CodeFuse-MFTCoder
CodeFuse-MFTCoder is a multi-task fine-tuning framework designed to enhance the multi-task capabilities of Large Language Models, especially on Code-LLMs. Unlike traditional single-task fine-tuning, it can handle multiple tasks simultaneously, balancing the differences in data volume, difficulty, and convergence speed among various tasks by combining diverse loss functions. This approach increases fine-tuning efficiency and performance. Additionally, the framework incorporates efficient training optimization techniques and supports almost all well-known open-source models. Moreover it ranked first on the BigCode Leaderboard for its MFT performance of CodeFuse-Deepseek model.