Create your Gitee Account
Explore and code with more than 14 million developers,Free private repositories !:)
Sign up
Clone or Download
contribute
Sync branch
Cancel
Notice: Creating folder will generate an empty file .keep, because not support in Git
21 days ago
25 days ago
Loading...
README
Apache-2.0

Chitu「赤兔」

Ask DeepWiki

中文 | English

Chitu「赤兔」是一个专注于效率、灵活性和可用性的高性能大模型推理框架。

里程碑

  • [2026/02/06] 发布 v0.5.1,适配摩尔线程 GPU。
  • [2025/12/12] 发布 v0.5.0,重点提升集群部署场景的性能。
  • [2025/08/01] 发布 v0.4.0,大幅提升了一体机推理部署场景的性能和稳定性,适配昇腾、英伟达、沐曦、海光,支持 DeepSeek、Qwen、GLM、Kimi 等模型。
  • [2025/07/28] 发布 v0.3.9,首发支持华为昇腾 910B 推理部署智谱 GLM-4.5 MoE 模型。
  • [2025/06/12] 发布 v0.3.5,提供昇腾 910B 完整原生支持,提供 Qwen3 系列模型高性能推理方案。
  • [2025/04/29] 发布 v0.3.0,新增 FP4 在线转 FP8、BF16 的高效算子实现,支持 DeepSeek-R1 671B 的 FP4 量化版
  • [2025/04/18] 发布 v0.2.2,新增 CPU+GPU 异构混合推理支持,实现单卡推理 DeepSeek-R1 671B。
  • [2025/03/14] 发布 v0.1.0,支持 DeepSeek-R1 671B,提供 FP8 在线转 BF16 的高效算子实现。

简介

赤兔定位于「生产级大模型推理引擎」,充分考虑企业 AI 落地从小规模试验到大规模部署的渐进式需求,专注于提供以下重要特性:

  • 多元算力适配:不仅支持 NVIDIA 最新旗舰到旧款的多系列产品,也为国产芯片提供优化支持。
  • 全场景可伸缩:从纯 CPU 部署、单 GPU 部署到大规模集群部署,赤兔引擎提供可扩展的解决方案。
  • 长期稳定运行:可应用于实际生产环境,稳定性足以承载并发业务流量。

项目团队感谢广大用户及开源社区提出的宝贵意见和建议,并将持续改进赤兔推理引擎。 然而,受制于团队成员的精力,无法保证及时解决所有用户在使用中遇到问题。 如需专业技术服务,欢迎致信 solution@chitu.ai

测试数据

请参阅赤兔开发团队测试的性能数据,也欢迎分享您的自测数据

性能数据与您的硬件配置、软件版本、测试负载相关,多次测试结果可能存在波动。

安装使用

请参阅开发手册获取完整的安装使用说明。

对于在单机环境上快速验证的场景,建议使用官方镜像进行部署。目前提供适用于以下平台的镜像:

  • 英伟达(arch 8.0、8.9):qingcheng-ai-cn-beijing.cr.volces.com/public/chitu-nvidia_arch_80_89:latest
  • 英伟达(arch 9.0):qingcheng-ai-cn-beijing.cr.volces.com/public/chitu-nvidia_arch_90:latest
  • 沐曦:qingcheng-ai-cn-beijing.cr.volces.com/public/chitu-muxi:latest
  • 昇腾(A2):qingcheng-ai-cn-beijing.cr.volces.com/public/chitu-ascend_a2:latest
  • 昇腾(A3):qingcheng-ai-cn-beijing.cr.volces.com/public/chitu-ascend_a3:latest

查看支持的模型

更多模型请参见 支持的模型

参与开发

赤兔项目欢迎开源社区的朋友们参与项目共建,请参阅贡献指南

交流讨论

如果您有任何问题或疑虑,欢迎提交issue。

您也可以扫码加入赤兔交流微信群:

许可证

本项目采用 Apache License v2.0 许可证 - 详见 LICENSE 文件。

本代码仓库还引用了一些来自其他开源项目的代码片段,相关版权信息已在代码中以 SPDX 格式标注。这些代码片段的许可证信息可以在 LICENSES/ 目录下找到。

本代码仓库还包含遵循其他开源许可证的第三方子模块。您可以在 third_party/ 目录下找到这些子模块,该目录中包含了它们各自的许可证文件。

常见问题

中文 | English

致谢

非常感谢来自华为、沐曦、海光、燧原、智谱、中国电信、并行科技等各方的帮助。

在构建 Chitu 的过程中,我们从以下项目(按字母排序)中学到了很多,并复用了一些函数:

我们将持续为开源社区贡献更高效、更灵活、更兼容、更稳定的大模型推理部署解决方案。

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright Qingcheng.AI Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

About

高性能大模型推理引擎 expand collapse
README
Apache-2.0
Cancel

Releases

No release

Activities

Pushed new commit to public-main branch 4 hours ago, b078472...d5cbf84
Pushed new commit to public-main branch 1 day ago, c5fdab1...b078472
Pushed new commit to public-main branch 3 days ago, 01e106b...c5fdab1
Pushed new commit to public-main branch 6 days ago, 4b3fbd8...01e106b
Pushed new commit to public-main branch 7 days ago, 938e7f2...4b3fbd8
Load More
can not load any more
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
1
https://gitee.com/thu-pacman/chitu.git
git@gitee.com:thu-pacman/chitu.git
thu-pacman
chitu
chitu
public-main

Search