CPM-Bee is a completely open source, commercially available pedestal model with tens of billions of parameters in Chinese and English. It is also the second milestone of CPM-Live training. It adopts the Transformer auto-regressive architecture (auto-regressive), pre-trains on trillions of high-quality corpus, and has strong basic capabilities. Developers and researchers can adapt various scenarios based on the CPM-Bee base model to create domain-specific application models.
👐 Open source and commercially available: OpenBMB has always been adhering to the open source spirit of "letting large models fly into thousands of households". The CPM-Bee pedestal model will be completely open source and commercially available to promote the development of large model fields. We encourage scientific research institutions, enterprises and individual developers around the world to freely innovate on the CPM-Bee base model under the premise of abiding by the open source license agreement.
💫 Excellent bilingual performance in Chinese and English: The CPM-Bee pedestal model has undergone strict screening and matching on the pre-training corpus, and has outstanding performance in bilingual Chinese and English. For details, please refer to the evaluation tasks and results.
📖 Ultra-large-scale high-quality corpus: The CPM-Bee pedestal model is trained on over one trillion corpus, which is one of the models with the most corpus in the open source community. At the same time, we have strictly screened, cleaned and post-processed the pre-training corpus to ensure the quality.
OpenBMB large model system ecological support: The OpenBMB large model system has a series of tools for high-performance pre-training, adaptation, compression, deployment, and tool development. The CPM-Bee base model will support all tool scripts to efficiently support developers to carry out stage use.
🔨 Dialogue and tool usage ability: Combining OpenBMB's exploration of instruction fine-tuning and tool learning, we fine-tuned on the basis of the CPM-Bee base model, and trained an instance model with powerful dialog and tool usage capabilities, API and internal testing Will be open soon.
Description: CPM-Bee is a pedestal model, that is, it is pre-trained from scratch. We encourage users to adapt/fine-tune/align on their own scenes and data before using it. For example, WebCPM uses CPM-Bee as the base, adapts to the serialized data retrieved by human networks, and obtains the ability of complex question answering and online retrieval. We will release more models based on the CPM-Bee base model in the future.
Visit Official Website
