Large-scale model reasoning library for calculating graphs - laptops can run large models without a graphics card
This large model reasoning library JittorLLMs has the following characteristics:
Low cost: Compared with similar frameworks, this library can greatly reduce hardware configuration requirements (by 80%). Without a graphics card, 2G memory can run large models, and everyone can implement local deployment of large models on ordinary machines; it is currently Largest model library known to have the lowest deployment cost;
Wide support: currently supports 4 large models: ChatGLM large model; , lowering the threshold for users of large models.
Portability: Users do not need to modify any code, but only need to install the Jittor version of torch (JTorch) to realize the migration of the model, so as to adapt to various heterogeneous computing devices and environments.
Fast speed: The loading speed of large models is slow. The Jittor framework uses zero-copy technology to reduce the loading cost of large models by 40%. At the same time, through the automatic compilation and optimization of meta-operators, the computing performance is improved by more than 20% compared with similar frameworks.
Visit Official Website
