ModelTC / LightCompress

What it does

• 💥Comprehensive Algorithm Support. Provides a broad range of ✨SOTA compression algorithms, including ✅quantization, ✅mixed-precision quantization, and ✅sparsity, while maintaining accuracy consistent with the original repositories. ✨Quantiza
• 💥Supported Formats. Supports both ✨quantization (integer and floating-point) and ✨sparsity, specifically including ✅weight-activation, ✅weight-only, ✅mixed-precision quantization, as well as ✅structured and ✅unstructured sparsity.
• 💥Wide Model Support. Offers support for a diverse array of ✨LLM models, including ✅LLama, ✅Mistral, ✅InternLM2, ✅Qwen2, among others, as well as ✅MOE(DeepSeekv2, Deepseek-R1) and ✅VLM(Llama3.2-vision, Qwen2-vl) models (see Supported Model L
• 💥Multi-backend Compatibility. Seamlessly integrates with various backends for enhanced deployment flexibility. Multiple quantization settings and model formats are compatible with a wide range of backends and hardware platforms, such as ✅VL
• 💥Performance Efficiency. Enables quantization of large LLMs, such as ✨Llama3.1-405B and ✨DeepSeek-R1-671B, with PPL evaluation on a single A100/H100/H800 GPU.