CASE-Lab-UMD / LLM-Drop

What it does

• Significant Speedup & Memory Savings. Achieves up to 2.1× inference speedup and over 40% KV cache memory reduction without requiring specialized hardware kernels.
• Unified Dropping Taxonomy. Systematically dissects and compares Block Drop, Attention-Layer Drop, MLP-Layer Drop, and Joint Layer Drop under a standardized framework.
• High Performance Retention. Retains >95–98% of core reasoning and general language capabilities (MMLU, GSM8K, ARC-c, HellaSwag) through importance-aware layer selection.
• Orthogonal Quantization Synergy. Easily pairs with post-training 4-bit quantization (AWQ / GPTQ) for compounding latency and memory benefits.
• Plug-and-Play Hugging Face Integration. Output models use standard auto_map configurations for seamless loading via AutoModelForCausalLM.