DevOps & Infrastructure
Awq Quantization
Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.
- Publisher / source
- Orchestra-Research/AI-Research-SKILLs (found on skills.sh, an awesome list)
- Upstream record
- Orchestra-Research/AI-Research-SKILLs
- Version
- Version 3
- Published
- Publication review
- Approved for publication
- Content fingerprint
9a875e53113c11b00ab417fc393eb504f8be97d50c2e6901c762b991388bf717