A unified library of SOTA model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
GitHub repository
https://github.com/NVIDIA/Model-OptimizerLicense
Apache-2.0
Stars
5,497
This is an independent tool overview. Check the project repository for current releases, compatibility, and installation requirements.