Data Science & Analytics
Evaluation
Evaluates accuracy of quantized or unquantized LLMs using NeMo Evaluator Launcher (NEL). Triggers on "evaluate model", "benchmark accuracy", "run MMLU", "evaluate quantized model", "run nel". Handles deployment, config generation, and evaluation execution. Always load when validating completed runs or reporting evaluation scores, including standalone launching-evals analysis. Not for quantizing models (use ptq), standalone serving (use deployment), or comparing completed baseline-vs-quantized results (use compare-results).
- Publisher / source
- NVIDIA/Model-Optimizer (found on GitHub top repositories)
- Upstream record
- NVIDIA/Model-Optimizer
- Version
- Version 1
- Published
- Publication review
- Approved for publication
- Content fingerprint
ee1893665ec1dd989de4c37eeb4b95ce451146e92bbd81cf4201bad2fcc43e9b