Software Development
Modelopt Benchmark Model Kernels
Inspect Hugging Face decoder layers on meta tensors and plan or run per-rank BF16, FP8, and NVFP4 GEMM or fused-MoE microbenchmarks with the bundled scripts and a local FlashInfer checkout. Use when choosing a model, GPU, TP, EP, or M/token-concurrency sweep; deriving common fused QKV and gate/up shapes without loading checkpoint weights; or using FlashInfer benchmark utilities. Do not use for end-to-end server throughput or request latency.
- Publisher / source
- NVIDIA/Model-Optimizer (found on GitHub top repositories)
- Upstream record
- NVIDIA/Model-Optimizer
- Version
- Version 1
- Published
- Publication review
- Approved for publication
- Content fingerprint
8ab731076bccb431776cc6255b909b4da3561e317803f9364f4c0b3d9fed60f9