Data Science & Analytics
Benchmark Model Kernels
Inspect Hugging Face decoder layers on meta tensors and plan or run per-rank BF16, FP8, and NVFP4 GEMM or fused-MoE microbenchmarks with the bundled scripts and a local FlashInfer checkout. Use when choosing a model, GPU, TP, EP, or M/token-concurrency sweep; deriving common fused QKV and gate/up shapes without loading checkpoint weights; or using FlashInfer benchmark utilities. Do not use for end-to-end server throughput or request latency.
- Publisher / source
- NVIDIA/Model-Optimizer (found on GitHub top repositories)
- Upstream record
- NVIDIA/Model-Optimizer
- Version
- Version 1
- Published
- Publication review
- Approved for publication
- Content fingerprint
d1e3238ea29cfd944230b01777d0ad726170e04a86c2f48e8db585f6622aae0e