AI & Machine Learning
MoE Training
Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional compute increase. Covers MoE architectures, routing mechanisms, load balancing, expert parallelism, and inference optimization.
- Publisher / source
- Orchestra-Research/AI-Research-SKILLs (found on skills.sh, an awesome list)
- Upstream record
- Orchestra-Research/AI-Research-SKILLs
- Version
- Version 2
- Published
- Publication review
- Approved for publication
- Content fingerprint
f13f184fe031c84700d72c15dd90161a96f3b48ab6b50b250e6b86cfc4c87354