AI & Machine Learning
Long Context
Extend context windows of transformer models using RoPE, YaRN, ALiBi, and position interpolation techniques. Use when processing long documents (32k-128k+ tokens), extending pre-trained models beyond original context limits, or implementing efficient positional encodings. Covers rotary embeddings, attention biases, interpolation methods, and extrapolation strategies for LLMs.
- Publisher / source
- Orchestra-Research/AI-Research-SKILLs (found on skills.sh, an awesome list)
- Upstream record
- Orchestra-Research/AI-Research-SKILLs
- Version
- Version 4
- Published
- Publication review
- Approved for publication
- Content fingerprint
6426eb21a38b8a314078292b8172d889b7f2b1a226901fcda78ac4d28a98a5e7