Context Engineering

What it is

Context engineering broadens prompt engineering from "wording a single instruction" to designing the entire context a model sees on each call: system instructions, few-shot examples, retrieved documents, tool definitions, conversation history, and memory. As applications became agentic and long-running, what fills the finite context window — and in what order and proportion — became as important as the phrasing of any one instruction.

The goal is to supply the smallest set of high-signal tokens that lets the model succeed: relevant, current, well-structured information, without noise or overflow. It treats the context window as a scarce, curated resource and manages retrieval, compaction, and selection to keep it that way.

When to use

• Building agents or multi-turn apps where context accumulates over time.
• Combining instructions, retrieved data, tools, and history into one call.
• Long contexts degrade quality or exceed the window and must be curated or compacted.
• You need to decide what information to include, exclude, and prioritize per request.

How it works

1. Inventory everything competing for the context window: instructions, examples, retrieved data, tools, history.
2. Select only the highest-signal, most relevant information for the current step.
3. Structure and order it so key material is prominent and unambiguous.
4. Compact, summarize, or drop stale content to stay within the window and avoid distraction.

Illustrative structure

The structure is a budgeted assembly of the context, not a single sentence: [concise system instructions] + [a few well-chosen examples] + [only the retrieved passages relevant now] + [needed tool definitions] + [compacted history] — each part sized to fit the window and kept relevant.

Pitfalls

• Overstuffing the window buries key information and can degrade quality ("lost in the middle").
• Irrelevant or outdated context distracts the model as much as missing context starves it.
• Naively appending full history eventually overflows the window and inflates cost.
• Poorly ordered context — critical instructions late or scattered — reduces adherence.

Frequently asked questions

Sources

• Anthropic — Effective context engineering for AI agents
• Liu et al. (2023), Lost in the Middle: How Language Models Use Long Contexts

Related techniques

• Retrieval-Augmented Generation (RAG)
• Few-Shot Example Selection
• System Prompts