Chain-of-Thought Prompting

What it is

Chain-of-thought (CoT) prompting elicits a series of intermediate reasoning steps rather than jumping straight to an answer. Wei et al. showed that prompting large models to "think step by step" — either by demonstrating reasoning in examples or by a simple zero-shot instruction — unlocks reasoning abilities that direct-answer prompting leaves untapped.

The explicit trace gives the model room to decompose the problem, track intermediate results, and arrive at a more reliable conclusion. It also makes the model's process auditable, which helps with debugging and trust.

When to use

• The problem requires multiple steps: arithmetic, logic, planning, or multi-hop reasoning.
• Direct answers are frequently wrong or skip necessary intermediate work.
• You need a visible reasoning trace to audit or debug the model's process.
• The task benefits from the model "showing its work" before committing to an answer.

How it works

1. Instruct the model to reason step by step before answering, or demonstrate the reasoning in examples.
2. Let the model generate intermediate steps that build toward the solution.
3. Have it state the final answer clearly, separated from the reasoning.
4. Optionally parse only the final answer for downstream use while keeping the trace for review.

Illustrative structure

The structure appends a reasoning directive to the task: "[question] Let's think step by step." — or, in few-shot form, each example shows the worked steps ("First… then… therefore…") before its answer, so the model mirrors that reasoning shape on the new question.

Pitfalls

• Verbose reasoning increases tokens, latency, and cost; not every task needs it.
• A fluent-looking chain can still reach a wrong answer — reasoning is not a correctness guarantee.
• Very small models benefit far less; CoT is an emergent behavior of larger models.
• Exposing the full chain to end users may leak intermediate errors or sensitive intermediate steps.

Frequently asked questions

Sources

• Wei et al. (2022), Chain-of-Thought Prompting Elicits Reasoning in LLMs
• Kojima et al. (2022), Large Language Models are Zero-Shot Reasoners

Related techniques

• Self-Consistency
• Task Decomposition
• Few-Shot Prompting