Top-p (Nucleus Sampling)
Top-p, or nucleus sampling, restricts token selection to the smallest set of most-probable tokens whose cumulative probability reaches a threshold p. The model then samples only from this "nucleus." Unlike a fixed top-k cutoff, the set size adapts to how confident the model is at each step.
Definition
Top-p (Nucleus Sampling)Nucleus sampling was introduced to address the tendency of pure sampling to occasionally pick very unlikely tokens and of greedy or top-k decoding to produce repetitive, degenerate text. By keeping only the head of the probability mass — say the top 90% — top-p dynamically widens the candidate set when the model is uncertain and narrows it when the model is confident.
Top-p is frequently used together with, or as an alternative to, temperature. Providers typically advise tuning one at a time. A value of 1.0 disables the cutoff, allowing the full distribution.
Frequently asked questions
What is Top-p (Nucleus Sampling)?
Top-p, or nucleus sampling, restricts token selection to the smallest set of most-probable tokens whose cumulative probability reaches a threshold p. The model then samples only from this "nucleus." Unlike a fixed top-k cutoff, the set size adapts to how confident the model is at each step.
Sources
Related terms
← Full AI & prompt engineering glossary
Put Top-p (Nucleus Sampling) to work with Prompeteer, the Agentic Contextual AI Platform →