Fine-Tuning
Fine-tuning adapts a pre-trained model to a specific task or style by continuing training on a smaller, curated dataset, updating the model's weights. Unlike prompting, which steers a frozen model at inference time, fine-tuning changes the model itself so the new behavior is baked in.
Definition
Fine-TuningA pre-trained model already encodes broad language and world knowledge. Fine-tuning takes that base and trains it further on labeled examples for a narrower objective — a domain vocabulary, an output format, or a task the base model handles inconsistently. It can improve reliability and reduce the prompt length needed to elicit desired behavior.
Full fine-tuning updates all parameters and is expensive; parameter-efficient methods such as LoRA update only a small set of added weights. Fine-tuning is most worthwhile when prompting and retrieval have been exhausted and there is enough high-quality task data to learn from.
Frequently asked questions
What is Fine-Tuning?
Fine-tuning adapts a pre-trained model to a specific task or style by continuing training on a smaller, curated dataset, updating the model's weights. Unlike prompting, which steers a frozen model at inference time, fine-tuning changes the model itself so the new behavior is baked in.
Sources
Related terms
← Full AI & prompt engineering glossary
Put Fine-Tuning to work with Prompeteer, the Agentic Contextual AI Platform →