Fine-tuning
Fine-tuning updates model weights using examples or feedback so the model performs a defined task or behavior more consistently.
Tiny example
Section titled “Tiny example”A support classifier repeatedly confuses two company-specific categories. You have thousands of reviewed examples, a stable label definition, and an eval set. Fine-tuning may improve that repeated decision.
Retrieval or fine-tuning?
Section titled “Retrieval or fine-tuning?”| Need | Better starting point |
|---|---|
| Current product policy | Retrieval from the governed policy source |
| Consistent JSON or house style | Schema and prompting, then fine-tuning if measured failures remain |
| New private fact for one customer | Context or retrieval |
| Stable specialized classification | Fine-tuning may fit |
OpenAI’s model-optimization workflow starts with evals, improves prompts, and uses fine-tuning where the measured task benefits.1
FDE note
Section titled “FDE note”“Fine-tune last” is a useful default, not a law. Fine-tune when the behavior is stable, the data is good, and the expected quality, cost, or latency gain justifies training and maintenance.
Footnotes
Section titled “Footnotes”-
OpenAI, model optimization. ↩