Fine-tuning takes a model that has already been pre-trained on general text and trains it further on a smaller, focused dataset so it performs well on a particular task or follows a desired style. It is far cheaper than pre-training because it reuses everything the base model already learned.
The technique was central to Google’s 2018 BERT paper (Devlin et al.), which showed that one pre-trained model could be fine-tuned to top many language benchmarks with only small task-specific changes. A more behavior-focused form appears in OpenAI’s 2022 InstructGPT paper, “Training language models to follow instructions with human feedback” (Ouyang et al.), whose abstract observes that “making language models bigger does not inherently make them better at following a user’s intent” — and uses fine-tuning on human-written demonstrations as a first step toward fixing that.
Fine-tuning ranges from updating all of a model’s parameters to lightweight methods that adjust only a small fraction, trading cost against flexibility.
Why business readers should care: Fine-tuning is how a general model is turned into something specialized for your domain, tone, or compliance rules. It is usually the most practical lever a company has over model behavior short of building a model from scratch.