Parameter-efficient fine-tuning, commonly abbreviated PEFT, is the umbrella term for techniques that adapt a large pretrained model to a new task without retraining all of its parameters. The base model’s weights stay frozen, and only a small number of extra or selected parameters are trained. Because frontier models have tens or hundreds of billions of parameters, full fine-tuning is prohibitively expensive in both compute and storage, especially when you want many task-specific variants; PEFT methods get comparable quality while touching a tiny fraction of the weights.
The family includes adapters (small inserted layers), prefix-tuning and prompt tuning (learned continuous prompts prepended to the input), and low-rank methods such as LoRA and its quantized variant QLoRA. Hugging Face’s PEFT library packages these methods and integrates with the Transformers, Diffusers, and Accelerate libraries, making it routine to fine-tune large models on consumer hardware and to store each task’s customization as a small add-on rather than a full model copy.
PEFT is one of the main reasons the open-weight ecosystem is so customizable. For a business, it is the difference between needing a GPU cluster to specialize a model and being able to do it on a single card, while keeping the cost of maintaining dozens of fine-tuned variants low.