7 terms
Distillation is a training method in which a smaller student model learns to reproduce the outputs of a larger teacher model at lower inference cost.
DPO, or direct preference optimization, trains a language model on pairs of preferred and rejected answers without a separate reward model or RL loop.
Fine-tuning continues training a pretrained model on a smaller, task-specific dataset so it adopts a new skill, format, style or domain without starting over.
LoRA, or low-rank adaptation, fine-tunes a large AI model by training small low-rank weight matrices while the original weights stay frozen.
Pretraining is the first, largest stage of model training, in which a neural network learns general patterns from vast unlabeled data before any adaptation.
QLoRA is a fine-tuning method that trains LoRA adapters on a frozen 4-bit quantized model, cutting GPU memory needs for adapting large language models.
RLHF (reinforcement learning from human feedback) trains a language model to prefer responses that people rate higher, using a learned reward model.