Fine-tuning and LoRA

In short: In fine-tuning, a finished model is trained further with additional examples so it masters a task or style better. LoRA does this with very little memory.

In more detail: Instead of changing all billions of parameters, LoRA (low-rank adaptation) trains small extra matrices that end up only a few megabytes in size. They can be put on the base model or swapped. QLoRA combines this with a quantised base model.

In Depth

• When useful: fixed style, technical language, particular output format. For new knowledge RAG is often better.
• Data: Hundreds to thousands of good examples are often enough.
• Hardware: With QLoRA a 7-billion model can be adapted on a card with 12 to 16 GB of VRAM.
• Risks: overfitting, loss of general knowledge, privacy of training data.

See also: Parameters (AI model), Quantisation