In-context learning: zero-shot and few-shot

In short: A large language model can solve new tasks without its weights being changed, purely through instructions and examples in the prompt. This is called in-context learning.

In more detail: Zero-shot: only an instruction. One-/few-shot: a few examples of input and desired output. The model recognises the pattern and continues it.

In detail

• Few-shot example: “Rate the sentiment. ‘Great film’ → positive. ‘Boring’ → negative. ‘Quite okay’ →”
• Why it works: in pre-training the model learns to continue patterns in texts. Examples in the prompt activate the matching pattern.
• The weights stay the same: nothing is learned after the conversation. The effect applies only within the context.
• Tips: clear role, specify the answer format, 3 to 5 varied examples, show edge cases too.
• Chain-of-thought prompting: examples with intermediate steps (“think step by step”), see Reasoning models and chain of thought.
• Limits: order and wording of examples change results, long prompts cost tokens.
• Alternative: fine-tuning for fixed tasks with many examples, see Fine-tuning and LoRA.

See also: Prompt engineering, Context window