Parameters (AI model)

In short: Parameters (weights) are the numbers in which a neural network stores its knowledge. “7B” means seven billion parameters.

In more detail: The more parameters, the more knowledge and abilities, but also more memory and slower execution. Quality also depends on the training data and method. Modern small models (7 to 14 billion) beat older, much larger ones today.

In Depth

• Size classes: under 3B (phone, edge), 7 to 14B (laptop, 8 GB to 16 GB), 30 to 40B (strong graphics card), 70B (two cards or Mac), over 100B (server, workstation).
• Dense and MoE models: mixture of experts separates total and active parameters.
• Training: Training needs a multiple of the memory of running (weights, gradients, optimiser state).
• Bits: Each parameter takes 0.5 to 4 bytes depending on quantisation.

See also: VRAM and model size, Fine-tuning and LoRA