Classification and regression

In short: Classification predicts a category (cat or dog, spam or not spam). Regression predicts a number (price, temperature).

In more detail: Both are forms of supervised learning and differ in output type and loss function.

In detail

• Binary classification: two classes. Output often a probability between 0 and 1 (sigmoid).
• Multi-class: one probability per class, summing to 1 (softmax), see Softmax and probabilities.
• Multi-label: several classes possible at once (a photo with dog and car).
• Regression: output is a real number. Loss often mean squared error (MSE).
• Classification loss: cross-entropy, see Loss functions: MSE and cross-entropy.
• Classic examples: linear regression, logistic regression, decision trees (Decision trees and random forests).
• Boundary: a decision boundary separates classes in feature space, linear models draw a straight line, neural networks curved surfaces.
• Language models: are basically classifiers: at every step they choose the next token from the vocabulary.

See also: Linear and logistic regression, Metrics: accuracy, precision, recall and F1