Data, features and labels

In short: Data is the raw material of every model. Features are the properties the model sees, the label is the correct answer it should predict.

In more detail: In house price prediction, area, number of rooms and location are the features, the price is the label. The quality and quantity of the data determines model quality more than almost any technique.

In detail

• Data set: rows = examples, columns = features. Table, images (pixels), text (tokens), audio (waveform).
• Split: training (for learning), validation (for tuning), test (for honest evaluation at the end). Typically 80/10/10.
• Preprocessing: fill missing values, check outliers, scale (normalise), encode categories as numbers (one-hot).
• Feature engineering: build better features from raw data (weekday from date). Deep learning often learns features itself.
• Label quality: wrong labels harm. Labelling is often manual work (crowdsourcing).
• Bias: if data under-represents certain groups, the model learns them worse, see Bias, fairness and data quality.
• Data leakage: if test data ends up in training, performance looks better than it is.
• Quantity: simple models need hundreds, image models millions, language models trillions of tokens.

See also: What is machine learning?, Training, validation and test