Autoencoders and VAE

In short: An autoencoder learns to compress data into a small code and to reconstruct it from that. The code (latent space) contains the essentials. A VAE turns this into a model that can create new data.

In more detail: The network has a bottleneck: input → encoder → small vector → decoder → reconstruction. Loss: difference between input and output. It learns without labels (unsupervised).

In detail

• Use: noise removal, anomaly detection (badly reconstructable data is unusual), dimensionality reduction, preliminary stage of image generation.
• VAE (variational autoencoder): the encoder outputs a distribution (mean and spread) instead of a point. An additional loss (KL) keeps the latent space smooth. You can draw random points and decode them to get new examples.
• Latent space: nearby points give similar images, interpolating gives smooth transitions.
• Diffusion: Stable Diffusion uses a VAE to bring images into a smaller latent space, see Image generation with diffusion models.
• VQ-VAE: discrete codes instead of numbers, basis of many image and audio tokenizers.
• GAN as competitor: generator and discriminator compete, see GANs: generator versus discriminator.

See also: Principal component analysis (PCA), Kinds of learning: supervised, unsupervised, reinforcement