How Autoencoders and Variational Autoencoders Turn Raw Data Into Learnable Latent Representations

What Is the Basic Architecture and Principle of an Autoencoder?

An autoencoder is a neural network architecture built to compress input data into its essential features through an encoder, then reconstruct the original input from that compressed representation through a decoderCITE:E1. IBM (IBM) describes this compress-then-reconstruct structure as the core mechanism of the modelCITE:E1. The gap between what the decoder outputs and the actual ground truth is defined as the reconstruction error, which serves as the signal the network learns fromCITE:E2.

How Is an Autoencoder Applied to Image Data, Using MNIST as an Example?

TensorFlow (TensorFlow) demonstrates the architecture on handwritten digit images: an encoder first turns each image into a lower-dimensional latent representation, and a decoder then turns that representation back into an imageCITE:E3. In the dataset used for this tutorial, each image measures 28x28 pixelsCITE:E4. Each MNIST image is originally represented as a vector of 784 integers, with each integer ranging from 0 to 255 to encode a single pixel's intensityCITE:E6.

How Does a Variational Autoencoder Improve on the Traditional Autoencoder's Latent Space?

A variational autoencoder (VAE) maps input data into the parameters of a probability distribution, such as the mean and variance of a Gaussian, instead of mapping it onto a single latent vector the way a traditional autoencoder doesCITE:E5. TensorFlow states that this distribution-based mapping produces a continuous, structured latent space, which is useful for image generationCITE:E5. The shift changes what the latent space is used for: a traditional autoencoder's latent vector is optimized for reconstructing a specific input, while a VAE's distribution parameters are structured so that sampling from the latent space can produce new, unseen outputsCITE:E5.

What Is the Theoretical Foundation and Learning Objective of the VAE?

The VAE's theoretical foundation traces to Kingma and Welling's 2013 paper, "Auto-Encoding Variational Bayes," which asks how to perform efficient inference and learning in directed probabilistic models that have continuous latent variables, intractable posterior distributions, and large datasetsCITE:E7. That framing — inference and learning under intractable posteriors at scale — is the problem the VAE's probability-distribution encoding, described by TensorFlow, is built to solveCITE:E7CITE:E5.

What This Means

Across these sources, the same underlying idea recurs at increasing levels of formality: IBM and TensorFlow describe an encoder-decoder structure evaluated by reconstruction error and demonstrated on 28x28-pixel, 784-integer MNIST vectorsCITE:E1CITE:E2CITE:E3CITE:E4CITE:E6, while the VAE reframes that same latent space as distribution parameters to enable generation rather than only reconstructionCITE:E5, a reframing whose formal justification goes back to Kingma and Welling's 2013 treatment of inference under intractable posteriorsCITE:E7.