Eigenfaces Facial Recognition

The Problem

A face image is just a big grid of pixel values, but most of that information is redundant, since every face shares the same basic structure (eyes, nose, mouth in roughly the same places). Eigenfaces, from Turk and Pentland's 1991 paper, asks: what if we found the handful of directions faces vary the most along, and represented every face as a combination of just those?

Approach

1. Loaded 100 face images (64×64 grayscale) from LFWcrop, a cropped subset of the Labeled Faces in the Wild dataset.
2. Standardized and centered every image (subtracted the mean face from the entire set).
3. Computed the eigenfaces: the eigenvectors of the covariance matrix of the training set, found efficiently via the standard trick of decomposing a much smaller matrix instead of the full pixel-by-pixel covariance matrix.
4. Reconstructed 16 unseen test faces as a linear combination of the eigenfaces.
5. Compared test faces to training faces by projecting both into "face space" and measuring Euclidean distance.

Results

The mean face below is the average of all 100 training faces blended together, a smooth, feature-less baseline every other face is measured against.

The eigenfaces themselves look ghostly, since each one represents a direction of variation rather than a real person. Together, they form the basis every face in the dataset can be reconstructed from.

Reconstructing unseen test faces from just those eigenfaces produces recognizably face-like images, but with less individual detail than the real test photos. This is the honest limitation of the method: with only 100 training faces, the learned "face space" doesn't fully capture features specific to people the model has never seen. More training faces would sharpen the reconstructions.

Original test faces (top) vs. their reconstructions from the eigenfaces alone (bottom).

Tools