Project 3

In this project, I create a "morph" animation of my face into someone else's face, compute the mean of a population of faces and extrapolate from a population mean to create a caricature of myself. A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors.

Image 1

Part 1. Defining Correspondences

I took a picture of myself and my favorite celebrity, Jennifer Lawrence, for my morph!

Image 1
Me
Image 2
JLaw

The warp will be controlled by defining a correspondence between the two pictures. For instance, a possible correspondence would be eye to eye, ear to ear, mouth to mouth, etc. I labeled 61 correspondences by hand among the two images.

Image 1
Using a tool to label correspondences

Now I need to provide a triangulation of these correspondences that will be used for morphing.

  1. Calculate the mean point set between the two image correspondences.
  2. Compute the Delaunay triangulation on the mean point set.
  3. Use the same triangulation for each source image.
Note that using the same triangulation means that each correspondence will be part of the same triangle while the points individually are distinct.

Image 1
Me Triangulation
Image 2
JLaw Triangulation

Part 2. Computing the "Mid-way Face"

Before computing the whole morph sequence, I computed the "mid-way face" of my images. This involves computing the average shape, warping both faces into that shape, and averaging the colors together.

My algorithm does the following:

  1. Loop through all triangles in the Delaunay triangulation.
  2. Compute an affine warp for each triangle in the two images, to warp the given image's triangle to the mean shape triangle.
  3. Get all points inside the mean triangle.
  4. Inverse warp these mean triangle points to get the pixel value in the source image.
Image 1
"Mid-way Face"

Part 3. The Morph Sequence

To create a morph sequence from one face to another, we can repeat the morphing process from Part 2 over a time interval t. The variable t actually allows us to linearly interpolate the amount of warping and cross-dissolving. We increase each parameter from [0, 1] over the course of the morph. Here is the full morph sequence from my face to Jennifer Lawrence's face!

Morphing Animation
Morph Sequence GIF

Part 4. The "Mean Face" of a Population

I picked the Danes dataset of annotated faces. It contains 40 different Danish human faces, all without glasses. The gender distribution is 7 females and 33 males.

In order to compute the "mean face" of this population, I:

  1. Compute the average face shape of the whole dataset.
  2. Morph each of the faces in the dataset into the average shape.
  3. Compute the average face of the population.
Image 1

Here is the average triangulation on an example face!

Image 1
Image 1
Image 1
Image 1
Image 1
Image 1
Image 1
Image 1

Some funky morphed faces (into average Danes face).

Image 1
"Mean Face"

Can I morph my own face into the average Danes face geometry? And can I morph the average Danes face into my geometry?

Image 1
Image 1
Image 1
Image 1

Not really.... it's hard to align my image well to the Dane image.

Part 5. Caricatures: Extrapolating from the mean

I created caricatures of my face by extrapolating from the population mean in the previous part. I chose different warping coefficients, alpha, that are outside a typical morph timeframe (where alpha should be between 0 and 1). This should stress my face's features (alpha < 0) or the average face's features (alpha > 1).

Morphing Animation
Alpha=-0.5
Morphing Animation
Alpha=1.5

Bells & Whistles

I will change my face into the average chinese male.

Morphing Animation
Me
Morphing Animation
Average Chinese Male

I will show morphing just the shape, just the appearance, and both.

Morphing Animation
Shape
Morphing Animation
Appearance
Morphing Animation
Both