CS180-Proj3: Face Morphing and Modelling a Photo Collection

David Wei
david_wei@berkeley.edu

1. Introduction

In the first part of this project, I morphed two face images using Affine Transformation. I obtained 100 correspondences for each of the two faces and computed their average coordinates. Triangulation was conducted on the correspondences and for each triangle, Affine Matrixes were generated to stretch the triangles from the original image to the Midway image. I used Cross Dissolve to bind the color. I furtherly generated a sequence of 51 morphed images using different Morph Weight to produce the morphing GIF. In the second part, I computed the mean face of 12 Brazilian faces, and stretched my face into the shape of the mean face. I also computed the mean face of 12 smiling Brazilian faces to add a smile on my grim portrait.

2. Correspondences and Triangulation

To morph the close-ups of Elon Musk and George Clooney by Martin Schoeller, I manually selected a total of 100 correspondences on each image. In this way, the two faces can be morphed by aligning the correspondences. I conducted Delaunay Triangulation on Elon's correspondences and adapted Elon's triangulation on George's image. Every pixel in the two close-ups belongs a same triangle. The correspondences and triangulation of the images are shown in Figure 1.

Figure 1a
Figure 1a: Correspondences and triangulation on Elon Musk's closeup.
Figure 1b
Figure 1b: Correspondences and triangulation on George_Clooney's closeup.

3. Midway Face and the Morph Sequence

Midway Face is a generated morphed image where ideally the shape and color of the corresponding face elements are the mean of that on the two faces. To generate the Midway Face, I computed the average coordinates of the correspondences to represent the mean shape of each face elements. The faces of Elon and George are aligned to the mean correspondences by conducting Affine Transformation on each triangle. After aligning the two faces, the midway face is calculated by obtaining the arithmetic mean of two aligned faces. The midway face is shown in Figure 2.

Figure 2a
Figure 2a: Elon Musk.
Figure 2b
Figure 2b: Midway.
Figure 2c
Figure 2c: George_Clooney.

To present a more vivid process of morphing, I generated a sequence containing 51 morphed images of Elon and George's closeups, where each image is assigned with a different Morph Weight. Morph Weight controls the extent of morphing and is within the range of [0, 1]. A morphed image with a weight of 0 is identical to the original Elon's closeup. As the weight increases, the morphed correspondences become closer to that of George's image, and the color of the image dissolves from Elon's face to George's. When the Morph Weight reaches 1, the output image becomes identical to the original George face. This morphed image sequence is played at 50ms per frame to produce the morphing GIF shown in Figure 3.

Figure 3
Figure 3: Morphing sequence of Elon Musk and George_Clooney.

4. Mean Face of Population

Using coordinates, we get to align and morph two faces into a midway face, or Mean Face. The same process can be adapted on a larger population. I aligned 12 Brazilian male faces from the FEI Face Database using 68 correspondences and computed their mean. The 12 face closeups, the closeups after alignment, and the mean face are shown in Figure 4, 5, 6, respectively.

Figure 4
Figure 4: Faces of Brazilian male.
Figure 5
Figure 5: Faces of Brazilian male after alignment.
Figure 6
Figure 6: Mean face of Brazilian male.

I aligned my face to the mean face to generate a Brazilian version of me, but still kept my color. This is similar to the morphing progress introduced in Section 3 only without the Cross-Dissolve. The output image is shown in Figure 7.

Figure 5a
Figure 7a: My face.
Figure 5b
Figure 7b: My face Brazilian version.

5. Make Me Smile! (Bells & Whistles)

In order to generate a smiling face using the original grim portrait of myself, I aligned my face to the mean face of a smiling Brazilian population. Figure 8, 9, 10 shows the smiling population, their mean face, and my face before and after alignment, respectively.

Figure 8
Figure 8: Faces of smiling Brazilian male.
Figure 9
Figure 9: Mean face of smiling Brazilian male.
Figure 10a
Figure 10a: My face.
Figure 10b
Figure 10b: My face smiling.

I also set the Morph Weight as 1.5 and -0.5 respectively to produce Caricatures.

Figure 11a
Figure 11a: Caricature of me smiling.
(morphed at 1.5 towards smiling Brazilian face)
Figure 11b
Figure 11b: Caricature of me sad.
(morphed at -0.5 towards smiling Brazilian face)

* Finished on Oct 2, 2024.