CS180-Proj2: Fun with Filters and Frequencies

David Wei
david_wei@berkeley.edu

1. Introduction

By applying filters and analyzing frequencies, images can be processed and combined in interesting ways. In the first part of this project, edge detection is conducted by applying the Finite Difference Filter. Gaussian Filter is applied to get rid of the unnecessary wrinkles. Then, images are sharpened by stacking its edges onto itself. The second part of this project consists of two image binding tasks. The first task generates Hybrid Image by adding the high frequency of one image to the low frequency of another. Both successful and failing attempts are introduced. The second task blends images by applying the Gaussian Stack and the Laplacian Stack.

2. Edge Detection by Blurred Finite Difference

I detected the edges by applying the Finite Difference Filters dx and dy, denoted by dx = [1, -1], dy = [1, -1]^T respectively. The horizontal and vertical edges are calculated by conducting convolution on the image using the filters, and are combined into a single edge image using Euclidean Distance. To erase the wrinkles, I conducted binaryzation on the edge image. Figure 1 shows the original Cameraman image and its horizontal, vertical and combined edge image before and after their binaryzation.

Figure 1a
Figure 1a: Original Cameraman image.
dx
Figure 1b
Figure 1c: Vertical edge image before binaryzation.
Figure 1b
Figure 1c: Vertical edge image after binaryzation.
dy
Figure 1d
Figure 1d: Horizontal edge image before binaryzation.
Figure 1e
Figure 1e: Horizontal edge image after binaryzation.
combined
Figure 1f
Figure 1f: Combined edge image before binaryzation.
Figure 1g
Figure 1g: Combined edge image after binaryzation.

The edge image shown in Figure 1f contains unwanted noise. To remove the noise, I smoothed the original image by applying Gaussian Filter with a kernel size of 9 and a sigma of 1.5 before edge detection. The edge image produced this time is shown in Figure 2a. In another attempt, I generated a Blurred Finite Difference Filter by conducting convolution on the Gaussian Filter and the two Finite Difference Filters, and applied the combined filters directly on the original image. The result of this second attempt is shown in Figure 2b.

Figure 2a
Figure 2a: Noiseless edge image by applying Gaussian Filter before edge detection.
Figure 2b
Figure 2b: Noiseless edge image by combining Gaussian Filter with Finite Difference Filters.

The noiseless edge image obtained by the two attempts are almost identical.

3. Image Sharpening

Image sharpening can emphasize the details of the image, making the image "clearer" when observed by human eyes. I sharpened the images using the Unsharp Masking technic. First, a Gaussian Filter is applied to the target image to generate a blurred image, which contains the low frequency features of the image. Sigma in the Gaussian Filter is always set as 1/6 of the kernel size. Then, the high-frequency details are obtained by subtracting the low frequency features from the original image. These details are emphasized in the final image by \[ IMG_{sharpened} = IMG_{original} + \alpha \times IMG_{high-frequency}. \] Figure 3, 4 shows the sharpening result of the taj and forest image.

Figure 3a
Figure 3a: Original taj image.
 
Figure 3b
Figure 3b: Sharpened taj image.
Gaussian kernel size = 7; alpha = 1
Figure 4a
Figure 4a: Original forest image.
 
Figure 4b
Figure 4b: Sharpened forest image.
Gaussian kernel size = 9; alpha = 1.5

In order to evaluate my image sharpening approach, I took the tiger image, blurred it, and then sharpened it. The high frequency details lost in the blurring process should be restored. Figure 5 shows the result.

Figure 5a
Figure 5a: Original tiger image.
Figure 5b
Figure 5b: Blurred tiger image.
Gaussian kernel size = 7
Figure 5c
Figure 5c: Sharpened blurred tiger image.
Gaussian kernel size = 15; alpha = 2

The tiger in the sharpened image seems more vivid than the tiger in the blurred image. This indicates that a lower resolution image can show similar visual effects on human eyes using image sharpening.

4. Hybrid Images

Human eyes can only identify color changes in a range of frequencies. When the colors change too swift or too slow on an image, humans can't identify the change. I combined two images by adding the low frequency features of one image and the high frequency details of the other. In this way, when looked afar, only the low frequency features from the first image would be noticed. When looked closely, only the high frequency details of the second image would be noticed. The following are three attempts of generating Hybrid Image.

4.1 Who is Spiderman? Take off your glasses!

By combining the high frequency details of Spiderman with mask on and the low frequency features of the face Peter Parker casted by Tobey Maguire, Spiderman should have mask on when looked closely. However, when look afar (or simply take off your nearsighted glasses like Peter did), Spiderman will reveal his true identity!

Figure 6a
Figure 6a: Spiderman with mask on.
Used as high frequency details; Gaussian kernel size = 13
Figure 6b
Figure 6b: Tobey Maguire in Spiderman suit.
Used as low frequency features; Gaussian kernel size = 17
Figure 6c
Figure 6c: Spiderman hybrid image.

I analyzed the two Spiderman images in frequency domain before and after their filtering using Fast Fourier Transform, Figure 7 shows the result.

Figure 7a
Figure 7a: High frequency image in frequency domain (before filtering).
Figure 7b
Figure 7b: Low frequency image in frequency domain (before filtering).
Figure 7c
Figure 7c: High frequency image in frequency domain (after filtering).
Figure 7d
Figure 7d: Low frequency image in frequency domain (after filtering).

4.2 Wednesday Cartoon/Real-Person Hybrid

I combined the high frequency details of the Wednesday cartoon version and the low frequency features of Wednesday casted by Jenna Ortega. The cartoon Wednesday should appear when looked closely, and the Netfilx Wednesday will appear when looked afar.

Figure 8a
Figure 8a: Wednesday cartoon version.
Used as high frequency details; Gaussian kernel size = 13
Figure 8b
Figure 8b: Wednesday starred by Jenna Ortega.
Used as low frequency features; Gaussian kernel size = 17
Figure 7c
Figure 8c: Wednesday hybrid image.

4.3 Golden Gate Bridge History/Now Hybrid (Failed)

I combined the high frequency details of Golden Gate Bridge image filmed last century and the low frequency features of the bridge filmed recently. Figure 9 shows the result.

Figure 9a
Figure 9a: Golden Gate Bridge filmed last century.
Used as high frequency details; Gaussian kernel size = 13
Figure 9b
Figure 9b: Golden Gate Bridge filmed recently.
Used as low frequency features; Gaussian kernel size = 17
Figure 9c
Figure 9c: Golden Gate Bridge hybrid image.

The results are not satisfying as the older Golden Gate Bridge image can barely be seen even when looked closely. I believe this is because one of the photos are grayscale while the other is colored. It can be hard for a grayscale image to gain human eyes' attention when it is mixed with a colored image.

5. Multiresolution Blending

When blending two images together, it would seem more natural if the high-frequency patterns are not stacked together while the low-frequency background color change gradually between the images. I implemented Laplacian Stack to obtain the image features under different frequencies. Laplacian Stack is calculated using the Gaussian Stack. Each layer in the Gaussian Stack blurs the image from the last layer using the Gaussian Filter, with an increasing larger kernel size as the layers go deep. The first layer in the Gaussian Stack is the original image. Each layer in the Laplacian Stack is denoted by \[ lstack[i] = gstack[i] - gstack[i + 1]; \] \[ lstack[-1] = gstack[-1]. \] Each layer in the Laplacian Stack contains a certain range of frequency, and should be blended in different ways.

The images are blended under the instruction of a mask image. Each layer is assigned with a blurred version of the mask image. The deeper the layer, the more blurred (using larger Gaussian Filter) the mask image is. First, a blend image for each layer is generated, denoted by \[ blend[i] = lstack1[i] * mask[i] + lstack2[i] * (1 - mask[i]). \] Then, the blended layers are added together, and the result is the blending output.

Figure 10 shows my attempt of blending the apple image (Figure 10a) and the orange image (Figure 10c) using mask image (Figure 10b). Figure 11 shows the Laplacian Stack of this process, where the left column is the Laplacian Stack of the apple image, the right column is the Laplacian Stack of the orange image, and the middle column is the blended layers. The blending output is shown in Figure 10d.

Figure 10a
Figure 10a: Apple image.
Blended from the left.
Figure 10b
Figure 10b: Mask image.
 
Figure 10c
Figure 10c: Orange image.
Blended from the right.
Figure 10d
Figure 10d: Oraple image.
Figure 11
Figure 11: Laplacian Stack of the Oraple image blending process.

I also blended earth and moon images to generate moonearth (imagine when human inhabits half of the moon), and blended my friend Mike and a kangaroo to generate Mikaroo (a scornful kangaroo). The results are shown in Figure 12 and 13 respectively.

Figure 12a
Figure 12a: Earth image.
Blended from the top.
Figure 12b
Figure 12b: Mask image.
 
Figure 12c
Figure 12c: Moon image.
Blended from the bottom.
Figure 12d
Figure 12d: Moonearth image.
Figure 13a
Figure 13a: Image of Mike.
Contributed the scornful eyes.
Figure 13b
Figure 13b: Mask image.
 
Figure 13c
Figure 13c: Image of a kangaroo.
Contributed to all other parts except the eyes.
Figure 13d
Figure 13d: A scornful kangaroo.

Bells and Whistles

When trying different ways of recoloring the moonearth image, I accidentally used the moonearth image generated last time as the new mask image. The blending result is amazing. It looks like lunar surface, but has clouds and oceans. I cannot explain this phenomenon, but here is the image.

Figure 14a
Figure 14a: Earth image.
 
Figure 14b
Figure 14b: Mask image (the blending result from last time).
Figure 14c
Figure 14c: Moon image.
 
Figure 14d
Figure 14d: Moonearth Ultra image.

* Finished on Sep 23, 2024.