Project 1: Image Filtering and Hybrid Images

Example of a series of scaled versions of the same hybrid image.

Above is an example of a hybrid image, which is constructed from a picture of a cat and a picture of a dog. The same hybrid image is scaled to different sizes in order to simulate looking at the picture from different distances. We can see a cat in the leftmost picturea but a dog in the rightmost picture whereas the rightmost picture is essentially what we see by looking at the leftmost picture from far away.

This hybrid image is made by adding the high-frequency portion of the cat picture and the low-frequency portion of the dog picture. This process is done on Matlab in this project. It requires the following steps:

  1. Generate a gaussian fileter with a variable standard deviation called cutoff_frequency
  2. Filter out the high frequencies of the dog picture by applying the gaussian filter.
  3. Filter out the low frequencies of the cat picture by applying the gaussian filter and subtracting the result from the original picture.
  4. Add the pictures from step 1 and step 2 to generate the hybrid image.
  5. Scale the hybrid picture to show the viewing effect.

For step 1 and step 2, I am not using Matlab's built-in imfilter function. Instead, I wrote my own version of this funtion called my_imfilter. This function takes in an image and a filter and outputs the filtered image.

Function my_imfilter

My filter does the following:

  1. Pad the image with zeros on the edges depending on the size of the filter.
  2. Do the same convolution on the three RGB channels.
  3. For each channel, cycle through each pixel position of the original image size
  4. For each pixel position, extract a window of the size of the filter around the pixel position.
  5. Do a dot product of the window and the filter and sum up all the numbers.
  6. Assign the sum to the pixel in the output image.

I used the simplest padding method, zero padding. Although this method causes a black frame around the output image, it does not hurt too much since the black frame looks like a picture frame that actually decorates the image :). Also, the high-frequency image is generated by subtracting the filtered image from the original image, which creates a white frame around the edges. When the two filtered images are combined, the white and black frames are smoothed out.

Example of code

This line of Matlab code extracts a window of pixels of the same size of the filter. The position of the center of the window is the position where the convolution result will be placed in the result picture.


% fsize is a an array of filter sizes. eg. fsize of a 3*4 filter is [3 4]
% j, k is the row and column index into pixels
window = padded(j:j+fsize(1)-1, k:k+fsize(2)-1, i);

Hybrid images and intermediate images of different image pairs in a table

Above are the hybrid images generated from different image pairs. Due the nature of original images, each image pair requires a different cutoff frequency to generate a good result. In general, I need to set the cutoff frequency high if I want the high-frequency image to stand out in the hybrid image and vise versa.

Comparing the results of different image pairs, I found that the two images need to have a similar background in order to generate a good hybrid image. The result background color is determined by the background color of the low-frequency image. Therefore, The high-frequency part of the hybrid image will look unreal if the backgrounds do not match. The fish and marine image is a good example. An exception is black and white as they can both be used as a good background color, for example, the Einstein and Marilyn image.

Additional discovery

Pictures of dark color are better used as low-frequency image.

Pictures of bright color are better used as high-frequency image.

This is consistent with the conclusion from the last paragraph. If the image with bright color is used as the low-frequency image, the background is likely to be bright and easily cover the dark-colored image. We can see the undesired result below using the cat image as the low-frequency image and the dog image as the high-frequency image.

In order to see the dark-colored high-frequency dog, I had to set the cutoff frequency relatively high. However, it looks like the outline of the dog is awkwardly imposed on the cat picture.