Project 1: Image Filtering and Hybrid Images

Example of a hybrid image

Hybrid images are static images that change in interpretation as a function of the viewing distance. The picture on the right side is an example of a hybrid image. Looking at the picture from a short distance, one can see an image of a fish. Viewed from a distance in which the fine detail blurs, the picture of a submarine emerges.

A hybrid image is the sum of a low-pass filtered version of the one image and a high-pass filtered version of a second image. The following brief steps describe how to create a hybrid image:

  1. Implement function my_imfilter() to do the image filtering.
  2. Use my_imfilter() to remove the high frequency from image1.
  3. Use my_imfilter() to blur image2 to get a temporary image. Subtract the temporary image from the original version of image2 to receive a high-pass filtered version of image2.
  4. Sum up the low frequencies of image1 and high frequencies of image2 to get a hybrid image.

Algorithm of Image Filtering

In this project, a function named my_imfilter(image, filter) is implemented to filter images. The function loads an image to be processed together with a filter to be used and then output a filtered image. Since input images are colored, the function first separates each color channel, then filters, and finally combines the filtered version of RGB channel together to get the result.

The color image is represented as a three-dimensional array in MATLAB. Red, green, blue channel is the first, second, third dimensional of the array, respectively. The separation and combination are implemented by the codes below.


%separate each color channel
image_r = image(:,:,1);
image_g = image(:,:,2);
image_b = image(:,:,3);
 
%filter
image_r = my_filter2(image_r, filter);
image_g = my_filter2(image_g, filter);
image_b = my_filter2(image_b, filter);
 
%create the RGB image
output(:,:,1) = image_r;
output(:,:,2) = image_g;
output(:,:,3) = image_b;

Since the filtering repeats three times, a function named my_filter2() is defined to make the code more concise and more clear. All the process of filtering is done in my_filter2().

The image first needs to be preprocessed before filtering. Zeros are added to the boundaries of the image. For a filter of size (m, n), (m-1)/2 rows and (n-1)/2 columns of zeros need to be padded around the image array so that the filtered image and the original image have the same size. The padding is done by the following steps:

  1. Get the size of filter (m, n) and the size of image (M, N).
  2. Horizontal concatenate the zero matrix of size ((m-1)/2, N) on the top and bottom sides of image.
  3. Vertical concatenate the zero matrix of size (M+m-1, (n-1)/2) on the left and right sides of image.

After the preprocessing, the image can be filtered. An output pixel's value is determined as a weighted sum of input pixel values. This can be represented by the following equation:

$$\large g(i, j) = \displaystyle\sum_{k, l} f(i+k, j+l)h(k, l).$$

In order to filter the image, a double circulation is put into use to go through the image array and calculate the values of each destination pixel, as follows.


%loop over all the pixels and calculate
for i=1:size_i(1)
    for j=1:size_i(2)
        a = i + size_f(1) - 1;
        b = j + size_f(2) - 1;
        im_tmp = image(i:a, j:b); %take the source part from image array
        m_tmp = im_tmp.*filter; %element-wise multiplication
        output(i, j) = sum(m_tmp(:)); %sum of all elements
    end
end

Test Results

The followings is the test results of the function my_imfilter(). The following images show the results of identify filter, small blur with a box filter, large blur, oriented filter (Sobel operator), high pass filter (discrete Laplacian operator), high pass "filter" alternative, respectively. The identify filter should do nothing so that the first image is also the original image.

Hybrid Images

For a fish and submarine hybrid image, the two original images look like this:

The low-pass (blurred) and high-pass versions of these images look like this:

Adding the high and low frequencies together and a hybrid image is generated:

For a dog and cat hybrid image, the two original images look like this:

The low-pass (blurred) and high-pass versions of these images look like this:

Adding the high and low frequencies together and a hybrid image is generated:

The original picture of Marilyn and Einstein images look like this:

The low-pass (blurred) and high-pass versions of these images look like this:

Adding the high and low frequencies together and a MarilynEinstein image is generated:

There are also two other examples, one is a combination of bike and bicycle:

And another is a integration of bird and plane: