Project 1: Image Filtering and Hybrid Images

In this project, a convolutional filter function was written in Matlab and used to generate hybrid images. These hybrid images combined the filtered low frequencies of one image with the high frequencies of another image. As the human eye is more perceptive to high frequencies when an image is nearby and low frequencies when the image is far away, the hybrid image will appear to be two different objects depending on the distance from the image. The following sections describe the implementation of the filtering algorithm and how this algorithm was used to create hybrid images. Several examples of the generated hybrid images will also be provided.

Filtering Algorithm

The my_imfilter() function accepts as arguments a color or grayscale image of arbitrary size and a filter. The filter is assumed to have an odd width and height (in this case the center point is unambiguous), but otherwise accepts any size and does not have to be square. The output of the function is a filtered image of the same size as the input image. The algorithm begins by initializing the output to a zero matrix and determining several size parameters:


% Initialize empty result with same size as image
output = zeros(size(image));

% Get size info
[H, W, C] = size(image); % Image size and color channels
[fH, fW] = size(filter); % Height and width of filter
hpad = (fH-1)/2; % Size of padding (along height)
wpad = (fW-1)/2; % Size of padding (along width)

Since the process of filtering reduces the size of the output image for filter sizes greater than one, the image is padded along each outer edge by half the size of the filter in that direction. This is done by mirroring the image across the edge. Mirroring ensures that the edges of the output image appear natural to the image, where other approaches such as padding with all zeros leave an unnatural black edge:


% Add padding around image equal to half the size of filter in each
% direction (use mirror image)
image = padarray(image, [hpad, wpad], 'symmetric', 'both');
H = H + 2 * hpad;
W = W + 2 * wpad;

After the image is padded, the filtering is performed by convolving the filter with the image. This operation is not a true convolution as the filter is not mirrored, but for symmetric filters the two operations are functionally equivalent. For color images, the filtering is done separately for each of the color channels. For each pixel in the original image, the filter is multiplied by the overlapping portion of the image, and the result is summed across both dimensions. For the summations, the Matlab sum() function is used rather than two additional for loops for improved efficiency and code cleanliness. Finally, the center pixel of the filter is set to the result of this summation and assigned to the corresponding pixel on the output image:


% Perform filtering
for c = 1:C
    for h = hpad+1:H-hpad
        for w = wpad+1:W-wpad
            fh = h-hpad:h+hpad;
            fw = w-wpad:w+wpad;
            output(h-hpad,w-wpad,c) = sum(sum(filter .* image(fh, fw, c)));
        end
    end
end

To verify correct operation of the filtering algorithm, a set of six different filters was applied to an image of a cat, including the following:

  1. Identity filter
  2. Box filter (small blur)
  3. Gaussian filter (large blur)
  4. Sobel filter
  5. Laplacian filter
  6. High pass "filter" (subtract low frequencies)

The resulting images are shown in the table below:


Identity filter

Box filter (small blur)

Gaussian filter (large blur)

Sobel filter

Laplacian filter

High pass "filter" (subtract low frequencies)

As expected, the identity filter is the same as the original image. The Box and Gaussian filters produce a blurring effect to varying degrees. Each of the remaining three filters act like high pass filters, detecting high frequency edges.

Hybrid Image Generation

Once the filtering algorithm was implemented as a Matlab function, it was used to generate hybrid images. To generate the hybrid image, one image was blurred using a Gaussian filter, and the second image was high-passed by applying the same Gaussian filter and subtracting the result from the original image. The two images were then summed together to produce the hybrid image:


%% Filtering and Hybrid Image construction
cutoff_frequency = 7; % Standard deviation in pixels of Gaussian filter
filter = fspecial('Gaussian', cutoff_frequency*4+1, cutoff_frequency);

% Remove the high frequencies from image1 by blurring it.
low_frequencies = my_imfilter(image1, filter);

% Remove the low frequencies from image2.
high_frequencies = image2 - my_imfilter(image2, filter);

% Combine the high frequencies and low frequencies
hybrid_image = low_frequencies + high_frequencies;

The hybrid image generation algorithm was first tested with images of a cat and dog, where the high frequencies were taken from the cat and the low frequencies were taken from the dog. The figure below shows the resulting hybrid image:

+ =

To simulate distance, the hybrid image can be made gradually smaller:

The smaller the image, the more it appears to be a dog since low frequencies are more easily detected by the human eye at larger distances. Several other pairs of images were also tested and are shown below. For each pair, the cutoff frequecy was tuned to produce the best results:

A motorbike. (cutoff = 7)


It's a bird... It's a plane... (cutoff = 5)


Marilyn Monroe grows a moustache. (cutoff = 5)


Something's fishy... (cutoff = 7)

As it turns out, this hybrid image process can also be applied to sets of three images with only a slight modification of the code. Instead of a single cutoff frequency, two cutoff frequencies can be used. One of the three images is low passed with the lower cutoff frequency (higher standard deviation), another image is high passed with the higher cutoff frequency (lower standard deviation), and the third image is filtered to mid frequencies by band passing it between the two frequencies. The modified code is shown below:


%% Two cutoff frequencies and filters are used for three images
cutoff_frequency1 = 9;
cutoff_frequency2 = 2;

filter1 = fspecial('Gaussian', cutoff_frequency1*4+1, cutoff_frequency1);
filter2 = fspecial('Gaussian', cutoff_frequency2*4+1, cutoff_frequency2);

% High frequencies are removed from the first image using the first cutoff frequency
low_frequencies = my_imfilter(image1, filter1);

% Low frequencies are subtracted from the second image using the second cutoff frequency
high_frequencies = image2 - my_imfilter(image2, filter2);

% Mid frequencies are created by subtracting the lowest frequencies and filtering out the highest frequencies
mid_frequencies = my_imfilter(image3, filter2) - my_imfilter(image3, filter1);

% Combine the low, mid, and high frequencies
hybrid_image = low_frequencies + mid_frequencies + high_frequencies;

This triple hybrid image approach was tested with images of a lion, tiger, and bear (oh my!). The resulting images are shown below:

In the largest image, you can clearly see the high frequencies of the tiger. However, by around the third image, the mid frequencies of the bear are the most apparent. As the image continues to get smaller, the low frequencies of the lion predominate.