Computer Vision Project

Project 1: Image Filtering and Hybrid Images

In this project, we superimpose a lowpass filtered image with a highpass filtered image to create an image whose interpretation by a person depends on their distance from the image.

Images of Interest

We consider some pairs of images to form some hybrid images. From left to right, these pairs are a cat and a dog, Marilyn Monroe and Albert Einstein, and a plane and a bird.

Filtering

First, my code zero pads the original image for the corner cases. Then, the filter walks over each of the windows in the image and performs the dot product to compute the output pixel.

The my_imfilter code is shown below. Speed-ups come from using the dot and sum functions to perform the dot product and from operating over all color channels simultaneously.


function output = my_imfilter(image, filter)

if any(rem(size(filter), 2) == 0)
    error('Both filter dimensions must be odd');
end

output = zeros(size(image));
offsets = floor(size(filter)/2);
padded_image = padarray(image, offsets);
matrix_dot_product = @(A, B) sum(dot(A, B), 2);
filter_repmat = repmat(filter, 1, 1, size(image, 3));
for i = 1:size(image, 1)
    for j = 1:size(image, 2)
        rows = i:i+size(filter,1)-1;
        cols = j:j+size(filter,2)-1;
        output(i,j,:) = matrix_dot_product(padded_image(rows, cols, :), filter_repmat);
    end
end

Filtering results

Some filtering results are shown above. From left to right, these results come from the identity filter, a Gaussian filter, a Sobel filter, and a Laplacian filter. The Gaussian filtered image has some dark pixels at the borders due to the zero padding in the original image.

Hybrid images from lowpass and highpass filters

The first column shows the lowpass filtered images, the second shows the highpass filtered images, and the third column shows the resulting hybrid images at different sizes. At larger sizes, the higher frequencies (i.e. edges) are present with colors (like the dog's fur color or Einstein's skin and hair color) being the only thing present from the lower frequencies. As the hybrid image gets smaller, the higher frequencies become less noticeable and the image looks just like the pre-lowpass filtered image.

Hybrid images from bandpass and bandstop filters

We can also form hybrid images that look like one image from near and far and look like a different image in between. To do this, we need to use a bandpass filter to keep some intermediate frequencies and a bandstop filter to remove those same intermediate frequencies. To form the bandpass filter, I based my code on the MATLAB tutorial linked here that makes use of the window method, a method which converts a filter of infinite extent (like the sinc filter) to one of finite extent. The code to generate the filter is shown below.


function filter = make_bandpass(filter_size, sigma, cutoff1, cutoff2)
%MAKE_BANDPASS Forms a 2-D bandpass filter.
%   FILTER = MAKE_BANDPASS(FILTER_SIZE, SIGMA, CUTOFF1, CUTOFF2) returns a
%   2-D bandpass filter based on the window method. FILTER_SIZE can be
%   either a number or a vector of two numbers. SIGMA is the standard
%   deviation of the Gaussian filter in pixels. CUTOFF1 and CUTOFF2 are the
%   cutoff frequencies for the bandpass filter, with CUTOFF1 < CUTOFF2 and
%   each variable being between 0 and 1.

if cutoff >= cutoff2
    error('First cutoff frequency must be less than second cutoff frequency.');
end

if any([cutoff1 cutoff2] < 0 | [cutoff1 cutoff2] > 1)
    error('Cutoff frequencies must be between 0 and 1.');
end

[f1, f2] = freqspace(filter_size, 'meshgrid');
Hd = ones(filter_size);
r = sqrt(f1.^2 + f2.^2);
Hd((r < cutoff1) | (r > cutoff2)) = 0;
win = fspecial('gaussian', filter_size, sigma);
win = win ./ max(win(:));
filter = fwind2(Hd, win);

Once the bandpass filter is formed, we use it and its complementary bandstop filter to form the hybrid image. The relevant code and some sample images are shown below.


function make_bandpass_hybrid_image(name1, name2, filter_size, sigma, cutoff1, cutoff2)
%MAKE_BANDPASS_HYBRID_IMAGE forms the bandpassed, bandstopped, and hybrid
%images.
%   MAKE_BANDPASS_HYBRID_IMAGE(NAME1, NAME2, FILTER_SIZE, SIGMA, CUTOFF1,
%   CUTOFF2) forms the bandpassed, bandstopped, and hybrid images. NAME1
%   refers to the name of the image to be bandstop filtered and NAME2 to
%   the image to be bandpass filtered. FILTER_SIZE can be either a number
%   or a vector of two numbers. SIGMA is the standard deviation of the
%   Gaussian filter in pixels. CUTOFF1 and CUTOFF2 are the cutoff
%   frequencies for the bandpass filter, with CUTOFF1 < CUTOFF2 and each
%   variable being between 0 and 1.

image1 = im2single(imread(sprintf('../data/%s.bmp', name1)));
image2 = im2single(imread(sprintf('../data/%s.bmp', name2)));
filter = make_bandpass(filter_size, sigma, cutoff1, cutoff2);
bandpassed_frequencies = my_imfilter(image2, filter);
bandstopped_frequencies = image1 - my_imfilter(image1, filter);
hybrid_image = bandpassed_frequencies + bandstopped_frequencies;
figure(1); imshow(bandpassed_frequencies+0.5);
figure(2); imshow(bandstopped_frequencies);
vis = vis_hybrid_image(hybrid_image);
figure(3); imshow(vis);

The first column shows the bandstop filtered images, the second shows the bandpass filtered images, and the thrid column shows the resulting hybrid images at different sizes. Unlike the highpass filter, the bandpass filter seems to find the edges and then blur them, such as with Marilyn Monroe's blurred outline. The bandstop filter, on the other hand, seems to do selective blurring as it removes certain frequencies. This is most noticeable with Einstein as his face is mostly unaffected, but his clothes and hair have a glow to them.