Computer Vision Project 1

Project 1: Image Filtering and Hybrid Images

A hybrid image is one that combines the low-spatial frequencies of one image with the high spatial frequencies of another image. For humans, high frequencies dominate visison at close distances while low frequencies become more significant as one moves further away from the image. Therefore, it is possible to combine two images into one such that it resembles one of them at close range and the other when you move further away.

In this project, we create Hybrid images using Matlab.

Example 1

In this example, we use the following images of a cat and a dog to combine images.

Resolving images to high and low frequencies and combining them into a hybrid, we get the following image,

If you look at the above image from a close range, it looks like a cat. However, the further you move away, the further it resembles a dog. We can formalize this by shrinking the image to various lengths using downsampling as shown below.

Code

We use a Gaussian filter to blur the image and filter only the low pass frequencies through. The high frequencies are obtained by subtracting the low-frequency image from the original image.


filter = fspecial('Gaussian', cutoff_frequency*4+1, cutoff_frequency);

Here, 'cutoff_frequency' is a parameter we tune to give us the best results.

Before we apply the filter to the image, we must pad it so that we can apply the filter to elements at the corners.


I = im2double(image);

[dim1, dim2, ~] = size(I);
[fil1, fil2] = size(filter);

padsize1 = floor(fil1/2)
padsize2 = floor(fil2/2)

paddedI = padarray(I, [padsize1 padsize2], 'symmetric', 'both');

We then apply the filter to every element of the image.


zerosI = zeros(size(paddedI));

for i=1+padsize1:dim1+padsize1
    for j=1+padsize2:dim2+padsize2
        for k=1:3
            box = paddedI(i-padsize1:i+padsize1, j-padsize2:j+padsize2, k);
            val = box.*filter;
            zerosI(i,j,k) = sum(sum(val));
        end
    end
end

outputI = zerosI(1+padsize1:padsize1+dim1, 1+padsize2:padsize2+dim2, :);

Here, 'outputI' represents the output image that is returned.

Benchmarks

Here, we test our filtering function against standard known filters to check whether the results are correct.


Identity Filter	Small Blur Filter	Large Blur Filter

Sobel Filter	Laplacian Filter	High Pass Filter

Example 2

In this example, we will try to combine The Scream with The Son of Man.

The hybrid image obtained is,

And downsampling it, we get

Improving Code Efficiency

The filtering code above goes through every pixel and calculates its new value based on the filter. Since the filter is fixed, we can improve the run time of the code by applying the filter to all elements of the array at once. The most straightforward way of doing this is by applying the filter pixel by pixel to the entire image.

Thus, given a filter and an image, we go through the filter pixel by pixel and apply it to the entire image. However, all pixels in the filter apply only to the element at the center. Therefore, we must shift the intermediate result array so that the filter pixel acts on the right image pixel.

Since we apply the multiplication operation to the entire array at once, Matlab can speed it up for us. Therefore, we cut down on the runtime.

The preliminary setup code is the same as in the other implementation. The filtering/convolution operation is performed as follows,


zerosI = zeros(size(paddedI));

for i=1:fil1
    for j=1:fil2
        val = filter(i,j);
        mulpadI = val*paddedI;
        sumI = circshift(mulpadI,[padsize1-i, padsize2-j]);
        zerosI = zerosI+sumI;
    end
end

outputI = zerosI(1+padsize1:padsize1+dim1, 1+padsize2:padsize2+dim2, :);
size(outputI);

We used Run and Time in Matlab to measure how the improved implementation fared against the original.

For the Cat-Dog images,