Computer Vision Project

Project 1: Image Filtering and Hybrid Images

The project aims to implement a MATLAB function to do image filtering, and then use it to create hybrid images [1]. Filtering involves an element-wise multiplication of entries of an image matrix I with a filter F.

Hybrid images are created by filtering one image to retain the low frequencies, filtering another image to remove them and then adding the two. With appropriate frequency cutoffs and image alignment, one can get a resultant image which looks like different images at different resolutions.

Filtering

Zero Padding

In order to find the value of the boundary elements after filtering, the original image first needs to be padded with zeroes on all sides. This is done using the padarray function in MATLAB. The code snippet below will pad an input image with (n-1)/2 zeroes, which is he minimum number required to meaningfully process the boundary elements.

%filCols and filRows are the number of columns and rows in the filter respectively. 
padded_image = padarray(image,[(filCols-1)/2 (filRows-1)/2]);

Implementation

After this, the actual filtering is done. A basic implementation would involve one for loop to iterate over each image channel, two to iterate over each element in the image and two more to iterate over the filter elements and then adding them together. A slightly improved code which vectorises some of the elements for faster processing is shown below.

%% Filtering
col_iterator=(filCols+1)/2:imCols+(filCols-1)/2; % A vector corresponding to the column containing the centre pixel
row_iterator=(filRows+1)/2:imRows+(filRows-1)/2; % A vector corresponding to the row containing the centre pixel

for i=1:imchannels
    for l=-(filCols-1)/2:(filCols-1)/2
        for m=-(filRows-1)/2:(filRows-1)/2
            output(col_iterator,row_iterator,i) = output(col_iterator,row_iterator,i)...
            		+ padded_image(col_iterator+l,row_iterator+m,i)*filter(l+(filCols+1)/2,m+(filRows+1)/2);
        end
    end
end

Correctness

The image matrix obtained after filtering does not match exactly with the one obtained using imfilter, possibly due to differences in padding methods or floating point errors. To obtain a measure of the closeness of the two outputs, one possible way could involve taking the norm of the error between the two matrices.

error=0;
for i=1:imchannels
	error=error + norm(image_myimfilter(:,:,i) - image_imfilter(:,:,i));
end

A table showing a comparison of output of the implemented function with imfilter for a given test image, and the error norm as described above, is shown below.

Output
Output of imfilter
Error Norm	0	9.2679e-06	8.9322e-06	1.0880e-05	9.5166e-06

Hybrid Images

Two images are taken and filtered with a Gaussian blur. A high frequency image is obtained by subtracting low frequencies from the original image. The MATLAB code for creating a hybrid image from two images is shown below:

filter1 = fspecial('Gaussian', cutoff_low*4+1, cutoff_low);
low_frequencies = my_imfilter(image1,filter1);

filter2 = fspecial('Gaussian', cutoff_high*4+1, cutoff_high);
high_frequencies = image2 - my_imfilter(image2,filter2);

% Combine the high frequencies and low frequencies
hybrid_image = low_frequencies + high_frequencies;

The figure above shows the creation of ahybrid image from a fishand a submarine. A filtered image of the fish containing low frequencies is combined with a high frequency image of the submarine and the last image in the sequence is the resultant hybrid image. The same image, when seen at different resolutions looks as follows:

What looks like a submarine from up close, when seen at a lower resolution, looks like a fish because our eyes only retain low frequencies when seeing smaller images or things from afar. Some more hybrid images at two different resolution scales are shown below:

Results in a table

Observations

The cutoff frequency needs to be tuned for each image separately. Higher values of cutoff frequency result in more blurring of the low frequency image and are thus better to use for sharper images.
Hybrid images which do not superimpose correctly seem to retain some high frequency halos even when viewed at low resolutions.
Some images worked better in the greyscale mode. A hybrid between a motorcycle and a bicycle where the motorcycle's low frequencies and bicycle's high frequencies are kept does not seem natural in the RGB mode because the eyes cannot make sense out of the red colour which seems to appear around the handlebars. The same, however, in the greyscale mode can be made to look like a shadow, thus making the image more believable.

References

[1] Oliva, Aude, Antonio Torralba, and Philippe G. Schyns. "Hybrid images." ACM Transactions on Graphics (TOG). Vol. 25. No. 3. ACM, 2006.

Sarang Joshi