Project 1: Image Filtering and Hybrid Images

The objective of this project is to implement a image filtering function and use it to create hybrid images. Hybrid images are images that are interpreted differently based on the viewing distance. This is possible due to the ability of the human vision to perform multiscale processing of images. This awesome illusion can can be obtained by combining two images that are filtered such that one image has only low frequency components while the second image has only high frequency components as specified in the SIGGRAPH 2006 paper by Oliva, Torralba, and Schyns. Apart from combining two images at two different spatial scales, producing aesthetically pleasing hybrid images is also dependent upon the alignment, perceptual grouping and color of the two component images. In this project, a filtering function is implemented that replicates the functionality of the native imfilter provided by MATLAB and hybrid images are obtained for different set of images while studying the effect of filtering and cut-off frequency.

Examples of hybrid image.

Implementing a Spatial Filter

Filtering is a neighborhood operation, in which the value of any given pixel is determined by applying some algorithm over the neighbouring pixels. In the spatial domain, filtering is obtained by taking the dot product of the filter over all the pixels of the image. However for filters with dimension greater than one, the filter cannot be applied to the boundary pixels. To overcome this problem we need to pad the image so that the filter can be applied over all the pixels. Several different padding techniques were experimented with and the symmetric padding technique was used as it provides a reasonable way to replicate content close to reality. Thus the following steps were followed in implementing a filtering function that replicates the imfilter function provided by MATLAB.

  1. Pad the image based upon the filter size - half of the filter size in each dimension
  2. Iterate over the depth i.e. colors of the image
  3. For each color, iterate over the rows
  4. For each row, iterate over the columns
  5. For each column i.e. a pixel, take the dot product between the filter and the pixel with its neighborhood

    6. Code Snippet 

    
%example code
for num_color=1:size(image,3)
    r_offset = (r_pad_size-1)/2;
    c_offset = (c_pad_size-1)/2;
    for r=(size(filter,1)+1)/2:size(image,1)- r_offset
        for c=(size(filter,2)+1)/2:size(image,2)- c_offset
            %Apply the dot product and take sum of the elements
            output(r-r_offset,c-c_offset,num_color)=sum(sum(filter.*image(r-r_offset:r+r_offset, 
            c-c_offset:c+c_offset, num_color)));
        end
    end
end

Generating hybrid images

Hybrid images are obtained by combining two images at two different spatial scales. We need to combine a low pass filtered image with a high pass filtered image. The cut-off frequency is tuned for every pair of images by observing the output hybrid image and varying the same to get a visually pleasing hybrid image. Below steps are used to obtain the hybrid image

  1. Select a cut-off frequency
  2. Filter one of the image using low pass filter
  3. Apply high pass filter i.e. Image - filter(Image) to the second image
  4. Add both the images to obtain the hybrid image

    5. Code Snippet 

    
%example code
low_frequencies = my_imfilter(image1,filter); % Low pass filter applied to image 1
high_frequencies = image2 - my_imfilter(image2, filter); % High pass filter applied to image 2
hybrid_image = low_frequencies + high_frequencies; % Generating hybrid image

Results

Image 1 Image 2 Hybrid Downsampled Version Cut-off Frquency
7
4
4
6
8
7

The above filter was tested over the given 5 pairs of closely aligned images. The cut-off frequency was varied for each pair and the frequency that produced an aesthetically pleasing hybrid image was selected. The Tech Tower and Bank of America Tower, two of the most iconic buildings in Atlanta were combined to produce a hybrid image. Both the images were aligned before producing the hybrid image. It is evident how the Tech Tower morphs into the Bank of America Building as a function of the viewing distance

Conclusion

A spatial filter was implemented that can support multidimensional images and arbitrary shaped filters. The filter was tested by generating hybrid images for several pair of images. The effect of cut-off frequency, alignment, color and perceptual grouping on the generation of hybrid images as mentioned in the paper "A. Oliva, A. Torralba, P.G. Schyns (2006). Hybrid Images. ACM Transactions on Graphics, ACM Siggraph, 25-3, 527-530." were observed and analysed.