Project 1: Image Filtering and Hybrid Images

hyber cat+dog

Fig.1. Example of a hybrid image made up of one image of cat and the other image of dog.

The project aims at utilizing an image filtering function to create hybrid images. The general idea to realize the situation that when people see the hybrid image at different distances, they are able to see different interpretations from the same hybrid image, which is achieved by manipulating the frequencies of the original images.

This project report mainly contains three parts shown as follows:

  1. Filtering

    1.1. Pad with Zeros

    1.2. Mirror the Image Boundary

  2. Hybrid Image

    2.1. Same Cutoff Frequency

    2.2. Different Cutoff Frequency

  3. Conclusion

1.Filtering

1.1. Pad with Zeros

Linear filtering operators involves weighted combinations of pixels in small neighborhoods, which means if the filter is desired to be centered on pixels at the image boundary, parts of the filter should be out of bounds. In order to solve this problem, various padding methods have been developed. The one that will be introduced here is padding with zeros. In other words, pixels outside the source image are set to zeros.

In this part, my_imfilter_Zero.m, DotProduct.m and proj1_test_filtering_Zero.m will be used. As shown from the following codes, the width and height of both image and filter are supposed to be acquired first in order to get the padding size. For zero padding, left zeros array and top zeros array are created which equal to right zeros array and down zeros array, respectively. Then, the new image with padding comes into being.

The core codes in my_imfilter_Zero.m and DotProduct.m are as below:


    %% The width and height of both image and filter
    [ImageWidth]=size(image,2);
    [ImageHeight]=size(image,1);
    [FilterWidth]=size(filter,2);
    [FilterHeight]=size(filter,1);
    
    %% Pad with Zeros
    PadLeft = zeros((FilterWidth-1)/2,ImageHeight)';
    PadUp = zeros(ImageWidth+(FilterWidth-1),(FilterHeight-1)/2)';
    
    %% Filtering
    for channel = 1:1:size(image,3)
        ImageColor = squeeze(image(:,:,channel));
        ImageNew(:,:,channel) = DotProduct(ImageWidth,ImageHeight,FilterWidth,FilterHeight,ImageColor,filter,PadLeft,PadUp);
    end
    

    %% Dot Product
    ImageNewR=zeros(ImageWidth,ImageHeight)';
    
    for WCordinate=1:1:ImageWidth
        for HCordinate=1:1:ImageHeight
            EachF=PadComplete(HCordinate:(FilterHeight-1)+HCordinate,WCordinate:(FilterWidth-1)+WCordinate);
            ImageNewR(HCordinate,WCordinate)=sum(sum(EachF.*filter));
        end
    end
    

The core idea of filtering is to get the filtered image's each pixel by calculating the sum of the dot product of the neighboring matrix of each pixel in the padding image and the filter which is accomplished in DotProduct.m. In order to let the code work for both color image and grayscale image, I write a for loop to determine whether to do the dot product for just once or three times for each R, G, B channel.

Then simply using proj1_test_filtering_Zero.m, we can get the images after using identify filter, small blur box filter, large blur filter, oriented filter as well as two different high pass filters-one using Discrete Laplacian and the other simply subtracting low frequency content.

The original images and the results of filtered image are displayed as follows:

Fig.2. Original marilyn image and Filtered marilyn image using the method of zero padding

Fig.3. Original dog image and Filtered dog image using the method of zero padding

It is easily to be observed from the figures that for both filtered color and filtered grayscale images, zero padding darkens the edges. Moreover, if you take a look at the third and fourth figure, it can be easily seen that the fourth one using Gaussian filter is more smooth than the third one using box filter due to the fact that Gaussian filter removes 'high-frequency' components from the image which means it is a loss-pass filter.

1.2. Mirror the Image Boundary

The other method used in this project is mirroring the image boundary which means reflecting pixels across the image edge.

In this session, my_imfilter_Mirror.m, DotProduct_Mirror.m and proj1_test_filtering_Mirror.m will be used to realize the filtering function. The methods of obtaining the size of image and filter, doing dot product and making it work for both color and grayscale images are similar. The only difference is to replace zero components with the boundary components of the image. I still tried to use the padding method utilized in the first session at first but it is a little complicated thus, I choose to use the padarray(A, PADSIZE, 'symmetric') function which reduces much code. The code for this is shown as follows:


    %% Filtering
    for channel = 1:1:size(image,3)
        ImageColor=squeeze(image(:,:,channel));
        PadComplete=padarray(ImageColor,[(FilterHeight-1)/2,(FilterWidth-1)/2],'symmetric');
        ImageNew(:,:,channel)=DotProduct_Mirror(ImageWidth,ImageHeight,FilterWidth,FilterHeight,PadComplete,filter);
    end
    

The results of using various filters are shown as below:

Fig.4. Filtered marilyn image using the method of mirror padding

Fig.5. Filtered dog image using the method of mirror padding

From the upon figures, it can be seen that the image after using identify filter does nothing regardless of the padding method used. Besides, it is obviously that the darkening edges disappear, owing to mirror padding's preserving colors near the borders, compared to the previous results in the padding with zeros part. Additionally, the conclusion for identify filter and the comparison between Gaussian filter and box filter is still effective.

2.Hybrid Image

Hybrid image is a technique that changes the two interpretations of the static images as a functions of viewing distance. And it is generated by superimposing two images-one is filtered with a low-pass filter and the other one is filtered with a high-pass filter. There are two parameters defining the hybrid images which are the frequencies cut of the low resolution image and high resolution image respectively. Thus, I will first try to use the same cutoff frequencies of both low and high resolution image. And then use two different cutoff frequencies for two images respectively.

In this session, proj.m, proj_DiffCutoffFreq.m, my_imfilter_Mirror.m and DotProduct_Mirror.m will be used for hybrid images since the method of mirroring the image boundary shows better results than the method of just padding with zeros. The code for hybrid image is shown as follows and Gaussian filter is applied for the low frequency part. For the high frequency part, it can be easily obtained by subtracting the low frequency part from the original image. This portion of code is displayed here:


    filter = fspecial('Gaussian', cutoff_frequency*4+1, cutoff_frequency);
    low_frequencies = my_imfilter_Mirror(image1,filter);
    high_frequencies = image2-my_imfilter_Mirror(image2,filter);
    hybrid_image = high_frequencies + low_frequencies;

2.1. Same Cutoff Frequency

The following series of figures - low frequency image, high frequency image, hybrid image and hybrid images in different scales - are the resulting images for the same cutoff frequency:

Fig.6. Original two images and Hybrid image: low Frequency-dog, high frequency-cat, cutoff frequency=7.

Fig.7. Original two images and Hybrid image: low Frequency-marilyn, high frequency-einstein, cutoff frequency=3.5.

Fig.8. Original two images and Hybrid image: low Frequency-fish, high frequency-submarine, cutoff frequency=6.5.

Fig.9. Original two images and Hybrid image: low Frequency-bird, high frequency-plane, cutoff frequency=4.5.

Fig.10. Original two images and Hybrid image: low Frequency-motorcycle, high frequency-bicycle, cutoff frequency=4.

I also try to use other set of images that I have found which are very similar other than just using those given to make hybrid images. The first two figures are the original ones - batman and cat. The following results of images are for low frequency, high frequency, hybrid image and hybrid image in different scales.

Fig.11. Original two images and Hybrid image: low Frequency-cat, high frequency-batman, cutoff frequency=2.5.

We can see that most of the hybrid images have good interpretations but for the bicycle-motorcycle and plane-bird hybrid image, the motorcycle and bird seems to dominate the whole images respectively. Therefore, I will then try to set two different cutoff frequencies to see if there is any improvement.

2.2. Different Cutoff Frequency

The following series of figures - low frequency image, high frequency image, hybrid image and hybrid images in different scales - are the resulting images for the different cutoff frequency:

Fig.12. Hybrid image: low Frequency-motorcycle, high frequency-bicycle, motorcycle cutoff frequency=8, bicycle cutoff frequency=3.

Fig.13. Hybrid image: low Frequency-bird, high frequency-plane, bird cutoff frequency=7, plane cutoff frequency=2.5.

It apparently indicates that after setting the cutoff frequencies for motorcycle and bird to higher values while the cutoff frequencies for bicycle and plane to lower values, the hybrid images shows better results which means if the cutoff frequencies of both original images are manipulated properly, more perfect hybrid image results are capable of achieving than the one just simply setting them the same.

3.Conclusion

To sum up, the first part completes the padding as well as filtering work and tests different kinds of filters from which we can get the conclusion that the padding method of mirroring the image boundary reveals better results than that of zero padding due to the disappearing of darkening margin. Besides, the results using Gaussian filter are more smooth than that using box filter. For the hybrid image portion, it is obviously that better hybrid images are able to acquire if different cutoff frequencies are assigned properly.

Thanks for watching. This is all my work for CS 6476-Project 1-Image Filtering and Hybrid Images. I really appreciate your time. If you have any problem, please contact me and my email address is xwu72@gatech.edu.