Weiyang Liu>

Project 1: Image Filtering and Hybrid Images

Image Filtering

Image filters are one of the most important building blocks in computer vision. On one hand, they serve as a fundamental tool to operate images, e.g., add soft blur, sharpen details, accentuate edges, remove noise, etc. On the other hand, they also can be viewed as the stacked units of successful convolutional neural networks (CNNs), execept that CNNs have further imposed non-linearity. Specifically, image filters use a collection of pixel values in the vicinity of a given pixel to determine its final output value.

Hybrid Images

The hybrid images explore the human visual perception for different visual frequencies. Hybrid images, by definition, refer to static images that are generated by superimporsing two images at two different spatial scales. Specifically, the low-spatial scale is obtained by filtering one iamge with a low-pass filter, while the high spatial scale is obtained by filtering a second image with a high-pass filter. The final image is composed by adding these two filtered images.

Project Objective

1. Finish a convolutional filter with given filtering template.
2. Generate the hybrid images by combinating low-frequency images and high-frequency images.
3. Give my own hybrid image examples.

Experimental Details

For image filtering, I adopt the suggested pipeline. First, I extend the original image in a mirror fashion, and then start convolution operation. My detailed code is shown as follows:

%% whether the input is a path of image
if ischar(image)==1
    image_pixel = imread(image);
else
    image_pixel = image;
end
%% define parameters
image_height =  size(image_pixel,1);
image_width = size(image_pixel,2);
channel_num = size(image_pixel,3); % Gray-scale as 1. Color as 3.
filter_height = size(filter,1);
filter_width = size(filter,2);
new_image = zeros(image_height+filter_height-1,image_width+filter_width-1,3);
%% extend the original image for filtering (mirror extension)
for i = 1:channel_num
    % upper left
    new_image(1:(filter_height-1)/2,1:(filter_width-1)/2,i) = ...
        image_pixel(1:(filter_height-1)/2,1:(filter_width-1)/2,i);
    % upper right
    new_image(1:(filter_height-1)/2,end-(filter_width-1)/2+1:end,i) = ...
        image_pixel(1:(filter_height-1)/2,end-(filter_width-1)/2+1:end,i);
    % lower left
    new_image(end-(filter_height-1)/2+1:end,1:(filter_width-1)/2,i) = ...
        image_pixel(end-(filter_height-1)/2+1:end,1:(filter_width-1)/2,i);
    % lower right
    new_image(end-(filter_height-1)/2+1:end,end-(filter_width-1)/2+1:end,i) = ...
        image_pixel(end-(filter_height-1)/2+1:end,end-(filter_width-1)/2+1:end,i);
    % upper middle
    new_image(1:(filter_height-1)/2,(filter_width-1)/2+1:end-(filter_width-1)/2,i) = ...
        image_pixel((filter_height-1)/2:-1:1,:,i);
    % left middle
    new_image((filter_height-1)/2+1:end-(filter_height-1)/2,1:(filter_width-1)/2,i) = ...
        image_pixel(:,(filter_width-1)/2:-1:1,i);
    % right middle
    new_image((filter_height-1)/2+1:end-(filter_height-1)/2,end-(filter_width-1)/2+1:end,i) = ...
        image_pixel(:,end:-1:end-(filter_width-1)/2+1,i);
    % lower middle
    new_image(end-(filter_height-1)/2+1:end,(filter_width-1)/2+1:end-(filter_width-1)/2,i) = ...
        image_pixel(end:-1:end-(filter_height-1)/2+1,:,i);
    % original image
    new_image((filter_height-1)/2+1:end-(filter_height-1)/2,(filter_width-1)/2+1:end-(filter_width-1)/2,i) = ...
        image_pixel(:,:,i);
end
%% compute the image pixel value after convolution filtering
for c = 1:channel_num
    for i = 1:image_height
        for j = 1:image_width
            A = filter.*new_image(i:i+filter_height-1,j:j+filter_width-1,c);
            output(i,j,c) = sum(A(:));
        end
    end
end

For hybrid image generation, I also use the very straightforward way as suggested. I generate the low-frequency image using the Gaussian filter and generate the high-frequency image using the subtraction of a Gaussian filtered image from the original one. The code is simple, as shown below:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Remove the high frequencies from image1 by blurring it. The amount of
% blur that works best will vary with different image pairs
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
low_frequencies = my_imfilter(image1,filter);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Remove the low frequencies from image2. The easiest way to do this is to
% subtract a blurred version of image2 from the original version of image2.
% This will give you an image centered at zero with negative values.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
high_frequencies = image2 - my_imfilter(image2,filter);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Combine the high frequencies and low frequencies
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
hybrid_image = low_frequencies + high_frequencies;