Oleg Filatov

Project 1: Image Filtering and Hybrid Images

Implementing my_imfilter

Since the task was to implement our own version of MATLAB's built-in imfilter function, first, the understanding of what imfilter does is crucial. On the most surface level, imfilter can be described as a function that takes in an image and a filter as inputs, and outputs the image with the filter applied to it. To implement it ourselves, however, this explanation isn't nearly detailed enough.

A filter can be represented as a 2D matrix of values. An image can be represented as a 3D matrix (e.g., RGB images). To apply filter, we need to perform convolution operation on every pixel in the original image, meaning that the filter matrix needs to be rotated 180 degrees first, and then find the new pixel value by performing the element-wise multiplication operation on the filter and a slice of the original image matrix that is the same size as the filter and with the pixel in question serving as the center, and then sum all of the product results to get the new value for the pixel at this location. The rotated filter matrix is also known as kernel, mask, or convolution matrix. In case of an image not being greyscale, we just need to visualize the image matrix as z separate 2D matrices, where z represents the number of elements representing each individual pixel (e.g., 3 matrices in case of RGB), and run convolution on those separately visualized matrices one at a time while at the same time assembling the resulting output matrix with the same dimensions as the original one. For the purpose of this project, we were told that performing rotations on filters was not necessary because all of them are symmetric (thus the rotation would change exactly nothing).

Another important thing to consider while implementing is padding, since filters use values of surrounding pixels in order to calculate new value for a chosen pixel, and there are cases when there are no surrounding pixels or not enough of them (e.g., corner/edge pixels). For this task, I implemented my own padding functionality (which uses black pixels as padding):

pad_row_num = floor(size(filter, 1)/2);
pad_col_num = floor(size(filter, 2)/2);

img_height = size(image, 1);
img_width = size(image, 2);
img_depth = size(image, 3);

row_padding = zeros(pad_row_num, img_width + 2*pad_col_num, img_depth);
col_padding = zeros(img_height, pad_col_num, img_depth);

new_image = [col_padding image col_padding];

new_image = [row_padding; new_image; row_padding];

Part of the code dealing with actual application of the filter matrix on the image can be seen below. The code works just fine whether it is grayscale or RGB:

for i=(1+pad_row_num):1:(size(new_image, 1)-pad_row_num);
    for j=(1+pad_col_num):1:(size(new_image, 2)-pad_col_num);
        for k=1:1:size(new_image, 3);
            surrounding_matrix = reference_img(i-pad_row_num:i+pad_row_num, j-pad_col_num:j+pad_col_num, k);

            conv_matrix = surrounding_matrix .* filter;

            s = sum(conv_matrix(:));

            new_image(i, j, k) = s;
        end
    end
end

Before returning the output, we should remember to remove the padding we created, so that the image has its original dimensions:

row_cutoff = size(new_image,1) - pad_row_num;
col_cutoff = size(new_image,2) - pad_col_num;

new_image = new_image(1+pad_row_num:row_cutoff, 1+pad_col_num:col_cutoff, :);

output = new_image;

Testing my_imfilter

To test my_imfilter I simply ran proj1_test_filtering as suggested. The results can be seen below, with images corresponding to filter descriptions in the same order:

1. Identity matrix: a matrix that can be represented as a square matrix with 1 in the center and 0's everywhere else. Applied to an image, should return the original image because of how applying a matrix to an image works (which was discussed above).
2. Gaussian filter: a smoothing low-pass filter that uses Gaussian function to blur the image. The higher the standard deviation passed to the Gaussian function, the stronger the blur effect is.
3. Laplacian filter: an edge detection filter that works by making areas of high intensity gradient stand out. A 2nd order derivative operator.
4. Sobel filter: an edge detection filter similar to Laplacian filter, except it is a 1st order derivative operator. What visually makes it different from Laplacian filter, due to the nature of the algorithm, is noise sensitivity and anisotropy.

Hybrid images

For the purpose of this assignment, we refer to hybrid images as images that are created by applying a low-pass filter to one picture, a high-pass filter to another (that cuts off at the same frequency as the low-pass one), and then adding them together. At an optimal cutoff frequency (which differs for every pair of images), an illusion of an object morphing into another one is created as the distance between the observer and the image is increased/decreased. The distance change can also be emulated by simply decreasing the size of the image. The effect is achieved due to the fact that while looking at objects far away, human brain does subsampling and changes the perceived dominant frequencies. The choice of the cutoff frequency affects the distance/size at which the moprhing will happen. Below, you can see all of the hybrid images created using image pairs provided to us, as well as the cutoff frequencies I found to be the best for each image through the trial and error method.

Cutoff frequency value used: 4

Cutoff frequency value used: 7

Cutoff frequency value used: 5

Cutoff frequency value used: 6

Cutoff frequency value used: 3