Project 1: Image Filtering and Hybrid Images

Example output of sobel filter

In this project I implemented the code to generate hybrid images. This was done by implementing the filtering operation and extracting low frequencies from one image and the high frequencies from the other image and summing it up. The low frequencies correspond to blurrier image which can be obtained by convolving the image with a guassian filter. The high frequency component of an image can be obtained by subtracting the blurrier component of the image.

Based on the examples provided the following five hybrid images were constructed
  1. cat and dog
  2. einstein and marilyn monroe
  3. bird and plane
  4. fish and submarine
  5. bicycle and motorcycle

Implementation details

I first implemented the code as a loop. Let filter_rows and filter_cols denote the number of rows and columns in the filter respectively. To calculate each pixel in the output image, I was doing filter_rows * filter_cols multiplication operations. This was very slow and took more than 5 minutes when the filter size was large. The next thing I tried was using the matlab inbuilt functions im2col and bsxfun which allowed me to unfold the image for each of the distinct filter positions (while sliding) and do a matrix vector multiplication as a single operation. This improved the speed of the code manifold. Simple padding with zeros was used

Example of code with highlighting

The following is the code snippet from my implementation


%example code
filter = imrotate(filter, 180);

%Get the size of input image and the filter
[rows, cols, channels] = size(image);
[filter_rows, filter_cols] = size(filter);

nr_rows_to_pad = (filter_rows-1)/2;
nr_cols_to_pad = (filter_cols-1)/2;
padded_image = padarray(image, [nr_rows_to_pad, nr_cols_to_pad ]);

output = zeros(rows, cols, channels);
for channel=1:channels;
  img = padded_image(:,:,channel);
  %Using the loop version of convolution was very slow. im2cols takes a
  %sliding convolution kernel(matrix) and emits one column for each
  %distinct position of the sliding matrix. The values are read columnwise.
  img_to_col = im2col(img, [filter_rows, filter_cols]);
  filter_as_col = reshape(filter, [filter_rows*filter_cols 1]);
  %multiply the filter with the col in a single operation
  conv = bsxfun(@times, img_to_col, filter_as_col);
  
  %Take the sum to compute the final value of the pixel
  conv = sum(conv);
  
  %Reshape back to an image
  output(:,:, channel) = col2im(conv, [1 1], [rows cols]);
end

The following is the output produced for the five pairs

Note that the filter was rotated by 180 degree before convolving with the image. This is because filter operation is actually convolving with the filter rotated by 180 degree.