Project 1: Image Filtering and Hybrid Images

The final result

The core of this project is the algorithm that filters the image using a provided kernel. Below I describe a few optimizations and extensions that attempt to maximize the utility and speed of this implementation. I also present several examples of the hybrid image pipeline in operation.

The Hybrid Image Algorithm

Each image below is an example of hybridizing two images by taking only the high frequencies from the first image, and the low frequencies from the second. These frequencies are separated using a Gaussian filter. This removes the high frequencies from an image, leaving only the low frequencies. To produce only the high frequencies, the algorithm simply subtracts the low frequencies from the raw image, leaving only the high frequencies. Then the high frequencies from the first image are averaged with the low frequencies from the second image, by computing the mean of the two images for each channel.

Low Frequency High Frequency Final Result

Various Sizes

This is a useful interpretation, because you can begin to see the low frequency image (shown on the right for reference) as the size is decreased. For example, in the smaller images above, you can see the dog, and Marilyn Monroe, respectively.

Implementation

There are two salient features of this implementation:
  1. Border Option Passthrough
  2. Vectorization Optimizations

Border Option Passthrough

This implementation uses padarray to pad the image before convolution. This has the added benefit of facilitating other padding methods such as symmetic padding and "replicate" padding automatically. This requires adding a varargin parameter to the function signature, but in turn allows these modes to be used automatically.


function output = my_imfilter(image, filter, varargin)

...

borderOption = 0;
if size(varargin, 1) == 1
    borderOption = varargin{1};
end

...

newImage = padarray(image, [verticalPad, horizontalPad], borderOption);

end

This allows the 'symmetric' and 'replicate' border options to be passed directly through to the padarray function without affecting normal operation.

Vectorization Optimizations

Most Matlab functions are implemented in C or Fortran, making native Matlab functions several orders of magnitude faster than an equivalent function written in Matlab itself. Therefore, this implementation attempts to maximimize the amount of native matlab functions used.


for y = 1:imageHeight
    for x = 1:imageWidth
        
        % Make the filter have the same number of dimensions as channels
        depthFilter = repmat(filter, 1, 1, channels);

        % Extract the ROI to convolve with
        ROI = newImage(y:y+(filterHeight-1), x:x+(filterWidth-1), :);

        % Compute each channel's dot product separately
        convMat = dot(double(ROI), depthFilter, 4);
        % Sum the products
        outputValue = sum(sum(convMat, 1), 2);
        % Store at the current location
        output(y, x, :) = outputValue;
    end
end

This section of code performs the actual filtering of the image. Instead of iterating over each channel, it creates a filter that has the same depth as the image, and uses the built-in dot product function to compute the prodcts of each filter element and each image element. Finally, it sums over the x and y dimensions to get the output of that convolution. This value is stored directly into the current location on the output image.