Project 1: Image Filtering and Hybrid Images
Image filtering
Image of a cat that we will play with.
Image filtering is implemented using a straight-forward double for loop that iterates through every single pixel in the image. At each pixel, we extract the subwindow which has the pixel at the center and the size equal to the filter's size. We then do a dot product between the subwindow and the filter matrix. The result is a scalar product, which will then be the new value of the pixel.
The following code snippet shows how this is done in MATLAB:
for x = 1 : size(image, 1)
for y = 1 : size(image, 2)
sub_window = mono_layer(x : x + size(filter, 1) - 1, y : y + size(filter, 2) - 1);
new_layer(x, y) = sum(sum(sub_window .* filter));
end
end
Example of a padded image (padding size is 20 pixels).
But how do we handle the pixels around the edges? The subwindow can't fit there. One solution to this problem is to pad the images with black pixels (or zeros in technical terms). To determine how much we should pad on the left, we look at the size of the filter. Let's say the filter has size m x n (or m rows and n columns). Remember that in the algorithm above we cut out a subwindow m x n around the current pixel. There are (n-1)/2 pixels on the left and (n-1)/2 pixels on the right of the current pixel (provided that n is an odd number). There are also (m-1)/2 pixels above and (m-1)/2 pixels below the current pixel. Therefore, if we just pad the left and the right of the image with (n-1)/2 and the top and bottom with (m-1)/2, the subwindow will always be valid.
The padding can be done as follows:
pad_matrix = floor(size(filter) ./ 2);
padded_image = padarray(image, pad_matrix);
Note that flooring here has the same effect as subtracting one before division: floor(n/2) = (n-1)/2 when n is odd. However, I did it this way so that it would work for even n also (which is irrelevant for this project).
We're almost there!
To make a fully functional image filtering, we need to be able to handle color images. The way we handle it is to work on each color channel separately. In this case, we are using RGB images, so we need to loop through the 3 different channels, perform the filtering, and update the layer.
Examples of image filtering
Let's try out a few image filters on the cat image:
- Identity filter
- Small blue with a box filter
- Large blur
- Oriented filter (Sobel operator)
- High pass filter (Discrete Laplacian)
- High pass filter alternative
|
|
The algorithm runs fairly fast because most filters have small size. The bottleneck is likely to be the double for loop as MATLAB doesn't handle loops very well.
An improvement for this algorithm is to use reflection instead of padding. With padding, you can see the black borders in the blur filters, which is an undesirable result.
Hybrid image
A hybrid image is the result of taking the sum of the low frequencies from one image and the high frequencies from a second image.
First, we want to extract the low frequencies from the first image. To do this, we simply blur out all the high frequencies. The blur filter we use is Gausian. By changing the parameters of the Gausian filter (hsize and sigma), we can fine tune the range of low frequencies we will extract.
Next, we take the high frequencies from the second image by subtracting the original image with a blurred version.
Finally, we add both the low-frequency and high-frequency images together.
The image at different scales:
The algorithm performs quite fast and does exactly as expected. From far away it looks like a dog while in close up it looks more like a cat.
Here are some extra hybrid images:
The order of two images in the algorithm matters a lot. It is often the case that only one order works well. The hybrid image usually looks better if the high-frequency image has fine and recognizable details.
Fish and submarine
Bird and plane
Einstein and Marilyn
Motorcycle and bicycle
Apple and orange
Fox and lion
