CS 143 / Project 1 / Image Filtering and Hybrid Images
Scaling of a hybrid cat/dog image.
Image filtering allows us to perform mathematic operations on images to modify their appearances in certain ways. Some filters blur images, some filters sharpen them, some just flat out remove features of an image to emphasize others within it. The hybrid image process in this project filters two source images such that one contains high frequencies and one contains low frequencies, and then the two are merged together. The theory behind it is that an image filter can be treated the same as adding a bunch of image filters together. In that sense, our hybrid imaging process is effectively just a filter made up of a bunch of smaller filters to generate the image from two source images.
How We Filter Images
Essentially what we do implementation-wise is treat the image and filter as two matrices. We pad the image an appropriate amount depending on the size of the filter. There are multiple possiblities in terms of padding (mirror the image, extend the image, perhaps even speculate the image, lots of options here), but for the purpose of the project we just chose to leave the padded values as 0. We then slide the filter through each part of the image, to make a new image (that has the same dimensions of the original image) as the filtered output by recording the dot product at each part of the padded image we place the filter over. In the context of the hybrid image problem, this works for creating a low frequency filter by just replacing the color at a given location with something similar to what the other colors are like. On the other hand, if this were negated (for the high frequency filter), it would essentially tone down a lot of the surrounding color and in its place sharpen the distinct edges. Both these two possibilities come into play for making the hybrid images. Long story short, matrix magic is the backbone of the image filtering process.Making Hybrid Images
For different images, different frequency cut-offs were necessary with the blurring. The frequency cut-off is used in the code to determine how much the images are blurred and sharpened for the low frequency images and high frequency images respectively to create the hybrid image. Essentially one has to strike a balance between not blurring too much that the colors don't mesh together well from a distance, as well as not sharpening too much that the blurred image isn't noticeable when viewing the hybrid image from a distance. The basic idea is to try to get an orientation and frequency cut-off that allows for the colors of one image to sort of color in the other. Here is an example containing the low frequency filtered image and the high frequency filtered image that are merged into a hybrid image with a basic matrix addition operation (goes back to the idea that a big filter can be represented as the sum of a bunch of smaller filters). Note how the low frequency dog image essentially provides the color while the high frequency cat image essentially provides the sharp features.
|
You will find a table below of hybrid images along with the source images that were filtered together to generate them. As you can see, some turned out more natural than others and a lot of how natural it looks has to do with a combination of how the images are blurred/sharpened and meshed together as well as the relative orientations of the images. For example, note that while the filters are done effectively well for the bicycle and motorcycle, no matter how good it is, it is going to be somewhat awkward due to the bicycle taking up significantly more space in the top portion of the image while the motorcycle doesn't (i.e. they don't overlap as well as some of the other images). For a given row, the hybrid images are the first two and the source images are the last two.
Hybrid Image Combinations with their Original Source Images
|
|
|
|
|