Christopher Polack

Project 1: Image Filtering and Hybrid Images

Our first project focuses on hybrid images. These images take advantage of the fact that our vision relies on different frequencies in order to make sense of an image from different distances (or images of different sizes). When we are further away from the image (i.e. when the image is smaller) we rely more on the lower frequencies of the image. From closer distances (i.e. when the image is larger) we use high frequency details in the image to enhance our understanding.

To the left are examples images of low and high frequency images extracted from an image of a tree. The low frequency image has most of the color data of the image, so when this is subtracted to construct the high frequency image much of the color is lost.

Although it appears that the low and high frequency images are calculated in separate ways, they are in fact closely related. The low frequency image is simply a convolution of the original image with a 2D Gaussian filter. The high frequency image is obtained by subtracting the low frequency convolution from the original image. The high frequency image is shifted to account for negative values that result from the subtraction.

There is one parameter available to customize the formation of the hybrid images. A so-called "cutoff frequency value" controls the crossover point at which the two images are joined. A lower value preserves more high frequencies of the image and only uses the very highest frequencies of the second image. A large value blurs the first image more (uses lower frequencies) and uses more than just the very highest frequencies of the second image.

Hybrid Images

A hybrid image is constructed by taking two different, but similar, images and overlaying the low frequencies of one with the high frequencies of another. When the image is viewed from a close distance, the image from which high frequencies were extracted dominates the interpretation of the image. When the image is small, the low frequencies dominate the image and the second image is understood.

Below are examples of hybrid images from the provided demo files.

Important Notes

There are a few things to note from the above demo images. It is clear that is some of these situations, the hybrid image seems more realistic in one direction than in the other. The bird to airplane picture seems more unnatural than the airplane to bird picture, for example. This could be due to the fact that some objects are simply more understandable based on either their low frequency or high frequency characteristics. The shape of an airplane is more distinct than its texture, while the texture of a bird is more distinct than its shape.

More Examples

Below are some examples I created to demonstrate the concept of hybrid images.

Failure Examples

Hybrid images fall apart in many cases. Most of the failures occur when the characteristics of the images are not taken into account.

Consider the images of the brick wall and the crowd of people. The above hybrid image works because the brick wall pattern resides in low frequencies, while the details of a crowd of people are encoded in high frequencies. When these two images are swapped, the resulting image does not make sense. There are no distinguishing low frequency qualities of a crowd of people, therefore both images look like a brick wall.

When the images are too distinct from one another, the hybrid image can also fail. In the following example, I tried to make hybrid images using a watermelon and a tennis ball. However, the distinct markings on each (and the distinct color of each) made the hybrid image very ineffective.