CS 7321 Winter 1998

PS#3 Solutions by Rob Orr

First I read in the images and created the Gaussian pyramids. Creating the Gaussian pyramids requires:

create a temporary images that is padded by two extra rows of pixels all around
center the original image in teh expanded template
fill extra rows in by using formula on the paper (g(-1) = 2*g(0) - g(1), g(-2) = 2*g(0)-g(2))
convolve expanded image by 5x5 filter; this can be done seperably in the x and y directions by convolving in each direction with the 1x5 filter [ 0.05 0.25 0.40 0.25 0.05].
delete even numbered rows and columns and return the resulting image
This process is repeated until the resulting image is 1x1. It is also repeated for all 3 color channels and for both original images.

for each level of the pyramid, subtract the expanded Gaussian image from the next level "up" the pyramid from the current level Gaussian; the resulting image is the Laplacian
expand() simply interpolates between pixels by running the seperable 5x5 filter over the image after it has been expanded by adding zero-filled rows and columns between the existing pixels
Again, this process is repeated until the resulting image is 1x1; also repeated for all color channels and both original images.

To create the splined image, I simply glued the Laplacian images together at each level, and summed them.

At each level of the pyramid (and for each color channel), one half of each Laplacian image (one-half left hand, one-half right hand) and joined and a new images is created. At the seam, the pixels are averaged from each image.
Start at the highest level of the pyramid (1x1 image). This image is expanded and is added to the spliced Laplacian from the next lower level in the pyramid. Repeat until you reach the lowest level.
I combined all the color channels together in one image and scaled the image so that there were no out-of-bounds values.

I made the following assumptions

The original images would be of size ((2^N)+1) X ((2^N)+1)
The seam where the images should be joined would occur at exactly halfway across the original images
The seam should be vertical and should extend from the top of the images to the bottom
The textures of the images would not vary greatly and averaging only on the seam would be satisfactory

I think the major weakness of my solutions are:

It does not accomodate arbitrary size images
It does not allow for arbitrarily joining the images (at any place and along any curve)
It does not do averaging over more than one seam pixel; larger texture differences are not dealt with very well

I think that this can be improved by doing the following

allow arbitrary size images
allow joining in any place and along any curve (incorporate masking)
average over 1 or 2 pixels on either side of the seam, depending on amount of texture difference
try to detect the amount of texture difference between the images and adaptively average along seam
allow the splining of more than two images together (either iteratively or in one pass)

lefthand

Figure 1a: This is the first image. It's my left hand.

Figure 1b: This is my second image. It's my right hand.

Figure 2a: Viola! All fingers!

Figure 2b: And of course, all thumbs.

Figure 3a-j: The Gaussian pyramid for the left hand images.

Figure 4a-j: The thresholded Laplacian pyramid for the left hand images:

What Gaussian and Laplacian pyramids are, how to create and use them, and why they are useful
How to combine two images seamlessly
Images can be represented at various feature sizes and they can be spliced and recombined at these various feature size levels (the essence of this assignment)

spline.m : this script reads the original images, creates the Gaussian and Laplacian pyramids, and splines the two images using the Laplcian pyramids
reduce.m : this function implements the reduce() function, and is used in creating the Gaussian pyramid
expand.m : this function implements the expand() function, and is central to creating the Laplacian pyramid