- The Gaussian and Laplacian Pyramids for the first fish image:
- The Gaussian and Laplacian Pyramids for the second fish image:
- The Matlab routines that created the above images
- The splined images as they were expanded and convolved
- The Matlab code that expanded the splined images
To perform a multiresolution spline the Gaussian and Laplacian pyramids for both of the images to be splined must be built.
The basis for a multitresolution spline as described by Burt and Adelson is the Gaussian and Laplacian pyramid. To spline a color image this involves the creating both Gaussian and Laplacian pyramids for each of the Red, Green and Blue color maps for each of the images.
Each level of the Gaussian pyramid is forming from an image by convolving the image with a 5x5 separable mask built from the 5-element row of the Pascal triangle and then reducing the result by removing the even numbered rows and columns in the image.
This process is simplified by making the image square with 2N+1 rows and columns. In the examples that follow all images are 257x257 pixels. This results in the formation of Gaussian levels G0 which is the original image and 257x257 pixels in size through G8 which is 1 pixel in size.
Once the Gaussian pyramid is formed each of the levels G8 through G1 is first expanded by adding rows and columns of zeroes between each existing row and column and then convolved with the same 5x5 separable mask. The resulting filtered expansion is then subtracted from the next lower Gaussian pyramid level. For example: Level G7 which is 3x3 is expanded to 5x5, convolved with the 5x5 mask and then subtracted, element by element, from level G6. The result of this subtraction is the Laplacian pyramid level L6.
Once all the levels of the Laplacian pyramid is formed the different levels can be reassembled to form the original image through a process of expanding and filtering. Starting with level L8 (in my example) each level is first expanded, convolved with the 5x5 mask and then added to the level below.
For example: Level L7 which is a 3x3 matrix is first expanded to a 5x5 matrix by adding rows and columns of zeroes. It is then filtered and added element by element to level L6.
Problems encountered:
As can be seen from the Laplacian pyramid level L0 there is some deteriation of the image. I was unable to
determine the source for these drop outs but my best guess is that:
1)Default conversion from double to uint8 (or vice versa) is causing undiscovered problems or
2)There is some kind of rounding error at work here.
In any case these dropouts either did or did not occur in the final image with a mysterious randomness
that kept me from divining the underlying cause.
Back to the TOP
The Gaussian and Laplacian Pyramids for the first fish image:
The original full color image:
|
The Red color map: 
|
The Green color map: 
|
As an example, this is the Gaussian pyramid for the Blue color map:
...And this is the Laplacian pyramid for the Blue color map:
Back to the TOP
The Gaussian and Laplacian Pyramids for the second fish image:
The original full color image:
|
The Red color map: 
|
The Green color map: 
|
As an example, this is the Gaussian pyramid for the Blue color map:
...And this is the Laplacian pyramid for the Blue color map:
Back to the TOP
The Matlab routines that created the above images
This is the main Matlab routine
This code separates the color image into three matrices
This code creates the Gaussian and Laplacian pyramids
The splicing process.
Once the Laplacian pyramids have been built for each of the two images they must be spliced together into one image for each level of the Laplacian before the expansion and convolution that will bring them back together into a final splined image.
To splice the image, a "receiver" image of zeroes that is the same size as the Laplacian at that level is built. Then for each of the Laplacian levels all pixels to the left of the center column of pixels for one image is placed in the receiver image on the left, all pixels to the right of the center column of pixels for second image is placed in the receiver imageon the right and the center column of pixels is calculated by averaging the center column of pixels from each of the two images.
This process is demonstrated by the images seen below:
Back to the TOP
The spliced laplacian images
|
The Red color map: 
|
The Green color map: 
|
The Blue color map: 
|
The Matlab code that produced the spliced images
This code creates the spliced Laplacian levels
The rebuild process.
Once the Laplacian levels for all three color maps for the spliced images are produced they are then combined to produce one output image for each color map. This combination process starts with the highest Laplacian level (in this case L8). First the Laplacian level is expanded by inserting rows and columns of zeroes between each existing row and column of pixels, a process that expands that Laplacian level to the size of the level below it. For example: Laplacian level L8, a 1x1 matrix is expanded to 3x3. Next that expanded level is convolved with the same 5x5 mask used in previous steps and then added, pixel by pixel to the Laplacian level below it. In our example, the expanded L8 is convolved and then added to Laplacian level L7 to produce a 3x3 output image at level L7. This output image is then expanded from a 3x3 to a 5x5, convolved and added to Laplacian level L6 to create a 9x9 output image at level L6. This process continues until level L0 is added to the output from level L1 to produce the final image for that particular color map.
Clearly there is a problem here. Not only are the drop outs seen before occuring
but also the convolution process
is not having the desired effect. In fact if there is any effect at all it is not evident in the output
of these routines.
Back to the TOP
The splined images as they were expanded and convolved
|
The Red color map:
|
The Green color map:
|
...And this is the Laplacian pyramid for the Blue color map:
The Matlab code that expanded the splined images
This is the main Matlab routine that saves the output of the splice operation
as files
and then controls the rebuilding of all three color maps
This code that actually does the rebuilding
Assembling the Laplacians for each color map into one color image.
Once the output image from Laplacian level L0 is produced for each of the color maps all that remains to be done is to display them as an RGB color image.
The final splined color image
This code that actually does the rebuilding
Conclusions and lessons learned.
I am not certain what caused the various difficulties that I ran across in this project however I have my suspicions.
Matlab is powerful in that it does so much for you that you can get quite a bit done without understanding exactly what it is that is going on. What I mean by this is that if you don't specify something explicitly then Matlab will handle it for you through some kind of default. This is a good thing as long as the thing it defaults to isn't the wrong thing. More than several times I had to back up and start a different approach once the underlying assumption was better understood. The double for calculations and uint8 for show tripped me up until this past week.
Another problem that occurred more than once was the need to do the old "if the output images don't look right, run it again just to make sure it isn't confused" routine. The dropouts seen in my images were a problem for most of the past two weeks but I finally got the software to do what it should when rebuilding the Laplacian pyramids on the test intensity image that I used. They suddenly reappeared once I started working with the color images for the project. I then was unable to fix the problem an simply had to move forward, dropouts and all, otherwise this project might run into the next Quarter.
Finally it seems to me that there are some underlying bugs (undocumented features) in Matlab that can lead to a great deal of frustration. It is possible that the Macintosh version of the software isn't as stable as other versions. This is a problem that I have encountered numerous times with other software packages adapted for the Mac.
Jim Rowan is a Ph.D. student in Georgia Tech"s College of Computing
To contact Jim Rowan by email: jrowan@cc.gatech.edu