CS7321: Computer Vision - Problem Set 3
Solution By: Ying Chen

How I solved it

Basically I followed the method described by the papers. Because color images are to be used, so, before anything, I read in the images, divide them into their R,G,B components, and merge the images at their component. To merge the images, first build the Laplacian pyramids for the images to be merged, then, build the Gaussian pyramin of the mask. Then, combining the images together at each individual level, and later reconstruct the resulting Laplacian image to an actual image.

What I did first was to create the kernel , given an a as the initial prarmeter.

Then, I created the reduce.m , following the equation in the book. One has to first padd the image with 4 extra pixels for the border, then at each odd pixel (starting at the original 0,0, to N,N), take the weighted sum of its neighbors with the kernel. It works similar to the matlab conv2 function, except that only the odd pixel values are stored. Thus, I first did a conv2 of the padded image with the kernel, then sample it at the odd ixels of the original image.

Then, I create the expand.m, also by the paper. Although, I think the equation is not quite right in the spline paper (so I used the one from the compression p aper). Expand is a little more difficult to implement than reduce. The reason is that we have to take into account when the value (i-m)/2 or (j-n)/2 becomes integer, and only use those values. Instead of going throught the two for loops (for each i,j of the new image) I also use convolution to help speed things up a little. Basicly, I noted that there is 4 possible ways of sampling the neighboring pixels depending on if i or j is odd or even. That is, for i/j = odd/odd, the pixels sampled will be the 4 corners around the center pxel, for i/j = even/even, the pixels sampled will be all of the 9 pixels in the 3x3 area center around the pixel(i/2,j/2). So, I generated the sub kernels for each case, and simply place the resulting convolution values at the appropriate place of the image. Before doing the convolution, I also padded the image so that the result of the convolution will be correct.

Then I start building pyramid, either Gaussian or Laplacian (because you still have to create the Gaussian when you want the Laplacian). It was not very difficult in terms of actually how to create it. The important thing is to calculate the actual levels of the pyramid. (This is where the image size will be picked on) The only difficult part is to figure out a way to use matlab to store those values. I decided that I will use a 3D array, row_of_image * col_of_image * level - to store these values. (I know that the pyramid actually can be stored with less than 1/2 of one image size)

After building the pyramid, I wrote a function call reconstr - that collaps a laplacian pyramid and return the resulting image.


My Assumptions
  1. Image input is rgb
  2. Both images are the same size.
  3. The image is of size MR*2^N + 1 by MC*2^N +1.
  4. Matlab has enough storage space to store the pyramids and process it.
Improvements

I can think of things that can speed things up and not reducing this o2 machine to a crawl..

I probably want to use a better space/structure organization in matlab. (I am allocation too much for data that only has one pixel..). Right now - for each image - that can be reduced into N levels, amount of memory per pyramid = row * col * N

Results: some images

The image that brought the o2's to a crawl
Left Image Merged Image Right Image
Similar image - but different texture...
Left Image Merged Image Right Image
Similar image - but different in hue...
see how color just sort of blended into it?
Left Image Merged Image Right Image

And the very last one with a variable mask

Matlab codes
Other Things..