In this assignment you will experiment with the Heeger-Bergen texture synthesis algorithm. You will write the histogram matching function which makes up the inner loop of the algorithm and test it on a variety of textures.
Be sure to do the following when you turn in the assignment:
Grading: This is the graduate level version. I expect most students in 7495 will do this one. You will be asked to do an additional experiment and answer some more detailed questions about the performance of the method. The answers from the students in 7495 will determine the mean for this assignment track. Any students from 4495 who do this version of the assignment will be graded on a more forgiving curve.
Software Issues: Download the latest version of the Viper2 library. We have created two new projects, TextureSynthesis and TextureCode. This problem set is based on a public domain implementation of the Heeger-Bergen algorithm. A report describing this implementation is available. One drawback of this code (for our purposes :-) is that it is based on yet another standard image processing library know as TIPS. In order to circumvent this problem without rewriting all of the code, we have created code which converts between TIPS format and the VisionSDK format which you are used to. This code has already been placed in the appropriate places in the project, so you should not need to understand the TIPS format in detail or write any conversion code yourself to do this problem set. If you think you have to do format conversion, please see Jianxin or me to make sure there is no confusion.
In their seminal 1995 SIGGRAPH paper on texture synthesis, Heeger and Bergen proposed a remarkably simple algorithm based on matching image statistics at multiple spatial scales. As was discussed in class, the algorithm has two conceptual parts: (1) an oriented pyramid decomposes an input texture into filter responses at multiple scales. These filter responses encode the structure of the texture pattern over different spatial neighborhood sizes in the image. (2) a histogram matching step iteratively modifies the statistics of a noise image until it conforms to the statistics of the input texture. This matching is done over histograms of filter responses, and then the pyramid representation is inverted to reconstruct an image. At that point the noise image has been converted to a new instance of the input texture. All aspects of the Heeger-Bergen algorithm have been implemented, with the exception of the histogram matching step which you are asked to write below. Note that you should read the Heeger-Bergen paper in order to answer the questions below.
1. Write the function MatchHistogram, which does histogram computation, CDF computation, etc. and implements the Heeger-Bergen inner loop. Pseudocode for this function along with a description of the necessary subroutines can be found in this report. A stub for this function already exists in the file ttexture.h within the TextureCode project in Viper2. The code is already set up with right sequence of function calls, which is DoSynthesis() calls IMatchTexture() calls IMatchHistogram() which calls your function MatchHistogram(). DoSynthesis is the entry point in the TextureSynthesis project. IMatchTexture() implements the Heeger-Bergen main loop. All image conversion is taken care of along the way. Your code simply needs to work with buffers of double values which represent image levels in the pyramid decomposition. You will find several sample image textures in the Data subdirectory of the TextureSynthesis project. Test your code on the marble.bmp texture. A correct output texture, marble-output.bmp, (for the default interface settings) has been provided so that you can check your code. Since it is a randomized algorithm, the outputs will match only qualitatively, not on a pixel-by-pixel basis.
2. Run the algorithm on the following groups of input textures. For each group, characterize the performance of the algorithm. Which aspects of the texture did it get correct? Which aspects did it fail to represent? Can you explain the failures in terms of the representation of texture which is used by the algorithm?
a) marble.bmp, grass.bmp, fiberglass.bmp
b) fur.bmp, cantaloupe.bmp
c) panda.bmp
3. How would you characterize the ability of the method to handle textures with significant large-scale spatial structures (i.e. textures whose elements span 10's or 100's of pixels in the input image)? How about the ability to handle structures at different rotations? Explain how the number of pyramid levels and orientations affect the performance. What prevents us from using a very large number of pyramid levels and orientations? (Note that the pyramid code you are using here cannot implement more than 4 orientations, so this is a general question).
4. It is interesting to consider combining the statistics of multiple textures together in doing synthesis (i.e. marbelized fiberglass). The easiest way to do this is to simply combine two image textures together in a single input image (by appending their rows together for example). This will have the affect of adding the histograms for the two textures together. Combine the marble and fiberglass textures in this manner and synthesize an image from their combined statistics. Describe the result. Can you suggest any way to improve it?
Instructions for submitting: Make a zip file containing your source code, your executable, and all of the output textures you created in response to the questions above. Your executable should compute and display the marble texture case (the default setting of the dialog box). Submit a hardcopy of your answers to the questions above to Jianxin. A folder will be created to receive your hardcopy as usual.