Project 2: Local Feature Matching

For this project, I implemented the three major steps of a local feature matching algorithm. Here is a broad overview of the three steps:

  1. Detect interest points:

    I used the Harris detector to detect interesting corner/edge points in both the images. I used the original Harris-Stephens(1) paper as a guide. It works like this:

    Alternative text - include a link to the PDF!

     I tried using R = Det(M)/Tr(M) instead of the R function defined above but in the end the above R function gave better performance. I tried different values of sigma for the gaussian and k for the R function too. There are other detectors too. The Shi-Tomasi detector is pretty close to the workings of Harris detectors but might give a better performance.

  2. Extract features:

    The next step is to extract features out of these interest points. I used a SIFT like detector that gives out 128 dimensional features. Here is a broad overview of how it works:

    Alternative text - include a link to the PDF!

    3. There are many parameters to play around with here. Some of them are feature width, shape of the extracted window, sigma of gaussian etc. I also raised each bin in the histogram to a power less than 1 (0.6 is default) to get better results.

  3. Match features:

    I used the nearest neighbor distance ratio formula from our text book (4.18) to find the best matches.

Results

Go to this result page to view the the result of the above program on different sets of images.

Go to this Directory to view all the images organized into folders by name.

I got the following results on the provided images:
Notre dame: 87 total good matches, 13 total bad matches. 0.87% accuracy
Mt. Rushmore: 40 total good matches, 38 total bad matches. 0.51% accuracy
Episcopal Gaudi: 1 total good matches, 30 total bad matches. 0.03% accuracy
As you can see, my basic detector can handle slight translational changes well (as seen in Notre Dame, and to some extent in Mt. Rushmore results). However, it performs abysmally if the images are at a different scale and/or angle.
Case by case results are discussed on the result page.

References

1. Harris and Stephen, A combined corner and edge detector, The Plessey Company, http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.434.4816&rep=rep1&type=pdf
2. Eq. 4.18, Computer Vision: Algorithms and Applications - Richard Szeliski, http://szeliski.org/Book/