Pulkit Sachdeva

Project 2: Local Feature Matching

The aim of this project was to create a local feature matching algorithm using techniques like Harris corner detector, SIFT, etc. We basically try to match interest points in two different images and try to find and match features in both images.

I implemented three functions to achieve this. These functions basically were

1. get_interest_points : Discover interest points in images
2. get_features : Construct feature vectors for each interest point
3. match_features : Match features between images

Discovering Interest Points

For get_interest_points, I basically implemented Harris's corner detector. I did this by generating a cornerness score matrix. I then found points whose surrounding window gives large corner response by using a threshold. I then found local maximas using non maximum supression.

Get Feature Vectors

The next step is to generate feature vectors for each interest point. I initially started with just using plain image patches and converting them to vectors, but in order to improve accuracy, I ended up implementing the SIFT pipeline. I basically did this by splitting area around each interest point into cells, and loooking at orientation of the gradient in each cell and using that to form vectors.

Matching Features between images

I implemented the nearest neighbor distance ratio test to match features between two images. This helps define how close a match is. I basically look at the minimum distance between two features in two images, and the second minimum distance. The ratio between these metrics gives confidence, and we sort this to get top 100 matches.

Results For Notre Dame Image

Comparison between images - Notre Dame

Accuracy for Notre Dame images : 91%

Results For Mount Rushmore Image

Comparison between images - Mount Rushmore

Accuracy for Mount Rushmore images : 34%

Results For Episcopal Gaudi Image

Comparison between images - Episcopal Gaudi

Accuracy for Episcopal Gaudi images : 6%

Findings