Project 2: Local Feature Matching

Notre Dame Matching

For the project we had to implements 3 different parts of the Feature Matching Pipeline.

  1. find_interest_points
  2. get_features
  3. match_features

find_interest_points

For this part of the project I implemented Harris Corner Detection in order to find points of interest. After finding the Harris corner score I then performed a gray-scale dilation algorithm in order to do my non-maxima suppression. Then I removed values below some threshold. Lastly I removed points close to the edge of the image and returned the remaining points. My interest point detecter usually finds 700-1000 points per image

Notre Dame Harris corners

get_features

For this part of the project we were tasked with creating a feature vector from our interest points. I did a simplified version of sift. I took the gradient of the image and then for a window around the interest points I created a histogram based on the magnitude and direction of the gradient in a 4x4 cell. I did this for each 4x4 cell in the surrounding window. The code for that is shown below. I initially tried to add a scale invariance by normalizing my histogram along the dominant orientation; however, this proved to be ineffective. After the feature vector is created from the histograms I raise it to a small power and then normalize the vector. Sift features brought me from 43% accuracy to 80%


    for j = 1:feature_width/4
        for k=1:feature_width/4
            cWindowDir = feature_window_dir(((j-1)*4) + 1:4*j, ((k-1)*4)+1:4*k);
            cWindowMag = feature_window_mag(4*(j-1)+1:4*j, 4*(k-1)+1:4*k);
            h = zeros(8,1);
            dirFlat = reshape(cWindowDir, [], 1);
            %disp(size(dirFlat));
            magFlat = reshape(cWindowMag, [], 1);
            for v = 1:size(dirFlat, 1)
                [cY, cX] = ind2sub([4 4], v);
                cY = ((j-1)*4)+cY;
                cX = ((k-1)*4)+cX;
                degrees = wrapTo360(atand(dirFlat(v)));
                d_idx = ceil(degrees/45) + 1;
                if d_idx == 9
                    d_idx = 1;
                end
                h(d_idx) = h(d_idx) + (magFlat(v)*g(cY, cX));
            end
            fVec = [fVec; h];
        end

match_feature

This was perhaps the easiest part. For this part I took the euclidean distance for every combination of vectors and found the 2 closest for any given feature. I added the index of the nearest neighbor and the feature I was looking at to the list and then added the confidence value of nn2/nn1. Lastly the values were sorted based on the confidence. This means that the higher confidence values are at the beginning of the list

Results

The detector worked really well for Mount Rushmore and Notre Dame with accuracies of 74% and 80% respectively; however, due to the fact that I removed the rotation correcting code Episcopal Gaudi did not perform well having an accuracy of 0%. This is likely because of the rotation as well as the massive difference in scale and lighting means that the detector was overall just too far off. Interest points were still detected in reasonable places.