For this project, I worked to implement a basic utility for finding SIFT-like features centered around Harris corners in an image and match these features with features extracted from another image of the same scene.
The first part of this project finds Harris corners in an input image in get_interest_points. The function begins by blurring the image with a Gaussian filter, followed by finding the vertical and horizontal gradient responses over the entire image. These gradient response images are then blurred again, and the pairwise products of the gradient intensities are calculated for each pixel and used to calculate the cornerness of that point.
With cornerness calculated for each pixel, all that remains is to select proper interest points. First, the function runs a sliding window across the image and, if the center of the sliding window is the local maximum in that window, then that center pixel is designated a local maximum in a binary image. A second binary image is created by finding the pixels with cornerness above a certain threshold, and the union of the two images gives the interest points for the image.
The second part of this project generates SIFT-like features for each point of interest found in the first part. The function first finds generates a four-dimensional matrix relating each pixel within the bounds of a feature box (in the case of my implementation, 24x24 pixels) to each spatial bin in that feature, with each pixel contributing more to the bin in which they land when close to the center of the bin than when on the side or in the corner. The function then finds the magnitude and direction of the gradient at each pixel in the image using imgradient.
Now, the function visits each interest point in the image and sets up a feature_width-size box around it. For each pixel in that feature box, the function finds the direction of the image gradient at that pixel and finds the two directional bins (there being 8 of them in each spatial bin, from 0 to 315 at 45-degree intervals) the gradient "lies between." The function then assigns the coefficients to each of those bins, inversely proportional to the "distance" of the gradient direction to each bin's center, and multiplies the gradient magnitude at that point by both the directional bin coefficients and the spatial bin coefficients corresponding to its location within the feature block and adds it to its appropriate bin entries on the line of the feature vector matrix corresponding to the correct interest point.
Finally, the last part of this project matches sets of features from two different images in match_features. First, the function selects the shorter set of features and uses it to set the output number of matches. Then, for each feature in the shorter list, the function finds the Euclidean distance from that feature to all other features in the longer list and sorts the longer list by these differences. This makes the matching feature's nearest match and next-nearest match easily accessible. Using these distances, the function calculates the nearest neighbor distance ratio for the proposed match between the matching feature and its nearest match in the longer list and logs it along with the proposed match. Once all features in the shorter list are matched to features in the longer list, the list of feature matches is sorted by NNDR and the matches and their NNDRs are returned. This list of features, being sorted by confidence, can then be easily pruned down to only consider the most confident matches.