RANSAC based matches on two images of the Episcopal Gaudi in Spain
The goal of this project was to introduce us students to the concepts of camera and scene geometry. This was done in three steps:
Additionally, I implemented coordinate normalization to improve the estimates for the Fundamental Matrix for extra credit.
The goal of this section was to compute the projection matrix that goes from 3D coordinates to 2D image coordinates. This can be done using the following equation:
and a slight modification of changing m34 to 1 to adjust the scale for the matrix. Once the Camera Projection Matrix was calculated, calculating the center of the camera was as simple as carrying out the following equation . After implementing both of those equations in the appropriate files, I was able to come up with the results shown below for the two provided point sets.
1st Point Set: 2D Projection Matrix Evaluation | 2nd Point Set: 2D Projection Matrix Evaluation |
1st Point Set: 3D Projection Matrix Evaluation & Camera Center | 2nd Point Set: 3D Projection Matrix Evaluation & Camera Center |
The goal of this section was to estimate the mapping of points in one image to epipolar lines in another image using the Fundamental Matrix. This was done by using SVD decomposition to solve the following equation:
Similarly to the first part of the project, I had to set m33 as 1 to adjust the scale correctly. Due to this, I only needed to find 8 matching points to create the fundamental matrix. My results are shown below:
Image 1: Epipolar lines and matches | Image 2: Epipolar lines and matches |
The final portion of this project involved using RANSAC in conjunction with the Fundamental Matrix Estimation to correctly match points in images based on unreliable point correspondences computed with SIFT. By using RANSAC to get the best estimation, and filtering those matches down to the 30 best ones, I was able to produce the results below:
Mount Rushmore Image 1: Epipolar lines and 30 best matches | Mount Rushmore Image 2: Epipolar lines and 30 best matches |
Mount Rushmore RANSAC 30 best matches |
It is quite clear, that using RANSAC, we can do a extremely good job at matching points across the 2 images. Based on a quick visual scan of the matched points, they all seem to be correct and valid matches for the Mount Rushmore image. For the sake of being thorough, I also ran it without limiting the number of matches to produce the following results:
Mount Rushmore Image 1: Epipolar lines and all the matches | Mount Rushmore Image 2: Epipolar lines and all the matches |
Mount Rushmore RANSAC with all the matches |
It is much harder to tell if all the points are matched correctly due to the density of the lines, but they seem to be quite accurate.
In order to improve the accuracy of RANSAC on the Episcopal Gaudi image, I wrote additional code to normalize the Fundamental Matrix for extra credit. This was done using the following equation:
Normalizing the Fundamental Matrix showed quite a difference in the resulting matches for the Episcopal Gaudi image pair. Without the normalized Fundamental Matrix, the results were as follows:
Episcopal Gaudi Image 1: Epipolar lines (non-normalized) and 30 best matches | Episcopal Gaudi Image 2: Epipolar lines (non-normalized) and 30 best matches |
Episcopal Gaudi RANSAC (no normalization) 30 best matches |
Unlike the Mount Rushmore example, running RANSAC alone on the Gaudi image pair was entirely accurate. You can see multiple incorrect matches in the images, my favorite one being the match between a cloud and a window. After normalizing the Fundamental Matrix, I was able to produce the results shown below:
Episcopal Gaudi Image 1: Epipolar lines (normalized) and 30 best matches | Episcopal Gaudi Image 2: Epipolar lines (normalized) and 30 best matches |
Episcopal Gaudi RANSAC (normalized) 30 best matches |
As you can see, running RANSAC when using a normalized Fundamental Matrix drastically improves the results and matching on the Episcopal Gaudi pair of images.