Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC

Overall, performance on project 3 greatly outweighed performance on project two. Matches were much more accurate. However, on both projects, Gaudi examples could be improved! However, performance on project three still outweighed the results determined without maximizing inliers.

Essentially, by generating a mapping between 2-D and 3-D world coordinates, we can generate correspondences between images using epipolar lines, RANSAC estimation of Fundamental Matrix, and therefore draw better congruencies between images.

Part 1

As suggested, projection matrix was derived from singular value decomposition on the linearized formulas provided in order to move from 3D world to 2D camera coordinates. Computing this relationship will allow us to draw more inferences about more accurate matches by giving us more knowledge on our world/space. Results were transformed back into matrix form.


%Our resultant projection matrix is roughly equal to -1.67 * provided
The projection matrix is:
    0.7679   -0.4938   -0.0234    0.0067
   -0.0852   -0.0915   -0.9065   -0.0878
    0.1827    0.2988   -0.0742    1.0000

Given the projection matrix, solving for the camera center was as easy as...


The estimated location of camera is: <-1.5126, -2.3517, 0.2827>

Compared to the projected camera center...

Part 2

Epipolar line generation examples

Using similar methodology as Part 1, but on a different set of equations, specifically using the non-transpose version of the definition of the Fundamental Matirx, we solve another singular value decomposition. Below is the result of Part 2, epipolar lines.


F_matrix =

   -0.0000    0.0000   -0.0019
    0.0000    0.0000    0.0172
   -0.0009   -0.0264    0.9995
 

Part 3

Using RANSAC to determine # of inliers per each Fundamental Matrix, and assuming Fundamental Matrix and Inliers are relatively correct, I took random samples of the inliers set which contained the most inliers to return as most accurate matches. Therefore, it was determined that the matrix which gave the most inliers to be the 'Best'!


ransam = randsample(size(inliers_a,1),30);
inliers_a = inliers_a(ransam,:);
inliers_b = inliers_b(ransam,:);

Fundamental Matrix estimation with RANSAC Results

We see in these two images above that epipolar lines pass through alike-points!

Compared to performance in project two, the selected matches for this method are much more accurate. In project two we had some stray lines and some accurate, while solving for inliers with RANSAC greatly improved performance.

Here we can see that the epipolar lines run through the equivalent descriptors in each image, indicating that our fundamental matrix is at least relatively accurate descriptions of the world/space.

And for Mount Rushmore:

As a threshold, I decided to limit the error (deviation from 0) in the equation to one percent for accurate matches. As for # of iterations, I used the expected number of iterations of RANSAC to find the "right" solution, using percent of innacurate input correspondences.

Very accurate sample of correspondence!

As expected, finding the best fundamental matrix using RANSAC on Gaudi resulted in a less than ideal product. There are some fair matches, others innacurate! This was expected because I did not normalize the coordinates before computing the fundamental matrix. I assume these innacurate matches are due to the fact that some of the fundamental matrix estimates were just plain wrong/innacurate, but that we still used the best combination with the most inliers, randomly sampling from this. This will lead to some accurate matches getting deleted, and some innacurate not getting pruned!

However, this performance greatly outweights performance from project two, as I had hoped and included in my writeup for project 2.

^^ = reference to results on Gaudi from Proj 2!