Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC

The aim of this project was to estimate the camera projection matrix, which maps 3D world coordinates to image coordinates, as well as the fundamental matrix, which relates points in one scene to epipolar lines in another. The three modules this project involved are

  1. Estimating Projection Matrix
  2. Estimating Fundamental Matrix
  3. Implementing RANSAC

I also implemented the extra credit part, which involved implementing normalization before running the 8 point algorithm.

Part 1 - Projection Matrix

The goal is to compute the projection matrix that goes from world 3D coordinates to 2D image coordinates. I used SVD here to solve for the parameters. The matrix I got at the end was :


    0.4583   -0.2947   -0.0140    0.0040
   -0.0509   -0.0546   -0.5411   -0.0524
    0.1090    0.1783   -0.0443    0.5968

Also, the residual I got was <0.0445> and the estimated location of camera is: <-1.5127, -2.3517, 0.2826> All these match reference solutions provided on project page. The images generated by the code are provided below.

Part 2 - Fundamental Matrix

The next part of this project was estimating the mapping of points in one image to lines in another by means of the fundamental matrix. I have tried to do this in two ways, once without normalizing original point values, and another by normalizing it (graduate credit). I used the 8 point algorithm to do this.

Fundamental Matrix Estimation without Normalization

Results for estimation without normalization are

As you can see, the results are pretty good, and the epipolar lines pass through given points.

Fundamental Matrix Estimation with Normalization (EXTRA CREDIT)

To do this, I centered the points for each image at mean, applied constraints by using standard deviation. Results for estimation without normalization are

As you can see, the results are pretty good here too, and the epipolar lines pass through given points. These results for these particular example images don't exhibit major changes from the results for the fundamental matrix without normalization.

Part 3 -Fundamental Matrix with RANSAC

I tried using RANSAC to estimate fundamental matrix. I used 2000 iterations, and after various trials, set the threshold to 0.0005 .Results for estimation without normalization are

MT RUSHMORE

As you can see,selected inliers lie on epipolar lines. The matches obtained here are also really good, as seen below .

NOTRA DAM

The matches obtained here are also really good, as seen below. However as compared to the Mt. Rushmore example, we see few(1 or 2) mismatches in the arrow image .

EPISCOPAL GAUDI

The matches obtained here are also really good, as seen below. We again see few(1 or 2) mismatches in the arrow image .

Part 3 -Fundamental Matrix (NORMALIZED) with RANSAC - GRADUATE CREDIT

I tried using the normalized eight point algorithm for creating fundamental matrixRANSAC to estimate fundamental matrix. The results are

MT RUSHMORE

We get nice results for this.

NOTRA DAM

The matches obtained here are also really good, as seen below. Also, it can be clearly seen that the matches with Notre Dame image here are really good between images, and definately better than the method used earlier, where we did not use the normalized 8 point algorithm

EPISCOPAL GAUDI

The matches obtained here are also really good, as seen below. Also, quality of matches is better than the earlier method(which did not have normalization).

Conclusion

For the examples we used, it could be clearly seen that using a normalized 8 point algorithm gave better results than the basic 8 point algorithm while counting inliers.