Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC

Part I: Calculating Projection Matrix and Camera Center

A projection matrix converts 3D world coordinates into 2D image coordinates.

Using SVD to calculate the projection matrix, we get:


The projection matrix is:
    0.4583   -0.2947   -0.0140    0.0040
   -0.0509   -0.0546   -0.5411   -0.0524
    0.1090    0.1783   -0.0443    0.5968

The total residual is: <0.0445>

The projection matrix M can be used to calculate the camera center as follows:


The camera center that we get using this formula on the previous the set of coordinates is:


The estimated location of camera is: <-1.5127, -2.3517, 0.2826>


Part II: Estimating the Fundamental Matrix

The fundamental matrix maps points in image 1 to points in image 2.

Estimating the fundamental matrix without normalizing the coordinates and using SVD:

F_matrix =

   -0.0000    0.0000   -0.0019
    0.0000    0.0000    0.0172
   -0.0009   -0.0264    0.9995

The results of estimating the fundamental matrix work much better when the points and the fundamental matrix are normalized. This can be done by centering the points of both images using the mean and scaling the points using the standard deviation.



Here, cu and cv are the means of the x and y coordinates respectively, s is the inverse of the combined standard deviation of the x and y coordinates, and T is the product of the scale and offset matrices.

Estimating the fundamental matrix after normalizing the coordinates and using SVD:

F_matrix =

   -0.0000    0.0000   -0.0003
    0.0000   -0.0000    0.0022
   -0.0000   -0.0031    0.0712

As you can see, the epipolar lines fit through the points with more precision after normalizing the coordinates.

Part III: Finding the best Fundamental Matrix using RANSAC

RANSAC follows 3 simple steps: Sample, Solve and Score.
  1. I sampled 8 points at random with a threshold error of 0.005.
  2. The fundamental matrix that fit these 8 points was found using the steps in part II
  3. The number of inliers for this fundamental matrix were counted based on the threshold error.

These steps were repeated 1177 times, as these are the number of iterations required to acquire a probability of 0.99 of getting atleast 1 iteration with no outliers, assuming that the outlier ratio is 50%. The best fundamental matrix found from these iterations (the one with most inliers) is used to match both images.

Results

Mount Rushmore

Before Normalization (137 Inliers)
After Normalization (656 Inliers)
Epipolar Lines Before Normalization
Epipolar Lines After Normalization

Notre Dame

Before Normalization (142 Inliers)
After Normalization (588 Inliers)
Epipolar Lines Before Normalization
Epipolar Lines After Normalization

Episcopal Gaudi

Before Normalization (162 Inliers)
After Normalization (387 Inliers)
Epipolar Lines Before Normalization
Epipolar Lines After Normalization

Woodruff

Before Normalization (146 Inliers)
After Normalization (211 Inliers)
Epipolar Lines Before Normalization
Epipolar Lines After Normalization

Sacre Coeur

Before Normalization (70 Inliers)
After Normalization (86 Inliers)
Epipolar Lines Before Normalization
Epipolar Lines After Normalization