Project 3: Camera Calibration and Fundamental Matrix Estimation with RANSAC

Part I: Camera Projection Matrix


From the given equation above, camera projection matrix, M, can be found by the least squares regression with the condition of m34=1 as:


The camera center, C, can be obtained by C=-Q-1m4 where M=(Q|m4). The calualted center below is close enough to the given center point through the small enough residual as 0.0445.


Most of epipolar lines are passing the correspoinding points while some are close to.

Visualization


Visualization of the projected points and camera center.

Part II: Fundamental Matrix Estimation

Based on the eqauation, w'Fw=0 where w and w' represents the coordinates [x,y,1] of the image 1 and 2, respectively. Using the least squares regression which is similar way to Part I, the fundamental matrix, F, can be estimated as below after the rank reduction.


Result


Epipolar lines via the fundamental matrix.

Part III: Fundamental Matrix with RANSAC

RANSAC has following stpes:

  1. Selection - 8 points pair from the two images.
  2. Estimation of F - Using the selected point, estimate F.
  3. Counting the number of liniers - w'Fw should be close to zero if F is correct (Threshold 0.005).
  4. Update - After comparing the current F and the best F so far, update the result and go to the selection until the number of iterations.

Result


Matching result comparison on Mount Rushmore: SIFT(VL) vs SIFT(VL+RANSAC with F)


Epipolar lines on Mount Rushmore via the RANSAC with the fundamental matrix.


Matching result comparison on Notre Dame: SIFT(VL) vs SIFT(VL+RANSAC with F)


Epipolar lines on Notre Dame via the RANSAC with the fundamental matrix.


Matching result comparison on Episcopal Gaudi: SIFT(VL) vs SIFT(VL+RANSAC with F)


Epipolar lines on Episcopal Gaudi via the RANSAC with the fundamental matrix.

Extra Credit: Normalization

By normalizing the coordinates of the points on the images, we can get a better result from spurious points as well. The normalization can be done by the transform matrix, T, which is the proudct of the scale and offset matrices as below, where s is a scale, cu and cv are the mean coordinates.


After the normalization of the fundamental matrix in Part II, it changes to:


We have more accurate results from figures in Part II which all epipolar lines pass the corresponding points as shown below. Moreover, the results on Episcopal Gaudi after normalization become significantly better.


Epipolar lines via the normalized fundamental matrix.

More result


Matching result comparison on Mount Rushmore: before vs after normalization


Epipolar lines on Mount Rushmore via the RANSAC with the normalized fundamental matrix.


Matching result comparison on Notre Dame: before vs after normalization


Epipolar lines on Notre Dame via the RANSAC with the normalized fundamental matrix.


Matching result comparison on Episcopal Gaudi: before vs after normalization


Epipolar lines on Episcopal Gaudi via the RANSAC with the normalized fundamental matrix.