Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC

Part 1: Camera Projection Matrix

After coding part 1, the following projection matrix was found with a residual of 0.0445:


    0.7679   -0.4938   -0.0234    0.0067
   -0.0852   -0.0915   -0.9065   -0.0878
    0.1827    0.2988   -0.0742    1.0000

Scaling the matrix by a factor of -1.6756 gives the following matrix which matches the provided matrix in all but a few entries, with those only being off by 0.001, most likely due to numerical error from the scaling factor:


   -0.4583    0.2947    0.0140   -0.0040
    0.0509    0.0546    0.5410    0.0524
   -0.1090   -0.1783    0.0443   -0.5968

Using this, the camera center was found to be <-1.5126, -2.3517, 0.2827>, again within numerical error of the provided center. With the normalized points used as a base out of the way, I tested the harder points from picture b. This provided a camera center of <303.0967, 307.1842, 30.4223> and the following projection matrix:


   -2.0466    1.1874    0.3889  243.7330
   -0.4569   -0.3020    2.1472  165.9325
   -0.0022   -0.0011    0.0006    1.0000

Using that projection matrix and camera center, the following point visualization was obtained, revealing pretty good correspondences:

Part 2: Fundamental Matrix Estimation

For my estimation of the fundamental matrix, I used singular value decomposition to solve the linear system for the matrix. I also set the lowest eigen value of the solved matrix to 0 in order to estimate the rank 2 matrix. This ended up slightly lowering the accuracy for the baseline, but ended up giving much better results when error is introduced.

The baseline images with epipolar lines shown. The results are pretty good, but the rank reduction estimate does cause the lines to move slightly off center. The fundamental matrix estimated for these image pairs is:


   -0.0000    0.0000   -0.0019
    0.0000    0.0000    0.0172
   -0.0009   -0.0264    0.9995

Without rank 2 estimation (left) the epipolar lines are inconsistent and some cross over when error is introduced, but a few still line up correctly. With rank 2 estimation (right), the epipolar lines are more consistent, but the they do not match the points quite as well

Part 3: Fundamental Matrix with RANSAC

My implementation of RANSAC uses a maximum of 2000 iterations while attempting to reach an inlier to outlier ratio of 99%. To accomplish this, it estimates the fundamental matrix with 8 randomly sampled points from the correspondences that vlfeat found. It then checks the error of the estimated matrix and counts any point with and error of less than 0.01 as an inlier. It then sorts the found inliers by this error and returns the top 30. This gave pretty good results, even for the Gaudi pair. Below are a few samples from my results: