Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC

The goal of this project is to estimate the camera projection matrix, which maps 3D world coordinates to image coordinates, as well as the fundamental matrix, which relates points in one scene to epipolar lines in another. It is consisted of three parts.

  1. Part I: Camera Projection Matrix.
  2. Part II: Fundamental Matrix Estimation.
  3. Part III: Fundamental Matrix with RANSAC.

Part I: Camera Projection Matrix

To get projection matrix, we use n 2d image coordinates and n 3d image coordinates, and then compose homogeneous linear system. Solve for m entries using linear least squares using the method below.

The camera center is <-1.5126, -2.3517, 0.2827>, and the projection matrix is:

0.7679 -0.4938 -0.0234 0.0067
-0.0852 -0.0915 -0.9065 -0.0878
0.1827 0.2988 -0.0742 1.0000

Part II: Fundamental Matrix Estimation

To compute the optimal fundamental matrix, we use an improved method by normalizing the coordinates before computing the fundamental matrix.

Estimate of the fundamental matrix for the base image pair is:

-0.0000 0.0000 -0.0001
0.0000 -0.0000 -0.0011
-0.0000 -0.0015 0.0345

And image pair is like:

Part III: Fundamental Matrix with RANSAC

In this part, we sample point correspondences computed with SIFT, and each time apply RANSAC method to find the best point correspondences and based on that compute the fundamental matrix. This is done by the code below. 


            % compute number of inline points and fundamental matrix
            sample = randsample(n, 9);
            Fmatrix = estimate_fundamental_matrix(matches_a(sample,:), matches_b(sample,:));

            count = length(find_index_in_threshold(matches_a, matches_b, n, threshold, Fmatrix));
            if(count > num_inliers)
                num_inliers = count;
                best_Fmatrix = Fmatrix;
            end

I set threshold = 0.002 and iteration = 5000 for the four image pairs. And the results are shown as below.

Mount Rushmore: number of inliers 639.

Notre Dame: number of inliers 516.

Episcopal Gaudi: number of inliers 399.

Woodruff Dorm: number of inliers 200.