Computer Vision Project

Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC

Part 1: Projection Matrix & Camera Center Calculations

The first step of the project was calculate the projection matrix, given a 2D-to-3D mapping, and then, given the previously calculated projection matrix, calculate the camera center.

During the projection matrix calculation step, a for loop was used to create the matrix for the system of equations. To solve the equations, single value decomposition was used, as shown below:


% SVD breaks the matrix down into U, S, and V
[~, ~, V] = svd(A);
M = reshape(V(:,end),[],3)';

Results: Normalized Points

The normalized points served as a baseline and a test for checking implementation correctness:


    0.4583   -0.2947   -0.0140    0.0040
   -0.0509   -0.0546   -0.5411   -0.0524
    0.1090    0.1783   -0.0443    0.5968


The total residual is: <0.0445>

The estimated location of camera is: <-1.5127, -2.3517, 0.2826>

Results: Challenging Points

After the being able complete the easy points, the calculations were also performed on the harder points (not normalized). The results were a lot less accurate (expectedly so), as the results show:


    0.0069   -0.0040   -0.0013   -0.8267
    0.0015    0.0010   -0.0073   -0.5625
    0.0000    0.0000   -0.0000   -0.0034


The total residual is: <15.5450>

The estimated location of camera is: <303.1000, 307.1843, 30.4217>

Part 2: Fundamental Matrix Calculation

Part 2 of the project consisted of estimating the fundamental matrix, a function that is also used in part 3.

The creation of the fundamental matrix was done in the following for loop and SVD.


for i = 1:n
    a = Points_a(i, :);
    b = Points_b(i, :);
    
    u_a = a(1, 1);
    v_a = a(1, 2);
    u_b = b(1, 1);
    v_b = b(1, 2);
    
    A(i, :) = [u_a*u_b  v_a*u_b  u_b  u_a*v_b  v_a*v_b  v_b  u_a  v_a  1];
end

[~, ~, V] = svd(A);
[U, S, V] = svd(reshape(V(:, end), [3 3])');

In addition to accurately computing the fundamental matrix, an important part of the function was to be able to complete the computations in an efficient amount of time. For this, preallocation was done in the function.


% Preallocate A
A = zeros(n, 9);

Results: Epipolar Lines

Results: Epipolar Lines (from Noisy Points)

Part 3: Fundamental Matrix Estimation with RANSAC

The final segment of the project consisted of estimating the fundamental matrix using RANSAC, given unreliable point matches. With RANSAC, erroneous matches can be overcome with multiple iterations of random sampling and modeling.

Results: Mount Rusmore

For the Mount Rushmore image pair, the results were consistent, giving good matches and similar lines over multiple runs of RANSAC at 1500 iterations.

Results: Notre Dame (Demonstrating RANSAC Variability)

Below are two sets of results from the Episcopal Gaudi, both using the same parameters (1500 iterations, 0.85 ratio threshold for RANSAC). These sets were selected to illustrate the one of the major effects of RANSAC, which is random results, some of which are inaccurate or suboptimal.

Christopher Tam