Project 3: Camera Calibration and Fundamental Matrix Estimation with RANSAC

Part 1: Camera Projection Matrix

To get the projection matrix, a system of equations (Ax = Y) is created, following the template contained in the skeleton code's comments. After the A matrix and the Y vector have been created, the vector form of the M matrix is calculated, with a 1 being appended to the end of the vector, and the vector form is reshaped into the 3x4 M matrix.


M = A \ Y;
M = [M;1];
M = reshape(M, [], 3)';
    

After this, the camera center is extracted from the M matrix using the equation -Q^-1 * m4, where Q is the first three columns of the M matrix, and m4 is the fourth column of the M matrix.


Center = -inv(M(:,1:3)) * M(:,4);
    

Results

Projection Matrix:

[0.7679, -0.4938, -0.0234, 0.0067]
[-0.0852, -0.0915, -0.9065, -0.0878]
[0.1827, 0.2988, -0.0742, 1.0000]

Residual: 0.0445

Camera Center: <-1.5126, -2.3517, 0.2827>


Part 2: Fundamental Matrix Estimation

To estimate the fundamental matrix for a set of points, the 8-point method is used. The regression equations for the 8 points are given in the project description. These equations are refactored into matrix form (Af = 0). This matrix is then solved using Singular Value Decomposition. After this, the F matrix is constrained to the det(F) = 0 property using Singular Value Decomposition.


for i=1:num_points
    u = Points_a(i,1);
    v = Points_a(i,2);
    up = Points_b(i,1);
    vp = Points_b(i,2);
    
    A(i,:) = [u*up, v*up, up, u*vp, v*vp, vp, u, v, 1];
end

[U, S, V] = svd(A);
f = V(:, end);
F = reshape(f, [3 3])';
[U, S, V] = svd(F);
S(3, 3) = 0;
F_matrix = U * S * V';
    

Results

Estimated Fundamental Matrix:

[-5.36e-07, 7.90e-06, -0.00188]
[8.83e-06, 1.21e-06, 0.017]
[-0.0009, -0.02642, 0.9995]

Part 3: Fundamental Matrix with RANSAC

To estimate the fundamental matrix using RANSAC, firstly 8 correspondences are chosen at random. I chose 8 since the 8-point algorithm requires a minimum of 8 points of correspondence. Nine points seemed to be unnecessary. After the points are selected, the estimated F matrix is calculated for those points. To check the accuracy of the estimated F matrix, each correspondence pair is checked against the estimated F matrix. The metric for accuracy is that the equation transpose(X) * estimated_F * X' < 0.008. This process is repeated 10,000 times. The fundamental matrix that produces the largest number of inliers is then selected as the best fit F matrix for the data set.


for iteration=1:num_iterations
    
    % Getting the random points
    random_indices = randperm(num_matches, num_points);
    subset_matches_a = zeros(num_points, 2);
    subset_matches_b = zeros(num_points, 2);
    for i=1:num_points
        random_index = random_indices(i);
        subset_matches_a(i,:) = matches_a(random_index,:);
        subset_matches_b(i,:) = matches_b(random_index,:);
    end
    
    % Estimating F for the subset of points
    estimated_F = estimate_fundamental_matrix(subset_matches_a, subset_matches_b);
    
    % Finding the number of inliers
    cur_inliers_a = [];
    cur_inliers_b = [];
    
    for i=1:num_matches
        x = [matches_a(i,:), 1];
        xp = [matches_b(i,:), 1];
        matchVal = abs(xp * estimated_F * transpose(x));
        
        if (matchVal < threshold)
           cur_inliers_a = [cur_inliers_a; x(1:2)];
           cur_inliers_b = [cur_inliers_b; xp(1:2)];
        end
    end
    
    if (size(cur_inliers_a, 1) > size(inliers_a, 1))
           Best_Fmatrix = estimated_F;
           inliers_a = cur_inliers_a;
           inliers_b = cur_inliers_b;
    end
end
    

Epipolar Results

Mt. Rushmore
Woodruff
Notre Dame
Gaudi