KAUSHIK SANTHANAM

1. Camera Projection matrix estimation

This function determines the projection matrix, which is a 3 X 4 matrix that describes the mapping of a pinhole camera from 3D points in the world to 2D points in an image. A homogeneous linear system of equations for moving from 3D world coordinates to 2D image coordinates using the projection matrix, M is considered. A set of corresponding normalized 2D and 3D points are substituted in the equations and the unknown values of M are determined. The total residual between the projected 2D location of each 3D point and the actual location of that point in 2D image must be very small. This projection matrix M can be divided into a matrix K of intrinsic parameters and a matrix [R/T] of extrinsic parameters. Camera centre is then determined as -(Q^-1)*m4; where m4 is the last column of M and Q is the first 3X3 elements of M.

Example code

%calculate projection matrix
M = zeros(3,4);
[m m_1] = size(Points_2D);

A = zeros(m * 2, 11);

A(1:2:end) = [Points_3D ones(m,1) zeros(m,4) -Points_2D(:,1).*Points_3D(:,1) -Points_2D(:,1).*Points_3D(:,2) -Points_2D(:,1).*Points_3D(:,3)];
A(2:2:end) = [zeros(m,4) Points_3D ones(m,1) -Points_2D(:,2).*Points_3D(:,1) -Points_2D(:,2).*Points_3D(:,2) -Points_2D(:,2).*Points_3D(:,3)];

B = reshape(Points_2D.',2*m,1);

M = A\B;
M = [M;1];
M = reshape(M,[],3)';

 
%computing camera centre

Q = M(:, 1:3);
m4 = M(:, 4);
Center = - inv(Q) * m4;

The projection matrix is :

The total residual is: 0.0445.

The estimated location of camera is: -1.5126, -2.3517, 0.2827

2. Fundamental matrix estimation

To obtain the epipolar geometry, we need to estimate fundamental matrix F. This can be done using some initial point correspondences between the images. Equation below shows that each matching pair of points between the two images provides a single linear constraint on F. This allows F to be estimated linearly (up to the usual arbitrary scale factor) from 8 independent correspondences.

(f11uu’ + f12vu’ + f13u’ + f21uv’ + f22vv’ + f23v’ + f31u + f32v + f33) = 0, where u,v,u’ and v’ are corresponding point locations in two images.

The linear squares estimate of F is full rank; however, the fundamental matrix is a rank 2 matrix. In order to reduce the rank, we can decompose F using singular value decomposition into the matrices U ΣV' = F. We can then estimate a rank 2 matrix by setting the smallest singular value in Σ to zero thus generating Σ2 . The fundamental matrix is then easily calculated as F = U Σ2 V'.

Example Code

%compute fundamental matrix

[m m_1] = size(Points_a);

w = zeros(m, 9);
w = [Points_a(:, 1).*Points_b(:, 1) Points_a(:, 2).*Points_b(:, 1) Points_b(:, 1) Points_a(:, 1).*Points_b(:, 2) Points_a(:, 2).*Points_b(:, 2) Points_b(:, 2) Points_a(:, 1) Points_a(:, 2) ones(m, 1)];

[U, S, V] = svd(w);

q = V(:, 9);
q=reshape(q', 3, 3);

[u, s, v] = svd(q);
s(3, 3) = 0;

F_matrixtranspose = u * s * v';
F_matrix = F_matrixtranspose';

Output

3. Fundamental matrix with RANSAC

The fundamental matrix is computed with unreliable point correspondences computed with SIFT. Least squares regression is not appropriate in this scenario due to the presence of multiple outliers. RANSAC is used in order to determine the fundamental matrix. 8 random corresponding matched points from two images are taken and the fundamental matrix is solved and determined. The number of inliers for this fundamental matrix is computed. To determine the points that correspond to inliers for a particular fundamental matrix, a distance measure or threshold is taken. Thus, a threshold is picked between the inliers and outliers and this threshold is varied in order to get better results.This process is iteratively done and the fundamental matrix with the highest number of inliers is taken as the final fundamental matrix.

Example Code

%Fundamental Matrix with RANSAC

start = 1;
ending = 4000;
max_count = 0;

for index_1 = start:ending
    
    i_count = 0;
    indices = randsample(size(matches_a, 1), 8);
    
    Points_a = matches_a(indices, :);
    Points_b = matches_b(indices, :);
    
    Fundamental_matrix = estimate_fundamental_matrix(Points_a, Points_b);
    
    for index_2 = 1:size(matches_a, 1)
        val = [matches_b(index_2, :) 1] * Fundamental_matrix * [matches_a(index_2, :) 1]';
        if abs(val) < 0.05
            i_count = i_count + 1;
        end
    end
    
    if(i_count > max_count)
            max_count = i_count;
            Best_Fundamental_matrix = Fundamental_matrix;
    end
end
   
val = zeros(size(matches_a, 1), 1);
    
for index_3 = 1:size(matches_a, 1)
       val(index_3) = abs([matches_b(index_3, :) 1] * Best_Fundamental_matrix * [matches_a(index_3, :) 1]');
end

[X I]=sort(val, 'ascend');      
   
i_a = matches_a(I(1:max_count), :);
i_b = matches_b(I(1:max_count), :);

Output

Mount Rushmore

Output

Notre Dame