Ramamurthy Siripuram

The goal of the first part of this project was to estimate the projection matrix of a camera. The projection matrix is the matrix that when multiplied by a set of 3d corrdinates will return the corresponding 2d coordinates on the picture the camera will take. This was accomplished by mapping a list of known 3d coordinates to a list of complementary 2d coordinates. Using pairs of 3d and 2d coordinates I was able to use single value decomposition to solve for the projection matrix.

The other goal was to calculate the position of the camera. This was accomplished by taking the 3x3 matrix made by the 9 left most cells of the projection matrix and the 3x1 matrix composed of the 3 right most cells of the projection matrix, called Q and m4 respectively. Then taking the negation of the matrix multiplication on the inverse of Q with m4

The results from using the given normalized data was as follows

The projection matrix is:
    0.4583   -0.2947   -0.0140    0.0040
   -0.0509   -0.0546   -0.5411   -0.0524
    0.1090    0.1783   -0.0443    0.5968


The total residual is: <0.0445>

The estimated location of camera is: <-1.5127, -2.3517, 0.2826>

The image below shows the error of the projection matrix. The total residual was less than 1, so the algorithm is very accurate.

Part 2: Fundamental Matrix

The goal of this part was to calculate the fundatmental matrix of 2 images. The fundamental matrix calculates the correspoding epipolar line on one image given a point on the other image.

The following is the code used:

A = zeros(numPoints, 9);
rnge = 1:numPoints;
for i = rnge
    u = Points_a(i, 1);
    v = Points_a(i, 2);
    
    up = Points_b(i, 1);
    vp = Points_b(i, 2);
    
    A(i, :) = [u*up u*vp u v*up v*vp v up vp 1];
end

[~, ~, V] = svd(A);
F_matrix = reshape(V(:, end), [3 3]);

The loop iterates through the matched points to construct the matrix representing the system of equations containing all of the points. Then an svd of the matrix gives the fundamental matrix. Below is the fundamental matrix calcuated from the given data.

-6.60698417012683e-07	7.91031620841432e-06	-0.00188600197690856
8.82396296136233e-06	1.21382933020625e-06	0.0172332901072654
-0.000907382302152516	-0.0264234649901809	0.999500091906723

These are the images with which the above fundamental matrix was calculated. The epipolar lines have been drawn on them.

Part 3: Fundamental Matrix Estimation with RANSAC

The goal of this part was to find the fundamental matrix of two real photographs where feature matches were actually calculated. Here there wont be perfect correspondence between matches.

The following is the code used:

quality = 0; %percentage of points within threshold (inliers)
inliers = 0; %number of inliers
threshold = 0.1; %threshold. Points must be within +/- this threshold in order to be a inlier
while q < 0.75
    smpl = randsample(numTotal, samplesize);
    Fmatrix = estimate_fundamental_matrix(matches_a(smpl,:), matches_b(smpl,:)); %calculate fundamental matrix with random set of points
    l = ones(numTotal, 1);
    x = [matches_a l matches_b l]; %construct system of equations that represents the points
    helper = @(ind) (x(ind,1:3) * Fmatrix * x(ind,4:6)'); %helper function to calculate xT * F * x'
    s = arrayfun(helper, 1:size(x,1));
    log_s = s < thr & s > -1*thr; %find the points that are with the threshold
    inliers = sum(log_s); %get number of points within the threshold
    inliers_a = matches_a(log_s, :); %get inliers from a
    inliers_b = matches_b(log_s, :); %get inliers from b
    new_q = inliers/numTotal;
    if new_q > q
        q = new_q;
        Best_Fmatrix = Fmatrix;
    end
end

I take advantage of the fact that xT * F * x' = 0 if F is a good fundamental matrix. So a estimated fundamental matrix where a lot of the matches meet that criteria will be a good fundamental matrix. In my code I set the minimum qualification for a fundamental matrix to be that 75% of all matches must have 0.1 > xT * F * x' > -0.1.

Mount Rushmore

Notre Dame

Episcopal Gaudi

Woodruff Dorm