Harry Lane

Part 1: Camera Projection Matrix

Part 1 of this project dealt with estimating the projection matrix and calculating the camera center from that estimation. The projection matrxix is fairly easy to estimate. First a system of equations is created from the 2D image coordinates and 3D world coordinates. A solution to those equations is then found with the following code:

	M = A \ Y;
	M = [M, 1];
	M = reshape(M, [], 3)'

This code gave an estimation of the projection matrix as this:

[
    0.7679   -0.4938   -0.0234    0.0067;
   -0.0852   -0.0915   -0.9065   -0.0878;
    0.1827    0.2988   -0.0742    1.0000;
]

From the projection matrix, the camera center can be found with the following code:

	Q = M(1:3, 1:3);
	m4 = M(:, 4);
	Center = -inv(Q) * m4;

With that code, and the previous projection matrix, the camera center comes out to be:

	( -1.5126,   -2.3517,    0.2827)

Part 2: Fundamental Matrix Estimation

Similar to Part 1, the next part deals with estimating lines in one image from points in another. This involves constructing another system of equations, but this is solved in a slightly different manner. Instead of using the '\' operator, svd is used twice. The first time solves f from Af = 0, while the second resolved the constraint that the det(F) = 0. We can write that with this code:

	[U, S, V] = svd(A);
	f = V(:, end);
	F = reshape(f, [3 3])';

	[U, S, V]= svd(F);
	S(3, 3) = 0;
	F_matrix = U * S * V';

And you end up with a fundamental matrix estimation of:

[
   -0.0000    0.0000   -0.0019
    0.0000    0.0000    0.0172
   -0.0009   -0.0264    0.9995
]

Part 3: Fundamental Matrix with RANSAC

The final part of the project deals with the fact that we rarely have perfect correspondance for a pair of point sets. Instead, there will most likely be two sets of points which match somewhat, but contain noise, much like you would get after collecting SIFT features and matching them. Using these point sets will create an inaccurate fundamental matrix, but we can improve its accuracy with RANSAC. Using RANSAC, we can ignore the outlier matches and just focus on matches that are truly accurate. My implementation of RANSAC looked like this:

    indicies = randsample(size(matches_a, 1), s);
    a_samples = matches_a(indicies, :);
    b_samples = matches_b(indicies, :);
    [guess] = estimate_fundamental_matrix(a_samples, b_samples);
        
    count = 0;
    guess_inliers = zeros(size(matches_a, 1));
    
    for i=1:size(matches_a,1)
        temp = [matches_b(i, :) 1] * guess * [matches_a(i, :) 1]';
        if (abs(temp) < distance_threshold)
            count = count + 1;
            guess_inliers(i) = 1;
        end
    end
    
    temp_confidence = count / size(matches_a, 1);
    
    if (temp_confidence > confidence)
        inlier_indexes = find(guess_inliers);
        inliers_a = matches_a(inlier_indexes, :);
        inliers_b = matches_b(inlier_indexes, :);
        Best_Fmatrix = guess;
        confidence = temp_confidence;
    end

This was repeated 1000 times with a sample size of 8 and a distance_threshold of .01 and those values were found through experimentation. With this algorithm, we can find epipolar lines for a pair of images that look like this:

Or this: