The assigned project focused on the projection of coordinates from a 3 dimensional space to a 2 dimensional one as well as using epipolar geometry to solve the point correspondence problem. The biggest lesson that this project taught me was that if you set up your given data in a format that an algorithm can use (SVD in my case), then you can solve for the missing parts fairly accurately. The parts that needed to be completed where:
For the camera calibration problem, I set up my system of linear equations as non-homogenous so I could use SVD. After I found the projection matrix to transform the 3D world to 2D, I was able to center the camera based on the matrix that was calculated by taking the negative inverse of the first 3 columns then multiplying it by the fourth column. I found that the most challenging part of this problem was just setting up the matrix. I did it like so:
rows = size(Points_3D);
rows = rows(1);
A = zeros(2 * rows, 12);
Y = zeros(2 * rows, 1);
%Setting up matrix
for i=1:rows
row_3D = Points_3D(i, :);
row_2D = Points_2D(i, :);
ind = i * 2;
Y(ind - 1, 1) = row_2D(1);
Y(ind, 1) = row_2D(2);
A(ind - 1, 4) = 1;
A(ind, 8) = 1;
A(ind - 1, 12) = -row_2D(1);
A(ind, 12) = -row_2D(2);
for j = 1:3
item = row_3D(j);
A(ind - 1, j) = item;
A(ind, j + 4) = item;
A(ind - 1, j + 8) = item * -row_2D(1);
A(ind, j + 8) = item * -row_2D(2);
end
end
This problem initially felt very similar to the previous one as the challenging part was setting up the matrix. After I set it up, I used SVD to estimate the fundamental one. I noticed that every row of the fundamental matrix always has [u' v' 1] at every 3 indexes, so I saved that and multiplied it to generate the matrix. The code used to set up the initial matrix is below.
rows = size(Points_a);
rows = rows(1);
A = zeros(rows, 9);
for i=1:rows
row_a = Points_a(i, :);
row_b = Points_b(i, :);
mB = [row_b(1) row_b(2) 1];
A(i, 1:3) = mB .* row_a(1);
A(i, 4:6) = mB .* row_a(2);
A(i, 7:9) = mB;
end
[U, S, V] = svd(A);
f = V(:, end);
F_matrix = reshape(f, [3 3]);
We used RANSAC to grab random subsets of corresponding pairs to be able to build our matrix. We build our matrix several times until we come up with a correct one, which makes the algorithm fairly robust to outliers. I iterated 5000 times and used a threshold of .05. The code is depicted below.
finalInliers = 0;
Best_Fmatrix = zeros(3, 3);
rows = size(matches_a, 1);
finalList = zeros(rows, 1);
for i=1:5000
sampleIndex = ceil(rand(10, 1) .* rows);
M = estimate_fundamental_matrix(matches_a(sampleIndex, :), matches_b(sampleIndex, :));
curr = 0;
currList = zeros(rows, 1);
for j = 1 : rows
dis = abs([matches_b(j, :) 1] * M * [matches_a(j, :) 1]');
if (dis < .05)
curr = curr + 1;
currList(j) = dis;
else
currList(j) = 2^1500;
end
end
if curr > finalInliers
Best_Fmatrix = M;
finalList = currList;
finalInliers = curr;
end
end
[~, d] = sort(finalList);
if (finalInliers > 200)
inliers_a = matches_a(d(1:200), :);
inliers_b = matches_b(d(1:200), :);
else
inliers_a = matches_a(d(1:finalInliers), :);
inliers_b = matches_b(d(1:finalInliers), :);
end