Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC

Example of a right floating element.

For this project I first first had to do the calculate projection matrix function. For this algorithm I chose to do this the linear method as it is easier to formulate and solve. I set the scale to one so the u and v would be easily caculated. Using the u ,v , X, Y, Z I were able to get the projection matrix by forming a matrix using the homogeneous linear system. A very useful method I found was the svd() function which helped me solve the system of equations necessarily to produce the projection matrix. My resulting matrix after running the function was

0.4583 -0.2947 -0.0140 0.0040

-0.0509 -0.0546 -0.5411 -0.0524

0.1090 0.1783 -0.0443 0.5968

Here is some example code of how I accomplished this calculation assuming A already contains the u,v,X,Y,Z combination matrix


	[U, S, V] = svd(A);
	M = V(:,end);
	M = reshape(M,[],3)';
	%M will now hold the projection Matrix

After I have gotten the projection matrix I then did the function for caculating camera center. I did this similiarly to the way shown in lecture by splitting my matrix into a 3x3 and a 1x3. I then proceeded to matrix multiply the inverse of the 3x3 with the 1x3 which gave me the center. This is much like the x = K[R t]X but since the 3x3 part of the matrix M is already the K*R part we can solve it by just doing inverse(Q) * m4. After running my part1 code I got the camera was located at <-1.5127, -2.3517, 0.2826>

Here is some sample code of how I calculated the camera center


	B = M(:,[1 2 3]);
	C = M(:,4);
	Center = -inv(B)*C;

Part1 Output

For part2 I was implementing a estimation of the fundamental matrix function. To do this I did something similar to what I did for the projection matrix. I first created the matrix that uses a combination of u and vs, and I iterated through the matrix in order to fill it up. I then did the same svd function I used in projection matrix in order to get a 3x3 fundamental matrix. Now using the fundamental matrix we can see the epipolar lines as they pass through the corresponding points

Here is some sample code of how I calculated the fundamental matrix


	%Matrix A is populated and svd to solve the linear system
	[x,y] = size(Points_a);
	A = zeros(x,9);
	for row = [1:1:x]
	    A(row,1) = Points_a(row,1)*Points_b(row,1);
	    A(row,2) = Points_a(row,1)*Points_b(row,2);
	    A(row,3) = Points_a(row,1);
	    A(row,4) = Points_a(row,2)*Points_b(row,1);
	    A(row,5) = Points_a(row,2)*Points_b(row,2);
	    A(row,6) = Points_a(row,2);
	    A(row,7) = Points_b(row,1);
	    A(row,8) = Points_b(row,2);
	    A(row,9) = 1;

Part2 Output

For part3 I implemented the ransac fundamental matrix. The ransac fundamental matrix function. For this algorithm I had to play with a lot of different thresholds and number of iterations to find the one I wanted. For each iteration I found some random samples which I then matches through the matches a and matches b. I then created a fundamental matrix using those points and used that to create the inliners. Using that I created cutoffs to see what the inliers were as well as the ransac fundamental matrix. I noticed that running this algorithm, the images can sometimes be inconsistent in which I attribute to the points being coplanar and the camera not being a pinhole camera and may have lens distortion.

Here is some sample code I used to calculate the inliners


	for i = 1 : rows
		  if abs([matches_b(i, :) 1] * fundMatrix * [matches_a(i, :) 1]') < upperbound
		  	  linesB = [linesB; matches_b(i, :)];
			  linesA = [linesA; matches_a(i, :)];
		  end

Here are some examples of my part3 that I ran on different images