Andrea Hu

Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC

Part 1: Camera Projection Matrix

The goal of this part is to estimate the projection matrix and camera center from the a set of 2D and 3D points.

1. Generate a matrix representing a system of equations to solve for M. Since we are given the 2D and 3D coordinates, we can directly use these values to set up the system of equations with minimal manipulation of values. Each row in the 2D coordinate matrix corresponds to u and v. Each row in the 3D coordinate matrix corresponds to x, y, and z. We need two rows in the matrix per mapping of 2D to 3D coordinates.
2. Take the singular value decomposition of the matrix and retrieve the right singular vectors and reshape it into a 3x3 matrix. This matrix is the projection matrix.
3. Split the projection matrix into Q and m4. Calculate [-1 * Q \ m4] to get the coordinates of the camera center. This was the formula provided to us to use.

Part II: Fundamental Matrix Estimation

1. Set up a system of equations in the form of a matrix to solve for F. We can get these values by manipulating each point in Points_a and Points_b.
2. Take the singular value decomposition of the matrix and retrieve the right singular vectors and reshape it into a 3x3 matrix. This is in full rank.
3. Take the singular value decomposition of the matrix resulting from step 2. To break it down into U, S, and V.
4. Set the smallest singular value to 0. To get a rank 2 matrix.
5. Multiply U, S, and V transpose to get F in rank 2.

Part III: Fundamental Matrix with RANSAC

1. Select 8 random indices and use them to retrieve the corresponding points in matches_a and matches_b. These will be used to calculate a fundamental matrix based on these values.
2. Estimate a fundamental matrix based off of those values.
3. For each pair of points in matches_a and matches_b, get the absolute value of the error when multiplied with the Fundamental matrix. If the error is less than a pre-set delta, increment the number of inliers. We keep a running count of inliers instead of keeping an array for performance reasons.
4. If the number of inliers is greater than that generated by a previous Fundamental matrix, update the best fundamental matrix and the largest number of inliers counted. This allows us to keep track of the best fundamental matrix and its number of inliers. We will keep updating this in a loop.
5. Repeat steps 1-4 for a set number of iterations.
6. Now that we are done iterating, loop through the points once more with the best fundamental matrix to get lists of at most 30 inliers. This is what we will return to the user.

Parameters

These values for the following parameters in ransac_fundamental_matrix.m provided optimal results.

• Delta = 0.0055. This is the maximum error.
• Iterations = 7000. This is the number of times we iterate steps 1-4.
• Number of random points = 8. This seemed to produce less error than other values.

Estimate of the Projection Matrix and the Camera Center

This is the output from running proj3_part1.

	The projection matrix is:
	    0.4583   -0.2947   -0.0140    0.0040
	   -0.0509   -0.0546   -0.5411   -0.0524
	    0.1090    0.1783   -0.0443    0.5968


	The total residual is: <0.0445>

	The estimated location of camera is: <-1.5127, -2.3517, 0.2826>

Estimate of the Fundamental Matrix for the Base Image Pair

This is the result from running proj3_part2.

	F_matrix =

	   -0.0000    0.0000   -0.0019
	    0.0000    0.0000    0.0172
	   -0.0009   -0.0264    0.9995

Here are the results from running proj3_part3.

Mount Rushmore: SIFT, epipolar lines, and matches

Notre Dame: SIFT, epipolar lines, and matches

Gaudi: SIFT, epipolar lines, and matches

Woodruff: SIFT, epipolar lines, and matches

Matches for images (enlarged)