Project 3: Camera Calibration and Fundamental Matrix Estimation with RANSAC

Part I: Camera Projection Matrix

For part one of the project I was tasked with calculating the projection matrix given corresponding 2D and 3D points. I started by looping over the points and adding them to a matrix two lines at a time (line 1: [Xn, Yn, Zn, 1, 0, 0, 0, 0, -(Un*Xn), -(Un*Yn), -(Un*Zn)] , line 2: [0, 0, 0, 0, Xn, Yn, Zn, 1, -(Vn*Xn), -(Vn*Yn), -(Vn*Zn)]). Once the matrix was compiled I created the M matrix by dividing the matrix I created by the image points, setting the last element to 1, and reshaping it. I then calculated the camera center using the M matrix found above by creating a Q matrix which was the first three columns of the M matrix and the m4 matrix which was the last column. I then was able to find the camera center by calculating the negative inverse of the Q matrix multiplied by the m4 matrix.

Part I Results:

Part II: Fundamental Matrix Estimation

For part two of the project I needed to calculate the fundamental matrix given corresponding point pairs to estimate the mapping of points in one image to another image. For this I iterated through the points and created a matrix line by line of the format: [(Ua*Ub), (Va*Ub), Ub, (Ua*Vb), (Va*Vb), Vb, Ua, Va, 1]. I then computed the svd of the matrix I created, fixed the scale and reshaped it. Since the least squares estimation of the fundamental matrix is full rank I then computed another svd on it, set the smallest singular value to 0, and then calculated F = U Σ2 V' to find the rank 2 fundamental matrix.

Part II Results:

Part III: Fudamental Matrix with RANSAC

In part three of the project I implemented the Fundamental Matrix with RANSAC. I needed to use RANSAC because the match pairs from this image set would not necessarily be perfect as they were derived from VLFeat’s SIFT matching. This meant that I would need to use RANSAC to pare down the number of points I used to create the Fundamental Matrix. To do this I first randomly selected 8 points. From these 8 points I computed the fundamental matrix in the same way from part 2. I then computed the number of inliers for the fundamental matrix. I did this by multiplying the transpose of the homogeneous coordinates in image a with the fundamental matrix and then again by the homogeneous coordinates in image b for each match pair as determined by the SIFT calculations. This provided me a number that showed if those points could fit with the fundamental matrix within a certain amount of error (sigma = .008). If the point was relatively close to the fundamental matrix (0) I considered it to be an inlier and found a confidence rating by dividing the number of inliers by the number of total matches. I then repeated this process from the beginning until I had a confidence rating that was sufficiently high (85% of matched points were inliers). I then proclaimed the fundamental matrix associated with that to be the fundamental matrix for the images.

Part III Results:

Note: Sigma and Confidence have been weakened for submission to speed up the program for the grader.