Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC

For this project we were required to perform camera calibration and fundamental matrix estimation. We first estimated the fundamental matrix for a scene with ground truth correspondences and then moved on to estimating using the RANSAC method from points from SIFT.

Camera Projection Matrix

The camera projection matrix goes from world 3D coordinates to 2D image coordinates. I chose to use the method of fixing the last element (m34) to 1 and finding the remaining coefficients. This involved calculating the pairs of points using least squares with the \ operator. After getting the projection matrix, the camera center is computed by:

Q = M(1 : 3, 1 : 3);
Center = -inv(Q) * M( : , 4);

Projection matrix evaluation Camera center location

Fundamental Matrix Estimation

This part of the project involves estimating the mapping of points in one image to lines in another. By using 8 pairs of points we can solve the matrix and compute F using singular value decomposition. We then estimate a rank 2 matrix by setting the last value to 0 and then solving F = UΣ2V'. Using this matrix estimation we can display epipolar lines.

Epipolar lines Pic A Epipolar lines Pic B

RANSAC Fundamental Matrix

The final part of the project is to calculate the fundamental matrix using RANSAC. We start by using the point correspondences generated from VLFeat's SIFT feature. After choosing 8 point correspondences, we solve the fundamental matrix using the function from part 2 and count the number of inliers. An inlier is determined by a distance metric that is capped at a threshold. Finally, we'll return the fundamental matrix that provides the most inliers.

Mount Rushmore all SIFT matches
Mount Rushmore Epipolar lines pic 1 Mount Rushmore Epipolar lines pic 2
Mount Rushmore RANSAC matches
Notre Dame RANSAC matches
Episcopal Gaudi RANSAC matches (less accurate than the others)