Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC

Part One

For the projection matrix section part one I setup the regression equations as described, set the m34 variable to 1, and solved using the / operator on the matrices. The output of this (after some reshaping of the solution vector) was a projection matrix which mapped 3D world coordinates to their correspondig 2D image coordinates. For the camera center section I used the Q inverse and fourth column of the projection matrix to solve for the point. All of my output is listed below.

Part Two

Part two was similar to part one in that there were regression equations that needed to be setup to estimate the fundamental matrix. This time, however, I solved using the singular value decomposition to solve the constrained optimization problem that was defined in the project description. This solution wasn't the final answer, since the matrix still needed to be reduced to rank two. This was done with another SVD and by setting the smallest singular value equal to 0. The resulting epipolar lines on the classroom image seemed to be spot on and are shown below for one of the images in the pair.

Part Three

Part three incorporated the code from part two into the RANSAC algorithm for estimating a fundamental matrix. The algorithm repeatedly samples a small number of potential pairs in the images and constructs fundamental matrices based on them. After several thousand iterations it picks the best matrix, based on the number of inliers. I implemented the algorithm as described in the class slides with the following parameters: probability that at least one sample is free from outliers: .99, outlier ratio: .5, number of correspondence pairs: 9 (these first three were used to computer the number of iterations), and an error threshold of .005. These numbers were chosen based partially on recommendations from course materials and also due to trial and error on the images. Unfortunately, I was never able to get a fundmental matrix that looked 100% correct on any of the images, but the remaining inliers tended to be somewhat accurate. I thought it was interesting that even an incorrect fundamental matrix had enough geometric significance to filter out some incorrect matches and keep mostly correct ones. The matches would be different on different iterations, and would usually center around where the epipolar lines intersect. This is most apparent on the Mount Rushmore images. The results on some of the image pairs are shown below.