Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC
The goal of this project is to introduce you to camera and scene geometry. Specifically we will estimate the camera projection matrix, which maps 3D world coordinates to image coordinates, as well as the fundamental matrix, which relates points in one scene to epipolar lines in another. The camera projection matrix and the fundamental matrix can each be estimated using point correspondences. To estimate the projection matrix (camera calibration), the input is corresponding 3d and 2d points. To estimate the fundamental matrix the input is corresponding 2d points across two images. You will start out by estimating the projection matrix and the fundamental matrix for a scene with ground truth correspondences. Then you'll move on to estimating the fundamental matrix using point correspondences from from SIFT.
Part I: Camera Projection Matrix
The goal is to compute the projection matrix that goes from world 3D coordinates to 2D image coordinates.
I used nonhomogeneous linear system to solve for fundamental matrix.
Here is the result of part 1.
The estimated location of camera is: <-1.5126, -2.3517, 0.2827>
Part II: Fundamental Matrix Estimation & Normalizing Coordinates
The next part of this project is estimating the mapping of points in one image to lines in another by means of the fundamental matrix.
I used 8-point algorithm to solve for fundamental matrix.
however, there is some small error for the lines. The error is mitigated after normalizing the coordinates. Here is the result after normalizing the coordinates. You can see the lines pass through exactly through the red points, especially for the top and bottom ones.
Part III: Fundamental Matrix with RANSAC
For two photographs of a scene it's unlikely that you'd have perfect point corresponence with which to do the regression for the fundamental matrix. So, next you are going to compute the fundamental matrix with unreliable point correspondences computed with SIFT (as you might get from project 2). As discussed in class, least squares regression is not appropriate in this scenario due to the presence of multiple outliers. In order to estimate the fundamental matrix from this noisy data you'll need to use RANSAC in conjunction with your fundamental matrix estimation.
I follow the RANSAC algorithm, first randomly select 8 point pairs, compute the fundamental matrix corresponding these points, then count the number of inliears as score. And loop for 1000 time pick the best model based on the score.
Mount Rushmore (with normalized coordinates)
Notre Dame (with normalized coordinates)
Gaudi(without normalized coordinates)
The Gaudi pair doesn't give good result, but it can be improved with normalizing coordinates. Here is the result after normalizing coordinates.
