Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC

Part 1: Projection matrix and Camera center

Fig 1. 2D-Point location estimation

Fig 2. Camera center estimation (Red cross is camera center)

Given a set of corresponding 2D coordinate points on the image and the 3D coordinate points in the world, the projection matrix were calculated. The two input set of points were rearranged so that the singular value decomposition can be applied to solve for M, the projection matrix. As a result, M is:

0.4583 -0.2947 -0.0140 0.0040
-0.0509 -0.0546 -0.5411 -0.0524
0.1090 0.1783 -0.0443 0.5968

Figure 1 is a visualization of comparison between the original 2D points (Actual Points), and the projected 2D points calculated by the 3D points and M (Projected Points). The residual from calculating the projected points were 0.0445, which is quite low. Then the center of the camera were calculated by taking the inverse of the first three columns of M and multiplying it by negative of the fourth column of M. The camera center result is also visualized in figure 2, which had values of (-1.5127 -2.3517, 0.2826).

Part 2: Fundamental Matrix

Similar to part 1 in calculating the projection matrix, the fundamental matrix for part 2 were also calculated using singular value decomposition. But for the fundamental matrix, both of the two input set of points were 2D coordinates from 2 separate images. According to the definition of the fundamental matrix, if the two coordinate points from each image exist in the same point in the 3D plane, multiplying one coordinate point, the fundamental matrix, and the transpose of the other coordinate point should yield 0. Using that definition, the input coordinates were rearranged to calculate the fundamental matrix through two SVD operations. The two images above are the output after drawing epipolar lines based on my calculations for the fundamental matrix:

-5.3626e-07 7.9036e-06 -0.0019
-8.8354e-06 1.2132e-06 0.0172
-9.0738e-04 -0.0264 0.9995

Part 3: RANSAC

In order to implement RANSAC to choose the best fundamental matrix (F), several steps were repeated for 2000 iterations unless certain threshold were met (number of matches captured by F / total number of matches). First, 8 random matches were chosen every iteration, then were used to calculate F, which were evaluated and stored. During the evaluation, the number of inlines were chosen by using the definition of the fundamental matrix, in which if the resulting value is less than the specified threshold, the point is considered inline. The result images shown above only shows the top 30 matches from my algorithm, chosen based on how close the inliers are from F. But because only 30 are shown for visualization purposes, re-running the procedure may result in varied epipolar line visualizations from the ones shown above.