Computer Vision Project

The goal of this project is to estimate the camera projection matrix and the fundamental matrix using point correspondences. To estimate the projection matrix, the input is corresponding 3d and 2d points. To estimate the fundamental matrix the input is corresponding 2d points across two images.

Details

Part I: Camera Projection Matrix

In this part, we use given 3D and 2D coordinates to calculate the projection matrix, as well as the camera center position. The method of setting up the linear equation system is described in lectures and in assignment, so I will not repeat it here. The result of my projection matrix is:

        |  0.4583    -0.2947    -0.0140    0.0040 |
        | -0.0509    -0.0546    -0.5411   -0.0524 |
        |  0.1090     0.1783    -0.0443    0.5968 |

and my camera position is <-1.5127, -2.3517, 0.2826>.

Part II: Fundamental Matrix Estimation

The fundamental matrix is calculated by some corresponding 2D coordinates in two images. After the linear system is built, we use the same method as before to solve for the matrix we want. In this part, however, it is important to normalize input data in order to give numerical stability to the solution. The fundamental matrix I got in this part is:

            | -1.3519e-7    1.8540e-6   -4.6340e-4 |
            |  1.2818e-6   -3.1551e-7     0.0037   |
            | -2.7125e-5   -0.0051        0.1192   |

The resulting epipolar lines are shown below.

Part III: Fundamental Matrix with RANSAC

In this part, we use SIFT extracted points from images instead of given ground truth points. The challenge here is that there are so many points and many of them are outliers. So we use RANSAC to try to exclude those outliers. In order to fit a good model, the threshold which used in RANSAC to reject a fitting is very important and is very sensitive. In my specific implementation, the threshold is around 7e-9 to 9e-9 and each pair need different tuning. Also, because the randomness of this algorithm, sometimes it takes several tries to get a good result. Some "good" resulting pictures on the three pairs are shown below.