Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC
Keypoint correspondences between Episcopal Gaudi image pairs with normalization
The goal of this project is to estimate the camera projection matrix and fundamental matrix with RANSAC. This project can be divided into three parts:
- Estimate camera projection matrix and camera center
- Estimate fundamental matrix for base image pair
- Estimate fundamental matrix with RANSAC
Estimate camera projection matrix and camera center
To compute the projection matrix that goes from 3D world coordinates to 2D image coordinates, I used the following equation
To solve this equation, I set m34 to 1 and then find the remaining coefficients using linear least squares.The result of my matrix is shown as below
To get camera center, we only need to apply the equation C=-Q^-1*m4, where Q is the first three columns of matrix M and m4 is the last column of matrix M.
My result of camera center is <-1.5126, -2.3517, 0.2827> using normalized 3D points
Actual projected
Camera center view
Estimate fundamental matrix for the base image pair
To calculate the fundamental matrix, I use the following equation, where (u,v) is the point in left image
and (u',v') is the corresponding point in right image.
The above equation is the same as below, which can be solved using linear least squares.
But the least squares estimate of F is of rank 2, so we need to reduce the rank to 1 by forcing det(F)=0.
The result of fundamental matrix without normalization for base image pair is:
The following two images show the epipolar lines crossing through the corresponding point
Estimate fundamental matrix with RANSAC
To estimate the fundamental matrix with RANSAC, I use the following steps
- Pick 8 points randomly to build the fundamental matrix with function wrote in part2.
- Count the number of inliers that support the model.
- Repeat the above two steps 2000 times and record the model that has the most number of inliers.
In step 2, to count the number of inliers, I check if the absolute value of P'FP less than 0.01, where
P' is the point in right image, P is the corresponding point in left image and F is the fundamental matrix. If the value is less than 0.01, I then increase the number of
inliers and record the distance to the model.
In this part, I only return the first 30 points which has the closest distance to the model.
Extra Credit
In this part, I improve the fundamental matrix by normalizing the coordinates before computing the fundamental matrix. I apply the following equation,
where c_u and c_v are the mean coordinates and s is the scale. The first matrix of the right hand side is the scaling matrix, the second matrix of the
right hand side is the offset matrix. The product of scaling matrix and offset matrix is the transform matrix.
To calculate s, I first estimate the standard deviation after substracting the means, then use 1/std to get s.
The final step is to use the scaling matrix to adjust the fundamental matrix so it can operate on the original coordinates, which
can be done with the following equation:
