Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC
Part - I
We had to compute camera projection matrix given 3D points and their corresponding 2D image co-ordinates. It was straightforward to set up matices for linear regression. I used SVD to extract the values of projection matrix
The projection matrix is :-
0.4583 -0.2947 -0.0140 0.0040
-0.0509 -0.0546 -0.5411 -0.0524
0.1090 0.1783 -0.0443 0.5968
Once I had the projection matrix, I used the given formula to calculate camera center
Camera center :-
-1.5127
-2.3517
0.2826
Part - II
The fundamental matrix maps points in one image to lines in another image. Given corresponding points in 2 images, we can estimate the fundamental matrix. It was again straightforward to set up matrices for linear regression. I used SVD to calculate full rank matrix first and later reduced rank to 2 to get correct mapping.
F_matrix :-
-0.0000 0.0000 -0.0019
0.0000 0.0000 0.0172
-0.0009 -0.0264 0.9995
The following images show the resulting epipolar lines
Part - III
We again try to estimate fundamental matrix but this time using SIFT matches on the Mount Rushmore pair. Since these matches are riddled with outliers, we use RANSAC to get a robust estimate. I used 2000 runs, taking 9 random matches each time to get the estimate for F_matrix. I used a threshold of 0.005 to decide which matches are inliers and which ones are not.
The F_matrix I obtained is :-
0.0000 -0.0000 0.0379
0.0000 0.0000 -0.0367
-0.0377 0.0332 -0.9973
This gave me just 165 inliers matches. But using fundamental matrix removed some really bad matches that the SIFT wrapper gave. Below, I show the results for Mount Rushmore pair and also the Gaudi pair.
The results for the Guadi pair are not that good. Normalization makes it much better.
Extra credit
For graduate extra credit, we had to implement normalization of image co-ordinates which helps in getting better estimates for fundamental matrix. I used the techniques suggested on project page. For scaling, I just used inverse of absolute maximum value for each image separately. Finally, I adjusted the resulting estimate to account for the transformations. Normalization gave a huge boost to my results. I was able to capture almost all of the correct matches. Below I show the results for Mount Rushmore pair and the Gaudi pair.
The results for the Guadi pair are much better now.
