Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC

The goal of this project is to get an introduction to camera and scene geometry. Specifically we will estimate the camera projection matrix, which maps 3D world coordinates to image coordinates, as well as the fundamental matrix, which relates points in one scene to epipolar lines in another. The camera projection matrix and the fundamental matrix can each be estimated using point correspondences. To estimate the projection matrix (camera calibration), the input is corresponding 3d and 2d points. To estimate the fundamental matrix the input is corresponding 2d points across two images. The project is split into 3 parts

  1. Camera Center and Projection Matrix Estimation
  2. Fundamental Matrix Estimation
  3. Fundamental Matrix with RANSAC

Camera Projection Matrix

Projection Matrix Estimate

The total residual is: 0.0445

The estimated location of camera is: <-1.5127, -2.3517, 0.2826>

Fundamental Matrix Estimate

Using simple points without normalisation

Epipolar Lines through Image Part a

Epipolar Lines through Image Part b

Using points with normalisation

Fundamental Matrix with RANSAC

RANSAC is run for around 1000 iterations using 8-point algorithm 

%example code
while n_iter<1000
    idx = randperm(size(matches_a,1));
    Best_Fmatrix = estimate_fundamental_matrix(matches_a(idx(1:8),:), matches_b(idx(1:8),:));
    val_thresh=0;
    no_inliers = 0;
    val_matches = [];
    ind_matches = [];
    for i=1:size(matches_a,1)
    	val_thresh = HomoPoints_a(i,:)*Best_Fmatrix*HomoPoints_b(i,:)'; 
    	if(abs(val_thresh)<0.1)
        	no_inliers = no_inliers + 1;
        	val_matches = [val_matches val_thresh];
        	ind_matches = [ind_matches i];
 
	    end
    end
    
    if(no_inliers>best_sofar_inliers)
        
        [~,ind] = sort(val_matches,'ascend');
        if(size(ind_matches,2)<30)
        	indices = ind_matches(ind(:));
        else
            indices = ind_matches(ind(1:30));
        end
    end
end

Results in a table

Mount Rushmore inliner points and epipolar lines using normal points without normalisation

The table above shows RANSAC output for two sets of images. The outliers are rejected based on a threshold in 8 point algorithm. We notice marked improvement in distinctive keypoint detection

Extra Credits:

Tagging images with normalised coordinates for fundamental matrix estimation

Mount Rushmore feature points and epipolar lines using normal points without normalisation
Notre Dame feature points and epipolar lines using normal points without normalisation