Project 3 / Camera Calibration and Fundamental Matrix Estimation with RANSAC

Example of a right floating element.

The goal of this project was to apply concepts of Cameras and Scene Geometry. It was broken into 3 parts.

  1. Estimate the Projection Matrix and the Camera Center
  2. Estimate the Fundamental Matrix
  3. Estimate the Fundamental Matrix using unreliable SIFT matches using RANSAC

Part 1: Estimate the Projection Matrix and Camera Center

Part 1 consisted of first finding the projection matrix and then calculating the camera center based on that matrix. To solve for the Projection Matrix, M, calculating A, the matrix of unknown camera parameters from the 2D and 3D coordinates. This is done by putting the image and world coordinations into the form: [X1 Y1 Z1 1 0 0 0 0 -u1X1 -u1Y1 -u1Z1; 0 0 0 0 X1 Y1 Z1 1 -v1X1 -v1Y1 -v1Z1; . . . Xn Yn Zn 1 0 0 0 0 -unXn -unYn -unZn; 0 0 0 0 Xn Yn Zn 1 -vnX1 -vnY -vnZn ] Where X, Y, and Z are the World (3D) coordinates, and u and v and the image (2D) coordinates.

Example of code with highlighting

The javascript in the highlighting folder is configured to do syntax highlighting in code blocks such as the one below.


	%get u and v from image coordinates
   r = Points_2D(k, :);
   u = r(1, 1);
   v = r(1, 2);
    
   %get X, Y, Z from the dimensions of the world coordinates
   s = Points_3D(k,:);
   X = s(1, 1);
   Y = s(1, 2);
   Z = s(1, 3);
   
   %odd rows - u values
   A(i, :) = [X Y Z 1 0 0 0 0 -u.*X -u.*Y -u.*Z -u];
   %even rows - v values
   A(i + 1, :) = [0 0 0 0 X Y Z 1 -v.*X -v.*Y -v.*Z -v];

Results

The projection matrix is: -0.0069 0.0040 0.0013 0.8267 -0.0015 -0.0010 0.0073 0.5625 -0.0000 -0.0000 0.0000 0.0034 The total residual is: <15.5450> The estimated location of camera is: <303.1000, 307.1843, 30.4217>

Part 2: Estimate the Fundamental Matrix

Part 2 I solved using the 8 Point Algorithm. Like in part one, it involed taking corresponding points and manipulating them into a matrix of form A(k, :) = [ua*ub ua*vb ua va*ub va*vb va ub vb 1]; Where k is a number from 1 to the size of the image.

Example of getting the corresponding points


	%get u and v from the current row of image a
   a = Points_a(k, :);
   ua = a(1, 1);
   va = a(1, 2);
   
   %get u and v from current row of image b
   b = Points_b(k, :);
   ub = b(1, 1);
   vb = b(1, 2);

Results

Part 3 : Estimate the Fundamental Matrix using unreliable SIFT matches using RANSAC

After finding the matches using RANSAC I sampled them randomly to produce the set of inliers for a and b for the best fundamental matrix. I'm sorry this report is so bad I ran myself out of time :(

Results