CS4480: Homework #2: Adding CG to the Real World

 

Do both parts in groups of 3 people, and submit results by noon on each deadline.

 

Part I: Calibration (due Feb. 12th)

1)      Groups check out camera + tripod AFTER printing and signing this form: www.cc.gatech.edu/classes/AY2001/cs4480_spring/images/checkout2001.pdf

2)      Film calibration pattern & short action scene with pattern in “home” location and static camera

3)      Continue filming while moving camera around (slow spin, walk, etc.) w/o pattern

4)      Digitize footage using DVRaptor s/w and capture cards in computer lab

5)      Extract 12+ interesting frames from calibration into stills (bmp or tif) with numbered filename suffixes (e.g. frame_21.tif)

6)      Pull down a copy of the calibration_tool directory for local use

7)      Follow tutorial to calibrate internal camera parameters using stills (copy results into a text file)

8)      Follow tutorial to calibrate camera extrinsics for image with pattern in home position; save translation and rotation matrices to same text file, and submit it to HW2 directory (like HW1)

 

Part II: Compositing (due Feb. 19th)

This part of the assignment walks you through mapping the results of Part 1 into the camera model for Maya. Your task is follow the directions, and then augment either your “home image” or a sequence shot from that camera position with CG elements of your choice. 75% of the task is the technical aspect of mapping the real-world calibration to a virtual camera, and 25% is your creativity and attention to detail in adding the CG elements. Since you have only one week for this part, focus on quality instead of quantity – we’re happy to see one image done really well vs. a long sequence where things float and the lighting is off. Of course, a well done sequence is better still.

The “guide” for part 2 is available here. Turn in your results in the same place as Part I (in /net/hi21/fx/HW2/your_group_name/)

 

Part III: Optional match-move experimentation (also best turned in with Part II, but since it’s optional, anytime you feel like it).

This part is NOT necessary to complete this assignment, and is purely for experimentation purposes. Do this if you’re eager, if you have time, or if you are thinking of having a moving camera in your shot. MayaLive is a newly available plugin for Maya (Window->Plugins->Mayalive), and has its own documentation (available from the help-menu). Its purpose is to allow the “automatic” tracking of features in an image sequence where the camera was moving. This then produces the equivalent of an extrinsic calibration of the real camera, but does so for each frame of the sequence, and without an explicit calibration pattern. The task: insert CG elements into your video (just like Part II), while using footage shot with a moving camera. Give us feedback on what you thought of MayaLive as a tool, and use this to help you decide to use/not use moving cameras in your final project effect.

 

 

 

Extracted from http://www.visualfx.com/demo.htm:

“

WHAT YOU ALWAYS WANTED TO KNOW BUT CAN NEVER GET ANYONE TO TELL YOU

(a.k.a. what computer graphics supervisors at ILM would like to see on a reel)

 

Show some Live Action/CG Integration. It is rare to see this on a student reel but as it is what we do at ILM, it gives us some understanding of the individual’s understanding of matching realistic lighting in CG as well as compositing. By sticking to an all CG environment (which is much easier to do than the integration we do) it is difficult to determine how your “eye” will create a photo realistic “look”.

“

 

Specifics:

These are temporary groups, so your final project can be done in whatever groups of 3 you choose. Group assignments for HW2 are on the SWIKI.

 

 

Part 1:

1.      Check camera inventory sheet when checking camera out, so you’re not held responsible for things which were already missing. For this project, you should not need to borrow equipment for more than a day, but you have it for few days just in case. You can use the scratch miniDV tape that comes with the bag to start, but since it goes back at check-in time, you should buy one or more of your own. These can be close to $10 if you buy in bulk (feel free to share shopping tips on the SWIKI). When using a blank tape for the first time, record it from start to finish with the lens covered, so it has a consistent time-code. This will prevent problems when dumping to the drive.

2.      Remember to switch to manual focus! You can print a calibration file of your own (make the squares all the same and line them up) and mount it on a perfectly planar surface. You can also use the larger calibration pattern located in the test-space of CoC 259 (middle of the GVU lab – the curtained area). Remember to measure the exact width and height of a square. For example, measure width of 4 boxes (B, W, B, W) and divide total by 4 to get an accurate X-dimension. Record in cm. for later. Origin is in upper left of the pattern pictured here, and I skipped the first and last row of boxes since the printer chopped them short. Move the pattern around so it takes up most of the field of view, and try to explore the space: show different positions and orientations, keeping the origin-corner as the left-top most corner in the image. Without moving the camera, move around in the scene or do something simple – this is just to show that it’s a sequence and not a single frame. Keep it short (30sec or less), and possibly make it relate to the 3D graphics you’ll be adding later. Don’t worry if you don’t yet know where you’ll be adding stuff to the scene (just glare in random directions for now). Snap multiple “home” position images with the pattern appearing bigger or smaller (see #8 below).

 

3.      Keep this under 30 sec. also, and don’t move the camera too drastically. This will be used only if we can obtain some match-move software licenses, and will only be important for the more ambitious groups.

4.      Each camera has a DV cable, which doesn’t look like the RCA, or the S-Video cables in the bag. It connects to the panel on the front of the DVFX PC’s in the States PC cluster. DVRaptor software is installed on all those machines, and you don’t have much need for the Navi software right now, unless you already recorded so much data that you want to easily index to the tape. Turn the camera to VCR mode before starting the software, and you’ll be able to control the “deck” using the on-screen controls. Video plays back on the LCD of the camera, and may not show up in the Raptor software’s preview window – that’s ok. I usually use the batch capture utility (in one of the menu options). It lets you mark the start (in) and stop (out) points of each part of the tape that I want to dump to a separate file. Once marked and given a file-destination, “ADD” each desired clip to the batch-capture list. You can save and update this list if things break down for some reason. If you get an error that the “in point” can’t be found, you may need to reboot the PC (so save that list to a safe place). Dump to E since it’s fast and hopefully not very fragmented, and either capture the data to smaller clips (calib clip, action clip, moving camera clip, etc.), or segment a larger file in Premiere so that you don’t have to burn drive-space by saving video which you won’t use. That said, don’t trash data you “might” want later. CD-burners should be working now (hint-hint).

5.      Premiere is ok for this, just export a frame instead of a movie. Don’t dare change the image size or use lossy compression! You will want to copy & rename the extracted images so that the last one still has a number less than 100. It hurts the calibration software to have to run through 2000 non-images to get to frame_2001. Just count from 1, and everything will be fine.

6.      It seems to run faster when using a local copy: \net\cpl\NT\OpenCV_alpha3\calib\TOOLBOX_calib\

7.      Follow the “First Calibration Example” on this page: \net\cpl\NT\cvl\CVL_html\appPage\calib_doc\index.html. The calib_gui program’s interface is pictured below. Basically, follow the onscreen instructions/questions and the hints that appear in the images to perform all the tasks leading to “Exit” (starting in the top left and going across). I’ll put up a trace of the program when I used it. Reprojection error MUST be less than a pixel, and ought to be around 0.1 pixels if you throw out the really bad images and keep around 8 good ones. Reprojection: the corner-finder puts little crosses when it finds something, and the corresponding point from the 3D model of your calib pattern is rendered using the calculated calibration as a circle. These should line up.                                                                                       

8.      What can I say - it’s the box in the lower left. If you add/suppress it into the list of active images, you can visualize (and sanity-check) where it was relative to the other calibration images. If it’s way off (because, say, it appeared too small in the image), you’ll want to use a different image as the “home”.