CS7322: Computer Vision II: Final Project Proposal

Adding graphical objects to video sequences

Antonio Haro

The Idea:

I propose to take the Kronos head tracking system and add the capability of mapping a graphical model (a hat) to the head. That is, a graphical hat will be mapped to a person's head that will scale, rotate, translate, etc. as the person's head does so.
 

The Domain and the Scope:

Data will initially be from video sequences created by me, although they could very well come from a movie at some point. (If the system is robust enough, it should not matter.)

This is an interesting problem to work on because the motion information that will be gathered by the system will be used to augment the scene, in addition to trying to understand what is going on in the scene. Being able to do this kind of thing is important in doing modeling/animation. This kind of information can also be used to create better special effects systems as current systems work on a frame by frame basis, which seems tedious and, most likely, unnecessary.
 

Approach:

Certain already available software components will be augmented to allow for the type of usage/functionality described. The systems are based on optical flow, which should be good enough for this application.
 

Proposed Time-line:

I do not have a particular timeline in mind, but I do have certain goals/mileposts in mind:

• Get the system working on a sequence; output the affine transformation parameters to stdout
• Find/create a hat model
• Add this model to a sequence
• Make the model stay on the person's head
• Transform the model as it 'sits' on the person's head such that it looks like it really is on their head
• Perform some type of model rendering to make the model look realistic (maybe take scene lighting levels into account, etc.)