|
Part 1 (due 2/27) |
Part 2 (due 3/13) | Extra Credit
(due 3/13) | Hints
PART 1: Retargeting, due 2/27
Motivation:
One of the big topics in motion capture is retargetting of
motion capture data. For example, let's say we have motion data of someone
kicking a target 3 feet off the ground. Now what if that target is 5 feet off
the ground? How do we retarget the motion data to accomplish that goal without
having to recapture the motion?
In this half of the assignment, you will be given several clips of motion
capture data and will need to implement a retargeting system to accomplish new
goals and yet satisfy the imposed constraints.
The motion capture data is in /net/cpl/data/mocap/AnimClass. Also included will
be instructions on how each motion should be retargeted.
To view the data simply run Maya. Open the script editor (Window | General
Editors | Script Editor) and do a File | Open Script and point it to the
bvhimport.mel script located in the AnimClass directory. Once the script is
loaded into the window, press Ctrl+Enter to execute the script. It will open up
a window, and you should point it to some mocap data.
To ensure smooth playback, go to Window | Settings/Preferences | Preferences.
Click the Settings Category on the left-hand side and make sure that Time is set
to NTSC (60 fps). Click on the Timeline category and make sure that Playback
Speed is set to Half (15 fps).
Background material:
-
Gleicher. "Retargetting Motion to New Characters. Proceedings of
SIGGRAPH 98". In Computer Graphics Annual Conferance Series. 1998. [PDF]
-
Witkin , Popovic, Motion warping, SIGGRAPH
1995 [ACM|PDF]
-
D. Tolani, A. Goswami, and N. Badler: "Real-time
inverse kinematics techniques for anthropomorphic limbs." Graphical Models 62
(5), Sept. 2000, pp. 353-388. [PDF]
Take a look at some motion data. Click on a particular joint and
periodically hit stop and start. Notice how the joint angles (the rotation
values for the joint) change over time. Also note how the translation
values do NOT change. In order to define a pose for a 3D character at time t,
we only need to specify the translation for the root joint (in this case, the
hips), and traverse down the skeletal hierarchy, adjusting joint angles.
If you read through the BVH file you will actually see the BVH hierarchy defined
in the first half of the file. The second half describes the joint angle data.
For more information on the BVH format please see
http://www.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/BVH.html
* biovision.com is NO LONGER Biovision's homepage!
Let's look at some IK examples. Open the scene AnimClass/ikexample.mb.
There are two two-bone, three-joint skeletons. On the left is a skeleton with IK
handles, on the right is a non-IK skeleton. Click on the end effector (the
bottom-most pointing joints) and using the translate tool, move them about.
Notice the difference? The skeleton with the IK handles behaves like a left arm
while the skeleton without the IK handles behaves like a, well, like a skeleton
without IK handles. Read through the Maya tutorial on IK setup (Help | Library |
Character setup | scroll down to Skeletons) for more information on IK.
Why all this talk about IK? Well, besides the fact that it's a very important
topic in computer animation, it could really help you with this project. Read
on.
OK, let's get to the bottom of this part of the assignment. What you will need
to do is look at the various clips of mocap data and READ the associated info
file. The info file will describe all the various constraints and parameters
associated with the motion as well as how it should be retargetted. You may, if
you choose, retarget the motion in some other way BUT PLEASE LET US KNOW BEFORE
YOU DO SO.
Read through the papers mentioned above. You will need to implement a retargeter
that will, given some motion capture data and a set of constraints (e.g., hand
must be at position XYZ at time T) smoothly retarget the original motion to the
new motion. Your retargetter must not violate any normal human joint
constraints - e.g., no oddly-bending knees, no heads twisting around in circles,
etc. You will need to pick 5 motions to retarget.
What to turn in:
You will need to turn in 5 sets of retargetted BVH data as well
as a writeup
describing your retargetting system. You will be graded on how well your
motions follow the given constraints. Samples of code does not have to be
submitted, but may be asked for.
Submit in
/net/hi21/ca2002/PS2/Part1/<username> (make a directory).
(Include a README.txt file to say which files are which)
Part 2: Motion blending and warping,
due 3/13
Now that you have successfully re-targetted the motion data to
fit new
constraints, you will have a chance to show off your creative skills by
blending and warping the motion clips together. For example, if you have a
walking clip and a running clip, can you interpolate the end of the walk
with the beginning of the run in order to create a smooth transition from a
walking clip to a running clip? We will consolidate all the motions from
part 1 together into a directory, and you can pick and choose from the
directory the motions you would like to blend together.
Background materials:
-
Bodenheimer, B., Rose, C., Rosenthal, S., and Pella, J.,
"The
Process of Motion Capture: Dealing with the Data'', Computer Animation
and Simulation '97, Eurographics Animation Workshop, Sept. 1997, D.
Thalmann and M. van de Panne, eds., Springer-Verlag, Wien, pp. 3-18. [PDF|PS|Bodenheimer's
Web Page]
-
Arikan and Forsyth, "Interactive Motion Generation From
Examples", ACM Transactions on Graphics (TOG) , Proceedings of the 29th
annual conference on Computer graphics and interactive techniques, 21(3),
(SIGGRAPH 2002) July 2002 [PDF]
-
Rose, C., Cohen, M., and Bodenheimer, B.,
``Verbs
and Adverbs: Multidimensional Motion Interpolation'', IEEE Computer
Graphics and Applications, v. 18, no. 5, Sept. 1998, pp. 32-40. [PDF|PS|Bodenheimer's
Web Page]
-
Kovar , Gleicher , Pighin, "Motion Graphs" ACM
Transactions on Graphics (TOG) , Proceedings of the 29th annual conference on
Computer graphics and interactive techniques, 21(3), (SIGGRAPH 2002) July
2002 [PDF]
In particular, pay attention to the last two on how they do their transition
selection. String together several clips of data and see if you can smoothly
blend between them.
You are required to write a GENERIC interpolator that can
combine motion clips together. You must combine AT LEAST 2 sets of 3 motions
together, 2 sets of 4 motions together, and 1 set of "your choice", resulting in
5 new clips of data. For example, if there are 26 clips of motion, you
could turn in
clip 1 -> clip 6 -> clip 14
clip 9 -> clip 22 -> clip 25
clip 14-> clip 10 -> clip 2 -> clip 1
clip 17-> clip 26 -> clip 17 -> clip 4
clip 3 -> clip 5 -> clip 22 -> clip 9 -> clip 15 -> clip 20 -> clip 6
In the AnimClass directory is a 3D model of a stick figure, our
beloved Woody (woody.mb). Load your newly-created clip into Maya and see if you
can animate Woody with your mocap data. You will need to scale Woody to fit your
skeleton and you will need to put him into the initial pose of your mocap data.
Finally, you will need to skin the model to the skeleton in order for the model
to be driven by the skeleton. See the Maya tutorial on Character Setup for more
information.
What to turn in:
You must turn in 5 blended clips of data, along with a writeup indicating which
clips you blended together, and a description of your algorithm. Samples of code
does not have to be submitted, but may be asked for. You will be graded on how
well your motions are interpolated.
Submit in
/net/hi21/ca2002/PS2/Part2/<username> (make a directory).
(Include a README.txt file to say which files are which)
Implement footskate cleanup. A big problem in motion capture
is the fact that the feet often slide. For example, look at the
footskate.avi video in the AnimClass directory. Note how the feet "slide"
along the ground as they plant. The movie is compressed with Microsoft's
MPEG 4 codec, so most windows machines should be able to view it.
You will implement a version of footskate cleanup to, well, clean up the
footskate!
Background materials:
Read through the paper. Are variable-length bones the solution? Think of
other solutions, and either implement them or this paper's solutions to solve
some of the footskate issues you may see in the data from part I. You
should turn in at least three examples of "cleaned" data. Your writeup
should include your algorithm and the clips you cleaned. Samples of code do
not have to be submitted, but may be asked for.
Submit in
/net/hi21/ca2002/PS2/EC/<username> (make a directory).
(Include a README.txt file to say which files are which)
HINTS:
I'M LOST! WHERE DO I START?
Code:
Startup code (Windows-based, email spark@cc for other platforms) is located
in the AnimClass/code directory. Included are the header files and a .lib
file (for Visual C++).
Start a Visual C++ Project and include the headers and library file in your
project.
The code is what you will need to get started, and does reading and writing
of BVH data. For example, you can do something like:
int main( int argc, char *argv[] )
{
BVH myBVH;
myBVH.read( argv[1] );
// read in command line argument as bvh file
// munge some data
myBVH.write( "somefile.bvh" );
}
The data is stored in a structure called (of all things) "data", while the
joint names and offsets are stored in an array. Look through the provided
.H files for details. To access the joint angle data, you would do
something like
myBVH.data[frame#].body_parts[bodypart#].data[0] = 23.56;
note that the ending data element is an array of size 3 holding the XYZ
values, e.g., data[0] = X, data[1] = Y, data[2] = Z. Again, read through
the .H files for more information.
Mail spark@cc with any questions.
Retargetting:
How do you figure out how to retarget the data? OK, here's the naive
approach, which can actually work quite well (although it doesn't always!).
Assume we're looking at the PickingUpABox.bvh data file. Load the data
into Maya, and you can see pretty much exactly where the subject leans
over and picks up the box. Hit pause at the point where the subject
actually bends over. To aid with the retargetting visualization you can
do two things:
-
add a box to the scene
-
add IK handles to the skeleton (see the Maya tutorial on IK)
Assume the box is a few units higher than it really is and reposition the
skeleton so that it is properly positioned to pick up the newly-placed box.
Note: the IK handles can make it easier to position the skeleton for this
task, but it is NOT necessary. You can always simply adjust the joint
angles manually using Maya.
Make a note of the new joint angles (located in the channel box to the right
of the Maya window) by clicking on various joints. So, if these are the new
"goal" joint angles, how do you go about interpolating these angles from
their orignal values to their new values, WITHOUT VIOLATING HUMAN JOINT
CONSTRAINTS? That is, make sure the elbow doesn't bend backwards!
Motion blending:
How do you start blending data? Well, read through the papers mentioned
above (the first two in particular for basics) and you should get an idea of
the naive approach. Essentially, the naive approach will find similarities
towards the end of clip 1 and the beginning of clip 2 and do a smooth linear
interpolation between the two. Naturally, a linear interpolation isn't the
best way to do it - can you think of other ways? Cubic splines, perhaps?
|