Video Textures

Video Textures are derived from video by changing the order in which the recorded frames are played. We play frames out of the original order only at places where it is unnoticeable for the viewer, thus using the finite duration input clip to generate a smoothly playing infinite video.

We have extended this concept to video sprites. Instead of playing back whole frames, we extract images of a character from video, and concatenate these sprites to form new animations. Video sprites are similar in concept to old-fashioned computer game sprites, but instead of a few dozen hand-drawn views of the character, we use up to 30000 video images. We have explored several discrete optimization techniques to find a good sequence of frames, that at the same time looks visually smooth and shows the desired action.

If you want to know more about video textures, look at the web pages of our publications:

Arno Schödl, Irfan Essa. Controlled Animation of Video Sprites. To Appear in Proceedings of the First ACM Symposium on Computer Animation (held in Conjunction with ACM SIGGRAPH 2002), San Antonio, TX, USA, July 2002..
Arno Schödl and Irfan Essa. Machine Learning for Video-Based Rendering. In Todd K. Leen, Thomas G. Dietterich, and Volker Tresp, editors, Advances in Neural Information Processing Systems, volume 13, pages 1002-1008. MIT Press, USA, 2001.
Arno Schödl, Richard Szeliski, David H. Salesin, and Irfan Essa. Video texturesProceedings of SIGGRAPH 2000, pages 489-498, July 2000.

Video Texture Highlights

Fish Tank. This is the final example from the SIGGRAPH 2000 presenation. It is a complete fish tank, populated with artificial fish sprites. The tank includes two sets of bubbles, two independently swaying plants, and a small number of independently moving fish.
Candle flame. The classic: Our first video texture. The red curves show the possible transitions from one frame in the original video clip to another.

Video Textures for Patriotism

The video-textured continuously waving Old Glory was shown on the surtitle monitor during the breaks of  the October 11th and 13th performances of Charles Gounod's opera "Faust" in the New Orleans Opera.

Other Video Texture links

Video Rewrite: The previous work most similar to video textures.

Xiaoli Zhu has done a class project about video textures at SUNY in Fall 2000.