Video Mesh: Three-Dimensional Video Editing
The video mesh data structure represents the structural information in an input video as a set of deforming texture mapped triangles augmented with mattes. The mesh has a topology that resembles a "paper cutout." This representation enables a number of special effects applications such as depth of field manipulation, object insertion, and change of 3D viewpoint.
Visit http://groups.csail.mit.edu/graphics/videomesh/ for more info and the additional files mentioned in the video.