This thesis describes a project to investigate the generation of novel special film effects using spatio-temporal volume processing. An algorithm for generating special effects was developed and implemented within a special toolkit, called S-TFX. This toolkit was used to successfully generate an extensive collection of novel special effects. Some of these effects include taking an arbitrary cross-section of the video sequence, forcing a particular object in the scene to stop moving and changing the direction that an object moves in without disturbing other objects in the scene.
The algorithm constructs a spatio-temporal volume from an input video sequence so that the video can be visualised as a single volume rather than a sequence of individual frames. The algorithm extracts the frames from the video and stacks them in order from front to back to create the spatio-temporal volume with time increasing along the Z-axis. The algorithm then defines two surfaces within the volume, the front surface and the back surface. The two surfaces can be defined in several different ways, parametrically, interactively or using volume segmentation. Using these surfaces the algorithm generates a set of N-2 intermediate surfaces between the front and back using interpolation or warping. This set of surfaces, including the front and back, are then used to sample the spatio-temporal volume to generate N new frames. The new frames can then be viewed as a new video sequence.
All the effects were discussed and explained on the basis of the interactions between the spatio-temporal objects in the scene and the sampling planes used to generate the effects. It was concluded that this work should be extended to consider both the scientific and artistic aspects of generating special film effects using spatio-temporal volume processing.
I wish to thank BBC Wildlife for supplying the original wildlife footage used in this project. Thanks also to Mario Kleiner at the Max Planck Institute for Biological Cybernetics Tübingen for supplying some high-resolution multi-view video sequences. I also wish to thanks Dr Stavri Nikolov and Dr Alan Chalmers for their help and advice during this project. I would also like to acknowledge the creators of Vtk at Kitware for developing an indispensable tool.