Movies mean reels of film. Commercial movie-making uses a master film from which others are copied. Film is a physical thing that can be damaged, soiled or broken like any other object. But unlike a scratch on a single car, a scratch or other artifact on an old movie master can affect the film's usefulness to viewing audiences and the fortunes of the company that owns it. If the master film is marred, each copy will also be marred. Even if the master film is converted to digital form for making video copies, the artifacts will persist. Everything on the old film, trash and all, is converted to electronic data that go onto the video copy.

Another difficulty with movies and other videos is the existence of several formats. It would be useful for film companies to be able to change films from one format to another so that current films could be easily converted to video, and older films could be made to fit today's video and film equipment. Format has to do with the technicalities of converting movies to digitized videos that can be shown on TV. One format problem involves resolution. The U.S. standard for TV is 525 scan lines and 60 hertz (Hz) - the frame rate. The European standard is 625 scan lines and 50 Hz. High-definition TV will have a different pair of numbers.

A second format problem concerns how to preserve the natural speed of motion depicted in a film when translating, for example, from a format that requires a speed of 24 frames per second to one that calls for 30. Because of the need to compensate for these differences in resolution and film speed, translation from one format to another is not a trivial process.

A Mathematical Approach to Repairing and Converting Films

This ATP project with Mathematical Technologies Inc. (MTI), a small company formed in 1981, has solved many of the problems of reformatting and removing defects from films. MTI specializes in bringing mathematical theory to commercial applications via new programming technologies, and its defect-removal research during the ATP project was particularly successful. Researchers developed mathematical algorithms to create data for filling in damaged areas of the digitized versions of movie-frame images, a process that essentially restores the images to their original quality. Texture matching is an important problem that had not been anticipated but had to be solved in order to repair severe, wide scratches and other defects involving substantial amounts of missing data.

The MTI technology can remove tears, splotches, scratches, dust motes, liquid-spill marks and other unwanted visual defects from movies. Methods for using the new technology, as well as a specialized user-friendly screen display from MTI, have been integrated into post-production processing at a number of facilities in Hollywood and elsewhere.

MTI researchers succeeded in developing some components needed for format conversion. Work on other components is still experimental. The researchers thoroughly investigated motion compensation (which concerns the way moving objects are detected in a movie) and determined how to make adjustments for motion so that the new technology does not create new artifacts. Specifically, they estimated the frame-to-frame motion of objects and developed technology for the rapid calculation of the most significant motions. This technology is critical both for restoration of damaged images and for translating between film and video recording standards.

New Products and Services for Film and Video Industries

Commercialization is under way. Near the end of the ATP project, MTI established a division called Digital Restoration Services that sells movie-restoration software and services, and the company is developing other products that would use the ATP-funded technology, too. MTI has invested heavily in the development of new software for film and video post-production since the ATP project was completed. A new state-of-the-art algorithm for converting from ordinary video resolution to high-definition-TV resolution was demonstrated at the National Association of Broadcasters trade show in April 1998.

The new MTI offerings face competition from several other products - virtually all of them from abroad. Competitor products, however, tend to focus on the "artistic" end of the of the movie restoration business rather than on the "technical" end. MTI's products focus on the technical end, and the company reports it is currently the only one to provide such software technology for automated restoration.

MTI initially intended to develop applications for motion-compensated reformatting and standards conversion, as well as restoration. After the ATP project began, the company decided to focus almost exclusively on restoration, based on a reassessment of the market for conversion software and services. It planned to offer film-restoration software running at commercially viable speeds (perhaps three to four times slower than real-time) on graphics workstations or high-performance personal computers costing well under $100,000. MTI succeeded, and it is offering the software for use with contemporary and archived movies. In addition, the company says it is about two years ahead of where it would have been without the ATP funds.

Restored "Ten Commandments"

Viewers of the many films, both contemporary and archival, restored with MTI's technology have benefited. When Cecil B. DeMille's "Ten Commandments" was restored with the ATP-funded technology and re-released, a commentator on the television program "Entertainment Tonight" reported that "the difference between the original and this new vibrant version is a revelation. . . . Digital technology is the modern miracle that's made it possible."

As MTI's mathematical algorithm technology is applied to more films, more viewers will benefit. Further benefits will emerge if the technology is used in other areas. It has potential applications, for example, in forward-looking infrared imagery, which is used by the military to detect objects at night, and may also be useful in medical imaging procedures such as ultrasound and fluoroscopy. Additional benefits will materialize if the technology for standards conversion is completed.

The film-restoration technology already commercialized promises spillover economic benefits to the viewing public and to owners of films with defects. Many films of historical interest, once they are restored with the new technology, will be available to viewers. The number of viewers will grow over the years as the restored or enhanced films are shown again and again, so spillover benefits will grow, as well. If the reformatting technology is completed and commercialized, additional benefits will accrue.

Go to other sections of Chapter 7: INFORMATION, COMPUTERS, AND COMMUNICATIONS
	Computer Recognition of Natural Handwriting
	Chinese Character-Recognition Methods for Computer Data Entry
	Three-Dimensional Anatomy of Human Body, With Animation, for Medical Training
	Packing More Data Into Optical Data-Storage Disks
	Mathematical Technology to Restore or Enhance Movies
	A User-Friendly Programmer's Tool for Writing Parallel-Processing Software