NIST Advanced Technology Program
Return to ATP Home Page
ATP Historical Award Statistics Business Reporting System Surveys EAO Economic Studies and Survey Results ATP Factsheets EAO Home Page
NIST SP 950-2 Status Report - Number 2

Performance of 50 Completed ATP Projects

Status Report - Number 2
NIST SP 950-2

Chapter 5 - Information Technology

Mathematical Technologies Inc. (MTI)
Mathematical Technology
to Restore or Enhance Movies

Many old movies are extremely valuable. If they were made for entertainment, reviving them for current showing can earn sizable profits in addition to providing viewing pleasure to consumers. The film archives of some movie studios, in fact, are worth hundreds of millions of dollars. Documentary movies with footage of important people, industrial processes, and current or historical events have great value, too, for educational and archival purposes.

COMPOSITE PERFORMANCE SCORE
(Based on a four star rating.)
star    star

Old Movies: a Resource Too Valuable to Waste
Movies mean reels of film. Commercial moviemaking uses a master film from which others are copied. Film is a physical thing that can be damaged, soiled, or broken like any other object. But unlike a scratch on a single car, a scratch or other artifact on an old movie master can affect the film’s usefulness to viewing audiences and the fortunes of the company that owns it. If the master film is marred, each copy will also be marred. Even if the master film is converted to digital form for making video copies, the artifacts will persist. Everything on the old film, trash and all, is converted to electronic data that go onto the video copy.


A frame from the movie Amacord, shown first with several areas that are damaged, and then shown after digital restoration automatically re-moved the damaged spots and replaced them with the original images.
A frame from the movie Amacord, shown first with several areas that are damaged, and then shown after digital restoration automatically re-moved the damaged spots and replaced them with the original images.

Another difficulty with movies and other videos is the existence of several formats. It would be useful for film companies to be able to change films from one format to another so that current films could be easily converted to video, and older films could be made to fit today’s video and film equipment. Format has to do with the technicalities of converting movies to digitized videos that can be shown on TV. One format problem involves resolution. The U.S. standard for TV is 525 scan lines and 60 hertz (Hz) — the frame rate. The European standard is 625 scan lines and 50 Hz. High-definition TV will have a different pair of numbers.

A second format problem concerns how to preserve the natural speed of motion depicted in a film when translating, for example, from a format that requires a speed of 24 frames per second to one that calls for 30. Because of the need to compensate for these differences in resolution and film speed, translation from one format to another is not a trivial process.

A Mathematical Approach to Repairing and Converting Films
This ATP project with Mathematical Technologies Inc. (MTI), a small company formed in 1981, has solved many of the problems of reformatting and removing defects from films. MTI specializes in bringing mathematical theory to commercial applications via new programming technologies, and its defect-removal research during the ATP project was particularly successful. Researchers developed mathematical algorithms to create data for filling in damaged areas of the digitized versions of movie-frame images, a process that essentially restores the images to their original quality. Texture matching is an important problem that had not been anticipated but had to be solved in order to repair severe, wide scratches, and other defects involving substantial amounts of missing data.

The MTI technology can remove tears, splotches, scratches, dust motes, liquid-spill marks and other unwanted visual defects from movies. Methods for using the new technology, as well as a specialized user-friendly screen display from MTI, have been integrated into post-production processing at a number of facilities in Hollywood and elsewhere.

MTI researchers succeeded in developing some components needed for format conversion. Work on other components is still experimental. The researchers thoroughly investigated motion compensation (which concerns the way moving objects are detected in a movie) and determined how to make adjustments for motion so that the new technology does not create new artifacts. Specifically, they estimated the frame-to-frame motion of objects and developed technology for the rapid calculation of the most significant motions. This technology is critical both for restoration of damaged images and for translating between film and video recording standards.

New Products and Services for Film and Video Industries
Commercialization is under way. Near the end of the ATP project, MTI established a division called Digital Restoration Services that sells movie-restoration software and services, and the company is developing other products that would use the ATP-funded technology, too. MTI has invested heavily in the development of new software for film and video post-production since the ATP project was completed. A new state-of-the-art algorithm for converting from ordinary video resolution to high-definition-TV resolution was demonstrated at the National Association of Broadcasters trade show in April 1998.

The new MTI offerings face competition from several other products — virtually all of them from abroad. Competitor products, however, tend to focus on the artistic end of the of the movie restoration business rather than on the technical end. MTI’s products focus on the technical end, and the company reports it is currently the only one to provide such software technology for automated restoration.

MTI initially intended to develop applications for motion-compensated reformatting and standards conversion, as well as restoration. After the ATP project began, the company decided to focus almost exclusively on restoration, based on a reassessment of the market for conversion software and services. It planned to offer film-restoration software running at commercially viable speeds (perhaps three to four times slower than real-time) on graphics workstations or high-performance personal computers costing well under $100,000. MTI succeeded, and it is offering the software for use with contemporary and archived movies. In addition, the company says it is about two years ahead of where it would have been without the ATP funds.

Restored Ten Commandments
Viewers of the many films, both contemporary and archival, restored with MTI’s technology have benefited. When Cecil B. DeMille’s Ten Commandments was restored with the ATP-funded technology and rereleased, a commentator on the television program “Entertainment Tonight” reported that “the difference between the original and this new vibrant version is a revelation. . . . Digital technology is the modern miracle that’s made it possible.”

As MTI’s mathematical algorithm technology is applied to more films, more viewers will benefit. Further benefits will emerge if the technology is used in other areas. It has potential applications, for example, in forward-looking infrared imagery, which is used by the military to detect objects at night, and may also be useful in medical imaging procedures such as ultrasound and fluoroscopy. Additional benefits will materialize if the technology for standards conversion is completed.

The film-restoration technology already commercialized promises spillover economic benefits to the viewing public and to owners of films with defects. Many films of historical interest, once they are restored with the new technology, will be available to viewers. The number of viewers will grow over the years as the restored or enhanced films are shown again and again, so spillover benefits will grow, as well. If the reformatting technology is completed and commercialized, additional benefits will accrue.

Project Highlights

PROJECT:
To develop generic software technology that can repair, enhance, or reformat movie and video sequences, enabling the restoration of damaged movies, enhancement of military images, and conversion between digital image formats.
Duration: 5/1/1993 — 8/31/1995
ATP Number: 92-01-0053

FUNDING (in thousands):

ATP
$989
86%
Company
136
12%
Total
$1,125

ACCOMPLISHMENTS:
MTI developed technology to remove artifacts (unwanted defects) from movies, whether archived or newly created. It made progress in developing the reformatting technology, but this work is still experimental. The company also:

  • formed MTI Digital Restoration Services early in 1996, a division now actively marketing software and restoration services in video post-production;
  • received jacket-cover credit for restoration work on the laser disc version of the 1958 film A Night to Remember, about the sinking of the Titanic;
  • participated, via MTI Digital Restoration Services, in acclaimed restorations of recent rereleases of Federico Fellini’s Amarcord and Cecil B. DeMille’s Ten Commandments; and
  • had its software used in the perfection or restoration of hundreds of new and old films for new video releases since 1995.

COMMERCIALIZATION STATUS:
Commercialization is in progress. Film-restoration software and services are being sold by MTI Digital Restoration Services, and MTI has other products under development. Viewers of many films, both current and archival, are benefiting from what they do not see: defects removed by the technology.

OUTLOOK:
In the restoration of old movies and the polishing of new releases, there are excellent expectations for the mathematical algorithm technology. It also has potential applications in forward-looking infrared imagery and in medical imaging areas like ultrasound and fluoroscopy. Completion of the technology for conversion between formats will widen applications further, particularly in high-definition TV.

Composite Performance Score:
star    star

COMPANY:
Mathematical Technologies Inc. (MTI)
1 Richmond Square
Providence, RI 02906-5139

Contact: Donald E. McClure
Phone: (401) 831-1315
Number of employees: 4 at project start, 6 at the end of 1997

Return to Table of Contents or go to next section.

Date created: April 2002
Last updated: April 12, 2005

Return to ATP Home Page

ATP website comments: webmaster-atp@nist.gov  / Technical ATP inquiries: InfoCoord.ATP@nist.gov

NIST is an agency of the U.S. Commerce Department
Privacy policy / Security Notice / Accessibility Statement / Disclaimer / Freedom of Information Act (FOIA) /
No Fear Act Policy / NIST Information Quallity Standards / ExpectMore.gov (performance of federal programs)

Return to NIST Home Page
Return to ATP Home Page Return to NIST Home Page Go to the NIST Home Page