 |
Performance
of 50 Completed ATP Projects
Status
Report - Number 2
NIST SP 950-2
Executive
Summary
|
| More
efficient use of energy, more efficient medical treatments that cost
less and cause less pain, the capacity to process growing volumes
of data, better vehicles, and improved position in the highly competitive
international electronics market—these are among the many significant
achievements of projects supported by the Advanced Technology Program
(ATP) over its first decade. Results from the first 50 completed projects
are strong for ATP, with estimated benefits far outweighing the entire
cost of ATP to date. |
The First 50 Projects
Policymakers, program administrators, business managers,
and others in this country and abroad have eagerly awaited a comprehensive
look at results from ATP-funded research projects. This report provides
at least partial answers by assessing the first 50 completed projects—approximately
10 percent of the projects funded by the ATP from 1990 through 2000. The
performance metrics show how each of the 50 projects performed in terms
of new technical knowledge created and disseminated, direct commercialization
of new technologies, and overall project effectiveness.
Project Characteristics
The majority of the first 50 completed projects are single-applicant
projects led by small businesses. Although only 16 percent are joint ventures,
84 percent involved collaborative relationships. Nearly half had close
R&D ties with universities, and more than half formed collaborative
arrangements with others to pursue commercialization.
ATP’s designated
technology area Electronics/ Computer Hardware/Communications comprised
the largest group of projects, followed by Manufacturing, Biotechnology
and Advanced Materials/Chemicals, and, last, Information Technology.
The ATP spent an average
of $1.5 million per single-applicant project and an average of $4.9 million
per joint-venture project. Across the 50 projects, the average total cost
(ATP plus industry) per project was $4.2 million, and the median project
length was three years. Together, ATP and industry spent a total of $208
million on the 50 projects, with ATP and industry sharing total research
costs
roughly equally.
ATP’s Mission
and Operations
The National Institute of Standards and Technology (NIST),
a part of the Department of Commerce’s Technology Administration,
administers ATP. The ATP and industry share research costs for projects
characterized by ambitious scientific and technological goals, and by
a strong potential to improve the competitiveness of U.S. businesses and
offer substantial economic return to the United States. The ATP seeks
to accelerate the development and application of enabling technologies
whose benefits will extend well beyond the direct benefits to the ATP
award recipients. The focus is on collaborative, multidisciplinary research
and on civilian technologies that appear likely to be commercialized in
the marketplace with private sector funding once the high technical risks
are reduced. The projects funded are selected in rigorous competitions
on the basis of their technical and economic merit, as determined by peer
review.
Since 1990, ATP has
committed funding of $1.6 billion in research costs for 522 projects,
with companies contributing a similar amount in matching funds for the
research, and much more in the post-project periods for follow-on commercialization.
More than 1,000 companies, universities, and nonprofit laboratories lead
the projects or participate as members of research joint ventures. More
than 1,000 additional organizations are involved as subcontractors and
informal collaborators.
Study Scope, Approach,
and Organization
This report comprises one element of ATP’s evaluation
program, providing a systematic and comprehensive look at a large group
of ATP projects, and shedding light on the performance of the program
at large.
At the report’s
core are 50 mini-case studies covering the first-completed projects and
investigating the performance of the projects several years after completion.
Chapter 1, an overview,
provides aggregate descriptive statistics, and then presents aggregate
output statistics—first, for knowledge creation/dissemination, and,
second, for progress toward commercial goals. It then uses all of the
outputs to construct a composite performance score to indicate overall
project effectiveness. The result is a four-star system of ratings, with
scores ranging from zero to four stars.
For a group of top-rated,
four-star projects, the chapter examines estimates of partial net benefits
and considers their implications for the overall success of ATP to date.
It also provides summary examples of strong three-star projects.
Because technology
development and commercialization take time and are characterized by unexpected
breakthroughs and failures, future updates of these projects may alter
the findings reported here.
Overall Project Performance
As expected with projects that tackle difficult research
problems, not all of the projects are equally successful. Sixteen percent
of the projects are top-rated in terms of overall project performance.
Twenty-four percent are in the bottom group in terms of project performance.
Sixty percent make up the middle group.
The top performing
projects not only solved challenging and significant technical problems,
but also made the new technical knowledge available to others, and directly
used that knowledge to accelerate commercial use of the technology—three
dimensions of performance that figure prominently in achieving the long-run
success of the ATP. Among this group, half of the top performing projects
received awards for their technical accomplishments from outside organizations,
and more than half of the single-company project leaders received outside
recognition for their business accomplishments. All of the top-rated projects
forged collaborative relationships, and all attracted private capital
for their follow-on commercialization efforts. Among the single-company
projects in the group, all the companies expanded their employment substantially.
All had a very strong outlook for continued progress, and, in fact, have
continued to make strong progress.
The lowest scoring
projects were not without accomplishments. Most performed research and
produced patents or technical publications, or gave presentations. But
none won awards or showed sustained direct progress toward commercialization,
and the outlook for direct commercial action of the award recipients was
poor or uncertain at best.
The large mid-scoring
group of projects had solid—and, in some cases, outstanding—
technical accomplishments, and, in some cases, made substantial progress
toward commercialization. In other cases, the projects were strong technically,
but showed little follow-on commercial progress. In a few cases, the projects
produced a technology with commercial strength but did little to disseminate
knowledge to others—a public-interest goal of the program. In yet
other cases, there was moderate progress in creating and disseminating
knowledge and moderate progress toward commercial goals. This middle group,
although not featured in the discussion of net benefits in this report,
will likely yield substantial net benefits overall.
From a portfolio perspective,
the results look strong for ATP: the estimated net benefits attributed
to the program from the top-performing projects alone far exceed the entire
cost of ATP to date, suggesting that the program is on track to produce
a high return for the nation.
In addition, there
is evidence that the benefits are extending well beyond those enjoyed
by the award recipients. For example, when patients receive superior medical
treatments at lower cost, there are spillover benefits. When consumers
buy high quality products and do not pay the full value for the additional
benefits they receive, there are spillover benefits. When other companies
increase their productivity or value added by using ATP-funded technologies,
there are spillover benefits. When others acquire and use productively
the knowledge from project findings, there are spillover benefits. Several
examples illustrate technology developments and commercial progress of
this first group of projects. (See box.)
| Project
Examples |
|
The
National Center of Manufacturing Sciences (NCMS)
Ann Arbor, Michigan, led a joint venture to achieve dramatic technical
advances in manufacturing printed wiring boards (PWBs). As a foundational
component to any larger electronics assembly, PWBs are essential
to many other technologies, and are used in the manufacture of products
ranging from computers to toys to vehicles. And, advances in PWBs
improve the position of U.S. companies in the very competitive world
electronics market.
The research
team made advances in materials, soldering, imaging, and chemical
processes. A new single-ply fiberglass PWB, which allows substantial
cost savings, has become the industry standard. New process methods
increase dramatically the yield of boards without flaws. A new surface
finish protects the board in multiple soldering applications. A
new process for attaching thin copper plating to fiberglass reduces
processing time and materials cost. A novel interconnect structure
has the potential to revolutionize the fabrication of PWBs by enabling
much higher wiring density. And, according to an in-depth economic
study, because the research effort was collaborative, the new capabilities
were developed at a research cost-savings of at least $35.5 million.
The various
joint-venture participants and their licensees have successfully
commercialized component technologies arising from the project,
resulting in substantial productivity improvements. Award-winning
papers, patents, and new process technology helped convey the information
to the hundreds of small companies that make up most of the industry.
The president of NCMS credited the project with literally saving
what was then the $7 billion U.S. PWB industry—a key segment
of a $20 billion domestic electronic interconnection industry employing
over 200,000 people.
|
|
Engineering
Animation, Inc., Ames, Iowa, developed core algorithms that
enabled the creation of three-dimensional images from sets of two-dimensional
cross-sectional images of human body parts, and animation for selected
organs. After an initial failure to commercialize a high-cost system
that incorporated the technology, the company adapted the technology
for CD-ROMs and print publications in 1995, and then bundled it
with medical books. The company went on to leverage its ATP-funded
technology in a multiplicity of applications featuring three-dimensional
animations which utilize computer visualization and computational
dynamics—in sectors as diverse as medical education, entertainment,
manufacturing design, transportation, and investigation of the Oklahoma
City bombing.
Founded by two
professors and two graduate students in 1990, the company had 20
employees at the time of its ATP award. Now its employees number
approximately 1,000, and 1999 sales totaled $71 million. The company
started receiving recognition from other organizations for its technical
progress in 1994, while it was working on the ATP project. It also
has received extensive recognition for its business achievements,
including acknowledgments by Individual Investor, Business Week,
and Forbes ASAP magazines as one of the best technology companies
in the country.
|
Aastrom
Biosciences, Inc., Ann Arbor, MI, received an ATP award to develop
a process for growing stem cells outside
of the body in 1990, when this was a new concept. Aastrom designed,
constructed, and validated a desktop-size bioreactor with the capacity
to produce large amounts of stem and other cells from small amounts
of bone marrow and umbilical cord blood.
The journey
from university research to commercializing its AastromReplicall™
System has been a long path for Aastrom—one that is still underway,
despite unabated effort and strong progress. The company has extended
the time for its expected commercialization date several times.
An earlier in-depth
economic study estimated that the replication system, once implemented,
would save approximately $134 million (in 1997 dollars) in the costs
of providing bone-marrow transplants for cancer treatment, compared
to the best alternative technique. The study conservatively attributed
about $47 million of the cost savings to ATP. The study also identified
potential benefits of pain reduction and better patient outcomes
from the technology but did not quantify them.
Results from
recent clinical trials point to an additional benefit—enabling
cancer patients without donors to receive stem cell transplants.
Aastrom’s replication system can expand tiny amounts of matching
cord blood into sufficient quantities for adult transplantation.
According to the director of medical oncology at Hackensack University
Medical Center, “these results suggest that we may have found
a new treatment approach that will enable more patients to receive
treatment for this very serious and often fatal disease.” According
to the American Cancer Society, 30,000 new cases of leukemia are
expected in 2000 and approximately 20,000 people will die from the
disease this year, making new, more effective treatments of great
value to society.
|
Peer Recognnition
of Technology Achievements
The knowledge created by each project is the source of
its future economic benefit, both for the innovator and for others who
acquire the knowledge. Knowledge created by the 50 projects ranges from
mathematical algorithms underlying new software tools, to the science
of growing human tissue, to new techniques for fabricating high-temperature
superconducting devices. Recognition of technical achievements by outside
organizations, including trade associations, foundations, and technical
journals, indicates that others see considerable value in the projects.
In 1996 alone, the projects claimed the following awards:
- R&D
magazine–an R&D 100 award to American Superconductor, Inc.,
in Westborough, Massachusetts, for its development of CryoSaver current
leads;
- Industry Week
magazine–one of 25 Technology of the Year Awards to American
Superconductor, Inc., for applications of superconducting wire;
- Industry Week
magazine–one of 25 Technology of the Year Awards to Engineering
Animation, Inc., in Ames, Iowa, for its interactive 3D visualization
products used in the manufacturing sector for product development;
- Discover
magazine–one of 36 finalists for Technology of the Year to HelpMate
Robotics, Inc., in Danbury, Connecticut, for the HelpMateRobot used
in hospitals;
- Microwave &
RF magazine–one of the Top Products of 1996 to Illinois
Superconductor, Inc., in Mt. Prospect, Illinois, for cellular phone
site filters and superconducting ceramics;
- Computerworld
magazine–finalist for the Smithsonian Innovator Medal to Molecular
Simulations, Inc., in San Diego, California, for advances in software
to help scientists simulate and visualize complex molecules.
Dissemination of New
Technical Knowledge
Dissemination of the new knowledge provides spillover benefits
to other companies who, in turn, may use the knowledge to increase and
broaden the national benefits from the ATP investment. Dissemination takes
place in several ways. Patents, publications, and presentations provide
a convenient avenue for others to acquire the knowledge. All but 1 of
the 50 projects produced one or more of these outputs.
The extensive collaborative
activities of the projects have provided another avenue for the spread
of knowledge. Eighty-four percent of these projects entailed collaborations,
including other companies, universities, national laboratories, nonprofit
consortia, and other organizations and individuals.
Release of new products
to the market also disseminates new technical knowledge. Others can use
the products and they may also attempt to discover how the products work
by observation, testing, and reverse engineering. More than 60 percent
of the projects placed commercial products or processes in the marketplace,
providing others with the ability to collect information about the new
technologies.
Workshops, websites,
and evaluation studies also facilitate information flows. The ATP has
organized and sponsored numerous public workshops over the years, in which
the companies have presented nonconfidential aspects of their ATP-funded
research and engaged in open discussions. The ATP has also made project
information available on its website (www.atp.nist).
Evaluation reports, such as this one, are an additional source of information
for the public.
Commercial Progress
in Applying the New Technologies
If the new knowledge is to yield economic benefits to the
nation, the award recipients, their collaborators, or the companies who
acquire that knowledge must put it to use. A second focus of the study,
therefore, is on the commercialization progress of the award recipients,
and in some cases their direct collaborators. The study does not include
commercialization activities of companies who acquire project knowledge
indirectly, although these activities may be as important or more important
than those of the award recipients and their collaborators.
Sixty-six percent
of the 50 projects had one or more products or processes in the market
when they were assessed, and another 14 percent expected to shortly. Thus,
despite the difficulty of moving from the research stage to commercialization,
companies in 80 percent of the projects either sold product, or used or
licensed to others process improvements stemming from their research,
or they were about to do so at the time they were contacted by study analysts.
Whether or not widespread diffusion of a technology results from these
commercial activities, it is highly significant that products and processes
are actually on the market.
An indicator that
a small research-oriented company is on the path toward commercialization
is company growth. A recent look at Fortune’s “Fastest Growing
100 Com-panies” list found 2 of the 31 then-small ATP-funded companies
on the list.
Capitalized value
of some of the ATP-funded companies has increased by hundreds of millions
of dollars. Nearly a fifth grew in employment by more than 500 percent
from the beginning of the project to several years after the project had
completed, and 61 percent grew in employment by more than 100 percent.
Several of the companies that were small when they received the ATP award
have grown out of that size category. Nineteen of the 31 small companies
at least doubled in size; 4 companies grew more than 1,000 percent.
Not all the small
companies grew—a little more than one-fifth experienced no change
or decreases in staff and not all kept their momentum going beyond the
period of ATP funding. But, as a group, the small companies funded by
ATP grew rapidly as they parlayed their new technical capabilities into
business opportunities.
The study results
point up the importance of the ATP’s two-path approach to realizing
national benefits. First, the direct commercialization effort by the award
recipient provides a path for the accelerated use of the technology by
U.S. companies. Second, the knowledge created by a project may disseminate
to others who may use it for economic benefit whether the award recipients
do or not. One path may provide an avenue for benefits when the other
does not, and both paths may yield larger, accelerated benefits compared
to having only a single route to impact.
What Difference Did
ATP Make?
The focus of evaluation is not just on the performance
of projects, but on the difference ATP made to the outcomes. The results
of the more detailed studies cited here emphasize effects attributed to
ATP. In addition, the mini-cases attempt to establish retrospectively
the impact that the ATP had on project outcomes.
For 44 of the 50 projects
responding to the question of what difference ATP made, 59 percent would
not have been undertaken at all without ATP funding, and 41 percent would
have begun at a later date or proceeded at a slower pace. (Personnel changes,
severe company financial distress, or lack of clarity in responses to
interview questions made it impossible to include 6 of the 50 projects
in this tabulation.)
Other effects attributed
to the ATP by the leaders of these projects include the fostering of collaborative
arrangements for research and commercialization activities and the ability
to raise additional capital.
Examples of company
comments about the role of the ATP include:
- Torrent Systems,
Inc.—It is doubtful that the technology could have been successfully
developed at all; venture capital funding had been sought but was unavailable.
- AlliedSignal,
Inc.—The company would have needed another five years to reach
this stage of development.
- Diamond Semiconductor
Group, LLC—The company would have been unable to do the research
or survive as a company; its only other alternative then was to become
part of a foreign company.
- Integra LifeSciences
Corporation—Without
ATP I don’t know that we could have proceeded. We would be at least
five years or more behind where we are.
- Nonvolatile
Electronics, Inc.—ATP
funding enabled the project to be done, prevented the company from failing,
and improved the company’s ability to attract capital from other
sources.
- FSI International,
Inc.—The award enabled FSI to collaborate with Massachusetts
Institute of Technology researchers.
- Light Age, Inc.—The
visibility generated by winning the ATP award helped Light Age establish
agreements with research partners and, coupled with the success of the
ATP project, enabled it to secure additional funding from private investors.
- Thomas Electronics,
Inc.—Without the ATP award, the company would have struggled
along with its conventional CRT technology and would have stood virtually
no chance of competing with other display-component suppliers, all of
which are foreign companies.
Return to Table
of Contents or go to next section.
Date created: April
2002
Last updated:
April 12, 2005
|
