STATUS REPORT
NUMBER 1

NIST SPECIAL PUBLICATION 950-1

William F. Long
Business Performance Research Associates, Inc.
Bethesda, Maryland 20814

March 1999

CONTENTS

Acknowledgements
Executive Summary
Introduction

CHAPTER 1 - Overview of Completed Projects

Characteristics of the Projects
Timeline of Expected ATP Project
    Activities and Impacts
Gains in Technical Knowledge
Dissemination of New Knowledge
Commercialization of the New Technology
Broad-Based Economic Benefits

CHAPTER 2 - Biotechnology

Aastrom Biosciences, Inc.
Aphios Corporation
Molecular Simulations, Inc.
Thermo Trilogy Corporation
Tissue Engineering, Inc.

CHAPTER 3 - Chemicals and Chemical Processing

BioTraces, Inc.

Return to Main Page.

Before turning to economy-wide benefits, it is useful to consider the impact of ATP funding on the research that led to them. Project leaders from each company were questioned about the role ATP funding played in their projects. Their answers are presented in the detailed discussions of Chapters 2-8 and summarized in Table 5.

Table 5. Impact of ATP funding on Conducting Projects

For all 38 completed projects, awardees were asked whether the project would have been done at some point without ATP funding. Answers were received for 32 of the projects. ⁽¹⁴⁾ For 21 of them (66%), the companies and other organizations indicated they would not have done the project at all without ATP funding. ⁽¹⁵⁾ For the other 11 projects, they said they would have done the project at some later date or slower pace. For the 32 projects as a whole, none would have been completed in the same time frame without ATP funding, and 21 of them would not have been completed at all, according to officials at the companies and other organizations. For the 11 companies whose projects would have been delayed without ATP funding, the typical lag reported was about two years, with a wide variance around the average. A lag of just 24 months may seem short, but its effects can be substantial when the costs and benefits of accelerating the technology development are considered. For illustrations, see the detailed treatments of two ATP projects, Aastrom Biosciences and Tissue Engineering, later in this section, where acceleration of the availability of new medical treatment technology is shown to have a potentially large impact on societal benefits.

Receipt of an ATP award also enhanced the ability of some of the companies to raise additional capital and acquire partners. Thirteen of the 32 responding companies reported that the ATP award helped them raise additional capital (four of them were among the companies that conducted an IPO after receiving ATP funds), and 23 said it boosted their ability to find partners. ⁽¹⁶⁾

Assessing Private and Social Returns from New Technology

Counting the number of projects that would not have been done without ATP funding provides some limited information on the benefits of the program, as does tabulating the number of months that projects would have been delayed if they would have been done, but on a delayed schedule. These limited data are insufficient to assess whether the ATP awards for the 38 completed projects were good uses of public funds, however. More detailed assessment is needed.

The value of the ATP-funded research can be assessed by probing the benefits and costs of projects and the return on the ATP investment. It should be kept in mind, however, that full diffusion of technologies generally takes considerably more time than has elapsed for these projects, and at this time their ultimate, long-run outcomes cannot be known with certainty.

The Mansfield Study of Private and Social Returns

More than 20 years ago Professor Edward Mansfield ⁽¹⁷⁾ of the University of Pennsylvania established general procedures for economists to follow when compiling estimates of the private and social returns from groups of innovations (new products or processes). His work focused on estimating "consumer surplus" benefits to consumers of new and improved goods and services resulting directly from commercial activities of the innovators - a type of spillover effect.

Mansfield's method and estimates addressed market spillovers and those knowledge spillovers which generate benefits via the development of new or improved competitive goods and services by imitators of the original innovating companies. He did not address other kinds of knowledge spillovers, such as use of the new knowledge in a research process leading to other new technologies in a different industry. Hence, for the type of enabling technologies that ATP funds, Mansfield's approach could be expected to capture an important, but partial, share of the total impact.

Case Studies of Seventeen Innovations

Mansfield based his analysis on 17 extensive individual case studies. His procedures have been upgraded over the years, but they still constitute a good starting point for any empirical study of the effects of innovation. He and his colleagues collected annual data for: cost, revenue and profits from the innovating firm; cost, revenue and profits from other firms in the same industry for competitive products or processes they introduced after imitating the new product or process; cost, revenue and losses from the innovating firm or other firms in the industry for products or processes the new product or process supplanted; cost, revenue and profits for producer goods from other firms that purchased the new product or licensed the new process; and cost and benefit data from final users for consumer goods.

Once these data were in hand, they were used to calculate: the annual costs of the innovation; the annual private dollar returns to the innovator; the annual dollar returns to all other parties (competitive firms, purchasing firms, final users); the net annual social dollar returns, by summing all these annual dollar returns (netting out any negative values); the annual private dollar return (using data for the innovating firm alone); the private rate of return; and the social rate of return.

Data Requirements for the Mansfield Analysis

The landmark results published by Mansfield, et. al., have been cited numerous times in the economics and technology policy literature, usually in the context of examining differences between the private and social returns from innovation. The focus here, however, is on a different aspect - the amount of data required to support his analysis. Table 6 presents data from the Mansfield study showing when the 17 innovations entered the market and how many years of data were available for the empirical estimates. In most cases, Mansfield was able to draw on 11 to 18 years of historical data for the older innovations. For more than a third of them, however, some projected data were used. ⁽¹⁹⁾

Table 6. Years of Data Available for Estimating Effects of 17 Innovations, Mansfield, et. al. (1977)

Few data of the type collected in the Mansfield study exist for new products and processes generated by the 38 ATP projects, since the technologies are still so young. Most of these innovations have multi-application potential, making their evaluation even more complex. And most of their benefits and many of their costs are yet to come. Economists can, nevertheless, project the values of these items (as Mansfield, et. al., did in some cases) in order to calculate the private and social returns. The earlier an analysis is conducted, relative to the year of the innovation, the greater the necessity to use projected data and, consequently, the greater the uncertainty in the results. Uncertainty in results is unavoidable at this time for benefit-cost evaluations of this kind for ATP-funded projects.

After a sufficient number of years have passed, an exercise like the Mansfield study - relying on more years of empirical data - can be performed for the innovations that emerge from these 38 ATP projects. That exercise will be much easier if data are collected and carefully archived along the way. The ATP is doing that as part of its evaluation plan. ⁽²⁰⁾

A Portfolio Approach to Costs and Benefits for the 38 Projects

The ATP awarded $64.6 million to the 38 completed projects described in Chapters 2-8 and contributed another $9.4 million to the 12 terminated projects (see Appendix B), bringing total ATP spending on the 50 projects completed or terminated by March 1997 to $74.0 million.

Since it is not expected that every project will be fully successful - all research goals reached, commercialization achieved, widespread dissemination of the knowledge and extensive benefits realized from the use of the resulting goods and services - it is more reasonable to assess the effectiveness of ATP awards as a group of funded projects, as an "investment portfolio," much as an investor in stocks and bonds might do. Pursuing that line of thought with the combined set of 50 completed and terminated projects leads to a simple question: For its investment of $74.0 million, what has the public received, or is likely to receive, in return?

Expected Returns for Just Three of the Projects

This study did not attempt to estimate returns to project participants or to society for the entire portfolio of 38 projects. To do so would entail an involved process requiring detailed economic evaluation case studies and a much larger effort than was allocated for this report. But for three of the projects, such detailed estimates have been calculated by other researchers. ⁽²¹⁾

Aastrom Biosciences: Stem-Cell Therapy Cost Reductions

The availability of ATP funds enabled Aastrom Biosciences to achieve its results one to two years earlier than it would have otherwise. This finding implies that benefits from the use of the company's new AastromReplicell(tm) System would start one to two years sooner.

Benefits of several kinds are expected to result from use of the System, as noted in Chapter 2. One of these is a reduction in the cost of stem cell therapy for cancer patients after chemotherapy or radiation treatments. Other benefits are reductions in the patient's pain and in the risk of complications.

Economists at the Research Triangle Institute (RTI), a consulting firm in North Carolina, have calculated estimates of the value of accelerating the availability of the System, using only the reduction in procedure cost. ⁽²²⁾ RTI economists estimated the number of cancer patients who would use the System in its first year of availability (16,000), estimated the annual growth in applications of the System, determined the cost reduction per patient, and used conventional present-value calculations to get a current value for the cost reduction effect. RTI conducted the calculations assuming the System would be available with ATP funding at the beginning of the year 2000 and repeated the calculations for the "without ATP funding" case that assumes the Systems would be available 18 months later.

RTI estimated that the System, once implemented, would save about $87 million (in 1997 dollars) in medical treatment costs without the acceleration provided by ATP support and $134 million with the acceleration. The difference, $47 million, is the estimated additional value, in terms of cost savings, created by the ATP funds, based on this one application area. Other applications of stem-cell therapy using the System are also expected, which will likely add to the future benefits.

This estimate considers only cancer treatment cost savings. Besides these benefits, the typical patient is expected to have less pain, suffering and trauma when stem cells are collected if the System is used instead of an alternative procedure. However, the value of the pain and trauma reduction is not included in the calculations because data for those effects were not available.

It is also expected that the stem-cell mixture that is injected back into the patient will be freer of cancer cells, leading to a better eventual outcome, if the System is used, but value was not assigned in the RTI study to that beneficial effect, either. Finally, with lower cost and less trauma, stem-cell therapy might become a possibility for some cancer patients who would otherwise not receive it. Stem-cell therapy is expected to increase survival chances for some of these patients, but the value of their prolonged lives is also not included in the estimate.

According to the estimates calculated by RTI, we can expect the additional returns to society attributable to ATP's award to Aastrom Biosciences to be on the order of $47 million, at least. Funding by ATP for the Aastrom project was $1.2 million. And the contribution by ATP to all 50 completed and terminated projects was $74.0 million. Since the RTI estimates from the use of the Aastrom System product were based on only one of several kinds of potential benefits, it seems clear that returns from this project alone are likely to account for a substantial percentage of the ATP expenditure for all 50 projects.

Auto Body Consortium: Higher Quality Car Bodies

While the economic and social impact of the Aastrom System is almost entirely in the future, the Auto Body Consortium's ATP project is already producing measurable benefits, as noted in Chapter 4. Chrysler, a member of the consortium, is making its Concorde line with the new dimensioning technology, as discussed in its marketing literature. Cars in this line are assembled in a plant that has already implemented the new technology and has the capacity to assemble about 250,000 cars per year. To date, the new technology has been implemented in six of the 10 Chrysler plants in North America, and each is expected to produce a minimum of 200,000 cars in 1998.

In a detailed study of this ATP project, Consad Research Corporation (Consad), a consulting firm in Pittsburgh, Pennsylvania, estimated a range of $10 to $25 per vehicle in production cost savings. ⁽²³⁾ Multiplying the smaller number ($10) by the minimum number of cars to be assembled in the six Chrysler plants yields an estimate of at least $12 million in production cost savings for 1998 alone. Multiplying by the larger number ($25) results in a savings estimate of $30 million.

Every one of those cars produced in 1998 will also cost less to maintain, with the producers saving on warranty costs and consumers saving on out-of-warranty costs. Consad estimated maintenance savings of $50 to $100 per car over its life, implying that for these 1.2 million cars (six plants producing 200,000 cars each), between $60 million and $120 million in maintenance costs will be saved over the life of the cars. Only a small portion of those maintenance savings have been realized so far, because none of these cars has been on the road for much more than a year.

Actual current savings have also already been realized by General Motors, the other automobile assembler involved in the project. The new technology has been implemented in 16 of its 31 plants in North America. Since the number of cars produced per plant by GM is comparable to that by Chrysler (at least 200,000 per year), GM will realize production cost savings of at least $32 million in 1998, and the figure could be as high as $80 million. And maintenance savings over the life of these cars would be between $160 and $320 million.

The estimates do not take into account cost savings from extending the technology to the other 4 Chrysler and 15 GM plants. The savings for those additional plants are still in the future, but the likelihood of these savings occurring in the U.S. economy is high.

Once again, a comparison with the size of the portfolio investment is in order. At least $44 million ($12 million at Chrysler and $32 million at GM) in production cost savings were expected to be realized in 1998 alone. The savings could be as high as $110 million. Comparable savings at the six Chrysler and 16 GM plants in 1999 and beyond are expected, as well. The Consad study projected economywide benefits of about $3 billion in the year 2000 due to resulting quality improvements in U.S.-produced automobiles and associated market share gains. ⁽²⁴⁾

Tissue Engineering: New Materials to Repair Damaged Ligaments

The availability of ATP funds enabled Tissue Engineering to achieve its results two years earlier than it would have otherwise, as noted in Chapter 2. RTI, which also included this project in its detailed case studies, ⁽²⁵⁾ estimated that products using a new prosthesis material - animal-derived extracellular matrix, or ADMAT - based on technology developed by Tissue Engineering with ATP support, would reach the market in 2001.

The RTI study focused again on a single application of ADMAT in calculating benefits from the use of this technology, namely, the repair of damaged knee ligaments (specifically, anterior cruciate ligaments, or ACLs). To estimate the number of potential users, RTI questioned officials at Wright Medical Technologies, a partner with Tissue Engineering, who provided an estimate of the number of persons who damage their ACLs annually. Based on that estimate, RTI estimated that the number using the Tissue Engineering technology would start at 9,000 in the first year of availability and grow to 72,000 10 years later. In addition, the RTI study explicitly incorporated benefits from the improvement in the quality of life for such persons, using a "quality-adjusted-life-years" index value.

RTI concluded that the total benefit to persons who receive the treatment is expected to approximate $33 billion with the support of the ATP funds. Without that support, it is expected to be on the order of $18 billion, because without the ATP funding benefits are not expected to start to accrue until 2003. Thus, about $15 billion of the expected net benefits from the new technology was estimated to be attributable to ATP funding. ⁽²⁶⁾

The difference in the sizes of RTI's estimated benefits from uses of the Aastrom Biosciences and Tissue Engineering technologies occurs for two major reasons. One is that the number of potential users of Tissue Engineering's ADMAT (patients with ACL damage) is larger than the number of potential candidates for bone marrow transplantation using Aastrom's System. The other is that the estimated patient benefits for ACL repair includes an estimate of the value of improvements to the patient's quality of life, whereas the estimates for bone marrow transplant benefits reflect only treatment cost savings and include no values for physical benefits to the patient.

These estimates for benefits to be received by users of the Tissue Engineering technology are so much larger than the ATP contribution to the project - $2 million - that making a comparison seems beside the point. What seems clear, though, is that the expected benefits appear to be much larger than the cost expended to achieve them.

Projected Benefits From ATP Contribution in Three Projects Exceed Total ATP Costs

The value of the projected benefits resulting from the ATP contribution in just the three ATP projects profiled above would greatly exceed total ATP costs to date. Cost savings already realized by Chrysler and GM as a result of the Auto Body Consortium project appear likely to be larger than the $74.0 million that ATP put into all 50 projects addressed in this report, not to mention the larger gains to the economy from quality improvements. If Aastrom Biosciences succeeds in bringing its product to market and if the RTI estimate of the value of the acceleration of market availability proves accurate, the return from ATP's assistance to the Aastrom project alone would cover more than half of all ATP funds provided for these 50 projects.

In addition, the estimated social return attributed to the ATP for the Tissue Engineering project is in the billions of dollars. The value of those benefits obviously swamps the $74.0 million in ATP funding for the 50 projects. Indeed, if the ADMAT technology proves to be anywhere nearly as beneficial as the RTI estimates predict, its benefits would swamp all ATP funding for all projects since the beginning of the program. Even if the expected number of patients who would benefit were cut, for example, by 80 percent and the expected benefit per patient were reduced by a like percentage, the estimated return from the ATP's contribution to this technology would still be more than half a billion dollars.

Potential Benefits from Other Projects

Based on the investigations of projects conducted for this study, considerable evidence suggests that others among the 38 projects are also quite promising in terms of their future benefits potential.

To mention only a few of the additional promising technologies that have resulted from this first group of 38 completed projects, consider first the Torrent Systems Project. It was found, for example, that an early user of its computer software technology expected to generate between $50 and $100 million per year in increased revenue on a $17 million investment in a system incorporating Torrent's technology, and that other users were also adopting the technology.

As another example, it was found that the software technology of Engineering Animation is being used to improve the training of doctors, among other things, and that patients in a particular surgical procedure were having better outcomes as a result of the company's imaging software. To these we can add other projects that were found to have produced promising technologies - technologies that may facilitate better weather forecasts, improve communications, enable new drug discovery, improve electronic devices, and lower loss of limb and life globally by improving detection of old land mines and toxins.

Preparing the Way for Future In-Depth Studies

Although this study does not provide a detailed quantitative analysis of the benefits deriving from these 38 completed ATP projects, it does document a number of project performance characteristics that will be useful for detailed estimates of returns. The presentations of project status in Chapters 2-8 contain many references to relevant markets, the role that the technology plays in those markets, the position of the innovating firm relative to other firms in the vertical chain leading to final purchase by users, and other characteristics that would be used in such a study. It also documents progress as of a point in time.

Go to other sections of Chapter 1: Overview of Completed Projects
 Characterstics of the Projects
Timeline of Expected ATP Project Activities and Impacts
Gains in Technical Knowledge
Dissemination of New Knowledge
Commercialization of the New Technology
Broad-Based Economic Benefits

Date created: March 1999
Last updated: April 12, 2005