Journal of Technology Transfer,
VOL. 23 (2): 3-4
SYMPOSIUM OVERVIEW
Rosalie T. Ruegg
This collection
of papers relates to the evaluation of the Advanced Technology Program
(ATP), the nation's civilian technology program charged with improving
the competitiveness of U.S. businesses. The program accomplishes
its mission by sharing the costs with industry of particularly challenging
research projects that are expected to accelerate the development
and commercialization of enabling technologies with strong potential
for generating broad-based benefits to the nation.
The Economic Assessment
Office (EAO) of the Advanced Technology Program seeks to measure the
economic impacts of ATP's funding of high-risk, enabling technologies,
and also to increase understanding of underlying relationships between
technological change and economic phenomena so as to further the ability
of the program to achieve its mission. To this end, the EAO compiles
data, conducts economic studies, and commissions studies by outside research
organizations and economists on the economic issues related to the projects
it funds.
Evaluation has been practiced
by the ATP from its inception, first as a management tool for the program
and later to meet the mandated requirements of the Government Performance
and Results Act (GPRA). As emphasized in a previous issue of the Journal, "Science
and Technology (S&T) and Research and Development (R&D) programs
are required as never before to regularly demonstrate the relevance and
value added of their programs" (Jordon 1997, p. 3). Indeed, the
ATP has met nearly continuous demand for measures of impact of the program
since the day it was established. It is probably the most highly scrutinized
program relative to its budget size of any government program to date.
Yet the ATP funds long-term research projects, most of which are still
in the research phase. It funds research underlying the development of
enabling technologies which are expected to have benefits extending substantially
beyond the direct ATP award recipient. Technology diffusion takes time
under the best of circumstances, and tracking and measuring externalities,
or spillover effects is complicated and difficult. Available evaluation
tools are insufficient to that challenging task.
Despite inherent methodological
and measurement challenges, the ATP's evaluation efforts are yielding
improved methods and tools of analysis, early progress indicators, and
projections of long-term impacts. ATP economists are tracking progress
throughout the lives of the projects it funds and into the post-project
period, compiling in the process an extensive database that is used for
portfolio profiling and economic analysis. Through a cooperative arrangement
with the National Bureau of Economic Research, the ATP commissions studies
by leading economists working in the field of technological change, and
these studies are extending the state of the art in the field.
This special issue of
the Journal of Technology Transfer features some recent ATP evaluation
studies. Some were performed by economists in ATP's Economic Assessment
Office; some by academics under contract to the ATP; and one by a consulting
firm. The papers presented are selected to illustrate a variety of evaluation
issues. A number of other evaluative studies are underway.
For background and perspective,
the first paper, by Rosalie Ruegg of the Advanced
Technology Program, provides an overview of the ATP. It explains how
the program operates, how it seeks to accomplish its mission, and what
it has funded to date. The paper identifies the major components of ATP's
evaluation program, and identifies research areas of particular interest
to the evaluation program. Because the ATP is a much misunderstood program,
this paper seeks to provide a basis for clearer understanding of its
rationale and guiding principles.
The second
paper, by Adam Jaffe of Brandeis University, reflects the importance
to the ATP of generating and measuring economic spillovers. It illustrates
with simple models how the benefits of ATP projects may extend beyond
the direct ATP award recipients through market and knowledge spillover
effects, and also discusses network spillovers. Jaffe recommends
that the ATP fund projects whose expected social benefits are large
and substantially exceed expected benefits to the awardees; guidance
which the ATP seeks to follow. He also recommends that evaluation
efforts include measures of spillover effects. This paper is presented
early in the collection because it contributes to the understanding
of later papers that refer to spillover effects.
The third
paper, by Jeanne Powell of the ATP's Economic Assessment Office,
presents and analyzes data from ATP's "Business Reporting System" to
evaluate short-and-medium-term project effects. The paper describes
ATP's principal data collection tool, and identifies a number of
pathways through which ATP-funded technologies are generating impact.
The enabling nature of the funded technologies is suggested by the
many potential applications that have thus far been identified.
A fourth
paper, which examines ATP's effects on the speed with which research
is conducted and technology is commercialized, is provided by Frances
Laidlaw, formerly an industry consultant with the ATP and now a technology
planner with Motorola. Laidlaw's paper questions not only whether,
and by how much, the ATP accelerates research, but also the value
of acceleration; whether saving time during the research stage translates
into saving time downstream; and whether project participants realize
any time-related benefits outside the walls of the project. Acceleration
of technology development and commercialization is of keen interest
because it is part of the ATP's mission, and is one of several ways
that the ATP can affect economic outcomes of the projects it funds.
The fifth
paper in the collection, by Albert Link, an economics professor
at the University of North Carolina at Greensboro, investigates the
effects on research efficiency of collaboration in an ATP-sponsored
joint venture. The project in question aimed to advance dramatically
the technical capabilities of makers of printing wiring boards which
provide the backbone of electronics products. His work also examines
early productivity effects resulting from the new technical capabilities.
In the sixth
paper, researchers at CONSAD Research Corporation report on a
detailed case analysis performed for another research joint venture
sponsored by the ATP, this one on new dimensional control technology
for discrete manufacturing. It analyzes the multiple impacts of the
new technology applied in the automobile manufacturing sector, and
uses a macroeconomic model to project national economic impacts resulting
from quality improvements in vehicles made by U.S. producers.
The seventh
paper examines special considerations in modeling the social
benefits of medical technologies. Prepared by Andrew Wang, an economist
in ATP's Economic Assessment Office, the paper is inspired by a study
conducted by economists at the Research Triangle Institute that is
nearing publication. That study develops a framework for evaluating
ATP-funded medical technologies, and applies the framework to project
preliminary estimates of private, social, and public expected net
benefits of seven tissue engineering projects funded by the ATP.
Wang's paper provides a thoughtful exposition of several of the key
concepts employed by the RTI study to estimate spillover benefits
to patients who receive more effective, less painful, or less costly
treatments as a result of the new tissue engineering technologies.
The last
paper in the collection, by ATP economist Connie Chang, signals
ATP's interest in counterpart programs to the ATP that are operated
in most other industrialized countries. She shows similarities and
differences in the programs by dissecting their design features.
The ATP requires a detailed understanding of these other programs
in order to take advantage of their evaluation programs in terms
of the implications for the ATP. This understanding is also needed
by the ATP to implement its Congressionally required evaluation of
eligibility of U.S. subsidiaries of foreign-owned companies to participate
in the ATP.
Many additional evaluation
studies of the ATP are currently underway on a variety of topics. Most
of this work will be reported in ATP publications, as well as in journals
such as this. ATP welcomes comments and advice from the evaluation community
on ways to improve the modeling and analysis of economic benefits from
the advanced technologies it funds.
Please note: NIST papers
are a contribution of the National Institute of Standards and Technology
and therefore are not subject to copyright.
Rosalie T. Ruegg
Special Issue Editor
Director, Economic Assessment Office
Advanced Technology Program
Reference
Jordon, Gretchen B. "Symposium
Overview." Journal of Technology Transfer vol. 22, no. 2,
Summer 1997, pp. 3_4.
 Return
to Table of Contents
Date created: September
1998
Last updated:
August 4, 2005
|
