Abstract
This study empirically
evaluates the impact of consortia funded by the U.S. Advanced Technology
Program (ATP) on the research productivity of participating firms. We
find that there is a positive association between the intensity of participation
in research consortia and the overall research productivity of participants.
Our analysis suggests that participation in one additional ATP-funded
research consortium per year would generate an increase in patenting
for that firm in that year of nearly 8%. We also find that consortia
have a positive impact on the research productivity of participants
in the technological areas targeted by the consortia. This positive
impact of consortia is higher when the average technological proximity
(as measured by the degree to which the patenting portfolios of participating
firms are similar) of participants is high. There is preliminary evidence
that large firms conducting intensive research and development (R&D)
tend to benefit more from their participation in consortia.
Japanese data validate
the findings of data from ATP projects regarding the positive association
between the average technological proximity of firms in a consortium
and patenting in the technological areas targeted by the consortium,
as well as an increase in patenting outcomes (i.e., increase in patenting
activity) over pre-consortium levels in the technologies targeted by
the consortium. Japanese data support ATP’s focus on pre-commercial
research. Qualitative data show a positive association between patenting
outcomes and research projects that Japanese firms perceive to be more
“basic” or “pre-commercial,” as opposed to projects
that are close to commercialization. In addition, Japanese data suggest
that bringing makers of rival products for the same market into the
same consortium is detrimental to patenting outcomes. Few ATP projects
have this type of horizontal structure. The results of the authors’
research on Japanese research consortia were published in the American
Economic Review in March 2002.
Acknowledgements
No research effort
of this magnitude is ever accomplished without the help of many people.
We would like to first express our gratitude to the staff of the Economic
Assessment Office of ATP. Special thanks go to Holly Jackson and Jeanne
Powell for help with initial data acquisition, to Robert Sienkiewicz
and Richard Spivack for help in arranging interviews, obtaining additional
data, and providing us with excellent feedback at several points in
the research process, and to John Hewes, Jack Boudreaux, Connie Chang,
and other reviewers at ATP who helped shape the final study. We also
wish to thank Adam Jaffe of Brandeis University and the National Bureau
of Economic Research (NBER) for his guidance and suggestions. Susan
Colligan and Julie Peters of the NBER helped administer our grant efficiently
and effectively, and we are grateful for their efforts.
To protect the confidentiality
of our interviewees, we are unable to acknowledge by name the government
officials and firm managers we interviewed in both the United States
and Japan to obtain invaluable insights on how both ATP and Japan’s
Ministry of Trade and Industry (MITI, now known as the Ministry of Economy,
Trade and Industry, or METI) research consortia actually operated. However,
this information was a critical input to our research. We are therefore
very grateful to the numerous officials and executives who generously
shared their insights with us.
The staff of the
Japanese Patent Office shared their vast information resources with
us at a fraction of the cost normally charged to private sector data
users. Officers at the Japanese Patent Office also helped us create
a mapping to link the technological goals of Japanese research consortia
with the International Patent Classification system. In generating a
similar “mapping” for ATP-sponsored consortia in the United
States, we benefited from the excellent assistance of Bailey Services,
Inc. We also benefited from informal discussions with Sam Petuchowski
of Bromberg and Sunstein, LLP. We also acknowledge the generosity of
Bronwyn Hall at the University of California, Berkeley, and Adam Jaffe
in sharing their data resources with us. We could not have completed
this project without their help.
Finally, a number
of research assistants worked on assembling much of the data used in
this report. We would like to thank Shun-Li Yao and, especially, Kaoru
Nabeshima, both Ph.D. students in the UC-Davis doctoral program in economics,
for their outstanding research assistance. Mariko Sakakibara received
help from Makoto Nakayama, Jina Kang, and Heather Berry of the UCLA
Management Ph.D. program, Jon Wolf at the UCLA Department of Economics,
and Yumiko Kawanishi in the doctoral program of East Asian Languages
and Cultures at UCLA.
Executive
Summary
In pursuit of its
legislative mandate to strengthen the competitiveness of technology-intensive
U.S. firms and industries, the Advanced Technology Program (ATP) supports
research consortia to promote and stimulate “pre-commercial”
research by private firms. A number of theoretical arguments in the
economic literature support the use of this policy instrument. ATP’s
funding decisions and evaluation efforts, however, must be based on
empirical facts rather than theoretical appeal. In this study, we describe
the results of our empirical evaluation of the impact of ATP-funded
consortia on the research productivity of participating firms.
This study builds
upon our earlier work (Branstetter and Sakakibara, 1998, 2000) where
we examine the impact of the Japanese government–sponsored research
consortia. To extract as much useful information as possible from our
quantitative data set, we analyze the ATP-funded consortia at three
different levels of aggregation:
- the impact of
consortia participation on the overall research productivity of the
participating firm;
- the impact of
participation at the consortium level; and
- the impact of
consortia participation at the level of the firm-consortium pair.
We describe below
our analytical framework, our findings using both U.S. and Japanese
data, and their implications. We acknowledge the limitations of our
study and make a number of suggestions on how future ATP-affiliated
researchers could build upon and extend our results. Finally, we point
to complete documentation of the database we used to study the impact
of ATP-funded research consortia.
THE IMPACT
OF CONSORTIA PARTICIPATION ON THE OVERALL RESEARCH PRODUCTIVITY FIRMS
To what extent did
participation in an ATP-funded research consortium contribute to an
overall expansion of research productivity among participating firms?
We pursued this question by developing an original, firm-level data
set of research inputs and outputs for a set of firms that participated
in ATP-funded research consortia and a set of control firms that were
never involved. Using such data, we analyzed the statistical relationship
between the frequency of participation in ATP-funded consortia and the
relative research productivity of industrial firms. Our analysis used
a conceptual framework known in the economic literature as the “knowledge
production function.” Innovative output was measured using patent
data. Innovative input was measured as research and development (R&D)
spending. We fully recognize the many shortcomings of patent data as
a measure of innovative output. Our decision to base our analysis on
patent data is discussed and defended in the body of the report.
We found the relationship
between participation and research productivity to be positive, statistically
significant, and robust to changes in the specification of our statistical
model. In other words, holding all other factors constant, we find evidence
that firms participating in more ATP-funded research consortia generate
more patents per unit of R&D spending than firms that participate
in fewer ATP consortia or do not participate at all. Our econometric
estimates suggest that, at the margin, a firm that participates in an
ATP-funded consortium will realize a nearly 8% increase in research
productivity per year.
THE IMPACT OF
PARTICIPATION AT THE CONSORTIUM LEVEL
What is the impact
of participation in ATP consortia on the collective patenting of participating
firms in the technological areas targeted by the consortia? What kinds
of consortia are the most successful at promoting the research productivity
of participating firms? With the help of outside experts, we constructed
a mapping from the stated technological goals of ATP-funded consortia
to the relevant patent classes of the U.S. Patent and Trademark Office
(USPTO) patent classification system. This mapping allows us to examine
patenting in the technological areas targeted by a consortium. Our results
indicate that consortium participation increased patenting in the targeted
areas above pre-consortium levels. In addition, “technological
proximity” of consortia, as measured by the degree to which the
patenting portfolios of participating firms are similar, and “pre-consortium
technology strength” are positively associated with patenting in
the technological areas targeted by the consortium.
THE IMPACT OF
CONSORTIA PARTICIPATION ON FIRM-CONSORTIUM PAIRS
What type of firm
receives the largest benefits from participation in an ATP-funded research
consortium? We examine the constituent firms of each consortium and
analyze the impact of the consortium on each firm separately. Our preliminary
results indicate that larger firms with higher R&D budgets (i.e.,
technologically more progressive firms) tend to benefit more from participation
than other firms. In the absence of panel data on the research inputs
and outputs of smaller firms, it is difficult, however, to come to any
definitive conclusions about the effect of size or overall R&D spending
on research outcomes.
ADDITIONAL WORK USINIG JAPANESE DATA
Japanese data were
used as a statistical “testing ground” for the analytical
framework that was applied to U.S. data. Japanese government support
of research consortia began in the late 1950s, which allows us to examine
the long-run effects of consortia. (ATP’s first projects began
in 1990–91 and the time series data used in our U.S. analysis extend
only through 1995.) Results from Japanese data suggest that much of
the impact of research consortia is felt long after the inception of
the project. In fact, evidence from Japanese consortia suggests that
some of the strongest effects are felt after the official cessation
of the consortia. This means that the relatively short time series
of data available on participating firms in ATP-funded research consortia
will tend to underestimate the total impact of the consortia.
RESULTS AND IMPLICATIONS
We find evidence
that the impact of participation in ATP-funded consortia on the research
productivity of participating firms is positive at all three levels
of analysis examined in this paper:
- the impact of
consortia participation on the overall research productivity of the
participating firm;
- the impact of
participation at the consortium level; and
- the impact on
the firm-consortium pair.
First, we find that
there is a positive statistical association between the intensity of
participation in research consortia and the overall research productivity
of the participating firms. Second, at the consortia level, we find
a positive impact of consortia on the research productivity of participating
firms in the technological areas targeted by the consortia. Furthermore,
we find that this positive impact of consortia is higher when the average
“technological proximity” of participating firms is high.
This is a measure that ATP could calculate for prospective firms as
a screening device for selecting the most meritorious project proposals.
Third, we find less clear-cut evidence concerning which types of firms
benefit most from participation. Our preliminary results suggest that
larger firms with higher R&D budgets benefit more from participation
than other firms. However, this finding should be viewed with caution
given the inherent flaws in our data set regarding research inputs and
outputs for smaller, private firms.
Results from these
three levels of analysis demonstrate that participation in ATP-funded
consortia is leading to verifiable, measurable increases in research
productivity, an indication that ATP is accomplishing its mission.
LIMITATIONS OF
THE STUDY AND ISSUES FOR FURTHER ANALYSIS
We confronted three
important limitations in our data. The first most serious problem is
that our data series covers only a four-year period, from 1991 to 1995.
Information on the total patenting of participating firms is based on
relevant data that extend to 1994 or early 1995 from the Regional Economic
Issues (REI) Patent Database developed and maintained at the Case Western
Reserve University Center for the Study of Regional Economic Issues.
Information on the total R&D spending of participating firms is
based on Standard & Poor’s COMPUSTAT database of financial,
statistical, and market information on more than 7,500 publicly held
companies; information taken from COMPUSTAT extends to 1995 (but could
have been expanded to 1997). Relatively few ATP projects began before
1995, and almost none were completed by then. Our patent data effectively
end in 1995, just as ATP was expanding its support of research consortia.
Results from our Japanese data indicate that the full impact of participation
in a research consortium is only realized over fairly long periods of
time. A large share—perhaps the largest share—of the benefits
from participation in ATP is missing from the data set. This means that
the data are likely to underestimate the overall impact of ATP-funded
research consortia. The fact that we find positive, statistically significant
benefits of participation in spite of this data truncation problem suggests
that the positive effects are real.
Second, the data
set lacks information on the research inputs and outputs of some smaller
firms involved in ATP-supported research consortia. While it is relatively
easy to obtain data on the large, publicly traded firms that were part
of these projects, it is difficult to obtain similar data on small,
privately held firms that were often leaders of the joint ventures.
Lack of data on smaller firms limits our ability to estimate the impact
of ATP-funded consortia on their research productivity. If smaller firms
benefit more from consortia participation than do larger firms, then
our data may underestimate the impact of ATP-funded consortia on smaller
firms.
Finally, we were
unable to analyze the ATP’s Business Reporting System (BRS) survey
data to its fullest extent due to confidentiality constraints. As a
result, we used consortium-level averages, rather than individual firm
responses, in our analysis of firms’ perceptions of the benefits
of consortium participation on research outcomes. Using individual firm
responses would have enhanced our ability to establish statistical links
between survey variables and research outcomes.
Future ATP-affiliated
researchers could expand our data set in ways that would solve some
of these problems. First, publicly available databases on firm patenting
and other firm characteristics could be used to expand the time series
dimension of our data set beyond 1995. This would allow a more comprehensive
analysis of the long-term benefits of ATP-funded projects. Second, ATP
is addressing the coverage of small- and medium-size, private firms
through the extension of BRS. Third, ATP staff could conduct additional
firm-level analysis using BRS survey data to fully utilize the available
data and avoid compromising data confidentiality.
DATABASE CONSTRUCTION
The Appendix
documents the database developed for this project, which was provided
to ATP with the first draft of the study in October 1998. The database
contains two key components. Table A1
is a mapping from the stated technological goals of ATP-funded research
consortia to the corresponding patent classes in the U.S. patent classification
system. This mapping, completed with the assistance of Bailey Services,
Inc., allows us to measure patenting by participating firms in the targeted
classes before, during, and, in principle, after the cessation of a
consortium. Table A2 is a list of ATP-funded
consortium projects, the firms participating in those projects, and
their associated Standard & Poor’s CUSIP (Committee on Uniform
Security Identification Procedures) identifying codes. The CUSIP codes,
a standard method in identifying issuers of securities, in conjunction
with Standard & Poor’s COMPUSTAT database, may be used to update
our data set. The Appendix also contains a complete description of the
way in which key variables were constructed and the statistical software
package used in our analysis. We hope these tables and documentation
will be a useful, enduring data resource for ATP.
Date created: January
24, 2003
Last updated:
February 10, 2003
