 |
|
|
|
|
|
|
|
|
 |
 |
 |
 |
 |
 |
 |
 |
|
|
|
||||||||||||||||||
|
|
A Toolkit
for Evaluating Public R&D Investment AbstractEvaluation is an essential component of publicly funded R&D programs, both in support of program management and public policy. The Advanced Technology Program (ATP) has emerged over its first decade as a leader in evaluation, engaging nationally prominent evaluators to apply new and existing methods in building an analytical and empirical basis for ATP’s operations and performance. This report draws from a body of 45 studies commissioned by ATP between 1990 and 2000 and analyzes the methods and techniques used and examines the findings of those studies. These studies have increased understanding not only of ATP but also of the dynamics of innovation systems and the relationships between public and private sector funding of R&D. The findings examined are organized around five major themes: firm/industry effects, collaboration effects, spillover effects, interfaces and comparisons with other programs, and measures of overall program performance. The extensive toolkit of evaluation methods presented in the report illustrates how those methods can be used to answer a variety of stakeholder questions. Methods include survey, descriptive and economic case study, bibliometrics, historical tracing, econometrics, expert judgment, social network analysis, cost index, and a composite performance rating system constructed from indicator metrics. Additionally, the use of analytical and conceptual modeling to explore a program’s underlying relationships and process dynamics is considered. The political economy of ATP is discussed, and an evaluation framework and an overview of evaluation best practices are provided. The report integrates and condenses a large body of related research and thus provides ATP with a convenient reference work, toolkit, and planning guide. For those administrators of other programs, public policy makers, and evaluators, the report also serves as an evaluation toolkit by providing a logical framework for program evaluation, illustrating the use of evaluation methods and techniques, providing an overview of evaluation principles and practices, organizing a body of knowledge on how public-private partnership programs function, and contributing to an understanding of what evaluation is and how it is practiced in the field of R&D. Keywords: Advanced Technology Program, assessment, economic evaluation, evaluation methods, impact analysis, logic models, public policy, public-private partnership program, R&D, spillovers, technology. This research was conducted between July 2001 and December 2002. Return to Table of Contents or go to Executive Summary. Date created: July 13,
2004 |
 |
ATP website comments: webmaster-atp@nist.gov / Technical ATP inquiries: InfoCoord.ATP@nist.gov. NIST is an agency of the U.S. Commerce Department |
 |