A Toolkit for Evaluating Public R&D Investment
Models, Methods, and Findings from ATP's First Decade

CHAPTER 9: A Crosscutting Look at Study Findings

This chapter rotates the analytical perspective found in Part II to focus on the findings of the evaluation studies supported by ATP. It highlights the general principle articulated in Chapter 2: that taking multiple analytical and empirical perspectives and evaluation approaches to complex questions contribute to more reliable, credible, and acceptable findings.

This crosscutting analysis of ATP’s evaluation studies is organized around the following major themes: (1) firm/industry effects, (2) collaboration effects, (3) spillover effects, (4) comparisons and interfaces with other programs, and (5) measures of overall ATP performance, including portfolio analysis, social returns, and impacts on competitiveness. A table at the beginning of each thematic section presents a listing of reports from which the findings are drawn and provides a further breakdown by sub-themes treated in the section.

Direct Private Firm Effects

Private firms play a central role in ATP’s operations. As Spender put it, “ATP harnesses private firms’ resources to the public interest without seeking to outguess market forces.” ²⁹⁴ The term “public-private partnership” further recognizes that the program works only in conjunction with private-sector partners. ATP is designed to attract private firms as partners and to influence their behavior, even as it relies on the special knowledge that firms have of their relevant markets, and on their profit motive and their sharing of the costs to increase program efficiency. It is not surprising, then, that a number of evaluation studies have provided insights as to how and why firms engage with ATP to develop new technologies, how they are affected by that engagement, and the significance of the interactions to the firms. This section summarizes related findings on private firm effects drawn from 13 studies listed in Table 9–1. Besides author and publication date, the table’s column headings indicate the six major sub-themes to be covered. ²⁹⁵

Addressing a “Financing Gap”

Private firms face barriers to innovation that have a long development and commercialization time horizon. The Harvard-MIT study sought to develop a better understanding of “the decision-making process within firms, and within outside financing sources, as it relates to the funding of early-stage, high-risk technology projects...” ²⁹⁶   It took as a point of departure a key premise leading to the establishment of ATP:

There was evidence that firms were systematically under investing in leading-edge technologies and failing to commercialize the products of their own research activities effectively. These concerns, buttressed by academic arguments pointing to a potential market failure in the area of early-stage technological developments, motivated new proposals for the role of government in the innovation system. (p. 1.)

The Harvard-MIT study looked to venture capitalists and others to help define the financing conditions faced by innovative firms. The views expressed by study participants “make clear the existence of a serious gap between the public resources available for academic and national laboratory research and the ability of private venture investors to finance research to reduce the new technical ideas to commercial form...the ‘Valley of Death’ in R&D...” ²⁹⁷

Table 9–1. Studies Informing the Impacts of ATP on Private Firms

According to David Morgenthaler, a past chair of the National Venture Capital Association and a participant in the Harvard-MIT study, “A useful way of looking at technical risk is to assess at what stage of development of a technology an institutional venture capitalist should invest.” He provided as guidance the list below. ²⁹⁸ In his view, there is “little or no role for government in the later stages of development” but in the earlier stages of R&D, where venture capitalists generally do not want to invest, private firms need support by the government. Mogenthaler’s list of why venture capitalists should and should not invest helped to define where government support is needed: Venture capitalists should:

• ... never invest to discover new scientific phenomena
• ... almost never invest to prove the scientific principle
• ... rarely invest to develop an enabling technology
• ... often invest to use a new technology to develop a product
• ... very often invest to revise and improve a product
• ... very often invest to produce a later-generation product
• ... very often invest to broaden a product line
• ... very often invest to apply a product to another application (p. 106)

Morganthaler elaborated as follows:

...it does seem that early stage help by the government in developing platform technologies and financing scientific discoveries is directed exactly at the areas where institutional venture capitalists cannot and will not go. In the analogy of the horse race, the role of the government can be to improve the bloodlines of the horses and give them some preliminary schooling. (pp. 107–108)

In a related study conducted by Harvard University researchers, ²⁹⁹ it was found that:

...most funding for technology development in the phase between invention and innovation comes from individual private equity “angel” investors, corporations, and the federal government—not venture capitalists. (p. 3)

And, further,

...the federal role in early-stage technology development is far more significant than may be suggested by aggregate R&D statistics. In general, … federal technology development funds complement, rather than substitute for, private funds. (p. 10)

The struggle of small private firms to find funding for the innovations they eventually brought to ATP has been documented in case studies. Gompers and Lerner interviewed seven small, innovative companies in the Boston area to find out what role “ATP funding has played in the company’s evolution.” ³⁰⁰ They found reasons each of the companies turned to ATP for funding, and concluded, “The Advanced Technology Program has substantially expanded and enhanced the R&D activities of our seven-company sample.” ³⁰¹ They concluded that despite the growth in venture capital funds during the mid-1990s, less than one tenth of one percent of business startups annually received venture financing. ³⁰²

Servo used the experiences of several firms to describe how ATP funding fit within the landscape of potential funding sources. As recounted by one firm spokesperson in her study, “We sought ATP funding as a means to develop novel process technology which exceeded our existing financial capability and which we were unable to convince venture capitalists and angel investors to back due to high technical risk.” ³⁰³

ATP’s Status Reports also documented circumstances in which firms were unable to obtain private sector financing, and sought ATP assistance in order to accomplish their innovation. For example, in documenting the ability of the two young proprietors of Calimetrics, Inc., to find financing for their pit depth modulation (PDM) technology, the Status Report gave the following account: ³⁰⁴

When first starting out, O’Neill and Wong had confidence in PDM’s technological potential, but the fledgling company did not have the means for sustained research. With a kick-start of $25,000 from family and friends, the company sought funding from venture capitalists and other private sources. All, however, rejected investment in PDM development, deeming the technical and market risk too high. The company faced the classic Catch-22 situation of innovation financing: funding was not available because the technology was unproven, yet it could only be proven if substantial financial resources were brought to bear on the problem. (p. 108)

Detailed case studies also shed light on the financing gap. According to Ehlen, who investigated the joint venture to develop flow-control machining technology, the end users of the technology tend not to fund the research of their suppliers and are unlikely to adopt a new process not yet proven to work; the suppliers lack the capital to do extensive in-house research—“particularly not high-risk research”; and university researchers lack the interest and resources to self-fund such a research program on their own. But with ATP funding, a supply-chain collaboration was formed to undertake the project. ³⁰⁵

Why do companies turn to ATP for funding? Several company spokespeople appearing at the National Research Council’s (NRC) workshops on ATP offered their own explanations. For example, David Ayares of PPL Therapeutics, Inc., a company working on xeno-organ transplantation, said: ³⁰⁶

To raise money, we aggressively pursued deals with several large pharmaceutical companies and with other xenograft companies. We wrote a business plan, which we showed to venture capitalists and private investors. We also pursued joint ventures. None of these efforts bore fruit. Risks and costs were too high, and payoffs too distant for potential investors. (p. 147)

Feldman and Kelley, in their survey study of a large group of 1998 ATP applicants, found that most companies turned down by ATP did not proceed with their R&D projects. If funding were readily available from other sources, it would seem that more of the companies would have been able to pursue their projects without ATP. ³⁰⁷ And, according to Powell and Lellock in their analysis of ATP Business Reporting System (BRS) data, ATP award recipients increased their own investments in their ATP-funded project as a result of obtaining ATP. ³⁰⁸ These findings provide evidence that ATP funding is complementary to, not a substitute for, private sources of R&D funds; that is, that it is filling a funding gap.

Influence on R&D Investment Choices

While some of the evaluation studies focused on the availability of funds and the timing of investment, others investigated the effects of ATP funding on firms’ R&D investment choices. Identified effects included: (1) increased innovativeness of research programs, (2) willingness to commit to longer-term research programs, and (3) ability to tackle larger scale and broader scope problems and opportunities.

Powell and Lellock, in their analysis of BRS data, found evidence of increased innovativeness attributed by firms to ATP. Figure 9–1, for example, indicates, “With the assistance of ATP funding, organizations are pursuing different R&D than they would have undertaken without ATP funding.” ³⁰⁹ Powell and Lellock found that, relative to what the project participants thought would have happened without ATP, 73% of participants in ATP projects “were more willing to accept technical risk” while 67% had “increased interest in performing longterm research” and 61% “had increased the R&D scope.” ³¹⁰

Figure 9–1. Change in the Nature of Industry R&D

Source: Business Progress Reports from 403 organizations in 198 ATP projects funded 1993–1997 — after one or more years of ATP funding.

BRS data also provided specific examples of estimates by project participants of ATP’s effect on their innovativeness. The participants identified the single most important parameter associated with technical success, and then specified the baseline metric, their project goal metric, and the expected value to be attained without ATP assistance, all in terms of the most critical parameter. This approach allowed comparisons. Table 9–2 provides a few examples drawn by Powell and Lellock from the BRS database. One of the examples is for gene sequencing. At the beginning of a project to develop advanced gene sequencing technology, the rate of sequencing was one gene per day. With ATP funding, the goal was to develop the ability to sequence 100 genes per day. Without the ATP award, the participants projected that they could only achieve a sequencing rate of five genes per day within the planned time frame. ³¹¹

Table 9–2. Illustrative Metrics Showing ATP’s Impact on Firms’ Innovation Goals

Source: Drawn from Powell and Lellock, Development, Commercialization, and Diffusion of Enabling Technologies: Progress Report, p. 11.

Feldman and Kelley provided additional evidence that ATP has influenced firms’ R&D investment choices. In their comparison of winners and non-winners, they found evidence that ATP encourages firms to pursue new technical areas, outside the scope of the firm’s past R&D activities. They found that among all applicants, 28% proposed in a technical area new to the company, but of award winners, 47% proposed in a new technical area. They concluded “these differences suggest that the ATP’s cost-sharing partnership with industry is indeed underwriting the efforts of firms...to initiate risky projects in new technical areas.” ³¹²

Halo Effect from ATP Award

As indicated earlier in the report, the existence of a “halo effect” on firms receiving ATP funding has been a recurring theme among project participants. In ATP’s first survey, Solomon Associates found 100% of single-firm award recipients and 60% of joint venture participants reporting that their ATP award had “enhanced the organization’s credibility in some way which resulted in impacts ranging from enhancing the image of the company with potential investors to increased public awareness of the technology.” ³¹³ This was mentioned by many of the respondents as “the single most important effect that the ATP award has  had on your organization thus far....” ³¹⁴

In ATP’s second major survey, which focused on participants in the first three ATP competitions, Silber & Associates replaced halo terminology with “increased credibility.” It concluded that 90% of participants benefited to a “great” or “moderate extent” from enhanced credibility associated with the award. ³¹⁵ According to Silber’s report, “Credibility is almost equal in importance to the direct financial impact of the award, particularly since increased credibility can mean increased business activity.” ³¹⁶

The on-going BRS survey of ATP project participants used the terms “increased credibility” and “halo effect” interchangeably. Powell and Lellock’s analysis of BRS survey data showed 93% of participants citing increased credibility due to their ATP award. ³¹⁷ According to the researchers, “The ‘halo effect’ may be expected to be of particular benefit to ATP-funded small businesses, which have little if any market presence and typically very limited financial resources at the time of the ATP award.” ³¹⁸

Among five major ATP effects Laidlaw identified as helping project participants, was a halo effect. ³¹⁹ Laidlaw’s reporting of interviewee comments suggested the connection between the ability of firms to carry out high-risk research, as discussed in the previous section, and the halo effect:

We didn’t round up private funding until after we got ATP funding. It’s a high-risk project, and we had previously operated by bootstrapping, which would not have lasted long. (Laidlaw, quoting an interviewee, p. 30)

Feldman and Kelley’s survey of 1998 winners and non-winners rigorously tested the hypothesis that ATP confers a halo effect. ³²⁰ To do this, they took into account other factors that could have explained improvement in the firms’ ability to attract funding from other sources. As stated in the NRC’s condensed version of the Feldman and Kelley study, ³²¹

Our analysis concludes that winning an ATP award significantly increases the firm’s success in attracting additional funds from other sources for R&D activities. Our findings provide strong evidence that the ATP award confers a halo effect on winners that makes them more likely to attract other funding when compared to non-winners of the same size, and age with projects of similar business and technical quality. Thus, our results confirm that the ATP award appears to send a market signal that certifies that the firm and the technology are promising. (p. 207.)

Acceleration of Technology

Because helping U.S. businesses accelerate development and commercialization of technology is a legislated mandate for ATP, a number of evaluation studies have investigated and measured this effect. The term “acceleration,” as used here, encompasses a wide scope of changes in timing, ranging from months to years, to cases where the technology would not have been developed at all without ATP. And, as noted by Laidlaw:

...it appears that the ATP is helping to overcome two types of economic efficiency problems related to speed to market: 1. the difficulty of obtaining funding at all to undertake long-run, high-risk, enabling technology development; and 2. the difficulty of implementing coordinated, collaborative R&D management practices needed to speed the conduct of R&D and the commercialization of the resulting technology. (p. iii)

Surveys of project participants provided the earliest evidence that participating in ATP was allowing firms to speed up technology development and commercialization. In the first ATP survey, Solomon Associates found that 69% of participants had saved time in developing their technology—usually in the one to five year range—and 35% of these considered the time saved of critical importance. ³²² Three years later, Silber & Associates found 95% of participants reporting a shortened R&D cycle because of ATP, with most reducing their R&D cycle by at least two years. ³²³ ATP’s BRS, which has regularly tracked acceleration, added questions about the participants’ window of opportunity to questions about the amount of time saved and the importance of saving time. In a further departure from the earlier surveys, the BRS looked at acceleration in terms of the various applications of the technology, making it not strictly comparable to the Solomon and Silber results.

Figure 9–2, reproduced from a BRS report, ³²⁴ shows the three aspects of timing tracked by this survey: (1) the estimated reduction in time to market, (2) importance of speed, and, to put time into perspective, (3) the perceived length of the window of opportunity. As Powell and Lellock observed:

...nearly all the companies expect some reduction in the time it will take to complete the R&D phase and bring their products to market/or implement new production processes as a result of ATP funding... A reduction of at least two years is anticipated for 65% of all applications... The importance of speed-to-market is considered ‘important’ or ‘critical’ for 98% of applications... Further emphasizing the importance of acceleration, the window of opportunity for 73% of the applications to enter the marketplace is considered to be within two years after ATP funding ends; i.e., it appears that companies believe they would miss the opportunity, or a significant part of it, without the acceleration enabled by ATP funding. (pp. 11–12)

Figure 9–2. Importance of Timing

Source: Business Progress Reports for 974 applications being pursued by 464 companies in 258

The Feldman-Kelley survey provided additional evidence of an acceleration effect of ATP on R&D. As shown by Table 9–3, winners proceeded with their proposed R&D, while less than two-fifths of the study’s control group of non-ATP-winners had started any aspect of the R&D project they proposed to ATP a year after being unsuccessful in the ATP competition. ³²⁵ The results implied that the winners also might not have started without ATP, but otherwise provided no specific estimates of time saved.

Table 9–3. The Extent to Which Non-Winners Pursue the Proposed R&D Project Without ATP Funding

*Companies reporting more than one type of alliance are included twice.

Source: ATP’s Business Reporting System. Data for 747 applications being pursued by 356 companies in 198 ATP projects funded 1993–1997—after one or more years of ATP funding.

Laidlaw’s interview-based analysis of the acceleration effect produced a clearer understanding of how and why ATP caused acceleration to take place, and even placed rough values on it. In her interviews with ATP award recipients, she asked why saving time is important to the firms. She investigated how the firms actually saved time through participating in ATP and whether the effects extended throughout the firm, beyond the ATP-funded project. ³²⁶

Laidlaw found that 96% of interviewees assigned a rating of “very important” to reducing cycle time, in order to meet competition. ³²⁷ When asked how much time they had saved due to ATP participation, the median response was “by 50% or three years.” ³²⁸ Table 9–4, from the study, represents Laidlaw’s attempt to obtain quantitative estimates of the dollar value of a year of time saved. For the 54% of surveyed firms responding with quantitative estimates, “The median estimate of the economic value of reducing the applied research cycle time by just one year is $5 million to $6 million.” ³²⁹

Table 9–4. Estimates of Economic Value of a One-Year Reduction in Applied Research Cycle Time, in Order of Decreasing Value

Firms interviewed by Laidlaw identified five principal factors that helped them cut time. These are summarized in Table 9–5. The most frequently mentioned factor was “ATP’s requirement for an integrated business and R&D plan with its emphasis on concurrent engineering…” ³³⁰ Spokespeople mentioned the integrated planning requirement more often than ATP’s funding as an important factor in accelerating their technology development. Elaborating, firms spoke of the positive effect on their timing of having a well-laid-out plan and following the plan without interruption, the stability lent by ATP’s involvement, early initiation of work with potential customers and users, integrating the voice of the customer, using a systematic approach, developing definable timelines, benchmarking, and selecting potential technology applications early in the process, and enhancing documentation procedures.

Table 9–5. ATP Effects that Helped Interviewees to Reduce Cycle Time

But, if ATP’s funding effect is defined more broadly—taking into account both ATP’s direct funding plus the improved ability of firms to attract additional financial assistance from others through ATP’s “halo effect”—funding is the effect most frequently mentioned by firms as speeding their technology projects. ³³¹

Laidlaw found that 86% of those interviewed expected the time saved in the R&D stage to flow through to later project stages. ³³² Thus, speeding R&D typically speeded commercialization.

She found that 86% of firms found a carry-over of time improvements to their other technology development projects. Table 9–6 summarizes the responses, identifying ways the time reductions were transferred to other projects.

Table 9–6. Carryover of Cycle-Time Improvements to Other Projects

Source: Adapted from Laidlaw, Acceleration of Technology Development by the Advanced Technology Program, 1997, Table 10.

Developing enabling, generic technology—”technology platforms”—reportedly was a major factor allowing firms to speed other projects involving multiple applications of the ATP-funded technology. Applying the ATP-required integrated R&D and business planning to other projects reportedly also helped firms speed other projects. The ability to apply methodologies and processes developed in ATP projects to other projects was another factor identified as helping firms speed their other projects. Some firms further reported that they had developed a company culture emphasizing speed as a result of their ATP project participation. ³³³

Project case studies provided another look at ATP-induced acceleration of technology  development. The RTI case studies, for example, estimated project acceleration ranging from “one to at least 10 years” for the seven tissue engineering projects included in the study. ³³⁴ In a more complicated case, the firms in the ATP-funded Printed Wiring Board joint venture reportedly would have delayed about half their research by at least one year without ATP’s assistance, and would not have undertaken the other half at all. ³³⁵

The set of mini-case studies for the completed projects provided yet another look at acceleration. Table 9–7 summarizes the acceleration effects of ATP funding for 44 of the first 50 completed projects, those for which data were available. ³³⁶ The majority of participants in completed projects reported they would not have proceeded at all without ATP funding. ³³⁷

Table 9–7. Effect of ATP Funding on Completed Projects

Source: Advanced Technology Program, Performance of 50 Completed ATP Projects, Status  Report 2, 2001, p. 24.

Increased Firm Productivity

ATP’s evaluation investigated program-induced changes in patenting by award recipients to gauge program impact on firm research productivity. Two of the evaluation studies examined—the Darby et al., study and the Sakakibara- Branstetter study—provided statistical evidence about how participation in ATP affected patenting behavior. ³³⁸

In determining how to best measure the impact of ATP participation, Darby et al., identified patents as “arguably the single best measure of commercial capture of  advanced technology.” ³³⁹ They found this to be an appropriate measure because a “major purpose of ATP is to increase commercial capture of advanced technology.” ³⁴⁰ They produced an adjusted patent count measure by deflating the patent count of ATP participants using a factor for year-to-year changes in the average rate of patenting for all U.S. assignees of U.S. patents. ³⁴¹ They estimated an increase in patenting averaging between five and 30 patents per firm per year of participation, attributed to ATP.

In addition to estimating an increase in firm patenting due to ATP participation, Darby et al., concluded that ATP-funded organizations accounted for an unusually high percentage of all U.S. patents granted to U.S. entities during the period 1988–1996. They did not attribute this fact to ATP, but rather suggested it signals “the very top, leading edge technology firms are getting involved [with ATP] as well as the top universities.” ³⁴³

The Sakakibara-Branstetter study also examined the impact of participating in ATP projects on the overall research productivity of firms that participated in ATP-sponsored joint ventures, again using patent data as a proxy for productivity. ³⁴⁴ Their study compared ATP participants with a control group over a common time period, rather than taking the before-and-after approach of the Darby et al., study.

Sakakibara and Branstetter’s findings suggested that participation in ATP raises the research productivity of participating firms. ³⁴⁵ They found that participating in ATP joint ventures increased patenting in the targeted technology areas above the level of patenting that participating firms showed prior to participating in the project. They estimated that, at the margin, each instance of participation in an ATP project raised a firm’s research productivity, as measured by patents, by 8% per year. ³⁴⁶ They found evidence that consortia in which participating firms are technologically proximate were systematically more successful than consortia in which participating firms are involved in diverse technologies. ³⁴⁷

Reflecting the attention given in ATP’s authorizing legislation to small businesses, ATP’s evaluation program has addressed the following question: “Have small firms been able to compete successfully against larger firms for ATP awards?” Powell analyzed BRS data to help answer the question. The data showed that the majority of ATP participating companies, including subcontractors, are small, and that 61% of awards have gone to projects led by small firms. ³⁴⁸

Source: Powell, Business Planning and Progress of Small Firms Engaged in Technology Development through the Advanced Technology Program, 1999, p. 45.

“New technical knowledge must be used if economic benefits are going to accrue to the nation. This generally means the introduction into the market of a new product or process by the innovating firm, its collaborators, or other companies that acquire the knowledge.” ³⁵⁰

Innovating firms making progress toward commercialization may experience growth, increased capitalized value, higher sales, increased revenue, and return on investment. Their close collaborators and licensees are also positioned to make early commercial progress. The activities of the awardees, their collaborators, and licensees comprise ATP’s “direct path to impact.” ³⁵¹

Commercial Progress

Table 9–9 summarizes the commercialization progress of the first 50 completed ATP projects. Examples of the early technologies and the products and processes developed from them are given in Table 9–10. ³⁵² The individual project summaries in the Status Report provide accounts of each innovator’s specific progress toward taking their technology further into commercialization.

Table 9–9. Progress of Participating Companies in Commercializing New Technologies

Source: Advanced Technology Program, Performance of 50 Completed ATP Projects, Status Report 2, 2001, p. 17.

Employment Gains Among Small Innovating Firms

Company growth, as indicated by employment increases, is easier to isolate among ATP participants by examining small, single-applicant company recipients of ATP funding rather than large, multi-divisional firms, joint ventures, or nonprofit organizations. Employment changes in larger companies, non-profits, and joint ventures are too complex to link directly to the ATP project. But for small companies, “Rapid growth is generally a signal that the small innovating company is on the path to taking its technology into the market.” ³⁵³ The Status Report provides an overview of employment changes experienced by small companies included in the first 50 completed projects. Figure 9–3 shows the distribution of small companies by their percentage change in employment.

Figure 9–3. Employment Change at 31 Small Companies Receiving a Single-Company Award

Source: Advanced Technology Program, Performance of 50 Completed ATP Projects, Status Report 2, 2001, p. 18.

As the Status Report points out, “Sixty percent have at least doubled in size; four of them have grown more than 1000 percent.” ³⁵⁴

Returns to the Innovators

Returns to innovators can be measured in several ways. For the seven firms that carried out the tissue engineering projects—the technology focus of the RTI studies—the authors computed a composite measure of direct company returns. Although they calculated a measure of private returns for each project, to preserve proprietary information, they presented the results as a composite. The estimated composite internal rate of return (IRR) to the companies was 12%.

Consistent with ATP’s goal of generating broad-based benefits, the estimated composite private return was much lower than the composite social return of which it is a part. ³⁵⁵

Pelsoci’s case study of closed-cycle air refrigeration (CCAR) estimated revenues to private company innovators. ³⁵⁶ The study projected the present value of projected revenues from CCAR installations in the food service markets, volatile organic compound market, and liquid natural gas industries to total $65 million. (The IRR was not provided.)

Table 9–10. Examples of Products and Processes from the First 50 Completed ATP Projects

Source: Extracted from Advanced Technology Program, Performance of 50 Completed ATP Projects, Status Report 2, 2001, Appendix A, pp. 253–258.

Finally, Ehlen’s case study of flow-control machining technology estimated increases in annual sales attributable to the project, focusing on three groups: (1) the main technology developer, Extrude Hone Corporation; (2) the first-line users of the technology, a group of aluminum casters; and (3) the automakers who would purchase the affected engines and install them in vehicles. ³⁵⁷

As Table 9–11 illustrates, for this process technology, the developer’s change in annual sales, shown in the bottom row, was relatively small. Again, the findings are indicative that the benefits to the innovator are much less than estimated social benefits.

Table 9–11. Impact on Industries of Near-Term, Five-Year Implementation Path for Flow-Control Machining Technology

Source: Ehlen, Economic Impacts of Flow-Control Machining Technologies: Early Applications in the Automobile Industry, 1999, p. 45.

Collaboration Effects

One of ATP’s legislated mandates is to “aid industry-led United States joint research and development ventures...” Evaluation studies have variously focused on the structure and formation of new ventures, the role of universities in joint ventures, factors determining their success, their stability over time, and their benefits and costs. This section draws findings from 10 studies listed in Table 9–12. The discussion is organized along the lines of the sub-themes identified in the last five column headings of the table.

Table 9–12. Studies Extending Knowledge about Collaboration Activities

Collaborative Activity, New Venture Formation, and Attribution to ATP

Table 9–13. Summary of Study Findings on Frequency of Collaboration

The earliest survey of ATP participants by Solomon Associates, published in 1993, found that 46% of those companies interviewed after one year in the program had formed strategic alliances to further advance the technology associated with their ATP project. Some others were in the process of forming collaborative relationships, and others said it was “too early.” Only one company indicated no collaboration and no near-term plans to collaborate. ³⁵⁸

While the collaborative activities identified by Solomon were linked directly to ATP projects, the survey made no attempt at establishing a counterfactual comparison. Rather, it merely reported the frequency of collaboration associated with ATP projects, and captured some participants’ comments concerning the benefits.

Silber & Associates’ 1995 survey measured the frequency of collaborative activity, and also identified the nature of the collaborations, the number of partners, the degree of success, and the extent to which the relationships were new ones, “formed expressly to carry out the ATP project.” Silber concluded: ³⁵⁹

The ATP projects fostered new relationships among U.S. companies. Only one joint venture involved a consortium of companies that previously worked together. According to the participating companies, the others were formed expressly for the ATP. (p. 23)

The Silber survey specifically investigated the extent of collaborative activities among single-company applicant projects, finding that 52% of the single company applicants had formed collaborations and that they brought an average of four outside companies into their projects as subcontractors. The study found that each joint venture project included an average of six members, and 43% of the joint venture members “forged subcontracting relationships with an average of five additional companies.” ³⁶⁰

Powell and Lellock, from their analysis of BRS data, provided further evidence that collaborative activities are extensive among ATP projects, and that ATP played an important role in stimulating the collaborative activities. Analyzing data from 415 participants in 198 ATP projects—including single-company applicants and joint ventures—they reported that of the 86% of respondents reporting collaboration, 69% reported that ATP “to a great extent” was responsible for bringing about the collaboration. ³⁶¹ Powell and Lellock also