Part 4 - Measures and Results

Dependent Variables

To provide a robust and comprehensive assessment of success, this study incorporates three distinct dimensions of the benefits to firms collaborating in R&D alliance—perceptual measures of success, patent measures, and financial measures. First, perceptual measures allow participants to qualitatively assess the extent to which the alliance achieved success in R&D collaboration. Perceptual measures have been found to be important for measuring success in R&D alliances because success in R&D alliances often means development or acquisition of knowledge as the collaborating firms pursue technology objectives (Anderson and Narus, 1990; Tuchi, 1996; Hebert and Beamish, 1997; Sakakibara, 1997). Also, additional in-house R&D is usually necessary to fully benefit from the outcomes of R&D alliances, making it difficult to evaluate alliances from their tangible results alone (Sakakibara, 2001). Use of managerial assessments of alliance performance received some initial criticism on concerns about potential bias and inaccuracy. But research by Geringer and Hebert (1991) shows that there is high correlation between subjective assessments of performance and objective financial measures based on accounting data. Thus, if properly conducted, managerial assessments of alliance performance are a reasonable way to measure alliance performance (Anderson and Weitz, 1989; Anderson, 1990; Child and Yan, 1999; Das and Teng, 2000). We use a survey-based measure for the firm's assessment of the overall success of the R&D alliance in terms of delivering value to the firm (Overall_Value).

Patents have been widely used as a measure of R&D output. We use patent application by the firm (Patent_Application) as a measure of outcomes from the research joint venture project. With indicator variables to control for differences in the propensity to patent across technology fields, patents are a good measure of the creation and formal ownership of intellectual property. Finally, we use an indicator of financial value from technology commercialization or implementation that the firm has already realized as a result of the research joint venture project. This measure (Commercialization) indicates whether the firm has received actual revenues from sales or licensing, or achieved actual cost savings, from commercializing or implementing technology developed in the R&D alliance. During interviews we were told that the most successful projects not only achieve technical objectives but also result in commercialized technology that generated some financial value for the company. While the amount of value may vary with a variety of factors (e.g., strength of competition, willingness of customers to adopt new technology, emergence of alternative technologies, etc.), we consider any positive value created from the research joint venture project to be an indicator of the success of the project.

Table 3 presents a cross-tabulation of the three outcomes variables used in this study. The five response categories for Overall_Value are shown in the header column, and the four possible response combinations for Patent_Application and Commercialization are shown in the header row. From the top three rows of the table, we see that firm respondents may have a poor or neutral perceptual assessment of overall value of the alliance, even while the firm may have generated patent applications or financial value from technology commercialization. From the bottom two rows, we see that firm respondents may have a positive perceptual assessment of the overall value of the R&D alliance, even if the firm has not (yet) generated tangible outcomes such as patent applications or financial value from technology commercialization.

Independent Variables

To empirically test the hypotheses outlined earlier, we construct variables to be employed in regression analysis. The independent variables include both project-level (i.e., alliance-level) variables and firm-level variables. We use both administrative data and survey data in constructing these variables. From administrative data, we construct the following project-level variables:

• Proj_Company_Count is the number of company partners in the alliance;
• Proj_Company_Distance is a measure of the geographic distance between company partners;
• Proj_Total_Budget is the total project budget for the research joint venture project;
• Proj_Tech_Bio, Proj_Tech_Chem, Proj_Tech_Elec, Proj_Tech_IT are indicator variables for the technology area of the project, i.e., Biotechnology, Chemistry/Materials,
  Electronics, Information Technology;
• Proj_Endyear_00_02, Proj_Endyear_03_04 are indicator variables for the end year of the project, i.e., 2000-2002, 2003-2004.
  

Firm-level variables constructed from administrative data include:

• Lead_Company is an indicator for the formal lead company for the research joint ven ture;
• Small_Company is an indicator that the company has fewer than 500 employees.

From survey data, we construct the following project-level variables:

• Proj_Ambitiousness is a measure of the ambitiousness of the R&D alliance project goals relative to other R&D initiatives in the industry;
• Proj_Current_Competitor is an indicator for whether the alliance includes companies that are competitors in product markets;
• Proj_Contract_Governance is a measure of satisfaction with the formal alliance agreement and governance procedures developed in the alliance;
• Proj_Goodwill_Trust is a measure of goodwill trust among the alliance partners.

Firm-level variables constructed from survey data include:

• General_Alliance_Experience is a measure of company project personnel's previous experience with inter-firm R&D collaborations in general;
• Partner_Specific_Experience is a measure of company project personnel's prior experience working with personnel from selected alliance partners in the current collaboration;
• Frequency_Communication is a measure of the frequency of communication by company project personnel with personnel from alliance partners;
• Joint_Work_Interaction is a measure of company project personnel's interaction with personnel from alliance partners to carry out joint work on project tasks;
• RD_Employment is the number of R&D employees at the company location;
• FTE_Technical is the number of company full-time equivalent technical staff that work on the project, on average per year.

Regression Results

The variables employed in regression analysis are displayed in Table 4, which also provides additional details on the definition and construction of the variables. Table 5 provides summary statistics for the variables. The average firm in the sample has a perceptual measure of the Overall_Value of the alliance outcomes that is above the neutral value of "3", which represents a survey response that the alliance was "neither successful nor unsuccessful" in delivering overall value to the company. The mean values of Patent_Application and Commercialization indicate that 37 percent of firms in the sample filed a patent application, and 29 percent of firms achieved revenues or cost savings from commercialization of technology developed in the alliance.

Table 6 presents the matrix of correlation coefficients for the variables used in the regression analysis. The three outcomes variables Overall_Value, Patent_Application, and Commercialization are correlated with each other. The three outcomes variables are each positively correlated with Proj_Ambitiousness, Frequency_Communication, and FTE_Technical. Two of the outcomes variables—Overall_Value and Patent_Application—are negatively correlated with Proj_Company_Count and Proj_Current_Competitor, and positively correlated with Proj_Con-tract_Governance and Lead_Company.

Notes:
* indicates statistical significance at the 0.10 level
** indicates statistical significance at the 0.05 level
*** indicates statistical significance at the 0.01 level
Source: Advanced Technology Program, Survey of ATP Joint Ventures

Among the independent variables, we find that Frequency_Communication is negatively correlated with Proj_Current_Competitor, and positively correlated with Lead_Company and FTE_Technical. We find that Proj_Contract_Governance is negatively correlated with Proj_Company_Count and Proj_Company_Distance, and positively correlated with General_Alliance_Experience and Frequency_Communication. We find that Proj_Goodwill_Trust is negatively correlated with Proj_Current_Competitor, Partner_Spe-cific_Experience, and Proj_Company_Distance, and positively correlated with Proj_Com-pany_Count, Frequency_Communication, and Proj_Contract_Governance.

Tables 7, 8, and 9 present the results from multiple regression analysis used to test the hypotheses presented earlier. The three tables present results for the three outcomes variables Overall_Value, Patent_Application, and Commercialization, respectively. Equations (1abc) provide alternative specifications for testing Hypothesis (3) and (7ab) relating to geographic distance of partner companies, and the frequency of communication and interaction. Equations (2ab) provide alternative specifications for testing Hypothesis (4ab) relating to general and partner-specific alliance experience. Equations (3) and (4) present alternative specifications for testing Hypothesis (5) and (6), respectively. The other variables of interest for testing the hypotheses are included in all of the regression specifications.

Table 7 presents regression results for the outcome variable Overall_Value. The coefficient for Frequency_Communication is consistently positive, supporting the hypothesis that frequent communication is a key alliance management factor for success. Equations (1abc) show that geographic distance and face-to-face work interaction have no effect, and therefore the frequency of communication is independent of geography and face-to-face interaction. Equations (2ab) show that the prior alliance experience variables do not have an effect on the perceptual measure of overall value. The coefficient for Proj_Contract_Governance is consistently positive, which supports the hypothesis that contractual provisions and governance procedures are important to alliance success. Somewhat surprisingly, we find that Proj_Good-will_Trust has a marginally significant negative coefficient in some equation specifications. Also contrary to expectations, we find in Equation (3) that R&D_Employment has a negative coefficient. In Equation (4), we find no effect of FTE_Technical on the perceptual measure of outcomes. We find in all specifications that the number of companies in the alliance, and the presence of competitors, do not have an effect on the perceptual measure of overall value. This finding suggests that alliance designers are successful in structuring the alliance to have the appropriate number and types of partners. The coefficient for Proj_Ambitiousness is positive in all specifications, which indicates that projects with ambitious goals deliver greater perceived value to the participants. The coefficient for Lead_Company is also generally positive, indicating that more active or leading members of an alliance are more likely to experience successful outcomes.

Table 8 presents regression results for the outcome variable Patent_Application. Equations (1abc) show that frequency of communication has a positive effect on the outcomes variable, while geographic distance and face-to-face interaction do not have any effect. Contrary to expectations, we see in Equations (2ab) that General_Alliance_Experience and Partner_Spe-cific_Experience have negative coefficients. Proj_Contract_Governance has a marginally statistically significant positive effect in some specifications, and Proj_Goodwill_Trust has a statistically significant negative effect in most specifications. Equation (3) shows that R&D_Employment has no effect, and Equation (4) shows a marginally significant positive coefficient for FTE_Technical. The number of companies in the alliance, indicated by Proj_Com-pany_Count, has a marginally significant negative effect in most specifications, while the presence of competitors has no effect. Proj_Ambitiousness has a statistically significant positive effect in most specifications, and Lead_Company has a positive effect in all specifications.

Notes:
* indicates statistical significance at the 0.10 level ** indicates statistical significance at the 0.05 level *** indicates statistical significance at the 0.01 level

Table 9 presents regression results for the outcome variable Commercialization. Again, we find the coefficients for Frequency_Communication and Proj_Ambitiousness to be consistently positive. In Equation (4), we find that FTE_Technical has a positive coefficient, indicating that greater allocation of technical personnel to the project increases the likelihood of successful outcomes. We do not find the other main variables of interest to have any effect on the commercialization outcomes variable.

Notes:
* indicates statistical significance at the 0.10 level
** indicates statistical significance at the 0.05 level
*** indicates statistical significance at the 0.01 level

Return to Table of Contents or go to next section of interim report.

Date created: August 29, 2006
Last updated: September 13, 2006

ATP website comments: webmaster-atp@nist.gov  / Technical ATP inquiries: InfoCoord.ATP@nist.gov.

NIST is an agency of the U.S. Commerce Department
Privacy policy / Security Notice / Accessibility Statement / Disclaimer / Freedom of Information Act (FOIA) /
No Fear Act Policy / NIST Information Quallity Standards / ExpectMore.gov (performance of federal programs)