11. Survey of ATP Joint Ventures: Methodology and Response Rates

Joint ventures have unique characteristics and may differ from single company projects in a number of important ways. Beginning in the Spring of 2003, ATP conducted a survey of all companies and nonprofit organizations that embarked on an ATP-funded joint venture project between 1991 and 2001. The Survey of ATP Joint Ventures explores the characteristics of joint ventures and factors leading to project success.

Survey Development

ATP contracted with Westat, a survey services firm, to assist with survey development, implementation, and administration. ATP provided Westat with draft questions that were based partly on prior surveys of ATP-funded companies and partly on research hypotheses of interest to ATP. Two versions of the survey were developed: one for companies and one for nonprofit organizations.¹ Universities and federal laboratories participating in ATP joint ventures were not surveyed.

A small pretest of companies actively participating in ATP-funded projects explored whether items might be interpreted in different ways by different respondents, and whether certain items were especially difficult to accurately comprehend and answer. Respondents were asked to complete a draft copy of the questionnaire prior to the interview. During the interview, respondents were asked a series of specific probes to clarify their interpretation and response to certain questions. No major revisions to the content or form of the survey resulted from the cognitive interviews conducted for the pretest.

Data Collection

The survey used a mixed-mode methodology of web and telephone administration. Although both modes were available, the majority of respondents completed the web version of the survey. Attempts to follow-up with nonrespondents with requests to do a telephone interview typically resulted in the respondent completing the web survey rather than a telephone version of the survey.

All companies and nonprofit organizations that embarked on an ATP-funded joint venture projects between 1991 and 2001, and for whom a respondent could be located, were included in the sample. Initially, 486 companies and 105 organizations were identified as potential members of the survey sample.

Over the course of the survey fielding, 54 of these companies or organizations were removed from the eligible pool as it became clear that there was no knowledgeable respondent for a particular company due to business closings, staff turnover, or aborted projects. This resulted in a total of 537 eligible respondents representing 142 projects.²

Data collection was carried out from May 1, 2003 through November 15, 2003. Following standard survey procedures, multiple contact attempts were made in order to maximize survey response rates. Advance letters describing the purpose of the survey were mailed to company contact persons associated with each ATP joint venture project. One week later, email letters reiterating the survey's purpose and providing login information were sent to these same contact persons. Three reminder emails were sent to nonrespondents over the course of several weeks. After eight weeks, Westat began contacting nonrespondents by telephone to collect the survey data.

Survey Response Rates

Of the 537 profit and nonprofit companies/organizations eligible to respond, a total of 479 surveys were completed³, for an overall response rate of 89%. Among 443 eligible for-profit companies, 397 provided responses (including 10 by telephone) yielding a response rate of 90%. These 397 respondents represented all 1 42 projects funded by ATP during this period. Of the 94 eligible nonprofit organizations, 82 provided responses (including 4 by telephone) yielding a response rate of 87%. The 94 respondents from nonprofit organizations participated in 50 joint venture projects (see Figure 11.1).

Figure 11.1 - Comparison of the number of respondents to the Survey of ATP Joint Ventures versus the eligible sample (start years 1991-2001)

Response Rate Comparisons

If respondents from companies and organizations that completed the survey differ in some way from those that did not respond, these differences might create biased survey results. To evaluate the possibility of nonresponse bias, response rates were compared by:

• Company size
• Project age
• Technology area
• Amount of project award
• Joint venture size
• Project duration

Statistical testing on the response rate differences in each of the above comparisons revealed few significant differences. This suggests that there is little nonresponse bias in the survey, at least with respect to these characteristics.

Company size

Response rates are similar across companies, regardless of company size.

• Large companies are Fortune 500⁴ companies, small companies are those with fewer than 500 employees, and all other companies were placed in the medium category (see Table 11.1).

Project age

Company partners on more recently concluded projects were more likely to respond (see Figure 11.2).

• Respondents associated with joint venture projects that ended 4 to 9 years prior to survey administration were less likely to complete the survey than were respondents whose projects ended 1 to 3 years prior to survey administration (89% and 85% vs. 96%, respectively).
• This general pattern, however, did not hold for respondents who represented projects that were due to end in 2004. Respondents in this group were equally likely to respond to the survey as those whose projects ended 1 to 9 years prior to survey administration.

Figure 11.2 - Response rates by joint venture project end year

Technology area

Response rates were similar across project technology areas.

• Statistical testing on the differences across technology areas revealed no significant differences, suggesting nonresponse bias does not vary by the project’s technology area (see Table 11.2).

Amount of project award

Response rates showed little variation by ATP award size.

• The observed response rates vary by less than 6%, suggesting that the amount of the ATP award had little bearing on a respondent’s willingness to complete the survey (see Table 11.3).

Joint venture size

Company partners in smaller joint ventures were more likely to respond.

• Respondents associated with larger joint venture projects, in general, were less likely to complete the survey than respondents associated with smaller joint venture projects.
• Respondents from joint ventures with 2 partners were significantly more likely to complete the survey than respondents from joint ventures with 3 to 5 partners or with 11 or more partners.
• Statistical testing on the four remaining possible comparisons showed no significant differences (see Figure 11.3).

Figure 11.3 - Response rates by number of joint venture partners

Project duration

Response rates varied little by project duration.

• Statistical testing of the differences in project duration, measured in years, revealed no significant differences, suggesting that respondents were equally willing to complete the survey regardless of project duration (see Figure 11.4).

Figure 11.4 - Response rates by number of joint venture partners

____________________
Note: Only statistically significant results are reported in this publication.

1. Given the similarity of the two surveys, they are referred to as a single survey throughout this publication. However, statistical highlights presented in the accompanying factsheets are based on the responses of for-profit companies, except where otherwise specified.
2. Organizations may participate in more than one ATP project. For example, a company may be a joint venture lead in one ATP project and a joint venture participant in another ATP project.
3. It was possible that no eligible member of a particular joint venture would respond to the survey. This possibility, however, did not occur. Thus, of the 537 eligible respondents, the 479 actual respondents still represented 142 joint venture projects.
4. For ATP, small companies have fewer than 500 employees, large companies are Fortune 500 companies, and medium size companies are all others.

Return to Table of Contents or go to next section of Survey.

Date created: August 2, 2006
Last updated: September 1, 2006

ATP website comments: webmaster-atp@nist.gov  / Technical ATP inquiries: InfoCoord.ATP@nist.gov.

NIST is an agency of the U.S. Commerce Department
Privacy policy / Security Notice / Accessibility Statement / Disclaimer / Freedom of Information Act (FOIA) /
No Fear Act Policy / NIST Information Quallity Standards / ExpectMore.gov (performance of federal programs)