NIST GCR 05–873
Customer Satisfaction Findings from the Advanced Technology Program’s
Survey of ATP Applicants 2002

1.  Introduction

Effective organizations monitor customer satisfaction, and the Advanced Technology Program (ATP) is no exception. ATP has several types of customers. Ultimately, ATP’s customers are the U.S. taxpayers who enjoy the benefits of the innovative technologies that ATP funds. In addition, ATP’s direct customers are the organizations that ATP interacts with on a daily basis during the course of project selection and program management. This group of customers includes all applicants to ATP competitions, including both applicants who receive funding from ATP and those who do not.

ATP is a partnership between government and industry that awards funding for the development of innovative, high-risk technologies that have the potential to create widespread social and economic benefits. ATP selects projects for funding in accordance with rigorous technical and economic criteria. ATP has held 44 competitions from 1990 through 2004, providing $2.3 billion in awards, while industry has provided an additional $2.1 billion as cost share, for a total of $4.4 billion of high-risk research.

Companies apply to ATP in response to an announced competition. They may apply either as single company applicants or as joint ventures. Single applicants can receive up to $2 million over three years, and joint ventures can receive ATP funding for up to five years, with no funding limitation other than the announced availability of funds. A joint venture must include at least two separately owned, for-profit companies and may also include universities, national research labs or non-profit organizations. Of the 768 projects awarded to date, 550 were to single company applicants and 218 were to joint ventures. More than 165 universities and 30 national laboratories participate in ATP projects either as subcontractors or as partners on joint ventures.

From January through July, 2004, ATP conducted a Survey of ATP Applicants to its 2002 competition. One section of the survey was devoted to customer satisfaction issues. The customer satisfaction questions addressed applicant perceptions and experience during the proposal preparation and review process. A total of 17 questions with 27 survey items covered the following topic areas:

• Applicant perceptions of the proposal process, including the ATP decision process;
• Satisfaction with the Proposal Preparation Kit provided by ATP and the newly introduced Electronic Submission System;
• The usefulness of information sources about ATP, such as the ATP website, proposers’ conferences, information booths and industry or company colleagues;
• Courtesy, promptness and help received from ATP staff;
• Satisfaction with timeliness, clarity and comprehensiveness of the debriefing received by applicants who were not selected for an award;
• Company time and cost devoted to proposal preparation.

Staff from ATP and Westat, a survey research firm under contract to ATP, collaborated in developing the survey. The customer satisfaction survey incorporates questions from a 2001 customer satisfaction survey conducted for ATP by the U.S. Census Bureau and questions on applicant perceptions and time and cost for proposal preparation from the Survey of ATP Applicants 2000. The similarity of survey items in the 2002 and 2000 Survey of ATP Applicants enables comparison of survey results from the 2002 and 2000 competitions. Where such comparisons can be made, they are presented in this report. Results were the same or very similar between the two competition years.

Methodology

Virtually all companies applying for funding in the year 2002 award competition were included in the survey. A limited number of company applicants were considered ineligible (e.g., companies that submitted incomplete proposals, companies that withdrew from awarded projects, and those whose funding awards were delayed until May, 2004). Joint venture partners that were not for-profit companies (such as universities, national laboratories or non-profit organizations) were not included in the survey. Altogether, 891 applicants were eligible to respond to the survey, including 144 companies that were selected for an ATP award and 747 companies that were not selected for funding.

Proposals for the 2002 competition were accepted in three batches (June, August, and September). Proposals that did not meet the criteria for funding in the first two batches could be resubmitted in a later batch. In addition, some applicants submitted proposals for more than one project. Since we did not want to burden applicants by asking them to respond for multiple proposals, we developed the following rules for these situations:

• Awardees submitting more than one proposal (either due to resubmission or multiple projects) were asked to respond to the survey questions based on the proposal awarded funding;
• Nonawardees who submitted a proposal in more than one batch were asked to respond based on their most recent proposal;
• Nonawardees who submitted proposals for more than one project were asked to respond based on one proposal that was randomly selected.

Data Collection

Data collection was carried out from January 2004 through July 2004. The survey used a mixed-mode methodology that included web and mail surveys, followed up by telephone interviews with those companies that did not respond by web or mail. Following standard survey procedures, multiple contact attempts were made in order to maximize survey response rates. Advance letters describing the purpose of the survey were mailed to company contact persons who were responsible for the 2002 ATP project proposal. For the web survey, emails containing a link to the survey web site and unique login credentials were sent about one week after the advance letter. Additional emails were sent to nonresponding applicants about one and three weeks after the initial email. For the mail survey, questionnaires were mailed about one week after the advance letter, with a second mailing of the questionnaire to nonresponding applicants three weeks after the initial questionnaire mailing. For both modes, Westat eventually tried to contact nonresponding applicants by telephone to collect the survey data.

Of 891 applicants eligible to respond, a total of 587 responses were received, for an overall response rate of 66%. Among the 144 ATP awardees invited to respond, 129 responses were received (117 by web, 12 by phone interview), yielding a response rate of 90%. Of the 747 nonawardees, 458 responses were received (195 by web, 64 by mail, and 199 by phone interview), yielding a response rate of 61%.

The following sections of this report present customer satisfaction results from the Survey of ATP Applicants 2002. The results are presented by topic area: applicant perceptions of the proposal process, views of the proposal preparation kit and electronic submission system, views of the usefulness of information sources, satisfaction with ATP staff, nonawardee views of the proposal debriefing and time and cost for proposal preparation. The 27 customer satisfaction items that comprise this portion of the survey are reproduced in the appendix.

Return to Table of Contents or go to next section.

Date created: July 29, 205
Last updated: August 4, 2005

ATP website comments: webmaster-atp@nist.gov  / Technical ATP inquiries: InfoCoord.ATP@nist.gov.

NIST is an agency of the U.S. Commerce Department
Privacy policy / Security Notice / Accessibility Statement / Disclaimer / Freedom of Information Act (FOIA) /
No Fear Act Policy / NIST Information Quallity Standards / ExpectMore.gov (performance of federal programs)