|
NIST GCR
02-834
Benefits and Costs of ATP Investments in Component-Based Software
This report assesses the
economic impact of the Advanced Technology Program's (ATP's) focused
program in Component-Based Software Development. From
1994 to 2000, ATP provided a total of $42 million in public funds to
support 24 separate technology development projects in this emerging
field. Quantitative and qualitative analyses presented in this
report demonstrate that ATP's support created significant economic benefits.
Two-thirds of the funded
projects achieved their technical objectives; many of the firms and one
joint venture proceeded to release successful commercial products based
on the technologies developed. ATP was credited by many of these
firms with enabling their R&D efforts, accelerating the
technology development process, and increasing the probability of
technical and commercial success.
When the 24 funded projects
are viewed as an investment portfolio, the social returns exceed any
reasonable benchmarks for public or private investment. The estimated
80 percent internal (social) rate of return reflects a substantial benefit
to the nation in excess of the return to the companies funded and is
an indication of inefficient capital markets for projects with high technical
risks. The calculated net present value (NPV) of $840 million and
benefit to cost ratio (B/C) of 10.5 suggest that the funds were a worthwhile
expenditure of public funds.
Return to Table
of Contents.
The authors would like
to acknowledge the assistance of many on the ATP staff, especially our
primary contacts Jeanne Powell and Barbara
Cuthill. Jeanne provided RTI with the project documents needed
for our study, reviewed deliverables for accuracy and readability, and
coordinated this report's internal review at ATP. Barbara contributed
much of the technical background for the study, as well as a wealth of
information about the individual firms and their software development
projects. We would also like to thank Kurt Wallnau, from the Software
Engineering Institute at Carnegie Mellon University, who reviewed a draft
version of this report and provided many helpful comments.
In conducting the eight
in-depth case studies, we relied heavily on structured interviews with
the principal investigators from the funded firms. We owe a great
deal of thanks to Gary Falacara from Aesthetic Solutions, Robert Glushko
of Commerce One, Barbara Hayes-Roth from Extempo Systems, Jeff Rees at
Hewlett-Packard (formerly at Intermetrics), Stanley Schneider of Real-Time
Innovations, Elaine Kant from SciComp, Brendan Madden at Tom Sawyer Software,
and Gregor Kiczales of Xerox.
Finally, we would like
to thank Taylor Bingham of RTI for his leadership and guidance in the
initial stages of this study. Brian Kropp and Michelle Bullock
provided valuable research and analytical support.
Return to Table
of Contents.
Component‑based software
development (CBSD) is a relatively new software production paradigm that
focuses on building large software systems by assembling readily available
components. Historically, about 85 percent of all large software
projects have been customized applications used within a single firm,
involving a minimal amount of code reuse. Systems programmers would
employ proprietary or shared libraries for a minor portion (up to 20 percent)
of the new application's code, but the remainder would be written specifically
for the project at hand. The remaining 15 percent of programs,
sold as packaged software to the retail market, have increasingly used
embedded components, but even in these programs, the overall levels of
reuse are quite low.
ATP's focused program in
component-based software development is an effort to change the paradigm
of custom application development to a "buy, don't build" approach for
most, if not all, software projects. The basic concept is that
computer code should be reused rather than rewritten whenever possible. Greater
reuse is expected to produce the following benefits:
- reduce the cost of
developing and maintaining software systems,
- increase the reliability
of software, and
- yield greater synergies
across portions of software code and applications.
The goal of ATP's focused
program in component-based software was to develop the technologies that
enable reuse of software code and automation of software development. ATP
focused-program competitions held in 1994, 1995, and 1997 resulted in
the selection of 24 software development projects. Sixteen of the
technologies developed during these projects are currently being used
in commercial applications.
This study uses case-study
methodology to analyze ATP's investment in the portfolio of projects
funded by the ATP's component-based software focused program. Research
Triangle Institute (RTI) performed an evaluation of the economic impact
of ATP's investment in this focused program, conducting eight in-depth
case studies to support this estimation. The conclusions from the
study are that the focused program was highly successful from a social
perspective, yielding benefits of $1.5 billion on a combined public and
private investment of about $119 million, measured in 2000 dollars. In
addition, the study found a number of qualitative benefits related to
assisting firms in strengthening their planning and management functions
and enhancing the credibility of the mostly small software firms that
were funded.
Case studies are an important
part of ATP's economic analysis strategy. They provide an in-depth
view of how ATP-funded technologies lead to economic benefits for the
awardees, other companies, and consumers. Case studies also provide
qualitative details about how ATP funding affects the investment decisions
of companies and the success of projects. Ideally, case studies
apply state-of-the-art methodologies that provide credible quantitative
estimates of the economic outcomes of ATP's investments in these technologies.
Understanding the component-based
market and the role of ATP within the market is key to developing an
estimation strategy to measure the qualitative and quantitative benefits
of the ATP focused program in component-based software.
A component is an independent
piece of software that interacts with other components in a well-defined
manner to accomplish a specific task. Components can be used to
build both custom enterprise-critical software and "off-the-shelf," shrink-wrapped
software. Although the ATP program focused on particularly challenging
and complex large-scale software development for commercial and industrial
applications, benefits from the new innovations will spill over to small-scale
or simple applications. As a software component market emerges,
most software developers are expected to shift focus from design and
implementation processes in large-scale and enterprise-critical situations
to solving application problems through adaptation of existing components
or combinations of components.
The concept of using components
to assemble software systems has been in existence for several decades. Firms
have reused up to 20 percent of their code internally since software
was first developed. However, several market failures have slowed
the development of a component-based market, including lack of interoperability,
network externalities, lack of trust in externally developed code, and
lock-in effects. ATP's focused program in component software was
designed to overcome the technical challenges associated with these barriers
to success.
Interoperability
Interoperability is a measure
of how well different pieces of technology function together. For
example, one application might generate results that another application
can use. The lack of interoperability slows the development of
a component-based market. ATP addressed this problem by selecting
projects for funding that focused on component technologies with multiple
applications. ATP also funded projects that focused on enterprise-wide
applications, where the technical challenge of maximizing interoperability
was a goal.
Network Externalities
Components must interact
with other components and infrastructures to create a valuable software
product. For example, a new spell-checking component has no value
unless basic word-processing components exist that can interact with
the new component. As a network of interacting components is developed,
the initial spell-checking component becomes more valuable. This
concept is called a "network externality." Each individual component
is more valuable than it is perceived to be because it adds value to
all other components.
However, when an individual
firm is making market decisions about what research to pursue for specific
products, the firm does not take into account the network benefits that
are being created because it cannot capture these benefits as profits. This
divergence between the social and private values of a new component has
slowed the investment in technology development needed to spawn a market
of component-based products. By providing federal funding to cost-share
high-risk R&D in this area, ATP can compensate for the discrepancy
between the social and private returns from a new component, and thereby
help stimulate development of reusable components and their commerce.
Lock-In
Legacy software and large,
highly customized software systems are abundant in large firms. These
systems contain massive amounts of information about how the company
operates, and they further identify critical data that must be collected
and analyzed. Significant resources are spent on maintaining and
updating these systems and monitoring their performance. Migrating
a complex, monolithic system to a newer component-based system, in which
updates and improvements are handled quickly and efficiently, could lower
maintenance costs. However, firms have been reluctant to change
to component-based systems. The main explanation is that the short-run
costs of migrating from the existing system to a component-based system
are prohibitively high, even though the long-term benefits could be substantial. ATP's
funding of the risky and complex aspects of component-based technology
development was anticipated to enable a multitude of component software
products. As more companies develop software component products,
more firms are expected to adopt component-based systems for their economic
benefits.
ATP's funding of the risky
phases of technology development, while products are still many years
away, gives companies a chance to pursue ideas that the private sector
either will not fund or not fund in a timely manner. As a result,
new products emerge that would not have been possible without ATP.
In 1994, ATP held a series
of workshops with industry and academia to discuss the potential for
a focused program on component-based software. These workshops,
along with various white papers from the technical community, resulted
in the ATP recommendation for a focused program. Another workshop
was held in 1995 to update the program scope. Over the course of
three rounds of funding, ATP committed a total of nearly $70 million
and industry committed an additional $55 million of cost-sharing funds
to a total of 24 projects. Of the 24 projects, 18 were completed
by the time of this study, two are still under way, and four failed to
complete. Two-thirds have yielded commercial products. To
date, three-fourths of the projects have reached the commercialization
phase even though some of the resulting products are not yet generating
revenues.
Historically, the process
of developing and using software components has been extremely complex. A
number of barriers have created substantial technical risk to innovation
in this area, including a lack of the following elements:
- automated tools for
building, locating, and adapting components;
- interfaces for nonprogrammers
to enable them to use the components; and
- specification of
interface semantics for bridging applications.
The ATP's focused program
for component-based software development sought to support development
of such tools, with special emphasis on tools that enable complex, large-scale,
commercial applications that can be used by a wide spectrum of industries. ATP
focused its funding efforts in two separate directions: building
components and building component architecture. Developing the
basic building blocks for components and architectures that apply to
all component developers and users is likely necessary to the maturation
of component-based software. Once component-based software markets
have emerged, software developers are expected to specialize and focus
on specific applications of components. The component-based software
industry would supply automated high-performance software tools and architectural
services that could be incorporated into the development of computer
systems. This model will allow a programmer to focus on the problem
at hand without being concerned about syntax or the programming process.
ATP investments across
a number of technology areas helped to eliminate or reduce market failures
to develop risky new technologies in a variety of ways. ATP maintains
a Web site with information to help participants conceptualize and write
proposals, complete required audits and reports, and, through its R&D
Alliance Network, establish successful cooperative research efforts. ATP
sponsors a number of seminars across the country in which funded firms
can participate.
Among the benefits of ATP
participation hypothesized at the focused program's inception, RTI found
the strongest evidence for the following impacts:
- accelerating the
technology's development and adoption,
- increasing the likelihood
of the technical success of each funded project, and
- widening the applications
of ATP-funded technologies.
For the component software
study, RTI developed an economic model to estimate the benefits attributable
to ATP-supported component-based software R&D projects, most of which
resulted in the creation of new software prototypes. Because the
final products will have a wide variety of potential uses and customers,
data collection for a price-index type analysis of benefits to users
of the technology would have been difficult or impossible to accomplish. However,
RTI developed a simple model that allowed an estimate of consumer and
producer surplus directly from data provided by the target companies. Although
this type of economic analysis is theoretically straightforward, this
is the first ATP evaluation in which it has been used.
The firms engaged in software
R&D are unlike companies typically envisioned in basic economics
theory in many respects. Although they sell final products in competitive
markets, their ability to differentiate their products gives them a good
deal of market power in new, highly differentiated product areas and
thus the ability to raise prices above marginal and average cost. Because
of the nature of the R&D process, however, these firms must commit
significant funds far in advance of achieving either technical or commercial
feasibility. R&D costs become the equivalent of high fixed
costs. These precommercial, high fixed costs, along with low marginal
costs of reproducing software and highly differentiated products, produce
many of the features of natural monopoly markets. However, the
rapid pace of technological development in software limits the scope
and duration of the market power the innovating firms exert, as expanded
product markets and competitor companies inevitably emerge around the
new idea and product.
Many of the companies involved
in component-based software are small start-ups that face severe financing
constraints. Because the bulk of their expenditures occur prior
to earning any revenues, indeed before technical feasibility has even
been established, these firms have difficulty obtaining capital from
loans or equity participation. Funding by ATP or another source
of patient capital is thus the only way that these technology development
projects can be undertaken. In addition, these firms are concerned
about the possibility of other firms entering the market, so they must
price at a point that is high enough to cover the fixed costs of production,
but low enough to slow the entry of other firms into the market. They
will set prices only high enough to recover their fixed development costs
over the entire product life cycle. In the detailed case studies,
RTI estimated the demand curves, R&D costs, and marginal costs of
production to determine the quantitative benefit of ATP-funded projects
in component-based software. In addition, RTI assessed the extent
to which other qualitative benefits occurred as a result of these projects.
RTI separated the economic
benefits from ATP investment into quantitative and qualitative benefits
and used a portfolio approach to calculate the quantitative returns to
ATP-funded projects. RTI selected eight projects that were expected
to generate significant economic returns, including firms of different
sizes and at least one firm from each of the three funding rounds. Benefits
were estimated from these eight successful projects and compared to the
costs for the entire set of 24 projects. This is equivalent to
assuming that the remaining 16 projects generated zero economic benefit,
an extremely conservative approach.
From each of the eight
in-depth case studies, estimates of the total benefits attributable to
the ATP project were made. This stream of benefits was then compared
to total outlays by ATP and industry to generate portfolio measures of
performance. Table ES-1 shows several measures of economic performance
attributable to the ATP program in component-based software.
Table ES-1. Quantitative
Measures of Economic Performance
Net
Present Value (in 2000 dollars) |
$840
million |
Benefit-Cost
Ratio |
10.5 |
Internal
Rate of Return |
80% |
The ATP component-based
software focused program was successful. The benefit-cost ratio
for the entire portfolio of CBSD projects was projected to be 10.5; that
is, with cash flows adjusted to 2000 dollars, a return of $10.5 was projected
for every dollar of ATP and industry investment. The net present
value of the investment, projected to be $840 million, describes the
net total benefit to the nation, in 2000 dollars, based on a 7 percent,
OMB-designated discount rate. The internal (social) rate of return
on investment of 80 percent describes the projected rate of return to
the nation.
These measures provide
a conservative estimate of the net benefits expected from the technologies
that ATP funded because they include benefits from just eight of the
projects but costs of all 24; the measures also assume very limited life
spans of resulting products relative to their likely potential. Most
of the products for which benefits were estimated have already generated
some revenues to the ATP-funded companies and benefits to customer-users. Three
of the projects-the Commerce One JV, and the Tom Sawyer and Intermetrics
single-company projects-generated enough returns independently to cover
the entire cost of the focused program.
In addition to the benefits
discussed above, the funded firms reported that ATP made several contributions
to their success that are more difficult to quantify. Standardized
ATP requirements for proposal writing, record keeping, and progress reporting
motivated firms to be more thorough in their project planning and execution. The
ATP funding requirements and management environment were well suited
to the unpredictable nature of R&D, as both patience and flexibility
on the part of ATP supported the firms' efforts. The participation
of ATP also imparted a "halo effect" of increased credibility, and lowered
barriers to commercialization of the technologies developed.
The quantitative and qualitative
analyses presented in this report demonstrate the significant impact
of ATP's investment in component-based software technology. Two-thirds
of the funded projects achieved their technical objectives; many of these
firms and one joint venture created successful commercial products based
on the technologies developed during their ATP project. When the
24 funded projects are viewed as an investment portfolio, the 80 percent
internal rate of return, $840 million net present value, and 10.5 benefit-to-cost
ratio suggest that the focused program was a worthwhile expenditure of
public funds.
Return to Table
of Contents or go to next Chapter.
Date created: December
3, 2002
Last updated:
August 2, 2005
|
