This is the accessible text file for GAO report number GAO-06-817R 
entitled 'NASA: Long-Term Commitment to and Investment in Space 
Exploration Program Requires More Knowledge' which was released on July 
26, 2006. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

July 17, 2006: 

The Honorable Sherwood Boehlert:
Chairman:
The Honorable Bart Gordon:
Ranking Minority Member:
Committee on Science:
House of Representatives: 

Subject: NASA: Long-Term Commitment to and Investment in Space 
Exploration Program Requires More Knowledge: 

The National Aeronautics and Space Administration (NASA) plans to spend 
nearly $230 billion over the next two decades implementing the Vision 
for Space Exploration. In January 2006, NASA publicly released its 
Exploration Systems Architecture Study (ESAS), which is an effort to 
identify the best architecture and strategy to implement the 
President's 2004 Vision for Space Exploration (Vision)[Footnote 1]. The 
cost estimate for implementing the ESAS through fiscal year 2011 
exceeds $31 billion. The estimate through fiscal year 2018 is over $122 
billion, and the estimate through fiscal year 2025 is nearly $230 
billion[Footnote 2]. These estimates include the architecture, robotic 
precursor missions, supporting technologies, and funding needed to 
service the International Space Station (IS[Footnote 3]S). NASA plans 
to implement this architecture through a "go as you can afford to pay" 
approach, wherein lower-priority efforts would be deferred, descoped, 
or discontinued to allow NASA to stay within its available budget 
profile. This approach assumes NASA's budget will increase moderately 
to keep pace with inflation. Given the long-term fiscal imbalances that 
will challenge the entire federal government now and in the future, it 
would be prudent for NASA to establish a program that reduces the risk 
that significant additional funding, beyond moderate increases for 
inflation, will be required to execute the program[Footnote 4]. 
Government leaders will have to make difficult decisions to resolve 
such challenges, and the debate over the potential cost and the federal 
government's role in implementing the Vision are emblematic of the 
challenges the nation will need to resolve in the years ahead. 

Because of the significance of this investment, competing demands on 
the federal discretionary budget, and the importance of the success of 
NASA's exploration program to the future of U.S. human spaceflight, you 
requested that we assess (1) the extent to which NASA has identified 
the architecture and costs necessary to implement the Vision, (2) 
whether NASA's exploration architecture cost estimates fit within the 
agency's projected available budgets, and (3) the risks associated with 
NASA's acquisition strategy for the CEV project. 

We presented our preliminary findings to your staff in May 2006. 
Because of your committee's interest in how NASA is implementing the 
Vision for Space Exploration, we are enclosing the full briefing that 
supported that May presentation with this report (see encl. II), along 
with a summary of our findings and conclusions. We are recommending 
that the NASA Administrator modify the current CEV acquisition strategy 
to ensure that the agency does not commit itself, and in turn the 
federal government, to a long-term contractual obligation prior to 
establishing a sound business case at the project's preliminary design 
review. In written comments, NASA nonconcurred with our recommendation 
and stated that it has the appropriate level of knowledge to proceed 
with its current acquisition strategy. As a result of its 
nonconcurrence, we are including as a matter for congressional 
consideration that the Congress should consider restricting NASA's 
appropriations and obligations for the CEV project to only the amount 
of funding necessary to successfully complete the project's preliminary 
design review. 

Background: 

The Vision includes plans to explore the moon, Mars, and 
beyond.[Footnote 5] The first step in implementing the Vision is to 
retire the space shuttle after completing assembly of the ISS by the 
end of the decade. NASA currently plans to retire the space shuttle in 
2010, creating a potential gap in U.S. human spaceflight of up to 4 
years before development of the CEV and the CLV is complete. Congress 
has voiced concern over the United States not having continuous access 
to space, and NASA has made it a priority to minimize the gap by 
accelerating the CEV project to have it in service as close to 2010 as 
possible. NASA's Exploration Systems Mission Directorate's (ESMD) 
Constellation program is responsible for the development of both the 
CEV and the CLV. NASA awarded concept development contracts for the CEV 
project to both Lockheed Martin and Northrop Grumman in July 2005 and 
plans to award a contract for design, development, production and 
sustainment in September 2006. That contract could extend through 2019. 
For the CLV, NASA plans to award a sole-source contract for the first 
stage of the CLV to ATK-Thiokol, the manufacturer of the Shuttle's 
Reusable Solid Rocket Motor, in October 2006. Also, the agency plans to 
award Pratt & Whitney Rocketdyne, the developer of the Space Shuttle 
Main Engine (SSME) and J-2 engines, a sole-source contract for 
development of the J-2X engine in November 2006. These contractors are 
currently planning their respective efforts under interim contract 
arrangements. NASA has started in-house preliminary design work on the 
CLV upper stage structures and avionics and plans to begin awarding 
competitive contracts for production of these items in May 2007. 

Despite many successes in the exploration of space, such as landing the 
Pathfinder and Exploration Rovers on Mars, the loss of life, 
unsuccessful missions, and unforeseen cost overruns have recently 
increased the level of concern over the benefits of such exploration, 
particularly with regard to human spaceflight activities. NASA has had 
difficulty bringing a number of projects to completion, including 
several efforts to build a second generation of reusable human 
spaceflight vehicle to replace the space shuttle. NASA has attempted 
several expensive endeavors such as the National Aero-Space Plane, the 
X-33 and X-34, and the Space Launch Initiative, among others. While 
these endeavors have helped to advance scientific and technical 
knowledge, none have completed their objective of fielding a new 
reusable space vehicle. We estimate that these unsuccessful development 
efforts have cost approximately $4.8 billion since the 1980s. The high 
cost of these unsuccessful efforts and the potential costs of 
implementing the Vision make it important that NASA achieve success in 
its new exploration program. 

Our past work has shown that developing a sound business case, based on 
matching requirements to available and reasonably expected resources 
before committing to a new product development effort, reduces risk and 
increases the likelihood of successful outcomes.[Footnote 6] At the 
heart of a business case is a knowledge-based approach to product 
development that is a best practice among leading commercial firms and 
successful government system developers. For a program to increase its 
chances of delivering a successful product, high levels of knowledge 
should be demonstrated before managers make significant program 
commitments. In essence, knowledge supplants risk over time. This 
building of knowledge can be described as three levels that should be 
attained over the course of the program: 

(1) At program start, the customer's needs should match the developer's 
available resources in terms of availability of mature technologies, 
time, human capital, and funding. 

(2) Midway through development, the product's design should be stable 
and demonstrate that it is capable of meeting performance requirements. 

(3) By the time of the production decision, the product must be shown 
to be producible within cost, schedule, and quality targets, and have 
demonstrated its reliability. 

Our work has shown that programs that have not attained the level of 
knowledge needed to support a sound business case have been plagued by 
cost overruns, schedule delays, decreased capability, and overall poor 
performance. With regard to NASA, we have reported that in some cases 
the agency's failure to define requirements adequately and develop 
realistic cost estimates--two key elements of a business case--resulted 
in projects costing more, taking longer, and achieving less than 
originally planned. 

Summary: 

Although NASA is continuing to refine its exploration architecture cost 
estimates, the agency cannot at this time provide a firm estimate of 
what it will take to implement the architecture. The absence of firm 
cost estimates is mainly due to the fact that the program is in the 
early stages of its life cycle. According to NASA cost-estimating 
guidance, early life cycle phase estimates are generally based upon 
parametric models, which use data from projects with similar attributes 
to predict cost because there are usually many unknowns and actual cost 
or performance data are not available. NASA preliminarily identified 
the resources needed to implement the architecture as outlined in the 
architecture study primarily through the use of such models. NASA 
conducted a cost risk analysis of its preliminary estimates through 
fiscal year 2011. On the basis of this analysis and through the 
addition of programmatic reserves (20 percent on all development and 10 
percent on all production costs), NASA is 65 percent confident that the 
actual cost of the program will either meet or be less than its 
estimate of $31.2 billion through fiscal year 2011. For the cost 
estimates for beyond 2011, when most of the cost risk for implementing 
the architecture will be realized, NASA has not applied a confidence 
level distinction. Since NASA released its preliminary estimates, the 
agency has continued to make architecture changes. For example, 
following the issuance of the architecture study, NASA conducted 
several analysis cycles during which various aspects of the 
architecture have evolved, such as the diameter of the CEV, the engine 
used to support the upper stage of the CLV, and the size of the 
Reusable Solid Rocket Booster on the CLV. While these changes, and 
others, are appropriate for this phase of the program, when concepts 
are still being developed, they leave the agency in the position of 
being unable to firmly identify program requirements and needed 
resources, which can also be expected at this phase of the program. 
According to NASA officials, once they receive more detailed contractor 
inputs, the agency will be able to produce higher-fidelity estimates of 
program cost. NASA plans to commit to a firm cost estimate at the 
preliminary design review (PDR) in 2008, when the program's 
requirements, design, and schedule will all be baselined. 

NASA will be challenged to implement the architecture recommended in 
the study within its projected budget. Whether using the architecture 
study estimates of funds available or NASA's Fiscal Year 2007 Budget 
Submission for ESMD that was based on the architecture study cost 
estimates, there are years when NASA does not have sufficient funding 
to implement the architecture. Some yearly shortfalls exceed $1 
billion, while in other years the funding available exceeds needed 
resources. NASA maintains that the architecture could be implemented 
within the projected available budgets through fiscal year 2011 when 
funding is considered cumulatively. In addition, NASA preliminarily 
projects multibillion-dollar shortfalls for ESMD in all fiscal years 
from 2014 to 2020, with an overall deficit through 2025 of over $18 
billion. In the short term, NASA is attempting to address this problem 
within the Constellation program by redirecting funds to that program 
from other ESMD activities to provide a significant surplus for fiscal 
years 2006 and 2007 to cover projected shortfalls beginning in fiscal 
year 2009. In addition, the Constellation program has requested more 
funds than required for its projects in several early years to cover 
shortfalls in later years. For example, the Exploration Communication 
and Navigation Systems project within the Constellation program plans 
to roll over $56.2 million from the fiscal year 2007 budget to make up 
for budget shortfalls in fiscal years 2008, 2009, and 2010. NASA 
officials stated the identified budget phasing problem could worsen 
given that changes made to the exploration architecture following 
issuance of the study will likely add to the near-term development 
costs, where the funding is already constrained. In addition, NASA's 
estimates beyond 2010 are based upon a surplus of well over $1 billion 
in fiscal year 2011 due to the retirement of the space shuttle fleet in 
2010. However, NASA officials said the costs for retiring the space 
shuttle and transitioning to the new program are not fully understood, 
and thus the expected surplus could be less than anticipated. 

NASA's current acquisition strategy for the CEV places the project at 
risk of significant cost overruns, schedule delays, and performance 
shortfalls because it commits the government to a long-term product 
development effort before establishing a sound business case. NASA 
plans to award a contract for the design, development, production, and 
sustainment of the CEV in September 2006--before it has developed key 
elements of a sound business case, including well-defined requirements, 
a preliminary design, mature technology, and firm cost estimates. The 
period of performance for the contract scheduled for award in September 
2006 will extend through at least 2014, with the possibility of 
extending through 2019. This contract will comprise all design, 
development, and test and evaluation activities, including production 
of ground and flight test articles and at least four operational CEVs. 
Although NASA is committing to a long-term contract, it will not have 
the elements of a sound business case in place until the project level 
PDR in fiscal year 2008. Awarding a contract for design, development, 
production, and sustainment of the project as NASA has planned places 
the CEV project at increased risk of cost growth, schedule delays, and 
performance shortfalls. At PDR, NASA will likely (a) have the increased 
knowledge necessary to develop a sound business case that includes high-
fidelity, engineering-based estimates of life cycle cost for the CEV 
project, (b) be in a better position to commit the government to a long-
term effort, and (c) have more certainty in advising Congress on 
required resources. 

Implementing the Vision over the coming decades will require hundreds 
of billions of dollars and a sustained commitment from multiple 
Administrations and Congresses over the length of the program. The 
realistic identification of the resources needed to achieve the 
agency's short-term goals would provide support for such a sustained 
commitment over the long term. With a range of federal commitments 
binding the fiscal future of the United States, competition for 
resources within the federal government will only increase over the 
next several decades. Consequently, it is incumbent upon NASA to ensure 
that it is wisely investing its existing resources. As NASA begins to 
implement the Vision with several key acquisition decisions planned to 
occur this fall, it will be essential that the agency ensure that the 
investment decisions it is making are sound and are based upon high 
levels of knowledge. NASA should make the prudent decision now to 
ensure that it has attained the appropriate level of knowledge to 
support a sound business case before it commits to the project. 
However, under the current acquisition strategy for CEV, key knowledge-
-including well-defined requirements, a preliminary design, mature 
technology, and firm cost estimates--will not be known until over a 
year after the expected contract award date. Nevertheless, NASA plans 
to commit the government to a long-term contract. This approach 
increases the risk that the project will encounter significant cost 
overruns, schedule delays, and decreased capability. Given the nation's 
fiscal challenges and those that exist within NASA, the availability of 
significant additional resources to address such issues, should they 
occur, is unlikely. With the impending decisions pertaining to the CEV, 
NASA has the opportunity to establish a firm foundation for the entire 
Constellation program by ensuring that the appropriate level of 
knowledge is available before proceeding with its acquisition strategy 
and committing the government to a long-term design, development, and 
production effort. 

Recommendation for Executive Action: 

Because of the importance of the CEV project to NASA's overall 
implementation of the Vision, NASA should focus on ensuring that its 
acquisition approach for the CEV project does not place the government 
at risk by committing to a long-term design and development effort 
without the knowledge needed to make wise investment decisions. We 
therefore recommend that the NASA Administrator modify the current CEV 
acquisition strategy to ensure that the agency does not commit itself, 
and in turn the federal government, to a long-term contractual 
obligation prior to demonstrating, through the establishment of a sound 
business case at the project's preliminary design review, that the 
project is affordable and executable. 

Matter for Congressional Consideration: 

Based on its response to our report, it appears that NASA plans to 
proceed with its acquisition strategy for the CEV and award a long-term 
contract for the project, although it continues to lack sufficient 
knowledge and a sound business case for doing so. Congress is currently 
being asked to approve NASA's fiscal year 2007 funding request and will 
be asked to approve fiscal year 2008 and perhaps the fiscal year 2009 
funding requests for the CEV project before NASA has demonstrated such 
knowledge and has provided evidence, based on that knowledge, that the 
project will be executable within existing and expected resources. In 
light of the fact that NASA plans to award the contract for the CEV in 
September 2006, Congress should consider restricting annual 
appropriations and limiting NASA's obligations for the CEV project to 
only the amount of funding necessary to support activities needed to 
successfully complete the project's preliminary design review. 

Agency Comments and Our Evaluation: 

In written comments on a draft of this report (see encl. I), NASA 
nonconcurred with our recommendation that it modify the current CEV 
acquisition strategy to ensure that the agency does not commit itself, 
and in turn the federal government, to a long-term contractual 
commitment prior to establishing a sound business case at the project's 
preliminary design review. NASA stated that it has the appropriate 
level of knowledge to proceed with its acquisition plan to "down 
select" to a single Crew Exploration Vehicle prime contractor in 
September 2006. NASA added that it is maximizing competition by 
soliciting from industry a development, production, and management 
approach with an emphasis on life cycle cost. In the area of technology 
maturity, NASA stated that it has a plan and process in place to 
address the Thermal Protection and Landing subsystems technology risks 
through in-house development work and collaboration with the prime 
contractor. NASA also noted that during its design, development, and 
test and evaluation effort, the agency will be using an end-item award 
fee, which would make all award fees subject to a final evaluation to 
determine how well the product met requirements, including cost and 
schedule. 

The CEV acquisition strategy is not knowledge-based in that it calls 
for maturing technologies, designing systems, and preparing for initial 
production concurrently--an approach that our work has shown carries 
the increased risk of cost and schedule overruns and decreased 
technical capability. Therefore, we disagree with NASA's statement that 
it has the appropriate level of knowledge to proceed with its current 
acquisition strategy and award a long-term contract for the project 
prior to obtaining sufficient knowledge. Specifically: 

 In its response, NASA suggests that there would be no benefit in 
retaining two prime contractors for the CEV project through the 
preliminary design review and that the best return on its investment 
would be gained by down-selecting to one contractor and awarding the 
contract in September 2006. Contrary to NASA's response, addressing our 
recommendation would not preclude the agency from down-selecting to one 
contractor. The thrust of our recommendation is that NASA should lessen 
the government's obligation to the project at such an early stage when 
realistic cost estimates have yet to be established and requirements 
are not fully defined, and therefore limit the scope of the contract to 
activities needed to successfully complete the preliminary design 
review. At that point the project should have in place a sound business 
case for proceeding and hence be in a better position to justify 
continued investment. Implementation of the recommendation could be 
accomplished through various means, including by retaining two 
contractors through the preliminary design review and awarding a 
contract at that time or by down-selecting as planned in September 2006 
and limiting the scope of the contract as described above. 

* NASA's suggestion that it is maximizing competition by soliciting 
from industry its development, production, and management approach and 
that it will receive firm competitive prices from industry for 
completion of development and demonstration of two vehicles has little 
basis. First, while the current structure will allow for competition in 
the short term, the benefits of such competition will be short-lived. 
Without well-defined requirements, mature technologies, an approved 
preliminary design, and realistic cost estimates, NASA has insufficient 
information to ensure that it is obtaining firm competitive prices for 
the work conducted for the entirety of Schedule A--especially for 
activities beyond the project's preliminary design review. 

Because NASA continues to refine the project's requirements, as 
demonstrated by the numerous changes to the exploration architecture as 
discussed in our report, it cannot provide a firm estimate of project 
cost. Without such information, it will likely be difficult for NASA to 
establish realistic "not-to-exceed" prices for Schedule B activities. 
Under the current strategy, NASA will not have high-fidelity, 
engineering-based estimates of life cycle costs for the CEV until the 
preliminary design review. As outlined in this report, projects with 
cost estimates based on early, evolving designs and top-level 
requirements are at increased risk of cost growth relative to estimates 
based on mature designs and detailed requirements--which could be 
achieved at the preliminary design review. According to NASA, it plans 
to obtain this and further knowledge about program cost, schedule, and 
risk elements following the contract award and in conjunction with the 
contractor. In the absence of such information, it is not clear how 
NASA can substantiate its statement that it has the knowledge necessary 
to commit to activities beyond the project's preliminary design review. 
Further, it cannot provide Congress with assurance of the 
appropriateness of requested funding for the project. 

 NASA stated that its current acquisition strategy for the CEV 
minimizes the government's obligation during development by dividing 
the CEV contract into three separate schedules. All three schedules, 
however, will be awarded in September 2006 as part of one contract. 
Although NASA plans to include language in the negotiated CEV contract 
to state that the minimum quantity under Schedule B will not be 
applicable until that schedule's period of performance begins in 2009-
-a step that would lessen the government's obligation during 
production--it will continue to be responsible for all Schedule A 
activities at the time of contract award. These activities include all 
design, development, and test and evaluation activities, as well as the 
production of two operational vehicles. Contractually obligating the 
government to even these Schedule A activities, before it has 
established a sound business case to support such a commitment, is not 
in line with our knowledge-based approach and is ultimately not in the 
best interest of the government. 

* NASA's investment in identifying and maturing the Thermal Protection 
and Landing Subsystems is a step in the right direction to ensure that 
these technologies are mature and available when needed. NASA has no 
guarantee, however, that these critical technologies will be mature by 
the time of the project's preliminary design review--the point at which 
our work has shown that technologies should be mature in order to 
decrease the risk of cost and schedule growth. NASA's proposed 
commitment to the project for activities beyond the preliminary design 
review before retiring these technology risks increases the likelihood 
that the project will experience schedule delays and cost overruns. 

* NASA maintains that program risks have been marginalized and that the 
agency will utilize incentives, including end-item award fees, to 
ensure contractor performance. NASA suggests that the incentives it 
plans to use in the form of end-item award fees will be a powerful tool 
for meeting cost schedule, technical, and quality goals. The use of 
these tools, however, does not compensate for proceeding with a risky 
acquisition, nor do they lessen NASA's responsibility to implement an 
executable program from the start. For them to function as intended, 
NASA needs to address the more fundamental issues related to its 
acquisition strategy, including its lack of a sound business case for 
the CEV project. 

* Finally, the use of cost-reimbursable contracting, while appropriate 
for early development and design efforts, places most of the cost risk 
for the project on the government. Given the nature of this effort, it 
is likely that the project will change significantly as it moves 
forward. Therefore, any scope changes or schedule slips could translate 
into additional contract cost for NASA. Such cost impacts could be 
minimized if NASA limited its contractual obligation to those 
activities needed to achieve a successful preliminary design review, as 
we recommended. In addition, limiting the scope of the CEV contract 
would allow both NASA and Congress to assess the project's progress at 
the preliminary design review and to decide if continued investment in 
the project is prudent and in the best interest of the government. 

It is important to note that Congress will continue to be asked to make 
funding commitments in advance of CEV project events that would 
demonstrate that the project has the knowledge necessary to support a 
sound business case. Specifically, NASA's funding request for fiscal 
years 2007 and 2008 are scheduled to be approved before the CEV holds 
its preliminary design review. Since the preliminary design review is 
currently scheduled for March 2008, this may also be the case for 
fiscal year 2009. Congress should safeguard against a situation in 
which contractual and budget decisions could hinder its ability to tie 
further investments in the CEV project to demonstrated progress at the 
preliminary design review. As such, we have included a matter for 
congressional consideration. 

We also received technical comments from NASA, which have been 
addressed in the report, as appropriate. 

Scope and Methodology: 

To assess the extent to which NASA has identified the architecture and 
costs necessary to implement the Vision and whether NASA's exploration 
architecture cost estimates fit within the agency's projected available 
budgets, we reviewed and analyzed NASA's Exploration Systems 
Architecture Study, fiscal year 2007 budget request, ground rules and 
assumptions provided from the Constellation program to project level 
management estimators to perform the bottom up review, guidance for use 
in preparing the fiscal year 2008 budget request, NASA cost-estimating 
guidance in the NASA Cost Estimating Handbook, and congressional 
hearings and testimonies pertaining to NASA and the Vision. We also 
conducted interviews with NASA headquarters officials from the Cost 
Analysis Division, the Exploration Systems Mission Directorate, and 
Constellation program officials, Constellation program and CEV project 
officials at Johnson Space Center; CLV project officials at Marshall 
Space Flight Center; and cost analysts from the Kennedy Space Center. 
During these interviews, we discussed the methodologies used in 
preparing the ESAS and subsequent cost estimates, architecture changes 
after the ESAS and the trades being considered, budgeting issues, and 
procurement strategies and activities. 

To assess the risks associated with NASA's acquisition strategy for the 
CEV project, we reviewed and analyzed CEV project documentation, 
including draft project plans, draft requirements documents, technology 
development plans, documentation included in the contract request for 
proposals, and past NASA human spaceflight acquisition programs. We 
compared NASA's plans for the CEV with criteria contained in GAO best 
practices work on systems acquisition. We also conducted interviews 
with NASA headquarters officials from the Exploration Systems Mission 
Directorate and Constellation Systems officials, Constellation program 
and CEV project officials at Johnson Space Center, and CLV project 
officials at Marshall Space Flight Center. 

We conducted our work from January 2006 to May 2006 in accordance with 
generally accepted government auditing standards. 

As agreed with your offices, unless you announce its contents earlier, 
we will not distribute this report further until 10 days from its date. 
At that time, we will send copies of the report to NASA's Administrator 
and interested congressional committees. We will also make copies 
available to others upon request. In addition, the report will be 
available at no charge on GAO's Web site at [Hyperlink, 
http://www.gao.gov]. 

Should you or your staff have any questions on matters discussed in 
this report, please contact me at (202) 512-4841 or lia@gao.gov. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. Principal 
contributors to this report were James L. Morrison, Assistant Director; 
Rick Cederholm; Shelby S. Oakley; Guisseli Reyes; Sylvia Schatz; and 
John S. Warren, Jr. 

Signed by: 

Allen Li: 
Director: 
Acquisition and Sourcing Management: 

Enclosures: 

Enclosure I: 

Comments from the National Aeronautics and Space Administration: 

National Aeronautics and Space Administration: 
Office of the Administrator: 
Washington, DC 20546-0001: 

July 6, 2006: 

Mr. Allen Li: 
Director Acquisition and Sourcing Management: 
United States Government Accountability Office: 
Washington, DC 20548: 

Dear Mr. Li: 

NASA has reviewed the Government Accountability Office (GAO) draft 
report entitled "NASA: Long-Term Commitment to and Investment in Space 
Exploration Program Requires More Knowledge (GAO Code 120515, Report 
Number GAO-06-817R)." Thank you for the opportunity to provide comments 
on the recommendation in the report. 

NASA embraces GAO's recognition that a "knowledge-based" approach 
reduces risks and increases the likelihood of successful outcomes. As 
the primary steward for achieving the Vision for Space Exploration, 
NASA fully recognizes the importance of investing its resources wisely 
and maintaining stakeholder confidence in its performance. NASA has the 
appropriate level of knowledge to proceed with its knowledge-and 
performance-based acquisition plan to "down-select" a single Crew 
Exploration Vehicle (CEV) prime contractor in September 2006. The 
Agency's acquisition strategy and plans capitalize on the benefits of 
competition, focus on performance, and address the inherent risk of 
complex development projects. Accordingly, NASA nonconcurs with GAO's 
recommendation that the NASA Administrator modify the current CEV 
acquisition strategy. 

NASA is maximizing competition by soliciting from industry their 
development, production, and management approach with an emphasis on 
Life Cycle Cost (LCC) for the CEV. While in this competitive 
environment, NASA will receive firm competitive prices from industry to 
complete development of the CEV and demonstrate one pressurized crew 
vehicle and one pressurized unmanned vehicle. Under this competition, 
NASA will also establish not-to-exceed prices for production of 
required CEVs to support the current flight manifest through 2019. The 
foundation of the CEV acquisition strategy is focused on gaining 
commitment from industry for a design solution and controlling LCC 
through competition and incentives. 

NASA has diligently invested the time and resources in the formulation 
phase of the CEV project in order to develop the knowledge necessary to 
commit to a long-term design and development effort. In May 2005, the 
Exploration Systems Architecture Study (ESAS) was initiated with one of 
its tasks being to provide a complete assessment of the top-level CEV 
requirements. As a result of the ESAS, the architecture and the top- 
level requirements for the CEV were chosen. With the level of knowledge 
gained through the Agency's investment in the ESAS and with its 
acquisition strategy, NASA perceives no benefit in retaining two prime 
contractors through Preliminary Design Review (PDR) at an estimated 
cost of $1 billion each. Instead, NASA has determined that a better 
return on its investment would be gained by competitively issuing two 
Phase 1 CEV prime contracts for conceptual design and trade studies 
against the ESAS architecture for an estimated cost of $46 million each 
and including the CEV requirements as part of the CEV Phase 2 contract 
competition. Additionally, NASA established an intra-agency CEV Smart 
Buyer team which performed trade studies and design analyses that were 
used by the CEV Project Office to understand and verify the 
appropriateness of the requirements incorporated into the CEV Phase 2 
solicitation and evaluation of proposals. With knowledge gained from 
ESAS, the Smart Buyer team, and the CEV Phase 1 contracts, NASA is now 
in a sound position to "down-select" a single prime contractor, thereby 
base-lining an industry approach and commitment to meet the desired 
outcomes of the CEV project. 

NASA's business approach is consistent with GAO's knowledge-based 
recommendation and recognition that knowledge replaces risk over time. 
The CEV acquisition strategy contains separate contract schedules and 
design reviews which are equivalent to GAO's defined: 

Knowledge Points. NASA's acquisition strategy minimizes the 
Government's obligation during development by dividing the CEV contract 
into three different schedules: 

* Schedule A for Design Development, Test, and Evaluation (DDT&E). 

* Schedule B for production beyond Schedule A. 

* Schedule C for sustaining activities during production and operation. 

Schedule A is authorized at contract award and continues through the 
first flight demonstration of each design variant of the CEV. Schedule 
A executes the formulation phase of the project such that NASA can 
utilize the contractor's knowledge to develop a set of validated 
requirements, including component specifications and mature 
technologies by the project's PDR. The current CEV strategy will allow 
NASA and the contractor to attain further appreciation and knowledge 
about the project and its required resources to provide firm cost, 
schedule, and risk elements. At this point, the Non-Advocacy Review 
(NAR) is typically scheduled immediately following the baseline of the 
project's preliminary design. 

Authorization of Schedule B is planned post PDR, NAR, and the Critical 
Design Review (CDR) and is currently limited to a minimum production 
quantity of two units. Authorization of Schedule C is planned at 
approximately the same time as Schedule B. The CEV strategy does not 
commit the Agency to any production until the NAR milestone is met. 
Additionally, utilizing Delivery Orders (Indefinite Delivery Indefinite 
Quantity) for Schedules B and C provides NASA with the ability to order 
only the units and the sustaining engineering necessary, with 
appropriate incentives, when the requirements and costs are better 
understood by NASA and industry. NASA will not commit to Schedule B or 
C activities until it is time to implement that portion of the 
contract. To mitigate concerns about the minimum production quantity of 
two units under the production contract (Schedule B), language will be 
included in the final negotiated CEV contract that will make explicit 
that the minimum quantity will not be applicable until the period of 
performance of Schedule B begins. First production orders are planned 
to be placed in the fall of 2009, nine months after the baseline of the 
CEV's critical design or CDR (Knowledge Point 3). 

NASA has a plan and process in place to address technology risks 
through in-house development work and collaboration with the prime 
contractor. For example, NASA has identified two areas where the 
additional technology maturation is needed: the Thermal Protection 
Subsystem and the Landing Subsystem. NASA has in-house advanced 
development plans (ADP) to develop these technologies with Prime 
Contractor participation through PDR. While the Prime Contractor will 
participate with the ADP, it will not assume development responsibility 
until after PDR. 

Incentives are a critical element in the business case for the CEV 
project. During DDT&E, NASA will use an end-item award fee. This makes 
all award fees subject to final determination only after the product 
has been demonstrated to meet requirements, including cost and 
schedule. This is a powerful tool for the NASA project manager and 
provides incentive to all elements of the project: cost (including life 
cycle costs), schedule, technical, and most importantly, quality. There 
will be inherent motivation toward schedule performance by means of 
concluding each project milestone with an award fee determination. 
Since no provisional payments will be made, industry will not receive 
interim payments until the completion of an established project 
milestone. A slip in schedule will be reflected both in a delay in 
receipt of the interim payment and in the NASA evaluation that will 
eventually follow. 

In summary, NASA is confident that its acquisition strategy and plans 
for selecting a CEV Prime Contractor are based on sound business case, 
will establish a firm foundation for the Constellation Program, and are 
in the Government's best interest. 

Sincerely, 

Signed by: 
Shana Dale: 
Deputy Administrator: 

Enclosure II: 

May 2006: 

NASA: Long-Term Commitment to and Investment in Space Exploration 
Program Requires More Knowledge: 

Why GAO Did This Study: 

In January 2006, the National Aeronautics and Space Administration 
(NASA) publicly released its Exploration Systems Architecture Study 
(ESAS), which aimed to identify the best architecture and strategy to 
implement the President's 2004 Vision for Space Exploration (Vision). 
The ESAS architecture supports development of a new Crew Exploration 
Vehicle (CEV), Crew Launch Vehicle (CLV), Cargo Launch Vehicle (CaLV), 
and other supporting systems, which are part of NASA's Exploration 
Systems Mission Directorate's (ESMD) Constellation program. The 
architecture also calls for various Research and Technology (R&T) and 
Robotic Lunar Exploration Program (RLEP) projects. 

The cost estimate for implementing the ESAS through fiscal year 2011 
exceeds $31 billion. The estimate through fiscal year 2018 is $122 
billion and the estimate through fiscal year 2025 is nearly $230 
billion. These estimates include the architecture, robotic precursor 
missions, supporting technologies, and funding needed to service the 
International Space Station (ISS). 

Because of the significance of this investment, competing demands on 
the federal discretionary budget, and the importance of the success of 
NASA's exploration program to the future of U.S. human spaceflight, we 
assessed (1) the extent to which NASA has identified the architecture 
and costs necessary to implement the Vision, (2) whether NASA's 
exploration architecture cost estimates fit within the agency's 
projected budgets, and (3) the risks associated with NASA's acquisition 
strategy for the CEV. 

Summary: 

Although NASA is continuing to refine its exploration architecture cost 
estimates, the agency cannot at this time provide a firm estimate of 
what it will take to implement the architecture. The absence of firm 
cost estimates is mainly due to the fact that the program is in its 
early stages. NASA preliminarily identified the resources needed to 
implement the architecture as outlined in the ESAS. However, since that 
time, NASA has continued to make architecture changes. For example, 
following the issuance of the ESAS, NASA undertook several analysis 
cycles in which various aspects of the architecture have evolved, such 
as the diameter of the CEV, the engine used to support the upper stage 
of the CLV, and the size of the Reusable Solid Rocket Booster on the 
CLV. These changes, and others, are appropriate for this phase of the 
program, when concepts are being developed, but leave NASA in the 
position of being unable to firmly identify program requirements and 
needed resources. NASA plans to commit to a firm cost estimate at the 
preliminary design review (PDR) in 2008, when the programs' 
requirements, design, and schedule will all be baselined. 

NASA will be challenged to implement the ESAS architecture with its 
projected budget. Whether using the ESAS estimates of funds available 
or NASA's fiscal year 2007 budget submission that was based upon the 
ESAS estimates, there are years when NASA does not have sufficient 
funding to implement the architecture. Some yearly shortfalls exceed $1 
billion, while in other years the funding available exceeds needed 
resources. NASA maintains that the architecture could be implemented 
within its projected available budgets through fiscal year 2011 when 
funding is considered cumulatively. In the short term, NASA has 
redirected funds to the Constellation program from other ESMD 
activities to provide a significant surplus for fiscal years 2006 and 
2007 to cover projected shortfalls for the program beginning in fiscal 
year 2009. The identified budget phasing problem in ESAS could worsen, 
given that changes to the architecture following the ESAS will likely 
add to the near term development costs, where funding is already 
constrained. In addition, NASA anticipates a significant surplus in 
fiscal year 2011 because of the retirement of the space shuttle fleet 
in 2010. However, the transition costs are not fully understood. 

NASA's acquisition strategy for the CEV places the project at risk of 
cost overruns, schedule delays, and performance shortfalls because it 
commits the government to a long-term product development effort before 
establishing a sound business case. NASA plans to award a contract for 
design, development, production, and sustainment of the CEV in 
September 2006--before it has developed well-defined requirements, a 
preliminary design, mature technology, and firm cost estimates. This 
information is not expected until the project-level PDR in fiscal year 
2008. At that point, NASA will likely (a) have the increased knowledge 
necessary to develop a sound business case that includes high-fidelity, 
engineering-based estimates of life cycle cost for the CEV project, (b) 
be in a better position to commit the government to a long-term effort, 
and (c) have more certainty in advising Congress on required resources. 

Implementing the Vision: 

NASA plans to bring the President's Vision to reality over the next 
several decades by: 

* conducting exploration activities in low-Earth orbit; for example, 
flying the space shuttle to complete assembly of the ISS; 

* exploring beyond low-Earth orbit; for example, establishing sustained 
exploration of the moon and Mars; 

* developing transportation that supports exploration; for example, 
building crew exploration vehicles; and: 

* pursuing opportunities for international and commercial 
participation. 

Exploration Contracts: 

NASA awarded concept development contracts to both Lockheed Martin and 
Northrop Grumman for the CEV project in July 2005. NASA plans to down-
select to one contractor and award a contract for development, 
production, and sustainment of the CEV in September 2006. That contract 
could extend through 2019. 

NASA plans to award a sole-source contract for the first stage of the 
CLV to ATK-Thiokol, the manufacturer of the Shuttle's Reusable Solid 
Rocket Motor, in October 2006. Also, the agency plans to award Pratt & 
Whitney Rocketdyne, the developer of the Space Shuttle Main Engine 
(SSME) and J-2 engines, a sole-source contract for development of the J-
2X engine in November 2006. These contractors are currently planning 
their respective efforts under interim contract arrangements. NASA has 
started in-house preliminary design work on the CLV upper stage 
structures and avionics and plans to begin awarding competitive 
contracts for production of these items in May 2007. 

Original Exploration Systems Architecture Study Overview: 

The ESAS outlined the recommended architecture and strategy for 
implementation of the Vision. The primary vehicles and elements of the 
architecture include the CEV, the CLV, the CaLV that includes the Earth 
Departure Stage (EDS), and the Lunar Surface Access Module (LSAM). The 
diagram below outlines a launch mission for crew and cargo, utilizing 
rendezvous locations in low-Earth and low-lunar orbits. 

[See PDF for Image] 

Source: NASA. 

[End of Figure] 

The original ESAS architecture is described below. Changes made to the 
architecture since the release of ESAS are described in later sections. 

CEV: The CEV is a reusable, Apollo-derived cone-shaped capsule launched 
atop the CLV. The CEV consists of a Command Module (CM), a Service 
Module (SM), and a Launch Abort System (LAS). The CEV is sized at 5.5 
meter diameters for lunar polar missions carrying a crew of four, and 
is also reconfigurable to accommodate up to six crew members for 
missions to ISS. The vehicle uses a Low Impact Docking System (LIDS) 
for ISS and lunar missions. The vehicle is reusable for up to 10 
missions and will land on land with a water landing as a backup. The SM 
utilizes a pressure-fed liquid oxygen (LOX)/methane propulsion system. 

CLV: The CLV consists of a shuttle-derived four-segment Reusable Solid 
Rocket Booster (RSRB) first stage and a newly designed upper stage with 
one modified, and now expendable, SSME. It will launch 25 metric tons 
to low-Earth orbit and serve as the long-term crew launch capability 
for the United States. 

CaLV: The CaLV will use a heritage shuttle external tank-derived LOX/ 
liquid hydrogen core stage propelled by five redesigned SSMEs. Attached 
to this core stage are two newly developed five-segment RSRBs, allowing 
over 100 metric tons to be launched to low-Earth orbit. The upper 
stage, which also serves as the EDS, uses an external tank-derived LOX/ 
liquid hydrogen system and will employ two Saturn-derived J-2 engines. 

LSAM: The LSAM is an expendable two-stage module launched atop the 
CaLV. The descent stage will utilize a LOX/liquid hydrogen propulsion 
system while the ascent stage will use a pressure-fed LOX-methane 
propulsion system. A crew cabin will be located on the ascent stage and 
will have an airlock to allow docking with the CEV. The LSAM will be 
able to land at any location on the lunar surface and will house a four-
member crew for up to 7 days. 

Cost Estimating: 

Cost-Estimating Process: 

NASA's Cost Estimating Handbook outlines cost-estimating processes in 
relation to acquisition life cycle phases. 

* In Pre-Phase A, there are many unknowns. At this point, the most 
effective cost-estimating approach is a parametric or analogous 
methodology, i.e., data from projects with similar attributes is used 
to predict the cost. 

* In Phase A, conceptual designs are better defined and a better 
understanding of the system requirements and technical risks exists. 
But, parametric or analogous cost-estimating techniques are still used, 
because detailed data may still be unavailable. 

* In Phase B, system designs are defined below the subsystem level. At 
this point, estimating methodologies evolve to more detailed parametric 
or engineering buildup estimates supported by technical experts. By the 
end of Phase B, specific data are available to prepare a full life 
cycle cost estimate. 

* In Phases C and D, cost estimates are refined to include actual data. 
At this point, the preferred cost methodology is an engineering buildup 
based on the lowest level of detail available, including overhead, 
labor, and material costs. 

Firm Cost Estimates Cannot Be Developed at This Time: 

NASA's cost estimates for implementing its exploration architecture are 
preliminary--a fact that NASA has acknowledged since the ESAS was 
publicly released. As part of the ESAS effort, NASA laid out the cost 
estimates for implementing the recommended architecture. Because the 
ESAS effort was an early life cycle activity, Pre-Phase A, the majority 
of the individual estimates were based upon parametric models, with 
little actual data. 

The ESAS process evaluated the cost of various alternative exploration 
architectures based upon high-level program requirements. The 
recommended architecture costs totaled: 

* over $31 billion dollars through fiscal year 2011, 

* over $122 billion through fiscal year 2018, and: 

* close to $230 billion through fiscal year 2025[Footnote 7] 

NASA conducted a cost risk analysis of the estimates through fiscal 
year 2011. This analysis provided a 65 percent confidence level for the 
estimate (i.e., NASA is 65 percent certain that the actual cost of the 
program will either meet or be less than the estimate). To obtain this 
level of confidence in the estimates, NASA included programmatic 
reserves--20 percent on all development and 10 percent on all 
production costs. NASA only conducted the risk analysis through the 
first flight date of the CEV at the time of ESAS--2011--leaving the 
estimates through 2018 and 2025, when most of the cost risk for 
implementing the architecture will be realized, with no confidence 
level distinction. According to NASA officials, the cost risk analysis 
lacked quality because of the evolving nature of the requirements for 
the architecture and the compressed time frames with which they had to 
conduct the analysis. According to NASA officials, once they receive 
more detailed contractor inputs, the agency will be able to produce 
higher-fidelity estimates of program cost. NASA has stated that it 
would not commit to a cost estimate for implementing the exploration 
architecture until the Constellation program's PDR, which will occur in 
late fiscal year 2008. At that time, the requirements, design, 
schedule, and cost will all be baselined. 

NASA refined the architecture several times since ESAS. As a result of 
these changes, the costs associated with the architecture have also 
changed. As part of the fiscal year 2007 budget formulation process, 
NASA made two major changes to plans laid out in the ESAS. First, the 
requirement for use of a LOX/methane engine on the CEV service module-
-a high-risk development--was removed, and the approach for meeting the 
propulsion requirement was left to the discretion of the contractor. 
Second, the first flight of the CEV was delayed until no later than 
2014. 

NASA's Life Cycle for Flight Systems and Ground Support Projects 
through Phase D: 

[See PDF for Image] 

Source: NASA. 

SRR = System Requirements Review
SDR = System Definition Review
NAR = Non=Advocate Review
Pre-Nar = preliminary Non-Advocate Review
PDR = Preliminary Design Review
CDR = Critical Design Review

[End of Figure] 

Subsequent to the submission of NASA's fiscal year 2007 budget, the 
Constellation program conducted an internal bottom-up review (BUR) of 
program costs. The goal of the BUR was to identify the funding it would 
take to "get the job done," which, according to the BUR guidance, means 
conducting the first flight of the CEV to the ISS by 2012 and first 
lunar mission by 2017. This review attempted to determine the cost 
impact of several major changes that were made to the architecture. 
These changes included a reduction in CEV diameter from 5.5 to 5 
meters, use of a five-segment RSRB and a Saturn-derived J-2x engine on 
the upper stage of the CLV, deletion of the unpressurized cargo CEV, 
the addition of an ISS docking system (Androgynous Peripheral 
Attachment System), and the inclusion of a Ka Band for High Definition 
Television on the CEV. Some of these architecture changes may help 
lessen technology development risks in the future program due to the 
planned commonality between the CLV and CaLV launch systems. While the 
results of this review were an attempt to provide more fidelity to the 
Constellation program's cost estimates, given the continued lack of a 
firm program baseline for requirements, design, and schedule, along 
with a continued lack of input from contractors, it is unlikely that 
the program had the level of detail available to support a true 
estimate of total costs this early in the program life cycle. 

ESMD is conducting a follow-on review to the Constellation program's 
BUR as NASA enters its fiscal year 2008 budget formulation cycle. As 
part of this latest review, NASA has continued to evaluate changes to 
the program architecture and schedule, such as the use of the RS-68 
engine on the CaLV and the delay of the first lunar mission to either 
fiscal year 2019 or fiscal year 2020. 

The continued evolution of the exploration architecture serves to 
highlight the preliminary nature of architecture itself and its 
associated cost estimates. Although NASA is continuing to refine its 
cost estimates for implementing the architecture to provide a more 
reliable estimate of cost, history suggests that program costs could 
increase significantly over estimates. In 2004, CBO reported that 
fulfilling the Vision could require the addition of billions of dollars 
to NASA's estimates of cost or extending the schedule for the first 
lunar landing by several years. Applying NASA's average cost growth 
figure of 45 percent to the ESAS cost estimates, assuming NASA business 
as usual, would result in an increase of almost $14 billion over the 
$31 billion it estimates it will need through 2011. With a significant 
increase in NASA budgets unlikely, given the current national fiscal 
imbalance, this level of cost growth could result in an unsustainable 
long-term exploration program. 

Cost Estimate Issues:

Historically, NASA has shown that it lacks a clear understanding of how 
much its programs will cost and how long they will take to achieve 
their objectives. NASA's cost estimates have often been unreasonable 
when committing to programs because of several factors, including 
inadequate requirements definition; changes in program content; and 
inadequate processes to establish priorities, quantify risks, and make 
informed investment decisions. GAO has reported on these issues for 
several years in both its high-risk series and in specific reviews of 
programs where NASA failed to apply discipline to its cost estimates to 
ensure those estimates were reasonable. For example, in 2002, GAO 
reported that since 1995, estimates for completion of the ISS had 
increased by $13 billion and the scheduled completion date had slipped 
4 years. Also, in 2004, GAO conducted a review of 27 other NASA 
programs and reported that the initial baseline estimates for over half 
of those programs were understated. 

Costs for NASA programs have historically been greater, on average, 
than initial estimates anticipated. A 2004 Congressional Budget Office 
(CBO) examination of 72 NASA programs spanning the past 30 years found 
that costs of NASA programs have increased, on average, 45 percent from 
initial budget estimates.

Funding Shortfalls: 

Expected Budget Challenges Architecture Implementation: 

NASA will be challenged to implement the exploration architecture, 
given the agency's expected budget profile. The ESAS effort defined the 
recommended architecture and preliminary costs, which NASA contends 
would allow the program to be accomplished within available budgets 
through fiscal year 2011. However, phasing issues still needed to be 
resolved. On an annual basis, NASA cannot afford to implement the 
architecture, although, cumulatively, for fiscal years 2007-2011, the 
agency says it has the money available. Beginning with fiscal year 2014 
and for the remainder of the decade, where the anticipated available 
budgets were adjusted for inflation, the ESAS cost projections show 
yearly multibillion-dollar shortfalls with an overall deficit through 
2025 of over $18 billion. 

The projected ESMD available budget figures used in the ESAS were 
developed well in advance of NASA's fiscal year 2007 President's budget 
submission. However, using the updated budget estimates from the fiscal 
year 2007 budget, the phasing issue becomes more pronounced when 
compared to ESAS estimated costs. As shown in the chart below, ESAS 
estimates could be accommodated within the ESMD available budget 
through fiscal year 2007. From fiscal year 2008 through fiscal year 
2010, however, NASA anticipates annual budget shortfalls for 
implementing the architecture within ESMD to exceed $1 billion per 
year. This shortfall could be partially offset, at least within the 
Constellation program, by a carryover of approximately $1 billion in 
both fiscal years 2006 and 2007 as a result of funds redirected from 
R&T activities within ESMD to that program. In addition, NASA officials 
stated the Constellation program has requested more funding than 
required for its projects in several years to cover shortfalls in later 
years. For example, the Exploration Communication and Navigation 
Systems project within the Constellation program plans to roll over 
$56.2 million from the fiscal year 2007 budget to make up for budget 
shortfalls in fiscal years 2008, 2009, and 2010. 

[See PDF for image] 

Source: NASA (data) and GAO (analysis). 

[End of figure] 

NASA's approach, however, appears to be contrary the agency's stated 
"go as you can afford to pay" approach to implement priority missions 
within available resources. In addition, the surplus shown in fiscal 
year 2011 is dependent upon dollars becoming available from the 
retirement of the space shuttle fleet, even though NASA officials 
stated the costs associated with retiring the space shuttle and 
transitioning to new architecture are not fully understood and the 
expected surplus could be less than anticipated. The shortfall 
presented by the fiscal year 2007 budget would not allow NASA to 
accomplish the stated program objectives within available resources 
over the next 5 years. 

In addition, changes to the architecture implementation schedule have 
not been consistent within the Constellation program. As previously 
stated, NASA moved the scheduled initial operational capability (IOC) 
date of the CEV to no later than 2014 during the fiscal year 2007 
budget formulation process. This change, along with modifications to 
the architecture, allowed NASA's estimates to meet its overall budget 
profile, despite continued year-to-year budget phasing issues. However, 
because of NASA's focus on minimizing the gap between the retirement of 
the space shuttle and the first flight of the CEV to the ISS, the 
program continued to attempt to meet the earlier IOC date for the CEV 
through its various analysis cycles. The earlier 2012 IOC date was 
retained as the planning date during the bottom-up review process, the 
Phase II request for proposal to the contractors involved CEV 
development, and the recent announcement concerning its intention to 
purchase the J-2x engine for the CLV from Pratt & Whitney Rocketdyne. 

The 2012 date for CEV IOC, in addition to changes the Constellation 
program made to the architecture during the BUR process, did not 
alleviate issues with the short-term funding profile. According to 
Constellation program officials, the net result of these changes will 
add more cost to the early years of the program, when funding is 
already constrained and phasing issues persist. Although the results of 
the BUR will not be released, indications from Constellation program 
officials are that the estimated costs of the program are higher than 
the ESAS estimated costs and available funding per NASA's budget 
profile. 

In the meantime, NASA continues to look for ways to resolve its budget 
phasing issues, such as by making additional changes to the exploration 
architecture. As the Constellation program executes its budget 
formulation process for the fiscal year 2008 budget cycle, it is 
currently analyzing options to the current architecture in an attempt 
to reduce development and production costs. For example, NASA recently 
announced that it intends to use five RS-68 engines instead of five 
SSMEs for the CaLV core stage, which would also require the CaLV core 
stage diameter to be increased to approximately 33 feet to accommodate 
the additional propellant needed by the RS-68 engines. 

NASA Funding Approach: 

The NASA Administrator recently testified that the agency is facing 
challenges to ensuring adequate funding for the priorities of the 
President and Congress within available budgetary resources. He stated 
that NASA has adopted a "go as you can afford to pay" approach to 
funding its exploration missions. This approach assumes NASA's top line 
budget will grow at the moderate rate identified in the President's 
fiscal year 2007 budget request. 

Under this approach, NASA would implement its priority missions within 
available resources and planned budgets through the redirection of 
funding for longer-term and lower-priority R&T elements within ESMD. As 
a result, several ESMD R&T programs and missions were discontinued, 
descoped, or deferred. That funding, in turn, was shifted into the 
Constellation Program to accelerate development of the CEV and the CLV. 
 
Gap in Human Spaceflight: 

The Vision called for retirement of the space shuttle fleet by the end 
of this decade and that the CEV should be available no later than 2014, 
creating a potential gap in human spaceflight of up to 4 years. The 
NASA Administrator has stated that it is a priority of the agency to 
close this gap and that the agency has taken steps to have the CEV in 
service as close to 2010 as possible. 

On the basis of lessons learned from the period between the end of the 
Apollo Program and the first flight of the space shuttle, the 
Administrator outlined several reasons why the CEV should not be 
delayed. These reasons include the potential for: 

* stagnation in the aerospace industry, 

* loss of critical expertise, 

* withering of the industrial base, 

* higher overall program costs, 

* program schedule delays, and: 

* loss of leadership in space exploration. 

Congress has also voiced its concern over the potential gap in human 
spaceflight. In the National Aeronautics and Space Administration 
Authorization Act of 2005, Congress stated it is the policy of the 
United States to have the capability for human access to space on a 
continuous basis. 

CEV Project: 

Best Practices: 

GAO has frequently reported on the importance of developing a sound 
business case before committing resources to a new product development 
effort. The business case in its simplest form is demonstrated evidence 
that (1) the need for the product is valid and that it can best be met 
with the chosen concept, and (2) the chosen concept can be developed 
and produced using existing and reasonably expected resources. 

GAO has undertaken a best practices body of work on how leading 
developers use a knowledge-based approach to develop products that 
reduces risks and increases the likelihood of successful outcomes. This 
type of approach is based on the premise of attaining knowledge about a 
program and the resources available before making a contractual or 
financial commitment. Knowledge that supports a sound business case 
includes well-defined requirements, a preliminary design, mature 
technology, and realistic cost estimates. 

Use of this approach has enabled leading organizations to deliver high- 
quality products on time and within budget. Conversely, GAO has also 
reported that major systems that have not established a sound business 
case have been plagued by cost overruns, schedule delays, decreased 
capability, and overall poor performance. NASA's track record in 
developing systems has not been good. GAO and others have reported that 
NASA has had numerous problems with its programs and projects, 
including underestimating program complexity and technology maturity, 
and inadequate review and systems engineering processes. 

Lack of Sound Business Case Puts CEV Acquisition at Risk: 

NASA's acquisition strategy for the CEV places the project at risk of 
cost overruns, schedule delays, and performance shortfalls because it 
commits the government to a long-term product development effort before 
establishing a sound business case. In September 2006, NASA plans to 
award a contract for design, development, production, and sustainment 
of the CEV--before it has developed well-defined requirements, a 
preliminary design, mature technology, and firm cost estimates for the 
project. The CEV project might not have all the elements of a sound 
business case in place until the project-level PDR in March 2008. At 
the completion of the PDR, NASA will approve the selected prime 
contractor's preliminary design based on detailed, validated 
requirements. Further, CEV project officials indicated that the CEV 
project plans to retire all technology risks by the PDR. At that point, 
NASA will likely have the increased knowledge necessary to develop a 
sound business case that includes high-fidelity, engineering-based 
estimates of life cycle cost for the CEV project. With this business 
case in hand, NASA would be in a better position to commit the 
government to a long-term design and development effort. NASA officials 
disagree and have stated that it is appropriate for them to proceed 
with the contract award because the agency is selecting "a designer, 
not a design" for the CEV. In reality, by awarding a contract as 
planned in September 2006, NASA is not only committing to an unknown 
design but to production and long-term sustainment of the CEV as well. 

The CEV contract scheduled for award in September 2006 will have three 
schedules. At the time of contract award, NASA will be responsible for 
fee earned and the reasonable, allowable, and allocable costs incurred 
by the contractor in the performance of Schedule A and the minimum 
quantities under Schedules B and C. 

* Schedule A is for design, development, test and evaluation of the 
CEV. Deliverables under Schedule A include all test articles and two 
operational CEV vehicles--one human-rated variant and one pressurized 
cargo variant. 

* Schedule B is for production beyond the two operational vehicles 
delivered under Schedule A. The CEV request for proposal states that 
the "guaranteed minimum" quantity for Schedule B is "two CEV," the type 
of which, according to NASA officials, is undetermined. 

* Schedule C is for sustainment in support of operations and in support 
of Schedule B activities. 

CEV Timeline: 

[See PDF for Image] 

Source: NASA (data) and GAO (presentation). 

Note: Contract award for all schedules is planned for September 2006. 
Schedule B and C performance periods are from 2009 to 2014 with an 
additional 5-year performance option to end in 2019. 

[End of Figure] 

An important step in developing a sound business case is defining 
requirements. The acquisition strategy for the CEV lays out a series of 
reviews to validate and approve CEV requirements. These reviews result 
in approved system-level requirements at the October 2006 System 
Requirements Review (SRR), and approved subsystem-level requirements at 
the April 2007 System Definition Review (SDR) and culminate with 
validated and approved component-level requirements at the March 2008 
PDR. Under the current CEV strategy, NASA will select the winning 
contractor about 1 month before the system level requirements are 
approved at the SRR, over a year and a half before detailed component- 
level requirements are approved at the PDR. 

Another aspect of a sound business case is having mature technologies 
before committing to product development. The CEV's acquisition 
strategy is predicated upon using mature technologies as the basis for 
system development. However, contractors will also be given discretion 
to include immature technologies in areas where technology advancement 
is critical to meeting requirements. NASA has independently identified 
technology risks and implemented advanced technology development 
projects to address risks in the areas of the thermal shielding needed 
for reentry and the landing systems needed for ground landings. CEV 
project officials also expect that each contractor's proposal will 
include additional technology development risks. Under the current CEV 
strategy, NASA is awarding a contract for product development and 
production of the first two variants of the CEV before it has resolved 
these technology development risks. 

Past Development Attempts: 

NASA has tried unsuccessfully to develop a number of vehicles to 
replace the shuttle over the past three decades. In the 1980s NASA 
initiated the National Aero-Space Plane (NASP) to build and test a 
manned experimental flight vehicle for demonstrating single-stage-to- 
orbit space launch and sustained hypersonic cruise capability. NASA 
canceled the program as it was experiencing cost overruns, schedule 
delays, and technology problems. GAO reported that from 1986 to 1993 
NASA spent $398 million for the NASP program. 

In the 1990s, NASA began the X-33 program to develop single-stage-to 
orbit technology and the X-34 to demonstrate reusable two-stage-to - 
orbit technologies. According to a 2006 Congressional Research Service 
report, NASA terminated the X-33 and X-34 in March 2001--after spending 
over $1.4 billion--because the cost to complete them was too high 
relative to the benefits. In 1999, GAO reported that technical problems 
and unrealistic cost estimates on the X-33 project alone led to cost 
overruns of $75 million and over a year's delay. 

In 2004, after the announcement of the Vision, NASA canceled the Space 
Launch Initiative (SLI) program, which was to provide both launch 
capabilities and an emergency crew return from the ISS. NASA's 
Inspector General reported that NASA did not verify and validate basic 
requirements for its second generation space transportation, while GAO 
reported that key management controls could not be implemented until 
such requirements were defined. GAO estimates that from 2001 to 2005 
NASA provided the SLI program with about $3 billion in funding. 

Appendix: 

Scope and Methodology: 

To assess the extent to which NASA has identified the architecture and 
costs necessary to implement the Vision and whether NASA's exploration 
architecture fits within the agency's projected available budgets, we 
reviewed and analyzed NASA's Exploration Systems Architecture Study, 
fiscal year 2007 budget request, ground rules and assumptions provided 
from the Constellation program to project-level management estimators 
to perform the BUR, guidance for use in preparing the fiscal year 2008 
budget request, NASA cost-estimating guidance in the NASA Cost 
Estimating Handbook, and congressional hearings and testimonies 
pertaining to NASA and the Vision. We also conducted interviews with 
NASA headquarters officials from the Cost Analysis Division, the 
Exploration Systems Mission Directorate, and the Constellation Program; 
Constellation program and CEV project officials at Johnson Space 
Center; CLV project officials at Marshall Space Flight Center; and cost 
analysts from Kennedy Space Center. During these interviews, we 
discussed the methodologies used in preparing the ESAS and subsequent 
cost estimates, architecture changes after ESAS and the trades being 
considered, budgeting issues, and procurement strategies and 
activities. 

To assess the risks associated with NASA's acquisition strategy for the 
CEV project, we reviewed and analyzed CEV project documentation, 
including draft project plans, draft requirements documents, technology 
development plans, documentation included in the contract request for 
proposals, and documentation for past NASA human space flight 
acquisition programs. We compared NASA's plans for the CEV with 
criteria contained in GAO best practices work on systems acquisition. 
We also conducted interviews with NASA headquarters officials from the 
Exploration Systems Mission Directorate, Constellation Program and CEV 
project officials at Johnson Space Center, and CLV project officials at 
Marshall Space Flight Center. 

Contributors: 

If you have any questions concerning this briefing, please call Allen 
Li at (202) 512-4841. Other key contributors to this briefing were 
James L. Morrison, Assistant Director; Rick Cederholm; Shelby S. 
Oakley; Guisseli Reyes; Sylvia Schatz; and John S. Warren, Jr. 

FOOTNOTES 

[1] The ESAS architecture supports the development of a new Crew 
Exploration Vehicle (CEV), Crew Launch Vehicle (CLV), a Cargo Launch 
Vehicle (CaLV), and other supporting systems. The architecture also 
calls for various Research and Technology (R&T) and Robotic Lunar 
Exploration Program (RLEP) projects. 

[2] All cost estimates related to the Vision are reported as inflated 
("real year") dollars. 

[3] NASA's cost estimate through 2011--$31.2 billion--included the 
costs of the R&T and RLEP projects needed to support the architecture. 
Its estimate for the first lunar landing--$104 billion--did not include 
$18 billion in funding for R&T and RLEP projects. To ensure 
consistency, the estimates for 2018 and 2025 are presented with R&T and 
RLEP funding included. 

[4] GAO, 21st Century Challenges: Reexamining the Base of the Federal 
Government, GAO-05-325SP (Washington, D.C.: Feb. 2005); 21st Century: 
Addressing Long-Term Fiscal Challenges Must Include a Reexamination of 
Mandatory Spending, GAO-06-456T (Washington, D.C.: Feb. 15, 2006); and 
Highlights of a GAO Forum: The Long-Term Fiscal Challenge, GAO-05-282SP 
(Washington, D.C.: Feb. 1, 2005). 

[5] The Vision includes a return to the moon that is intended 
ultimately to enable future exploration of Mars and other destinations. 
To accomplish this, NASA initially plans to (1) complete its work on 
the International Space Station by 2010, fulfilling its commitment to 
15 international partner countries; (2) begin developing a new manned 
exploration vehicle to replace the space shuttle; and (3) return to the 
moon no later than 2020 in preparation for future, more ambitious 
missions. 

[6] Examples of our best practices reports include GAO, Best Practices: 
Using a Knowledge-Based Approach to Improve Weapon Acquisition, GAO-04- 
386SP (Washington, DC.: Jan. 2004); Space Acquisitions: Committing 
Prematurely to the Transformational Satellite Program Elevates Risks 
for Poor Cost, Schedule, and Performance Outcomes, GAO-04-71R 
(Washington, D.C.: Dec. 4, 2003); Best Practices: Capturing Design and 
Manufacturing Knowledge Early Improves Acquisition Outcomes, GAO-02- 
701 (Washington, D.C.: Jul. 15, 2002); and Best Practices: Better 
Matching of Needs and Resources Will Lead to Better Weapon System 
Outcomes, GAO-01-288 (Washington, DC.: Mar. 8, 2001).

[7] NASA's cost estimate through 2011--$31 billion--included the costs 
of the R&T and RLEP projects needed to support the architecture. Its 
estimate for the first lunar landing--$104 billion--did not include $18 
billion in funding for R&T and RLEP projects. To ensure consistency, 
the estimates for 2018 and 2025 are presented with R&T and RLEP funding 
included. The estimates include $20 billion to service the ISS. 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, 

NelliganJ@gao.gov 

(202) 512-4800 

U.S. Government Accountability Office, 

441 G Street NW, Room 7149 

Washington, D.C. 20548: