This is the accessible text file for GAO report number GAO-11-364R 
entitled 'Additional Cost Transparency and Design Criteria Needed for 
National Aeronautics and Space Administration (NASA) Projects' which 
was released on March 3, 2011. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

GAO-11-364R: 

United States Government Accountability Office: 
Washington, DC 20548: 

March 3, 2011: 

The Honorable Charles Bolden: 
Administrator: 
National Aeronautics and Space Administration: 
300 E Street, SW: 
Washington, DC 20024-3210: 

Subject: Additional Cost Transparency and Design Criteria Needed for 
National Aeronautics and Space Administration (NASA) Projects: 

Dear Administrator Bolden: 

Today, GAO published its third annual assessment of selected large- 
scale NASA projects.[Footnote 1] During this assessment we identified 
several issues that merit your management attention. 

The federal government faces real fiscal limitations and will have to 
make difficult choices about upcoming priorities. This reality makes 
it more important than ever that NASA manage its programs and projects 
as efficiently and effectively as possible and within a budget that 
over recent years has remained relatively constant. It will also 
require that NASA make tough decisions about which projects to fund 
among core missions in science, aeronautics, and human space flight 
and exploration. Our work over the past three years has shown that 
NASA's major projects are frequently approved without evidence of a 
sound business case--ensuring a match between requirements and 
resources--and, therefore, cost more and take longer to develop than 
planned. Our March 2011 assessment found that 13 NASA projects that 
established baselines prior to fiscal year 2009 had experienced an 
average cost growth of almost 55 percent, with a combined increase in 
development costs of almost $2.5 billion from their baselines 
established at their Confirmation Review.[Footnote 2] While NASA has 
taken steps over recent years to help improve its acquisition 
management through several initiatives aimed at improving cost 
estimating and management oversight, the overall outcomes of these 
efforts will take time to become apparent. Based on the findings of 
our past three assessments, we are recommending that NASA (1) provide 
increased transparency into project costs to the Congress to conduct 
oversight and ensure earlier accountability and (2) develop a common 
set of measurable and proven criteria to assess the design stability 
of projects before proceeding into later phases of development. 

Lack of Transparency Into Early Project Development Costs: 

As we have reported, NASA does not provide enough transparency into 
costs during early project development to provide the Congress with 
sufficient information to conduct oversight and ensure earlier 
accountability.[Footnote 3] While there is a need to allow projects a 
period of time for discovery and to pursue different concepts, 
projects can spend significant sums of time and money before they 
enter implementation[Footnote 4] and are held accountable to a formal 
baseline. For example, authorized in 2005 and 2006 respectively, the 
Ares and Orion projects[Footnote 5] have spent over $9 billion 
combined, but neither has reached implementation. Furthermore, the 
James Webb Space Telescope was authorized in 1999, reached 
implementation in 2008, and spent nearly $2 billion during those 9 
years. The Senate Committee on Commerce, Science, and Transportation 
recently asked GAO to testify on how NASA can improve its overall 
transparency and accountability. 

Currently, NASA does not provide to the Congress cost and schedule 
information for projects in the early, critical phases of development 
and makes this information public only after the projects have been 
formally approved to enter implementation. Projects establish 
preliminary cost estimates in the formulation phase; these estimates, 
however, are for planning purposes only as they enable NASA decision- 
makers to better manage the overall portfolio of projects. NASA does 
not report deviations from these early estimates to the Congress. 
Although progress is not measured or reported externally against early 
planning baselines, cost growth and schedule delays can and do occur 
during the formulation phase. NASA's internal analysis of past 
projects indicates that there is an average of 14 percent growth in 
the development cost estimates during the formulation phase. 
Additionally, NASA does not report information on the funding that has 
been authorized to date for the projects in formulation, as it does in 
its annual budget submission for projects in implementation. 
Additional insight into costs would provide critical information to 
help match project requirements to resources and could better enable 
the Congress to make more informed decisions when approving the 
projects through the annual appropriations process. Our prior reports 
have shown that early insight into project progress can put decision-
makers in a better position to ensure projects have the resources 
necessary for success. 

Lack of Design Metric May Contribute to Project Cost Growth: 

During the course of our past three reviews of large-scale NASA 
projects and other work examining NASA's acquisition management, 
[Footnote 6] we found that NASA does not use a common, proven metric 
to assess design stability before allowing programs to move from the 
design phase to the test and integration phase[Footnote 7] of the 
development process. NASA's acquisition policy does not specify a 
metric by which a project's design stability is measured at the 
critical design review.[Footnote 8] Guidance in NASA's Systems 
Engineering Handbook, however, mirrors an accepted best practice 
[Footnote 9] that at least 90 percent of engineering drawings should 
be releasable by the critical design review, although projects are not 
required to follow this guidance.[Footnote 10] We have found that if 
design stability is not achieved at the critical design review, but 
product development continues, costly re-designs to address changes to 
project requirements and unforeseen challenges can occur. As shown in 
figure 1, nearly all of the projects we reviewed over the last three 
years held their critical design review without 90 percent of 
engineering drawings being releasable--failing to meet NASA's Systems 
Engineering Handbook guidance and our best practices criteria for 
design stability. The only two projects in our 2011 review that met 
the design criteria--Orbiting Carbon Observatory-2 and Tracking and 
Data Relay Satellite Replenishment--to date have experienced no cost 
growth; however, these projects are based heavily on previous designs. 
Ten projects did not meet the design criteria and, of those, nine had 
experienced development cost growth since their baseline. 

Figure 1: 2009-2011 Trend of Design Criteria for Projects in 
Implementation: 

[Refer to PDF for image: vertical bar graph] 

Year: 2009; 
Projects in implementation not meeting design criteria: 100%. 

Year: 2010; 
Projects in implementation not meeting design criteria: 100%. 

Year: 2011; 
Projects in implementation meeting design criteria: 83%; 
Projects in implementation not meeting design criteria: 17%. 

Source: GAO analysis data provided by NASA. 

Note: In our 2009, 2010, and 2011 assessments we reviewed 10, 9, and 
12 projects, respectively, that were in implementation and had held 
their critical design review. 

[End of figure] 

Some of the projects we reviewed in the past three years identified 
other activities that occurred prior to the critical design review as 
evidence of design stability. In addition to releasable engineering 
drawings, NASA relies on subject matter experts in the design review 
process and other methods to assess design stability. For example, a 
panel of experts within NASA provides an assessment of the technical 
and programmatic approach, risk posture, and progress against the 
project baseline at key decision points to be assured that the project 
has a stable design. Some projects rely on using engineering models 
and engineering test units to assess design stability. According to 
NASA's systems engineering policy, at the critical design review the 
project's design should be stable enough to support full scale-
fabrication, assembly, integration, and test.[Footnote 11] However, a 
recent study by the National Research Council[Footnote 12] found that 
the critical design review milestone for many NASA missions may be 
held prematurely--driven by schedule rather than driven by design 
maturity. Furthermore, we found that the majority of the 12 projects 
in our March 2011 review[Footnote 13] that had held their critical 
design review experienced increases in the number of engineering 
drawings that were expected after that review--an indication of an 
unstable design after the critical design review. This is particularly 
evident for the four projects that held their critical design reviews 
prior to fiscal year 2009--the projects that entered the test and 
integration phase in 2009 or before and have more of a history to 
track variances.[Footnote 14] As shown in figure 2 below, the four 
projects, on average, had a 107 percent increase in expected 
engineering drawings after the critical design review after having 
only 36 percent of drawings releasable at that review. The remaining 
eight projects that held their critical design review in fiscal year 
2009 or later have not reported a large increase in expected drawings. 
All of these projects will enter or have entered the test and 
integration phase in 2010 or later. A National Research Council study 
states that approval of an immature design at the critical design 
review can cause problems during test and integration such as 
integration difficulties and late design changes. An increase in 
design drawings is one indication of late design changes. 

Figure 2: Comparison of Design Drawing Increase for Projects with 
Critical Design Review (CDR) prior to and since Fiscal Year 2009: 

[Refer to PDF for image: vertical bar graph] 

Projects with CDR prior to FY 2009: 
Average drawings released at CDR: 36%; 
Average increase in expected drawings after CDR: 105%. 

Projects with CDR in FY 2009 or later: 
Average drawings released at CDR: 75%; 
Average increase in expected drawings after CDR: 10%. 

[End of figure] 

Regardless of how stability is measured, consistent, proven, and 
quantitative measures employed at the critical design review--such as 
the percentage of engineering drawings--can provide evidence that the 
design is stable and will meet performance requirements. These 
measures can also be an indication to decision-makers that the 
requisite knowledge has been attained to allow the project to proceed 
in its development lifecycle and better enable them to assess the 
performance of individual projects against the overall portfolio of 
projects. 

Conclusions: 

Over the past several years, our assessments have shown that NASA's 
major acquisitions have been marked by cost overruns and schedule 
delays. Given the multiple priorities that NASA must balance--coupled 
with the constraints that will continue to bind the government's 
fiscal resources for many years--ensuring that NASA is basing its 
acquisition decisions on correct, timely, and consistent information 
is critical. While NASA has undertaken several initiatives aimed at 
improving its acquisition management, the lack of transparency into 
the cost of projects in the early, critical phases of development 
leaves the Congress with incomplete knowledge to inform its oversight 
and ensure earlier accountability. While there is a need to allow 
projects a period of time for discovery and to pursue different 
concepts, particularly highly complex efforts, inadequate transparency 
into project progress for what sometimes amounts to 5 or more years 
can preclude effective oversight and accountability and make it even 
more difficult to stop projects that are not on track to meet the 
agency's goals with available resources. Finally, without a standard 
design metric to measure the projects' progress at crucial points in 
the development life cycle, NASA cannot be assured that its decisions 
will result in the best possible return on its investments. 

Recommendations for Executive Action: 

To provide increased transparency into project risks, we recommend 
that you: 

* Direct the Office of the Chief Financial Officer (OCFO) to provide 
more transparency into project costs in the early, critical phases of 
development so that the Congress has sufficient information to conduct 
oversight and ensure earlier accountability. Specifically, the OCFO 
should provide progress reports for NASA space flight programs and 
projects in formulation that include information on cumulative prior 
budget authority and current cost ranges in NASA's annual budget 
submission to the Congress to enhance the knowledge with which they 
base funding decisions for NASA. 

To provide NASA decision-makers with a mechanism to consistently and 
accurately judge design stability, we recommend that you: 

* Direct the Office of the Chief Engineer to develop a common set of 
measurable and proven criteria, such as the percentage of releasable 
design drawings, to assess design stability and to allow decision- 
makers to make more informed, consistent determinations of approval 
for projects to proceed from the final design phase to the assembly, 
integration, and test phase of the development process and amend 
NASA's systems engineering policy, accordingly. 

Agency Comments and Our Evaluation: 

In written comments on a draft of this report, the Deputy 
Administrator of NASA partially concurred with our first 
recommendation and concurred with our second the recommendation. 
NASA's comments are reprinted in the Enclosure. 

NASA partially concurred with our recommendation to provide more 
transparency into project costs in the early, critical phases of 
development in order to provide the Congress sufficient information to 
conduct oversight and ensure earlier accountability. While NASA agreed 
that clarity is needed into life-cycle cost range estimates, it stated 
that the estimates developed during formulation are not a baseline, 
and therefore not a basis for measurement of cost growth. Preliminary 
cost estimates are used by NASA to determine whether there is a good 
match between requirements and resources when designing a project. 
NASA stated that it currently provides these preliminary cost range 
estimates upon request, and is considering providing increased 
information for projects in early formulation in the annual budget 
submission to the Congress. NASA did not commit, however, to providing 
information to the Congress on the cumulative prior budget authority 
for projects in formulation, as we recommended. We understand that 
preliminary cost estimates are distinct from the baseline costs of the 
project and are not intended as a basis to measure cost growth. These 
early cost estimates do, however, provide the basis of the financial 
investment that the agency is committing the government to and should 
be transparent to Congressional decision-makers for their deliberation 
and decision-making. The lack of transparency into project progress 
early in development can preclude the ability of the Congress to 
conduct oversight and ensure accountability and make it even more 
difficult to stop projects that are not on track to meet the agency's 
goals with available resources. Similarly, given NASA's various 
accounting changes throughout the past decade, it is difficult if not 
impossible for the Congress and parties external to NASA to determine 
the cumulative budget authority or the investment to date that has 
been given to a particular project. Providing this type of budget 
information would allow Congressional decision-makers insight into the 
commitments to the project thus far to make informed decisions about 
their continued approval. 

NASA concurred with our second recommendation that the agency develop 
a common set of measurable and proven criteria to assess design 
stability. NASA stated that the drawing release metric is a legacy 
standard developed prior to the use of computerized drawings, and 
hence does not take into account improvements due to the use of this 
technology. We acknowledge this point, but our analysis of NASA 
projects shows that those projects that have met or come close to 
meeting the best practices drawing release metric have fared better 
with regard to cost and schedule than those projects that did not come 
close to meeting the metric. NASA recognizes the need for metrics for 
greater insight into project formulation and development and stated 
that it is developing a common set of measurable and proven criteria 
to assess design maturity, which it will provide to GAO no later than 
March 31, 2011. 

Scope and Methodology: 

The majority of our findings in this correspondence were based on work 
completed as part of our 2011 Assessments of Selected Large-Scale 
Projects. In the assessment, we reviewed cost growth and design 
maturity of 21 projects, each with an estimated life-cycle cost of 
over $250 million. We conducted interviews with project officials and 
examined the current phase of a project's development and how each 
project was advancing. NASA provided updated cost and schedule data as 
of February 2011 for 16 of the 21 projects. We reviewed and compared 
that data to previously established cost and schedule baselines. We 
assessed each project's cost and schedule and characterized growth in 
either as significant if it exceeded the thresholds that trigger 
reporting to the Congress under the law.[Footnote 15] We also reviewed 
NASA's February 2011 budget submission to the Congress and noted 
differences between cost information provided for projects in 
implementation versus cost information provided for projects in 
formulation. In addition, NASA provided cost and schedule information 
from previously reported projects that we used for historical 
analysis. Through a standardized data collection instrument, we 
gathered basic information about the projects as well as current and 
projected development activities for those projects, including 
information concerning the project's engineering design drawings. We 
used this data to assess design stability against GAO's established 
criteria for knowledge-based acquisitions and on other GAO work on 
system acquisitions. We reviewed and discussed NASA policies and in-
house practices related to evaluating the design stability of each 
project. We took appropriate steps to address data reliability. 

We conducted this performance audit in February 2011 in accordance 
with generally accepted government auditing standards. Those standards 
require that we plan and perform the audit to obtain sufficient, 
appropriate evidence to provide a reasonable basis for our findings 
and conclusions based on our audit objectives. We believe that the 
evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

We are sending copies of this report to interested congressional 
committees. In addition, the report will be available at no charge on 
GAO's Web site at [hyperlink, http://www.gao.gov]. If you have any 
questions, please contact me at (202) 512-4841 or chaplainc@gao.gov. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this letter. 

Key contributors to this report were Shelby S. Oakley, Assistant 
Director; Jessica M. Berkholtz; Richard A. Cederholm; Laura Greifner, 
Kristine R. Hassinger, and Roxanna T. Sun. 

Signed by: 

Cristina Chaplain:
Director:
Acquisition and Sourcing Management: 

[End of section] 

Enclosure: Comments from the National Aeronautics and Space 
Administration: 

National Aeronautics and Space Administration: 
Office of the Administrator: 
Washington, DC 20546-0001: 

February 23, 2011: 

Ms. Christina Chaplain: 
Director: 
Acquisition and Sourcing Management: 
United States Government Accountability Office: 
Washington, DC 20548: 

Dear Ms. Chaplain: 

The National Aeronautics and Space Administration (NASA) appreciates 
the opportunity to review the Government Accountability Office (GAO) 
draft correspondence entitled "Additional Cost Transparency and Design 
Criteria Needed for National Aeronautics and Space Administration 
(NASA) Projects" (GAO-11-364R). NASA values the continued open 
communications between NASA and the GAO team on the annual assessment 
of large-scale projects and appreciates the constructive comments 
arising as a result of this effort. 

NASA agrees with the GAO's concern about managing programs and 
projects as efficiently and effectively as possible, especially within 
a budget that is likely to be constrained due to the fiscal 
limitations currently faced by all Federal Government agencies. 
Consequently, NASA remains dedicated to continuous improvement of 
acquisition management processes and performance. We are pleased that 
GAO recognizes NASA's ongoing efforts to mitigate acquisition 
management risk and lay a stronger foundation for reducing project 
cost and schedule growth. As was highlighted in the annual assessment, 
NASA implemented a Joint Cost and Schedule Confidence Level (JCL) 
policy in 2009 to increase the likelihood of project success at the 
specified funding level. Since institution of this policy, the average 
cost growth for NASA's large scale projects has remained below the 15 
percent threshold which, if breached, requires special notification to 
Congress. NASA will continue to assess the effect of utilizing JCLs to 
manage project cost and schedule growth. With the completion and 
launch of three missions in 2011, NASA will have a better measure of 
the impact of our acquisition management improvement efforts over the 
last five years. 

In the draft report, GAO makes two recommendations to the 
Administrator intended to increase transparency into project risks and 
to provide NASA decision makers with a mechanism to consistently and 
accurately judge design stability, specifically: 

Recommendation 1: Direct the Office of the Chief Financial Officer 
(OCFO) to provide more transparency into project costs in the early, 
critical phases of development so that the Congress has sufficient 
information to conduct oversight and ensure earlier accountability. 
Specifically, the ()CFO should provide progress reports for NASA space 
flight programs and projects in formulation that include information 
on cumulative prior budget authority and current cost ranges in NASA's 
annual budget submission to the Congress to enhance the knowledge with 
which they base funding decisions for NASA. 

Management's Response: Partially concur. NASA agrees with the need to 
provide clarity on life-cycle cost range estimates, but the estimates 
developed during formulation are not a baseline, and therefore not a 
basis for measurement of cost growth. As recognized by GAO, these 
estimates are used by NASA decision makers to manage the overall 
portfolio of projects. Furthermore, these preliminary estimates are 
used to help NASA determine whether there is a good match between 
requirements and resources given the project design trades at that 
point in time. The decision to approve a mission to proceed to 
development involves a complex balance between a mission's technical 
design, scientific and technical priorities, schedule, available 
resources within a portfolio of projects, and overall risk assessment 
of mission success given the requirements and constraints. Trades 
between all of these areas may or may not lead to growth in the cost 
estimate. When a project is approved to proceed to development, these 
trades are complete; NASA confirms that the requirements match the 
available resources, and commits to complete the project within the 
life-cycle cost estimate based on this match. 

NASA currently provides preliminary cost range estimates, inclusive of 
prior year costs, for projects that have passed Key Decision Point B 
(KDP-B), upon request. For example, these ranges are published 
annually as part of the GAO's assessment of NASA's large-scale 
projects. NASA is also considering providing increased information for 
projects in early formulation in the annual budget submission. 

Recommendation 2: Direct the Office of the Chief Engineer to develop a 
common set of measurable and proven criteria, such as the percentage 
of releasable design drawings, to assess design stability and to allow 
decision-makers to make more informed, consistent determinations of 
approval for projects to proceed from the final design phase to the 
assembly, integration, and test phase of the development process and 
amend NASA's systems engineering policy, accordingly. 

Management's Response: Concur. NASA agrees that projects should have a 
stable design before starting assembly, integration, and test. 
Standing Review Boards (SRB) comprised of independent technical and 
programmatic experts are asked to assess design stability at 
Preliminary Design Review (PDR) and Critical Design Review (CDR) using 
a number of criteria that are outlined in NASA's system engineering 
and independent review policies. The drawing release metric referenced 
in NASA's Systems Engineering Handbook is a legacy standard developed 
prior to the use of computerized drawings and hence does not take into 
account improvements due to the use of this technology. For this 
reason, although NASA continues to use drawing release as one of many 
indicators, the Agency does not place as much emphasis on drawing 
release when considered holistically along with other criteria to 
assess design stability. NASA does recognize the need for metrics for 
greater insight into project formulation and development. NASA concurs 
with GAO's recommendation to develop a common set of criteria to 
assess design stability. NASA is developing a common set of measurable 
and proven criteria to assess design maturity. The metrics will be 
provided to GAO no later than March 31, 2011. 

NASA will continue to follow through with our new policies and 
management attention on cost and schedule growth and project 
management in the coming year. We are committed to continuous 
improvement in order to explore and utilize space in an affordable way 
for the benefit of the Nation. To this end, we look forward to 
continuing to work with the GAO to measure and improve our performance 
and management practices. 

Thank you for the opportunity to comment on this draft report. If you 
have any questions or require additional information, please contact 
Katie Gallagher at (202) 358-2185. 

Sincerely, 

Signed by: 

Lori B. Garver: 
Deputy Administrator: 

[End of section] 

Footnotes: 

[1] GAO, NASA: Assessments of Selected Large-Scale Projects, 
[hyperlink, http://www.gao.gov/products/GAO-11-239SP] (Washington, 
D.C.: Mar. 3, 2011). 

[2] The confirmation review is the point at which cost and schedule 
baselines are established and approved for the project. Project 
progress is measured against these baselines. 

[3] GAO, NASA: Issues Implementing the NASA Authorization Act of 2010, 
[hyperlink, http://www.gao.gov/products/GAO-11-216T] (Washington, 
D.C.: Dec. 1, 2010); GAO-11-239SP. 

[4] There are two phases in NASA's project life cycle--the formulation 
phase and the implementation phase. In the formulation phase, the 
project defines requirements--what the project is being designed to do--
matures technology, establishes a schedule, estimates costs, and 
produces a plan for implementation. In the implementation phase, the 
project carries out these plans, performing final design and 
fabrication as well as testing components and system assembly, 
integrating these components and testing how they work together, and 
launching the project. This phase also includes the period from 
project launch through mission completion. 

[5] As part of the Constellation Program, the Ares I Crew Launch 
Vehicle was designed to carry the Orion Crew Exploration Vehicle into 
low-Earth orbit for missions to the International Space Station and 
the Moon. 

[6] GAO, NASA: Implementing a Knowledge-Based Acquisition Framework 
Could Lead to Better Investment Decisions and Project Outcomes, 
[hyperlink, http://www.gao.gov/products/GAO-06-218] (Washington, D.C.: 
Dec. 21, 2005); GAO, NASA: Assessments of Selected Large-Scale 
Projects, [hyperlink, http://www.gao.gov/products/GAO-09-306SP] 
(Washington, D.C.: Mar. 2, 2009); GAO, NASA: Assessments of Selected 
Large-Scale Projects, [hyperlink, 
http://www.gao.gov/products/GAO-10-227SP] (Washington, D.C.: Feb. 1, 
2010); [hyperlink, http://www.gao.gov/products/GAO-11-239SP]; and 
[hyperlink, http://www.gao.gov/products/GAO-11-216T]. 

[7] During the design phase, the project completes its preliminary 
design and technology development and begins fabrication of test and 
flight article components, assemblies, and subsystems. In the test and 
integration phase, the project initiates system assembly, integration 
and test in preparation for flight. 

[8] According to NASA Interim Directive for NASA Procedural 
Requirements 7120.5D, Table 2-7 (Sept. 22, 2009), the critical design 
review demonstrates that the maturity of the design is appropriate to 
support proceeding with full-scale fabrication, assembly, integration, 
and test, and that the technical effort is on track to complete the 
flight and ground system development and mission operations in order 
to meet mission performance requirements within the identified cost 
and schedule constraints. Progress against management plans, budget, 
and schedule, as well as risk assessments, are presented. 

[9] GAO, Best Practices: Capturing Design and Manufacturing Knowledge 
Early Improves Acquisition Outcomes, [hyperlink, 
http://www.gao.gov/products/GAO-02-701] (Washington, D.C.: July 15, 
2002). 

[10] Engineering drawings are considered to be a good measure of the 
demonstrated stability of a product's design because the drawings 
represent the language used by engineers to communicate to the 
manufacturers the details of a new product design--what it looks like, 
how its components interface, how it functions, how to build it, and 
what critical materials and processes are required to fabricate and 
test it. Once the design of a product is finalized, the drawing is 
"releasable". 

[11] NASA Procedural Requirement 7123.1A, NASA Systems Engineering 
Processes and Requirements Appendix G, paragraph G.8 (Mar. 26, 2007). 

[12] Controlling Cost Growth of NASA Earth and Space Science Missions, 
National Research Council, The National Academies, Washington D.C. 
2010. 

[13] Sixteen projects in our 2011 review were in the implementation 
phase. Of those, twelve projects had held their critical design 
reviews and could provide information on the number of design drawings. 

[14] We began tracking the percentage of drawing releasable as a 
measure of the design stability of selected large-scale NASA projects 
in our 2009 assessment. Since then, NASA provides drawing information 
on projects included in our yearly assessment on a semiannual basis. 

[15] NASA is required to report to Congress if development cost of a 
program is likely to exceed the baseline estimate by 15 percent or 
more, or if a milestone is likely to be delayed by 6 months or more. 
42 U.S.C. § 16613(d). 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: