This is the accessible text file for GAO report number GAO-14-778 
entitled 'F-35 Sustainment: Need for Affordable Strategy, Greater 
Attention to Risks, and Improved Cost Estimates' which was released on 
September 23, 2014. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to the Committee on Armed Services, House of Representatives: 

September 2014: 

F-35 Sustainment: 

Need for Affordable Strategy, Greater Attention to Risks, and Improved 
Cost Estimates: 

GAO-14-778: 

GAO Highlights: 

Highlights of GAO-14-778, a report to the Committee on Armed Services, 
House of Representatives. 

Why GAO Did This Study: 

The F-35 Lightning II is intended to replace a variety of existing 
aircraft in the Air Force, Navy, and Marine Corps, while providing the 
most supportable, technologically advanced, lethal, and survivable 
aircraft to date. The F-35 is DOD's most expensive weapon system, with 
estimated sustainment costs of about $1 trillion. With the military 
services planning for the ability to deploy and maintain the F-35 
within 4 years, DOD is working to develop a sustainment strategy that 
will be both affordable and executable for the program's life cycle. 

GAO was mandated to review DOD's F-35 sustainment planning efforts. 
This report addresses the extent to which DOD has (1) developed an F-
35 sustainment strategy and addressed potential risks related to 
affordability and operational readiness and (2) developed a reliable 
O&S cost estimate for the program's life cycle. GAO analyzed 
documented plans and cost estimates and interviewed DOD and contractor 
officials. 

What GAO Found: 

The Department of Defense (DOD) currently has or is developing several 
plans and analyses that will make up its overall F-35 sustainment 
strategy, which is expected to be complete in fiscal year 2019. The 
annual F-35 operating and support (O&S) costs are estimated to be 
considerably higher than the combined annual costs of several legacy 
aircraft (see figure). DOD has begun some cost-savings efforts and 
established sustainment affordability targets for the F-35 program, 
but DOD did not use the military services' budgets to set these 
targets. Therefore, these targets may not be representative of what 
the services can afford and do not provide a clear benchmark for DOD's 
cost-savings efforts. In addition, DOD has not fully addressed several 
issues that have an effect on affordability and operational readiness, 
including aircraft reliability and technical-data rights, which could 
affect the development of the sustainment strategy. 

Figure: Comparison of the Annual Estimated F-35 Operating and Support 
(O&S) Cost at Steady State to Actual Legacy Aircraft O&S Costs in 
Fiscal Year 2010: 

[Refer to PDF for image: illustrated horizontal bar graph with 5 
photographs] 

Aircraft: F-35 A/B/C; 
Estimated O&S Cost: $19.9 billion[A]. 

Aircraft: F-15 C/D; F16 C/D; AV-8B; F-18 A-D; 
Estimated O&S Cost: $11.1 billion[B]. 

Source: GAO presentation of Department of Defense data; Cost 
Assessment and Program Evaluation office. GAO-14-778. 

Notes: For the purposes of this report, GAO defines steady-state 
operations as the period from 2036 to 2040, when, according to the 
services' plans, the number of F-35 aircraft and flying hours reaches 
its highest point and plateaus. 

[A] The F-35 cost presented is Cost Assessment and Program 
Evaluation's (CAPE) estimated total annual operating and support (O&S) 
cost for 2040 in base year 2012 dollars. 

[B] Legacy aircraft cost is based on a CAPE analysis of 2010 cost 
data, representing a high point for aircraft O&S budgets due to 
contingency operations at that time. 

[End of figure] 

It is unclear whether DOD's O&S cost estimates for the F-35 program 
reflect the most likely costs that the F-35 program will incur. DOD 
has two primary F-35 O&S estimates that each total around $1 trillion 
over a 56-year life cycle. These cost estimates are comprehensive in 
that they include all DOD-required program elements and are organized 
according to a standard O&S cost-estimating structure; however, 
weaknesses exist with respect to a few of the assumptions, and the 
estimates did not include all analyses necessary to make them fully 
reliable. For example, the estimates did not use reasonable fuel burn 
rate assumptions that reflect the likely future F-35 fuel usage. 
Further, one of the estimates did not use reasonable assumptions about 
part replacement rates and depot maintenance. Finally, while DOD took 
some steps to mitigate the uncertainties inherent in cost estimates, 
DOD officials did not conduct key analyses to determine the level of 
risk associated with the estimates. 

What GAO Recommends: 

GAO recommends that DOD develop better informed affordability 
constraints; address three risks that could affect sustainment, 
affordability, and operational readiness; and take steps to improve 
the reliability of its cost estimates. DOD concurred with all but one 
recommendation and partially concurred with the recommendation to 
conduct uncertainty analysis on one of its cost estimates, stating it 
already conducts a form of uncertainty analysis. GAO continues to 
believe that the recommended analysis would provide a more 
comprehensive sense of the uncertainty in the estimates. 

View [hyperlink, http://www.gao.gov/products/GAO-14-778]. For more 
information, contact Cary Russell at (202) 512-5431 or 
russellc@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

DOD Is Developing an F-35 Sustainment Strategy, but Key Risks to 
Affordability and Operational Readiness Remain: 

Improved Assumptions and Additional Analyses Could Increase 
Reliability of F-35 O&S Costs: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: GAO Scoring of 2013 F-35 Operating and Support (O&S) 
Estimates: 

Appendix III: Comments from the Department of Defense: 

Appendix IV: GAO Contact and Staff Acknowledgments: 

Related GAO Products: 

Tables: 

Table 1: JPO and CAPE 2013 Total Estimated O&S Costs: 

Table 2: Sample of F-35 Parts with Higher Removal Rates Than Assumed 
in Joint Program Office (JPO) 2013 Cost Estimate: 

Table 3: Summary Assessment of Joint Program Office (JPO) and Cost 
Assessment and Program Evaluation (CAPE) Cost Estimates: 

Figures: 

Figure 1: F-35A Air Force Conventional Takeoff and Landing Variant 
(CTOL): 

Figure 2: Timeline of Major Events in the F-35 Program, 2001-2040: 

Figure 3: Comparison of Annual Estimated F-35 Operating and Support 
(O&S) Cost at Steady State to Actual Legacy Aircraft O&S Costs in 
Fiscal Year 2010: 

Figure 4: Autonomic Logistics Information System (ALIS) Issue-
Resolution Process: 

Abbreviations: 

AFRS: Anomaly and Failure Resolution System: 

ALIS: Autonomic Logistics Information System: 

AT&L: Acquisitions, Technology, and Logistics: 

CAPE: Cost Assessment and Program Evaluation: 

CTOL: conventional takeoff and landing: 

CV: carrier-suitable variant: 

DFARS: Defense Federal Acquisition Regulation Supplement: 

DOD: Department of Defense: 

DOT&E: Director of Operational Test and Evaluation: 

EOTS: Electrical Optical Targeting System: 

FAR: Federal Acquisition Regulation: 

IOC: initial operational capability: 

IP: Intellectual Property: 

JPO: Joint Program Office: 

JRMET: Joint Reliability and Maintainability Evaluation Team: 

KPP: Key Performance Parameter: 

MFHBR: Mean Flight Hours Between Removals: 

O&S: operating and support: 

OSD: Office of the Secretary of Defense: 

OSD-LMR: Office of the Secretary of Defense Logistics and Materiel 
Readiness: 

PBL: Performance Based Logistics: 

R+M: reliability and maintainability: 

SAR: Selected Acquisition Report: 

SOS: System of Systems: 

STOVL: short takeoff and vertical landing: 

[End of section] 

United States Government Accountability Office: 
GAO:
441 G St. N.W. 
Washington, DC 20548: 

September 23, 2014: 

The Honorable Howard P. "Buck" McKeon: 
Chairman: 
The Honorable Adam Smith: 
Ranking Member: 
Committee on Armed Services: 
House of Representatives: 

The Department of Defense (DOD) has a continuing responsibility to 
procure weapon systems that successfully execute its national security 
mission. One such weapon system is the F-35 Lightning II--also known 
as the Joint Strike Fighter--which is intended to replace a variety of 
existing aircraft in the Air Force, Navy, and Marine Corps, while 
providing the most technologically advanced, lethal, supportable, and 
survivable aircraft to date. With DOD estimating its sustainment costs 
to be approximately $1 trillion over its life cycle, the F-35 is not 
only the most ambitious weapon system in DOD's history, but also the 
most costly. Recognizing that at least 70 percent of a weapon system's 
life-cycle costs stem from operating and supporting the system, in 
recent years DOD has made changes to its acquisition process to put an 
earlier emphasis on sustainment. Specifically, more attention has been 
placed on sustainment planning, assuring competition among suppliers, 
and identifying sustainment-related resource constraints at the front 
end of the acquisition process. However, the F-35 acquisition program, 
which began in October 2001, predates these changes, and DOD has only 
recently begun to focus on how it will sustain the nearly 2,500 
aircraft it plans to procure. Currently, the F-35 is 13 years into its 
acquisition strategy--a strategy that involves substantial overlap 
among development, testing, and production activities. With all three 
military services planning to deploy and maintain the F-35 within the 
next 4 years, and the Marine Corps planning to do so in less than 1 
year, DOD is working to develop a sustainment strategy for the F-35 
that will be both affordable and executable for the life cycle of the 
program. 

We have reported on DOD's acquisition of the F-35 for many years (see 
the Related GAO Products section at the end of this report). Our body 
of work has identified significant cost, schedule, and performance 
problems and found that those problems, in large part, can be traced 
to (1) decisions made at key junctures without adequate product 
knowledge; and (2) a highly concurrent acquisition strategy with 
significant overlap among development, testing, and manufacturing 
activities. In March 2012, DOD completed an extensive restructuring of 
the F-35 program by increasing the program's cost estimates, extending 
its testing and delivery schedules, and deferring near-term aircraft 
procurement quantities into the future. We concluded in June 2012 and 
in March 2013 that the restructuring actions should lead to more 
achievable and predictable outcomes, albeit at higher costs and with 
longer time frames than originally planned for testing and delivering 
capabilities to the warfighter.[Footnote 1] In March 2014, we found 
that problems encountered by DOD in completing software testing may 
hinder delivery of expected warfighting capabilities to most of the 
services.[Footnote 2] We have made numerous recommendations aimed at 
addressing these issues, and DOD has taken some actions to address 
them to varying degrees. 

In light of DOD cost estimates for sustainment of the aircraft 
amounting to approximately $1 trillion, and increased concerns over 
affordability in a fiscally constrained environment, the House Armed 
Services Committee report accompanying a National Defense 
Authorization bill for Fiscal Year 2014 mandated GAO to review DOD's 
sustainment planning efforts for the F-35 program.[Footnote 3] This 
report addresses the extent to which DOD has (1) developed a 
sustainment strategy for the F-35 program and addressed potential 
risks to affordability and operational readiness and (2) developed a 
reliable operating and support (O&S) cost estimate for the life cycle 
of the program. 

For each of our objectives, we reviewed relevant policy and procedures 
and collected information by interviewing officials from the Office of 
the Under Secretary of Defense (Acquisitions, Technology and 
Logistics), the Office of the Assistant Secretary of Defense 
(Logistics and Materiel Readiness), the Office of the Director for 
Cost Assessment and Program Evaluation (CAPE), the Office of the 
Director for Operational Test and Evaluation (DOT&E), the Office of 
the Director for Developmental Test and Evaluation, the Air Force, the 
Navy, the Marine Corps, and the F-35 Joint Program Office (JPO). We 
also gathered F-35 reliability and maintainability (R+M) data[Footnote 
4] from 2013 through 2014 that had been verified through DOD's Joint 
Reliability and Maintainability Evaluation Team (JRMET). To determine 
the reliability of these data, we collected information on how the 
data were collected, managed, and used through a survey and interviews 
and with relevant DOD officials. In addition to the data-reliability 
survey, we also reviewed the corresponding database user manual and 
related documentation to determine the limitations of the data. By 
assessing this information against GAO data-quality standards, we 
determined that the data presented in our findings were sufficiently 
reliable for presenting information about the aircraft's reliability 
in this report. 

To interview officials about and observe F-35 operations, maintenance, 
training, and developmental and operational testing, we conducted 
visits to Eglin Air Force Base, Marine Corps Air Station Yuma, and 
Naval Air Station Patuxent River--locations where these activities 
were occurring. In addition, we met with officials about F-35 
sustainment planning and costs at Naval Air Systems Command and we 
reviewed documentation and interviewed officials at Lockheed Martin--
the prime contractor--in Fort Worth, Texas, about the program's status 
with respect to development, operations, testing, and costs. To 
determine the extent to which DOD has developed a sustainment strategy 
and addressed key risks related to affordability and operational 
readiness, we reviewed DOD's guidance and policy for defense 
acquisitions and life-cycle sustainment planning and compared them to 
the program's Life Cycle Sustainment Plan and other planning 
documents, including business-case analyses. Furthermore, we reviewed 
DOD's risk-management and policy guidance for specific sustainment 
elements and compared it to current program risks and mitigation 
practices. 

To determine the extent to which DOD has developed a reliable O&S cost 
estimate for the F-35 program, we evaluated DOD's two primary cost 
estimates, the 2013 JPO and the 2013 CAPE office's O&S cost estimates, 
using GAO's Cost Estimating and Assessment Guide.[Footnote 5] The 
methodology outlined in the Cost Estimating and Assessment Guide is a 
compilation of best practices that federal cost-estimating 
organizations and industry use to develop and maintain reliable cost 
estimates throughout the life of an acquisition program. The JPO's 
estimate is used to drive budget planning and is required by DOD 
acquisition policy, and CAPE's estimate is used to assess 
affordability and is required at major acquisition milestones or by 
special request from DOD leadership. Generally, either the JPO's or 
CAPE's estimate is used as DOD's estimate of record and published in 
the Selected Acquisition Report (SAR)--a report that DOD is required 
to submit annually to Congress on each of its major defense 
acquisition programs. For the F-35, DOD decided to use the CAPE O&S 
estimate in the 2012 and 2013 SARs. We evaluated both the JPO and the 
CAPE O&S estimates because both estimates are used in DOD decision 
making. According to the Cost Estimating and Assessment Guide, there 
are four general characteristics of sound cost estimating: being well-
documented, comprehensive, accurate, and credible. For the purposes of 
this review, we conducted a limited assessment and evaluated two of 
these characteristics: comprehensive and credible. We chose to 
evaluate the estimates' comprehensiveness because if a cost estimate 
is not comprehensive then it cannot fully meet the characteristics of 
being well-documented or accurate. For example, if the cost estimate 
is missing some cost elements, then the documentation will be 
incomplete and the estimate will be inaccurate. We also included an 
assessment of the credible characteristic due to the substantial 
estimated cost of the program and to determine whether risks to costs 
were quantified. To determine whether the comprehensive and 
credibility characteristics were met, we reviewed CAPE and JPO cost-
estimating documentation, including data sources, assumptions, and 
calculations, and we interviewed cost-estimating officials from these 
offices. 

We conducted this performance audit from August 2013 to September 2014 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. Appendix I 
provides a more detailed description of our scope and methodology. 

Background: 

The F-35 Lightning II program is a joint, multinational acquisition 
intended to develop and field an affordable family of next-generation 
strike fighter aircraft for the United States Air Force, Navy, Marine 
Corps, and eight international partners.[Footnote 6] There will be 
three variants of the F-35. (1) The conventional takeoff and landing 
(CTOL) variant, designated the F-35A, will be a multirole, stealthy 
strike aircraft replacement for the Air Force's F-16 Falcon and the A-
10 Thunderbolt II aircraft, and will complement the F-22A Raptor (see 
figure 1). (2) The short takeoff and vertical landing (STOVL) variant, 
the F-35B, will be a multirole, stealthy strike fighter that will 
replace the Marine Corps' F/A-18C/D Hornet and AV-8B Harrier aircraft. 
(3) The carrier-suitable variant (CV), the F-35C, will provide the 
Navy a multirole, stealthy strike aircraft to complement the F/A-18 
E/F Super Hornet. The Marine Corps will also field a limited number of 
F-35C CVs. Lockheed Martin is the aircraft contractor and Pratt & 
Whitney is the engine contractor. 

Figure 1: F-35A Air Force Conventional Takeoff and Landing Variant 
(CTOL): 

[Refer to PDF for image: photograph] 

Source: Department of Defense. GAO-14-778. 

[End of figure] 

DOD began the F-35 program in October 2001 with a highly concurrent 
acquisition strategy, resulting in substantial overlap between 
development, testing, and production. The program was restructured in 
2003 and again in 2007 due to performance problems, cost growth, and 
schedule slips. After the program experienced a critical cost breach 
in 2010, the program was restructured once more, and in 2012 some near-
term procurement quantities were deferred to a future date and the 
program's cost estimates were increased.[Footnote 7] In 2013, DOD 
issued a report to Congress[Footnote 8] stating that the Marine Corps, 
Air Force, and Navy were planning to field initial operational 
capabilities (IOC) in 2015, 2016, and 2018, respectively.[Footnote 9] 
See figure 2 for a timeline of major events leading from the beginning 
of development. 

Figure 2: Timeline of Major Events in the F-35 Program, 2001-2040: 

[Refer to PDF for image: timeline] 

2001: Milestone B is approved and Lockheed Martin is selected as the 
prime contractor[A]. The program begins development. 

2006-2019: Low-rate production. 

2012: Program establishes new acquisition baseline and recertifies 
Milestone B after a cost-growth breach. 

2015: Program to begin operational testing, and Marine Corps to 
declare Initial Operational Capability (IOC)[B]. 

2016: Air Force to declare IOC. 

2018: Navy to declare IOC. 

2019: Planned milestone decision to enter full-rate production. 

2020-2040: Operations and support phase. 

Source: GAO analysis of Department of Defense information. GAO-14-778. 

[A] As part of DOD's acquisition process, Milestone B initiates the 
Engineering and Manufacturing Development Phase. 

[B] IOC is generally obtained when organizations or units scheduled to 
receive a system have received it and have the ability to employ and 
maintain that system. 

[End of figure] 

The program is currently in low-rate initial production, and the 
contractor is assembling the next lot of 36 aircraft with a scheduled 
delivery by the end of 2015. As of June 2014, 78 aircraft have been 
fielded, are flying, and are being maintained at Eglin Air Force Base, 
Marine Corps Air Station Yuma, Edwards Air Force Base, Nellis Air 
Force Base, and Luke Air Force Base. The program had achieved about 
18,250 cumulative flight hours for the fleet of aircraft; a total of 
200,000 flight hours are required in order for the fleet to reach R+M 
maturity, as outlined in the program's Operational Requirements 
Document. In March 2014, we found that the program had progressed in 
its developmental flight testing, but it continued to lag in testing 
of critical mission-systems software, delaying the delivery of 
expected warfighting capabilities.[Footnote 10] 

By full-rate production, planned for fiscal year 2019, DOD would 
generally be required to establish adequate sustainment and support 
systems for the F-35.[Footnote 11] Per DOD guidance for all weapon 
systems acquisitions, these sustainment and support systems should be 
defined in a support concept that is incorporated into a sustainment 
strategy. For the F-35, this concept should comprise the necessary 
plans to conduct operations, maintenance, and sustainment throughout 
the system's life cycle, with the F-35 Life Cycle Sustainment Plan 
serving as the principle document governing F-35 sustainment. 
According to the F-35 Operational Requirements Document, this concept 
must provide warfighting and peacetime capability with the lowest cost 
of ownership, and all variants must be able to deploy rapidly, sustain 
high mission reliability, and sustain a high sortie-generation rate. 
[Footnote 12] 

In developing this strategy, the program has made affordability its 
top priority, and the F-35 Program Executive Officer[Footnote 13] 
reiterated this priority in his April 2014 testimony.[Footnote 14] In 
this testimony, the Program Executive Officer also identified areas in 
which the program is focusing on driving down sustainment costs and 
identifying improvements to meet long-term sustainment needs in order 
to "produce a mutually beneficial sustainment enterprise that 
operates, manages, and supports the global system with relevant 
metrics and incentives, while meeting warfighter-defined readiness and 
cost objectives." However, the department has stated that it has a 
long history of starting programs that proved to be unaffordable, and 
the result of this practice has been costly program cancellations and 
dramatic reductions in inventory objectives.[Footnote 15] Recently 
updated DOD acquisition policy has placed more emphasis on including 
sustainment considerations as early as possible in the acquisition 
process.[Footnote 16] According to the policy, a successful program 
meets the sustainment performance requirements, remains affordable, 
and continues to seek cost reductions throughout the Operations and 
Support Phase that begins after the full-rate production decision. 

In a 2012 memorandum, the Under Secretary of Defense for Acquisitions, 
Technology and Logistics (AT&L) acknowledged that certain F-35 program 
areas including the Autonomic Logistics Information System (ALIS), 
reliability growth, and sustainment costs require additional 
improvement. 

* ALIS. ALIS is the primary sustainment tool for the F-35 and is 
intended to predict maintenance and supply issues, automate logistics 
support processes, and provide decision aids to help reduce life-cycle 
sustainment costs and improve force readiness. ALIS is one of three 
major components that make up the F-35 air system, along with the 
aircraft and the engine, and comprises both hardware and software. The 
F-35 program is delivering ALIS capabilities incrementally, which 
include: operations, maintenance, supply chain, customer-support 
services, training, tech data, system security and external interfaces. 

* Reliability and Maintainability (R+M). R+M measures aircraft 
performance to determine how often the aircraft experiences failures 
and how much time it takes to repair those failures. R+M is monitored 
through a series of metrics that measure the intended performance of 
the aircraft in meeting its requirements by maturity at a cumulative 
200,000 flight hours with at least 50,000 flight hours per variant. 
R+M drives sortie-generation rates and a reduced logistics footprint 
for the F-35, as well as informing program O&S costs, which are tied 
to the performance of the system at maturity. 

* O&S Costs. O&S consists of sustainment costs incurred from the 
initial system deployment through the end of system operations and 
includes all costs of operating, maintaining, and supporting a fielded 
system. The F-35 JPO develops an annual estimate for the O&S costs of 
maintaining and supporting the F-35 for 56 years.[Footnote 17] In 
2011, the estimate was $1.03 trillion; in 2012, it was $857 billion; 
and in 2013, it was $916 billion in then-year dollars. Additionally, 
CAPE conducts independent cost estimates on certain programs at 
different points in the acquisition cycle. In recent years, CAPE's F-
35 O&S cost estimate has been reported in DOD's annual SAR as the 
official O&S cost estimate for the program. In 2013, that estimate was 
about $1.02 trillion in then-year dollars. 

As stated earlier, recent changes to DOD's acquisition guidance and 
policy have also focused on making sustainment decisions earlier in 
the acquisition process, by identifying the attributes of an effective 
performance-based logistics arrangement and requiring strategies that 
identify and manage technical data rights. Both of these areas will 
play an important role in the future of the F-35 program. 

* Performance Based Logistics (PBL).[Footnote 18] An effective 
performance-based logistics strategy ties objective metrics for 
delivered logistical system performance to incentives that will 
motivate the support provider (i.e., the contractor). In recent years, 
F-35 program managers have reaffirmed their commitment to employing a 
robust performance-based logistics system for the F-35 to help 
determine options for pursuing the most value-driven performance-based 
agreements with the support provider. 

* Technical Data Rights. Technical data for weapon systems include the 
details necessary to ensure the adequacy of performance, as well as 
instructions for operation, maintenance, and other actions needed to 
support weapon systems.[Footnote 19] Technical data are an important 
part of a weapon system program, such as the F-35. Identifying 
technical data needs and costs early in the acquisition process can 
help program managers maximize the potential for future competition 
and develop the system's overall sustainment strategy. 

DOD Is Developing an F-35 Sustainment Strategy, but Key Risks to 
Affordability and Operational Readiness Remain: 

DOD's development of an F-35 sustainment strategy is an ongoing 
process that includes a number of plans, analyses, and decisions that 
must be finalized by the start of full-rate production--estimated to 
occur in fiscal year 2019. To date, the program's sustainment strategy 
continues to evolve but may not be affordable, and DOD has not fully 
addressed several key risks to long-term affordability and operational 
readiness, that, if not mitigated, could affect the development of the 
strategy. 

DOD Is Currently Developing an F-35 Sustainment Strategy, but It May 
Not Be Affordable: 

DOD has a number of sustainment planning documents that will make up 
the F-35 program's overall sustainment strategy, including the Life 
Cycle Sustainment Plan, which, according to DOD policy, will provide 
the basis for sustainment activities. DOD also has past and ongoing 
studies and analyses that have the potential to affect and inform the 
overall sustainment strategy, including two Business Case Analyses and 
several Level of Repair Analyses. These planning documents and 
analyses will provide the basis for the overall sustainment strategy 
that will dictate sustainment operations when the program enters full-
rate production. Furthermore, a recently launched initiative led by 
the F-35 JPO is working to establish a roadmap to the end of the 
design phase in 2018, which is intended to eventually result in a 
Future Support Construct--another element of sustainment planning that 
will inform decisions for long-term operations and support for the 
aircraft. The JPO plans to finalize the Future Support Construct in 
March 2015. 

However, the current sustainment strategy that DOD is developing may 
not be affordable. The program has continued to experience cost 
overruns, and the recent SAR estimated the O&S cost to sustain the 
system for 56 years to be approximately $1 trillion. Additionally, 
according to DOD officials, including officials within the Office of 
the Secretary of Defense (OSD), the current sustainment strategy is 
not affordable. According to CAPE analysis, the combined O&S costs of 
several legacy aircraft--the F-15C/D, F-16C/D, AV-8B, and F-18A-D 
fleets--in 2010 exceeded $11 billion. Comparatively, based on CAPE's 
2013 O&S cost estimate, the annual cost to sustain the F-35 will be 
about $19.9 billion (in base year 2012 dollars) in 2040--the end of 
its steady-state years.[Footnote 20] This $8.8 billion difference 
represents an increase of more than 79 percent in annual O&S costs for 
the F-35 as compared with several legacy aircraft (see figure 3). 
Moreover, the Program Executive Officer has continued to express 
concerns over the affordability of the program's sustainment approach, 
stating that "F-35 sustainment costs remain a concern" and that 
affordability continues to be a top priority for the program.[Footnote 
21] 

Figure 3: Comparison of Annual Estimated F-35 Operating and Support 
(O&S) Cost at Steady State to Actual Legacy Aircraft O&S Costs in 
Fiscal Year 2010: 

[Refer to PDF for image: illustrated horizontal bar graph with 5 
photographs] 

Aircraft: F-35 A/B/C; 
Estimated O&S Cost: $19.9 billion[A]. 

Aircraft: F-15 C/D; F16 C/D; AV-8B; F-18 A-D; 
Estimated O&S Cost: $11.1 billion[B]. 

Source: GAO presentation of Department of Defense data; Cost 
Assessment and Program Evaluation office. GAO-14-778. 

Notes: For the purposes of this report, we define steady-state 
operations as the period from 2036 to 2040, when, according to the 
services' plans, the number of F-35 aircraft and flying hours reaches 
its highest point and plateaus. 

[A] The F-35 cost presented is CAPE's estimated total annual O&S cost 
for 2040 in base year 2012 dollars. 

[B] Legacy aircraft cost is based on a CAPE analysis of 2010 cost 
data, representing a high point for aircraft O&S budgets due to 
contingency operations at that time. 

[End of figure] 

In 2012, DOD established affordability targets for the program, 
stating that the cost per flying hour for the Air Force, Marine Corps, 
and Navy F-35 variants could not exceed $35,200, $38,400, and $36,300 
respectively. However, DOD officials, including some at OSD, stated 
that they believe that based on this threshold, the program is not 
affordable. To address increasing costs, DOD has taken steps focused 
on cost reduction including, but not limited to, establishing a Cost 
War Room in 2013 and a Readiness Cell in 2014. The Cost War Room is a 
collaborative group comprising DOD, Lockheed Martin, and Pratt & 
Whitney personnel, established with the purpose of reducing costs and 
providing support for transitioning to performance-based logistics. 
The mission of the Readiness Cell is to identify opportunities to 
enable F-35 availability (a sustainment metric) at more than 60 
percent by 2015 across all three variants, thereby improving 
sustainment metrics that directly affect O&S costs. However, it is 
unclear whether these cost-saving measures can result in an affordable 
approach to the long-term operations and sustainment of the F-35 
because the established affordability targets that must be achieved 
may not be representative of what the services can actually afford. 

DOD's current acquisition policy states that affordability constraints 
for procurement and sustainment should be determined early in program 
planning processes and should be used to ensure that capability 
requirements are prioritized and that cost trade-offs occur as early 
as possible and throughout the program's life cycle.[Footnote 22] 
These constraints should not be based on cost estimates but rather on 
whether a component can afford the estimated costs of a system, and 
should be used to indicate whether actions must be taken to prevent 
exceeding the constraints.[Footnote 23] When DOD established 
affordability targets for the F-35 program in 2012, the methodology 
for determining that threshold was not informed by actual resource 
constraints within service budgets at the time. Specifically, DOD 
officials stated that the targets were determined by arbitrarily 
lowering CAPE's estimated F-35 Cost per Flight Hour by 10 percent. 
[Footnote 24] However, without informed affordability constraints, 
based on military service budgets, DOD cannot be sure whether the 
costs savings it achieves through current efforts will lead to an 
affordable sustainment strategy, and DOD may miss additional areas for 
savings. 

DOD Has Not Taken Steps to Fully Address Risks to Affordability and 
Operational Readiness That Could Affect the F-35 Sustainment Strategy: 

DOD faces the following three key risks to the long-term affordability 
and operational readiness of the F-35 program: (1) the performance of 
ALIS; (2) the R+M of the whole air system; and (3) the management of 
technical data rights. However, the department has not taken steps to 
fully address these risks, which could affect the development of the 
sustainment strategy. 

ALIS: 

ALIS is a system of systems that serves as the primary logistics tool 
to support operations, mission planning, and sustainment of the F-35. 
One official described ALIS as the brains of the aircraft, helping 
maintainers manage such things as aircraft health and diagnostics, 
supply-chain management, and necessary maintenance events. However, 
ALIS has experienced recurring problems, including user issues and 
schedule delays. Furthermore, DOD has stated that ALIS is one of the 
biggest impediments to the amount of sorties, or flights, that can be 
generated in support of testing and operations. The sortie-generation 
rate is a key performance parameter[Footnote 25] for the aircraft. 

For example, the integration of ALIS capabilities--which are fielded 
in increments--has been repeatedly delayed. The most recent timeline 
for ALIS estimates that the final version of ALIS will be released for 
testing in 2017--7 years after the originally planned release date. 
Additionally, ALIS's diagnostic system has not reached full 
functionality. Specifically, it has failed to meet basic requirements, 
including having the ability to identify faults and failures in the 
aircraft. That issue has resulted in time-consuming workarounds that 
could further affect sortie generation. Currently, DOD tracks issues 
with ALIS through an internal reporting mechanism that allows users in 
the field to identify problems with the system for potential fixes. 
For example, according to DOD officials, ALIS's Anomaly and Failure 
Resolution System (AFRS) is intended to identify the necessary 
maintenance to repair the aircraft, but it is not yet mature. 
Maintainers at an F-35 operational training site told us that, as a 
result, they have had to use multiple approaches to identify the best 
maintenance solution. Once identified, the maintainer can submit this 
solution as an update for AFRS. However, this update must first be 
reviewed by field support on site and then sent--in the form of an 
action request--to the contractor for approval before it is integrated 
into AFRS. Maintenance officials told us that they have submitted 
several thousand action requests to date and have thereby created a 
backlog, leaving maintainers to wait multiple days for an approval. 

Problems with ALIS are reported by users on an individual basis, and 
thus do not give DOD a complete view of ALIS's system performance. 
Once an issue is reported, DOD decides whether to (1) defer the issue, 
forcing a workaround for users, or (2) pay for the contractor to 
develop a "fix." For an overview of this process, see figure 4. 

Figure 4: Autonomic Logistics Information System (ALIS) Issue-
Resolution Process: 

[Refer to PDF for image: illustration] 

1. Diagnostic health codes are downloaded from the plane to Autonomic 
Logistics Information System (ALIS). 

2. Maintainer retrieves codes from ALIS and reports any issues to the 
contractor. 

3. Contractor receives issue reports from operations. 

4. Solution identified; or: Solution deferred[A]. 

5. Solution sent back to maintainer and ALIS is updated. 

Sample of ALIS capabilities: 
* Assess usage; 
* Perform aircraft maintenance; 
* Perform air vehicle post- and preflight servicing; 
* Plan mission; 
* Process diagnostics data; 
* Process sustainment data; 
* Provide logistics data. 

Source: GAO analysis of Department of Defense information; Lockheed 
Martin. GAO-14-778. 

[A] If a solution is not immediately identified by the contractor, a 
temporary solution is sent back to the maintainer, or a workaround is 
used, or both, until a permanent solution is identified. 

[End of figure] 

To fully understand system performance, DOD's Systems Engineering 
Guide for Systems of Systems (SOS) states that it is important to have 
a set of metrics that allow for an assessment of the SOS performance 
traced to user requirements because the SOS will likely evolve based 
on incremental changes in individual systems--similar to ALIS's 
incremental fielding.[Footnote 26] These metrics should measure the 
intended integrated behavior and performance of the SOS in actual 
operations versus the progress of the development of the SOS, allowing 
an assessment of SOS capabilities based on user requirements, such as 
functionality between interdependent subsystems. Additionally, 
according to the F-35 Operational Requirements Document, the F-35 must 
have a fully functional and effective logistics system (e.g., ALIS) 
aimed at ensuring operational readiness and availability and reducing 
overall life-cycle costs by the time the aircraft reaches maturity at 
200,000 flight hours for all three variants with at least 50,000 per 
variant. 

Although DOD has a performance-measurement process for the overall 
aircraft, it does not have a similar process, with metrics and 
targets, to determine and address the most significant performance 
issues with ALIS. DOD receives data on ALIS's availability from 
Lockheed Martin, and it tracks the progress of development of ALIS, 
but it does not track any other data to assess the performance or 
maturity of the system based on user requirements. Without a 
performance-measurement process that ties ALIS performance to user 
requirements, it is unclear how the program can effectively and 
efficiently address future ALIS performance issues. Having performance 
metrics and targets for ALIS that assess the intended integrated 
behavior and performance in actual operations could allow the program 
to determine future root causes of ALIS issues--a major tenet of risk 
mitigation, according to DOD's risk management guide--and address 
them.[Footnote 27] Furthermore, because the current process to address 
ALIS issues involves paying for ALIS fixes on an incremental basis, 
the program may face additional costs that are more difficult to 
predict. Moreover, without a performance-measurement process for ALIS, 
DOD cannot be sure that it is considering the most important 
performance-related information when developing future performance-
based logistics agreements for the program--agreements that will 
affect long-term sustainment costs. Finally, without a fully 
functional logistics system (i.e., ALIS), DOD may not be able to fully 
support the necessary performance parameters, such as sortie 
generation, to ensure operational readiness and availability. 

Reliability and Maintainability: 

DOD officials have continued to express concerns with the reliability 
and maintainability (R+M) of the aircraft. In his April 2014 
testimony, the Program Executive Officer stated that R+M remains an 
area for needed improvement. In that same month, the Director, 
Operational Test and Evaluation (DOT&E), stated that the growth rates 
and improvement programs that would be necessary to meet R+M 
requirements when the aircraft reaches maturity (200,000 flight hours) 
are what they termed "ambitious." Additionally, DOT&E stated that 
future efforts to improve R+M may be hampered by higher failure rates 
as the aircraft begins to fly with more aggressive maneuvering and 
more extensive use of mission systems. 

To measure the R+M of the aircraft, testers collect data on (1) the 
number of flight hours achieved before a failure occurs (reliability) 
and (2) the amount of time it takes to repair those failures 
(maintainability). These two primary measures are supported by various 
metrics and tracked along planned growth curves to measure progress in 
meeting requirements.[Footnote 28] Based on recent growth curves and 
reports for some of DOD's R+M metrics, some metrics have progressed, 
some continued to lag, and some have worsened. For example: 

* Mean Flight Hours between Failures (Design Controllable) is the 
average amount of flight hours achieved before a design-controllable 
failure[Footnote 29] occurs. As of March 2014, this metric was 
progressing in that the number of flight hours before a failure occurs 
was increasing for all three variants. For example, the average flight 
hours between failures for the F-35A--the variant with the most flight 
hours to date--was 5.2 in March 2014, surpassing the expectation at 
its current flight hours by about 1.2 and growing toward its 
requirement at maturity of 6.0. Moreover, this was an increase of 
about 1.8 average flight hours between failures since September 2013, 
as reported by GAO.[Footnote 30] 

* Mean Flight Hours between Critical Failures is the average amount of 
flight hours achieved before a failure occurs that results in the loss 
of a capability to perform a mission-essential function.[Footnote 31] 
As of March 2014, this metric was lagging well below its requirements 
at maturity, meeting an average of 42 percent of those requirements 
across all three variants. 

* Mean Time to Repair is the average time it takes a maintainer to 
repair a failed component or device. Currently, this metric is not 
improving in that as flight hours increase, it is taking maintainers 
longer to repair failed components for the F-35A and F-35C, and the 
amount of time it takes to repair failed components for the F-35B 
remains unchanged. Specifically, GAO reviewed R+M growth curves 
provided by DOD showing the historical growth of this metric from 2009 
for the F-35B and 2010 for the F-35A and F-35C to March 2014, and we 
observed that the metric is trending in the opposite direction of its 
predicted path for the F-35A and F-35C, and the metric is remaining 
steady, without improvement, for the F-35B. 

A tenet of F-35 sustainment is to continually improve R+M of the 
weapon system to drive down O&S costs. In an effort to improve R+M, 
DOD began a Reliability Improvement Plan in 2014 that focused on 
addressing the top 20 issues with the aircraft that affect R+M. While 
the current Reliability Improvement Plan is in the process of 
addressing some R+M issues, the plan focuses primarily on the 
aircraft's hardware and does not specifically address software and its 
potential effects on overall R+M. The F-35 is the most software-
intensive fighter aircraft DOD has procured to date. According to DOD 
officials, software reliability is an issue that needs attention; 
however, officials have also stated that the current process for 
tracking R+M is almost exclusively dedicated to hardware. 

To identify some software issues as they arise, users in the field use 
an internal system to submit requests to the contractor, but these 
requests are submitted on an individual basis and may not always be 
addressed immediately as it takes time to determine whether the issue 
is related to hardware or software. For example, officials told us 
that the Electrical Optical Targeting System, which is used to track a 
target, continues to fail. In this instance, testers reported the 
problem, and officials attempted to improve the capability with 
hardware changes. However, not all issues with the Electrical Optical 
Targeting System were fixed with the hardware changes, and officials 
have decided to also try to address the issue with software changes, 
causing users to identify workarounds in the meantime. As another 
example, officials discussed instances in which the diagnostics system 
signals to a maintainer that the landing gear failed, but it was 
actually a sensor near the landing gear that failed. Because software 
for isolating these types of failures is not yet mature, operators and 
maintainers on the ground may continue to check the landing gear 
without discovering the sensor issue and reporting it. Finally, 
officials stated that with the release of the next increment of 
software in 2015, a number of new issues may arise, and these issues 
may be related to software because the new software includes new 
processors that can affect mission systems on the aircraft. 

According to DOD reliability guidance, basic reliability activities 
should include software reliability assessments.[Footnote 32] 
Additionally, Naval Air Systems Command guidance advises that weapon 
systems programs should establish software R+M metrics as a key 
activity in terms of achieving supportability goals and developing a 
high level of confidence that the product will work reliably and can 
be maintained easily.[Footnote 33] Furthermore, DOD's Risk Management 
Guide states that effective risk management depends, in part, on 
continuous monitoring and reassessment to identify and address root 
causes of program risks.[Footnote 34] Currently there are no processes 
or metrics by which the program can monitor and assess software R+M 
and the full extent to which software issues may be contributing to 
the overall R+M issues of the aircraft. Without an assessment process, 
including metrics, for software R+M as a means of mitigating ongoing 
risks to R+M, program officials cannot have a full understanding of 
the root causes of R+M issues on the aircraft, or whether software 
failures contribute to the overall R+M outside of hardware failures. 
This can potentially lead to unforeseen costs in the future as 
software is continually upgraded and refined. Since O&S costs are 
based on meeting the required reliability at maturity (200,000 flight 
hours), there are increasing risks to O&S cost and future aircraft 
availability. 

Technical Data: 

In May 2011, we found[Footnote 35] that DOD needs access to technical 
data--recorded information used to produce, support, maintain, or 
operate a system[Footnote 36]--related to its weapon systems in order 
to help control costs and maintain flexibility in the acquisition and 
sustainment of those weapon systems. Technical data can enable the 
government to complete maintenance work in-house, as well as to 
competitively award acquisition and sustainment contracts. 
Additionally, we found that for service contracts pertaining to DOD 
weapon programs, which can involve products as well as support 
services, the lack of access to proprietary technical data and a heavy 
reliance on specific contractors for expertise creates limitations to, 
or even precludes, the possibility of competition.[Footnote 37] 

Within the past 5 years, acquisition reforms such as the Weapon 
Systems Acquisition Reform Act of 2009 and other initiatives seeking 
greater efficiency and cost savings in acquisitions have put greater 
emphasis on obtaining technical-data rights and on maintaining 
competition throughout the life cycle of weapon systems. In 2010, DOD 
announced its "Better Buying Power" initiative that outlines a series 
of actions and directives to promote competition, including ensuring 
that technical data requirements are considered at key milestones of 
the acquisition cycle.[Footnote 38] Specifically, in November 2010, 
the Under Secretary of Defense for Acquisition, Technology and 
Logistics issued a memorandum that requires program officials to (1) 
conduct a business case analysis that outlines the technical data 
rights the government will pursue to ensure competition, and (2) 
include the results of this analysis in certain strategies at 
Milestone B.[Footnote 39] However, according to DOD officials, because 
the F-35 program's initial Milestone B decision occurred back in 2001, 
and these initiatives started nearly a decade later, the F-35 program 
is getting a late start on its assessment of its technical data needs. 

In accordance with the Acquisition Strategy for the October 2001 
Milestone B, the F-35 program did not acquire technical data suitable 
for competition for the F-35 or its subsystems under its competitively 
awarded system development contract. Furthermore, as stated in the 
Acquisition Strategy, since the system design contract is in place, 
the acquisition of data suitable for competition may be prohibitively 
expensive and would not result in a commensurate return on investment. 
The current iteration of the Acquisition Strategy leaves the door open 
for acquiring current and future technical data that could support 
future competition through the life cycle. Specifically, DOD plans to 
assess the merits of acquiring operations, maintenance, installation, 
and training data that would support sustainment and operation 
efforts. Planning for follow-on procurements is to include data 
rights/management strategies that will support, to the extent 
practical, the options for competition through the remainder of the 
program life cycle. With respect to this assessment, and DOD's current 
requirements related to technical data, DOD officials confirmed the 
need for the F-35 program to begin identifying technical data needs 
and costs related to future sustainment of the aircraft. They said 
that DOD has begun discussions with contractor officials to determine 
who, pursuant to the applicable contract clauses, has the rights to 
specific technical data associated with the various hardware and 
software systems. According to JPO officials, technical-data rights 
could play a significant role in the development of the sustainment 
strategy for the F-35; however, the federal government does not have 
an understanding of the technical data it currently owns, what 
technical data it may still need, and how much it may cost to acquire 
those data to support the future sustainment of the F-35. 

According to current DOD acquisition policy, an Intellectual Property 
(IP) Strategy must be established and maintained for all defense 
acquisition programs in order to identify and manage the full spectrum 
of IP and related issues, such as technical data, from the inception 
of the program and throughout the life cycle. The IP Strategy 
describes how program management will assess program needs for, and 
acquire competitively whenever possible, the IP deliverables and 
associated license rights necessary for competitive and affordable 
acquisition and sustainment over the entire product life cycle. The IP 
Strategy is to be updated throughout the entire product life cycle, 
summarized in the Acquisition Strategy, and presented with the Life 
Cycle Sustainment Plan during the Operations and Support Phase. 
[Footnote 40] The IP Strategy is also to integrate, for all systems, 
the IP planning elements required under Subpart 207.106 (S-70) of the 
Defense Federal Acquisition Regulation Supplement (DFARS). 

The DFARS[Footnote 41] specifically states the following: 

* In accordance with Section 802(a) of the National Defense 
Authorization Act for Fiscal Year 2007 (Pub. L. No. 109-364) and DOD 
policy requirements, acquisition plans[Footnote 42] for major weapon 
systems and subsystems of major weapon systems shall: 

* assess the long-term technical data and computer software needs of 
those systems and subsystems; and: 

* establish acquisition strategies that provide for the technical data 
and computer software deliverables and associated license rights 
needed to sustain those systems and subsystems over their life cycle; 
the strategy may include: 

- the development of maintenance capabilities within DOD; or: 

- competition for contracts for sustainment of the systems or 
subsystems. 

According to the current F-35 acquisition strategy, the F-35 program 
will assess the merits of acquiring current and future technical data 
that could support future competition throughout the life cycle, and 
will help provide the basis for identifying the data and data rights 
required for the sustainment of the program. However, as of July 2014, 
DOD officials had not completed assessments of technical data rights, 
including a plan that would identify what technical data rights the 
federal government currently owns, what additional technical data 
rights it may need, and the costs associated with purchasing any 
necessary technical data. According to DOD officials, the program is 
planning to address these technical data rights issues as part of the 
future sustainment strategy that will document decisions for long-term 
operations and support for the aircraft. To date, although DOD 
officials have acknowledged the importance of technical data rights to 
the future of the F-35 program and have told us that discussions with 
contractor officials regarding ownership of those rights have begun, 
we have not seen any documentation confirming that assessments of 
technical data rights will be completed. Until DOD determines the 
technical data rights the federal government currently owns and its 
critical technical data needs and associated costs, the program runs 
the risk of being limited in its flexibility to make changes to 
sustainment plans and the overarching sustainment strategy. 
Furthermore, without a long-term IP Strategy, the program runs the 
risk of not being able to compete requirements for such things as 
spare parts. Unless technical data rights needs are considered up 
front, critical data and software may not be acquired, rendering them 
unavailable (or unaffordable) years later when seeking to maximize 
competition on a program during its sustainment phase. 

Improved Assumptions and Additional Analyses Could Increase 
Reliability of F-35 O&S Costs: 

DOD's most recent O&S cost estimates are comprehensive in that they 
include all DOD-required program elements and are organized according 
to a standard O&S cost-estimating structure; but weaknesses exist with 
respect to a few of the assumptions, and the estimates did not include 
all analyses necessary to make them fully reliable.[Footnote 43] 

DOD Created Two Primary Estimates of Operating and Support Costs for 
the F-35 across Its Life Cycle: 

There are two primary DOD F-35 O&S cost estimates, one developed by 
the JPO and one by CAPE. O&S costs include the direct and indirect 
costs of sustaining a fielded system, such as costs for replacement 
parts, fuel, maintenance, personnel, support facilities, and equipment 
(see table 1). The JPO's 2013 estimate, which is used to drive budget 
planning, lists the total F-35 O&S costs across a 56-year life cycle 
as approximately $916 billion in then-year dollars. CAPE's estimate, 
which is used to assess affordability and is required at major 
milestones or by special request, lists the same cost to be about 
$1.02 trillion in then-year dollars.[Footnote 44] Generally, either 
the JPO's or CAPE's estimate is used as DOD's estimate of record and 
published in the Selected Acquisition Report (SAR)--a report that DOD 
is required to submit annually to Congress on each of its major 
defense acquisition programs.[Footnote 45] 

Table 1: JPO and CAPE 2013 Total Estimated O&S Costs: 

Then-year dollars in billions: 

Operating and Support (O&S) cost element: Mission personnel; 
Joint Program Office (JPO): $212.9 billion; 
Cost Assessment and Program Evaluation (CAPE): $236.4 billion. 

Operating and Support (O&S) cost element: Unit-level consumption[A]; 
Joint Program Office (JPO): $167.9 billion; 
Cost Assessment and Program Evaluation (CAPE): $158.7 billion. 

Operating and Support (O&S) cost element: Maintenance; 
Joint Program Office (JPO): $270.6 billion; 
Cost Assessment and Program Evaluation (CAPE): $326.3 billion. 

Operating and Support (O&S) cost element: Unit-level maintenance[B]; 
Joint Program Office (JPO): $246.8 billion; 
Cost Assessment and Program Evaluation (CAPE): $282.9 billion. 

Operating and Support (O&S) cost element: Intermediate-level 
maintenance[C]; 
Joint Program Office (JPO): 0.0; 
Cost Assessment and Program Evaluation (CAPE): 0.0. 

Operating and Support (O&S) cost element: Depot-level maintenance; 
Joint Program Office (JPO): $23.8 billion; 
Cost Assessment and Program Evaluation (CAPE): $43.4 billion. 

Operating and Support (O&S) cost element: Contractor and sustaining 
support[D]; 
Joint Program Office (JPO): $158.8 billion; 
Cost Assessment and Program Evaluation (CAPE): 190.5 billion. 

Operating and Support (O&S) cost element: Indirect support[E]; 
Joint Program Office (JPO): $106.0 billion; 
Cost Assessment and Program Evaluation (CAPE): $104.6 billion. 

Operating and Support (O&S) cost element: Total; 
Joint Program Office (JPO): $916.3 billion; 
Cost Assessment and Program Evaluation (CAPE): $1.02 trillion. 

Source: GAO analysis of JPO and CAPE data. GAO-14-778. 

Notes: Both life-cycle cost estimates were developed in 2013 and span 
56 years--the JPO estimate from 2009 to 2065 and the CAPE estimate 
from 2010 to 2066. Figures in table may not sum due to rounding. 

[A] Consolidates fuel, training expendables, and other unit-level 
consumption. 

[B] Consolidates unit-level consumables, parts, and depot-level 
repairables, which include repairable individual parts, assemblies, or 
subassemblies that are required for the repair of the aircraft and 
related equipment. 

[C] Intermediate-level maintenance represents the cost of labor and 
materiel for maintenance that is not performed at the operational or 
depot levels, but by an intermediate-level maintenance organization. 
Intermediate-level maintenance activities may include calibration, 
repair, testing, and replacement of parts, components, or assemblies, 
and technical assistance. 

[D] Consolidates sustaining support and contractor support from CAPE 
estimate and sustaining support and continuing system improvements 
from JPO estimate. Sustaining support relates to the cost of support 
activities other than maintenance that can be attributed to a system 
and are provided by organizations other than operating units. 
Continuing system improvements relate to the cost of hardware and 
software modifications to keep the system operating and current. 

[E] Indirect support represents the cost related to general services, 
such as medical care for active-duty military and base operating 
support. 

[End of table] 

DOD F-35 Cost Estimates Are Comprehensive, but Weaknesses Exist in a 
Few Assumptions: 

The JPO and CAPE both created comprehensive F-35 O&S cost estimates, 
but there are weaknesses in a few of the assumptions used in the 
estimates. According to the GAO Cost Estimating and Assessment Guide, 
a good cost estimate depends on how well the program is defined in the 
technical baseline and work-breakdown structure. Both the JPO and CAPE 
used a DOD-approved technical baseline and a product-oriented O&S cost 
work-breakdown structure to develop their 2013 estimates. The 
technical baseline provides the comprehensive program description, 
which is used to determine what must be included in the estimate. A 
work-breakdown structure is a necessary program-management tool 
because it provides a basic framework for estimating costs, allowing 
for the estimates to be easily updated and compared to future 
estimates. Both DOD guidance and the GAO Cost Estimating and 
Assessment Guide highlight the importance of good documentation in 
cost estimating, stating that a complete cost estimate should be 
formally documented to serve as an audit trail of source data, 
methods, and results.[Footnote 46] In addition, the GAO Cost 
Estimating and Assessment Guide states that all assumptions and 
exclusions the estimate is based on should be reasonable, clearly 
identified, and explained.[Footnote 47] While the JPO and CAPE 
estimates are comprehensive, weaknesses exist because both estimates 
used unreasonable assumptions for fuel burn rates and have not clearly 
documented intermediate-level maintenance costs (see appendix II for 
more information on our analysis). 

* Fuel burn rates: The JPO and CAPE estimates do not use a reasonable 
assumption for the Marine Corps fuel burn rate across the life cycle 
of the aircraft. Both estimates use a fuel burn assumption of 1,493 
gallons per flying hour for the Marine Corps variant. While the rate 
used in the estimates was calculated based on service-planned missions 
at maturity and DOD-validated physics-based models, this burn rate is 
an almost exact match of the burn rate being observed at the 
operational sites. This may be a reasonable assumption for the short 
term, but it is likely that the fuel burn rates will increase in the 
long term. The burn rate of 1,493 gallons per flying hour used in the 
JPO and CAPE cost estimates reflects fuel burn data from aircraft 
flown using limited capabilities--at slower speeds and at lower 
altitudes than the F-35 will eventually fly. After 2015, the Marine 
Corps is planning on using the aircraft's increased capability, which 
will likely result in more fuel being used. In addition, shortly after 
2015, the Marine Corps is planning on deploying the F-35 to ships, 
where the aircraft's fuel-intensive vertical landing capability will 
be used more frequently, which will likely increase fuel usage. 
Consequently, the use of lower fuel burn rate across the entire life 
cycle of the Marine Corps variant is not a reasonable assumption. 

Similarly, the JPO estimate lowered its fuel burn rate assumption to 
1,480 gallons per flight hour from 1,558 gallons per flight hour for 
the Air Force variant. However, the Air Force stated that the more 
conservative assumption of 1,558 gallons per flight hour should be 
used across the life cycle because the F-35 has yet to use its full 
flight capabilities, weapons, or mission systems, which will likely 
increase the fuel burn rates in the long term. The higher fuel burn 
rate assumption for the Air Force would represent a $4.0 billion cost 
increase in base year 2012 dollars across the life cycle of the 
aircraft. 

* Intermediate-level maintenance. These costs are not clearly 
documented in either the JPO or the CAPE cost estimates. For the F-35, 
intermediate-level maintenance tasks include wheel and tire servicing, 
battery maintenance, structural airframe support, and inspections. The 
documentation supporting DOD's cost estimate states that intermediate-
level maintenance costs are not calculated for the F-35, but the 
program's maintenance planning documents state that intermediate-level 
maintenance will be used in the sustainment of the F-35, and it is 
currently being employed. Specifically, Marine Corps officials told us 
that intermediate-level personnel at Yuma are conducting some F-35 
activities, including tire, wheel, and battery servicing. In addition 
to the existing F-35 intermediate-level maintenance being conducted at 
Yuma, DOD has conducted at least five analyses to date, each resulting 
in a recommendation to repair a F-35 Marine Corps variant component at 
the intermediate level. These five component repairs represent more 
than $30 million in intermediate-level maintenance costs across the 
life cycle of the aircraft, but they are not documented in DOD's cost 
estimates. Moreover, all of the analyses to identify which components 
will be repaired at the intermediate level are not yet complete, and 
the costs related to intermediate-level maintenance could increase. 
According to DOD officials, the costs of these intermediate-level 
repairs are already included under mission personnel and unit-level 
maintenance elements of the cost estimates. However, this is not 
documented in the mission personnel or unit-level maintenance sections 
of the cost estimates. While DOD guidance states that intermediate-
level costs can be combined with unit-level costs, the guidance also 
states that this should be noted in the cost estimate's supporting 
documentation to avoid an interpretation that a portion of the 
maintenance costs were omitted. Without clear documentation of these 
costs it is unclear whether they are fully represented in the JPO and 
CAPE estimate. 

In addition, the JPO estimate does not include reasonable assumptions 
about part-replacement rates and depot maintenance. 

* Part replacement: The JPO estimate does not include reasonable 
assumptions for part replacement. Based on data from the Air Force and 
Marine Corps F-35 variants at testing and operational sites, parts are 
being replaced, on average, 15 to 16 times more frequently than the 
assumptions used across the life cycle of the JPO estimate (see table 
2). For example, a sensor that costs about $4,800 is being replaced 60 
to 129 times more frequently than anticipated across the life cycle of 
the JPO cost estimate. Another example is the battery charger unit, 
which costs about $60,000 to acquire new, and is being replaced 3 to 8 
times more frequently than anticipated across the life cycle of the 
JPO cost estimate. 

Table 2: Sample of F-35 Parts with Higher Removal Rates Than Assumed 
in Joint Program Office (JPO) 2013 Cost Estimate: 

Component/part: Thermal management system fan; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 10; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 22; 
Replacement cost per part (dollars): $17,732. 

Component/part: Fiber channel switch; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 13; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 9; 
Replacement cost per part (dollars): $33,485. 

Component/part: Data security module; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 13; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 12; 
Replacement cost per part (dollars): $239,256. 

Component/part: 270 volt battery unit; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 1[A]; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 2; 
Replacement cost per part (dollars): $132,717. 

Component/part: Data transfer cartridge; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 4; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 5; 
Replacement cost per part (dollars): $20,477. 

Component/part: Helmet display management computer; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 1[A]; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 3; 
Replacement cost per part (dollars): $209,266. 

Component/part: Signal processor; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 6; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 6; 
Replacement cost per part (dollars): $125,095. 

Component/part: Oxygen generation system; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 2; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 8; 
Replacement cost per part (dollars): $41,985. 

Component/part: Seat portion assembly; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 5; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 3; 
Replacement cost per part (dollars): $52,628. 

Component/part: Hot air sensor; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 60; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 129; 
Replacement cost per part (dollars): $4,797. 

Component/part: General processor; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 6; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 3; 
Replacement cost per part (dollars): $85,648. 

Component/part: Helmet display unit; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 3; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 5; 
Replacement cost per part (dollars): $124,917. 

Component/part: Pump module; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 19; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 13; 
Replacement cost per part (dollars): $36,660. 

Component/part: 270 volt battery charger unit; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 3; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 8; 
Replacement cost per part (dollars): $59,745. 

Component/part: Distribution valve; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 192; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 83; 
Replacement cost per part (dollars): $6,314. 

Component/part: Panoramic cockpit display electronics unit; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 2; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 1; 
Replacement cost per part (dollars): $146,179. 

Component/part: Integrated power package ventilation fan; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 6; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 9; 
Replacement cost per part (dollars): $4,574. 

Component/part: Average of all 195 Air Force and 189 Marine Corps 
variant parts analyzed[B]; 
Air Force variant: number of times more frequently removed than 
assumed by JPO: 16; 
Marine Corps variant: number of times more frequently removed than 
assumed by JPO: 15; 
Replacement cost per part (dollars): $47,624. 

Source: GAO analysis of JPO data. GAO-14-778. 

Notes: The parts presented in this table are taken from a JPO list of 
"high driver" replacement parts from November 2013, some of which will 
be addressed in the JPO's ongoing reliability-improvement efforts. The 
mean flight hours between removals (MFHBR) data used in this analysis 
were taken from operational and test Air Force F-35A (CTOL) and Marine 
Corps F-35B (STOVL) variants during a 12-month period ending on March 
31, 2014. The JPO assumptions for removals are developed by the 
contractor and represent the predicted frequency of removals (MFHBR) 
once the aircraft reaches maturity, which is when the entire F-35 
fleet reaches a cumulative 200,000 flying hours. Navy variant data 
were excluded from this analysis because of the limited number of Navy 
variant aircraft being maintained to date. 

[A] These frequencies are slightly higher than one, but have been 
rounded down. 

[B] This average includes the 17 "high driver" replacement parts 
listed above. 

[End of table] 

The part-replacement assumptions used by the JPO reflect the 
anticipated reliability of the aircraft at maturity--once the entire 
fleet has achieved 200,000 flight hours. According to JPO officials, 
the reliability issues causing the high part-replacement rates will be 
resolved once the aircraft reaches maturity, which is estimated to 
occur at the end of fiscal year 2019. The JPO increased the cost of 
replacing parts in the 2010 to 2019 portion of its estimate to reflect 
the lower reliability of the aircraft until maturity.[Footnote 48] 
However, according to officials from the Institute for Defense 
Analysis, who conducted a study of the F-35's R+M for DOT&E, the F-35 
program would have to achieve a higher reliability-growth improvement 
rate than has been observed in almost all other aircraft in order to 
meet the anticipated reliability by 2020.[Footnote 49] As previously 
stated, reliability improvement efforts are under way that could 
reduce these costs, but it is unlikely that these efforts will bring 
significant results in the near term because the current F-35 fleet 
must be modified into the configuration necessary for reliability 
improvement. As a result, for the next 10 years, the F-35 fleet will 
not represent the configurations necessary for reliability-growth 
improvement. In addition, according to the study of the F-35's R+M 
conducted by the Institute for Defense Analyses for DOT&E, future 
efforts to increase reliability may be hampered by increased failure 
rates as the fielded fleet starts flying more operationally 
representative missions, with more aggressive maneuvering and more 
extensive use of mission systems. 

CAPE adjusted its part-replacement assumptions for its 2013 estimate 
to reflect higher replacement rates across the life cycle of the 
aircraft. As a result, CAPE's 2013 estimated cost for parts across the 
life cycle of the program is $23 billion greater than the JPO's in 
base year 2012 dollars. CAPE officials told us that if they had used 
the actual replacement rates being observed at the F-35 sites, the 
part-replacement cost across the life cycle of the program would have 
been roughly $120 billion greater than the JPO's in base year 2012 
dollars. 

* Depot maintenance: Finally, the JPO estimate does not include a 
reasonable assumption regarding F-35 depot maintenance. The depot 
maintenance cost element includes the labor, materiel, and overhead 
costs for overhaul or rework of aircraft returned to a government or 
contractor repair facility. This O&S cost includes scheduled and 
unscheduled depot maintenance, as well as inspections. We have found 
this to be a cost element that has been underestimated in the past for 
legacy and fifth-generation aircraft, including the F/A-18 and F-22. 
[Footnote 50] The JPO reduced its assumptions related to the cost of 
depot maintenance by about half, from about $2 million per scheduled 
depot induction in the 2012 estimate to about $600,000-$1.3 million in 
the 2013 estimate. This change resulted in a $19.7 billion reduction 
in then-year dollars from the JPO's 2012 estimate. JPO officials 
stated that this reduction was the result of using a different cost-
estimation method, which had the effect of lowering the hourly cost 
per induction. According to the assumptions in the JPO's 2013 
estimate, the average cost per depot induction hour is roughly $205. 
However, in 2004, the average cost per depot induction was about $238, 
and it is likely that labor and materiel costs have increased over the 
past 10 years.[Footnote 51] To develop its hourly cost, the JPO used F-
16 data from Lockheed Martin contractors that had been adjusted for 
the F-35. According to the JPO's current assumptions, materiel costs 
would be 30 percent of the labor costs, but data for the AV-8 and F/A-
18 depot inductions used in the previous year's estimate indicate that 
depot induction materiel costs have historically been closer to 45 
percent of labor costs. As a result, it is likely that the depot 
maintenance hourly cost used in the JPO's 2013 estimate is not a 
reasonable assumption. 

DOD Has Not Conducted Uncertainty Analyses to Determine Credibility of 
Estimates: 

Every cost estimate contains a degree of uncertainty because of the 
many assumptions that must be made about the future. To mitigate this 
uncertainty, a variety of checks and analyses can be conducted to 
determine the credibility of the assumptions and the estimate as a 
whole. While the JPO and CAPE have done some cross-checks and 
sensitivity analyses to mitigate the uncertainty of their assumptions, 
they have not conducted uncertainty analyses to determine whether the 
estimates reflect the most likely costs. 

According to the GAO Cost Estimate and Assessment Guide and DOD 
guidance on cost estimating, a good practice for determining the 
reasonableness of an estimate's assumptions for certain cost elements 
is to cross-check them with other programs' estimates or other cost-
estimate methodologies to determine whether results are similar. JPO 
and CAPE cost estimators stated that they cross-checked their results 
of key cost elements against O&S estimates for other aircraft to 
determine whether they produced similar results. For example, 
according to a JPO cost-estimating official, the F-35 results for the 
depot maintenance cost element were three to five times greater than 
the same costs for legacy aircraft. In addition, CAPE conducted a 
cross-check comparing the cost per flying hour of legacy and F-35 
aircraft, which identified F-35 costs as being significantly higher 
than the legacy aircraft. The official stated that this was reasonable 
given the complexity of the fifth-generation aircraft and the 
additional capability that the F-35 offers. 

The JPO and CAPE have also conducted some sensitivity analyses, but 
they have not conducted uncertainty analyses to determine the 
credibility of their estimates. Sensitivity analyses examine how 
changes to individual assumptions and inputs affect the estimate as a 
whole. Both the JPO and CAPE cost estimators have told us that they 
conducted "what if" scenarios--varying the assumptions of certain 
variables individually to see how the total O&S costs may be affected. 
Although CAPE did not conduct a sensitivity analysis on its 2013 O&S 
cost estimate, it did conduct one for its 2011 F-35 O&S cost estimate, 
which contains many of the same assumptions as its 2013 estimate. 
However, as a best practice, a sensitivity analysis should be included 
in all cost estimates because it examines the effects of changing 
assumptions and ground rules. The JPO asked officials at the Office of 
the Secretary of Defense, Logistics and Materiel Readiness (OSD-LMR), 
to conduct a sensitivity analysis using the JPO cost model. OSD-LMR 
conducted this analysis, determining the cost elements that 
represented the highest percentage of cost and varying assumptions to 
determine which factors have the greatest effect on the total O&S 
cost. According to DOD guidance, sensitivity analyses are useful for 
identifying critical estimating assumptions, but they have limited 
utility in providing a comprehensive sense of overall uncertainty. In 
contrast, an uncertainty or quantitative risk analysis can provide a 
broad overall assessment of variability in the cost estimate. 
Similarly, the GAO Cost Estimating and Assessment Guide states that 
risk and uncertainty analyses should be performed to determine the 
level of risk associated with the estimate. JPO and CAPE cost 
estimators have not conducted uncertainty analyses to understand the 
potential range of costs around their point estimates and to convey 
their level of confidence that the estimates reflect the most likely 
costs. JPO and CAPE officials stated that they did not conduct 
uncertainty analyses because they are not a standard practice for DOD 
when creating O&S cost estimates. However, according to the GAO Cost 
Estimating and Assessment Guide, an estimate without a risk and 
uncertainty analysis is unrealistic because it does not assess the 
variability in the estimate. 

In April 2014 testimony, the F-35 program executive officer 
highlighted the importance of refining the cost estimate as a key 
component of DOD's risk-reduction efforts related to the F-35. 
[Footnote 52] Without fully reliable O&S cost information to develop 
accurate budget requests and address the affordability of the F-35 
program, the military services may be at risk of sustainment cost 
overruns in the near and long terms. The use of improved assumptions, 
as well as an understanding of the level of confidence in the 
estimates through the use of an uncertainty analysis, would also 
improve DOD's ability to make F-35 sustainment decisions. 

Conclusions: 

The F-35 program remains one of the top priorities for the future of 
our national defense, according to senior defense leadership. However, 
as we have previously reported, the F-35 program has experienced cost, 
schedule, and performance problems that have put it 7 years behind 
schedule and significantly over its original budget. With life-cycle 
sustainment costs estimated by DOD at about $1 trillion, the F-35 is 
approaching a critical juncture. In an era of significant budgetary 
pressures and competition for resources, with the Air Force, Navy, and 
Marine Corps each beginning to deploy and maintain F-35 aircraft 
within the next 4 years, and with full-rate production planned in 
fiscal year 2019, DOD must find ways to make the sustainment of the F-
35 program affordable. DOD has begun to devote attention to 
sustainment through the creation of plans and analyses that could help 
inform programmatic decisions and shape its overall sustainment 
strategy. One key decision is for DOD to determine what it can 
realistically afford with respect to the F-35 program, but until it 
identifies affordability constraints tied to the military services' 
budgets, it will continue to develop and field the most costly weapon 
system program in history without knowing whether the Air Force, Navy, 
or Marine Corps can pay for it. Furthermore, while DOD has begun 
testing and fielding the F-35 at sites around the country, the program 
faces several risks, including ALIS, software R+M, and uncertainties 
related to technical data rights, which could adversely affect DOD's 
sustainment strategy. If these risks remain unaddressed, sustainment 
costs could potentially increase well beyond current estimated levels, 
and operational readiness could suffer. 

Finally, although DOD's current cost estimates identify most 
sustainment costs and document cost-influencing assumptions, certain 
elements may be underestimated. In some areas, such as the assumptions 
underlying future fuel burn rates, continuing to underestimate these 
elements could translate to billions of dollars in cost increases over 
the life cycle of the program. Therefore, it is imperative that DOD 
provide an accurate picture of the current and future state of the 
program, as well as a greater degree of assurance that its cost 
estimates are credible--through the use of uncertainty analyses--to 
help inform the sustainment strategy. With more than 2,400 F-35 
aircraft still planned for purchase, DOD has an opportunity to chart a 
more informed, analytical, and affordable path for the program. 
However, sustainment traditionally represents about 70 percent of a 
weapon system's life-cycle cost, and until DOD develops a viable 
sustainment strategy, addresses key risks, and improves its cost 
estimates, the department will likely continue to present Congress 
with near-term funding requests without having full knowledge of the 
extent of the program's long-term financial requirements. 

Recommendations for Executive Action: 

We are making the following eight recommendations to improve DOD's 
sustainment planning and operating and support (O&S) cost estimates 
for the F-35 program. 

To help DOD develop an affordable sustainment strategy for the F-35, 
we recommend that the Secretary of Defense direct the Under Secretary 
of Defense for Acquisitions, Technology and Logistics to direct the F-
35 Program Executive Officer to establish affordability constraints 
linked to, and informed by, military service budgets that will help 
guide sustainment decisions, prioritize requirements, and identify 
additional areas for savings by March 2015, at which point the Future 
Support Construct decision will be approved. 

To help DOD address key risks to F-35 affordability and operational 
readiness, and to improve the reliability of its O&S cost estimates 
for the life cycle of the program, we recommend that the Secretary of 
Defense direct the F-35 Program Executive Officer to take the 
following five actions: 

* To enable DOD to better identify, address, and mitigate performance 
issues with the Autonomic Logistics Information System (ALIS) that 
could have an effect on affordability, as well as readiness, establish 
a performance-measurement process for ALIS that includes, but is not 
limited to, performance metrics and targets that (1) are based on 
intended behavior of the system in actual operations and (2) tie 
system performance to user requirements. 

* To develop a high level of confidence that the aircraft will achieve 
its R+M goals, develop a software reliability and maintainability 
(R+M) assessment process, with metrics, by which the program can 
monitor and determine the effect that software issues may have on 
overall F-35 R+M issues. 

* To promote competition, address affordability, and inform its 
overarching sustainment strategy, develop a long-term Intellectual 
Property (IP) Strategy to include, but not be limited to, the 
identification of: 

- current levels of technical data rights ownership by the federal 
government and: 

- all critical technical data needs and their associated costs. 

* To improve the reliability of the JPO F-35 O&S cost estimate, 
clearly document assumptions related to intermediate-level maintenance 
costs and revise assumptions related to fuel burn rates, part 
replacement, and depot-maintenance induction in its future F-35 O&S 
cost estimates to better reflect the current and future state of the F-
35 program. 

* To understand the potential range of costs associated with the JPO F-
35 O&S cost estimate, conduct uncertainty analyses on future JPO 
estimates. 

To improve the reliability of the Cost Assessment and Program 
Evaluation (CAPE) F-35 O&S cost estimate, we recommend that the 
Secretary of Defense direct the Director of CAPE to take the following 
two actions for future F-35 O&S cost estimates: 

* clearly document assumptions related to intermediate-level 
maintenance and revise fuel burn assumptions to better reflect the 
current and future state of the F-35 program and: 

* conduct uncertainty analyses to understand the potential range of 
costs associated with its estimates to reflect the most likely costs 
associated with the program. 

Agency Comments and Our Evaluation: 

In written comments on a draft of this report, DOD concurred with 
seven of the report's eight recommendations and partially concurred 
with one recommendation. DOD's comments are summarized below and 
reprinted in appendix III. DOD also provided technical comments, which 
we have incorporated into our report where appropriate. 

DOD concurred with the recommendation that the Secretary of Defense 
direct the Under Secretary of Defense for Acquisitions, Technology, 
and Logistics to direct the F-35 Program Executive Officer to 
establish affordability constraints linked to, and informed by, 
military service budgets. DOD stated that it established F-35 
affordability targets for unit recurring flyaway cost and for 
sustainment cost in March 2012 to guide development of an affordable 
strategy. DOD further stated that the F-35 program is currently 
operating under these affordability targets and using affordability 
goals to reduce life-cycle costs. We acknowledge that the department 
established affordability targets for sustainment in March 2012, but 
these actions do not fully address the intent of our recommendation 
because, as we note in our report, these affordability targets may not 
be representative of what the services can actually afford because the 
methodology for determining the targets was not informed by resource 
constraints within military service budgets. We further acknowledge in 
our report that DOD is currently engaged in cost-reduction efforts, 
but until DOD establishes affordability targets that are informed by 
military service budgets, the department cannot be sure whether the 
cost savings it achieves through current efforts will lead to an 
affordable sustainment strategy. If DOD develops affordability targets 
that are linked to and driven by military service budgets, this action 
should address the recommendation. 

DOD concurred with the recommendation that the Secretary of Defense 
direct the F-35 Program Executive Officer to establish a performance-
measurement process for ALIS that includes, but is not limited to, 
performance metrics and targets that (1) are based on intended 
behavior of the system in actual operations and (2) tie system 
performance to user requirements. DOD stated that the F-35 program 
will continue to use ALIS performance metrics and targets to assess 
effects on the operational requirements, such as the sortie-generation 
rate key performance parameter, through the remainder of system 
development and demonstration (SDD). DOD also stated that while the F-
35 program continues to assess these effects today, future incremental 
ALIS functional deliveries and additional operational experience will 
enable more-refined metrics over time which will help reduce 
performance risk and ensure a successful operational test and 
evaluation at the end of SDD. As our report states, we acknowledge 
that metrics for ALIS's availability are used to track the development 
of ALIS, but these metrics do not fully address the intent of our 
recommendation because there is no performance-measurement process 
that directly ties ALIS performance to user requirements. While the 
sortie-generation rate key performance parameter is one way in which 
to monitor performance, this metric is aimed at measuring the 
performance of the entire F-35 air system and not, specifically, how 
ALIS performs its functions in response to user requirements. We agree 
that, as DOD noted, performance metrics and targets may be refined 
over time to help reduce performance risk, but until DOD develops 
performance targets and metrics tied directly to ALIS functionality 
and user requirements, it may still face additional costs and risks to 
operational readiness and availability. If DOD develops these 
performance targets and metrics as part of a performance-measurement 
process, this action should address our recommendation. 

DOD concurred with the recommendation that the Secretary of Defense 
direct the F-35 Program Executive Officer to develop a software 
reliability and maintainability assessment process, with metrics, by 
which the program can monitor and determine the effect that software 
issues may have on overall F-35 reliability and maintainability (R+M) 
issues. DOD stated that the F-35 program has an established process 
for assessing software reliability and will continue to monitor and 
determine the effect that software issues may have on overall F-35 
R+M. DOD noted that this process includes a Failure Reporting Analysis 
Corrective Action System database, a Joint Reliability and 
Maintainability Evaluation Team, and a Test Data Scoring Board, and 
uses the Mean-Flight-Hours-Between-Failure metric. Although we agree 
that the components DOD lists are in place, and that DOD does have a 
process for assessing F-35 R+M, these actions do not fully address the 
intent of our recommendation because, as we state in our report, this 
process is aimed primarily at assessing the reliability of the F-35 
aircraft's hardware and not, specifically, its associated software. 
Specifically, the process does not determine and monitor software 
reliability using software-specific metrics and targets. We continue 
to believe that DOD's current process and associated metrics do not 
allow program officials to have a full understanding of the root 
causes of reliability issues and the extent to which software failures 
contribute to these issues. As our report states, the reliability of 
the aircraft is measured with a number of different metrics; however, 
according to DOD officials, the current process and associated metrics 
are almost exclusively dedicated to hardware. Since the F-35 is the 
most software-intensive aircraft DOD has procured to date, 
establishing a software reliability assessment process with metrics 
and targets, outside of hardware reliability, would allow DOD to 
determine the extent to which software failures contribute to the 
overall R+M of the F-35. If DOD develops such a process, this action 
should address our recommendation. 

DOD concurred with the recommendation that the Secretary of Defense 
direct the F-35 Program Executive Officer to, in order to promote 
competition, address affordability and, to inform its overarching 
sustainment strategy, develop a long-term IP Strategy to include, but 
not be limited to, the identification of (1) current levels of 
technical data rights ownership by the federal government and (2) all 
critical technical data needs and their associated costs. DOD stated 
that the F-35 acquisition strategy includes a data-management strategy 
whereby it will continue to assess the acquisition of technical data 
as specific needs arise throughout the life cycle and will determine 
the level of specified rights, critical technical data needs and 
associated costs, through business-case analysis, contract strategy 
development, and ongoing sustainment planning. If DOD develops a long-
term IP strategy that includes the identification of its current level 
of specified rights, critical technical data needs, and associated 
costs, this action should address the recommendation. 

DOD concurred with the recommendation that the Secretary of Defense 
direct the F-35 Program Executive Officer to clearly document 
assumptions related to intermediate-level maintenance costs and revise 
assumptions related to fuel burn rates, part replacement, and depot-
maintenance induction in its future F-35 O&S cost estimates. DOD 
stated that the Program Executive Officer will document intermediate-
level maintenance cost assumptions and revise fuel burn rates, part 
replacement, and depot-maintenance induction assumptions in future 
annual updates to the F-35 O&S cost estimate. We agree that, if the 
assumptions are revised and fully documented to reflect the current 
and future state of the program, this action should address the 
recommendation. 

DOD concurred with the recommendation that the Program Executive 
Officer conduct uncertainty analysis on future F-35 O&S cost 
estimates. DOD stated that it will ensure that the Program Executive 
Officer conducts risk and sensitivity analysis, in accordance with DOD 
guidance, in future annual updates to the cost estimate. If the 
Program Executive Officer performs an uncertainty or quantitative risk 
analysis on future JPO F-35 O&S estimates, this action should address 
the recommendation. 

DOD concurred with the recommendation that CAPE clearly document 
assumptions related to intermediate-level maintenance and revise its 
O&S cost estimate's fuel burn assumptions to better reflect the 
current and future state of the F-35 program. DOD stated that CAPE's 
current documentation of the O&S cost estimate reflects an approved 
two-level maintenance strategy (i.e., organizational-and depot-level 
maintenance only, no intermediate-level maintenance). DOD also stated 
that CAPE will ensure that if the strategy is revised to include any 
planned intermediate-level maintenance activities, the activities and 
associated costs will be fully captured within program estimates and 
documentation. However, the program's maintenance planning documents 
already state that intermediate-level maintenance will be used in the 
sustainment of the F-35, and it is currently being employed. In its 
written response, DOD further stated that any intermediate-level costs 
are already included in the estimate's mission personnel costs. While 
DOD guidance states that intermediate-level costs can be combined with 
unit-level costs, including mission personnel, the guidance also 
states that this should be noted in the cost estimate's supporting 
documentation to avoid an interpretation that a portion of the 
maintenance costs were omitted. However, as we note in the report, 
CAPE did not document intermediate-level maintenance costs in the 
mission personnel or unit-level maintenance sections of the cost 
estimates. Without clear documentation of these intermediate-level 
maintenance costs, it is unclear whether they are fully represented in 
the CAPE estimate. If CAPE clearly documents where and how 
intermediate maintenance costs are included in its future cost 
estimates, this action should address the recommendation. 

Related to revising its O&S cost estimate's fuel burn assumptions, DOD 
stated that CAPE will update its estimate to reflect the latest 
approved technical baseline and incorporate the latest approved fuel 
consumption actuals, as this information becomes available. As our 
report states, the estimate's fuel burn rate assumption should reflect 
the most likely future state of the program across its life cycle. To 
date, fuel consumption actuals are from aircraft flown using limited 
capabilities--at slower speeds and at lower altitudes than the F-35 
will eventually fly--and do not reflect the most likely future state 
of the program. We agree that, once fuel consumption data from 
aircraft flown using the full capability are available, these data 
could be used to update the assumptions. Until then, the estimate's 
fuel burn rate assumptions should be revised to reflect the future 
state of the program. If fully implemented, DOD's proposed actions to 
update its estimate should address the recommendation. 

Finally, DOD partially concurred with the recommendation that CAPE 
conduct uncertainty analyses to understand the total potential range 
of costs and to determine if its estimate reflects the most likely 
costs associated with the program. DOD stated that the department 
agrees with the value of understanding potential cost ranges, but 
stated that it considers the risk and sensitivity analyses regularly 
performed by CAPE to be a form of uncertainty analysis. As our report 
states, we acknowledge that CAPE has conducted some sensitivity 
analyses on previous F-35 O&S estimates, but it has not conducted an 
uncertainty or quantitative risk analysis on its F-35 O&S cost 
estimate. According to DOD guidance, sensitivity analyses are useful 
for identifying critical estimating assumptions, but they have limited 
utility in providing a comprehensive sense of overall uncertainty. In 
contrast, an uncertainty or quantitative risk analysis, such as a 
Monte Carlo simulation, can provide a broad overall assessment of 
variability in the cost estimate. Similarly, the GAO Cost Estimating 
and Assessment Guide states that risk and uncertainty analyses should 
be performed to determine the level of risk associated with an 
estimate. We continue to believe that without an uncertainty analysis, 
DOD cannot fully understand how much of the total potential O&S costs 
are accounted for in its nearly $1 trillion estimate. If CAPE performs 
an uncertainty analysis on future estimates, this action should 
address our recommendation. 

We are sending copies of this report to appropriate congressional 
committees; the Secretary of Defense; the Secretaries of the Air 
Force, Army, and Navy; and the Commandant of the Marine Corps. In 
addition, the report is available at no charge on the GAO website at 
[hyperlink, http://www.gao.gov]. 

If you or your staff has any questions about this report, please 
contact me at (202) 512-5431 or russellc@gao.gov. Contact points for 
our Offices of Congressional Relations and Public Affairs may be found 
on the last page of this report. Staff members making key 
contributions to this report are listed in appendix IV. 

Signed by: 

Cary Russell, Director: 
Defense Capabilities and Management: 

[End of section] 

Appendix I: Scope and Methodology: 

To address both of our objectives--that is, to evaluate F-35 
sustainment planning and risk and operating and support (O&S) cost 
estimates--we collected and analyzed information and interviewed 
officials from the following Department of Defense (DOD) offices: 

* Office of the Under Secretary of Defense (Acquisitions, Technology 
and Logistics); 

* Office of the Assistant Secretary of Defense (Logistics and Materiel 
Readiness); 

* Office of the Director for Cost Assessment and Program Evaluation 
(CAPE); 

* Office of the Director for Operational Test and Evaluation (DOT&E); 

* Office of the Deputy Assistant Secretary of Defense for 
Developmental Test and Evaluation; 

* Office of the Deputy Assistant Secretary of Defense for Systems 
Engineering; 

* Department of the Air Force; 

* Department of the Navy; 

* Headquarters Marine Corps; and: 

* F-35 Joint Program Office (JPO). 

We conducted visits to: 

* Eglin Air Force Base, Florida; 

* Marine Corps Air Station Yuma, Arizona; 

* Naval Air Station Patuxent River, Maryland; 

* Naval Air Systems Command, Maryland; and: 

* Lockheed Martin, Fort Worth, Texas. 

To assess the extent to which DOD has developed a sustainment strategy 
for the F-35 program and addressed potential risks related to 
affordability and operational readiness, we reviewed documentation of 
program plans and analyses with relevant sustainment elements, 
including the F-35 Life Cycle Sustainment Plan, the Weapon System 
Planning Document, the F-35 Future Support Construct, the F-35 
Autonomic Logistics Global Sustainment Concept of Operations, and the 
F-35 Operational Requirements Document. We reviewed documentation of 
ongoing analyses, including Level of Repair Analyses and Business Case 
Analyses. We compared these documents to DOD acquisition guidance and 
policy on sustainment planning, including the Defense Acquisition 
Guidebook, Interim DOD Instruction 5000.02 "Operation of the Defense 
Acquisition System," and DOD's Life Cycle Sustainment Plan and 
guidance. 

To determine the extent to which DOD has developed plans to mitigate 
sustainment risks, we conducted site visits (see locations above) and 
interviewed key DOD officials in order to collect information about 
ongoing operations, testing, and risks to the sustainment of the F-35. 
After identifying the three most significant risks to the sustainment 
of the program, we compared DOD's risk-mitigation efforts to DOD Risk 
Management Guide and other DOD acquisition guidance and DOD policy and 
guidance. Specifically, we reviewed DOD's efforts to monitor and 
improve the risks of (1) the performance of the Autonomic Logistics 
Information System (ALIS), (2) the reliability and maintainability 
(R+M) of the aircraft, and (3) the management of technical data. We 
then compared these efforts to DOD's Risk Management Guide, Interim 
DOD Instruction 5000.02, and other specific guidance and best 
practices, including DOD's System of Systems Guide, DOD's Manual for 
Improving Reliability, Availability, and Maintainability, and Naval 
Air Systems Command Software Logistics Primer. 

To determine the extent to which DOD has developed a reliable 
operating and support (O&S) cost estimate for the F-35 program, we 
evaluated both DOD's Joint Program Office (JPO) and its Cost 
Assessment and Program Evaluation (CAPE) office 2013 O&S cost 
estimates using GAO's Cost Estimating and Assessment Guide.[Footnote 
53] Specifically, the methodology outlined in the Cost Estimating and 
Assessment Guide is a compilation of best practices that federal cost-
estimating organizations and industry use to develop and maintain 
reliable cost estimates throughout the life cycle of an acquisition 
program. The JPO's estimate is used to drive budget planning and 
CAPE's estimate is used to assess affordability and is required at 
major milestones or by special request. Generally, either the JPO's or 
CAPE's estimate is used as DOD's estimate of record and published in 
the Selected Acquisition Report (SAR)--a report that DOD is generally 
required to submit annually to Congress on each of its major defense 
acquisition programs. For the F-35, DOD decided to use the CAPE O&S 
estimate in the 2012 and 2013 SARs. We evaluated both the JPO and the 
CAPE O&S estimates because both estimates are used in DOD decision 
making. According to the GAO Cost Estimating and Assessment Guide, 
there are four general characteristics of sound cost estimating: being 
well-documented, comprehensive, accurate, and credible. For the 
purposes of this engagement, we conducted a limited assessment and 
evaluated two of these characteristics: comprehensive and credible. We 
chose to evaluate the estimates' comprehensiveness because if a cost 
estimate is not comprehensive then it cannot fully meet the 
characteristics of being well-documented or accurate. For example, if 
the cost estimate is missing some cost elements, then the 
documentation will be incomplete and the estimate will be inaccurate. 
We also included an assessment of the credible characteristic due to 
the substantial estimated cost of the program and the need to better 
understand and quantify the F-35 O&S cost risk. To determine whether 
the comprehensive and credibility characteristics were met, we 
reviewed CAPE and JPO cost-estimating documentation, including data 
sources, assumptions, and calculations, and we interviewed cost-
estimating officials from these offices. In addition, to determine how 
much more frequently parts were being replaced than assumed in the JPO 
cost estimate (see table 2), we compared the mean flight hour between 
removals (MFHBR) data for the Air Force's F-35A (CTOL) and the Marine 
Corps' F-35B (STOVL) from a 12-month period ending in March 31, 2014, 
to the MFHBR assumptions used in the JPO estimate. 

We also gathered F-35 reliability and maintainability (R+M) 
data[Footnote 54] from 2013 through 2014 that had been verified 
through DOD's Joint Reliability and Maintainability Evaluation Team 
(JRMET). To determine the reliability of these data, we collected 
information on how the data were collected, managed, and used through 
a survey and interviews with relevant DOD officials. In addition to 
the data-reliability survey, we also reviewed user manual and related 
documentation to determine the limitations of the data. By assessing 
this information against GAO data-quality standards, we determined 
that the data presented in our findings were sufficiently reliable for 
presenting information about the aircraft's reliability in this report. 

We conducted this performance audit from August 2013 through September 
2014 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: GAO Scoring of 2013 F-35 Operating and Support (O&S) 
Estimates: 

We completed an overall assessment of the Joint Program Office (JPO) 
and Cost Assessment and Program Evaluation (CAPE) F-35 O&S estimates 
on the basis of two characteristics--comprehensiveness and 
credibility--and their associated best practices derived from the GAO 
Cost Estimating and Assessment Guide. After reviewing documentation 
that JPO and CAPE submitted for their 2013 F-35 O&S cost estimates, 
conducting interviews with JPO and CAPE cost-estimating officials, and 
reviewing relevant sources, we determined that these cost estimates 
are not fully reliable. While the estimates are comprehensive, not all 
of the assumptions that the estimates were based on were clearly 
documented or reasonable. In addition, the estimates are not fully 
reliable in that the JPO estimate is partially credible and the CAPE 
estimate is minimally credible. These evaluations are shown in table 3 
below. We determined the overall assessment rating by assigning each 
individual best practice rating a number: Not Met = 1, Minimally Met = 
2, Partially Met = 3, Substantially Met = 4, and Met = 5. Then, we 
took the average of the individual best practice assessment ratings to 
determine the overall rating for each of the two characteristics. The 
resulting average becomes the Overall Assessment as follows: Not Met = 
1.0 to 1.4, Minimally Met = 1.5 to 2.4, Partially Met = 2.5 to 3.4, 
Substantially Met = 3.5 to 4.4, and Met = 4.5 to 5.0. A cost estimate 
is considered reliable if the overall assessment ratings for each of 
the two characteristics are substantially or fully met. If any of the 
characteristics are not met, minimally met, or partially met, then the 
cost estimate does not fully reflect the characteristics of a high-
quality estimate and cannot be considered reliable. 

Table 3: Summary Assessment of Joint Program Office (JPO) and Cost 
Assessment and Program Evaluation (CAPE) Cost Estimates: 

Characteristic: Comprehensive; 
Overall assessment: Met (JPO); Met (CAPE); 

Best practice[A]: The cost estimate includes all operating and 
support costs; 
Individual assessment: Met (JPO); Met (CAPE). 

Best practice[A]: The cost estimate completely defines the program, 
reflects the current schedule, and is technically reasonable; 
Individual assessment: Met (JPO); Met (CAPE). 

Best practice[A]: The cost estimate work breakdown structure (WBS) is 
product-oriented, traceable to the statement of work/objective, and at 
an appropriate level of detail to ensure that cost elements are neither 
omitted nor double-counted; 
Individual assessment: Partially Met (JPO): Met (JPO); Met (CAPE). 

Best practice[A]: The estimate is based on reasonable and fully 
documented cost-influencing ground rules and assumptions; 
Individual assessment: Partially Met (JPO); Partially Met (CAPE). 

Characteristic: Credible; 
Overall assessment: Partially Met (JPO); Minimally Met (CAPE); 

Best practice[A]: The cost estimate includes a sensitivity analysis 
that identifies a range of possible costs based on varying major 
assumptions, parameters, and data inputs; 
Individual assessment: Met (JPO); Partially Met (CAPE). 

Best practice[A]: A risk and uncertainty analysis was conducted that 
quantified the imperfectly understood risks and identified the effects 
of changing key cost driver assumptions and factors; 
Individual assessment: Not met (JPO); Not met (CAPE). 

Best practice[A]: Major cost elements were cross-checked to see 
whether results were similar; 
Individual assessment: Minimally Met (JPO); Partially Met (CAPE). 

Best practice[A]: An independent cost estimate was conducted by a 
group outside the acquiring organization to determine whether other 
estimating methods produce similar results; 
Individual assessment: Met (JPO); Not applicable (CAPE)[B]. 

Source: GAO analysis of JPO and CAPE documentation and data. GAO-14-
778. 

[A] Best practices are derived from GAO, Cost Estimating and 
Assessment Guide: Best Practices for Developing and Managing Capital 
Program Costs, GAO-09-3SP (Washington, D.C.: March 2009). 

[B] We excluded the best practice of developing an independent cost 
estimate from our evaluation of CAPE's estimate. It would be 
unreasonable to expect the CAPE estimate to have its own independent 
cost estimate because the CAPE estimate was developed as an 
independent cost estimate of the JPO estimate. 

[End of table] 

[End of section] 

Appendix III: Comments from the Department of Defense: 

Assistant Secretary of Defense: 
Logistics and Materiel Readiness: 
3500 Defense Pentagon: 
Washington, DC 20301-3500: 

September 9, 2014: 

Mr. Cary Russell: 
Director, Defense Capabilities and Management: 
U.S. Government Accountability Office: 
441 G Street, N.W. 
Washington, DC 20548: 

Dear Mr. Russell: 

This is the Department of Defense (DoD) response to the GAO Draft 
Report, GAO-14-778, "F-35 Joint Strike Fighter: Need for Affordable 
Sustainment Strategy Greater Attention to Risks and Improved Cost 
Estimates," dated August 11,2014 (GAO Code 351852). 

Detailed comments on the report recommendations are enclosed. 

Sincerely, 

Signed by: 

Paul D. Peters: 
Principal Deputy: 

Enclosure: As stated. 

GAO Draft Report Dated August 11, 2014: 
GAO-14-778 (GAO CODE 351852): 

"F-35 Joint Strike Fighter: Need For Affordable Sustainment Strategy 
Greater Attention To Risks And Improved Cost-estimates" 

Department Of Defense Comments To The GAO Recommendation: 

Recommendation 1: To help the Department of Defense develop an 
affordable sustainment strategy for the F-35, we recommend that the 
Secretary of Defense direct the Under Secretary of Defense for 
Acquisitions, Technology, and Logistics to direct the F-35 Program
Executive Officer to establish affordability constraints linked to, 
and informed by, military budgets that will help guide sustainment 
decisions, prioritize requirements, and identify additional areas for 
savings by March 2015, at which point the Future Support Construct
decision will be approved. 

DoD Response: Concur. To guide development of an affordable strategy, 
the Department established F-35 affordability targets for unit 
recurring flyaway cost and for sustainment cost in March 2012. The F-
35 program is currently operating under these affordability targets 
and using affordability goals to reduce life cycle costs. 

Recommendation 2: To enable the Department of Defense to better 
identify, address, and mitigate performance issues with Autonomic 
Logistics Information System (ALIS) that could have an impact on 
affordability, as well as readiness, establish a performance measurement
process for ALIS that includes, but is not limited to, performance 
metrics and targets that (1) are based on intended behavior of the 
system in actual operations and (2) tie system performance to user 
requirements. 

DoD Response: Concur. Through the remainder of system development and 
demonstration (SDD), the F-35 program will continue to use ALIS 
performance metrics and targets to assess impacts to the operational 
requirements, such as the sortie generation rate key performance
parameter. While the F-35 program continues to assess these impacts 
today, future incremental ALIS functional deliveries and additional 
operational experience will enable more refined metrics and targets 
over time. These refined metrics and targets will help reduce 
performance risk and ensure a successful operational test and 
evaluation at the end of SDD. 

Recommendation 3: To develop a high level of confidence that the 
aircraft will achieve its reliability and maintainability goals, 
develop a software reliability and maintainability assessment process, 
with metrics, by which the program can monitor and determine the effect
that software issues may have on overall F-35 reliability and 
maintainability issues. 

DoD Response: Concur. The F-35 program has an established process for 
assessing software reliability and will continue to monitor and 
determine the effect that software issues may have on overall F-35 
reliability and maintainability. This process includes a Failure
Reporting Analysis Corrective Action System database, a Joint 
Reliability and Maintainability Evaluation Team, a Test Data Scoring 
Board, and uses the Mean-Flight-Hours-Between-Failure metric. 

Recommendation 4: To promote competition, address affordability, and 
inform its overarching sustainment strategy, develop a long-term IP 
Strategy to include, but not be limited to, the identification of: 

* Current levels of technical data rights ownership by the federal 
government. 

* All critical technical data needs and their associated costs. 

DoD Response: Concur. The F-35 acquisition strategy includes a data 
management strategy whereby the program will continue to assess the 
acquisition of technical data as specific needs arise throughout the 
life cycle. Furthermore, the program will determine the level of 
specified rights, critical technical data needs and associated costs, 
through business case analysis, contract strategy development, and 
ongoing sustainment planning. 

Recommendation 5: To improve the reliability of the JPO F-35 O&S cost 
estimate, clearly document assumptions related to intermediate-level 
maintenance costs and revise assumptions related to fuel bum rates, 
part replacement, and depot-maintenance induction in its future F-35 
O&S cost estimates to better reflect the current and future state of 
the F-35 program. 

DoD Response: Concur. The F-35 Program Executive Officer (PEO) will 
document assumptions related to intermediate-level maintenance costs 
in future annual updates to the JPO O&S cost estimate. Current 
documentation reflects an approved two-level maintenance strategy
(i.e., organizational and depot levels only, with no intermediate 
level maintenance). The F-35 PEO will also revise assumptions related 
to fuel bum rates, part replacement, and depot maintenance induction 
in future annual updates to the JPO O&S estimate. 

Recommendation 6: To understand the potential range of costs 
associated with the JPO F-35 O&S cost estimate, conduct uncertainty 
analysis on future JPO estimates. 

DoD Response: Concur. The Department will ensure that the F-35 PEO 
conducts risk and sensitivity analysis, according to DoD guidance, in 
future annual updates to the JPO F-35 O&S cost estimate. 

Recommendation 7: Clearly document assumptions related to intermediate-
level maintenance and revise fuel bum assumptions to better reflect 
the current and future state of the F-35 program. 

DoD Response: Concur. The current documentation of the CAPE O&S 
estimate reflects an approved two-level maintenance strategy (i.e., 
organizational and depot levels only, with no intermediate level 
maintenance). While the program does not pursue an I-level maintenance
strategy, the draft report correctly notes that the program does 
conduct some off-aircraft (or "0+") maintenance activities, such as 
wheels, tires, batteries, and support/test equipment, as defined in 
the F-35 Life Cycle Sustainment Plan, Annex A, Program Support Strategy.
However, the costs for these personnel are already captured in the 
estimate under Mission Personnel; these personnel are part of the 
Manpower Estimate Report (MER) provided by the Services to identify 
their squadron manning requirements. CAPE will ensure that if the
maintenance strategy is revised to include any planned intermediate-
level maintenance activities, those activities and their associated 
costs will be fully captured within the program's O&S estimate and 
associated documentation. CAPE will also update its estimate to 
reflect the latest approved technical baseline and incorporate the 
latest approved fuel consumption actuals, including fuel consumption 
information provided from the flight test program, as this information 
become available. 

Recommendation 8: Conduct uncertainty analysis to understand the 
potential range of costs associated with its estimates to reflect the 
most likely costs associated with the program. 

DoD Response: Partially concur. The Department considers the risk and 
sensitivity analyses regularly performed by CAPE to be a form of 
uncertainty analysis, as described in the 2007 OSD CAIG O&S Cost 
Estimating Guide, Section 5.3.4. The Department agrees with the value of
understanding potential O&S cost ranges and will ensure that CAPE 
continues to perform uncertainty analyses in accordance with DoD 
guidance. 

[End of section] 

Appendix IV: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Cary Russell, (202) 512-5431 or russellc@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, the following staff members 
made key contributions to this report: Alissa Czyz, Assistant 
Director; Jennifer Echard; Jeffrey Hubbard; Abishek Krupanand; Jason 
Lee; Jennie Leotta; Katie Mauldin; Carol Petersen; Amie Steele; Alyssa 
Weir; Cheryl Weissman; and Amanda Weldon. 

[End of section] 

Related GAO Products: 

Weapon Systems Management: DOD Has Taken Steps to Implement Product 
Support Managers but Needs to Evaluate Their Effects. [hyperlink, 
http://www.gao.gov/products/GAO-14-326]. Washington, D.C.: April 29, 
2014. 

Defense Acquisitions: Assessments of Selected Weapon Programs. 
[hyperlink, http://www.gao.gov/products/GAO-14-340SP]. Washington, 
D.C.: March 31, 2014. 

F-35 Joint Strike Fighter: Slower Than Expected Progress in Software 
Testing May Limit Initial Warfighting Capabilities. [hyperlink, 
http://www.gao.gov/products/GAO-14-468T]. Washington, D.C.: March 26, 
2014. 

F-35 Joint Strike Fighter: Problems Completing Software Testing May 
Hinder Delivery of Expected Warfighting Capabilities. [hyperlink, 
http://www.gao.gov/products/GAO-14-322]. Washington, D.C.: March 24, 
2014. 

F-35 Joint Strike Fighter: Restructuring Has Improved the Program, but 
Affordability Challenges and Other Risks Remain. [hyperlink, 
http://www.gao.gov/products/GAO-13-690T]. Washington, D.C.: June 19, 
2013. 

F-35 Joint Strike Fighter: Program Has Improved in Some Areas, but 
Affordability Challenges and Other Risks Remain. [hyperlink, 
http://www.gao.gov/products/GAO-13-500T]. Washington, D.C.: April 17, 
2013. 

Defense Acquisitions: Assessments of Selected Weapon Programs. 
[hyperlink, http://www.gao.gov/products/GAO-13-294SP]. Washington, 
D.C.: March 28, 2013. 

F-35 Joint Strike Fighter: Current Outlook Is Improved, but Long-Term 
Affordability Is a Major Concern. [hyperlink, 
http://www.gao.gov/products/GAO-13-309. Washington, D.C.: March 11, 
2013. 

Fighter Aircraft: Better Cost Estimates Needed for Extending the 
Service Life of Selected F-16s and F/A-18s. [hyperlink, 
http://www.gao.gov/products/GAO-13-51]. Washington, D.C.: November 15, 
2012. 

Joint Strike Fighter: DOD Actions Needed to Further Enhance 
Restructuring and Address Affordability Risks. [hyperlink, 
http://www.gao.gov/products/GAO-12-437]. Washington, D.C.: June 14, 
2012. 

Tactical Aircraft: F-22A Modernization Program Faces Cost, Technical, 
and Sustainment Risks. [hyperlink, 
http://www.gao.gov/products/GAO-12-447]. Washington, D.C.: May 2, 2012. 

Joint Strike Fighter: Restructuring Added Resources and Reduced Risk, 
but Concurrency Is Still a Major Concern. [hyperlink, 
http://www.gao.gov/products/GAO-12-525T]. Washington, D.C.: March 20, 
2012. 

Defense Logistics: Improvements Needed to Enhance Oversight of 
Estimated Long-term Costs for Operating and Supporting Major Weapon 
Systems. [hyperlink, http://www.gao.gov/products/GAO-12-340]. 
Washington, D.C.: February 2, 2012. 

Joint Strike Fighter: Implications of Program Restructuring and Other 
Recent Developments on Key Aspects of DOD's Prior Alternate Engine 
Analyses. [hyperlink, http://www.gao.gov/products/GAO-11-903R]. 
Washington, D.C.: September 14, 2011. 

Joint Strike Fighter: Restructuring Places Program on Firmer Footing, 
but Progress Is Still Lagging. [hyperlink, 
http://www.gao.gov/products/GAO-11-677T]. Washington, D.C.: May 19, 
2011. 

Defense Acquisition: DOD Should Clarify Requirements for Assessing and 
Documenting Technical-Data Needs. [hyperlink, 
http://www.gao.gov/products/GAO-11-469]. Washington, D.C.: May 11, 
2011. 

Joint Strike Fighter: Restructuring Places Program on Firmer Footing, 
but Progress Still Lags. [hyperlink, 
http://www.gao.gov/products/GAO-11-325]. Washington, D.C.: April 7, 
2011. 

Joint Strike Fighter: Restructuring Should Improve Outcomes, but 
Progress Is Still Lagging Overall. [hyperlink, 
http://www.gao.gov/products/GAO-11-450T]. Washington, D.C.: March 15, 
2011. 

Defense Management: DOD Needs to Monitor and Assess Corrective Actions 
Resulting from Its Corrosion Study of the F-35 Joint Strike Fighter. 
[hyperlink, http://www.gao.gov/products/GAO-11-171R]. Washington, 
D.C.: December 16, 2010. 

Tactical Aircraft: DOD's Ability to Meet Future Requirements Is 
Uncertain, with Key Analyses Needed to Inform Upcoming Investment 
Decisions. [hyperlink, http://www.gao.gov/products/GAO-10-789]. 
Washington, D.C.: July 29, 2010. 

Joint Strike Fighter: Significant Challenges and Decisions Ahead. 
[hyperlink, http://www.gao.gov/products/GAO-10-478T]. Washington, 
D.C.: March 24, 2010. 

Joint Strike Fighter: Additional Costs and Delays Risk Not Meeting 
Warfighter Requirements on Time. [hyperlink, 
http://www.gao.gov/products/GAO-10-382]. Washington, D.C.: March 19, 
2010. 

Joint Strike Fighter: Significant Challenges Remain as DOD 
Restructures Program. [hyperlink, 
http://www.gao.gov/products/GAO-10-520T]. Washington, D.C.: March 11, 
2010. 

Joint Strike Fighter: Strong Risk Management Essential as Program 
Enters Most Challenging Phase. [hyperlink, 
http://www.gao.gov/products/GAO-09-711T]. Washington, D.C.: May 20, 
2009. 

[End of section] 

Footnotes: 

[1] GAO, Joint Strike Fighter: DOD Actions Needed to Further Enhance 
Restructuring and Address Affordability Risks, [hyperlink, 
http://www.gao.gov/products/GAO-12-437] (Washington, D.C.: June 14, 
2012) and F-35 Joint Strike Fighter: Current Outlook Is Improved, but 
Long-Term Affordability Is a Major Concern, [hyperlink, 
http://www.gao.gov/products/GAO-13-309] (Washington, D.C.: Mar. 11, 
2013). 

[2] GAO, F-35 Joint Strike Fighter: Problems Completing Software 
Testing May Hinder Delivery of Expected Warfighting Capabilities, 
[hyperlink, http://www.gao.gov/products/GAO-14-322] (Washington, D.C.: 
Mar. 24, 2014). 

[3] H.R. Rep. No. 113-102, at 118 (2013). 

[4] Reliability and maintainability data measure aircraft performance 
to determine how often the aircraft experiences failures and how much 
time it takes to repair those failures. 

[5] GAO, Cost Estimating and Assessment Guide: Best Practices for 
Developing and Managing Capital Program Costs, [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009). 

[6] The international partners are the United Kingdom, Italy, the 
Netherlands, Turkey, Canada, Australia, Denmark, and Norway. These 
nations contributed funds for system development and signed agreements 
to procure aircraft. In addition, Israel and Japan have signed on as 
foreign military sales customers. 

[7] Section 2433 of title 10 of the United States Code, commonly 
referred to as Nunn-McCurdy, requires DOD to notify Congress whenever 
a major defense acquisition program's unit cost experiences cost 
growth that exceeds certain thresholds. This is commonly referred to 
as a Nunn-McCurdy breach. Significant breaches occur when the program 
acquisition unit cost or procurement unit cost increases by at least 
15 percent over the current baseline estimate or at least 30 percent 
over the original estimate. For critical breaches, as was the case 
with the F-35 in 2010, when these unit costs increase at least 25 
percent over the current baseline estimate or at least 50 percent over 
the original, DOD is required to take additional steps, including 
conducting an in-depth review of the program. Programs with critical 
breaches must be terminated unless the Secretary of Defense certifies 
to certain facts related to the program and takes other actions, 
including restructuring the program. 10 U.S.C. § 2433a. 

[8] U.S. Navy, U.S. Marine Corps, U.S. Air Force, Report to 
Congressional Defense Committees: F-35 Initial Operational Capability 
(June 2013). 

[9] IOC is generally obtained when organizations or units scheduled to 
receive a system have received it and have the ability to employ and 
maintain that system. 

[10] [hyperlink, http://www.gao.gov/products/GAO-14-322]. 

[11] Department of Defense, Office of the Under Secretary of Defense 
for Acquisitions, Technology and Logistics, Operation of the Defense 
Acquisition System, (Interim) Instruction 5000.02 (Washington, D.C.: 
Nov. 26, 2013). 

[12] Sorties, or flights, are generated in support of testing and 
operations. Department of Defense, Joint Strike Fighter, Operational 
Requirements Document (Washington, D.C.: Aug.19, 2008). 

[13] The Program Executive Officer position alternates between the 
Departments of the Navy and the Air Force, and reports to the Service 
Acquisition Executive of the other service. The Program Executive 
Officer is currently an Air Force Lieutenant General. 

[14] Fiscal 2015 Defense Authorization: Tactical Aircraft Programs, 
113th Cong. (April 2014) (statement of Lt. Gen. Christopher C. Bogdan, 
Program Executive Officer, F-35 Lightning II Joint Program Office). 

[15] DOD Interim Instruction 5000.02. 

[16] DOD Interim Instruction 5000.02. See also Defense Acquisition 
University, Defense Acquisition Guidebook (Washington, D.C.: May 15, 
2013). The guidebook is designed to complement policy documents such 
as DOD Interim Instruction 5000.02 by providing the acquisition 
workforce with discretionary best practices that should be tailored to 
the needs of each program. 

[17] Each aircraft has a 30-year service life, but the 2013 estimates 
span 56 years. 

[18] PBL is performance-based life-cycle product support whereby 
outcomes are acquired through performance-based arrangements that 
deliver warfighter requirements and incentivize product support 
providers to reduce costs through innovation. 

[19] See Defense Federal Acquisition Regulation Supplement (DFARS) 
clause 252.227-7013 for a full definition of "technical data." 

[20] For the purposes of this report we define steady-state operations 
as the period from 2036 to 2040, when, according to the military 
services' plans, the number of F-35 aircraft and flying hours reaches 
its highest point and plateaus. 

[21] Fiscal 2015 Defense Authorization: Tactical Aircraft Programs 
(statement of Lt. Gen. Christopher C. Bogdan). 

[22] Affordability goals are established at Milestone A, the entry 
into Technology Development. After systems engineering trade-offs are 
completed during the Technology Development phase, these affordability 
goals then become affordability caps at Milestone B, the start of 
System Development, when a match is to be made between requirements 
and resources. We refer to the goals and caps collectively as 
affordability constraints. 

[23] DOD Interim Instruction 5000.02. Prior to being incorporated into 
the interim instruction, affordability constraints were a part of the 
Better Buying Power initiative. 

[24] DOD officials, including the F-35 product support manager, have 
stated that linking affordability to actual service budgets will 
constitute a focal point of the program, moving forward. However, to 
date, we have not seen any documentation or evidence indicating that 
this effort is in progress. 

[25] A key performance parameter (KPP) is a performance attribute of a 
system considered critical to the development of an effective military 
capability. 

[26] The System Engineering Guide for Systems of Systems provides 
practitioners with practical guidance and is intended as reference 
only. 

[27] Department of Defense, Risk Management Guide for Acquisition 
(Washington, D.C.: August 2006). 

[28] Each R+M metric must meet its specific requirement at maturity--
200,000 flight hours for the fleet, with at least 50,000 per variant. 

[29] Design-controllable failures are those that can be attributed to 
deficiencies in design, but considered by the JPO to be fixable by 
design modification. 

[30] [hyperlink, http://www.gao.gov/products/GAO-14-322]. 

[31] The loss of a mission-essential function also downs the aircraft 
for that mission. 

[32] Department of Defense, Handbook: Reliability Growth Management 
(Washington, D.C.: Sept. 10, 2009). This handbook is for guidance only 
and is not to be cited as a requirement. 

[33] Naval Air Systems Command, Software Logistics Primer (Washington, 
D.C.: April 2010). This primer is for training purposes only. 

[34] DOD, Risk Management Guide for Acquisition. This guidance is not 
mandatory to follow, but program managers are encouraged to apply the 
fundamentals of the guide. 

[35] GAO, Defense Acquisition: DOD Should Clarify Requirements for 
Assessing and Documenting Technical-Data Needs, [hyperlink, 
http://www.gao.gov/products/GAO-11-469] (Washington, D.C.: May 11, 
2011). 

[36] Defense Federal Acquisition Regulation Supplement (DFARS) clause 
252.227-7013 defines technical data as "recorded information, 
regardless of the form of method of the recording, of a scientific or 
technical nature (including computer software documentation) … [but 
not including] computer software or data incidental to contract 
administration, such as financial and/or management information." 
Technical data for weapon systems include drawings, specifications, 
standards, and other details necessary to ensure the adequacy of item 
performance, as well as manuals that contain instructions for 
installation, operation, maintenance, and other actions needed to 
support weapon systems. 

[37] GAO, Federal Contracting: Opportunities Exist to Increase 
Competition and Assess Reasons When Only One Offer Is Received, 
[hyperlink, http://www.gao.gov/products/GAO-10-833] (Washington, D.C.: 
July 26, 2010). 

[38] Under Secretary of Defense for Acquisition, Technology and 
Logistics, "Better Buying Power: Mandate for Restoring Affordability 
and Productivity in Defense Spending," memorandum (June 28, 2010); 
"Better Buying Power: Guidance for Obtaining Greater Efficiency and 
Productivity in Defense Spending," memorandum (Sept. 14, 2010); 
"Implementation Directive for Better Buying Power--Obtaining Greater 
Efficiency and Productivity in Defense Spending," memorandum (Nov. 3, 
2010); and "Better Buying Power 2.0: Continuing the Pursuit for 
Greater Efficiency and Productivity in Defense Spending," memorandum 
(Nov. 13, 2012). 

[39] As part of DOD's acquisition process, Milestone B initiates the 
Engineering and Manufacturing Development phase and is normally the 
formal initiation of an acquisition program. DOD Interim Instruction 
5000.02. Milestone B for the F-35 was reached in 2001, and recertified 
in 2012. 

[40] DOD Interim Instruction 5000.02. 

[41] DOD, Defense Federal Acquisition Regulation Supplement (DFARS) 
Subpart 207.106 (S-70). The DFARS provides DOD implementation and 
supplementation of the Federal Acquisition Regulation (FAR). The DFARS 
contains requirements of law, DOD-wide policies, delegations of FAR 
authorities, deviations from FAR requirements, and policies and 
procedures that have a significant effect beyond the internal 
operating procedures of DOD or a significant cost or administrative 
effect on contractors. DFARS § 201.301. 

[42] Acquisition plans are generally developed before issuance of a 
solicitation, FAR § 7.104 and DFARS § 207.106. 

[43] A table scoring the cost estimates based on these characteristics 
and their corresponding best practices can be found in appendix II. 

[44] 10 U.S.C. § 2334. For the F-35, DOD decided to use the CAPE 
estimate for the 2012 and 2013 SARs. However, both estimates are used 
in DOD decision making, as noted above. Consequently, we evaluated 
both the JPO and the CAPE estimates. Total O&S costs are listed in 
then-year dollars, which are adjusted to show the effects of inflation 
over the life cycle of the program. 

[45] 10 U.S.C. § 2432. A full life-cycle cost analysis, including O&S 
costs, is required annually in the program's Selected Acquisition 
Report (SAR). Generally, the program office estimate is used as the 
estimate of record and published in the SAR, and, as a result, the JPO 
estimate is developed annually. 

[46] Office of the Secretary of Defense, Cost Analysis Improvement 
Group, Operating and Support Cost Estimating Guide (Washington, D.C.: 
October 2007) and [hyperlink, http://www.gao.gov/products/GAO-09-3SP]. 
The Cost Analysis and Improvement Group is now known as Cost 
Assessment and Program Evaluation (CAPE). 

[47] [hyperlink, http://www.gao.gov/products/GAO-09-3SP]. We are 
defining a reasonable assumption as one that is logical, credible, and 
acceptable given the available data and information. 

[48] The JPO increased the cost of replacement parts from 2010 to 2019 
of the estimate by using a reliability growth curve. This reliability 
growth curve roughly doubles the replacement part cost per flight hour 
at maturity and then decreases the cost on an annual basis, assuming 
that parts will be replaced less frequently until the aircraft reaches 
the contractor-predicted replacement rate at maturity in 2020. 

[49] According to Institute for Defense Analysis officials, this rate 
of improvement is not impossible, but has only been observed in 
dissimilar aircraft like the C-17. 

[50] GAO, Defense Management: DOD Needs Better Information and 
Guidance to More Effectively Manage and Reduce Operating and Support 
Costs of Major Weapon Systems, [hyperlink, 
http://www.gao.gov/products/GAO-10-717] (Washington, D.C.: July 20, 
2010). 

[51] This average price per depot hour includes labor, materiel, and 
overhead costs based on data from three air logistics centers--the 
Oklahoma City Air Logistics Center, Tinker Air Force Base, Oklahoma; 
the Ogden Air Logistics Center, Hill Air Force Base, Utah; and the 
Warner Robins Air Logistics Center, Robins Air Force Base, Georgia. 

[52] Fiscal 2015 Defense Authorization: Tactical Aircraft Programs 
(statement of Lt. Gen. Christopher C. Bogdan). 

[53] GAO, Cost Estimating and Assessment Guide: Best Practices for 
Developing and Managing Capital Program Costs, [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009). 

[54] R+M data measure aircraft performance to determine how often the 
aircraft experiences failures and how much time it takes to repair 
those failures. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO's commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
"E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO's actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO's 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548. 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548. 

[End of document]