This is the accessible text file for GAO report number GAO-10-717
entitled 'Defense Management: DOD Needs Better Information and
Guidance to More Effectively Manage and Reduce Operating and Support
Costs of Major Weapon Systems' which was released on July 20, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
July 2010:
Defense Management:
DOD Needs Better Information and Guidance to More Effectively Manage
and Reduce Operating and Support Costs of Major Weapon Systems:
GAO-10-717:
GAO Highlights:
Highlights of GAO-10-717, a report to congressional committees.
Why GAO Did This Study:
The Department of Defense (DOD) spends billions of dollars each year
to sustain its weapon systems. These operating and support (O&S) costs
can account for a significant portion of a system’s total life-cycle
costs and include costs for repair parts, maintenance, and personnel.
The Weapon Systems Acquisition Reform Act of 2009 directs GAO to
review the growth in O&S costs of major systems. GAO’s report
addresses (1) the extent to which life-cycle O&S cost estimates
developed during acquisition and actual O&S costs are available for
program management and decision making; (2) the extent to which DOD
uses life-cycle O&S cost estimates after systems are fielded to
quantify cost growth and identify its causes; and (3) the efforts
taken by DOD to reduce O&S costs for major systems. GAO selected seven
aviation systems that reflected varied characteristics and have been
fielded at least several years. These systems were the F/A-18E/F, F-
22A, B-1B, F-15E, AH-64D, CH-47D, and UH-60L.
What GAO Found:
DOD lacks key information needed to effectively manage and reduce O&S
costs for most of the weapon systems GAO reviewed—including life-cycle
O&S cost estimates and complete historical data on actual O&S costs.
The services did not have life-cycle O&S cost estimates developed at
the production milestone for five of the seven aviation systems GAO
reviewed, and current DOD acquisition and cost-estimating guidance
does not specifically address retaining these estimates. Also, the
services’ information systems designated for collecting data on actual
O&S costs were incomplete, with the Army’s system having the greatest
limitations on available cost data. without historic cost estimates
and complete data on actual O&S costs, DOD officials do not have
important information necessary for analyzing the rate of O&S cost
growth for major systems, identifying cost drivers, and developing
plans for managing and controlling these costs. At a time when the
nation faces fiscal challenges, and defense budgets may become
tighter, the lack of this key information hinders sound weapon system
program management and decision making in an area of high costs to the
federal government
DOD generally does not use updated life-cycle O&S cost estimates to
quantify cost growth and identify cost drivers for the systems GAO
reviewed. The services did not periodically update life-cycle O&S cost
estimates after production was completed for six of the seven systems.
The F-22A program office had developed an updated life-cycle O&S cost
estimate in 2009 and found a 47-percent ($19 billion) increase in life-
cycle O&S costs from what had been previously estimated in 2005. GAO’s
comparisons of estimated to actual O&S costs for two of the seven
systems found some areas of cost growth. However, notable changes such
as decreases in the numbers of aircraft and flying hours occurred in
both programs after the production milestone estimates were developed,
complicating direct comparisons of estimated to actual costs.
According to federal guidance, agencies should have a plan to
periodically evaluate program results as these may be used to
determine whether corrections need to be made and to improve future
cost estimates. However, DOD acquisition and cost estimation guidance
does not require that O&S cost estimates be updated throughout a
system’s life cycle or that information on program changes affecting
the system’s life-cycle O&S costs be retained. The services’
acquisition and cost-estimation guidance that GAO reviewed does not
consistently and clearly require the updating of O&S cost estimates
after a program has ended production.
DOD has several departmentwide and service-specific initiatives to
address O&S costs of major systems. One DOD program funds projects
aimed at improving reliability and reducing O&S costs for existing
systems. Other initiatives are aimed at focusing attention on O&S cost
requirements and reliability during the acquisition process. In a
recent assessment, DOD identified weaknesses in O&S cost management,
found deficiencies in sustainment governance, and recommended a number
of corrective actions. Many of DOD’s initiatives are recent or are not
yet implemented.
What GAO Recommends:
GAO recommends that DOD take steps to retain life-cycle O&S cost
estimates for major systems, collect additional O&S cost elements in
its visibility systems, update life-cycle O&S cost estimates
periodically after systems are fielded, and retain documentation of
program changes affecting O&S costs for use in cost analysis. DOD
concurred with GAO’s recommendations.
View [hyperlink, http://www.gao.gov/products/GAO-10-717] or key
components. For more information, contact Jack E. Edwards at (202) 512-
8246 or edwardsj@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Better Information and Guidance Could Help DOD to More Effectively
Manage and Reduce O&S Costs of Major Weapons Systems:
The Services Generally Do Not Use Updated Life-Cycle Estimates to
Assess O&S Cost Growth for Fielded Weapon Systems:
DOD Has Departmentwide and Service-Specific Initiatives to Address
Weapon System O&S Costs:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Results of OSD-Sponsored Analysis of C-17 Aircraft:
Appendix III: Analysis of Changes between Estimated and Actual O&S
Costs for the Navy's F/A-18E/F:
Appendix IV: Comments from the Department of Defense:
Appendix V: GAO Contact and Staff Acknowledgments:
Related GAO Products:
Tables:
Table 1: DOD's Current and Former Structure for Preparing and
Presenting O&S Cost Estimates:
Table 2: Changes in Army Helicopter Inventory, Flight Hours, and Unit-
Level Consumption Costs between Fiscal Year 1998 and Fiscal Year 2007:
Table 3: Percent Change in Army Helicopter Unit-Level Consumption
Costs per Flight Hour between Fiscal Year 1998 and Fiscal Year 2007:
Table 4: Organizations Contacted to Obtain Information on Major Weapon
System Operating and Support Costs:
Table 5: Comparison of Navy F/A-18E/F Total Estimated and Actual O&S
Costs for Fiscal Years 1999-2009:
Figures:
Figure 1: Comparison of Estimated and Actual O&S Costs for the Air
Force's F-22A (Fiscal Years 2005-2009):
Figure 2: Comparison of Estimated and Actual O&S Costs for the Navy's
F/A-18E/F (Fiscal Years 1999-2009):
Figure 3: Actual F-15E O&S Costs for Fiscal Years 1996-2009:
Figure 4: Actual B-1B O&S Costs for Fiscal Years 1999-2009:
Abbreviations:
DOD: Department of Defense:
IDA: Institute for Defense Analyses:
O&S: operating and support:
OSD: Office of the Secretary of Defense:
VAMOSC: Visibility and Management of Operating and Support Costs:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
July 20, 2010:
Congressional Committees:
The Department of Defense (DOD) spends billions of dollars each year
to sustain its weapon systems. These operating and support (O&S) costs
can account for a significant portion of a weapon system's total life-
cycle costs and include costs for, among other things, repair parts,
maintenance, and contract services, as well as the personnel
associated with a weapon system.[Footnote 1] Weapon systems are costly
to sustain in part because they often incorporate a technologically
complex array of subsystems and components and need expensive spare
parts and logistics support to meet required readiness levels. In
addition, the high tempo of military operations in Iraq and
Afghanistan has increased wear and tear on many weapon systems,
escalating their O&S costs well beyond peacetime levels. Many of the
key decisions affecting a weapon system's O&S costs are made while the
system is still in the acquisition process. For example, acquisition-
based decisions about the design, materials, and technology for the
system affect the logistics support that is eventually needed to keep
the system available and ready after it is fielded. Controlling O&S
costs is critical to ensure future affordability of defense budgets.
Life-cycle O&S costs for the F-35 Joint Strike Fighter, the newest
aircraft being acquired for the Air Force, Navy, and Marines, are now
estimated at about $916 billion and its operating costs per hour are
expected to exceed the legacy aircraft it is replacing.
To improve DOD's organization and procedures for acquiring weapon
systems, Congress passed the Weapon Systems Acquisition Reform Act of
2009 ("the Act").[Footnote 2] The Act contains provisions regarding
improving the accuracy of cost estimates for major defense acquisition
programs.[Footnote 3] The Act also directs GAO to review the growth in
O&S costs of major weapon systems and to provide the results of this
review not later than 1 year after the date the legislation is
enacted.[Footnote 4] We provided the preliminary results of our review
to the congressional defense committees in May 2010. This report
provides the final results of our review and addresses (1) the extent
to which life-cycle O&S cost estimates developed during acquisition
and actual O&S costs are available for major weapon system program
management and decision making; (2) the extent to which DOD uses life-
cycle O&S cost estimates for major weapon systems after they are
fielded to quantify cost growth and identify its causes; and (3) the
efforts taken by DOD to reduce O&S costs for major weapon systems.
The Act also requires us to review weaknesses in operations affecting
the reliability of financial information on the systems and assets to
be acquired under major defense acquisition programs. We are issuing a
separate report in response to that provision of the Act. Furthermore,
the Act directs the DOD Director of Cost Assessment and Program
Evaluation,[Footnote 5] within the Office of the Secretary of Defense
(OSD), to review existing DOD systems and methods for tracking and
assessing O&S costs on major defense acquisition programs. In
addition, we have conducted prior work on weapon system O&S cost
issues and made recommendations for better managing and controlling
costs. The Related GAO Products section at the end of this report
provides additional information on our prior work.
To conduct this review, we collected and analyzed data on seven major
aviation weapon systems: the Navy's F/A-18E/F; the Air Force's F-22A,
B-1B, and F-15E; and the Army's AH-64D, CH-47D, and UH-60L. We focused
on aviation systems to allow us to compare O&S cost growth, where
possible, across the selected systems. We selected systems that had
reached initial operating capability after 1980, incurred several
years of actual O&S costs, and had relatively large fleets. We also
selected the systems to reflect varied characteristics in terms of
military service,[Footnote 6] mission, support strategy, and aircraft
age. We did not select weapon systems with known limitations of
available cost data.[Footnote 7] The results from this nonprobability
sample cannot be used to make inferences about all aviation systems or
major weapon systems because the sample may not reflect all
characteristics of the population. We obtained life-cycle O&S cost
estimates, where available, that the services had developed to support
the decision to proceed with production of the aircraft.[Footnote 8]
We obtained data on actual O&S costs from the services' Visibility and
Management of Operating and Support Costs (VAMOSC) systems, which are
information systems designated by DOD as the authoritative sources to
provide visibility over these historical data. In our analysis of O&S
costs, we have adjusted DOD data to reflect constant fiscal year 2010
dollars, unless otherwise noted. We also interviewed and obtained
pertinent documents, including directives, instructions, regulations,
memorandums, and manuals, from acquisition, program management, cost
analysis, and logistics officials to understand how O&S costs were
estimated, tracked, managed, and controlled in accordance with
applicable laws and policies; and to identify factors that have led to
cost growth. Our scope and methodology is discussed further in
appendix I.
We conducted this performance audit from June 2009 to July 2010 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives. However, O&S
cost data for individual weapon systems have certain data reliability
limitations and are not consistent across all major systems. We
discuss some of these data limitations in our report.
Results in Brief:
DOD lacks key information needed to effectively manage and reduce O&S
costs for most of the weapon systems we reviewed--including life-cycle
O&S cost estimates and complete historical data on actual O&S costs.
The services did not have life-cycle O&S cost estimates developed at
the production milestone for five of the seven aviation systems we
reviewed, and the services did not have complete data on actual O&S
costs in their VAMOSC systems, with the Army system having the
greatest limitations on its available cost data. Without historical
life-cycle O&S cost estimates and complete data on actual O&S costs,
DOD officials do not have important information necessary for
analyzing the rate of O&S cost growth for major weapon systems,
identifying cost drivers, and developing plans for managing and
controlling these costs. In addition, at a time when the nation faces
fiscal challenges and defense budgets may become tighter, the lack of
this key information hinders sound weapon-system program management
and decision making in an area of high costs to the federal
government. DOD officials we interviewed noted that the department has
not placed emphasis on assessing and managing weapon system O&S costs
compared with other priorities. More specifically, our review showed
the following:
* Life-cycle O&S cost estimates developed during weapon system
acquisition to support production decisions were available for two
aviation systems--the Air Force's F-22A and the Navy's F/A-18E/F.
Neither the services nor OSD could produce the production milestone
O&S cost estimates for the other five systems. Under cost-estimating
best practices, the thorough documentation of cost estimates and the
retention of these estimates are essential in order to analyze changes
that can aid preparation of future cost estimates. However, current
DOD acquisition and cost-estimating guidance does not specifically
address retaining life-cycle O&S cost estimates and the support
documentation used to develop these estimates.
* The services' VAMOSC systems do not collect complete data on O&S
costs, and some cost data in these systems are inaccurate. While DOD
guidance recommends that the services collect actual data on O&S costs
using the same cost element structure used to develop life-cycle O&S
cost estimates, the guidance does not require use of this structure in
collecting and presenting these data. Our review showed that the Air
Force's and Navy's VAMOSC systems do not collect actual cost data for
several cost elements, and the Army's system collects actual data on
unit-level consumption costs (such as fuel, spare parts, and training
munitions) but does not consistently capture data for other major O&S
cost elements. Although we recommended in 2000 that the Army improve
its VAMOSC system, the Army has not made significant improvements.
According to Army officials, additional cost data in existing
information systems do not feed into the VAMOSC system.
DOD generally does not use updated life-cycle O&S cost estimates to
quantify cost growth and identify cost drivers for the weapon systems
we reviewed. Even though most of these weapon systems have been
fielded for at least a decade, the services did not periodically
update life-cycle O&S cost estimates for six of the seven systems.
According to Office of Management and Budget guidance on benefit-cost
analysis, agencies should have a plan for periodically evaluating
program results as these evaluations may be used to determine whether
corrections need to be made and to improve future cost estimates.
Further, cost-estimating best practices call for regular updating of
estimates to determine whether the preliminary information and
assumptions remain relevant and accurate, record reasons for
variances, and archive cost and technical data for use in future
estimates. However, for six of the seven aviation weapon systems we
reviewed, service officials could not provide current life-cycle O&S
cost estimates that updated previous estimates by, for example,
incorporating actual historical costs or comparing actual costs to the
prior estimates. In contrast, the F-22A program office had updated the
production milestone cost estimate, found a 47 percent ($19 billion)
increase in life-cycle O&S costs from what had been previously
estimated, and identified cost drivers for this increase. Our
comparisons of estimated to actual O&S costs for the two aviation
systems where data were available--the F-22A and the F/A-18E/F--
indicated there were some areas of cost growth. However, notable
program changes that occurred after the production milestone estimates
were developed, such as decreases in the total numbers of aircraft and
flying hours, complicated direct comparisons between estimated and
actual costs. Our analysis of actual O&S costs for the other five
weapon systems showed that costs had increased over time, but the
extent to which this cost growth was planned is uncertain because the
production milestone O&S cost estimates were not available. Despite
the Office of Management and Budget guidance and cost-estimating best
practices, current DOD acquisition and cost estimation guidance does
not require that O&S cost estimates be updated after a program has
been fielded, and service guidance that we reviewed is not clear and
consistent regarding the updating of O&S cost estimates. In addition,
although weapon system programs may experience significant changes
over time that affect the assumptions used to develop earlier life-
cycle O&S cost estimates, the DOD and service guidance that we
reviewed does not explicitly require the services to maintain
documentation of program changes for use in future cost analysis.
Without assessments aimed at quantifying and analyzing changes in a
weapon system's life-cycle O&S costs after it is fielded, the services
may miss opportunities to take corrective actions aimed at reducing
these costs and to improve future cost estimating.
DOD has several departmentwide and service initiatives to address O&S
costs of major weapon systems. DOD, for example, has a $25 million
program that funds specific projects aimed at improving reliability
and reducing O&S cost for existing systems. Other departmentwide
initiatives are aimed at focusing attention on O&S cost requirements
during the acquisition process, improving system reliability during
acquisition, and improving cost estimates and the availability of
contractor cost data. Some of these initiatives address factors we
previously identified as negatively affecting DOD's ability to manage
O&S costs. For example, DOD has introduced a total ownership cost
performance requirement for new weapon systems. The services also
identified initiatives to help them better manage O&S costs for
aviation systems. Although one Army command had an O&S cost-reduction
program, none of the services had cost-reduction programs implemented
servicewide. Moreover, in a recent departmentwide assessment, DOD
identified problems with weapon system product support. In addition to
identifying weaknesses in O&S cost management, DOD's study team also
found deficiencies in sustainment governance and recommended a number
of corrective actions. Many of DOD's initiatives are recent or have
not yet been implemented.
We are recommending that DOD take steps to retain life-cycle O&S cost
estimates for major weapon systems, collect additional O&S cost
elements in their VAMOSC systems, update life-cycle O&S cost estimates
periodically after systems are fielded, and retain documentation of
program changes affecting O&S costs for use in cost analysis. We are
also recommending that the Army develop a strategy for improving its
VAMOSC system. DOD, in its comments on a draft of this report,
concurred with our recommendations.
Background:
O&S Costs Constitute a Significant Portion of a System's Life-Cycle
Costs:
A system's life-cycle costs include the costs for research and
development, procurement, sustainment, and disposal. O&S costs include
the direct and indirect costs of sustaining a fielded system, such as
costs for spare parts, fuel, maintenance, personnel, support
facilities, and training equipment. According to DOD, the O&S costs
incurred after a system has been acquired account for at least 70
percent of a system's life-cycle costs and depend on how long a system
remains in the inventory.[Footnote 9] Many of the key decisions
affecting O&S costs are made during the acquisition process, and a
weapon system's O&S costs depend to a great extent on its expected
readiness level and overall reliability. In general, readiness can be
achieved either by building a highly reliable weapon system or
supporting it with a more extensive logistics system that can ensure
spare parts and other support are available when needed. If a weapon
system has a very high expected readiness rate but its design is not
reliable, O&S costs may be high and more difficult to predict.
Conversely, if the weapon system design has been thoroughly tested for
reliability and is robust, O&S costs may be more predictable.
O&S Costs Are Required to Be Estimated at Various Points during the
Acquisition Process:
DOD's acquisition process includes a series of decision milestones as
the systems enter different stages of development and production. As
part of the process,[Footnote 10] the DOD component or joint program
office responsible for the acquisition program is required to prepare
life-cycle cost estimates, which include O&S costs, to support these
decision milestones and other reviews. Under the current acquisition
process,[Footnote 11] decision makers at milestone A determine whether
to approve a program to enter into technology development. Although
very little may be known about the system design, performance,
physical characteristics, or operational and support concepts, DOD
guidance states that rough O&S cost estimates are expected to
primarily support plans that guide refinement of the weapon system
concept.[Footnote 12] At milestone B, a decision is made whether to
approve the program to enter into engineering and manufacturing
development. At this point, according to the guidance, O&S cost
estimates and comparisons should show increased fidelity, consistent
with more fully developed design and support concepts. At this stage,
O&S costs are important because the long-term affordability of the
program is assessed, program alternatives are compared, and O&S cost
objectives are established. The program must pass through milestone C
before entering production and deployment. DOD's guidance states that
at milestone C and at the full-rate production decision review, O&S
cost estimates should be updated and refined, based on the system's
current design characteristics, the latest deployment schedule, and
current logistics and training support plans. Further, the guidance
states that O&S experience obtained from system test and evaluation
should be used to verify progress in meeting supportability goals or
to identify problem areas. Lastly, O&S cost objectives should be
validated, and any O&S-associated funding issues should be resolved,
according to the guidance.
OSD's Cost Assessment and Program Evaluation[Footnote 13] office has
established guidance regarding life-cycle O&S cost estimates that are
developed at acquisition milestone reviews and has defined standards
for preparing and presenting these estimates.[Footnote 14] Current
guidance issued in October 2007 identifies O&S cost elements and
groups them into several major areas.[Footnote 15] This 2007 guidance--
which went into effect after the systems selected for our review
passed through the production milestone--updated and refined the
guidance issued in May 1992. The cost element structure in the 2007
guidance is similar to that of the 1992 guidance, with some key
differences. For example, separate cost elements for intermediate-
level and depot-level maintenance under the 1992 structure were
combined into one maintenance cost element area in 2007. Cost elements
for continuing system improvements were included under sustaining
support in 1992 but separately identified in the 2007 structure. Also,
cost elements for contractor support are no longer separately
identified as a single cost area in the 2007 structure but are spread
over other areas. Table 1 summarizes the 2007 and 1992 cost element
structure for O&S cost estimating and provides a description of DOD's
cost elements.
Table 1: DOD's Current and Former Structure for Preparing and
Presenting O&S Cost Estimates:
2007 cost element structure: Unit-level personnel;
1992 cost element structure: Mission personnel;
2007 description of cost elements: The direct costs of all operator,
maintenance, and other support personnel at operating units (or at
maintenance and support units that are organizationally related and
adjacent to the operating units).
2007 cost element structure: Unit operations;
1992 cost element structure: Unit-level consumption;
2007 description of cost elements: The unit-level consumption costs of
operating materials such as fuel, electricity, expendable stores,
training munitions, and other operating materials. Also included are
costs of any unit-funded support activities, training devices, or
simulator operations that uniquely support an operational unit,
temporary additional duty/temporary duty associated with the unit's
normal concept of operations, and other unit-funded services.
2007 cost element structure: Maintenance;
1992 cost element structure: Intermediate maintenance; Depot
maintenance;
2007 description of cost elements: The costs of labor (outside of the
scope of unit-level) and materials at all levels of maintenance in
support of the primary system, simulators, training devices, and
associated support equipment.
2007 cost element structure: [A];
1992 cost element structure: Contractor support;
2007 description of cost elements: The cost of contractor labor,
materials, and overhead incurred in providing all or part of the
logistics support to a weapon system.[B].
2007 cost element structure: Sustaining support;
1992 cost element structure: Sustaining support;
2007 description of cost elements: Costs for support services provided
by centrally managed support activities external to the units that own
the operating systems and that can be identified to a specific system
(excludes costs that must be arbitrarily allocated).
2007 cost element structure: Continuing system improvements;
1992 cost element structure: [C];
2007 description of cost elements: The costs of hardware and software
updates that occur after deployment of a system that improve a
system's safety, reliability, maintainability, or performance
characteristics to enable the system to meet its basic operational
requirements throughout its life. (Costs for system improvement
identified as part of the acquisition strategy or a preplanned product
improvement program and included in the acquisition cost estimate are
not included. Also, any improvements of sufficient dollar value that
would qualify as distinct major defense acquisition programs are not
included.)
2007 cost element structure: Indirect support;
1992 cost element structure: Indirect support;
2007 description of cost elements: Installation and personnel support
costs that cannot be directly related to the units and personnel that
operate and support the system being analyzed.
Source: GAO presentation of DOD guidance.
[A] In the 2007 cost element structure, contractor support costs are
reported under other cost element areas.
[B] This description applies to the 1992 cost element structure.
[C] In the 1992 cost element structure, continuing system improvement
costs were reported under the sustaining support cost element area.
[End of table]
The Services Have Developed Systems for Providing Visibility of Actual
O&S Costs:
Each of the services has developed a system for collecting,
maintaining, and providing visibility over historical information on
actual weapon system O&S costs. Collectively referred to as VAMOSC
systems, the Army's system is the Operating and Support Management
Information System; the Navy's system is the Naval Visibility and
Management of Operating and Support Cost system;[Footnote 16] and the
Air Force's system is the Air Force Total Ownership Cost system. These
systems were developed in response to long-standing concerns that the
services lacked sufficient data on the actual costs of operating and
supporting their weapon systems. For example, according to a Naval
Audit Service report, in 1975 the Deputy Secretary of Defense directed
the military departments to collect actual O&S costs of defense weapon
systems. In 1987, the Senate Committee on Appropriations requested
that each service establish a capability within 4 years to report
accurate and verifiable O&S costs for major weapon systems.[Footnote
17] In 1992, DOD required that the O&S costs incurred by each defense
program be maintained in a historical O&S data-collection system.
Related guidance tasked the services with establishing historical O&S
data-collection systems and maintaining a record of data that
facilitates the development of a well-defined, standard presentation
of O&S costs by major defense acquisition program.[Footnote 18]
According to DOD's guidance, the services' VAMOSC systems are supposed
to be the authoritative source for the collection of reliable and
consistent historical O&S cost data about major defense programs, and
it is incumbent upon the services to make the data as accurate as
possible. DOD's stated objectives for the systems include the
provision of visibility of O&S costs so they may be managed to reduce
and control program life-cycle costs and the improvement of the
validity and credibility of O&S cost estimates by establishing a
widely accepted database. According to the guidance, the O&S cost
element structure provides a well-defined standard presentation format
for the services' VAMOSC systems.
Prior GAO Reviews Identified Factors Negatively Affecting DOD's
Ability to Manage O&S Costs:
Our work in the late 1990s and in 2003 identified several factors
negatively affecting DOD's ability to manage O&S costs.[Footnote 19]
First, DOD used immature technologies and components in designing its
weapon systems, which contributed to reliability problems and acted as
a barrier to using manufacturing techniques that typically help reduce
a system's maintenance costs. In contrast, commercial companies ensure
they understand their operating costs by analyzing data they have
collected on equipment they are currently using. Second, DOD's
acquisition processes did not consider O&S costs and readiness as key
performance requirements[Footnote 20] for new weapon systems and
placed higher priority on technical performance features. Further, DOD
continued to place higher priority on enhanced safety, readiness, or
combat capability than on O&S cost management after system fielding.
Third, DOD's division of responsibility among its requirements-
setting, acquisition, and maintenance communities made it difficult to
control O&S costs, since no one individual or office had
responsibility and authority to manage all O&S cost elements
throughout a weapon system's life cycle. Fourth, the services' VAMOSC
systems for accumulating data to analyze operations and support
actions on fielded systems did not provide adequate or reliable
information, thus making it difficult for DOD to understand total O&S
costs.
We have also reported on the effect of DOD weapon system sustainment
strategies on O&S costs. For example, we reported in 2008 that
although DOD expected that the use of performance-based logistics
arrangements would reduce O&S costs, it was unclear whether these
arrangements were meeting this goal. The services were not consistent
in their use of business case analyses to support decisions to enter
into performance-based logistics arrangements. Also, DOD program
offices that implemented these arrangements had not obtained detailed
cost data from contractors and could not provide evidence of cost
reductions attributable to the use of a performance-based logistics
arrangement. Finally, we have reported on O&S cost issues associated
with individual weapon systems, including the Marine Corps' V-22
aircraft in 2009 and the Navy's Littoral Combat Ship in 2010.
Better Information and Guidance Could Help DOD to More Effectively
Manage and Reduce O&S Costs of Major Weapons Systems:
Life-Cycle O&S Cost Estimates for the Production Milestone Were Not
Available for Five of the Seven Systems Reviewed:
The services did not have the life-cycle O&S cost estimates that were
prepared at the production milestone for most of the aviation weapon
systems in our sample. Specifically, production milestone O&S cost
estimates were available for two of the seven systems we reviewed--the
Air Force's F-22A and the Navy's F/A-18E/F. We requested cost
estimates from a variety of sources, including service and OSD offices
that were identified as likely repositories of these estimates.
However, service acquisition, program management, and cost analysis
officials we contacted could not provide these estimates for the
Army's CH-47D, AH-64D, and UH-60L or the Air Force's F-15E or B-1B.
OSD offices we contacted, including the OSD Deputy Director for Cost
Assessment and offices within the Under Secretary of Defense for
Acquisition, Technology and Logistics, also could not provide the cost
estimates for these five systems.
Without the production milestone cost estimates, DOD officials do not
have important information necessary for analyzing the rate of O&S
cost growth, identifying cost drivers, and developing plans for
managing and controlling these costs. In addition, at a time when the
nation faces fiscal challenges and defense budgets may become tighter,
the lack of this key information hinders sound weapon-system program
management and decision making in an area of high costs to the federal
government. In a recent speech,[Footnote 21] the Secretary of Defense
stated that given the nation's difficult economic circumstances and
parlous fiscal condition, DOD will need to reduce overhead costs and
transfer those savings to force structure and modernization within the
programmed budget. DOD officials we interviewed noted that the
department has not placed emphasis on assessing and managing weapon
system O&S costs compared with other priorities. Moreover, our prior
work has shown that rather than limit the number and size of weapon
system programs or adjust requirements, DOD's funding process attempts
to accommodate programs. This creates an unhealthy competition for
funds that encourages sponsors of weapon system programs to pursue
overambitious capabilities and to underestimate costs.
DOD acquisition guidance has required the development of life-cycle
cost estimates for acquisition milestone reviews since at least 1980.
Based on the historical acquisition milestones for the five systems
with missing estimates, the approximate dates that the production
milestone life-cycle O&S cost estimates should have been prepared were
1980 for the Army's CH-47D, 1985 for the Air Force's F-15E, 1989 for
the Army's UH-60L and the Air Force's B-1B,[Footnote 22] and 1995 for
the Army's AH-64D. Additionally, DOD has been required to obtain
independent cost assessments since the 1980s. We requested any
independent estimates that had been prepared for the systems we
reviewed from the OSD Cost Assessment and Program Evaluation office,
but the office could not provide them.[Footnote 23] The service
estimates were prepared in 2000 for the F/A-18E/F and in 2005 for the
F-22A. While DOD officials could not explain why life-cycle O&S cost
estimates for the other five systems were not available, they said
that likely reasons were loss due to office moves, computer failures,
and purging of older files. Further, prior DOD and service guidance
may not have addressed the retention of cost estimates. The two
systems for which cost estimates were available had the most recent
production milestones of the systems in our sample.
Under GAO's guidance for cost-estimating best practices, issued in
2009, thorough documentation and retention of cost estimates are
essential in order to analyze changes that can aid preparation of
future cost estimates.[Footnote 24] However, with the exception of the
Army, current DOD and service acquisition and cost estimation guidance
do not specifically address requirements for retaining O&S cost
estimates and the support documentation used to develop the estimates.
For example, although DOD's cost-estimation guidance emphasizes the
need for formal, complete documentation of source data, methods, and
results, neither it nor DOD's acquisition policy specifically
addresses retention of cost estimate documentation.[Footnote 25]
* Naval Air Systems Command officials said they retained the
production milestone O&S cost estimates for the F/A-18E/F because this
was a good practice; however, they were not aware of any Navy guidance
that required such retention. While the Navy's current acquisition and
cost analysis instructions[Footnote 26] state that records created
under the instructions should be retained in accordance with the
Navy's records management guidance,[Footnote 27] the records
management manual does not clearly identify any requirements for
retaining acquisition cost estimates for aircraft.[Footnote 28] In
addition, we found that although the estimate for the F/A-18E/F was
retained, some of the supporting documentation was incorrect or
incomplete.[Footnote 29]
* The Air Force's acquisition and cost estimation guidance is also
unclear with regard to retention of cost estimates. An Air Force
acquisition instruction[Footnote 30] states that the program manager
is responsible for developing appropriate program documentation and
for maintaining this documentation throughout the life cycle of the
system, as well as maintaining a realistic cost estimate and ensuring
it is well documented to firmly support budget requests. However, we
did not find any references to retaining cost estimates specifically
related to acquisition milestones in either this instruction or other
Air Force acquisition and cost estimation guidance.[Footnote 31]
* Only the Army's current acquisition regulation states that all
documentation required by the milestone decision authority for each
milestone review must be retained on file in the program office for
the life of the program, although the regulation does not make
specific reference to retaining the O&S cost estimate.[Footnote 32]
The production milestones for the three Army systems we reviewed
predate the Army's current regulation, which was issued in 2003.
Complete Data on Actual O&S Costs Were Not Collected in the Services'
VAMOSC Systems:
The services' VAMOSC systems did not collect complete data on actual
O&S costs. The Air Force's and Navy's systems did not collect actual
cost data for some cost elements that DOD guidance recommends be
collected, and the Army's system was the most limited. Additionally,
we found that data for some cost elements were not accurate. DOD
guidance recommends--but does not require--that the cost element
structure used for life-cycle O&S cost estimating also be used by the
services to collect and present actual cost data. Such guidance, if
followed, could enable comparisons between estimated and actual costs.
Some O&S cost data that are not collected in the VAMOSC systems may be
found in other of the services' information systems or from other
sources. However, these data may not be readily available for the
purpose of analyzing weapon system O&S costs. Without complete data on
actual O&S costs, DOD officials do not have important information
necessary for analyzing the rate of O&S cost growth, identifying cost
drivers, and developing plans for managing and controlling these costs.
Air Force's VAMOSC System:
While the Air Force's VAMOSC system collected actual cost data on many
of DOD's recommended cost elements, it did not collect data on some
cost elements for the weapon systems we reviewed. For example, the Air
Force's VAMOSC system did not collect actual O&S costs for support
equipment replacement, modifications, or interim contractor support.
According to service officials, the F-22A, the F-15E, and the B-1B
incurred support equipment replacement and O&S modification costs, and
the F-22A incurred interim contractor support costs. Air Force
officials responsible for the VAMOSC system told us that actual cost
data on these three cost elements are contained in another information
system, the Air Force General Accounting and Finance System-
Reengineered, but the data are not identifiable because procurement
officials often do not apply the established accounting and budgeting
structure when they entered into procurement contracts. Further, the
Air Force lacks a standard structure for capturing contractor
logistics support costs that could provide additional visibility over
both procurement and O&S costs. For example, although program
officials said the F-22A was supported under interim contractor
support in 2006 and 2007,[Footnote 33] no F-22A interim support costs
were included in the VAMOSC system.
Further, according to officials, a recent change in the way the Air
Force funds repair parts also introduced inaccuracies into that
service's VAMOSC system. Starting in fiscal year 2008, the Air Force
centralized the funding of its flying operations at higher-level
commands that support a number of aircraft and bases. For example, the
Air Force Material Command now funds flying operations for most active
units. Prior to that time, the Air Force provided funding for repair
parts directly to lower-level organizational units that paid for each
part when ordered. Under the new process, the higher-level commands
provide funding for repair parts to the Air Force Working Capital Fund
based on the anticipated number of flying hours and an estimated rate
necessary to purchase repair parts per hour of use. Since repair parts
funding is now based on such estimates, there have been differences
between the amounts provided and the actual costs incurred. For
example, officials indicated that in fiscal year 2008 overpayments of
$430 million were provided for repair parts, and in fiscal year 2009
the overpayment amount was $188 million. Although the total
overpayment amount can be identified, the Air Force cannot identify
which specific programs overpaid, so the entire overpayment amount was
recorded against the B-1B's O&S costs in the Air Force accounting
system. VAMOSC system officials were aware of this inaccuracy and
removed the amount from the B-1B's O&S costs within the VAMOSC system.
However, because these officials said they do not have the information
necessary to apply the appropriate amount of the refund to the
appropriate programs, they placed the funds into an account not
associated with a particular weapon system. Therefore, the actual O&S
costs for repair parts reported by VAMOSC system could be inaccurate
for one or more weapon systems for at least the past 2 years.
Navy's VAMOSC System:
For the F/A-18E/F, the Navy's VAMOSC system collected data on many of
DOD's recommended cost elements but did not collect actual O&S costs
for interim contractor support costs, civilian personnel, and indirect
infrastructure costs by weapon system. Navy officials responsible for
the VAMOSC system told us it did not collect interim contactor support
costs because the Navy considers these to be procurement rather than
O&S costs. According to Navy officials, the F/A-18E/F incurred interim
contractor support costs prior to fiscal year 2003.[Footnote 34] Navy
officials are currently attempting to add direct civilian personnel
costs from the Navy's Standard Accounting and Reporting System.
However, since it is difficult to identify these costs by weapon
system, aggregated civilian personnel costs are currently captured
within a separate section of the VAMOSC system. In addition, Navy
officials said indirect infrastructure costs are captured in the
aggregate within a separate section of the VAMOSC system and are not
reported within the O&S costs of each weapon system. According to Navy
officials, these indirect infrastructure costs are not available by
weapon system because of the time and resources that would be
necessary to match real property records--indicating the use of the
facility--to command installation records that contain the costs to
operate the facility.
Further, we found that some of the cost elements in the Navy's VAMOSC
systems were not accurate. For example, the Navy's VAMOSC system did
not separately report F/A-18E/F costs for intermediate-level repair
parts and materials and supplies. According to Navy officials,
intermediate-level costs were included as unit-level repair parts and
materials and supplies due to the way the Navy's accounting system
captures these costs. Also, officials noted that support equipment
maintenance costs were inaccurate because some of these costs were
subsumed under other cost elements. Further, Navy officials said that
the VAMOSC system reported costs for all F/A-18E/F modifications,
including those that added capabilities and those that improved
safety, reliability, maintainability, or the performance
characteristics necessary to meet basic operational requirements.
According to OSD guidance, modifications to add capabilities are
considered a procurement cost and therefore should not be reported as
an O&S cost in the VAMOSC system. According to Navy officials, they
are unable to separate the different types of modification costs in
order to provide visibility for the O&S modification costs.
Army's VAMOSC System:
Compared with the Navy's and Air Force's systems, the Army's VAMOSC
system is the most limited in terms of actual O&S cost data collected.
For the three types of Army aircraft we reviewed, the VAMOSC system
consistently collected data for unit-level consumption cost elements:
fuel, materials and supplies, repair parts, and training munitions.
Costs for depot maintenance, while collected in the system, are not
presented in the OSD-recommended cost element structure. The system
does not include personnel cost data and instead provides a link to
another database. In addition, Army officials said the VAMOSC system
generally collected costs for only government-provided logistics
support and currently contained costs for two weapon systems supported
under contractor logistics support arrangements (the Stryker armored
combat vehicle and UH-72A Light Utility Helicopter).[Footnote 35]
Further, Army officials said that the costs for materials and supplies
and for repair parts were added to the VAMOSC system when the items
were transferred to the unit instead of when they were actually used.
Also, many of the costs were allocated based on demand, quantity, and
price assumptions. That is, if more than one weapon system used a
repair part, the costs for this part were allocated to each weapon
system based on the number of aircraft. While this may be a reasonable
allocation method, the VAMOSC system may not reflect the actual O&S
costs for the weapon systems that used the part.
We reported on deficiencies of the Army's VAMOSC system in 2000.
[Footnote 36] Our prior work found that the Army did not have complete
and reliable data on actual O&S costs of weapon systems. Specifically,
the Army's VAMOSC system did not collect data on O&S cost elements
such as contractor logistics support, supply depot support, and
software support. Further, we reported that the VAMOSC system did not
contain cost data on individual maintenance events, such as removal
and assessment of failed parts. We concluded that without complete O&S
cost data, Army program managers could not assess cost drivers and
trends in order to identify cost-reduction initiatives. Although we
recommended that the Army improve its VAMOSC system by collecting data
on additional O&S cost elements, the Army has not made significant
improvements. According to Army officials responsible for the VAMOSC
system, it was designed to collect information from other information
systems. Therefore, it cannot collect data on other cost elements
unless another information system captures these costs. According to
Army officials, two information systems that the Army is developing--
the General Fund Enterprise Business System and the Global Combat
Support System--may enable the service to collect additional O&S cost
data in the future.
Even with these planned information systems, it is unclear what
additional O&S cost data will be collected, how quickly the Army will
be able to incorporate the data into its VAMOSC system, what resources
may be needed, or what additional limitations the service may face in
improving its VAMOSC system. Army officials, for example, do not
expect the General Fund Enterprise Business System to become fully
operational until the end of fiscal year 2012, and full operation of
the Global Combat Support System will occur later, in fiscal year
2015. Army officials also said while they have requested that
additional O&S cost data be collected by weapon system, it is too
early to tell whether these data will be collected.
The Services Generally Do Not Use Updated Life-Cycle Estimates to
Assess O&S Cost Growth for Fielded Weapon Systems:
Life-Cycle O&S Cost Estimates Were Not Periodically Updated after
Fielding for Six of the Seven Systems Reviewed:
For six of the seven systems selected for our review, the services did
not periodically update life-cycle O&S cost estimates after the
systems were fielded, even though most of the systems have been in
DOD's inventory for over a decade. Only the program office for the F-
22A had updated its production milestone cost estimate. According to
Office of Management and Budget guidance on benefit-cost analysis,
[Footnote 37] agencies should have a plan for periodic, results-
oriented evaluation of the effectiveness of federal programs. The
guidance also notes that retrospective studies can be valuable in
determining if any corrections need to be made to existing programs
and to improve future estimates of other federal programs. In
addition, cost-estimating best practices call for such estimates to be
regularly updated.[Footnote 38] The purpose of updating the cost
estimates is to determine whether the preliminary information and
assumptions remain relevant and accurate, record reasons for variances
so that the accuracy of the estimate can be tracked, and archive cost
and technical data for use in future estimates. Despite the benefit-
cost analysis guidance and cost-estimating best practices, service
officials for six of the seven aviation weapon systems we reviewed
could not provide current, updated O&S cost estimates that
incorporated actual historical costs or analysis of actual costs
compared to the estimate prepared at the production milestone. While
cost estimates were prepared for major modifications to some of the
systems in our review, these estimates were limited in scope and did
not incorporate actual cost data.
The Air Force's updated life-cycle O&S cost estimate for the F-22A
illustrates the potential magnitude of changes in O&S costs that a
weapon system may experience over its life cycle. When the F-22A
program office updated the 2005 cost estimate in 2009, it found a 47-
percent increase in life-cycle O&S costs[Footnote 39]. The 2009
estimate of about $59 billion to operate and support the F-22A is $19
billion more than was estimated in 2005. The increase in life-cycle
O&S costs occurred despite a 34-percent reduction in fleet size from
277 aircraft projected in the 2005 estimate to 184 aircraft projected
in the 2009 estimate. The program office also compared the two
estimates to identify areas of cost growth. According to the program
office, the projected O&S cost growth was due to rising aircraft
repair costs, unrealized savings from using a performance-based
logistics arrangement to support the aircraft, an increased number of
maintenance personnel needed to maintain the F-22A's specialized
stealth exterior, military pay raises that were greater than forecast,
and personnel costs of Air National Guard and Air Force Reserve units
assigned to F-22A units that were not included in the production
milestone estimate.
A 2007 independent review by the Air Force Cost Analysis Agency also
projected future O&S cost growth for the F-22A. According to Air Force
Cost Analysis Agency officials, the review was initiated because cost
data showed the F-22A's cost per flying hour was higher than projected
in the 2007 President's Budget, prompting concerns that the future O&S
costs of the aircraft were underestimated. Specifically, the fiscal
year 2007 actual cost per flight hour was $55,783, about 65 percent
higher than the $33,762 projected in the 2007 President's Budget.
[Footnote 40] The Air Force Cost Analysis Agency estimated that in
2015 (when the system would be fully mature), the F-22A's projected
cost per flight hour would be $48,236, or 113 percent higher than the
$22,665 cost per flight hour in 2015 that had been estimated in 2005.
[Footnote 41] The estimated cost per flight hour increased:
* $8,174 because fixed O&S costs did not decrease in proportion to the
reductions in the number of planned aircraft (277 to 183) and annual
flight hours per aircraft (366 to 277);
* $4,005 because the formula used in the 2005 estimate to calculate
the cost to refurbish broken repair parts understated the potential
costs;
* $2,414 for engine depot maintenance costs due to higher-than-
previously-projected engine cycles[Footnote 42] per flying hour;
* $2,118 for higher costs for purchasing repair parts not in
production or with diminishing manufacturing sources;
* $2,008 because of unrealized economies of scale for repair parts due
to smaller quantity purchases (based on reduced aircraft and flying
hours);
* $1,670 for additional costs for munitions maintainers, training
munitions, and fuel consumption associated with a new capability--an
air-to-ground mission; and:
* $1,641 for additional maintenance due to lower levels of weapon
system reliability than projected in the production milestone O&S cost
estimate.
The remaining $3,542 cost per flight hour increase identified by the
Air Force Cost Analysis Agency's review was due to changes in
personnel requirements, a new requirement to remove and replace the
stealth coating mid-way through the aircraft's life, labor rate
increases, immature engine repair procedures, and the administrative
cost of Air National Guard units assigned to active duty units or
active duty units assigned to Air Force Reserve or Air National Guard
units.
Program Changes over Time Complicate Comparisons of Estimated to
Actual O&S Costs for Two Systems:
For the two aviation systems where both estimated and actual O&S cost
data were available, we found that although there were some areas of
cost growth, direct comparisons between estimated and actual costs
were complicated in part because of program changes that occurred
after the estimates were developed at the production milestone. For
example, the Air Force and Navy had fewer F-22A and F/A-18E/F
aircraft, respectively, in their inventories and flew fewer hours than
planned when the estimates were developed. In addition, a recent, OSD-
sponsored study of the Air Force's C-17 aircraft identified various
changes that can occur over a weapon system's life-cycle and lead to
O&S cost growth. For the C-17, these changes included factors internal
to the program, factors external to the program, and changes in
accounting methods. (The findings from that study are summarized in
appendix II.)
Analysis of Estimated and Actual O&S Costs for the F-22A:
Our analysis showed that actual O&S costs for the Air Force's F-22A
totaled $3.6 billion from fiscal years 2005 to 2009, excluding amounts
for interim contractor support. This amount compared to $3.8 billion
projected for these years in the 2005 production milestone O&S cost
estimate. (Fig. 1 shows estimated and actual costs for each year.)
However, the Air Force had 125 aircraft in its inventory in fiscal
year 2009 rather than the 143 aircraft projected in the 2005 cost
estimate. Also, the aircraft fleet actually flew 68,261 hours over
this time period rather than the 134,618 hours projected in the 2005
cost estimate. On a per flight hour basis, the fiscal year 2009 actual
O&S costs were $51,829, or 88 percent higher than the $27,559 forecast
in 2005 after accounting for inflation.
Figure 1: Comparison of Estimated and Actual O&S Costs for the Air
Force's F-22A (Fiscal Years 2005-2009):
[Refer to PDF for image: multiple line graph]
Constant fiscal year 2010 dollars in billions:
Fiscal year: 2005;
Actual costs: $0.1 billion;
Estimated costs: $0.2 billion.
Fiscal year: 2006;
Actual costs: $0.6 billion;
Estimated costs: $0.3 billion.
Fiscal year: 2007;
Actual costs: $0.8 billion;
Estimated costs: $1 billion.
Fiscal year: 2008;
Actual costs: $0.9 billion;
Estimated costs: $1.1 billion.
Fiscal year: 2009;
Actual costs: $1.1 billion;
Estimated costs: $1.2 billion.
Source: GAO analysis of Air Force data.
Note: The information presented in this figure is subject to
limitations in the data contained in the Air Force's VAMOSC system, as
discussed earlier in this report.
[End of figure]
The use of contractor logistics support for the F-22A further
complicated comparisons of actual costs to the estimated costs
developed in 2005. Although the F-22A has been supported under
contractor logistics support arrangements since before 2005, the
estimates included the costs for government-provided logistics support
of the aircraft. For example, for fiscal years 2005 through 2009, the
O&S cost estimate projected that contractor logistics support would
cost $736 million. However, actual contractor logistics support costs
for the F-22A were $2.1 billion. For fiscal years 2005 through 2009, F-
22A contractor logistics support costs were 60 percent of the total
actual O&S costs reported in the Air Force's VAMOSC system. Due to the
use of this support arrangement, however, the Air Force's VAMOSC
system reports all of the amounts paid to the F-22A contractor under a
single cost element instead of under multiple individual cost
elements. In contrast, program officials confirmed that various
contractor-provided cost elements--such as repair parts, materials and
supplies, depot maintenance, and sustaining support--were included in
the production milestone O&S cost estimate as separate items. Further,
according to officials, prior to 2008 the program office did not
obtain from the contractor cost reports that provide details of how
the amounts paid to the contractor were spent in terms of DOD's
recommended O&S cost elements by fiscal year.[Footnote 43] Therefore,
it is not possible to compare a significant amount of the actual O&S
costs for the F-22A to the production milestone estimate at the cost
element level.
Of the remaining F-22A O&S costs not covered by contractor logistics
support, mission personnel costs constituted the largest proportion--
approximately 22 percent--of the total actual O&S costs reported for
fiscal years 2005 through 2009. Compared with the estimates developed
in 2005, actual mission personnel costs were $34 million (20 percent)
higher for fiscal year 2008 and $113 million (62 percent) higher for
fiscal year 2009. The 2005 estimate provided for 1,335 maintenance
personnel for each F-22A wing (which was projected to number 72
aircraft), but according to Air Force officials the current authorized
personnel for an F-22A wing (now numbering 36 aircraft) is 1,051
maintenance personnel. While the number of aircraft per wing was
reduced by half, the number of personnel was reduced by about 21
percent. According to officials, although the change in wing
composition from three squadrons of 24 aircraft to two squadrons of 18
aircraft reduced personnel requirements, additional personnel who were
not included in the 2005 estimate are now required to support the
aircraft's added air-to-ground mission, an increased maintenance
requirement for the aircraft's stealth exterior, and other maintenance
requirements that were determined through a 2007 staffing study.
[Footnote 44] In addition, Air National Guard and Air Force Reserve
units were not included in the 2005 estimate, so the personnel costs
of these units resulted in higher actual costs. Finally, as noted in
the F-22A program office's 2009 update to the life-cycle O&S cost
estimate, military pay raises given to service members were greater
than forecast in the production milestone estimate.
Analysis of Estimated and Actual O&S Costs for the F/A-18E/F:
Our analysis for the Navy's F/A-18E/F showed that total actual O&S
costs for fiscal years 1999 through 2009 were about $8.7 billion. This
amount compares to the $8.8 billion projected for these years in the
1999 production milestone O&S cost estimate. However, program changes
complicate direct comparisons between estimated and actual costs, as
they do for the F-22A. For example, the Navy estimated that it would
have 428 aircraft in fiscal year 2009, but the actual number of
aircraft was 358, about 16 percent less. Similarly, the Navy estimated
that the aircraft fleet as a whole would fly 780,628 hours from fiscal
year 1999 through 2009, but the aircraft fleet actually flew 625,067
hours, or 20 percent less. On a per flight hour basis, the fiscal year
2009 O&S costs were $15,346, 40 percent higher than the $10,979
forecast in 1999. Although total actual costs were less than estimated
for the 11-year period, actual annual costs for fiscal years 2005
through 2009 have exceeded the annual estimates by an average of 10
percent after accounting for inflation (see figure 2).
Figure 2: Comparison of Estimated and Actual O&S Costs for the Navy's
F/A-18E/F (Fiscal Years 1999-2009):
[Refer to PDF for image: multiple line graph]
Constant fiscal year 2010 dollars in billions.
Fiscal year: 1999
Estimated costs: $0.1 billion;
Actual costs: $0.01 billion.
Fiscal year: 2000
Estimated costs: $0.2 billion;
Actual costs: $0.05 billion.
Fiscal year: 2001
Estimated costs: $0.2 billion;
Actual costs: $0.1 billion.
Fiscal year: 2002
Estimated costs: $0.4 billion;
Actual costs: $0.2 billion.
Fiscal year: 2003
Estimated costs: $0.5 billion;
Actual costs: $0.4 billion.
Fiscal year: 2004
Estimated costs: $0.8 billion;
Actual costs: $0.7 billion.
Fiscal year: 2005
Estimated costs: $1.0 billion;
Actual costs: $1.0 billion.
Fiscal year: 2006
Estimated costs: $1.1 billion;
Actual costs: $1.2 billion.
Fiscal year: 2007
Estimated costs: $1.3 billion;
Actual costs: $1.5 billion.
Fiscal year: 2008
Estimated costs: $1.5 billion;
Actual costs: $1.7 billion.
Fiscal year: 2009
Estimated costs: $1.6 billion;
Actual costs: $1.8 billion.
Source: GAO analysis of Navy data.
Note: The information presented in this figure is subject to
limitations in the data contained in the Navy's VAMOSC system, as
discussed earlier in this report.
[End of figure]
With regard to individual cost elements, our comparison of actual O&S
costs for fiscal years 1999 through 2009 to those projected in the
1999 estimate found that actual costs for fuel, modifications, depot
maintenance, and intermediate maintenance were higher than originally
estimated while training costs were much lower. (App. III presents a
more detailed comparison of actual and estimated O&S costs for the F/A-
18E/F.) In discussing findings from this comparison with cost analysts
at the Naval Air Systems Command, they provided the following
explanations for key changes we identified:
* Fuel costs were higher than estimated because the price of fuel has
increased overall since the estimate was developed in 1999. Further,
when the estimate was developed, it was assumed the F/A-18E/F
aircraft's fuel consumption would be similar to that of the F/A-18C/D.
However, this did not prove to be an accurate analogy, and the F/A-
18E/F's fuel consumption has been higher than that of the earlier
model aircraft. The analysts also said that some of the increased fuel
costs for fiscal year 2005 through 2009 may also be attributed to
increased refueling activity of the F/A-18E/F after the retirement of
the S-3B aircraft.[Footnote 45]
* Depot maintenance costs were higher than estimated, in part because
the engine was repaired by a contractor under a performance-based
logistics arrangement, but the estimate projected costs for government-
provided support. The government repair estimate included a large
initial investment of procurement funds--which are not considered O&S
costs--for spare parts. The Navy subsequently changed the engine
repair concept to a performance-based logistics arrangement with less
expensive spare parts and reduced the initial investment by about 15
percent. However, as a result of the new arrangement, depot
maintenance costs increased. Further, the 1999 estimate purposefully
excluded some engine depot-maintenance costs in order to keep a
consistent comparison with similar costs for the F/A-18A-D models.
(These costs were instead included in the estimate as costs for repair
parts.) However, after adjusting for these issues, actual engine depot
maintenance costs in fiscal years 2007 and 2009 were higher by a total
of approximately $64 million, and Navy officials could not explain
this variance. Additionally, the production milestone estimate
developed in 1999 included costs for support equipment replacement,
which are not captured in the Navy's VAMOSC system.
* Actual costs for aviation repair parts were higher than estimated
after removing the costs that should have been included as engine
depot-maintenance costs from the estimate.
* Intermediate-level maintenance costs were higher than projected
because the estimate did not include personnel costs for shore-based,
intermediate-level maintenance.
* Modification costs were higher than projected because the Navy's
VAMOSC system collected costs for all procurement-funded
modifications, including those that added capabilities, while the
estimate only projected costs for flight-safety modifications.
* Training costs were lower than estimated because the Navy's VAMOSC
system did not include actual nonmaintenance training costs such as
initial pilot and naval flight officer training and installation
support costs. These costs were included in the cost estimate.
Actual O&S Costs Increased for Five Systems, but Extent of Planned
Cost Growth Is Uncertain:
Although we did not have production milestone estimates of life-cycle
O&S costs for the Air Force's F-15E and B-1B or for the Army's AH-64D,
CH-47D, and UH-60L, we reviewed changes in actual O&S costs for each
system and found that costs increased over time for a variety of
reasons. As noted earlier, some cost elements are not maintained in
the services' VAMOSC systems or are not accurate, and our analysis was
subject to these limitations. Furthermore, we could not determine the
extent to which the cost growth was planned since the services could
not provide us with the O&S cost estimates developed for the
production milestone. According to service cost analysis officials,
actual O&S costs for these systems were likely higher than estimated
because such estimates are typically based on peacetime usage rather
than wartime usage assumptions. Further, service cost analysts said
that since the late 1990's actual costs for repair parts have grown
faster than the OSD inflation rates that are used to develop O&S cost
estimates.
Actual O&S Costs for the F-15E for Fiscal Years 1996 through 2009:
Total actual O&S costs for the Air Force's F-15E increased 82 percent
from $944 million in fiscal year 1996 to $1.7 billion in fiscal year
2009 (see figure 3). The number of F-15E aircraft increased 8 percent
from 200 to 215 during this time period, and the number of flight
hours increased 7 percent from 60,726 to 65,054. Per aircraft, O&S
costs increased 69 percent from $5 million to $8 million over this
period, and the cost per flight hour increased 70 percent from $15,538
to $26,408.
Figure 3: Actual F-15E O&S Costs for Fiscal Years 1996-2009:
[Refer to PDF for image: line graph]
Constant fiscal year 2010 dollars in billions.
Fiscal year: 1996;
F-15E O&S Costs: $0.9 billion.
Fiscal year: 1997
F-15E O&S Costs: $0.9 billion.
Fiscal year: 1998;
F-15E O&S Costs: $1.1 billion.
Fiscal year: 1999;
F-15E O&S Costs: $1.1 billion.
Fiscal year: 2000;
F-15E O&S Costs: $1.1 billion.
Fiscal year: 2001;
F-15E O&S Costs: $1.1 billion.
Fiscal year: 2002;
F-15E O&S Costs: $1.2 billion.
Fiscal year: 2003;
F-15E O&S Costs: $1.2 billion.
Fiscal year: 2004;
F-15E O&S Costs: $1.4 billion.
Fiscal year: 2005;
F-15E O&S Costs: $1.4 billion.
Fiscal year: 2006;
F-15E O&S Costs: $1.5 billion.
Fiscal year: 2007;
F-15E O&S Costs: $1.5 billion.
Fiscal year: 2008;
F-15E O&S Costs: $1.6 billion.
Fiscal year: 2009;
F-15E O&S Costs: $1.7 billion.
Source: GAO analysis of Air Force data.
Note: The information presented in this figure is subject to
limitations in the data contained in the Air Force's VAMOSC system, as
discussed earlier in this report.
[End of figure]
Our analysis found that personnel, fuel, repair parts, and depot
maintenance accounted for about 95 percent of the overall increase in
F-15E O&S costs from fiscal years 1996 to 2009. For example, actual
personnel costs grew by $73 million (19 percent) over the period. Most
of the growth was due to wage increases rather than increases in the
number of personnel. Also, fuel costs increased $142 million (18
percent) during these years. According to program officials, this
increase was mainly due to higher fuel prices rather than increased
consumption. Cost for repair parts grew $398 million (51 percent), and
program officials attributed some of this growth to higher costs for
materials used during depot repair, as well as higher prices paid for
labor, storage, and handling. Further, officials said that several
avionics systems on the F-15E have been replaced and the costs to
repair some of the new components are higher. Depot maintenance costs
increased $124 million (16 percent) and program officials said this
increase was due to increasing rates for depot work, noting that the
Air Logistics Centers increased their rates because of higher material
costs.[Footnote 46] Also, officials said that as aircraft age the
number of subsystems that require repair increases, which results in
additional tasks being required during planned depot maintenance. For
example, the F-15C/Ds that are expected to fly until 2025 will be
completely rewired in planned depot maintenance because the original
wiring is deteriorating. A similar program is planned in the future
for the F-15Es and is expected to significantly increase the cost of
planned depot maintenance for that aircraft.
Actual O&S Costs for the B-1B for Fiscal Years 1999 through 2009:
Annual actual O&S costs for the Air Force's B-1B increased 21 percent
from $1.1 billion in fiscal year 1999 to $1.3 billion in fiscal year
2009 (see figure 4). This cost growth occurred despite a 29 percent
reduction in the aircraft fleet from 93 to 66 during the same period.
Per aircraft, O&S costs increased 71 percent from $11 million to $19
million, and the cost per flight hour increased 23 percent from
$46,224 to $56,982.
Figure 4: Actual B-1B O&S Costs for Fiscal Years 1999-2009:
[Refer to PDF for image: line graph]
Constant fiscal year 2010 dollars in billions.
Fiscal year: 1999;
B-1B O&S Costs: $1.1 billion.
Fiscal year: 2000;
B-1B O&S Costs: $1.1 billion.
Fiscal year: 2001;
B-1B O&S Costs: $1.2 billion.
Fiscal year: 2002;
B-1B O&S Costs: $1.3 billion.
Fiscal year: 2003;
B-1B O&S Costs: $1.2 billion.
Fiscal year: 2004;
B-1B O&S Costs: $1.3 billion.
Fiscal year: 2005;
B-1B O&S Costs: $1.2 billion.
Fiscal year: 2006;
B-1B O&S Costs: $1.2 billion.
Fiscal year: 2007;
B-1B O&S Costs: $1.2 billion.
Fiscal year: 2008;
B-1B O&S Costs: $1.4 billion.
Fiscal year: 2009;
B-1B O&S Costs: $1.3 billion.
Source: GAO analysis of Air Force data.
Note: The information presented in this figure is subject to
limitations in the data contained in the Air Force's VAMOSC system, as
discussed earlier in this report.
[End of figure]
Our analysis showed that fuel, repair parts, and depot maintenance
accounted for 97 percent of the overall increase in B-1B O&S costs
from fiscal years 1999 through 2009. Fuel costs increased $89.4
million (40 percent), which program officials attributed mainly to
higher fuel costs and increased utilization of the aircraft in recent
years. Program officials reported that in each of the last 3 full
fiscal years (2007, 2008, and 2009), the hourly utilization rate per
aircraft was 46 percent, 51 percent, and 54 percent higher,
respectively, than in fiscal year 1999. According to the program
office, the increased cost for repair parts, which grew $51.9 million
(23 percent), was due to the increased cost of materials consumed in
the refurbishment of repair parts. Depot maintenance costs increased
$77.1 million (34 percent), and program officials said this increase
was due to higher utilization of aircraft, increased labor and
material costs,[Footnote 47] and changes in cost accounting.
Actual O&S Costs for the AH-64D, CH-47D, and UH-60L for Fiscal Years
1998 through 2007:
The Army's O&S data on unit-level consumption costs for the AH-64D, CH-
47D, and UH-60L showed that all three experienced significant cost
growth from fiscal years 1998 through 2007.[Footnote 48] However, as
table 2 shows, the size of the fleets and numbers of flying hours also
increased, with the AH-64D experiencing the greatest growth. According
to Army officials, fiscal year 1998 costs reflected peacetime training
only, whereas data for fiscal year 2007 also includes costs for units
deployed in Afghanistan and Iraq. O&S costs for deployed units
constituted more than half of the total O&S dollars for these aircraft
in fiscal year 2007.[Footnote 49]
Table 2: Changes in Army Helicopter Inventory, Flight Hours, and Unit-
Level Consumption Costs between Fiscal Year 1998 and Fiscal Year 2007:
Weapon system: AH-64D;
Number of aircraft: Fiscal year 1998: 24[B];
Number of aircraft: Fiscal year 2007: 567;
Number of aircraft: Percent increase: 2,263%;
Number of flying hours: Fiscal year 1998: 994;
Number of flying hours: Fiscal year 2007: 158,030;
Number of flying hours: Percent increase: 15,798%;
Unit-level consumption costs (constant fiscal year 2010 dollars in
millions)[A]: Fiscal year 1998: $5;
Unit-level consumption costs (constant fiscal year 2010 dollars in
millions)[A]: Fiscal year 2007: $757;
Unit-level consumption costs (constant fiscal year 2010 dollars in
millions)[A]: Percent increase: 15,384%.
Weapon system: CH-47D;
Number of aircraft: Fiscal year 1998: 415;
Number of aircraft: Fiscal year 2007: 446;
Number of aircraft: Percent increase: 7%;
Number of flying hours: Fiscal year 1998: 58,311;
Number of flying hours: Fiscal year 2007: 99,743;
Number of flying hours: Percent increase: 71%;
Unit-level consumption costs (constant fiscal year 2010 dollars in
millions)[A]: Fiscal year 1998: $189;
Unit-level consumption costs (constant fiscal year 2010 dollars in
millions)[A]: Fiscal year 2007: $488;
Unit-level consumption costs (constant fiscal year 2010 dollars in
millions)[A]: Percent increase: 159%.
Weapon system: UH-60L;
Number of aircraft: Fiscal year 1998: 453;
Number of aircraft: Fiscal year 2007: 605;
Number of aircraft: Percent increase: 34%;
Number of flying hours: Fiscal year 1998: 79,126;
Number of flying hours: Fiscal year 2007: 180,446;
Number of flying hours: Percent increase: 128%;
Unit-level consumption costs (constant fiscal year 2010 dollars in
millions)[A]: Fiscal year 1998: $103;
Unit-level consumption costs (constant fiscal year 2010 dollars in
millions)[A]: Fiscal year 2007: $498;
Unit-level consumption costs (constant fiscal year 2010 dollars in
millions)[A]: Percent increase: 382%.
Source: GAO analysis of Army data.
Notes: The information presented in this figure is subject to
limitations in the data contained in the Army's VAMOSC system, as
discussed earlier in this report.
[A] Due to the way the Army's VAMOSC system collects and reports
costs, these costs also include materials and supplies for
intermediate-level maintenance.
[B] The Army began fielding the AH-64D in fiscal year 1998.
[End of table]
Measured by flight hour, Army unit-level consumption costs increased
51 percent per flight hour for the CH-47D and 111 percent per flight
hour for the UH-60L, and decreased 3 percent per flight hour for the
AH-64D, from fiscal year 1998 to 2007. As discussed earlier in the
report, unit-level consumption costs reported in the Army's VAMOSC
system include fuel, materials and supplies, repair parts, and
training munitions. As shown in table 3, fuel costs increased by more
than 140 percent for all three systems, the costs of materials and
supplies and repair parts also increased for each system, and the cost
of training munitions decreased. The decreased cost of training
munitions drove the overall decrease in unit-level consumption costs
for the AH-64D, and a program official stated this was likely due to
the significant amount of training conducted during the initial
fielding of the AH-64D in 1998.
Table 3: Percent Change in Army Helicopter Unit-Level Consumption
Costs per Flight Hour between Fiscal Year 1998 and Fiscal Year 2007:
Weapon system: AH-64D;
Increase or decrease in unit-level consumption cost element: Fuel:
161%;
Increase or decrease in unit-level consumption cost element:
Materiel/supplies[A]: 67%;
Increase or decrease in unit-level consumption cost element: Repair
parts: 20%;
Increase or decrease in unit-level consumption cost element: Training
munitions: -74%.
Weapon system: CH-47D;
Increase or decrease in unit-level consumption cost element: Fuel:
142%;
Increase or decrease in unit-level consumption cost element:
Materiel/supplies[A]: 68%;
Increase or decrease in unit-level consumption cost element: Repair
parts: 35%;
Increase or decrease in unit-level consumption cost element: Training
munitions: -44%.
Weapon system: UH-60L;
Increase or decrease in unit-level consumption cost element: Fuel:
142%;
Increase or decrease in unit-level consumption cost element:
Materiel/supplies[A]: 42%;
Increase or decrease in unit-level consumption cost element: Repair
parts: 140%;
Increase or decrease in unit-level consumption cost element: Training
munitions: -54%.
Source: GAO analysis of Army data.
Notes: The information presented in this figure is subject to
limitations in the data contained in the Army's VAMOSC system, as
discussed earlier in this report.
[A] Due to way the Army's VAMOSC system collects and reports costs,
these costs also include materials and supplies for intermediate-level
maintenance.
[End of table]
Updated Estimates of Life-Cycle O&S Costs and Documentation of Program
Changes Are Generally Not Required after Weapon System Production
Decisions:
Even though periodic updates to life-cycle O&S cost estimates could
quantify any cost growth in major weapon systems and help identify
cost drivers, DOD acquisition and cost-estimating guidance do not
require that O&S cost estimates be updated after a program has
completed production. Service guidance that we reviewed does not
consistently and clearly require the updating of O&S cost estimates
after a program has completed production. Additionally, although our
review showed program changes can have a large effect on actual O&S
costs after cost estimates are developed at the production milestone,
DOD and service acquisition guidance do not require program offices to
maintain documentation of such changes for use in cost analysis.
Federal law requires that a full life-cycle cost analysis for each
major defense acquisition program be included in the programs' annual
Selected Acquisition Reports to Congress.[Footnote 50] Requirements
related to Selected Acquisition Reports, however, end when a weapon
system has reached 90 percent of production.[Footnote 51] In addition,
we found that for the systems we reviewed, the estimated O&S costs
included in the Selected Acquisition Reports were sometimes not
updated.[Footnote 52] For our sample, the estimated O&S costs included
in the annual reports for the F-22A remained unchanged from 2005
through 2007, and the services did not have current updated life-cycle
O&S cost estimates for the other six weapon systems we reviewed.
Further, while life-cycle costs are required to be reported in the
Selected Acquisition Reports, OSD officials noted that the calculation
of the estimate may be inconsistent. For example, cost analysts at the
Naval Air Systems Command maintain a cost-estimating model for the F/A-
18E/F that is regularly updated and used to develop O&S cost estimates
for the Selected Acquisition Reports and other analyses to improve
cost effectiveness. However, the methodology used to develop the
Navy's cost estimates for the Selected Acquisition Reports differs
from the methodology used to develop life-cycle cost estimates for
acquisition milestone decisions and includes significantly more
infrastructure costs. According to the Naval Air System Command
guidance, the estimates for the Selected Acquisition Reports are not
comparable to the acquisition milestone life-cycle cost estimates
without adjusting for the different ground rules and assumptions used.
The estimates for the Selected Acquisition Reports also are not
comparable to the costs reported in the Navy's VAMOSC system.
DOD acquisition policy requires the services to provide life-cycle O&S
cost estimates for decisions made during specific points in the
acquisition process, including the production decision, but neither
this policy nor DOD's cost-estimating guidance require O&S cost
estimates for systems that have been fielded. In a December 2008
memorandum, DOD also required that several metrics, including an
ownership cost metric,[Footnote 53] be reported quarterly for all
major weapon defense acquisition programs. However, this quarterly
reporting policy does not currently apply to weapon systems that have
completed production and are no longer reporting information in the
Selected Acquisition Reports to Congress. Of the weapon systems we
reviewed, program offices for the AH-64D, F-22A, and F/A-18E/F
currently provide Selected Acquisition Reports to Congress.
The Army regulation and Navy instructions we reviewed do not address
updating life-cycle O&S cost estimates for systems that have been
fielded.[Footnote 54] Although the Air Force has a directive requiring
annual updates to program cost estimates, it does not specifically
mention life-cycle O&S cost estimates. An Air Force directive issued
in August 2008 includes the requirement that major acquisition program
cost estimates be updated annually and used for acquisition purposes,
such as milestone decisions, and other planning, programming,
budgeting, and execution decisions.[Footnote 55] The directive also
states that it is applicable to organizations that manage both
acquisition and sustainment programs. However, as mentioned earlier,
service and OSD officials were unable to locate O&S cost estimates for
the F-15E and the B-1B aircraft. According to Air Force cost analysis
and policy officials, the requirement for annual cost estimate updates
is applicable to programs no longer in acquisition, but they are still
developing the Air Force instruction that will contain more specific
guidance for implementing the 2008 directive. The officials expect
that, once issued, the Air Force instruction will clarify the
requirement to update O&S cost estimates annually.
In addition, changes in weapon system programs affected the
assumptions used in production-milestone life-cycle O&S cost
estimates, but DOD and service acquisition guidance that we reviewed
do not explicitly require the services to maintain documentation of
program changes affecting O&S costs. According to federal standards
for internal control,[Footnote 56] information should be recorded and
communicated to management and others within the entity who need it
and in a form and within a time frame that enables them to carry out
their internal control and other responsibilities. Also, managers need
to compare actual performance to planned or expected results and
analyze significant differences.
DOD Has Departmentwide and Service-Specific Initiatives to Address
Weapon System O&S Costs:
Several Departmentwide Initiatives Address Weapon System O&S Costs:
DOD has several departmentwide initiatives to address weapon system
O&S costs. The DOD-wide Reduction in Total Ownership Costs-Special
Interest Program, initiated in 2005, is aimed at reducing weapon
system O&S costs by improving reliability and maintainability and
reducing total ownership costs in weapon systems that are already
fielded. Program funding totaled about $25 million in fiscal year
2009. For its 15 funded projects, DOD forecasts total ownership cost
savings for fiscal years 2006 through 2011 to be $9.5 billion, with an
average 60 to 1 return on investment.[Footnote 57] For example,
according to officials, the program is funding an effort to develop
trend analysis software to diagnose and resolve problems with the F/A-
18 aircraft.
Other departmentwide initiatives seek to better manage O&S costs of
major weapon systems during the acquisition process. Some of these
initiatives address factors we previously identified as negatively
affecting DOD's ability to manage O&S costs. In 2003, we reported that
DOD did not consider O&S costs and readiness as key performance
requirements for new weapon systems and placed higher priority on
technical performance features. In 2007, DOD began requiring the
services to establish an ownership cost metric during the requirements
determination and acquisition processes for weapon systems in order to
ensure that O&S costs are considered early in decision making.
According to current Joint Staff guidance,[Footnote 58] the ownership
cost metric and reliability metric are key system attributes of the
sustainment (or materiel availability) key performance parameter.
[Footnote 59] While the ownership cost metric includes many of OSD's
recommended O&S cost elements, such as energy (fuel, oil, petroleum,
electricity, etc.), maintenance, sustaining support, and continuous
system improvements, it does not include personnel and system-specific
training costs. In 2008, OSD expanded the use of the ownership cost
and materiel reliability metrics, along with the materiel availability
key performance parameter, to all major defense acquisition programs
that provide information to Congress in Selected Acquisition Reports.
In a July 2008 memorandum intended to reinforce the use of the life-
cycle metrics, OSD requested that these programs develop target goals
for each metric within 60 days.[Footnote 60] In a December 2008
memorandum, OSD asked the services to begin reporting against the
target goals on a quarterly basis.[Footnote 61] According to OSD
officials, they are working with the services to improve the accuracy
and submission of the reported cost information.
We also previously noted that DOD used immature technologies in
designing its weapon systems, which contributed to reliability
problems and acted as a barrier to using manufacturing techniques that
typically help reduce a system's maintenance costs. DOD has identified
insufficient reliability designed in the system during acquisition as
one of the key reasons for increases in O&S costs. Based on the
recommendation of the DOD Reliability Improvement Working Group, DOD's
primary acquisition instruction was updated in 2008 to include
guidance directing program mangers to develop reliability,
availability, and maintainability strategies that include reliability
growth as an integral part of design and development. Further, the
instruction states that reliability, availability, and maintainability
shall be integrated within systems engineering processes; documented
in system plans; and assessed during programmatic reviews.[Footnote 62]
DOD has also taken steps to improve the information available for cost
estimating and monitoring of actual O&S costs. In 2008, we reported
that for the performance-based logistics arrangements we reviewed,
program offices often did not have detailed cost data that would
provide insights regarding what the program office was spending for
various aspects of the support program.[Footnote 63] That same year,
DOD's primary acquisition instruction was updated to include a
requirement that sustainment contracts provide for detailed contractor
cost reporting for certain major programs to improve future cost
estimating and price analysis.[Footnote 64] However, the instruction
does not provide details as to the timing or content of such cost
reporting. Officials in OSD Cost Assessment and Program Evaluation are
currently drafting additional guidance to clarify the cost-reporting
requirement.
Additionally, OSD Cost Assessment and Program Evaluation initiated an
effort in 2008 to collect actual operational testing and evaluation
information and make it available to cost analysts for use in
developing weapon system cost estimates. According to OSD officials,
actual test data could improve these estimates by providing cost
analysts more accurate information. In support of the initiative, the
services have collected over 150 test data reports from their
operational testing agencies. Although cost analysis officials
indicated that they have not yet used the test data in preparing cost
estimates, there is a high level of interest in the information
contained in the test reports as evidenced by the number of times the
data have been accessed. Officials noted that research is ongoing,
particularly within the Army, to develop quantitative tools that link
operational test results with O&S cost estimates.
The Services Have Initiatives to Help Them Better Manage Aviation
System O&S Costs:
The services also identified initiatives to help them better manage
aviation system O&S costs. Although one Army command had an O&S cost-
reduction program, none of the services had cost-reduction programs
implemented servicewide. According to Army officials, the most direct
aviation O&S cost-reduction initiative within that service is the
Aviation and Missile Life Cycle Management Command's O&S Cost
Reduction program. Under the program, the command investigates fielded
aviation systems with high failure rates and high costs and attempts
to reduce costs by funding projects aimed at reliability improvements,
life-cycle extensions, and acquisition cost reductions. According to
Army officials, the annual budget for this program is $10 million to
$12 million per year, and most projects predict at least a 2.5 to 1
return on investment. Examples of funded projects include developing a
fuel additive and reducing corrosion in CH-47 aircraft blades.
Officials also noted that other Army initiatives during the last
several years include a renewed emphasis on the importance of
estimating total life-cycle costs during the weapon system acquisition
process and the establishment of draft guidance for the inclusion of
Operations and Maintenance funding projections within acquisition
program affordability charts used during certain weapon system
acquisition reviews.[Footnote 65] In addition, the Army conducts
annual weapon systems reviews at which program managers present
current and emerging life-cycle weapon system funding requirements
based on the latest Army or program office cost estimate developed for
the system. Army officials said these initiatives can help the Army in
better managing O&S costs.
While the Navy could not identify initiatives designed specifically to
reduce O&S costs for its aviation systems, Navy officials said the
Naval Aviation Enterprise, a working group of naval aviation
stakeholders, was established in 2004 to meet multiple goals,
including exchanging information to reduce O&S costs. Through cross-
functional teams, subject-matter experts collaborate to resolve
problems and improve operations. The Navy stated that, as a result of
this initiative, it achieved O&S cost savings of $50 million from its
flying-hour program in fiscal year 2005. Additionally, Navy officials
cited the establishment of Fleet Readiness Centers as an initiative
that could lead to O&S cost reduction in aviation systems. Created as
part of the Base Realignment and Closure process in 2005, the Fleet
Readiness Centers aim to improve maintenance efficiency and reduce
costs by combining intermediate-and depot-level maintenance personnel.
As a result, the Navy expects avoidance of unwarranted maintenance
procedures, reduced turnaround times, an increase in completed
repairs, and reduced maintenance costs. Although the Navy is expected
to achieve cost savings from the Fleet Readiness Centers, we reported
in 2007 that the projected savings are likely to be overstated.
[Footnote 66]
The Air Force also lacks initiatives specifically designed to reduce
O&S costs of aviation systems. Air Force officials noted, however,
that improved management of O&S costs could result from its
Expeditionary Logistics for the 21st Century program. The program is a
logistics process-improvement effort that was started in 2005 under a
larger program called Air Force Smart Operations for the 21st Century,
which is the guiding program for all transformation efforts within the
Air Force. Although one goal of the program is to reduce O&S costs by
10 percent, Air Force officials said program initiatives to date do
not focus on specific weapon systems.
DOD's Recent Assessment Identified Problems and Recommended Actions to
Improve Weapon System Product Support:
A DOD Product Support Assessment Team led by the Office the Under
Secretary of Defense for Acquisition, Technology and Logistics
recently concluded a year-long study of weapon system product support,
and in November 2009 issued a report with recommendations to improve
weapon system life-cycle sustainment.[Footnote 67] With regard to O&S
costs, the report cited inadequate visibility of O&S costs as one of
several problems that hinder weapon system life-cycle support
management. According to the report, DOD does not have adequate
visibility of O&S costs; lacks a process to systematically track and
assess O&S costs; and lacks valid, measurable sustainment metrics to
accurately assess how programmatic decisions will affect life-cycle
costs. Further, the report states that DOD cannot identify, manage,
and mitigate major weapon system cost drivers. To address identified
deficiencies in O&S cost management, the Product Support Assessment
Team recommended (1) establishing an O&S affordability requirement,
including linking O&S budgets to readiness, (2) developing and
implementing an affordability process with all DOD stakeholders (such
as the financial and program management communities), and (3)
increasing the visibility of O&S costs and their drivers across the
supply chain.
In addition to the deficiencies identified with regard to O&S cost
management, the Product Support Assessment Team also found
deficiencies in DOD's sustainment governance. Governance is defined by
the Product Support Assessment Team as the consistent and cohesive
oversight across the management, policies, processes, and decision
making for sustainment to ensure that sustainment information is a
critical component of weapon system acquisition and throughout the
life cycle. The report noted that every programmatic decision made
during the life cycle of a weapon system should be made with the
knowledge of how that decision will affect the life-cycle support of
that system. However, the report stated that this has been difficult
within DOD due to the lack of perceived relative importance of long-
term costs and lack of valid, measurable support metrics, especially
cost projections. To address identified deficiencies in sustainment
governance, the Product Support Assessment Team recommended (1)
strengthening guidance so that sustainment factors are sufficiently
addressed and governed at key life-cycle decision points, (2) issuing
DOD policy to require the services to conduct independent logistics
assessments[Footnote 68] prior to acquisition milestones, and (3)
creating a post-initial-operating-capability review that includes an
assessment of known support issues and potential solutions. OSD has
formed three Integrated Product Teams to further develop and lead the
implementation of the Product Support Assessment Team recommendations
over a 3-year period. While the report highlighted some of the
limitations on assessing and managing O&S costs, the current
recommendations do not identify specific actions or enforcement
measures.
One of the first changes resulting from the Product Support Assessment
Team recommendations was a new DOD effort in April 2010 to begin
reviews of sustainment costs for all acquisition category ID weapon
system programs[Footnote 69] and address sustainment factors at
milestone decision and other review points during the acquisition
process. Under new DOD guidance, program managers for these programs
are to use a sustainment chart to facilitate the reviews and provide
information on support strategy, metrics, and costs in a standardized
format.[Footnote 70] Specifically, the chart should include the
original O&S cost baseline, as reported in the initial Selected
Acquisition Report for the system, as well as current program costs
according to the most recent projections. Further, the current
estimated total O&S costs for the life cycle of the system should also
be included, along with the antecedent system's cost for comparison.
No Single Individual or Entity Is Empowered to Control O&S Costs:
A related factor that has historically challenged DOD's ability to
reduce weapon system O&S costs is that no single individual or entity
within the department is empowered to control these costs. A variety
of offices within the services and DOD are involved in the decision
making that affects sustainment. Though DOD has designated the program
manager as responsible for many aspects of weapon system life-cycle
sustainment planning, many decisions and processes are outside of the
program manager's control. Using aviation systems as an example, these
decisions and processes include budget determination, funding
processes, the number and pay of personnel assigned to support
aircraft, the number of aircraft procured, the number of hours flown,
the aircraft basing locations, and the rates charged by depot
maintenance facilities. After the aircraft are produced, program
managers have only a limited ability to directly affect O&S costs.
Army aviation officials, for example, indicated that during the
sustainment phase, program managers control only the budgets for
program-related logistics and engineering support, retrofit
modifications, and technical manuals, which account for only a small
percentage of total O&S costs.
In addition, it is likely that multiple individuals will serve as the
weapon system's program manager over its life-cycle. For example, the
average tenure for a program manager is roughly 17 months, whereas the
average life of a major weapon system often exceeds 20 years. This
turnover results in program managers bearing responsibility for the
decisions of their predecessors, making it difficult to hold the
program manager accountable for growth in the system's O&S costs.
Finally, a weapon system's long life-cycle also affects cost-reduction
initiatives, as it may take many years for some of the initiatives to
produce returns on investment.
Conclusions:
In the absence of key information on O&S costs for its major weapon
systems, DOD may not be well-equipped to analyze, manage, and reduce
these costs. While the military services are required to develop life-
cycle O&S cost estimates to support production decisions, DOD cannot
fully benefit from these estimates if they are not retained. If cost-
estimating best practices are followed, the estimates, among other
things, can provide a benchmark for subsequent cost analysis of that
system, enable the identification of major cost drivers, and aid in
improving cost estimating for future systems. Similarly, in the
absence of more complete historical data on a weapon system's actual
O&S costs in their VAMOSC systems, the services are not in a good
position to track cost trends over time, compare these actual costs
with previous estimates, and determine whether and why cost growth is
occurring. While all the services' VAMOSC systems have deficiencies,
the Army's system has the greatest limitations. We reported on these
limitations 10 years ago and recommended improvements, but the Army
has not made significant improvements since then.
Moreover, without periodically updating life-cycle O&S cost estimates
and documenting program changes affecting O&S costs after a system is
fielded, DOD managers lack information necessary to compare actual
performance to planned or expected results, as stated in federal
standards for internal control. DOD has begun to recognize that
greater management emphasis should be placed on better managing weapon
system O&S costs, as indicated by several current and planned
initiatives. The department furthermore has acknowledged deficiencies
in O&S cost visibility and noted that every programmatic decision made
during the entire life cycle of a DOD weapon system should be made
with the knowledge of how that decision will affect the life-cycle
sustainment of that system. Finally, citing the economic and fiscal
challenges the nation faces along with the prospects for greatly
reduced defense budgets, the Secretary of Defense highlighted the need
for DOD to take a more aggressive approach to reducing its spending
and finding efficiencies where possible in order to better afford its
force structure and weapon system modernization priorities. These
competing budget priorities provide additional impetus for DOD to
manage and reduce weapon system O&S costs.
Recommendations for Executive Action:
To improve DOD's ability to manage and reduce O&S costs of weapon
systems over their life cycle, we recommend that the Secretary of
Defense direct the Under Secretary of Defense for Acquisition,
Technology and Logistics and the Director of OSD Cost Assessment and
Program Evaluation to take the following five actions:
* Revise DOD guidance to require the services to retain life-cycle O&S
cost estimates and support documentation used to develop the cost
estimates for major weapon systems. This requirement should apply to
cost estimates developed by weapon system program offices and other
service offices, including cost analysis organizations. Furthermore,
this requirement should include cost estimates prepared during the
acquisition process as well as those prepared after a system is
fielded.
* Identify the cost elements needed to track and assess major weapon
systems' actual O&S costs for effective cost analysis and program
management, and require the services to collect and maintain these
elements in their VAMOSC systems. To the extent possible, data
collected on actual O&S costs should be comparable to data presented
in life-cycle cost estimates. To oversee compliance with this new
requirement, DOD should require the services to identify any gaps
where actual cost data are not being collected and maintained and to
identify efforts, along with timelines and resources, for filling
these gaps.
* Direct the Army to develop and implement a strategy for improving
its VAMOSC system. This strategy should include plans for
incorporating additional cost elements from other information systems,
time frames for expanding on existing cost elements, and resources
required to improve the VAMOSC system.
* Require the services to periodically update their life-cycle O&S
cost estimates for major weapon systems throughout their life cycle.
These updates should provide an assessment of cost growth since the
prior estimate was developed and account for any significant cost and
program changes.
* Develop guidance for documenting and retaining historical
information on weapon system program changes to aid in effective
analysis of O&S costs. DOD should determine, in conjunction with
service acquisition and cost analysis officials, the types of
information needed and the level of detail that should be retained.
We also recommend that the Secretary of Defense require that the
Director of OSD Cost Assessment and Program Evaluation retain any
independent life-cycle O&S cost estimates prepared by that office
along with support documentation used to develop these cost estimates
for major weapon systems.
Agency Comments and Our Evaluation:
In its written comments on a draft of this report, DOD generally
concurred with our recommendations, noting that the department is
committed to strengthening its O&S data availability as well as its
use of O&S estimates in the governance process for major defense
acquisition programs. DOD also stated that it will take steps to
update its policy to ensure that O&S cost estimates are retained,
along with supporting documentation. Specifically, the department
fully concurred with four recommendations and partially concurred with
two. The department's written comments are reprinted in appendix IV.
DOD also provided technical comments that we have incorporated into
this report where applicable.
DOD concurred with our four recommendations to revise guidance to
require the services to retain life-cycle O&S cost estimates and
support documentation used to develop the cost estimates; develop
guidance for documenting and retaining historical information on
weapon system program changes to aid in effective analysis of O&S
costs; require that the Director of the Cost Assessment and Program
Evaluation retain any independent life-cycle O&S cost estimates
prepared by that office, along with support documentation used to
develop these cost estimates for major weapon systems; and revise DOD
guidance to require the services to periodically update life-cycle O&S
cost estimates for major weapon systems throughout their life cycle
and assess program changes and cost growth. While DOD concurred with
our recommendation to periodically update life-cycle O&S cost
estimates for major weapon systems, the department noted that the Navy
is concerned about the additional cost and personnel related to this
requirement. We maintain that periodic estimates that quantify and
assess changes in weapon systems O&S costs will assist with the
identification of prospective areas for cost reduction and improve
DOD's ability to estimate O&S costs in the future. Therefore, the
resulting benefits from periodic analysis of O&S costs will likely be
greater than the incremental costs associated with the additional
resources.
DOD partially concurred with our recommendation to identify the cost
elements needed to track and assess major weapon systems' actual O&S
costs for effective cost analysis and program management, require the
services to collect and maintain these elements in their VAMOSC
systems, and require the services to identify elements where actual
cost data are not being collected and maintained, along with efforts
for filling these data gaps. However, the department noted that while
DOD will coordinate internally to address this issue, the Director of
the Cost Assessment and Program Evaluation office should be directed
to take this action in lieu of the Under Secretary of Defense for
Acquisition, Technology and Logistics. DOD's comments further noted
that these two OSD offices would coordinate with one another to
implement other recommendations we made. We have modified our
recommendations to reflect that both the Under Secretary of Defense
for Acquisition, Technology and Logistics and the Director of the Cost
Assessment and Program Evaluation office will need to play key roles
in implementing these recommendations.
DOD also partially concurred with our recommendation that the Army
develop and implement a strategy for improving its VAMOSC system. DOD
stated that while the Army will develop such a strategy, the Army
maintains that its military personnel costs are collected by a
separate database, the Army Military-Civilian Cost System, and
although the costs are not captured by weapon system fleet, the data
are sufficient for O&S cost -estimating purposes. The Army also
pointed out that it has made progress in collecting contractor
logistics support cost data. Specifically, the Army stated that
guidance issued in 2008 has led to cost-reporting requirements (that
is, requirements that the contractor provide details regarding support
costs by cost element) being included in new support contracts.
Further, the Army noted that a future information system should be
able to capture contractor support cost data. As we stated in our
report, new Army systems may improve the availability of actual O&S
cost data. However, these systems are still being developed. Even with
these planned information systems, it is unclear what additional O&S
cost data will be collected, how quickly the Army will be able to
incorporate the data into its VAMOSC system, what resources may be
needed, or what additional limitations the service may face in
improving its VAMOSC system. We based our recommendation on DOD
guidance regarding the VAMOSC systems. As we state in our report, DOD
required that the O&S costs incurred by each defense program be
maintained in a historical O&S data-collection system and designated
the services' VAMOSC systems as the authoritative source for these
cost data. Therefore, we continue to believe the Army needs a strategy
for improving the cost data available in its VAMOSC system.
While generally concurring with our recommendations, DOD's response
noted that there are over 150 major defense acquisition programs
across the departments and agencies, ranging from missile defense
systems to combat vehicles, with each program having unique challenges
in data reporting. Although DOD agreed that our report was reasonable
in its analysis of the seven programs reviewed, it emphasized that the
problems encountered with our sample may not be found across the
entire department. While we solicited DOD's and the services' inputs
to try to avoid selecting weapon systems with known data limitations,
we agree with DOD and our report clearly states that we selected a
nonprobability sample for our review and, therefore, the results
cannot be used to make inferences about all major weapon systems.
DOD's response also noted that while our report recognizes the recent
initiatives the department has established to track and prevent future
O&S cost growth, the effects of these initiatives are generally not
reflected in the systems we analyzed. According to DOD's comments, a
review of at least one pre-major defense acquisition program would
have allowed us to assess the potential long-term effect of these
initiatives with respect to controlling O&S cost growth. While we
agree that a review of the effectiveness of recent initiatives would
be beneficial in the future, many of the initiatives were only
implemented in the last several years and are likely too new to
demonstrate improvements. Further, the scope of our work was limited
to a comparison of the original O&S cost estimates developed for
selected major weapon systems to the actual O&S costs incurred in
order to assess the rate of cost growth. Therefore, we selected
systems that had previously passed through DOD's acquisition process,
achieved initial operating capability, and been fielded for at least
several years. These systems were not affected by DOD's recent
initiatives.
We are sending copies of this report to interested congressional
committees; the Secretary of Defense; the Secretaries of the Army, the
Navy, and the Air Force; the Under Secretary of Defense for
Acquisition, Technology and Logistics; and the Director, Office of
Management and Budget. In addition, the report will be available at no
charge on the GAO Web site at [hyperlink, http://www.gao.gov/].
If you or your staff have any questions concerning this report, please
contact me on (202) 512-8246 or edwardsj@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. Key contributors are listed in
appendix V.
Signed by:
Jack E. Edwards:
Director, Defense Capabilities and Management:
List of Committees:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Daniel K. Inouye:
Chairman:
The Honorable Thad Cochran:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
United States Senate:
The Honorable Ike Skelton:
Chairman:
The Honorable Howard P. McKeon:
Ranking Member:
Committee on Armed Services:
House of Representatives:
The Honorable Norman D. Dicks:
Chairman:
The Honorable C.W. Bill Young:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:
[End of section]
Appendix I: Scope and Methodology:
To conduct our review of growth in operating and support (O&S) costs
for major weapon systems, we collected and analyzed data on seven
major aviation systems: the Navy's F/A-18E/F; the Air Force's F-22A, B-
1B, and F-15E; and the Army's AH-64D, CH-47D, and UH-60L. We focused
on aviation systems to enable comparisons of cost growth, where
possible, across the selected systems. For example, some factors
driving cost growth in an aviation system may be more applicable to
other types of aircraft than to maritime or land systems. We selected
aviation systems that had reached initial operating capability after
1980 and had incurred several years of actual O&S costs, indicating a
level of maturity in the program. The newest system in our sample--the
F-22A--has been fielded for about 4 years, and the oldest system--the
CH-47D--has been fielded about 17 years. We limited our selection to
aviation systems that had relatively large fleets, avoiding low-
density systems for which cost data may have been anomalous. We also
selected the systems to reflect varied characteristics in terms of
military service, mission, and support strategy. However, we did not
include a Marine Corps aviation system in our sample because the Naval
Air Systems Command manages and supports all Marine Corps aircraft. We
also did not select systems with known limitations of available data
on actual O&S costs. For example, we have previously reported that
some systems supported under performance-based logistics arrangements
may not have detailed cost data available because the Department of
Defense (DOD) has not required the contractor to provide these data.
In considering which systems to select for our review, we also
obtained input from DOD and service officials. The results from this
nonprobability sample cannot be used to make inferences about all
aviation systems or about all major weapon systems because the sample
may not reflect all characteristics of the population.
The following is an overview of each system selected for our review:
* The F/A-18E/F Super Hornet is an all-weather attack aircraft as well
as a fighter. It performs a variety of missions including air
superiority, fighter escort, reconnaissance, aerial refueling, close
air support, air defense suppression, and day/night precision strike.
The F/A-18E/F entered full rate production in January 2000 and
established initial operational capability in September 2001. As of
the end of fiscal year 2009, the Navy had 358 F/A-18E/F aircraft.
* The F-22A Raptor is the Air Force's newest fighter aircraft and
performs both air-to-air and air-to-ground missions. Officials stated
that the program received approval to enter into full rate production
in April 2005 and established initial operating capability in December
2005. Currently, the Air Force plans to buy 187 F-22A aircraft.
* The F-15E Strike Eagle is a dual-role fighter designed to perform
air-to-air and air-to-ground missions. Officials indicated that the
program received approval to enter into full rate production in early
1986 and established initial operating capability in September 1989.
As of the end of fiscal year 2009, the Air Force had 223 F-15E
aircraft.
* The B-1B Lancer is a multimission long-range bomber designed to
deliver massive quantities (74,000 pounds) of precision and
nonprecision weapons. The Air Force received the first B-1B in April
1985 and established initial operating capability in September 1986.
As of the end of fiscal year 2009, the Air Force had 66 B-1B aircraft.
* The AH-64D Apache Longbow is the Army's heavy division/corps attack
helicopter. It is designed to conduct rear, close, and shaping
missions, as well as distributed operations and precision strikes. In
addition, the AH-64D is designed to provide armed reconnaissance
during day or night, in obscured battlefields, and in adverse weather
conditions. The original Apache entered Army service in 1984, and the
AH-64D followed in 1998. As of the end of fiscal year 2009, the Army
had 535 AH-64D aircraft.
* The UH-60L Black Hawk is a twin-engine helicopter that is used in
the performance of the air assault, air cavalry, and aeromedical
evacuation missions. The UH-60L is an update to the original UH-60A,
which entered Army service in 1979. As of the end of fiscal year 2009,
the Army had 564 UH-60L aircraft.
* The CH-47D Chinook is a twin-engine, tandem-rotor transport
helicopter that carries troops, supplies, ammunition, and other battle-
related cargo. Between 1982 and 1994, the Army upgraded all early
models--the CH-47A, B, and C models--to the CH-47D, which features
composite rotor blades, an improved electrical system, modularized
hydraulics, triple cargo hooks, and more powerful engines. As of the
end of fiscal year 2009, the Army had 325 CH-47D aircraft.
To determine the extent to which (1) life-cycle O&S cost estimates
developed during acquisition and data on actual O&S costs are
available for program management and decision making and (2) DOD uses
life-cycle O&S cost estimates for major weapon systems after they are
fielded to quantify cost growth and identify its causes, we identified
available cost estimates, compared the estimates with actual cost
data, and obtained additional information on how O&S costs are
tracked, assessed, managed, and controlled. We requested documentation
from the services and the Office of the Secretary of Defense (OSD) on
life-cycle O&S cost estimates that the services prepared during
acquisition to support the decision to proceed with production of the
aircraft in our sample. We also requested documentation of O&S cost
estimates that OSD may have independently prepared for this milestone
decision. We focused on the production milestone because, while life-
cycle cost estimates may be developed during earlier stages of the
acquisition process, DOD cost-estimating guidance states that cost
estimates for the production milestone should be based on the current
design characteristics of the weapon system, the latest deployment
schedule, and the latest operation and maintenance concept. In
addition, we requested documentation from the services for any current
updates to life-cycle O&S cost estimates that may have been developed
after the systems were fielded. We also obtained information from
weapon system program offices on their practices for retaining
information regarding program changes affecting O&S costs. To identify
requirements for conducting, updating, and retaining cost estimates,
we reviewed Office of Management and Budget guidance, DOD and service
acquisition and cost estimation guidance, and federal guidance on cost-
estimating best practices.
For actual historical data on weapon system O&S costs, we obtained
access to the services' Visibility and Management of Operating and
Support Costs (VAMOSC) systems that have been designated as the
authoritative sources of these data. We worked with service cost
analysis officials to understand how data in these systems are
organized and how to query them for data on our selected aviation
systems. To assess the reliability of the data, we surveyed cost
analysis officials. For example, we obtained information on specific
cost elements that were collected, data sources, and efforts to
improve the completeness and accuracy of collected data. We also
reviewed DOD and service guidance on the VAMOSC systems and cost
element structure, and we reviewed prior GAO and DOD assessments of
the availability of actual O&S cost data for DOD weapon systems. We
identified limitations in the data and discuss these in our report.
Taking these limitations into account, we determined that the
available data were sufficiently reliable to compare estimated to
actual costs for the F-22A and F/A-18E/F, the two systems in our
sample for which we were able to obtain the production milestone life-
cycle O&S cost estimate, and also to present an analysis of changes in
actual costs over time for the other five systems.
In comparing estimated to actual costs for the F-22A and the F/A-
18E/F, we analyzed differences that occurred each year, determined
which cost elements experienced the greatest changes over time, and
reviewed how actual program conditions compared to the assumptions
used to develop the production milestone cost estimate. In addition,
we met with cost analysis experts from the Center for Naval Analyses
and the Institute for Defense Analyses and obtained the results of an
Institute for Defense Analysis study on O&S costs for the Air Force's
C-17 aircraft that had been prepared at the request of the Office of
the Under Secretary of Defense for Acquisition, Technology and
Logistics. For the five weapon systems in our sample where production
milestone life-cycle O&S cost estimates were unavailable, we obtained
and analyzed data on actual O&S costs from the services' VAMOSC
systems. This analysis was subject to the limitations in the data that
we identified for each of the services' VAMOSC systems, as discussed
in the report. We met with officials responsible for each selected
weapon system to discuss issues related to the management of the
program and cost trends. In our analysis of O&S costs, we have
adjusted DOD data to reflect constant fiscal year 2010 dollars, unless
otherwise noted.[Footnote 71] Throughout this report, all percentage
calculations are based on unrounded numbers.
To identify efforts taken by DOD to reduce O&S costs, we interviewed
cognizant OSD and service officials involved in weapon system
acquisition, logistics, and program management. For specific
initiatives, we obtained documents that described their objectives,
time frames, and other information. In addition, we obtained and
reviewed pertinent guidance on performance management and internal
control practices in the federal government. We also reviewed a report
issued in November 2009 by the DOD Product Support Assessment Team.
Finally, we also consulted prior O&S studies performed by DOD, the
services' audit entities, and GAO.
During our review, we conducted work at the DOD and service offices as
shown in table 4 (located in the Washington, D.C., area unless
indicated otherwise).
Table 4: Organizations Contacted to Obtain Information on Major Weapon
System Operating and Support Costs:
Office of the Secretary of Defense:
* Office of the Assistant Deputy Under Secretary of Defense (Materiel
Readiness);
* Office of the Director, Defense Research and Engineering, Systems
Engineering;
* Office of the Director, Acquisition Resources and Analysis,
Acquisition Management;
* Office of the Director, Cost Assessment and Program Evaluation.
Joint Staff, Joint Chiefs of Staff:
Air Force:
* Directorate of Resource Integration, Weapon Systems Readiness
Division, Deputy Chief of Staff for Logistics, Installations and
Mission Support;
* Directorate of Transformation, Transformation Management Division,
Deputy Chief of Staff for Logistics, Installations and Mission Support;
* Air Force Fleet Viability Board, Deputy Chief of Staff for
Logistics, Installations and Mission Support, Wright-Patterson Air
Force Base, Ohio;
* Directorate of Studies, Analyses, Assessments and Lessons Learned;
* Directorate for Acquisition Integration, Assistant Secretary of the
Air Force for Acquisition;
* Air Force Cost Analysis Agency;
* Air Combat Command, Langley Air Force Base, Virginia;
* Air Force Materiel Command, Wright-Patterson Air Force Base, Ohio;
* Aeronautical Systems Center, Wright-Patterson Air Force Base, Ohio;
* F-15 Development Program Office (912th Aeronautical Systems Group),
Wright-Patterson Air Force Base, Ohio;
* B-1 Development Program Office (812th Aeronautical Systems Group),
Wright-Patterson Air Force Base, Ohio;
* F-22 System Program Office (478th Aeronautical Systems Group),
Wright-Patterson Air Force Base, Ohio;
* Oklahoma City Air Logistics Center, Tinker Air Force Base, Oklahoma;
* Warner-Robins Air Logistics Center, Robins Air Force Base, Georgia;
Army:
* Deputy Chief of Staff for Logistics
* Office of the Deputy Assistant Secretary of the Army for Acquisition
Policy and Logistics;
* Program Executive Office-Aviation, Redstone Arsenal, Alabama;
* Apache Helicopters Project Manager's Office, Redstone Arsenal,
Alabama;
* Cargo Helicopters Project Manager's Office, Redstone Arsenal,
Alabama;
* Utility Helicopters Project Manager's Office, Redstone Arsenal,
Alabama;
* Office of the Deputy Assistant Secretary of the Army for Cost and
Economics;
* Army Materiel Command Headquarters;
* Army Aviation and Missile Life Cycle Management Command, Redstone
Arsenal, Alabama;
* Aviation and Missile Research Development and Engineering Center,
Redstone Arsenal, Alabama.
Navy:
* Deputy Chief of Naval Operations (Fleet Readiness and Logistics);
* Deputy Chief of Naval Operations (Integration of Capabilities and
Resources);
* Assistant Secretary of the Navy (Research, Development, and
Acquisition);
* Naval Center for Cost Analysis;
* Naval Air Systems Command, F/A-18 Program Office, Naval Air Station
Patuxent River, Maryland;
* Naval Air Systems Command, Cost Department, Naval Air Station
Patuxent River, Maryland.
Other organizations:
* Center for Naval Analyses;
* Institute for Defense Analyses.
Source: GAO.
[End of table]
We conducted this performance audit from June 2009 through July 2010
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
[End of section]
Appendix II: Results of OSD-Sponsored Analysis of C-17 Aircraft:
This appendix provides further information on an Office of the
Secretary of Defense-sponsored study of operating and support (O&S)
cost growth for the Air Force's C-17 aircraft. The Institute for
Defense Analyses (IDA) conducted the study for the Office of the Under
Secretary of Defense for Acquisition, Technology and Logistics.
According to an IDA analyst, the study began in 2007 and was completed
in April 2009. We did not evaluate the study's methodology, results,
or conclusions.
The intent of the study was to demonstrate various analytic methods
for monitoring major weapon system reliability, maintainability,
availability, and O&S costs against baseline targets throughout the
life cycle. IDA obtained O&S cost estimates developed by the Air Force
during the acquisition of the C-17,[Footnote 72] compared them to
actual fiscal year 2009 O&S costs (estimated using DOD's recommended
cost element structure), and developed an updated life-cycle cost
estimate using actual O&S cost data.[Footnote 73] In its report, IDA
showed that the C-17's estimated life-cycle O&S costs increased from
$91.6 billion to $118.1 billion (29 percent) from 1985 through 2009.
[Footnote 74] The estimated cost growth occurred despite a decrease in
the total aircraft inventory from a projected 210 down to an actual
total of 190.[Footnote 75] Further, the study reported that the C-17's
cost per flight hour increased 43 percent from an estimated $13,989 in
1985 to an estimated $19,995 in 2009. According to the study, major
cost drivers were fuel consumption, materials and supplies, repair
parts, airframe overhaul, engine overhaul, and sustaining
engineering/program management.
According to IDA's report, the C-17 program experienced changes during
and after acquisition that affected the comparison of the updated O&S
cost estimates--developed using actual O&S costs--to the originally
estimated O&S costs. The report grouped the factors that caused O&S
cost growth into three categories: internal program factors, external
program factors, and accounting factors.
* According to an IDA analyst involved with the study, variances due
to internal program factors are defined as those that were influenced
by the aircraft's program managers. Such factors identified in the
study included system design, reliability, and maintenance support
concepts. For example, the report noted that the C-17 transitioned
from planned government-provided support to contractor logistics
support, and this change greatly complicated the analysis and became a
major aspect of the study. IDA attributed cost increases for
sustaining engineering/program management, contractor field service
representatives, contractor training support, and engine depot-
maintenance costs to this change in support concept. Further, the C-
17's airframe weight increased during development, which led to
increased fuel consumption and higher fuel costs.[Footnote 76]
Finally, system modifications increased in scope, which led to
additional cost increases.
* Changes in costs due to external program factors are defined as
those that were generally beyond the control of program managers,
according to the IDA analyst. These factors included changes to system
quantities or delivery schedules, basing and deployment plan changes,
and higher system-operating tempos due to contingencies. For example,
the change from 210 to 190 aircraft reduced total costs; a change to
the mix of active and reserve units from 73 percent active to 90
percent active increased costs; and personnel costs increased due to
growth in incentive pay, housing, and medical care costs.
* Finally, according to the IDA analyst, variances from accounting
factors are defined as those that resulted from differences in the way
costs were categorized over time. Accounting factor changes that
affected C-17 O&S costs included a change in the scope of DOD's
indirect costs; changes in personnel accounting; and changes to the
timing of the weapon system's phase-in, steady state, and phase-out
periods.
On the basis of its C-17 analysis, IDA concluded that any mechanism to
track and assess weapon system O&S costs against baseline estimates
would require a systematic and institutional methodology that does not
currently exist within DOD. According to the report, the
methodological approach that was used in the study was ad hoc, labor
intensive, and dependent on analyst judgment. The study suggested
that, in the absence a more systematic, institutional methodology, DOD
could instead track major O&S cost drivers--such as reliability, fuel
consumption, maintenance manning per aircraft, and dollars per
airframe overhaul. However, the exact metrics DOD used would depend on
how the department plans to use the data in managing the O&S costs of
its weapon systems and how the data would be used in decision making.
[End of section]
Appendix III: Analysis of Changes between Estimated and Actual O&S
Costs for the Navy's F/A-18E/F:
This appendix provides a detailed breakdown, by cost element, of total
estimated and actual operating and support (O&S) costs for the Navy's
F/A-8E/F for the period of fiscal years 1999 through 2009 (see table
5). The estimated costs were obtained from the Navy's O&S life-cycle
cost estimates prepared for the 1999 production milestone. Data on
actual O&S costs were obtained from the Navy's Visibility and
Management of Operating and Support Costs (VAMOSC) system.
Table 5: Comparison of Navy F/A-18E/F Total Estimated and Actual O&S
Costs for Fiscal Years 1999-2009:
Constant fiscal year 2010 dollars in millions.
Cost element: Manpower;
Total estimated O&S costs, fiscal years 1999-2009: $2,235;
Percent of total estimated costs: 25%;
Total actual O&S costs, fiscal years 1999-2009: $2,031;
Percent of total actual costs: 23%;
Change in total O&S costs: -$204;
Percent change: -9%.
Cost element: Unit-level operations;
Total estimated O&S costs, fiscal years 1999-2009: $3,573;
Percent of total estimated costs: 41[A];
Total actual O&S costs, fiscal years 1999-2009: $4,259;
Percent of total actual costs: 48[A];
Change in total O&S costs: $685;
Percent change: 19.
Cost element: Unit-level operations: Fuel;
Total estimated O&S costs, fiscal years 1999-2009: $792;
Percent of total estimated costs: 9;
Total actual O&S costs, fiscal years 1999-2009: $2,188;
Percent of total actual costs: 25;
Change in total O&S costs: $1,395;
Percent change: 176.
Cost element: Unit-level operations: Materials and supplies;
Total estimated O&S costs, fiscal years 1999-2009: $760;
Percent of total estimated costs: 9;
Total actual O&S costs, fiscal years 1999-2009: $555;
Percent of total actual costs: 6;
Change in total O&S costs: -$205;
Percent change: -27.
Cost element: Unit-level operations: Repair parts;
Total estimated O&S costs, fiscal years 1999-2009: $1,639;
Percent of total estimated costs: 19;
Total actual O&S costs, fiscal years 1999-2009: $1,363;
Percent of total actual costs: 16;
Change in total O&S costs: -$276;
Percent change: -17.
Cost element: Unit-level operations: Training expendable stores;
Total estimated O&S costs, fiscal years 1999-2009: $382;
Percent of total estimated costs: 4;
Total actual O&S costs, fiscal years 1999-2009: $153;
Percent of total actual costs: 2;
Change in total O&S costs: -$229;
Percent change: -60.
Cost element: Intermediate maintenance;
Total estimated O&S costs, fiscal years 1999-2009: $86;
Percent of total estimated costs: 1;
Total actual O&S costs, fiscal years 1999-2009: $452;
Percent of total actual costs: 5;
Change in total O&S costs: $366;
Percent change: 428.
Cost element: Depot maintenance;
Total estimated O&S costs, fiscal years 1999-2009: $280;
Percent of total estimated costs: 3;
Total actual O&S costs, fiscal years 1999-2009: $723;
Percent of total actual costs: 8;
Change in total O&S costs: $443;
Percent change: 159.
Cost element: Contractor support;
Total estimated O&S costs, fiscal years 1999-2009: $0;
Percent of total estimated costs: 0;
Total actual O&S costs, fiscal years 1999-2009: $79;
Percent of total actual costs: 1;
Change in total O&S costs: $79;
Percent change: [B].
Cost element: Sustaining support;
Total estimated O&S costs, fiscal years 1999-2009: $2,638;
Percent of total estimated costs: 30[A];
Total actual O&S costs, fiscal years 1999-2009: $1,139;
Percent of total actual costs: 13[A];
Change in total O&S costs: -$1,499;
Percent change: -57.
Cost element: Sustaining support: Sustaining engineering;
Total estimated O&S costs, fiscal years 1999-2009: $128;
Percent of total estimated costs: 2;
Total actual O&S costs, fiscal years 1999-2009: $14;
Percent of total actual costs: [C];
Change in total O&S costs: -$114;
Percent change: -89.
Cost element: Sustaining support: Modifications;
Total estimated O&S costs, fiscal years 1999-2009: $742;
Percent of total estimated costs: 8;
Total actual O&S costs, fiscal years 1999-2009: $946;
Percent of total actual costs: 11;
Change in total O&S costs: $204;
Percent change: 27.
Cost element: Sustaining support: Software maintenance;
Total estimated O&S costs, fiscal years 1999-2009: $71;
Percent of total estimated costs: 1;
Total actual O&S costs, fiscal years 1999-2009: $40;
Percent of total actual costs: 1;
Change in total O&S costs: -$30;
Percent change: -43.
Cost element: Sustaining support: Simulator operations;
Total estimated O&S costs, fiscal years 1999-2009: $62;
Percent of total estimated costs: 1;
Total actual O&S costs, fiscal years 1999-2009: $17;
Percent of total actual costs: [C];
Change in total O&S costs: -$44;
Percent change: -72.
Cost element: Sustaining support: Training;
Total estimated O&S costs, fiscal years 1999-2009: $1,635;
Percent of total estimated costs: 19;
Total actual O&S costs, fiscal years 1999-2009: $45;
Percent of total actual costs: 1;
Change in total O&S costs: -$1,591;
Percent change: -97.
Cost element: Sustaining support: Other;
Total estimated O&S costs, fiscal years 1999-2009: $0;
Percent of total estimated costs: 0;
Total actual O&S costs, fiscal years 1999-2009: $77;
Percent of total actual costs: 1;
Change in total O&S costs: $77;
Percent change: [B].
Cost element: Indirect support;
Total estimated O&S costs, fiscal years 1999-2009: $0;
Percent of total estimated costs: 0;
Total actual O&S costs, fiscal years 1999-2009: $36;
Percent of total actual costs: [C];
Change in total O&S costs: $36;
Percent change: [B].
Cost element: Total;
Total estimated O&S costs, fiscal years 1999-2009: $8,811;
Percent of total estimated costs: 100%;
Total actual O&S costs, fiscal years 1999-2009: $8,719;
Percent of total actual costs: 100%[D];
Change in total O&S costs: -$92;
Percent change: -1%.
Source: GAO analysis of DOD data.
[A] The percentages for the cost subelements listed under the unit-
level operations cost element and the sustaining support cost element
are shown separately and are also rolled up into the overall
percentages for these two cost elements.
[B] Since these costs were not included in the production milestone
estimate, a percentage increase or decrease could not be calculated.
[C] Percentage is less than 1 percent.
[D] May not add to 100 due to rounding.
[End of table]
[End of section]
Appendix IV: Comments from the Department of Defense:
Office Of The Assistant Secretary Of Defense:
Logistics And Materiel Readiness:
3500 Defense Pentagon:
Washington, DC 20301-3500:
June 29, 2010:
Mr, Jack E. Edwards:
Director, Defense Capabilities and Management:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, DC 20548:
Dear Mr. Edwards:
This is the Department of Defense (DoD) response to the GAO draft
report, GAO-10-717, "Defense Management: DoD Needs Better information
and Guidance to More Effectively Manage and Reduce Operating and
Support Costs of Major Defense Systems, dated May 21, 2010 (GAO Code
351370).
The Department generally concurs with the recommendations presented in
the report on more effectively managing and reducing O&S costs for
Major Defense Acquisition Programs (MDAPs). There are over 150 MDAPs
across the departments and agencies, ranging from missile defense
systems to combat vehicles. with each MDAP having unique challenges in
data reporting. The report is reasonable in its analysis of the seven
programs reviewed, but it does not necessarily follow that problems
encountered with that sample are to be found across the entire
Department. The Department requests that this point be highlighted in
GAO's report.
While the GAO report also indicates that the Department has made
significant changes, these changes arc generally not reflected in the
systems analyzed. Recent changes focus on tracking and preventing
future cost growth. GAO missed an opportunity to review pre-MDAPs to
assess the potential effectiveness of these initiatives. for instance,
for the Joint Lightweight Tactical Vehicle program, reliability and
maintainability are being emphasized in the design and engineering
phase of the program, with the goal of reducing future maintenance
cost and optimizing vehicle readiness, consequently reducing long-term
operating and support (O&S) costs. A review of at least one pre-MDAP
would have shed light on the long-term impact recent initiatives
should have on controlling O&S cost growth.
Further. the Department will take steps to update policy to ensure
that O&S cost estimates are retained along with supporting
documentation. The Department is committed to strengthening its O&S
data availability, as well as its use of O&S estimates in the
governance process for MDAPs at each major Milestone.
Detailed DoD comments on the draft GAO recommendations are provided in
the attachment. The DoD appreciates the opportunity to comment on the
draft report.
Sincerely,
Signed by:
Alan F. Estevez:
Principal Deputy:
Attachment: As stated:
[End of letter]
GAO Draft Report — Dated May 21, 2010:
GAO Code 351370/GAO-10-717:
"Defense Management: DoD Needs Better Information and Guidance to
More Effectively Manage and Reduce Operating and Support Costs of Major
Defense Systems"
Department Of Defense Comments To The Recommendations:
Recommendation 1: The GAO recommends that the Secretary of Defense
direct the Under Secretary of Defense (Acquisition, Technology and
Logistics) (USD(AT&L)) to revise DoD guidance to require the Services
to retain life-cycle operating and support (O&S) cost estimates and
support documentation used to develop the cost estimates for major
weapon systems. This requirement should apply to cost estimates
developed by weapon system program offices and other Service offices,
including cost analysis organizations. Furthermore, this requirement
should include cost estimates prepared during the acquisition process
as well as those prepared after a system is fielded.
DOD Response: Concur. This effort will require coordination between
USD(AT&L) and the Director of the Cost Assessment and Program
Evaluation.
Recommendation 2: The GAO recommends that the Secretary of Defense
direct the Under Secretary of Defense (Acquisition, Technology and
Logistics) (USD(AT&L)) to identify the cost elements needed to track
and assess major weapon systems actual operating and support (O&S)
costs for effective cost analysis and program management, and require
the Services to collect and maintain these elements in their
visibility and management of operating and support cost (VA.MOSC)
systems. To the extent possible, data collected on actual O&S costs
should he comparable to data presented in life-cycle cost estimates,
To oversee compliance with this new requirement, DoD should require
the Services to identify any gaps where actual cost data are not being
collected and maintained and to identify efforts, along with timelines
and resources, for filling these gaps.
DOD Response: Partially Concur. The Department agrees with the intent
of the GAO recommendation; however the Department recommends that the
Director of the Cost Assessment and Program Evaluation. Office of the
Secretary of Defense be directed to take this action in lieu of
USD(AT&L). This effort will require coordination between USD(AT&L) and
the Director of the Cost Assessment and Program Evaluation.
Recommendation 3: The GAO recommends that the Secretary of Defense
direct the Under Secretary of Defense (Acquisition. Technology and
Logistics) to direct the Army to develop and implement a strategy for
improving its Visibility and Management of Operating and Support Cost
(VAMOSC) System. This strategy should include plans for incorporating
additional cost elements from other information systems. time frames
for expanding on existing cost elements, and resources required to
improve the VAMOSC system.
DOD Response: Partially Concur. The Army will develop and implement a
strategy to improve its' VAIVIOSC system, consistent with the comments
below:
The magnitude of Army weapon systems (over 1 million) makes it very
difficult to capture every O&S cost element by system. The growth in
weapon system operating and support (O&S) costs is primarily related
to OPTEMPO costs, which the Army tracks in its VAMOSC system.
In regards to Military Personnel (MILPERS) costs, these costs are not
captured by fleet in the Army. The Deputy Assistant Secretary of the
Army for Cost and Economics (DASA(CE)) maintains the Army Military-
Civilian Cost System (AMCOS) that provides MILPERS costs for accurate
cost estimating purposes. Since manpower requirements for individual
systems are known, AMCOS has provided what is required for the
military personnel cost portion of system O&S cost estimating.
With respect to Contractor Logistic Support (CLS), the Army has
provided guidance on the reporting of contractor logistic support
costs. Army Regulation 700-142 (dated 26 March 2008) paragraph 2-
15(b)(4)(0 now requires the PM to coordinate with DASA(CE) and Deputy
Chief of Staff, G3/5/7, Training Directorate (DAMO-TR), to determine
if the system being fielded warrants a demand-based cost factor
developed by DASA(CE) in order to generate out-year operations and
maintenance funding for support. The PM must then provide updated cost
data to the DASA(CE) during the life of the system. This has resulted
in program managers adding CLS reporting requirements into the CLS
contracts. Significant progress has been made in capturing CLS costs
for Operating and Support Management Information System (OSMIS) — in
addition to Stryker and Light Utility Helicopter (LUI-1), Mine
Resistant Ambush Protected (MRAP) and Unmanned Aerial Systems (UAS)
that are currently reporting data to the Army VAMOSC system, new CLS
contracts include the requirement to report data. CLS reporting by
weapon system was designed into the Army's financial new system - CLS
reporting by weapon system was designed into General Fund Enterprise
Business System.
Recommendation 4: The GAO recommends that the Secretary of Defense
direct the Under Secretary of Defense (Acquisition, Technology and
Logistics) (USD(AT&L)) to require the Services to periodically update
their life-cycle operating and support (O&S) cost, estimates for major
weapon systems throughout their life cycle. These updates should
provide an assessment of cost growth since the prior estimate was
developed and account for any significant cost and program changes.
DOD Response: Concur. The Department of Navy views with concern that
this valid and necessary requirement that will require additional
resources in terms of cost and personnel across the cost estimating
and data provider organizations. This effort will require coordination
between USD(AT&L) and the Director of the Cost Assessment and Program
Evaluation.
Recommendation 5: The GAO recommends that the Secretary of Defense
direct the Under Secretary of Defense (Acquisition, Technology and
Logistics) (USD(AT&L)) to develop guidance for documenting and
retaining historical information on weapon system program changes to
aid in effective analysis of operating and support costs. DoD should
determine, in conjunction with Service acquisition and cost analysis
officials, the types of information needed and the level of detail
that should be retained.
DOD Response: Concur. This effort will require coordination between
USD(A F&L) and the Director of the Cost Assessment and Program
Evaluation.
Recommendation 6: The GAO recommends that the Secretary of Defense
require that the Director of OSD Cost Assessment and Program
Evaluation retain any independent life-cycle operating and support
(O&S) cost estimates prepared by that office along with support
documentation used to develop these cost estimates for major weapon
systems.
DOD Response: Concur.
[End of section]
Appendix V: GAO Contact and Staff Acknowledgments:
GAO Contact:
Jack Edwards, (202) 512-8246 or edwardsj@gao.gov:
Staff Acknowledgments:
In addition to the contact name above, the following staff members
made key contributions to this report: Tom Gosling, Assistant
Director; Tracy Burney; Sandra Enser; Kevin Keith; James Lackey;
Charles Perdue; Richard Powelson, Janine Prybyla; Jennifer Spence; and
Alyssa Weir.
[End of section]
Related GAO Products:
Joint Strike Fighter: Additional Costs and Delays Risk Not Meeting
Warfighter Requirements on Time. [hyperlink,
http://www.gao.gov/products/GAO-10-382]. Washington, D.C.: March 19,
2010.
Defense Acquisitions: Assessments of Selected Weapon Programs.
[hyperlink, http://www.gao.gov/products/GAO-10-388SP]. Washington,
D.C.: March 30, 2010.
Littoral Combat Ship: Actions Needed to Improve Operating Cost
Estimates and Mitigate Risks in Implementing New Concepts. [hyperlink,
http://www.gao.gov/products/GAO-10-257]. Washington, D.C.: February 2,
2010.
Defense Acquisitions: Army Aviation Modernization Has Benefited from
Increased Funding but Several Challenges Need to be Addressed.
[hyperlink, http://www.gao.gov/products/GAO-09-978R]. Washington,
D.C.: September 28, 2009.
Defense Acquisitions: Assessments Needed to Address V-22 Aircraft
Operational and Cost Concerns to Define Future Investments.
[hyperlink, http://www.gao.gov/products/GAO-09-482]. Washington, D.C.:
May 11, 2009.
GAO Cost Estimating and Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs. [hyperlink,
http://www.gao.gov/products/GAO-09-3SP]. Washington, D.C.: March 2009.
Defense Logistics: Improved Analysis and Cost Data Needed to Evaluate
the Cost-effectiveness of Performance Based Logistics. [hyperlink,
http://www.gao.gov/products/GAO-09-41]. Washington, D.C.: December 19,
2008.
Missile Defense: Actions Needed to Improve Planning and Cost Estimates
for Long-Term Support of Ballistic Missile Defense. [hyperlink,
http://www.gao.gov/products/GAO-08-1068]. Washington, D.C.: September
25, 2008.
Defense Acquisitions: Fundamental Changes Are Needed to Improve Weapon
Program Outcomes. [hyperlink,
http://www.gao.gov/products/GAO-08-1159T]. Washington, D.C.: September
25, 2008.
Military Base Closures: Projected Savings from Fleet Readiness Centers
Likely Overstated and Actions Needed to Track Actual Savings and
Overcome Certain Challenges. [hyperlink,
http://www.gao.gov/products/GAO-07-304]. Washington, D.C.: June 29,
2007.
Air Force Depot Maintenance: Improved Pricing and Cost Reduction
Practices Needed. [hyperlink, http://www.gao.gov/products/GAO-04-498].
Washington, D.C.: June 17, 2004.
Military Personnel: Navy Actions Needed to Optimize Ship Crew Size and
Reduce Total Ownership Costs. [hyperlink,
http://www.gao.gov/products/GAO-03-520]. Washington, D.C.: June 9,
2003.
Best Practices: Setting Requirements Differently Could Reduce Weapon
Systems' Total Ownership Costs. [hyperlink,
http://www.gao.gov/products/GAO-03-57. Washington, D.C.: February 11,
2003.
Defense Logistics: Opportunities to Improve the Army's and the Navy's
Decision-making Process for Weapons System Support. [hyperlink,
http://www.gao.gov/products/GAO-02-306]. Washington, D.C.: February
28, 2002.
Defense Acquisitions: Navy and Marine Corps Pilot Program Initiatives
to Reduce Total Ownership Costs. [hyperlink,
http://www.gao.gov/products/GAO-01-675R]. Washington, D.C.: May 22,
2001.
Defense Acquisitions: Higher Priority Needed for Army Operating and
Support Cost Reduction Efforts. [hyperlink,
http://www.gao.gov/products/GAO/NSIAD-00-197]. Washington, D.C.:
September 29, 2000.
Defense Acquisitions: Air Force Operating and Support Cost Reductions
Need Higher Priority. [hyperlink,
http://www.gao.gov/products/GAO/NSIAD-00-165]. Washington, D.C.:
August 29, 2000.
Financial Systems: Weaknesses Impede Initiatives to Reduce Air Force
Operations and Support Costs. [hyperlink,
http://www.gao.gov/products/GAO/NSIAD-93-70]. Washington, D.C.:
December 1, 1992.
Navy Fielded Systems: Operating and Support Costs Not Tracked.
[hyperlink, http://www.gao.gov/products/GAO/NSIAD-90-246]. Washington,
D.C.: September 28, 1990.
[End of section]
Footnotes:
[1] The department recently estimated that weapon system product
support costs in fiscal year 2008 were at least $132 billion.
According to DOD, product support encompasses materiel management,
distribution, technical data management, maintenance, training,
cataloging, configuration management, engineering support, repair
parts management, failure reporting and analysis, and reliability
growth. However, this amount does not include all O&S costs.
[2] Pub. L. No. 111-23 (2009).
[3] Major defense acquisition programs are estimated by the Under
Secretary of Defense for Acquisition, Technology and Logistics to
require an eventual total expenditure for Research, Development, Test
and Evaluation of more that $365 million, including all planned
increments, based on fiscal year 2000 constant dollars (approximately
$509 million in fiscal year 2010 dollars), $2.190 billion of
Procurement funding, including all planned increments (approximately
$3.054 billion in fiscal year 2010 dollars), or are designated as a
major defense acquisition program by the milestone decision authority.
Throughout this report, we use the term major weapon system to
describe a major defense acquisition program.
[4] Specifically, section 304 of the Act requires that we identify the
original O&S cost estimates for selected major defense weapon systems;
assess the actual O&S costs and the rate of growth; assess the factors
contributing to high rates of O&S cost growth; and assess measures
taken by DOD to reduce O&S costs for major weapon systems.
[5] The Act created this position, which includes many of the
functions formerly performed by the Director of Program Analysis and
Evaluation.
[6] We did not include a Marine Corps aviation system in our sample
because the Naval Air Systems Command manages and supports all Marine
Corps aircraft. References to the military services in this report do
not include the Marine Corps.
[7] For example, we previously reported that detailed historical cost
data were not available for many weapon systems supported under
performance-based logistics arrangements.
[8] While life-cycle cost estimates may be developed during earlier
stages of the acquisition process, DOD guidance states that the cost
estimate for the production milestone should be based on the current
design characteristics of the weapon system, the latest deployment
schedule, and the current logistics and training support plans. The
production milestone is also known as milestone C or, for older
systems, milestone III.
[9] DOD, Weapon System Acquisition Reform Product Support Assessment
(November 2009).
[10] The principal DOD policies on the weapon system acquisition
process are DOD Directive 5000.01, The Defense Acquisition System (May
12, 2003), and DOD Instruction 5000.02, Operation of the Defense
Acquisition System (Dec. 8, 2008).
[11] DOD's acquisition policies and terminology related to major
decision reviews have varied over the years. Most of our selected
weapon systems passed through the acquisition process prior to the
current structure. Although under the current process the milestone
names have changed, the milestone reviews held in the past similarly
focused on approvals for concept demonstration, system development,
and production.
[12] OSD, Operating and Support Cost Estimating Guide (October 2007)
and Operating and Support Cost Estimating Guide (May 1992).
[13] The 2009 Weapon Systems Acquisition Reform Act transfers the
functions and personnel of the Cost Analysis Improvement Group to the
Deputy Director for Cost Assessment, under the OSD Director for Cost
Assessment and Program Evaluation. For the purposes of this report, we
will refer to the current organization.
[14] DOD Directive 5000.04, Cost Analysis Improvement Group (Aug. 16,
2006).
[15] OSD, Operating and Support Cost Estimating Guide (October 2007).
[16] The Navy's VAMOSC system collects O&S costs for both Navy and
Marine Corps systems.
[17] S. Rep. No. 100-235, at 53 (Dec. 4, 1987).
[18] DOD 5000.4-M, DOD Cost Analysis Guidance and Procedures (December
1992).
[19] See the Related GAO Products section at the end of this report.
[20] Requirements for new weapon system capabilities are expressed as
key performance parameters, which are attributes or characteristics of
a system that are considered critical or essential to the development
of an effective military capability and make a significant
contribution to the characteristics of the future joint force.
[21] Remarks delivered by Secretary of Defense Robert M. Gates in
Abilene, Kansas (May 8, 2010).
[22] The Air Force's acquisition of the B-1B did not follow DOD's
milestone process. Milestone reviews for the B-1B program were not
conducted because the program was simultaneously engaged in full-scale
development and production. Instead of the milestone process, a system
of secretarial program reviews was used that involved the Air Force
briefing the Secretary of Defense on the program status, focusing
primarily on cost and schedule. We were unable to obtain any
acquisition O&S cost estimates for the B-1B, including a cost estimate
that the Air Force had planned to prepare in 1989.
[23] According to OSD Cost Assessment and Program Evaluation
officials, staffing levels did not allow them to begin developing
independent life-cycle cost estimates until the early 1990's. Although
they could not provide the estimate for the AH-64D, these officials
stated they developed an independent cost estimate for a later upgrade
to the program (Block 2), and are currently developing one for a new
upgrade (Block 3). When we requested that this office provide initial
and updated O&S cost estimates for the AH-64D during our review, we
were not informed of the existence of the Block 2 cost estimate or
provided this estimate.
[24] GAO, GAO Cost Estimating and Assessment Guide, [hyperlink,
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009).
[25] For this analysis, we reviewed guidance in DOD Directive 5000.01,
DOD Instruction 5000.02, DOD Directive 5000.04, DOD 5000.4-M, and
OSD's Operating and Support Cost Estimating Guide.
[26] Secretary of the Navy Instruction 5000.2D, Implementation and
Operation of the Defense Acquisition System and the Joint Capabilities
Integration and Development System (Oct. 16, 2008) and Secretary of
the Navy Instruction 5223.2, Department of the Navy Cost Analysis
(Dec. 16, 2008).
[27] Secretary of the Navy Manual M-5210.1, Records Management Manual
(November 2007 Revision).
[28] The guidance broadly describes categories of records and does not
explicitly include requirements for retaining acquisition cost
estimates, other than for ships.
[29] For example, the flying hours contained in the Cost Analysis
Requirements Description did not match the flying hours used to
develop the cost estimate. The Cost Analysis Requirements Description
is a complete description of the system whose costs are to be
estimated; it is intended to define the program to a sufficient level
of detail such that no confusion exists between the many parties who
may be concerned with estimating the program's cost. Additionally,
some data were not complete--there was not enough detail provided to
substantiate the factors chosen, the regression formulas used, or the
exact inflation factors that were employed.
[30] Air Force Instruction 63-101, Acquisition and Sustainment Life
Cycle Management (Apr. 17, 2009).
[31] Air Force Instruction 63-101; Air Force Instruction 63-1201,
Acquisition: Life Cycle Systems Engineering (July 23, 2007); Air Force
Instruction 65-508, Financial Management: Cost Analysis Guidance and
Procedures (Oct. 1, 1997); Air Force Policy Directive 65-5, Financial
Management: Cost and Economics (Aug. 5, 2008).
[32] Army Regulation 70-1, Army Acquisition Policy: Research,
Development and Acquisition (December 2003). This guidance is located
in an appendix to the regulation.
[33] Although the F-22A was supported by interim contractor support
before fiscal year 2006, OSD cost-estimating guidance states that
postoperational contractor support is an O&S cost. The F-22A reached
initial operating capability in December 2005. According to program
officials, subsequent interim contractor support costs were $730.6
million in calendar year 2006 and $632.1 in calendar year 2007. (We
adjusted DOD data to reflect constant fiscal year 2010 dollars.) While
not all of these costs may be O&S costs, none of the amounts were
included in the Air Force's VAMOSC system.
[34] As noted previously in our discussion of the F-22A, OSD cost-
estimating guidance states that postoperational contractor support is
an O&S cost. According to budget data provided by Navy officials,
interim contractor support costs could be as high as $86.6 million in
fiscal year 2000, $75.3 million in fiscal year 2001, and $73.3 million
in fiscal year 2002 (in then year dollars). However, these amounts
also include some funding for government activities. Further,
additional amounts for interim contractor support were funded out of
other budget line items but the amounts were unknown.
[35] While the use of contractor logistics support arrangements,
including performance-based logistics arrangements, also affect the
level of detail of costs reported in the Air Force and Navy VAMOSC
systems, these systems generally include contractor logistics support
costs in the aggregate. DOD relies heavily on these arrangements to
support selected weapon systems. We have previously reported that
detailed support cost data were often not available for DOD weapon
systems supported by contractor logistics support and performance-
based logistics arrangements. GAO, Defense Logistics: Improved Data
and Cost Analysis Needed to Evaluate the Cost-Effectiveness of
Performance Based Logistics Arrangements, GAO-09-41 (Washington, D.C.:
Dec. 19, 2008).
[36] GAO, Defense Acquisitions: Higher Priority Needed for Army
Operating and Support Cost Reduction Efforts, [hyperlink,
http://www.gao.gov/products/GAO/NSIAD-00-197] (Washington, D.C.: Sept.
29, 2000).
[37] Office of Management and Budget, Circular A-94, Guidelines and
Discount Rates for Benefit-Cost Analysis of Federal Programs (Oct. 29,
1992).
[38] [hyperlink, http://www.gao.gov/products/GAO-09-3SP].
[39] Throughout this report, all percentage calculations are based on
unrounded numbers.
[40] We adjusted DOD data to reflect constant fiscal year 2010
dollars. Further, the Air Force Cost Analysis Agency used the cost
data available in the Air Force VAMOSC system at the time of their
review. The fiscal year 2007 cost per flight hour currently reported
in the Air Force's VAMOSC system for the F-22A, in constant fiscal
year 2010 dollars, is $54,690.
[41] According to Air Force officials, the F-22A is currently expected
to reach maturity in late 2010, when the fleet has reached 100,000
flight hours.
[42] An "engine cycle" refers to one movement of the throttle from
idle to full power and then back to idle.
[43] According to program officials, during fiscal year 2008 the Air
Force modified the F-22A support contract and began requiring the
contractor to provide detailed cost reports to the program office by
fiscal year for both the airframe and the engines. However, officials
told us they did not yet have the finalized reports for fiscal year
2009 for use in our analysis. Air Force cost analysis officials
confirmed the data were not in the database that contains these
reports.
[44] According to the officials, the Air Force uses actual maintenance
data to periodically reassess the personnel necessary to support an
aircraft. In 2007, the Air Force performed such an assessment for the
F-22A, resulting in an increase of maintenance personnel assigned to
the F-22A.
[45] According to Navy officials, when F/A-18E/F aircraft are assigned
to tanking duty and obtain fuel both for aircraft consumption and
delivery to other aircraft via their tanking system, the total cost of
this fuel is charged to the tanking squadron and is reported in the
Navy's visibility system.
[46] In 2004, we reported that the per-hour price charged by the Air
Force's Air Logistics Center for F-15E repair and maintenance
increased 42 percent in 2003 and that the cost increases were due
largely to higher material costs, which accounted for 67 percent of
the increase. See GAO, Air Force Depot Maintenance: Improved Pricing
and Cost Reduction Practices Needed, [hyperlink,
http://www.gao.gov/products/GAO-04-498] (Washington, D.C.: June 17,
2004).
[47] In our 2004 report, [hyperlink,
http://www.gao.gov/products/GAO-04-498], we found the per hour price
charged by the Air Logistics Centers for work performed on the B-1B
increased 65 percent in 2003.
[48] We used fiscal year 1998 data as our starting point because that
was the first year that data for all three systems were available in
the Army's VAMOSC system. We ended our analysis in fiscal year 2007
because the Army had not yet transferred its fiscal year 2008 or 2009
data into the cost element structure recommended by DOD's guidance.
[49] According to officials, wartime O&S costs were 60 percent of the
total O&S costs reported in fiscal year 2007 for the AH-64D; 54
percent for the CH-47D; and 72 percent for the UH-60L.
[50] Selected Acquisition Reports, required under 10 U.S.C. § 2432,
are the standard, comprehensive, summary status report of major
defense acquisition programs periodically submitted to Congress. They
contain key cost, schedule, and performance information. The
requirement for submission of a Selected Acquisition Report may be
waived under certain circumstances. DOD began providing Selected
Acquisition Reports to Congress in the late 1960s.
[51] DOD is required to provide Selected Acquisition Reports to
Congress on major weapon systems until 90 percent of the total
quantity of items to be delivered under the program have been
delivered or 90 percent of planned expenditures under the program have
been made.
[52] In some cases, DOD included life-cycle costs in its Selected
Acquisition Reports; however, the life-cycle costs included in the
reports to Congress were not detailed enough to enable us to assess
the reasons for O&S cost growth.
[53] This metric is addressed further in our discussion of DOD
initiatives to address O&S costs.
[54] Army Regulation 70-1; Secretary of the Navy Instruction 5000.2D;
Secretary of the Navy Instruction 5223.2.
[554] Air Force Policy Directive 65-5.
[56] GAO, Standards for Internal Control in the Federal Government,
[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]
(Washington, D.C.: November 1999).
[57] According to officials, projected savings are based on estimated
return on investment developed prior to project funding.
[58] Chairman of the Joint Chiefs of Staff, Manual for the Operation
of the Joint Capabilities Integration and Development System (July 31,
2009).
[59] As noted earlier, a key performance parameter is an attribute or
characteristic of a system that is considered critical or essential to
the development of an effective military capability that makes a
significant contribution to the characteristics of the future joint
force. A key system attribute is considered most critical or essential
for an effective military capability but, since not selected as a key
performance parameter, provides decision makers with an additional
level of capability prioritization below the key performance parameter.
[60] Under Secretary of Defense for Acquisition, Technology and
Logistics, Memorandum for Secretaries of the Military Services,
Implementing a Life Cycle Management Framework (July 31, 2008).
[61] Under Secretary of Defense for Acquisition, Technology and
Logistics, Memorandum for Secretaries of the Military Departments,
Implementation of Life Cycle Sustainment Outcome Metrics Data
Reporting (Dec. 11, 2008).
[62] DOD Instruction 5000.02.
[63] [hyperlink, http://www.gao.gov/products/GAO-09-41].
[64] DOD Instruction 5000.02.
[65] These charts provide requirements and funding data for use in
these reviews. Currently, the charts include this information only for
Procurement; Research, Development, Test and Evaluation; and Military
Construction appropriations.
[66] GAO, Military Base Closures: Projected Savings from Fleet
Readiness Centers Likely Overstated and Actions Needed to Track Actual
Savings and Overcome Certain Challenges, [hyperlink,
http://www.gao.gov/products/GAO-07-304] (Washington, D.C.: June 29,
2007).
[67] DOD, Weapon System Acquisition Reform Product Support Assessment
(November 2009). Product support is defined as the package of support
functions required to maintain the readiness and operational
capability of weapon systems, subsystems, software, and support
systems.
[68] Independent logistics assessments are a compilation of
information that the services recognize as important to their life-
cycle product-support governance.
[69] DOD acquisition category I programs are major defense acquisition
programs or programs that have been designated as such by the
milestone decision authority. An acquisition category ID weapon system
is an acquisition category I program for which the milestone decision
authority is the Under Secretary of Defense for Acquisition,
Technology and Logistics.
[70] Under Secretary of Defense for Acquisition, Technology and
Logistics, Memorandum for Secretaries of the Military Departments,
Strengthened Sustainment Governance for Acquisition Program Reviews
(Apr. 5, 2010).
[71] Constant dollars measure the value of purchased goods and
services at price levels that are the same as those in the base or
reference year. Constant dollars do not contain any adjustments for
inflationary changes that have occurred or are forecast to occur
outside the base year. Therefore, the changes in these dollar amounts
will not be due to inflation.
[72] The study included the 1985 milestone II estimate, the 1989
milestone low-rate production estimate, and the 1995 milestone IIIB
estimate for the C-17.
[73] According to the IDA analyst, since the C-17 has been supported
by a contractor under a performance-based logistics arrangement,
actual historical O&S costs were not available in OSD's recommended
cost element structure. As part of the study, the amounts of the
various cost elements were estimated using the C-17 support contract,
budget data, and other available data in order to update the O&S cost
estimate.
[74] Amounts in constant fiscal year 2009 dollars. The estimates
assume a 30-year service life.
[75] Although the IDA analysis was based on a total of 190 aircraft,
Air Force officials stated that the current C-17 program of record as
of June 2010 is 223 aircraft.
[76] According to IDA's study, the C-17's current aircraft operating
weight is 6 percent higher than that used in the Air Force's 1985
milestone II estimate.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548: