This is the accessible text file for GAO report number GAO-11-746R entitled 'Military Readiness: Navy's Report to Congress on the Impact of Training and Crew Size on Surface Force Material Readiness' which was released on July 7, 2011. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO-11-746R: United States Government Accountability Office: Washington, DC 20548: July 7, 2011: Congressional Committees: Subject: Military Readiness: Navy's Report to Congress on the Impact of Training and Crew Size on Surface Force Material Readiness: Since 2000, the Navy has undertaken a number of initiatives to achieve greater efficiencies and reduce costs. For example, it reduced the workforce requirements for some of its ships and transitioned away from instructor-led training programs to more computer-based training. In June 2010, we reported[Footnote 1] the Navy lacked a firm analytical basis for some of the reductions it made to ship crew sizes and therefore could not be assured it had appropriately sized its crews to maintain material readiness and accomplish necessary tasks aboard its ships. We also reported the Navy lacked outcome-based performance measures to evaluate the impact of changes to training on trainees' job performance, knowledge, skills, and abilities once they report to their ships and therefore could not fully determine the effectiveness of the training changes it implemented and whether further adjustments were necessary. We recommended the Navy validate the underlying assumptions and standards it uses to calculate workforce requirements and, as necessary, based on this assessment, reevaluate its cruiser and destroyer workload requirements. We also recommended the Navy develop additional metrics to measure the effectiveness of its training. The Navy concurred with our recommendations. Citing our previous work and other sources, the House Armed Services Committee has expressed concern about the material readiness of the Navy's surface combatant ships. In House Report 111-491, which accompanied a proposed bill for the Fiscal Year 2011 National Defense Authorization Act (H.R. 5136), the committee directed the Secretary of the Navy to submit a report that describes: * the impact that changes in training and reductions in crew size have had on the material readiness of its ships, including: - the ships' ability to perform required maintenance tasks and pass required inspections; - any projected effects on the lifespan of individual ships; and: - any effects on overall reported readiness; * the methodology, including metrics, that the Navy used to make this assessment, and based on the results, any adjustments in training and manning that the Navy plans to make to address its findings; and: * the steps the Navy has taken to establish a stringent tool-control program, through appropriate commands, for all surface combatant ships similar to the tool-control program that exists for aviation squadrons, and describe the funding required to implement such a program.[Footnote 2] The Navy issued its report to Congress in February 2011.[Footnote 3] In addition, the House report directed GAO to provide a briefing to the Senate and House Armed Services committees that assesses the completeness of the Navy's report and describes the status of actions taken by the Navy to establish the tool-control program; and submit a follow-on report to the congressional defense committees that assesses the reasonableness of the Navy's methodology and conclusions and the impact of the tool-control program. In April 2011, we provided a briefing to the Senate and House Armed Services committees on the completeness of the Navy's report and the status of the tool-control program. We found that the Navy's report addressed or partially addressed each of the mandated reporting requirements. In addition, we found that the Navy had established a limited program to control certain types of equipment, but it had not established a stringent tool-control program for all of its surface combatant ships similar to the tool-control program that exists for aviation squadrons.[Footnote 4] According to the Navy's report and our interviews with Navy officials, the Navy had not implemented such a program for three primary reasons: * Safety--the primary reason for the aviation tool-control program is the concern that loose tools on a flight deck could lead to personnel injury or death. Misplaced tools on a ship do not generally pose the same dangers to personnel. * Cost--a misplaced tool on a flight deck could get drawn into an aircraft engine, which could lead to catastrophic failure that could cost the Navy millions of dollars. Misplaced tools on ships do not generally pose the same risk. * Practicality--the large number and wide distribution of tools required for the maintenance and repair of surface ships makes it impractical to track the location of all tools at all times. This report (1) describes the Navy's methodology for developing its report to Congress and the information presented in its report on conclusions and adjustments the Navy is making to training and manning and (2) assesses the reasonableness of the Navy's methodology and conclusions. As agreed with your staff, because the Navy has not implemented a stringent tool-control program as envisioned in the House report, we were not able to assess its impact and therefore are not addressing the topic in this report. In assessing the reasonableness of the Navy's methodology and conclusions, we reviewed prior GAO work and other sources to identify criteria, including generally accepted research standards and other principles that define a sound and complete quality study. On the basis of this review, we identified specific criteria that were relevant to the Navy's report, which were whether (1) the methodology supported accomplishing the objectives; (2) conclusions and recommendations were supported by analysis; and (3) data were verified and validated. With respect to data, the criteria suggests that data reliability assessments are essential to gathering and evaluating information needed to make a determination to use certain data. Examples include determining the accuracy and completeness of data, and using proper caveats to describe any data limitations. We used these criteria for our assessment. We also interviewed the Navy officials who were responsible for drafting the Navy's report to discuss how the report to Congress was prepared and the types of data used. Additionally, we obtained and reviewed the Fleet Review Panel report on Surface Force Readiness, [Footnote 5] which, according to Navy officials, was a key source document that was used to develop the Navy's report to Congress. We compared the findings, conclusions, and recommendations in the Fleet Review Panel report to the information in Navy's report to Congress. Finally, we reviewed a report on ship material readiness that was released the same month as the Navy's report to Congress to determine whether the findings and conclusions of the two reports were consistent. This second report was authored by the Naval Sea Systems Command, Surface Warfare Directorate, which is responsible for the maintenance and modernization of all the Navy's nonnuclear surface ships.[Footnote 6] We did not independently assess the validity of the various sources of data used to develop the Navy's report to Congress. We conducted this performance audit from February 2011 to June 2011, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. The Navy's Methodology Involved Drawing on Existing Studies and Data: The Navy's methodology for producing its report consisted of assigning responsibility for the report to the Fleet Readiness Division, which searched for sources of information to address the reporting requirements outlined in House Report 111-491, assembled the information from the sources that appeared to be relevant to training and manning readiness, and finally, vetted the Navy's report through a variety of Navy organizations and through the Navy's chain-of-command. To prepare its report to Congress, the Navy assigned responsibility to officials from its N43 office, the Fleet Readiness Division. These officials reviewed and assembled information from a variety of sources including: * total manning numbers and Navy Enlisted Classification distributions; * training changes relative to maintenance skill sets; * historical data from the Navy's Board of Inspection and Survey (which conducts material inspections of ships every 5 years), referred to as INSURV inspections; * readiness trends from the Defense Readiness Reporting System-Navy (which reports the readiness of Navy forces and the supporting infrastructure to accomplish designed and assigned missions); and: * findings and recommendations from the Fleet Review Panel on Surface Force Readiness.[Footnote 7] Drawing on this information, N43 officials assembled the report documenting the impact of changes in training and manning on ship readiness, conclusions, and proposed actions. For example: * On the basis of its report describing the impact of manning and training, the Navy's report to Congress noted the same general points cited in the Fleet Review Panel and the Naval Sea Systems Command report on ship material readiness. Specifically, the report stated that decisions made by the Navy over the past decade to increase efficiencies throughout the fleet, such as eliminating some training courses and shifting to more computer-based training resulted in crews arriving on board their ships not fully ready to perform apprentice- level maintenance actions and contribute effectively to existing material demands. In addition, it stated that the decreased manning aboard the Navy's surface combatant ships due to the optimal manning initiative, which removed sailors from ships through efficiency initiatives, contributed to declines in material readiness and an unmanageable workload burden on crews. The report also noted a growing backlog in ship maintenance due, in part, to reduction of personnel at the Navy's shore-support facilities, such as regional maintenance centers. In its report, the Navy listed some of the metrics it uses to track the material readiness of its ships, as well as shipboard manning and training. These metrics included historical data from the Navy's Board of Inspection and Survey, readiness trends from the Defense Readiness Reporting System-Navy, and broader metrics of training and manning trends, which the Navy's report states are monitored on a monthly basis. * The Navy makes several conclusions, including that: - there are many aspects to ship material readiness, with training and manning as two parts of the holistic material readiness picture; - changes in training and reductions in crew size over the past decade have had a detrimental effect on the overall readiness of the surface force; - actions are needed in the areas of training and manning to help improve the overall material readiness of the surface combatant ships; and: - once implemented, the combination of improved shipboard manning, additional shore-based billets, training improvements and other material readiness initiatives not addressed in the report to Congress will improve material readiness. * In addressing the adjustments it is making to improve material readiness, the Navy's report to Congress presents several actions that are completed, in process, or planned. For example, the report states that the Navy has already made organizational changes. Specifically, the report notes that Naval Sea Systems Command directed the creation of the Surface Warfare Directorate and a Deputy Directorate for Readiness. These two entities will manage the complete life-cycle support for all nonnuclear surface ships and address material readiness challenges. The Navy's report also noted planned actions to improve training and manning. The Navy stated it has already begun to implement changes in training such as an enhanced material readiness course at the Surface Warfare Officer School. Additionally, the Navy plans to increase the number of critical billets in optimally manned ships by 1,120 billets in fiscal years 2012 and 2013, and create 285 additional shore billets to reestablish the sea-shore flow between ship and intermediate-level maintenance organizations and to provide skilled craftsmen when sailors return to surface ships. Further, the report added that future budgets would include additional manpower to support the Fleet's regional maintenance centers. According to the N43 officials who developed the report, the report was reviewed by the Chief of Naval Operations and several other Navy offices including Naval Sea Systems Command, Fleet Forces Command, and the Naval Education and Training Command. While the Navy's Methodology and Conclusions Are Generally Reasonable, Additional Steps Would Have Enhanced the Report: The Navy used a reasonable methodology in preparing its report to Congress; however it did not perform sufficient analysis to verify the information used to support its conclusions and recommendations. Specifically, the Navy's methodological approach of relying on existing information and subject-matter experts supported accomplishing the report's objectives of describing the impact of training and manning changes on ship readiness, and related conclusions and recommendations. By reviewing and drawing upon published reports such as the Fleet Review Panel report and existing databases such as Defense Readiness Reporting System-Navy, and using reviewers knowledgeable about the subject matter, the Navy gathered relevant information on the status of ship readiness and insights on training and manning issues being experienced on board surface combatants. While the Navy reached specific conclusions and related recommendations in its report, it did not perform any independent analysis to verify source information, including taking steps to assess the reliability of any data. According to N43 officials involved in preparing the report, they reviewed the various source documents previously mentioned and extracted information to include in the Navy report, but did not do any independent analysis to confirm the validity of the data or the conclusions referenced in the source documents. As a result, the Navy's report did not include any discussion of data limitations or caveats to any of the information it presented, including its conclusions and recommendations. Based on our prior work and review of the source documents, we found examples where limitations in certain data exist or differences in data occurred. For example, in its report, the Navy stated that it relied on historical data from INSURV inspections as a source of information in assessing the impact of training and crew size reductions on ship material readiness. It also refers to INSURV data as an indicator of readiness trends. Specifically, it states that the Navy is taking steps to reverse the downward trend in material readiness and ensure that the expected service life of surface combatant ships is not affected. Citing INSURV inspection data for the period of 2008 to 2010, which shows a decrease in the percentage of unsatisfactory inspections, the Navy notes that a positive trend is occurring. However, in our June 2010 report on Navy training and manning, which includes INSURV inspection data over a similar period of 2 years, we concluded that "because of the relatively small number of inspections each year, it is not possible to draw the conclusion that the last 2 years of data from the Navy's independent board of inspection and survey (INSURV) represents a trend."[Footnote 8] In addition, the Navy outlined specific actions that it is taking or plans to take to address the declines in readiness due to manning and crew changes, which, in some cases, differ from the actions recommended in the Fleet Review Panel. For example, while the Fleet Review Panel report and the Navy's report both stipulate that additional sailors are needed aboard the Navy's surface combatant ships, the two reports concluded different numbers of personnel should be added. However, the Navy's report did not caveat its recommendations including providing additional context or rational for why its proposed actions differed from other studies. We are not making recommendations in this correspondence. Agency Comments: DOD was provided a copy of a draft of this report for review, but did not have comments. We are sending copies of this report to the appropriate congressional committees; the Secretary of Defense; and the Secretary of the Navy. This report will also be available at no charge on our Web site at [hyperlink, http://www.gao.gov]. Should you or your staff have any questions concerning this report, please contact me at (202) 512-9619 or pickups@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in enclosure I. Signed by: Sharon L. Pickup: Director: Defense Capabilities and Management: Enclosure: List of Committees: The Honorable Carl Levin: Chairman: The Honorable John McCain: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Daniel Inouye: Chairman: The Honorable Thad Cochran: Ranking Member: Subcommittee on Defense: Committee on Appropriations: United States Senate: The Honorable Howard McKeon: Chairman: The Honorable Adam Smith: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable C.W. "Bill" Young: Chairman: The Honorable Norman Dicks: Ranking Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: [End of section] Enclosure I: GAO Contact and Staff Acknowledgments: GAO Contact: Sharon L. Pickup, (202) 512-9619 or pickups@gao.gov: Staff Acknowledgments: In addition to the contact named above, key contributors to this report were Michael Ferren (Assistant Director), Cynthia Grant, Nicole Harms, James Krustapentus, Mary Jo LaCasse, and Joanne Landesman. [End of section] Footnotes: [1] GAO, Military Readiness: Navy Needs to Reassess Its Metrics and Assumptions for Ship Crewing Requirements and Training, [hyperlink, http://www.gao.gov/products/GAO-10-592] (Washington, D.C.: June 9, 2010). [2] See H.R. Rep. No. 111-491, at 261-262 (2010). [3] Department of the Navy, Report to Congress: Impact of Training and Crew Size on Surface Force Material Readiness (Washington, D.C.: February 2011). [4] The Commander, Naval Surface Force, U.S. Atlantic Fleet, had established a controlled equipage program for its surface ships but this program only tracks high-cost and sensitive items, not the full range of shipboard tools. [5] Fleet Review Panel, Final Report, Fleet Review Panel of Surface Force Readiness (Feb. 26, 2010). [6] Naval Sea Systems Command, The "Bridge" to the Fleet: Material Readiness Key to Warships Ready for Tasking (Feb. 15, 2011). [7] In September 2009, the Commanders of U.S. Pacific Fleet and U.S. Fleet Forces directed Vice Admiral Balisle, USN-Ret., to convene and lead a Fleet Review Panel to assess surface force readiness in the areas of manning, training, and maintenance and to recommend corrective actions. The Fleet Review Panel issued its report in February 2010. [8] GAO, Military Readiness: Navy Needs to Reassess Its Metrics and Assumptions for Ship Crewing Requirements and Training, [hyperlink, http://www.gao.gov/products/GAO-10-592] (Washington, D.C.: June 9, 2010). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: