This is the accessible text file for GAO report number GAO-07-367R entitled 'Defense Acquisitions: Air Force Decision to Include a Passenger and Cargo Capability in Its Replacement Refueling Aircraft Was Made without Required Analyses' which was released on March 6, 2007. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. March 6, 2007: Congressional Committees: Subject: Defense Acquisitions: Air Force Decision to Include a Passenger and Cargo Capability in Its Replacement Refueling Aircraft Was Made without Required Analyses: The United States Air Force has described aerial refueling as a key capability supporting the National Security Strategy and military warfighters on a daily basis. Currently, the Air Force uses two aircraft for aerial refueling: the KC-135 and the KC-10. While the KC- 10 fleet has an average age greater than 20 years, the KC-135 fleet averages more than 46 years and is the oldest combat weapon system in the Air Force inventory. Consequently, the Air Force intends to replace or recapitalize the KC-135 first. The Air Force began its KC-135 recapitalization efforts in fiscal year 2004, and officials presented a KC-135 recapitalization program to joint military decision makers in November 2006. This program proposed the inclusion of a passenger and cargo capability, which exists to some extent in the current aircraft, in the replacement air refueling aircraft. The Air Force proposal is part of the Joint Capabilities Integration and Development System[Footnote 1] process, which uses analyses to identify and assess such a proposal so as to inform decision makers who must allocate scarce resources. According to Air Force officials, the recapitalization process may cost between $72 billion and $120 billion and will span decades.[Footnote 2] This recapitalization takes place at a time when the Air Force faces fiscal constraints over the next few years, forcing officials to reconfigure the service's short-and long- term priorities in its fiscal year 2008 budget plan. The Air Force has begun this process by announcing the intention to reduce personnel levels by 40,000 members. Because of broad congressional interest, we are currently reviewing, under the Comptroller General's authority to conduct evaluations on his own initiative, the Analysis of Alternatives for the recapitalization of the KC-135 aircraft.[Footnote 3] To fully understand the Analysis of Alternatives for the KC-135 Recapitalization, we reviewed the requirements determination process, of which an analysis of alternatives is a part. The purpose of this report is to bring to your attention issues concerning the adequacy of Department of Defense (DOD) analyses used to determine the requirements for a refueling aircraft with passenger and cargo capabilities, for which a contract is to be awarded late in fiscal year 2007. Specifically, we reviewed (1) to what extent policy and implementing guidance were followed in identifying the passenger and cargo capability and in assessing the associated risk of not including that capability in the replacement refueling aircraft proposal and (2) to what extent decision makers, who validated and approved the capability as a requirement, relied on analyses as specified in policy and implementing guidance and the extent to which this reliance may affect initiation of the acquisition program.[Footnote 4] On December 15, 2006, we briefed congressional staff on our preliminary observations. This letter expands on the information discussed in that briefing and includes recommendations to the Secretary of Defense. We plan to complete our review of the Analysis of Alternatives and report the results in early summer 2007. To conduct our evaluation, we reviewed documents containing analyses supporting decision making in the DOD requirements process. This involved a review of joint and service policies and implementing guidance that form a framework for DOD's capability-based planning processes. We also reviewed DOD's Mobility Capabilities Study and service concepts of operations concerning air mobility and aerial refueling as well as the RAND Analysis of Alternatives for KC-135 Recapitalization and the corresponding DOD reviews of the Analysis of Alternatives. To assess the DOD recapitalization proposal, we reviewed DOD and Air Force key documents and analyses. Furthermore, we interviewed officials from the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics, the Joint Staff, Headquarters Air Force, U.S. Transportation Command, Air Mobility Command, Air Force Office of Aerospace Studies, and RAND Corporation. We also interviewed officials directly involved with presentations made to the Air Force Requirements for Operational Capabilities Council and to the Joint Requirements Oversight Council. We performed our work between May and December 2006 in accordance with generally accepted government auditing standards. The passenger and cargo capability proposed for the replacement refueling aircraft is one of a number of proposed capabilities and this review was limited to this single capability. Results in Brief: Mandatory Air Force policy requires Air Force organizations to use a formal capabilities-based approach to identify, evaluate, develop, field, and sustain capabilities that compete for limited resources. Contrary to mandatory Air Force implementing guidance, however, the Air Force proposal for a replacement refueling aircraft included a passenger and cargo capability without analyses identifying an associated gap, shortfall, or redundant capability. According to mandatory Air Force implementing guidance, analyses supporting the decision-making process should assess a capability based on the effects it seeks to generate and the associated operational risk of not having it. In this case, the supporting analyses determined neither need nor risk with regard to a passenger and cargo capability. Air Force officials could not provide supporting information sufficient to explain this discrepancy between the analyses and their proposal. Without sound analyses, the Air Force may be at risk of spending several billion dollars unnecessarily for a capability that may not be needed to meet a gap or shortfall. Military decision makers approved the passenger and cargo capability as a requirement although supporting analyses identified no need or associated risk. Mandatory Air Force implementing guidance states that senior leaders must use the documented results of analyses to confirm the identified capability requirement. The Air Force Requirements for Operational Capabilities Council validated, and the Chairman of the Joint Chiefs of Staff's Joint Requirements Oversight Council validated and approved, the replacement refueling aircraft proposal with a passenger and cargo capability. Following this approval of the oversight councils, DOD plans to solicit proposals and award a contract for the replacement of the refueling aircraft late in fiscal year 2007. However, including a passenger and cargo capability without analyses identifying an associated gap or shortfall could preclude the certification of the program by the Under Secretary of Defense, Acquisition, Technology and Logistics to Congress. Without this certification, the acquisition program for the replacement refueling aircraft cannot begin. Accordingly, we are recommending that the Secretary of Defense direct the Secretary of the Air Force to accomplish the required analyses that evaluate the proposed passenger and cargo capability so as to determine if there is a gap, shortfall, or redundancy, assess the associated risk, and then submit such documentation to the Joint Requirements Oversight Council for validation. Once these analyses are completed, we also recommend that the Secretary of Defense direct the Chairman, Joint Chiefs of Staff, to formally notify the Under Secretary of Defense for Acquisition, Technology and Logistics that such analyses have been completed as required prior to certification of the program to Congress. In written comments on a draft of this report, DOD disagreed with our recommendation to accomplish the required analyses to establish if there is a gap, shortfall, or redundancy and assess associated risks concerning the proposed passenger and cargo capability in the replacement refueling aircraft. DOD stated that through the Joint Capabilities Integration and Development System process, the Air Force presented analysis and rationale for the passenger and cargo capability. DOD further stated that its Joint Requirements Oversight Council and the Air Force concluded that the analysis was sufficient justification for the capability and the Joint Requirements Oversight Council validated the requirement. However, as our report points out, DOD did not perform the required analyses and failed to identify a gap, shortfall, or redundancy for the passenger and cargo capability. DOD agreed with our recommendation to formally notify the Under Secretary of Defense for Acquisition, Technology and Logistics once the required analyses have been completed. DOD stated that the Under Secretary of Defense for Acquisition, Technology and Logistics will consider whether the Joint Requirements Oversight Council has accomplished its duties with respect to the program, including an analysis of the operational requirements of the program. DOD also stated that the Department would again review the justification for a passenger and cargo capability prior to making a decision to initiate the acquisition program. However, DOD did not offer assurance that, as we recommended, the Air Force would accomplish the required analyses to determine if there is a gap, shortfall, or redundancy, assess the associated risk, and then submit such documentation to the Joint Requirements Oversight Council for validation. We continue to believe that our recommendations have merit and that the analyses required by mandatory guidance are necessary to inform the decision that begins the acquisition program. In light of the DOD comments, we have added a matter for congressional consideration to this report that suggests the Congress require that: * in addition to the certification described by section 2366a of title 10, United States Code, the Under Secretary of Defense for Acquisitions, Technology and Logistics make a specific certification that the Air Force employed a sound, traceable, and repeatable process producing analyses that determined if there is a gap, shortfall, or redundancy and assessed the associated risk with regard to passenger and cargo capability for the KC-135 Recapitalization, and: * consistent with service policy, these analyses are made available to the Joint Requirements Oversight Council prior to the Under Secretary's certification of the program pursuant to section 2366a of title 10, United States Code. DOD's comments are reprinted in enclosure I and our evaluation of the comments begins on page 12 of this letter. Background: With the use of its capabilities-based assessment system, DOD compares what the military has with what the military needs as it considers what it will buy. The military identifies and prioritizes these needs and proposes solutions or capabilities that address a capability gap or shortfall. A capability gap or shortfall is the military inability to achieve a desired effect by performing a set of tasks under specified standards and conditions. The gap may be the result of having no existing capability or lack of proficiency or sufficiency in an existing capability. When capabilities are validated and approved, they become requirements in the defense acquisition system. The KC-135 recapitalization, seeking a replacement refueling aircraft for the Air Force, has proceeded under this capabilities-based approach.[Footnote 5] In a capabilities-based approach, establishing the requirements for the replacement refueling aircraft involves consideration of capabilities beyond those of the current aircraft fleet. As described by DOD, the capabilities-based approach replaces the process of building plans, operations, and doctrine around an individual military weapon system as often occurred in the past.[Footnote 6] Instead, the capabilities-based approach requires that officials explicitly link the acquisition to appropriate and needed capabilities. For example, although the current KC-135 and the KC-10 refueling aircraft have a cargo and passenger capability, a replacement refueling aircraft proposal may or may not have such a capability, depending on needs of the warfighter and the supporting analyses. Air Force officials told us that based on this existing capability, they foresaw a need for the inclusion of a similar capability and included the passenger and cargo capability. However, in a capabilities-based approach, needs must be supported by analyses. Therefore, the inclusion of a passenger and cargo capability in the current proposal should not depend on what occurred in the past but what will be needed in the future. In our previous reporting concerning acquisition outcomes and best practices, we have noted the importance of matching warfighter requirements with available resources, a responsibility shared by the acquisition and requirements communities in DOD. As described in Air Force implementing guidance, there is within DOD a distinct separation between the requirements authority and acquisition authority.[Footnote 7] Under this guidance, this separation requires early and continued collaboration between both communities in order for the Joint Capabilities Integration and Development System process and acquisition process to work effectively. Mandatory Air Force implementing guidance describes the process of analyzing and prioritizing capabilities as establishing a common understanding of how a capability will be used, who will use it, when it is needed, and why it is needed. The guidance further describes that each capability, such as the passenger and cargo capability of the replacement refueling aircraft, is to be assessed based on the effects it seeks to generate and the associated operational risk of not having it. To avoid the risk of unnecessary spending on an unneeded capability, service guidance envisions fielding affordable and sustainable operational capability needed by the warfighter. Mandatory Air Force policy and guidance implement the Joint Capabilities and Integration and Development System, which includes analyses performed by the military service and oversight by both service and joint oversight councils. The Air Force, as sponsor of the KC-135 recapitalization, participates in the Joint Capabilities Integration and Development System process, which is intended to identify, assess, and prioritize needed joint military capabilities and associated risks. Mandatory Air Force guidance states that Air Force capabilities-based planning employs an analytically sound, repeatable, and traceable process to identify capability needs.[Footnote 8] The Air Force Requirements for Operational Capabilities Council, an instrument of the Air Force Chief of Staff and Secretary of the Air Force, reviews, validates, and recommends approval of all Air Force capabilities based requirements.[Footnote 9] After Air Force validation, the Chairman's Joint Requirements Oversight Council, responsible for reviewing military acquisitions, approves and validates warfighting capabilities.[Footnote 10] From a requirements standpoint, final approval of a proposal rests at the level of the Chairman of the Joint Chiefs of Staff. Following Air Force validation and joint approval, the Under Secretary of Defense for Acquisition, Technology and Logistics, who supervises DOD acquisition,[Footnote 11] must certify, as Milestone Decision Authority for the proposed refueling recapitalization, that the requirements community has accomplished its statutory duties and that the proposed program is in compliance with DOD policies and regulations.[Footnote 12] Absent this certification, the acquisition program for the replacement refueling aircraft cannot begin.[Footnote 13] In our prior reviews, we produced a number of products concerning aerial refueling requirements and related capabilities such as passenger and cargo capacity. In our August 1996 report, U.S. Combat Air Power: Aging Refueling Aircraft Are Costly to Maintain and Operate, we recommended consideration of a dual-use aircraft that could conduct both aerial refueling and airlift operations as a replacement for the KC-135.[Footnote 14] We recommended that the Secretary of Defense require that future studies and analyses of replacement airlift and tanker aircraft consider accomplishing the missions with a dual-use aircraft. DOD only partially concurred with this recommendation, expressing concern at that time about how a dual-use aircraft would be used and whether one mission area might be degraded to accomplish the second mission. In our June 2004 report, Military Aircraft: DOD Needs to Determine Its Aerial Refueling Aircraft Requirements, we recommended conducting a study to establish air refueling requirements and we also recommended that a comprehensive analysis of alternatives be conducted in support of the recapitalization of the KC-135.[Footnote 15] DOD concurred with this recommendation. The current proposal for recapitalization of the KC-135 considers a multirole aircraft, establishes air refueling requirements, and includes an analysis of alternatives. Air Force Analyses Did Not Identify a Need for Passenger and Cargo Capability as Required by Policy: Mandatory Air Force policy requires Air Force organizations to use a formal capabilities-based approach to identify, evaluate, develop, field, and sustain capabilities that compete for limited resources.[Footnote 16] According to DOD officials, the KC-135 recapitalization has proceeded under a capabilities-based approach. Contrary to Air Force implementing guidance, however, the Air Force's proposal for a replacement refueling aircraft included a passenger and cargo capability without analyses identifying an associated gap, shortfall, or a redundant capability. According to mandatory Air Force implementing guidance, analyses supporting the decision-making process should assess a capability based on the effect it seeks to generate and the associated operational risk of not having it.[Footnote 17] However, in this case, the supporting analyses determined neither need nor risk. Air Force officials could not provide information explaining this discrepancy between the analyses and their proposal. The four analyses that might have established the passenger and cargo requirement are the Capabilities Review and Risk Assessment, the Functional Area Analysis, the Functional Needs Analysis, and the DOD Mobility Capabilities Study. Mandatory Air Force policy directs the use of a Capabilities Review and Risk Assessment as a capabilities-based planning process that is traceable, repeatable, and defensible to identify Air Force-wide capability shortfalls, gaps, and areas for further study.[Footnote 18] According to mandatory Air Force implementing guidance, senior leaders use these findings to make comprehensive decisions that will yield the best results for the Air Force and joint warfighter.[Footnote 19] The Air Force conducted a Capabilities Review and Risk Assessment examining the Air Force concepts of operations to determine if a capability gap or shortfall existed and the assessment did not report a passenger and cargo capability gap to be addressed by an air refueling aircraft.[Footnote 20] The Functional Area Analysis and the Functional Needs Analysis are related assessments and both are a part of the Joint Capabilities Integration and Development System. According to mandatory Air Force implementing guidance, the Functional Area Analysis documents the military tasks needed to achieve military objectives.[Footnote 21] The Functional Area Analysis for the replacement refueling aircraft does not identify movement of passengers and cargo as a required task of the proposed refueling replacement aircraft. Mandatory Air Force implementing guidance also describes follow-on analysis, the Functional Needs Analysis, which compares the capability needs to the capabilities provided by existing or planned systems and identifies the gaps/ shortfalls or redundancies.[Footnote 22] Additional mandatory Air Force implementing guidance states that if the Functional Needs Analysis identifies a shortfall, Air Force planners must determine the consequence to the Air Force of having a specific amount of capability and the likelihood that the shortfall will have an adverse effect on the Air Force's ability to achieve desired effects for a given time period.[Footnote 23] The Functional Needs Analysis for the replacement tanker aircraft does not document a passenger and cargo capability gap or shortfall and made no mention of the consequence to the Air Force of having or not having this capability or the likelihood that a shortfall will have an adverse effect on the Air Force. In 2004, the Department of Defense identified the Mobility Capabilities Study as an effort to determine, among other things, refueling requirements and recapitalization needs.[Footnote 24] The Mobility Capabilities Study executive summary stated the analysis conducted by the study addressed five objectives including identifying mobility capability gaps, overlaps, or excesses and providing associated risk assessments.[Footnote 25] However, when the report of the Mobility Capabilities Study was released in 2005, it did not identify a passenger and cargo gap or shortfall, concluding instead that combined U.S. and host nation transportation assets were adequate to meet U.S. objectives with acceptable risk. The study did note that a passenger- and cargo-capable refueling aircraft could be used in a secondary mission role when not being used in its primary mission. However, the Mobility Capabilities Study also identified a refueling aircraft shortfall in all refueling-required scenarios but one and concluded that the number of aircraft needed to satisfy refueling needs ranges from 520 to 640 total aircraft, a range that exceeds the current Air Force inventory of 590 refueling aircraft.[Footnote 26] A possible shortage of refueling aircraft under some circumstances raises questions about the ability to employ a refueling aircraft in a passenger and cargo role and underscores the importance of analyses to guide decision-makers concerning a refueling replacement aircraft. Additionally, DOD previously expressed concern that a tanker with a passenger and cargo capability could inappropriately degrade the air refueling mission of the aircraft and concluded that such an option could only be accepted if supported by analyses.[Footnote 27] Mandatory Air Force implementing guidance governing the capabilities- based planning analyses discussed above states that capabilities-based planning employs an analysis process that identifies, assesses, and prioritizes needed military capabilities.[Footnote 28] These four analyses did not identify a passenger and cargo capability gap, did not establish that such a capability would represent a redundancy, and did not assess the risk of not acquiring such a capability. Without sound analyses, the Air Force may be at risk of spending several billion dollars unnecessarily for a capability that may not be needed to meet a gap or shortfall. Military Decision Makers Approved the Capability with Neither an Identified Need nor Risk Assessment: Military decision makers approved the passenger and cargo capability as a requirement although supporting analyses identified neither need nor risk. According to mandatory Air Force implementing guidance, the validation phase of the requirements determination process is the formal review process of a capabilities-based requirements document by the Air Force Requirements for Operational Capabilities Council or Chairman's Joint Requirements Oversight Council to confirm the capability need and operational requirement.[Footnote 29] The Air Force Requirements for Operational Capabilities Council validated and the Chairman's Joint Requirements Oversight Council validated and approved the replacement refueling aircraft proposal with a passenger and cargo capability. The Air Force Requirements for Operational Capabilities Council is the oversight body established to validate and recommend approval or disapproval of Air Force-sponsored proposals and requirements documents. This instrument of the Air Force Chief of Staff and Secretary of the Air Force first reviewed and then validated the proposal for a passenger and cargo capability in the replacement refueling aircraft. According to mandatory Air Force implementing guidance, the Air Force Requirements for Operational Capabilities Council ensures Air Force capabilities-based requirements documentation is prepared in accordance with Air Force and joint guidance, complies with established standards, and accurately articulates valid Air Force capabilities-based requirements.[Footnote 30] Although there was neither an identified need nor a risk assessment that supported inclusion of the passenger and cargo capability in the replacement refueling aircraft, the Air Force Requirements for Operational Capabilities Council validated the proposal in July 2006 and forwarded it for Joint Requirements Oversight Council consideration. The Chairman of the Joint Chiefs uses the Joint Requirements Oversight Council to help fulfill his statutory responsibility to provide advice to the Secretary of Defense on requirements prioritization.[Footnote 31] The Vice Chairman of the Joint Chiefs of Staff, on behalf of the Chairman, presides over the Joint Requirements Oversight Council and, in this role, assists the Chairman in identifying and assessing the priority of joint military requirements (including existing systems and equipment) to meet the national military and defense strategies.[Footnote 32] According to joint policy, Joint Capabilities Integration and Development System procedures support the Chairman and Joint Requirements Oversight Council in identifying, assessing, and prioritizing needed joint military capabilities and conducting risk assessments.[Footnote 33] In November of 2006, the Joint Requirements Oversight Council met, validated, and approved the passenger and cargo capability without the supporting analyses that identified a passenger and cargo need or an associated risk. Officials acknowledged that analyses that might have established the need and might have assessed the risk were not used in the Chairman's oversight council. Officials stated that decision makers used military judgment as the basis to include the passenger and cargo capability in the proposal for the replacement refueling aircraft. In our review of relevant DOD policy and guidance, we found mandatory Air Force guidance describing a capabilities-based process that incorporates subjective operational expertise in combination with objective analysis.[Footnote 34] However, we found decision makers did not use objective analysis in combination with their judgment as required.[Footnote 35] By including a passenger and cargo capability in the replacement refueling aircraft without supporting analyses that identify need and assess associated risk, the Air Force may be at risk of spending several billion dollars unnecessarily and DOD may not be able to certify the program as required by statute. The lack of analyses identifying and supporting the passenger and cargo capability affects the acquisition program directly. The Air Force intends to replace the fleet of more than 500 KC-135s, and the Mobility Capabilities Study of 2005 set the requirement for KC-135s at a range of between 520 to 640 aircraft. Replacement of this fleet is estimated to cost a minimum of $72 billion. Compared to a refueling aircraft without a passenger and cargo capability, the inclusion of the capability is estimated to increase costs by 6 percent. The Joint Requirements Oversight Council approval of the proposal of a replacement refueling aircraft with the passenger and cargo capability, without an established need supported by analyses and without an analysis of risk, could result in an unnecessary expenditure of at least $4.3 billion by our estimates. Lack of analyses may also affect initiation of the acquisition program. Pursuant to statute, the Under Secretary of Defense for Acquisition, Technology and Logistics must certify the program before initiation of an acquisition program.[Footnote 36] Among other items, this certification must include that (1) the Joint Requirements Oversight Council has accomplished its duties including an analysis of operational requirements and (2) the KC-135 refueling replacement aircraft program complies with DOD policies, regulations, and directives. Although responsible for reviewing and approving military needs, the Joint Requirements Oversight Council approved a capability that was not associated with a capability gap or shortfall, contrary to policy and implementing guidance. This could preclude certification of the program by the Under Secretary of Defense for Acquisition, Technology and Logistics. Without this certification, the acquisition program for the replacement refueling aircraft cannot begin.[Footnote 37] Conclusions: DOD and the military services are facing significant budgetary pressures as they determine the necessary equipment and personnel to meet varied and demanding missions. As we noted earlier in our report, the Air Force is faced with fiscal constraints and is considering reducing its personnel by tens of thousands. While funding these budgetary and personnel issues, the Air Force has indicated that its top acquisition priority is replacement of the KC-135 aerial refueling aircraft.[Footnote 38] Additionally, the Air Force has decided that the replacement aircraft is to include a passenger and cargo capability. However, it has reached the decision to add this capability without the benefit of supporting analyses that identified need and assessed associated risk. Accomplishing required analyses informs decision making and it is consistent with current Air Force policy. Mandatory Air Force implementing guidance states each capability is to be assessed on the effects it seeks to generate and the associated operational risk of not having it. Additionally, the Air Force recognizes the importance of policies that accurately determine requirements in an environment of limited resources to achieve the greatest Air Force capability.[Footnote 39] Accomplishing the required analyses related to the passenger and cargo capability in the replacement refueling aircraft informs decision making, complies with mandatory Air Force policy and implementing guidance, and may avoid unnecessary expenditures for capability that may be unneeded. Recommendations for Executive Action: We recommend that the Secretary of Defense direct the Secretary of the Air Force to accomplish the required analyses that evaluate the proposed passenger and cargo capability so as to determine if there is a gap, shortfall, or redundancy, assess the associated risk, and then submit such documentation to the Joint Requirements Oversight Council for validation. Once these analyses are completed, we also recommend that the Secretary of Defense direct the Chairman, Joint Chiefs of Staff, to formally notify the Under Secretary of Defense for Acquisition, Technology and Logistics that such analyses have been completed as required prior to certification of the program to Congress. Matter for Congressional Consideration: The Congress should consider requiring that: * in addition to the certification described by section 2366a of title 10, United States Code, the Under Secretary of Defense for Acquisitions, Technology and Logistics make a specific certification that the Air Force employed a sound, traceable, and repeatable process producing analyses that determined if there is a gap, shortfall, or redundancy and assessed the associated risk with regard to passenger and cargo capability for the KC-135 Recapitalization, and: * consistent with service policy, these analyses are made available to the Joint Requirements Oversight Council prior to the Under Secretary's certification of the program pursuant to section 2366a of title 10, United States Code. Agency Comments and Our Evaluation: In written comments on a draft of this report, DOD did not agree with one recommendation but agreed with a second recommendation. DOD's comments are included as enclosure I at the end of this report. DOD disagreed with our recommendation to accomplish the required analyses to establish if there is a gap, shortfall, or redundancy and assess associated risks concerning the proposed passenger and cargo capability in the replacement refueling aircraft. In its comments, DOD stated that through the Joint Capabilities Integration and Development System process, the Air Force presented analysis and rationale for the passenger and cargo capability. DOD further stated that its Joint Requirements Oversight Council and the Air Force concluded that the analysis was sufficient justification for the capability and the Joint Requirements Oversight Council validated the requirement. However, as our report points out, DOD did not perform the required analyses and failed to identify a gap, shortfall, or redundancy for the passenger and cargo capability. When interviewed, Joint Requirements Oversight Council officials told us that no analysis identifying a need for a passenger and cargo capability was presented to the council. Required analyses should establish an understanding of when and why a capability is needed and the risk of not having it. No such analysis was available to the Joint Requirements Oversight Council. Considering the requirement for analyses that separate needs from wants and the risk of unnecessary expenditures in this multi-year multi-billion dollar acquisition program, we continue to believe that our recommendation has merit and that the analyses required by mandatory guidance are necessary to inform the decision that begins the acquisition program. DOD agreed with our recommendation to formally notify the Under Secretary of Defense for Acquisition, Technology and Logistics once the required analyses have been completed. Acknowledging the responsibility established in section 2366a of title 10, United States Code, DOD stated that the Under Secretary of Defense for Acquisition, Technology and Logistics will consider whether the Joint Requirements Oversight Council has accomplished its duties with respect to the program, including an analysis of the operational requirements of the program. DOD also stated that the Department would again review the justification for a passenger and cargo capability prior to making a decision to initiate the acquisition program. However, DOD did not offer assurance that, as we recommended, the Air Force would accomplish the required analyses to determine if there is a gap, shortfall, or redundancy, assess the associated risk, and then submit such documentation to the Joint Requirements Oversight Council for validation. We believe that the time it could take to accomplish the required analyses and submit the analyses for revalidation by the Joint Requirements Oversight Council, could delay the Under Secretary's certification until just prior to the Milestone B decision, and may frustrate the congressional oversight that would otherwise be permitted under section 2366a.[Footnote 40] We believe that in a program committing $120 billion over several decades, the review confirming that needs are justified should occur as far in advance of program initiation as possible. We continue to believe that by including a passenger and cargo capability in the replacement refueling aircraft without required analyses that identify need and assess associated risk, the Air Force is at risk of spending several billion dollars unnecessarily. We also believe, as reported, that the absence of analyses identifying a capability gap, shortfall, or redundancy, and the Joint Requirements Oversight Council approval of the program without these analyses is contrary to policy and implementing guidance and could preclude certification of the program by the Under Secretary of Defense for Acquisition, Technology and Logistics. In light of the DOD comments on our report, we are proposing a matter for congressional consideration. We are sending copies of this report to other interested congressional committees; the Secretary of Defense; the Under Secretary of Defense, Acquisition, Technology and Logistics; the Secretary of the Air Force, and the Director, Office of Management and Budget. Copies of this report will also be made available to others upon request. In addition, this report will be available at no charge on the GAO Web site at http://www.gao.gov. If you have any questions regarding this report, please contact me at (202) 512-8365 or solisw@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report include Ann Borseth, Assistant Director; Grace Coleman; Oscar Mardis; Karen Thornton; and Steve Woods. Signed by: William M. Solis, Director: Defense Capabilities and Management: List of Addressees: The Honorable Carl Levin: Chairman: The Honorable John McCain: Ranking Minority Member: Committee on Armed Services: United States Senate: The Honorable Daniel K. Inouye: Chairman: The Honorable Ted Stevens: Ranking Minority Member: Subcommittee on Defense: Committee on Appropriations: United States Senate: The Honorable Ike Skelton: Chairman: The Honorable Duncan L. Hunter: Ranking Minority Member: Committee on Armed Services: House of Representatives: The Honorable John P. Murtha: Chairman: The Honorable C.W. Bill Young: Ranking Minority Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: Enclosure !: Comments from the Department of Defense: Office Of The Under Secretary Of Defense: 3000 Defense Pentagon: Washington, DC 20301-3000: Acquisition, Technology, And Logistics: Feb 20 2007: Mr. William M. Solis: Director, Defense Capabilities and Management: U.S. Government Accountability Office: 441 G Street, N.W. Washington, DC 20548: Dear Mr. Solis: This is the Department of Defense (DoD) response to the GAO Draft Report, 07-367R, "Air Force Decision to include a Passenger and Cargo Capability in Its Replacement Refueling Aircraft Was Made Without Required Analyses," dated January 23, 2007 (GAO Code 350973/GAO-07- 367R). The Department of Defense non-concurs with Recommendation 1 and concurs with Recommendation 2 in the draft report. Details of our responses are contained in the enclosure. The Department appreciates the opportunity to comment on the draft report. Sincerely, Signed by: Diane M. Wright: Acting Director: Portfolio Systems Acquisition: Enclosure: As stated: GAO Draft Report - Dated January 23, 2007 GAO Code 350973/GAO-07-367R: "Air Force Decision to Include a Passenger and Cargo Capability in Its Replacement Refueling Aircraft Was Made Without Required Analyses": Department Of Defense Comments To The Recommendations: Recommendation 1: The GAO recommended that the Secretary of Defense direct the Secretary of the Air Force to accomplish the required analyses that evaluate the proposed passenger and cargo capability so as to determine if there is a gap, shortfall, or redundancy, assess the associated risk, and then submit such documentation to the Joint Requirements Oversight Council for validation. DOD Response: Non-concur. Through the Joint Capabilities Integration and Development System (JCIDS) process, the Air Force presented analysis and rationale for the passenger and cargo capability. The Joint Requirements Oversight Council (JROC) and the Air Force concluded that the analysis was sufficient justification for the capability, and the JROC validated the passenger and cargo requirement. The Department will review the JROC justification and associated analysis prior to the Milestone B decision, as required by section 2366a of title 10, United States Code. Recommendation 2: The GAO recommended that the Secretary of Defense direct the Chairman, Joint Chiefs of Staff, once these analyses are completed, to formally notify the Under Secretary of Defense for Acquisition, Technology and Logistics that such analyses have been completed as required prior to certification of the program to Congress. DOD Response: Concur. In accordance with section 2366a of title 10, United States Code, the Under Secretary of Defense for Acquisition, Technology and Logistics will consider whether the Joint Requirements Oversight Council has accomplished its duties with respect to the program, including an analysis of the operational requirements for the program. [End of section] (350973): FOOTNOTES [1] The procedures established in the Joint Capabilities Integration and Development System (JCIDS) support the Chairman of the Joint Chiefs of Staff and the Joint Requirements Oversight Council in identifying, assessing, and prioritizing joint military capability needs as specified in Title 10 of the United States Code, sections 153, 163, 167, and 181. Chairman of the Joint Chiefs of Staff Instruction 3170.01E, Joint Capabilities Integration and Development System, ¶ 1 (May 11, 2005), hereinafter referred to as CJCSI 3170.01E (May 11, 2005). [2] This cost estimate is based on information provided by Air Force officials in congressional testimony. The cost represents the estimated total cost of procuring 520 replacement refueling aircraft over a 38- year period. This is based on an estimated range of current prices per plane based on costs cited in Air Force testimony, February, 2006. [3] According to Air Force implementing guidance, this is an analysis that helps decision makers select the most cost-effective alternative to satisfy an operational capabilities-based requirement. An Analysis of Alternatives is also an analysis of operational effectiveness and estimated life-cycle costs for alternative materiel systems. Air Force Instruction 10-601, Capabilities-Based Requirements Development, ¶ 4.4.1 (July 31, 2006), hereinafter cited as AFI 10-601 (May 31, 2006). [4] In policy and implementing guidance concerning capabilities-based assessments, the words needs and requirements are used interchangeably. For purposes of this report, we use the term capability to mean the military ability to achieve a desired effect by performing a set of tasks under specified standards and conditions. Also for purposes of this report, the term requirement means a system capability or characteristic required to accomplish approved mission needs. [5] In an interview regarding the Air Force briefing to the Joint Requirements Oversight Council, a senior DOD official told us the refueling recapitalization program is proceeding under a capabilities- based approach. [6] Prior to this new capabilities-based approach, DOD used a threat- and risk-based process to determine requirements. While threat is no longer the driving factor in determining requirements, risk management is still part of DOD acquisition strategy. [7] AFI 10-601, Capabilities-Based Requirements Development, ¶ 1.2 (July 31, 2006). [8] Air Force Instruction 10-604, Capabilities-Based Planning, ¶ 1.1.1 (May 10, 2006), hereinafter cited as AFI 10-604 (May 10, 2006). [9] AFI 10-601, ¶ 2.3.5.1 (July 31, 2006). [10] AFI 10-601, ¶ 2.3.5.2 (July 31, 2006). [11] 10 U.S.C. §133 (b)(1) (2006). [12] 10 U.S.C. 2366a §§ (a)(7) and (a)(10) (2006). [13] 10 U.S.C. § 2366a (a) (2006) and Department of Defense Instruction 5000.2, Operation of the Defense Acquisition System, ¶ 3.7.1.2 (May 12, 2003), hereinafter cited as DODI 5000.2 (May 12, 2003). [14] GAO, U.S. Combat Air Power: Aging Refueling Aircraft Are Costly to Maintain and Operate, GAO/NSIAD-96-160 (Washington, D.C.: Aug. 8, 1996). [15] GAO, Military Aircraft: DOD Needs to Determine Its Aerial Refueling Aircraft Requirements, GAO-04-349 (Washington, D.C.: June 4, 2004). [16] Air Force Policy Directive 10-6, Capabilities-Based Planning and Requirements Development, ¶ 1 (May 31, 2006), hereinafter cited as AFPD 10-6 (May 31, 2006). [17] AFI 10-601, ¶ 1.2 (July 31, 2006). [18] AFPD 10-6, ¶ 1.1 (May 31, 2006). [19] AFI 10-604, ¶ 3.1.4 (May 10, 2006). [20] The Global Mobility Concept of Operations describes the primary mission of air refueling as providing worldwide, day/night, adverse weather, probe/drogue, and boom air refueling on the same sortie to receiver-capable U.S., allied, and coalition military aircraft (including unmanned aircraft). Refueling aircraft are employed to support global attack, air bridge, deployment, redeployment, homeland defense, and theater support to joint, allied, and coalition air forces, and specialized national defense missions. They also are used to support special operations and U.S. nuclear forces. [21] AFI 10-604, ¶ 3.1.4.2 (May 10, 2006). [22] AFI 10-601, ¶ 2.2.1 (July 31, 2006). [23] AFI 10-604, ¶ 3.1.4.3 (May 10, 2006). [24] GAO-04-349, app. II, p. 34. [25] Department of Defense Mobility Capabilities Study, Executive Summary, Sec. II, p. 2 (December 2005). [26] The inventory of 590 air refueling aircraft comprises 114 KC- 135Es, 417 KC-135Rs, and 59 KC-10 aircraft. [27] GAO/NSIAD-96-160, app. I, p. 41. [28] Capabilities-based analyses that identify, assess, and prioritize include the Functional Needs Analysis, Functional Area Analysis, and the Capabilities Review and Risk Analysis. AFI 10-604, ¶¶ 1.1.1 and 3.1.4 (May 10, 2006). The Mobility Capabilities Analyses 2005 shared this objective of identifying mobility capability gaps, overlaps, or excesses and providing associated risk assessment. Department of Defense Mobility Capability Study, Executive Summary, section II, p. 2. [29] AFI 10-601, ¶ 2.3.5 (July 31, 2006). [30] AFI 10-601, ¶ 3.4 (July 31, 2006). [31] 10 U.S.C. § 153 (a)(4)(A) and 10 U.S.C. § 181 (b)(1) (2006) and Chairman of the Joint Chiefs of Staff Instruction 5123.01B, Charter of the Joint Requirements Oversight Council, Encl. A-1. [32] 10 U.S.C. § 181 (b)(1) (2006). [33] CJCSI 3170.01E, ¶ 1 (May 11, 2005) and Chairman of the Joint Chiefs of Staff Instruction 5123.01B, Charter of the Joint Requirements Oversight Council, Encl. A ¶ 2e(15), p. A-6 (April 15, 2004). [34] AFI 10-601, ¶ 1.4.1 (July 31, 2006). [35] Air Force Instruction 10-601 (July 31, 2006) also describes, at ¶ 1.4.3, the concept of Top-Down Direction, whereby higher authority, such as the Chief of Staff of the Air Force, may direct a sponsor to initiate the development and fielding of a new capability. Even if Top- Down Direction were used in this case, the sponsor would still be responsible for conducting appropriate analysis and producing the capabilities-based documents, pursuant to the mandatory guidance at AFI 10-601, ¶ 1.4.3. [36] 10 U.S.C. § 2366a (a) (2006). [37] 10 U.S.C. § 2366a (2006) and DODI 5000.2 ¶ 3.7.1.2 (May 12, 2003). [38] In October 2006, the Chief of Staff of the Air Force identified the service's top five procurement priorities as follows: (1) replacement refueling aircraft, (2) combat rescue helicopter, (3) space- based early warning and communications satellites, (4) the F-35 (Lightning II), and (5) the next-generation long-range strike bomber. [39] Air Force Policy Directive 16-5, Planning, Programming, and Budgeting System, ¶ 1 (29 July 1994). [40] In respect to acquisition programs, milestones are established in DODI 5000.2 and are the points where a recommendation is made and approval is sought regarding starting or continuing a program into the next phase. In this instance, the decision at Milestone B is to enter into the system development and demonstration phase pursuant to guidance prescribed by the Secretary of Defense and to begin the acquisition program. GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to www.gao.gov and select "Subscribe to Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Washington, D.C. 20548: Public Affairs: Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: