Defense Infrastructure:

Further Actions Needed to Support Air Force Electronic Warfare Evaluation Simulator Relocation Plans

GAO-11-123R: Published: Jan 26, 2011. Publicly Released: Jan 26, 2011.

Additional Materials:

Contact:

Brian J. Lepore
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

According to Department of Defense (DOD) officials, the Air Force Electronic Warfare Evaluation Simulator (the Simulator), located at Air Force Plant in Fort Worth, Texas, is an important asset for helping to protect U.S. and allied pilots and aircraft against the missile threats posed by adversaries. Most missiles use one of two electronic warfare technologies in order to pursue aircraft in flight and deliver an explosive warhead with the intent to inflict maximum damage. Small shoulder-launched missiles generally use infrared seekers that search for heat sources on an aircraft, while more sophisticated air-to-air and larger surface-to-air missiles can use radio waves and infrared seekers to determine an aircraft's location in flight. DOD continually develops and tests countermeasures to protect U.S. and allied aircraft from both types of missile threats. The Air Force Electronic Warfare Evaluation Simulator at Plant 4 is one of only two Air Force facilities of its kind that test countermeasures against heat-seeking missiles, and it is the only Air Force facility that currently houses the equipment necessary to test countermeasures against more sophisticated radio frequency surface-to-air missiles. The Simulator uses an array of computer hardware and software and other equipment to simulate the firing of a missile under various conditions and scenarios, precluding the need to actually fire and destroy a missile in the process. Conducting such tests provides DOD, the Department of Homeland Security, and allied governments with the necessary data to develop various countermeasures for use by military and commercial aircraft. The Test Resource Management Center within the Office of the Secretary of Defense (OSD) is tasked with reviewing Air Force and other services' proposals to change the test and evaluation infrastructure in accordance with OSD guidance and congressional direction.5 In a July 8, 2009, report, the center provisionally approved the Air Force's relocation proposal and submitted the report to congressional defense committees in response to congressional direction. Subsequently, on July 24, 2009, the House Appropriations Committee expressed concern about DOD's proposed relocation of the Air Force Electronic Warfare Evaluation Simulator. House Report 111-230 directed that funds shall not be obligated or expended to relocate the Simulator until a comprehensive cost-benefit analysis, reviewed by GAO, is provided to the congressional defense committees. Furthermore, the House report, noting that the Simulator's specialized test capabilities are a vital element of our national defense posture, directed that the study's findings should demonstrate the technical merits of any proposed relocation. In August 2009, the Test Resource Management Center submitted OSD's July 2009 report to us in response to the congressional direction in House Report 111-230 and, pending our review, has not submitted that report to the congressional defense committees. Our objectives for this review were to determine (1) to what extent OSD's report on the proposed relocation of the Air Force Electronic Warfare Evaluation Simulator includes a comprehensive cost-benefit analysis and (2) to what extent OSD has addressed the technical issues involved in the proposed relocation.

In its July 2009 report, OSD provided some limited cost and benefit information but did not include all expected costs and benefits associated with the proposal to relocate the Air Force Electronic Warfare Evaluation Simulator, and therefore the report does not constitute a comprehensive cost-benefit analysis. Since the report was issued, the Air Force provided us with additional estimated cost and benefit information intended to better support its relocation rationale. OSD's report identified the annual cost to operate, maintain, and modernize the Air Force Electronic Warfare Evaluation Simulator, but did not specifically identify any estimated onetime transition costs or other costs that may be associated with the relocation. Air Force officials subsequently provided us with an estimated total onetime transition cost of approximately $7 million that was not included in OSD's report. Additionally, during the course of our review, the Air Force identified other potential costs not included in this $7 million transition cost estimate or OSD's report, such as an estimated $3 million cost to temporarily retain Lockheed Martin personnel and an additional $200,000 to $300,000 cost to transfer its infrared test equipment to Eglin. OSD's report also identified the expected benefit of an annual recurring savings ranging from $2.8 million to $4.4 million, but did not include a detailed methodology supporting this estimate. The Air Force subsequently provided us with supporting information for this estimate as well as descriptions of the additional benefits expected. Nevertheless, the Air Force did not follow relevant guidance or best practices for completing a cost-benefit analysis. Air Force economic analysis guidance includes detailed procedures and a checklist for conducting such an analysis and obtaining a certification to ensure the reliability of cost estimates. OMB guidance also identifies several elements that should be included to promote independent analysis, and GAO-identified best practices for developing cost-benefit analyses include steps such as obtaining an independent cost estimate. Air Force officials told us they felt that the information they submitted was sufficient and indicated that conducting the kind of costbenefit analysis described in Air Force guidance would have been cost prohibitive. While Air Force guidance permits officials to secure a waiver under those and other circumstances, the Air Force office developing the cost-benefit analysis did not request such a waiver or provide us with documentation supporting its rationale that conducting such an analysis would have been cost prohibitive. Ultimately, by not following relevant guidance for conducting a comprehensive cost-benefit analysis, the Air Force may lack reasonable assurance that its proposal includes all the potential costs, benefits, and impacts associated with its relocation proposal and the proposal may also not sufficiently satisfy congressional direction to provide a comprehensive cost-benefit analysis. We are making two recommendations to improve DOD's proposal to relocate the Air Force Electronic Warfare Evaluation Simulator. First, we recommend that OSD, in consultation with the Air Force, revise the July 2009 cost-benefit analysis to adhere to internal Air Force guidance and identify all costs and benefits associated with the relocation proposal and submit it to the congressional defense committees. Second, we recommend that the Air Force finalize a transition plan that includes steps for staffing and training personnel to operate and maintain the relocated Air Force Electronic Warfare Evaluation Simulator capabilities and submit that plan to the congressional defense committees as well.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In our January 26, 2011 report entitled, Defense Infrastructure: Further Actions Needed to Support Air Force Electronic Warfare Evaluation Simulator Relocation Plans, we reported that DOD had not fully responded to congressional direction included in House Report 111-230 which directed that funds shall not be obligated or expended to relocate the Air Force Electronic Evaluation Simulator until a comprehensive cost-benefit analysis, reviewed by GAO, is provided to the congressional defense committees. Based on the results of our work we found that DOD provided some limited cost and benefit information in its analysis but did not include all expected costs and benefits associated with the proposal to relocate the Air Force Electronic Warfare Evaluation Simulator, and therefore the report did not constitute a comprehensive cost-benefit analysis. As a result, we recommended that the Secretary of Defense direct the Under Secretary of Defense (Acquisition, Technology and Logistics), in consultation with the Secretary of the Air Force, to revise the previously prepared cost-benefit analysis, in line with internal guidance and in consultation with the appropriate Air Force Comptroller and Financial Management offices and resubmit its analysis to the appropriate congressional defense committees. DOD concurred with our recommendation and in response OSD, in consultation with the appropriate Air Force Comptroller and Financial Management staff, revised its cost-benefit analysis and resubmitted the analysis to the congressional defense committees on June 3, 2011. We believe providing this additional information to the defense committees satisfies our recommendation and will assist the congressional defense committees in determining whether relocating the Air Force Electronic Warfare Evaluation Simulator is cost effective and in the best interest of national security.

    Recommendation: To satisfy congressional direction included in House Report 111-230 to provide a comprehensive cost-benefit analysis of the Air Force's proposal to relocate the Air Force Electronic Warfare Evaluation Simulator, the Secretary of Defense should direct the Under Secretary of Defense (Acquisition, Technology and Logistics), in consultation with the Secretary of the Air Force, to submit this revised analysis to the congressional defense committees.

    Agency Affected: Department of Defense

  2. Status: Closed - Implemented

    Comments: In our January 26, 2011 report entitled, Defense Infrastructure: Further Actions Needed to Support Air Force Electronic Warfare Evaluation Simulator Relocation Plans, we reported that DOD had not completed a transition plan to ensure that there are an adequate number of trained personnel available during and after the relocation of the Air Force Electronic Evaluation Simulator. For example, we found that the Air Force?s plan for training and maintaining personnel with the needed expertise to relocate and maintain the Simulator was still in development as of November 2010, and no Air Force personnel had begun receiving hands-on training to operate the Simulator. Although Air Force officials we met with expressed confidence that sufficient technical expertise currently exists at Wright-Patterson Air Force Base to operate the Simulator's radio frequency capability, the Air Force had not trained or finalized its plans to train Air Force personnel to specifically operate and maintain this one-of-a-kind equipment. We also found that no transition plan or agreement with Lockheed Martin, the current operators of the Simulator, had been finalized to document how the Air Force plans to ensure that there are an adequate number of trained personnel located at Wright-Patterson if the Simulator is relocated. As a result of our work, we recommended that DOD finalize a transition plan that includes steps for staffing and training personnel to operate and maintain relocated Simulator capabilities at Wright-Patterson. DOD concurred with our recommendation and in June 2011 revised its transition which included steps for staffing and training personnel to operate and maintain the Simulator's capabilities during and after the relocation. We believe providing this updated transition plan that identifies how personnel will be staffed and trained to operate the Simulator during and after the transition, satisfies our recommendation and will help decrease the risk associated with the relocation, and help ensure the Simulator will be fully operational within the planned transition time frame.

    Recommendation: To ensure an effective phased transition of the Air Force Electronic Warfare Evaluation Simulator's radio frequency capabilities from its current location to Wright-Patterson and to minimize the potential impact of a delayed transition on test customers, the Secretary of Defense should direct the Under Secretary of Defense (Acquisition, Technology and Logistics), in consultation with the Secretary of the Air Force, to finalize a transition plan that includes steps for staffing and training personnel to operate and maintain relocated Simulator capabilities at Wright-Patterson.

    Agency Affected: Department of Defense

  3. Status: Closed - Implemented

    Comments: In our January 26, 2011 report entitled, Defense Infrastructure: Further Actions Needed to Support Air Force Electronic Warfare Evaluation Simulator Relocation Plans, we reported that DOD had not fully responded to congressional direction included in House Report 111-230 which directed that funds shall not be obligated or expended to relocate the Air Force Electronic Evaluation Simulator until a comprehensive cost-benefit analysis, reviewed by GAO, is provided to the congressional defense committees. Based on the results of our work we found that DOD provided some limited cost and benefit information in its analysis but did not include all expected costs and benefits associated with the proposal to relocate the Air Force Electronic Warfare Evaluation Simulator. In addition, as part of its analysis, DOD did not consult with the appropriate Air Force comptroller and financial management staff to have its analysis certified, and therefore the report did not constitute a comprehensive cost-benefit analysis. As a result of our work, we recommended that the Secretary of Defense direct the Under Secretary of Defense (Acquisition, Technology and Logistics), in consultation with the Secretary of the Air Force, revise the previously prepared cost-benefit analysis, in line with internal guidance and in consultation with the appropriate Air Force Comptroller and Financial Management offices, and identity all expected cost and benefits associated with the proposal to relocate the Simulator. DOD concurred with our recommendation and in response, in June 2011 OSD, in consultation with the appropriate Air Force Comptroller and Financial Management staff, revised its cost-benefit analysis to included the additional cost and benefit data excluded from the original analysis. We believe the additional information provided in DOD's updated cost-benefit analysis satisfies our recommendation and will provide DOD and the congressional defense committees with the necessary information to determine whether relocating the Air Force Electronic Warfare Evaluation Simulator is cost effective and in the best interest of national security.

    Recommendation: To satisfy congressional direction included in House Report 111-230 to provide a comprehensive cost-benefit analysis of the Air Force's proposal to relocate the Air Force Electronic Warfare Evaluation Simulator, the Secretary of Defense should direct the Under Secretary of Defense (Acquisition, Technology and Logistics), in consultation with the Secretary of the Air Force, to revise the previously prepared cost-benefit analysis, in line with internal guidance and in consultation with the appropriate Air Force Comptroller and Financial Management offices, and identify all expected costs and benefits associated with the proposed relocation to determine whether the proposed relocation is cost effective and in the best interests of national security.

    Agency Affected: Department of Defense

  4. Status: Closed - Implemented

    Comments: In our January 26, 2011 report entitled, Defense Infrastructure: Further Actions Needed to Support Air Force Electronic Warfare Evaluation Simulator Relocation Plans, we reported that DOD had not completed a transition plan to ensure that there were an adequate number of trained personnel available during and after the relocation of the Air Force Electronic Evaluation Simulator. For example, we found that the Air Force?s plan for training and maintaining personnel with the needed expertise to relocate and maintain the Simulator was still in development as of November 2010, and no Air Force personnel had begun receiving hands-on training to operate the Simulator. Although Air Force officials we met with expressed confidence that sufficient technical expertise currently exists at Wright-Patterson Air Force Base to operate the Simulator's radio frequency capability, the Air Force had not trained or finalized its plans to train Air Force personnel to specifically operate and maintain this one-of-a-kind equipment. We also found that no transition plan or agreement with Lockheed Martin, the current operators of the Simulator, had been finalized to document how the Air Force plans to ensure that there will be an adequate number of trained personnel located at Wright-Patterson if the Simulator is relocated. As a result of our work we recommended that DOD finalize its transition plan and submit this plan to the appropriate defense committees. DOD concurred with our recommendation and on June 3, 2011 resubmitted its updated transition plan to the defense committees. During our review of the updated transition plan, we found that DOD has identified the steps it plans to take to ensure an adequate number of trained personnel will be available to assist in the relocation of the Simulator. Providing this updated transition plan to the defense committees satisfies our recommendation and will provide Congress and DOD some reassurance that the risk associated with the relocation will be decreased.

    Recommendation: To ensure an effective phased transition of the Air Force Electronic Warfare Evaluation Simulator's radio frequency capabilities from its current location to Wright-Patterson and to minimize the potential impact of a delayed transition on test customers, the Secretary of Defense should direct the Under Secretary of Defense (Acquisition, Technology and Logistics), in consultation with the Secretary of the Air Force, to submit this plan to the congressional defense committees.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Sep 10, 2014

Sep 9, 2014

Sep 8, 2014

Jul 31, 2014

Looking for more? Browse all our products here