This is the accessible text file for GAO report number GAO-03-451 
entitled 'Military Readiness: DOD Needs to Better Manage Automatic Test 
Equipment Modernization' which was released on April 11, 2003.



This text file was formatted by the U.S. General Accounting Office 

(GAO) to be accessible to users with visual impairments, as part of a 

longer term project to improve GAO products’ accessibility. Every 

attempt has been made to maintain the structural and data integrity of 

the original printed product. Accessibility features, such as text 

descriptions of tables, consecutively numbered footnotes placed at the 

end of the file, and the text of agency comment letters, are provided 

but may not exactly duplicate the presentation or format of the printed 

version. The portable document format (PDF) file is an exact electronic 

replica of the printed version. We welcome your feedback. Please E-mail 

your comments regarding the contents or accessibility features of this 

document to Webmaster@gao.gov.



Report to the Chairman, Subcommittee on National Security, Emerging 

Threats and International Relations, Committee on Government Reform, 

House of Representatives:



United States General Accounting Office:



GAO:



March 2003:



Military Readiness:



DOD Needs to Better Manage Automatic Test Equipment Modernization:



DOD Automated Test Equipment Modernization:



GAO-03-451:



GAO Highlights:



Highlights of GAO-03-451, a report to the Chairman, Subcommittee on 

National Security, Emerging Threats and International Relations, House 

Committee on Government Reform



Why GAO Did This Study:



The services have billions of dollars worth of outdated and obsolete 

automatic test equipment (ATE) used to test components on military 

aircraft or weapon systems. Department of Defense (DOD) policy 

advocates the development and acquisition of test equipment that can be 

used on multiple types of weapon systems and aircraft and used 

interchangeably between the services.



At the request of the Subcommittee’s Chairman, GAO examined the 

problems that the Air Force, Navy, and Marine Corps are facing with 

this aging equipment and their efforts to comply with DOD policy.



What GAO Found:



DOD and the services face growing concerns regarding obsolete automatic 

test equipment, given the high costs of modernizing or replacing it and 

its potential effect on aircraft readiness. The Navy and Air Force, for 

example, estimate that they will spend billions of dollars to modernize 

or replace this equipment, much of which was acquired in the 1970s and 

1980s. In the meantime, the aging testers are becoming increasingly out 

of date and more difficult to support. When the testers do not work 

properly, maintenance can suffer and readiness can be adversely 

affected.



Since 1994, DOD policy has advocated the acquisition of test equipment 

that can be used on multiple weapon systems and aircraft and can be 

used interchangeably between the services; progress in this regard has 

been slow. For example, although the Navy set out in 1991 to replace 25 

major tester types with one standard tester by 2000, budget cuts and 

delays in developing software have resulted in delays in completing the 

replacement of these obsolete testers until 2008. The Air Force has 

only recently initiated a test equipment modernization plan. However, 

little evidence suggests that consideration is being given to the 

acquisition of equipment that would have common utility for more than 

one weapon system as DOD policy advocates. For procurement of new 

weapon systems, the Air Force is giving little consideration to the use 

of a common tester, while a common tester is planned for use as the 

primary tester for the Joint Strike Fighter. 



Although DOD tasked the Navy as its Executive Agent for automatic test 

equipment in 1994, the agent has made only limited progress in 

achieving compliance across all the services with DOD policy advocating 

the development of common systems. While the Executive Agent can point 

to some successes in individual systems, its officials acknowledged 

that the organization does not have sufficient authority or resources 

to fully implement the policy and achieve the maximum commonality 

possible.



What GAO Recommends:



GAO recommends that the Secretary of Defense reemphasize the policy and 

reconsider the organizational placement and authority of the Executive 

Agent for ATE.



DOD concurred with GAO’s recommendations and agreed that its Executive 

Agent for ATE should be given the authority and resources to more 

effectively fulfill the Department’s oversight responsibilities.



www.gao.gov/cgi-bin/getrpt?GAO-03-451.



To view the full report, including the scope

and methodology, click on the link above.

For more information, contact Neal Curtin at (757) 552-8100 or 

curtinn@gao.gov.



[End of section]



Contents:



Letter:



Results in Brief:



Background:



Aging ATE Presents Major Challenges to DOD:



DOD Has Had Limited Success in Fostering Commonality:



Conclusions:



Recommendations for Executive Action:



Agency Comments and Our Evaluation:



Appendix I: Scope and Methodology:



Appendix II: Comments from the Department of Defense:



Related GAO Products:



Figures:



Figure 1: Schematic of an Automatic Test System, Including ATE and Test 

Program Set Components:



Figure 2: CASS Station:



This is a work of the U.S. Government and is not subject to copyright 

protection in the United States. It may be reproduced and distributed 

in its entirety without further permission from GAO. It may contain 

copyrighted graphics, images or other materials. Permission from the 

copyright holder may be necessary should you wish to reproduce 

copyrighted materials separately from GAO’s product.



Abbreviations:



ATE: automatic test equipment:



CASS: Consolidated Automated Support System:



DOD: Department of Defense:



GAO: General Accounting Office:



JSF: Joint Strike Fighter:



United States General Accounting Office:



Washington, DC 20548:



March 31, 2003:



The Honorable Christopher Shays

Chairman, Subcommittee on National Security,

Emerging Threats and International Relations

Committee on Government Reform

House of Representatives:



Dear Mr. Chairman:



One of the major challenges facing the military services is that of 

modernizing billions of dollars’ worth of aging and increasingly 

obsolete automatic test equipment[Footnote 1] (ATE) used to 

troubleshoot and diagnose components of aircraft or weapon systems. 

Since 1994, Department of Defense (DOD) policy has advocated a 

reduction in the proliferation of testers that are unique to one type 

of weapon system or aircraft, favoring the development or acquisition 

of testers that (1) are interoperable[Footnote 2] within a service and 

between the services and (2) can be used on many different components 

of multiple types of aircraft and weapon systems. As the services 

modernize ATE, they are challenged to adhere to DOD policy and reduce 

the number of unique testers.



ATE with its test program sets, including test software, an interface 

device that connects the ATE to the item being tested, and 

documentation make up an automatic test system. (See fig. 1.):



Figure 1: Schematic of an Automatic Test System, Including ATE and Test 

Program Set Components:



[See PDF for image]



[End of figure]



ATE is used by the services at all maintenance levels (from flight 

lines for routine maintenance to depots for major overhauls and 

factories for production and acceptance testing) to test electronic 

systems and components that are difficult or impossible to test 

manually, to isolate system malfunctions, and to verify that systems 

are operating properly. These testers can be made to examine a single 

aircraft system, various components of an aircraft, or multiple 

components of different aircraft.



Because of your concerns regarding DOD’s continued reporting of spare 

parts shortages and the potential impact that ATE obsolescence could 

have on the readiness of military aircraft, you asked us to determine 

whether DOD and the services are giving adequate attention to ATE 

modernization efforts. Specifically, our objectives were to identify 

(1) what problems the Air Force and Navy[Footnote 3] are facing with 

their ATE and (2) how successful DOD, the Air Force, and the Navy have 

been in addressing the proliferation of unique testers.



Our review included ATE for aircraft managed by the Air Force and the 

Navy and included information on ATE acquisition for two fighter 

aircraft currently under development: the multiservice Joint Strike 

Fighter (JSF) and the Air Force’s F/A-22. Our scope and methodology are 

described in more detail in appendix I. We performed our review from 

January 2002 through March 2003 in accordance with generally accepted 

government auditing standards.



Results in Brief:



DOD and the services face growing concerns regarding obsolete ATE, 

given the high costs of modernizing or replacing this type of equipment 

and its potential adverse impact on aircraft readiness. ATE acquired in 

the 1970s and 1980s is becoming increasingly out-of-date and more 

difficult to support. These obsolescence issues are further aggravated 

by new technologies that, in some cases, make ATE obsolete even before 

the new testers can be fully fielded. Also, older testers are kept much 

longer than initially planned because the weapon systems they support 

are being kept longer. Repair parts for older ATE are becoming 

increasingly scarce, as more contractors discontinue their support.



Although exact cost figures are not available, the services estimate 

that they will need several billion dollars in the coming years either 

to acquire new testers or modernize existing ones. Although the 

services do not maintain data that allow them to measure the extent to 

which obsolete ATE affects readiness, according to DOD readiness 

reports, only 28 percent of Air Force, Navy, and Marine Corps key 

aircraft models met their readiness goals in 2002. Although a 

combination of factors affects readiness goals, the availability of 

spare parts is a key contributor to readiness, and the performance of 

ATE significantly affects the supply of spare parts.



For years, DOD’s policy has aimed to minimize the acquisition of ATE 

that is unique to a particular weapon system; however, the 

implementation of this policy has been slow. In 1994, DOD appointed the 

Navy as its Executive Agent to oversee policy implementation; however, 

according to Executive Agent officials, the Navy has not had the 

authority or resources to effectively carry out this oversight. The 

services lack ATE modernization plans, and there is no DOD-wide 

approach to ensure that all ATE acquisitions and modernizations are 

identified in an early enough stage to ensure that commonality[Footnote 

4] and interoperability are adequately considered. Without sufficient 

information concerning the magnitude of the services’ modernization 

efforts or a departmentwide approach to accomplish ATE modernization, 

the department faces a very expensive and time-consuming ATE 

modernization effort, with the continued proliferation of unique 

testers and no assurance that resources are allocated in the most 

effective manner. As a result, some ATE modernization and acquisition 

planning is being done with little consideration to commonality. For 

example, Executive Agent officials said that they have not had contact 

with the Air Force’s F/A-22 project office concerning ATE since 1994, 

and it does not appear that commonality is being considered or that 

unique ATE development will be minimized.



We are making several recommendations aimed at reinforcing DOD’s stated 

goal of achieving more commonality of test equipment and strengthening 

the department’s oversight. DOD concurred with our recommendations and 

agreed to reemphasize its policy that common automatic test equipment 

be developed to the maximum extent possible. In addition, DOD agreed 

that its Executive Agent for ATE should be given the authority and 

resources to more effectively fulfill its oversight responsibilities. 

DOD’s comments on our report are reprinted in their entirety in 

appendix II.



Background:



It is estimated that DOD employs more than 400 different tester types. 

This equipment is used to diagnose problems in aircraft avionics and 

weapon system components so that the component can be repaired and 

replaced on the aircraft or put into the supply system for future use. 

For example, testers may be used to diagnose problems with aircraft 

radars, guidance and control systems, or weapon systems. According to 

DOD, the department spent over $50 billion in its acquisition and 

support of ATE from 1980 through 1992, and the procurement was 

characterized by the proliferation of testers designed to support a 

specific weapon system or component. These testers are quickly becoming 

obsolete and more difficult and costly to maintain because they may no 

longer be in production and parts may not be readily available. Over 

the years, various studies have criticized the continued proliferation 

of unique ATE and highlighted the need for the development and 

acquisition of testers that can be used to test more than one system or 

component.



In September 1993, the House Appropriations Committee recommended that 

the Secretary of Defense develop a DOD-wide policy requiring ATE 

commonality among the services, along with a formal implementation 

mechanism with sufficient authority, staffing, and funding to ensure 

compliance.[Footnote 5] In 1994, DOD established a policy stating that 

managers of DOD programs should select families of testers or 

commercial off-the-shelf components to meet all ATE acquisition needs 

and that the introduction of unique testers should be minimized. DOD 

designated the Navy at that time as its Executive Agent to oversee 

policy implementation in all services, and identified a goal of 

reducing life-cycle costs and providing greater ATE commonality and 

interoperability. Additional DOD guidance published in 1996 and 1997 

required that all ATE acquisitions be part of the approved families of 

testers or commercial off-the-shelf.



Aging ATE Presents Major Challenges to DOD:



DOD faces major challenges with aging and increasingly obsolete ATE. 

These problems include the high costs of maintaining and replacing 

ATE and the declining availability of spare parts for the aging 

testers. In addition, several DOD organizations, including the Navy 

Inspector General, have suggested that aging and obsolete ATE may 

adversely affect aviation readiness.



Modernization Costs Are Substantial:



Departmentwide estimates of funds needed for ATE modernization and 

acquisition are not readily available. However, according to Air Force 

and Navy ATE managers, most of the services’ ATE is obsolete and will 

need to be upgraded or replaced over the next several years. Our study 

confirmed that replacement and modernization costs would be 

substantial. The Navy, for example, spent about $1.5 billion from 

fiscal years 1990 through 2002 for the acquisition of its primary 

family of testers and plans to spend an additional $430 million through 

fiscal year 2007. Additionally, the Navy estimates that it plans to 

spend $584 million through fiscal year 2007 to adapt existing test 

program sets necessary to perform specific tests of the various 

aircraft components supported by this family of testers. The Navy also 

anticipates spending an additional $584 million to develop program test 

sets for new weapon system requirements.



Information on the Air Force’s spending for ATE modernization is 

somewhat sketchy, as limited data are available centrally for 

individual weapon systems. According to a recent study done for the Air 

Force, the service has not developed a plan that allows modernization 

funding requirements to be determined. However, estimates are available 

for selected systems. The F-15 fighter program office, for example, is 

spending approximately $325 million on just one tester that will be 

fielded in 2004. It also plans to upgrade its electronic warfare 

tester, which is one of seven primary testers for the aircraft, at a 

cost of over $40 million. A 2002 study of B-52 bomber ATE identified 

obsolescence issues associated with six of the aircraft’s seven major 

testers that will require more than $140 million in the near future. 

Similarly, the upgrade of a unique B-1 bomber tester is expected to 

exceed $15 million, even though the Air Force is considering replacing 

this tester and has already begun planning the acquisition. The latest 

estimate for the new tester is $190 million. Current ATE estimates for 

the F/A-22, which is still under development, are not available. 

However, estimates made early in the development phase exceeded 

$1.5 billion.



Readiness Could Be Adversely Affected:



ATE is becoming increasingly out-of-date and more difficult to support. 

And, according to service officials, using this outdated equipment to 

perform required tests in a timely manner is becoming increasingly 

challenging. Although the services could not quantify the extent that 

tester problems affect readiness, service officials noted that without 

adequate test equipment to diagnose problems, components cannot be 

repaired in a timely manner and the mission capability of military 

aircraft can be adversely affected. In August 2000, the Navy Inspector 

General identified shortfalls in ATE as having a negative impact on 

naval aviation and, in particular, on the availability of repaired 

components. During the same time frame, a Navy operational advisory 

group, recognizing the importance of ATE in maintaining readiness, 

ranked support equipment, including ATE, as one of its top 20 readiness 

issues.



We have issued several reports in the recent past addressing the 

shortage of spare parts--a potential result of ATE problems. In 

addition, according to DOD readiness reports, only 28 percent of Air 

Force, Navy, and Marine Corps key aircraft models met their readiness 

goals in fiscal year 2002. Although difficulties in meeting these goals 

are caused by a complex combination of interrelated logistical and 

operational factors, the shortage of spare parts was a major cause. ATE 

plays a significant role in the supply of available spares, since this 

equipment affects both how many parts are taken out of service for 

repair and how quickly they are repaired and returned. We reported that 

maintenance and repair facilities routinely work around spare parts 

shortages by removing a working part from one aircraft to replace a 

nonworking part in another aircraft, a practice called 

“cannibalization.”[Footnote 6] And, although the services do not record 

increases in cannibalizations that are caused by ATE problems, the 

services use cannibalization as a routine maintenance practice when 

testers are not available or not working properly.



In July 2001, we reported that as a result of ATE not working properly, 

unfilled requisitions were adversely affecting the mission capability 

of F-14 aircraft.[Footnote 7] In another case, more than 1,200 Air 

Force B-1 bomber components were backlogged and could not be repaired 

because of the same reason. Although we were unable to measure specific 

reductions in the readiness of F-14 and B-1 aircraft as a result of 

this problem, mission capable rates for the B-1 in fiscal years 1998-

2002 averaged approximately 55 percent, compared with the goal of 

67 percent, while mission capable rates for the F-14D, during the same 

period, averaged 67 percent, compared with a goal of 71 percent. 

Additionally, the Air Force’s 2002 B-52 study concluded that six of the 

seven major testers used to test B-52 components need to be modified or 

replaced or the availability of the aircraft will be adversely affected 

as early as 2006. Air Force officials believe that similar problems 

will continue unless the service undertakes a major ATE modernization 

or replacement program.



DOD Has Had Limited Success in Fostering Commonality:



Since the early 1990s, DOD policies have addressed the need for 

commonality in ATE acquisition and modernization. Although the services 

have been making some progress, efforts to comply with these policies 

have been slow. For example, although the Navy has developed a single 

family of testers to work on many of its aircraft components, after 

11 years, the replacement of its obsolete testers aboard aircraft 

carriers and shore maintenance facilities has not been completed. In 

addition, strategic planning for the modernization of automatic test 

equipment at Navy depots has only recently been initiated.



Historically, the Air Force has not had a service-level ATE 

standardization policy and has essentially pursued unique ATE solutions 

for each weapon system. Since individual aircraft program offices have 

been doing their own planning for modernization, the Air Force has 

given little consideration to having common ATE or testers that are 

interoperable with those of other services. Planning for the Air 

Force’s latest aircraft acquisition, the F/A-22, calls for the 

development of automatic test equipment that will be unique to that 

aircraft. In August 2002, the Air Force initiated a planning effort to 

determine its long-term servicewide ATE modernization needs.



The Navy Has Been Slow in Fielding Its Common Tester:



According to Navy reports, obsolete ATE results in higher backlogs and 

increased flying hour costs, and adversely affects aircraft readiness. 

The Navy recognized years ago, and prior to the establishment of DOD’s 

1994 ATE standardization policy, that its ATE was becoming obsolete. 

In the 1980s the Navy embarked upon an ATE standardization program to 

replace 25 of its testers with one standard ATE family, the 

Consolidated Automated Support System (CASS), to minimize unique types 

of testers. The Navy designed CASS to be used at maintenance activities 

both ashore and afloat. In 1991, the Navy began to produce CASS for the 

general purpose testing of equipment such as radios, radars, and 

electro-optics. (See fig. 2.):



Figure 2: CASS Station:



[See PDF for image]



[End of figure]



CASS’s replacement of 25 types of obsolete testers, in support of 

2,458 weapon system components, was scheduled for completion by 

fiscal year 2000. However, according to Navy officials, because of 

budget cuts that caused delays in developing the test program sets, 

only 4 of the 25 have been completely replaced by CASS, and 8 test sets 

have been partially replaced. Navy officials told us that the 

completion schedule has slipped to fiscal year 2008 for aircraft 

carriers and shore maintenance facilities and could be much longer for 

aviation depots.



The Navy reports that the replacement of these testers with CASS 

stations, when complete, will reduce the number of test-related 

enlisted occupational specialties from 32 to 4, thus reducing training 

requirements. In addition, CASS will reduce the requirement for test 

equipment operators aboard each aircraft carrier from 105 to 54, and at 

the same time reduce space requirements for testers from 2,700 to 1,900 

square feet. Spare parts needed to repair testers will be reduced from 

30,000 to 3,800. According to Navy officials, however, the revised 

completion schedule will not allow for the timely replacement of aging 

ATE, and these delays will adversely affect aircraft readiness.



In addition to schedule slippage, the original CASS equipment was 

fielded about 10 years ago, uses 15-year-old technology and, according 

to Navy ATE program managers, is in need of an upgrade. Accordingly, 

by 2006, the first production units will have reached the point where 

wear and obsolete components will drive supporting costs to 

unacceptable levels and create a need for replacement and 

modernization. The Navy has begun modernization planning for CASS, 

including upgrades through fiscal year 2014.



Integrating CASS into Navy depots may further delay ATE commonality 

within the service. For example, a 2001 Navy report, addressing total 

ATE ownership costs, noted that the depots have not maximized the use 

of CASS because of the limited availability of capital investment 

funds. In addition, at one depot we found some reluctance to use CASS. 

This depot had four CASS stations that had never been used--two were 

delivered in 1999 and installed in December 2000 and February 2001, 

while two others delivered in 2000 were still in crates. Depot 

officials said that they had elected not to put the equipment on-line, 

as they wanted to avoid paying for overhead and maintenance, especially 

without the workload to justify their use. They also noted that the 

development of the test program sets needed to use the CASS has been 

slow, thereby slowing the fielding of the equipment. The Navy has only 

recently begun a servicewide planning effort to modernize its depot-

level testers and determine how best to integrate CASS into its depot 

maintenance strategy.



Air Force’s Approach Has Resulted in Limited Commonality:



Unlike the Navy, the Air Force has not made commonality a priority but 

has pursued unique ATE solutions for each weapon system. In addition, 

it has only recently initiated efforts to collect information on ATE in 

its inventory, including the equipment’s condition and its need for 

modernization or replacement. Because the Air Force has not made 

concerted efforts to use one system to service multiple aircraft 

platforms, it has not taken advantage of efficiencies and potential 

savings such as those expected by the Navy as a result of CASS.



Although the Air Force is developing plans to modernize its ATE, and 

although its policy is to consider developing common testers, it does 

not yet have an overall plan to guide its modernization efforts and has 

made limited progress in this area. Furthermore, it does not have a 

process in place to ensure that commonality is given adequate 

consideration in its ATE acquisition and modernization.



The Air Force has been primarily upgrading--rather than replacing--

aging ATE; leaving ATE management up to individual program managers. In 

most cases, it relies on contractors to provide support for ATE, 

leaving it vulnerable to contractors who may decide to stop supporting 

testers when maintaining them is no longer profitable.



In early 2001, the Air Force organized the Warner Robins Air Logistics 

Center Automatic Test System Division to work with program offices 

on ATE issues. The Division has recently initiated efforts to establish 

a database of all contractors that are capable of supporting existing 

ATE to help identify emerging supportability issues. Although the 

office is responsible for fostering the adoption and use of common 

families of testers, it has no final decision-making authority 

regarding ATE modernizations and no control over funding decisions on 

these matters. Division officials told us that they work with 

individual project offices to encourage them to use common ATE, but 

individual project offices make the final decisions.



In our opinion, leaving these ATE decisions to the individual Air Force 

project offices has led to some questionable and unnecessary 

expenditures. For example:



* The Air Force will spend approximately $325 million to replace a 

tester for the F-15 with one that has been under development for almost 

10 years and is already obsolete. The new tester, called the Electronic 

System Test Set, is not expected to be fielded until 2004. However, 

this electronic tester already needs an upgrade that will cost more 

than $24 million. Because the new tester will not be able to perform 

all the required tests, the Air Force will have to keep the old tester 

too.



* The Air Force is spending over $15 million for an interim 

modernization of its intermediate automatic test equipment for its B-1 

aircraft while, at the same time, a new tester is being developed. If 

the Air Force had taken the necessary steps to replace this obsolete 

tester in a timely manner, these duplicative costs could likely have 

been avoided, and overall ATE modernization costs reduced. According to 

an Air Force official, the program office should have begun the 

acquisition of a replacement tester several years ago, but funding was 

not available. The service is now considering acquiring a replacement 

tester estimated to cost $190 million.



The Air Force’s Warner Robins Air Logistics Center Automatic Test 

System Division is developing a strategic plan that is expected to 

serve as a management plan for meeting long-term ATE needs. The 

Division plans to develop a baseline of its current tester 

capabilities, address supportability and sustainability issues, and 

determine whether tester failures adversely affect the availability of 

aircraft weapon systems. In addition, it will evaluate replacement and 

modernization alternatives, taking into account life-cycle costs and 

the potential for developing common testers. The plan’s implementation 

is expected to take years to complete.



Services’ Approaches in Developing Testers for Two New Aircraft Differ:



While most of our work focused on ATE for the current aircraft 

inventory, we also wanted to see how the services were approaching 

development of testers for two new aircraft, the Joint Strike Fighter 

and the F/A-22. We found that very different approaches are being taken 

in the development of ATE for these two aircraft. The JSF, for example, 

will have a single tester, made up almost entirely of commercial 

components, which will test all components on the aircraft. The F/A-22 

project office has no assurance that commonality is being considered in 

its tester development or that DOD’s policy to minimize unique ATE 

development is being followed.



The JSF originated in the early 1990s through the restructuring and 

integration of several tactical aircraft and technology initiatives 

already under way. The goal was to use the latest technology in a 

common family of aircraft to meet the future strike requirements of the 

services and U.S. allies. The JSF support strategy is built upon a 

single tester to be used by the Air Force, Navy, and Marine Corps, as 

well as by foreign partners, to test all avionics and weapon systems on 

the aircraft.



The JSF tester, referred to as the LM-STAR, is made up almost entirely 

of commercially available components, contributing to readily 

available spares and less complicated upgrades. It will be used during 

development and after the aircraft is fielded. Vendors participating in 

the development of avionics and weapon system components for the 

aircraft are required to produce these components so that their testing 

can be done by the LM-STAR. A total of $99 million has been allocated 

for the purchase and support of 88 of these testers during the 

development phase. While a final decision has not been made on whether 

maintenance support for the aircraft will be provided by the contractor 

or at a military facility, the system project office is taking steps to 

ensure that this tester can be used regardless of where maintenance is 

accomplished.



By contrast, Air Force F/A-22 program officials told us that they have 

not made a decision as to what testers will be used to support this new 

aircraft, which began development in 1991. The project office has not 

ensured that all components for the F/A-22 can be tested with a single 

tester. Project officials told us that the F/A-22 is a very complex 

aircraft and that opportunities to take advantage of common equipment 

will be limited. Yet, the same contractor that is developing the F/A-22 

is also involved in the JSF, which is also very advanced and complex 

and which uses a common family of testers. While current projections of 

ATE costs are not available, estimates made early in the F/A-22 

development phase exceeded $1.5 billion.[Footnote 8]



DOD Oversight Needs Strengthening:



In 1993, the House Appropriations Committee recommended that a DOD-wide 

policy be adopted requiring that the introduction of unique ATE be 

minimized and that DOD establish an oversight system with sufficient 

authority, staffing, and funding to ensure compliance. DOD established 

a policy requiring the services to minimize unique types of testers to 

reduce redundant investments and lessen long-term costs, leveraging its 

investments in testers across the entire DOD establishment. In 1994, 

DOD appointed the Navy as its Executive Agent for ATE to oversee the 

implementation of this policy. As part of the tasking, the Executive 

Agent for ATE was directed to establish a process so that programs 

proposing not to use the DOD-designated standard of ATE families would 

have to request a waiver. In accordance with the direction provided by 

DOD, the Executive Agent established a waiver process. According to 

data provided by the Executive Agent, since its inception, 30 requests 

for waivers were submitted for their review. Our analysis indicated 

that 15 of these requests resulted in waivers or concurrence. The 

remaining requests were never finalized, were returned to the 

originating office for further action, or were determined not to 

require waivers. According to Executive Agent officials, the Executive 

Agent makes recommendations concerning the waiver requests, but it does 

not have the authority to disapprove them.



Executive Agent officials told us however, that they have no assurance 

that all tester acquisitions and modifications are identified or that 

all required waivers are requested. As a result, they may not be aware 

of all ATE modifications or acquisitions or they may not be made aware 

of such until the process is already under way and it is too late to 

affect any change. For example, the Air Force did not request a waiver 

for a $77 million modification to ATE supporting the low altitude 

navigation and targeting infrared for night (LANTIRN). LANTIRN is a pod 

system that supports the F-15, F-16, and F-14 aircraft in low-level 

navigation and lazing targets. In its technical comments on our draft 

report, however, Air Force officials indicated that owing to the nature 

of the LANTIRN modification, a DOD waiver was not required. We continue 

to believe, however, that the Executive Agent should be notified of 

tester modifications of this magnitude.



In addition to having no assurance that all tester acquisitions and 

modifications are identified, Executive Agent officials told us they do 

not have the necessary enforcement authority or resources to 

effectively implement the waiver process even when they know of the 

planned acquisition or modification. For example, Executive Agent 

officials held several discussions with F/A-22 program officials, early 

in the development phase, concerning the use of common testers; 

however, there was no evidence of the Executive Agent’s involvement in 

F/A-22 ATE development since November 1994. Executive Agent officials 

do not know whether common testers are being considered.



As DOD’s Executive Agent for ATE, the Navy has achieved some success in 

encouraging the development of common testers and in dealing with 

technical issues affecting all services. In September 1998, the 

Executive Agent for ATE reported that DOD had avoided $284 million in 

costs by implementing DOD’s policy and cited one example in which the 

Army and the Navy achieved savings of $80 million by jointly developing 

an electro-optics test capability. Navy officials also told us that 

they believe ATE planning for the Joint Strike Fighter, which calls for 

vendors to use standardized test equipment or equipment having 

commercially available components, can also be considered an 

accomplishment. In addition, the Executive Agent established integrated 

process teams to research technical issues dealing with tester 

commonality, such as efforts to develop open systems architecture. In 

this regard, DOD provided funds to the Executive Agent during fiscal 

years 1995 to 1998 for its research and development efforts. Currently, 

the Navy is leading a joint service technology project aimed at 

demonstrating that the most advance technologies can be combined into a 

single tester. The Executive Agent also implemented a process whereby 

ATE modernization and acquisitions would be reviewed for compliance 

with DOD policy, and developed the ATE Selection Process Guide and the 

ATE Master Plan to aid the services in complying with DOD’s ATE 

policies.



ATE officials, responsible for oversight of ATE, noted that their role 

is essential; however, its current placement in one service (the Navy) 

makes it difficult to ensure other services comply with DOD guidance. 

A report recently prepared by a joint service working group[Footnote 9] 

noted continuing problems in the implementation of DOD policy, 

including ATE obsolescence, delays in modernization efforts, a lack of 

ATE interoperability among the services, upgrading difficulties, rising 

support costs, proliferation of equipment that is difficult to support, 

and systems that are not easily deployed.



Conclusions:



The services have made limited progress in achieving DOD’s commonality 

goals for ATE, as established in the early 1990s. The department does 

not have a joint service forum or body that can oversee the total scope 

of ATE acquisition and modernization and better promote ATE commonality 

and the sharing of information and technology across platforms and 

services. DOD does not have sufficient information concerning the 

magnitude of the services’ modernization efforts or a departmentwide 

approach to accomplish ATE modernization in the most cost-effective 

manner. Without such an approach, the department faces a very expensive 

and time-consuming ATE modernization effort, with the continued 

proliferation of unique testers. It will also have no assurance that 

resources are allocated in the most effective manner to exploit 

commonality and commercially available technology and products. A 

single entity within DOD--rather than in one service--may be in the 

best position to provide overarching oversight and coordination between 

the services in planning for the modernization of ATE. We believe that 

high-level management commitment within DOD and all the services will 

be needed to achieve a cultural change that fosters the development of 

common ATE.



Recommendations for Executive Action:



We recommend that the Secretary of Defense reemphasize the policy that 

common ATE be developed to the maximum extent possible. We also 

recommend that the Secretary reconsider whether placing its Executive 

Agent for ATE in the Navy--or any single service--is the most effective 

way to implement the policy. Wherever the Executive Agent is placed 

organizationally, we recommend that the Secretary give it authority and 

resources to:



* include representatives from all of the services, with a scope to 

include the oversight of ATE acquisition and modifications for all 

weapon systems;



* establish a mechanism to ensure that all ATE acquisitions and 

modernizations are identified in an early enough stage to be able to 

provide a comprehensive look at commonality and interoperability and to 

ensure a coordinated effort between service entities;



* direct the services to draw up modernization plans for its review so 

it can identify opportunities to maximize commonality and technology 

sharing between and within the services; and:



* continue efforts to research technical issues dealing with tester 

commonality such as the development of open system architecture and 

other joint service applications.



Agency Comments and Our Evaluation:



The Department of Defense provided written comments on a draft of this 

report, which are reprinted in their entirety in appendix II. The 

department also provided technical comments which we have incorporated, 

as appropriate, into the report. DOD concurred with our recommendations 

and agreed that it should reemphasize the policy that common automatic 

test equipment be developed to the maximum extent possible. DOD 

indicated that it would propose that an ATE acquisition policy 

statement be included in the next issuance of DOD Instruction 5000.2, 

“Operation of the Defense Acquisition System,” April 5, 2002. DOD also 

agreed to reconsider whether the placement of its Executive Agent in 

the Navy--or any single service--is the most effective way to implement 

its ATE policy. The department further concurred that an Executive 

Agent for ATE should have the authority and resources to direct the 

services to draw up modernization plans for its review to maximize 

commonality, interoperability, and technology sharing between the 

services. In this regard, DOD agreed that there should be a mechanism 

to ensure all automatic test equipment acquisitions and modernizations 

are identified in an early enough stage in order to have a coordinated 

effort among service entities. Finally, DOD agreed that the Executive 

Agent for ATE should include representatives from all services. DOD 

intends to use its authority recently published in DOD Directive 

5100.88, “DOD Executive Agent,” September 3, 2002, to reconsider the 

placement of the Executive Agent and to provide it with sufficient 

authority, resources, and mechanisms to carry out its responsibilities. 

In addition, DOD intends to include the funding for the Executive Agent 

as part of the Planning, Programming, Budgeting and Execution process 

and to identify such funding separately so that it is visible within 

the DOD budget.



As agreed with your office, unless you publicly announce its contents 

earlier, we plan no further distribution of this report until 30 days 

from the date of this report. At that time, we will send copies of this 

report to interested congressional committees; the Secretaries of 

Defense, the Navy, the Air Force, and the Army; the Commandant, U.S. 

Marine Corps; and the Director, Office of Management and Budget. We 

will also make copies available to other interested parties on request. 

In addition, the report will be available at no charge on the GAO Web 

site at http://www.gao.gov/. If you or your staff have any questions 

about the report, please contact me at (757) 552-8100.



Key contributors to this assignment were Ken Knouse, William Meredith, 

Harry Taylor, Hugh Brady, and Stefano Petrucci.



Sincerely yours,



Neal P. Curtin

Director, Defense Capabilities and Management:



Signed by Neal P. Curtin



[End of section]



Appendix I: Scope and Methodology:



We reviewed and analyzed available reports, briefings, documents, and 

records and interviewed officials at the Office of the Secretary of 

Defense and at Air Force and Navy headquarters organizations, 

Washington, D.C.; the Naval Air Systems Command located at Patuxent 

River, Maryland; Air Force Material Command and system program offices 

located at Wright-Patterson Air Force Base, Ohio; Warner Robins Air 

Logistics Center, Georgia; the North Island Naval Aviation Depot, 

California; the Navy’s Aviation Intermediate Maintenance Department, 

Oceana Master Jet Base, Virginia; and the intermediate maintenance 

department aboard an aircraft carrier based in San Diego, California. 

The Army was not included in the scope of this study because our focus 

was primarily on fixed-wing aircraft and because of the Army’s efforts 

to standardize its automatic test equipment (ATE) around a single 

family of testers, a situation similar to that of the Navy’s.



To identify the problems that Air Force and Navy aviation (including 

Marine Corps) is facing with regard to ATE, we interviewed personnel 

responsible for policies and oversight, obtained applicable regulations 

and other guidance, and analyzed data provided by the services on 

various testers. We provided a proforma for the Air Force’s and Navy’s 

use in documenting their inventory of ATE, identifying obsolete 

testers, and providing estimates of modernization and replacement time 

frames and cost. The Navy’s data on ATE were provided by the central 

office that manages common test equipment--PMA-260, within the Naval 

Air Systems Command, and the Air Force’s Automatic Test System Division 

at Warner Robins Air Logistics Center. We also discussed obsolescence 

issues and ATE problems with the managers of shore-based, aircraft 

carrier, and depot maintenance activities. We reviewed and analyzed our 

prior reports and ongoing efforts, and reports of other organizations 

to provide a historical and contextual framework for evaluating ATE 

policies and issues, for documenting readiness rates of selected 

aircraft, and documenting the processes put in place by the Department 

of Defense (DOD) to oversee the services’ efforts to acquire and 

modernize ATE.



To determine how successful DOD and the services have been in 

addressing the proliferation of unique testers, we held discussions 

with the responsible offices within each service and DOD, analyzed 

regulations and guidance, and reviewed studies and other documentation. 

We focused our work concerning this objective at the Navy office 

designated as DOD’s Executive Agent for Automatic Test Equipment--PMA-

260 within the Naval Air Systems Command--and the Air Force’s Automatic 

Test System Division at Warner Robins Air Logistics Center. At these 

offices, which have responsibility for ATE acquisition or sustainment, 

modernization, and oversight, we held discussions with responsible 

officials, obtained documentation regarding responsibilities and 

decisions, and reviewed files for specific ATE acquisition and 

modernization programs. We also obtained information from individual 

system program offices, for selected aircraft, located at Wright-

Patterson Air Force Base and selected Navy and Air Force depots and 

intermediate maintenance facilities. Because we found that Air Force 

testers are generally unique to specific aircraft, we selected the F-

15, B-1B, and B-2 for more detailed analysis, as these are considered 

to be front-line aircraft depended upon heavily by the Air Force to 

accomplish its mission. We also obtained information on ATE acquisition 

for two fighter aircraft currently under development: the Joint Strike 

Fighter and the F/A-22.



We performed our review from January 2002 through March 2003 in 

accordance with generally accepted government auditing standards.



[End of section]



Appendix II: Comments from the Department of Defense:



DEPUTY UNDER SECRETARY OF DEFENSE FOR LOGISTICS AND MATERIEL READINESS 

3500 DEFENSE PENTAGON WASHINGTON, DC 20301-3500:



MAR 24 2003:



Mr. Neal P. Curtin:



Director, Defense Capabilities and Management U.S. General Accounting 

Office:



Washington, DC 20548:



Dear Mr. Curtin:



This is the Department of Defense (DoD) response to the GAO draft 

report, GAO-03-451, “Military Readiness: DoD Needs to Better Manage 

Automatic Test Equipment Modernization,” dated February 24, 2003 (GAO 

Code 350137).



The Department concurs with all six recommendations. Additionally, 

since portions of the draft report supporting the recommendations could 

be misleading, technical comments are provided to improve the accuracy 

and clarity of the draft report. The Department appreciates the 

opportunity to comment on the draft report.



Sincerely,



Allen W. Beckett, Principal Assistant:



Signed by Allen W. Beckett:



Enclosure: As stated:



GAO CODE 350137/GAO-03-451:



“MILITARY READINESS: DOD NEEDS TO BETTER MANAGE AUTOMATIC TEST 

EQUIPMENT MODERNIZATION”:



DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS:



RECOMMENDATION 1: The GAO recommended that the Secretary of Defense 

reemphasize the policy that common automatic test equipment be 

developed to the maximum extent possible. (Page 15/Draft Report):



DoD RESPONSE:	Concur. We will propose an ATE acquisition policy 

statement for the next issuance of DoD Instruction 5000.2.



RECOMMENDATION 2: The GAO recommended that the Secretary of Defense 

reconsider whether placing its executive agent for automatic test 

equipment in the Navy - or any single Service - is the most effective 

way to implement the policy. (Page 15/Draft Report):



DoD RESPONSE:	Concur. DoD Directive 5100.88 “DoD Executive Agent”, 

September 3, 2002 was recently published in order to provide a DoD-wide 

definition of DoD Executive Agent, and to provide DoD approval 

authority for assigning DoD Executive Agent responsibilities, 

functions, and authorities within the DoD. The new Directive goes on to 

state that DoD Executive Agent assignments shall be identified in a DoD 

Directive and will cite the Secretary of Defense’s or the Deputy 

Secretary of Defense’s authority assigning DoD Executive Agency.



The DoD Directive assigning the Executive Agent must identify funding 

and other resource arrangements for the DoD Executive Agent to carry 

out assigned responsibilities, functions, and authorities; specify 

other DoD Components, if any, that provide operational missions or 

administrative or other designated activities in support of the DoD 

Executive Agent.



The DoD Executive Agent funding methods and resource requirements, 

including force structure to the extent permitted by law, shall be 

included as a part of the Planning, Programming, Budgeting and 

Execution process; and the funding and costs in support of each DoD 

Executive Agent assignment and associated arrangements shall be 

identified separately and shall be visible within the DoD budget.



DoD will reconsider assigning the Navy as Executive Agent for Automatic 

Test Equipment consistent with the new DoD Directive 5100.88 outlined 

above.



RECOMMENDATION 3: The GAO recommended that the Secretary of Defense 

give the executive agent the authority and resources to include 

representatives from all of the Services, with a scope to include the 

oversight of automatic test equipment acquisition and modifications for 

all weapon systems. (Page 15-16/Draft Report):



DoD RESPONSE: Concur. See response to Recommendation 2.



RECOMMENDATION 4: The GAO recommended that the Secretary of Defense 

give the executive agent the authority and resources to establish a 

mechanism to insure that all automatic test equipment acquisitions and 

modernizations are identified in an early enough stage so as to be able 

to provide a comprehensive look at commonality and interoperability and 

to ensure a coordinated effort among Service entities. (Page 16/Draft 

Report):



DoD RESPONSE: Concur. See response to Recommendation 2.



RECOMMENDATION 5: The GAO recommended that the Secretary of Defense 

give the executive agent the authority and resources to direct the 

Services to draw up modernization plans for its review so it can 

identify opportunities to maximize commonality and technology sharing 

between and within the Services. (Page 16/Draft Report):



DoD RESPONSE: Concur. See response to Recommendation 2.



RECOMMENDATION 6: The GAO recommended that the Secretary of Defense 

give the executive agent the authority and resources to continue 

efforts to research technical issues dealing with tester commonality 

such as the development of open system architecture and other joint 

Service applications. (Page 16/Draft Report):



DoD RESPONSE: Concur. See response to Recommendation 2.



[End of section]



Related GAO Products:



Defense Inventory: Better Reporting on Spare Parts Spending Will 

Enhance Congressional Oversight. GAO-03-18. Washington, D.C.: 

October 24, 2002.



Defense Inventory: Improved Industrial Base Assessments for Army 

War Reserve Spares Could Save Money. GAO-02-650. Washington, D.C.: July 

12, 2002.



Defense Inventory: Trends in Services’ Spare Parts Purchased from the 

Defense Logistics Agency. GAO-02-452. Washington, D.C.: April 30, 2002.



Defense Logistics: Opportunities to Improve the Army’s and Navy’s 

Decision-Making Process for Weapons Systems Support. GAO-02-306. 

Washington, D.C.: February 28, 2002.



Military Aircraft: Services Need Strategies to Reduce Cannibalizations. 

GAO-02-86. Washington, D.C.: November 21, 2001.



Defense Logistics: Actions Needed to Overcome Capability Gaps in the 

Public Depot System. GAO-02-105. Washington, D.C.: October 12, 2001.



Defense Logistics: Air Force Lacks Data to Assess Contractor Logistics 

Support Approaches. GAO-01-618. Washington, D.C.: September 7, 2001.



Defense Inventory: Navy Spare Parts Quality Deficiency 

Reporting Program Needs Improvement. GAO-01-923. Washington, D.C.: 

August 16, 2001.



Army Inventory: Parts Shortages Are Impacting Operations and 

Maintenance Effectiveness. GAO-01-772. Washington, D.C.: July 31, 2001.



Navy Inventory: Parts Shortages Are Impacting Operations and 

Maintenance Effectiveness. GAO-01-771. Washington, D.C.: July 31, 2001.



Air Force Inventory: Parts Shortages Are Impacting Operations and 

Maintenance Effectiveness. GAO-01-587. Washington, D.C.: June 27, 2001.



Defense Inventory: Information on the Use of Spare Parts Funding Is 

Lacking. GAO-01-472. Washington, D.C.: June 11, 2001.



Defense Inventory: Approach for Deciding Whether to Retain or 

Dispose of Items Needs Improvement. GAO-01-475. Washington, D.C.: 

May 25, 2001.



Military Aircraft: Cannibalizations Adversely Affect Personnel and 

Maintenance. GAO-01-93T. Washington, D.C.: May 22, 2001.



Defense Inventory: Army War Reserve Spare Parts Requirements Are 

Uncertain. GAO-01-425. Washington, D.C.: May 10, 2001.



FOOTNOTES



[1] The test hardware and software of an integrated assembly of 

stimulus, measurement, and switching components under computer control 

that is capable of processing software routines designed specifically 

to test a particular item or group of items.



[2] “Interoperability” is the ability of systems to provide data or 

material to and accept the same from other systems and to operate 

effectively together.



[3] The term “Navy,” as presented in this report, represents Naval 

Aviation, which includes the Marine Corps.



[4] Test equipment that can be used on multiple airframes and weapon 

systems.



[5] House Report No 103-254, Sept. 22, 1993.



[6] See U.S. General Accounting Office, Military Aircraft: Services 

Need Strategies to Reduce Cannibalizations, GAO-02-86 (Washington, 

D.C.: Nov. 21, 2001).



[7] See U.S. General Accounting Office, Navy Inventory: Parts Shortages 

Are Impacting Operations and Maintenance Effectiveness, GAO-01-771 

(Washington, D.C.: July 31, 2001).



[8] In providing technical comments on our draft report, the F/A-22 

project office reiterated that because of the complexity of this 

aircraft, opportunities to use common test equipment were limited. The 

project office indicated that designing one set of ATE to test all 

components could make that tester overly complex and expensive. In 

addition, the project office indicated that it had taken advantage of 

commercial testers and incorporated diagnostics into the avionics 

themselves. Finally, the project office indicated that the estimate for 

ATE of more than $1.5 billion made early in the development phase was 

correct but misleading since the support philosophy had changed. We 

continue to believe that the F/A-22 project office has not ensured that 

tester commonality is being considered. The project office was not able 

to provide information concerning the ATE used or planned for the F/A-

22 or estimates of ATE costs. Furthermore, there was no evidence of 

Executive Agent involvement in the F/A-22 program since November 1994, 

and Executive Agent officials do not know whether common testers are 

being considered.



[9] The DOD Executive Agent for ATE established this working group to 

develop a jointly funded demonstration project whereby the services 

would develop and share ATE innovative technologies for inclusion in 

future ATE acquisitions and modernizations.



GAO’s Mission:



The General Accounting Office, the investigative arm of Congress, 

exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability 

of the federal government for the American people. GAO examines the use 

of public funds; evaluates federal programs and policies; and provides 

analyses, recommendations, and other assistance to help Congress make 

informed oversight, policy, and funding decisions. GAO’s commitment to 

good government is reflected in its core values of accountability, 

integrity, and reliability.



Obtaining Copies of GAO Reports and Testimony:



The fastest and easiest way to obtain copies of GAO documents at no 

cost is through the Internet. GAO’s Web site ( www.gao.gov ) contains 

abstracts and full-text files of current reports and testimony and an 

expanding archive of older products. The Web site features a search 

engine to help you locate documents using key words and phrases. You 

can print these documents in their entirety, including charts and other 

graphics.



Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its 

Web site daily. The list contains links to the full-text document 

files. To have GAO e-mail this list to you every afternoon, go to 

www.gao.gov and select “Subscribe to daily E-mail alert for newly 

released products” under the GAO Reports heading.



Order by Mail or Phone:



The first copy of each printed report is free. Additional copies are $2 

each. A check or money order should be made out to the Superintendent 

of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. 

Orders should be sent to:



U.S. General Accounting Office



441 G Street NW,



Room LM Washington,



D.C. 20548:



To order by Phone: 	



	Voice: (202) 512-6000:



	TDD: (202) 512-2537:



	Fax: (202) 512-6061:



To Report Fraud, Waste, and Abuse in Federal Programs:



Contact:



Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov



Automated answering system: (800) 424-5454 or (202) 512-7470:



Public Affairs:



Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.



General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.



20548: