This is the accessible text file for GAO report number GAO-11-806 
entitled 'Weapons Acquisition Reform: Actions Needed to Address 
Systems Engineering and Developmental Testing Challenges' which was 
released on September 19, 2011. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to the Committee on Armed Services, U.S. Senate: 

September 2011: 

Weapons Acquisition Reform: 

Actions Needed to Address Systems Engineering and Developmental 
Testing Challenges: 

GAO-11-806: 

GAO Highlights: 

Highlights of GAO-11-806, a report to the Committee on Armed Services, 
U.S. Senate. 

Why GAO Did This Study: 

For the past 2 years, the Department of Defense (DOD) has been 
implementing the Weapon Systems Acquisition Reform Act (Reform Act) 
requirements for systems engineering and developmental testing. These 
activities are important to DOD’s ability to control acquisition 
costs, which increased by $135 billion over the past 2 years for 98 
major defense acquisition programs. 

GAO was asked to determine (1) DOD’s progress in implementing the 
Reform Act’s requirements and (2) whether there are challenges at the 
military service level that could affect their systems engineering and 
developmental testing activities. To do this, GAO analyzed 
implementation status documents, discussed developmental testing 
office concerns with current and former DOD officials, and analyzed 
military service workforce growth plans and test range funding data. 

What GAO Found: 

The new offices for systems engineering and developmental test and 
evaluation are continuing to make progress implementing Reform Act 
requirements. Since GAO’s 2010 report on this topic, the Deputy 
Assistant Secretaries for Systems Engineering and Developmental Test 
and Evaluation have issued additional policies and guidance, assisted 
more weapons acquisition programs in the development of acquisition 
plans, and provided input to senior leaders at Defense Acquisition 
Board meetings. DOD also designated the Deputy Assistant Secretary for 
Developmental Test and Evaluation for concurrent service as the 
Director of the Test Resource Management Center. This was an optional 
Reform Act provision, which places oversight of testing resources and 
acquisition program developmental testing activities under one 
official. Despite these steps, the developmental test and evaluation 
office reports having difficulty covering its portfolio of about 250 
defense acquisition programs with its current authorized staff of 63 
people. Current and former testing officials believe the office needs 
more influence and resources to be effective, but they said thorough 
analysis has not been done to determine the appropriate office size. 
Further, according to the Deputy Assistant Secretary for Developmental 
Test and Evaluation, a statutory provision that designates the Test 
Resource Management Center as a field activity may limit his ability 
to achieve management and reporting efficiencies that could be 
obtained by combining or shifting resources between the two 
organizations. GAO has a matter for Congressional consideration to 
allow shifting resources between the Test Resource Management Center 
and the developmental test and evaluation office. 

The military services are facing workforce challenges that could curb 
systems engineering and developmental testing efforts, if not properly 
addressed. The services planned to increase their systems engineering 
and test and evaluation career fields by about 5,000 people (14 
percent) and about 300 people (4 percent), respectively, between 
fiscal years 2009 and 2015 through hiring actions and converting 
contractor positions to government positions. The services have 
increased the systems engineering career field by about half of its 
projections and exceeded its planned growth for the test and 
evaluation career field through the end of fiscal year 2010. However, 
future growth may be difficult because of budget cuts and a 
clarification in DOD’s insourcing approach, which may make civilian 
hiring more difficult. For example, the services now plan to hire 
about 800 fewer systems engineers by 2015 than they originally 
projected. Further, cuts to development test ranges’ fiscal year 2012 
budgets of nearly $1.2 billion (17 percent) over the next 5 years 
could offset some of the workforce gains already achieved. Currently, 
the services lack metrics that could be used to justify funding 
levels, effectively allocate funding cuts, make workforce decisions, 
or make difficult decisions related to mothballing, closing, or 
consolidating test capabilities, if future budget cuts are necessary. 
To the extent DOD cannot provide adequate systems engineering and 
developmental testing support to its weapon systems portfolio, the 
risks of executing the portfolio within cost and schedule are 
increased. 

What GAO Recommends: 

GAO recommends that DOD assess the resources needed by the 
developmental test and evaluation office, develop a plan to implement 
the assessment, develop metrics to aid funding decisions, and report 
the effect budget cuts are having on the services’ ability to meet 
program office needs. GAO also has a matter for congressional 
consideration. DOD concurred with two recommendations, and offered 
clarifying language, which GAO incorporated, on the other two 
recommendations for which DOD partially concurred. 

View [hyperlink, http://www.gao.gov/products/GAO-11-806] or key 
components. For more information, contact Michael J. Sullivan at (202) 
512-4841 or sullivanm@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

DOD Is Continuing to Implement Reform Act Requirements, but Challenges 
Remain: 

Workforce Challenges Could Affect Services' Systems Engineering and 
Developmental Testing Efforts: 

Conclusions: 

Recommendations for Executive Action: 

Matter for Congressional Consideration: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: Comments from the Department of Defense: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Offices of the Deputy Assistant Secretaries of Defense for 
Systems Engineering and Developmental Test and Evaluation Workforce 
Growth: 

Table 2: Ongoing Efforts by Deputy Assistant Secretaries of Defense 
for Systems Engineering and Developmental Test and Evaluation to 
Implement Selected Reform Act Requirements: 

Table 3: Actual and Planned Military Service Systems Engineering and 
Test and Evaluation Career Field Growth: 

Figure: 

Figure 1: MRTFB Operation & Support Funding for Fiscal Years 2006-2015: 

Abbreviations: 

DOD: Department of Defense: 

MRTFB: Major Range and Test Facility Base: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

September 19, 2011: 

The Honorable Carl Levin: 
Chairman: 
The Honorable John McCain: 
Ranking Member: 
Committee on Armed Services: 
United States Senate: 

For the past 2 years, the Department of Defense (DOD) has been 
implementing provisions of the Weapon Systems Acquisition Reform Act 
of 2009[Footnote 1] (Reform Act) related to systems engineering and 
developmental testing. Greater attention to these activities provides 
the Office of the Secretary of Defense an opportunity to affect weapon 
acquisition plans early and assess technical risks throughout weapon 
system development. In addition, it could help control total 
acquisition costs, which we recently reported[Footnote 2] had 
increased by $135 billion over the past 2 years for DOD's current 
portfolio of 98 major defense acquisition programs.[Footnote 3] 

Last year, we reported[Footnote 4] on the status of DOD's initial 
efforts to implement Reform Act requirements related to systems 
engineering and developmental testing. We found that DOD had taken 
steps to implement the Reform Act requirements, including establishing 
new offices for the Director of Systems Engineering and the Director 
of Developmental Test and Evaluation within the Office of the 
Secretary of Defense. The offices have since been renamed as the 
offices of the Deputy Assistant Secretary of Defense for Systems 
Engineering and the Deputy Assistant Secretary of Defense for 
Developmental Test and Evaluation. We found that there were concerns 
about the amount of influence the Deputy Assistant Secretary for 
Developmental Test and Evaluation could have on weapon acquisition 
programs based upon where the office is placed organizationally within 
the department. However, we could not determine whether the office had 
the appropriate amount of influence because it was not tracking the 
extent to which its recommendations were being adopted or impacting 
weapon programs. We also identified issues the military services face 
as they enhance systems engineering and developmental testing 
activities on their weapon acquisition programs, including determining 
whether they have enough people to perform these activities, training 
the influx of new hires they expect, and addressing test range 
resource needs. 

Based on our initial work, the Senate Armed Services Committee asked 
us to continue to monitor DOD's efforts to implement the Reform Act 
provisions, as well as look at the military services' systems 
engineering and developmental test and evaluation capabilities. Our 
specific objectives were to determine (1) DOD's progress in 
implementing the Reform Act's systems engineering and developmental 
testing requirements within the Office of the Secretary of Defense and 
(2) whether there are challenges at the military service level that 
could affect their systems engineering and developmental testing 
activities. 

In conducting our work, we analyzed information obtained from the 
offices of the Deputy Assistant Secretaries of Defense for Systems 
Engineering and Developmental Test and Evaluation to determine the 
status of their efforts to implement the Reform Act legislation. We 
also solicited the views of current and former DOD developmental 
testing officials about the effectiveness of the developmental test 
and evaluation office. In addition, we interviewed officials and 
analyzed pertinent documents related to workforce and test range 
issues from 12 DOD test ranges, the Test Resource Management Center, 
and cognizant military service systems engineering, developmental 
testing, and personnel offices. For purposes of this report, we use 
the term systems engineering career field to refer to two Systems 
Planning, Research Development, and Engineering career fields--Systems 
Engineering and Program Systems Engineering. The test and evaluation 
career field is a combination of developmental and operational testing 
personnel. Both of these career fields represent a portion of the 
total systems engineering and developmental testing workforce. See 
appendix I for a more detailed explanation of our scope and 
methodology. 

We conducted this performance audit from September 2010 to September 
2011 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

Background: 

DOD significantly downsized its acquisition workforce in the 1990s and 
2000s as part of an overall reduction of military and civilian defense 
personnel after the end of the Cold War. According to DOD's April 2010 
Strategic Human Capital Plan Update, the department decreased its core 
acquisition workforce[Footnote 5] from about 146,000 people in fiscal 
year 1998 to about 126,000 in fiscal year 2008. Meanwhile, the number 
of major defense acquisition programs increased from 76 to 93 and 
total estimated costs[Footnote 6] to acquire them increased from 
nearly $805 billion to more than $1.6 trillion. The systems 
engineering and test and evaluation career fields were affected by the 
workforce cuts. For example, in 2008 the Defense Science Board 
reported[Footnote 7] that the Army's test and evaluation workforce was 
reduced by more than 55 percent from 1991 through 2007. In response to 
cuts in these career fields, DOD reduced its emphasis in these areas 
and/or relied on prime contractors to analyze and interpret 
developmental testing data. The Defense Science Board also found that 
many weapon programs were failing initial operational testing due to a 
lack of a disciplined systems engineering approach. Additionally, in 
2008 the National Academy of Sciences reported[Footnote 8] that there 
were no longer enough systems engineers to fill programs' needs. The 
Academy observed that DOD cannot outsource its technical and program 
management experience and intellect and still expect to acquire new 
systems that are both effective and affordable. 

Over the past several years, the Congress has called for DOD to 
improve its acquisition workforce and increase emphasis on systems 
engineering and developmental testing during weapon systems 
development. For example, the National Defense Authorization Act for 
Fiscal Year 2008[Footnote 9] required the Secretary of Defense to 
establish the DOD Acquisition Workforce Fund for the recruitment, 
training, and retention of DOD acquisition personnel, which includes 
the systems engineering and test and evaluation career fields. In 
addition, the Reform Act contains a number of systems engineering and 
developmental testing requirements aimed at helping weapon programs 
establish a solid foundation prior to the start of development. For 
example, the Deputy Assistant Secretaries are expected to review and 
approve acquisition planning documents for major defense acquisition 
programs, as well as monitor program activities. 

In response to the Reform Act legislation, DOD established new offices 
within the Office of the Secretary of Defense for systems engineering 
and developmental testing to provide assistance to program offices as 
they are developing their acquisition strategies prior to the start of 
development and then oversee program office efforts to implement the 
strategies. The systems engineering office has about 120 programs in 
its portfolio and the developmental test and evaluation office has 
about 250 programs. In addition, they provide advocacy, oversight and 
guidance for their respective workforces throughout DOD. The Secretary 
of Defense also announced plans to increase the number of people 
performing acquisition activities by almost 20,000 employees through 
new hiring actions and by converting contractor positions to 
government positions (insourcing) between fiscal years 2009 and 2015. 
[Footnote 10] The Strategic Human Capital Plan Update indicates that 
22 percent of the almost 20,000 position growth will be for systems 
engineering and 1 percent for test and evaluation, which would 
increase the size of the career fields by 16 percent and 5 percent, 
respectively. The majority of these new positions will be in the 
military services at headquarters, program offices, and test range 
locations. 

The services maintain test ranges for development testing activities. 
Currently, 24 ranges are designated as part of the Major Range and 
Test Facility Base (MRTFB)[Footnote 11] because they have unique test 
capabilities that are used by multiple services. The Test Resource 
Management Center is responsible for ensuring that the MRTFB is 
adequately funded and maintained while the services oversee the 
remaining non-MRTFB ranges and facilities, which perform more service- 
specific testing. 

DOD Is Continuing to Implement Reform Act Requirements, but Challenges 
Remain: 

The new offices of the Deputy Assistant Secretary of Defense for 
Systems Engineering and Developmental Test and Evaluation have 
continued to make progress implementing the Reform Act requirements. 
Both offices, which provide assistance to program managers and perform 
oversight for decision makers, have issued additional policies and 
guidance, and are overseeing and providing information on more weapon 
acquisition programs than they did last year. The systems engineering 
office will be staffed at 163 people in fiscal year 2012 and the 
developmental test and evaluation office will be staffed at 63 people, 
which is less than the developmental test and evaluation office 
originally projected. Yet, the offices are more reliant on contractor 
support than the Deputy Assistant Secretaries would like. In addition, 
many current and former testing officials continue to believe the 
developmental test and evaluation office does not have the resources 
or influence to effectively oversee and affect program decisions. 
However, it is unclear how many people are needed. 

Efforts to Implement Reform Act Requirements Are Progressing: 

DOD established the two offices for systems engineering and 
developmental test and evaluation within the Office of the Secretary 
of Defense in June 2009. Since then, both offices have increased 
staffing, which is enabling them to meet statutory requirements for 
assisting and overseeing their portfolios of defense acquisition 
programs. The table below shows the actual and planned workforces for 
both offices. 

Table 1: Offices of the Deputy Assistant Secretaries of Defense for 
Systems Engineering and Developmental Test and Evaluation Workforce 
Growth: 

Deputy Assistant Secretary for Systems Engineering: 

Workforce staffing (government and contractor employees): Government; 
June 2009 (actual): 17; 
Fiscal year 2010 (actual): 20; 
Fiscal year 2011 (authorized): 28; 
Fiscal year 2012 (planned): 28; 
Goal (planned): 28. 

Workforce staffing (government and contractor employees): Contractors; 
June 2009 (actual): 96; 
Fiscal year 2010 (actual): 107; 
Fiscal year 2011 (authorized): 114; 
Fiscal year 2012 (planned): 135; 
Goal (planned): 135. 

Workforce staffing (government and contractor employees): Total; 
June 2009 (actual): 113; 
Fiscal year 2010 (actual): 127; 
Fiscal year 2011 (authorized): 142; 
Fiscal year 2012 (planned): 163; 
Goal (planned): 163. 

Deputy Assistant Secretary for Developmental Test and Evaluation: 

Workforce staffing (government and contractor employees): Government; 
June 2009 (actual): 4; 
Fiscal year 2010 (actual): 10; 
Fiscal year 2011 (authorized): 9; 
Fiscal year 2012 (planned): 9; 
Goal (planned): 16. 

Workforce staffing (government and contractor employees): Contractors; 
June 2009 (actual): 20; 
Fiscal year 2010 (actual): 20; 
Fiscal year 2011 (authorized): 50; 
Fiscal year 2012 (planned): 50; 
Goal (planned): 50. 

Workforce staffing (government and contractor employees): Total; 
June 2009 (actual): 24; 
Fiscal year 2010 (actual): 30; 
Fiscal year 2011 (authorized): 63[A]; 
Fiscal year 2012 (planned): 63[A]; 
Goal (planned): 70[A]. 

Source: GAO presentation of information from the offices of the Deputy 
Assistant Secretary of Defense for Systems Engineering and the Deputy 
Assistant Secretary of Defense for Developmental Test and Evaluation. 

[A] Includes four detailees from the Test Resources Management Center. 

[End of table] 

Most of the staffing increases have been through hiring contractor 
employees. While both offices have also increased their government 
staff, hiring freezes have curbed their ability to hire additional 
government employees and for the developmental test and evaluation 
office to meet its authorized staffing goal. The systems engineering 
office was authorized 28 government employees for fiscal year 2011, 
but has not yet been given permission to advertise for all of the 
positions because of the hiring freeze. According to the Deputy 
Assistant Secretary for Developmental Test and Evaluation, the office 
was initially authorized to have 70 employees, but will be capped at 
63 in fiscal year 2011, 4 of which are detailees from the Test 
Resource Management Center. 

In their fiscal year 2010 joint annual report to the Congress, 
[Footnote 12] the Deputy Assistant Secretaries reported on their 
ongoing efforts to implement Reform Act requirements. The table below 
includes information that was included in the report or provided to us 
on their fiscal years 2009 and 2010 activities. As shown, the offices 
have continued or increased their activities. 

Table 2: Ongoing Efforts by Deputy Assistant Secretaries of Defense 
for Systems Engineering and Developmental Test and Evaluation to 
Implement Selected Reform Act Requirements: 

Reform Act requirement: Develop policies and guidance; 
Systems engineering: Fiscal year 2009: started updating systems 
engineering plan guidance and the Guide for Integrating Systems 
Engineering into DOD Acquisition Contracts. Published policy that 
expands reliability, availability, and maintainability guidance for 
acquisition programs and updated the Defense Acquisition Guidebook 
chapter on systems engineering. Fiscal year 2010: published Directive-
Type Memoranda on (1) Implementation of the Weapon Systems Acquisition 
Reform Act of 2009[A] and (2) Development Planning to Inform Materiel 
Development Decision Reviews and Support Analyses of Alternatives[B]; 
Developmental testing: Fiscal year 2009: published guidance on 
incorporating test and evaluation requirements into acquisition 
contracts. Updated required content in test and evaluation strategy 
and master plan documents to include reliability factors. Fiscal year 
2010: published Information Assurance Policy Crosswalk Working Group 
Report and drafted DOD Instruction establishing Developmental Test and 
Evaluation responsibilities. 

Reform Act requirement: Review, approve acquisition planning documents 
of major acquisition programs; 
Systems engineering: Fiscal year 2009: reviewed 22 and approved 16 
systems engineering plans. Fiscal year 2010: reviewed 25 and approved 
21 systems engineering plans; 
Developmental testing: Fiscal year 2009: reviewed and approved 25 
developmental test and evaluation plans. Fiscal year 2010: reviewed 
and approved 33 developmental test and evaluation plans and 4 test and 
evaluation strategies. 

Reform Act requirement: Monitor, review activities of major 
acquisition programs; 
Systems engineering: Fiscal year 2009: reviewed systems engineering 
activities for 35 programs and participated in 20 technical reviews. 
Fiscal year 2010: reviewed systems engineering activities for 39 
programs and participated in 31 technical reviews; 
Developmental testing: Fiscal year 2009: reviewed and assessed 
developmental testing activities for 17 programs. Fiscal year 2010: 
reviewed and assessed developmental testing activities for 38 programs. 

Reform Act requirement: Provide advocacy, oversight, and guidance for 
respective DOD acquisition workforce; 
Systems engineering: Fiscal year 2009: started assessing systems 
engineering competencies. Fiscal year 2010: continued competency 
assessment of workforce. Sponsored Defense Acquisition University 
course development and continued focus on education and training; 
Developmental testing: Fiscal years 2009: updated education and 
training requirements and validated certification requirements. Fiscal 
year 2010: continued fiscal year 2009 efforts and worked with 
components in developing long-term strategies to optimize workforce. 

Source: GAO presentation of Deputy Assistant Secretary of Defense for 
Systems Engineering and Deputy Assistant Secretary of Defense for 
Developmental Test and Evaluation information. 

[A] DOD Directive-Type Memorandum 09-027, Implementation of the Weapon 
Systems Reform Act of 2009 (Dec. 4, 2009, Incorporating Change 1, Oct. 
21, 2010). 

[B] DOD Directive-Type Memorandum 10-017, Development Planning to 
Inform Material Development Decision Reviews and Support Analysis of 
Alternatives (Sept. 13, 2010, Incorporating Change 1, May 16, 2011). 

[End of table] 

In addition to the progress highlighted in the table above, the 
offices are also taking actions in areas that they had not acted upon 
last year--issuing required guidance on the development and tracking 
of performance criteria and exercising a Reform Act option of 
designating the Deputy Assistant Secretary of Defense for 
Developmental Test and Evaluation for concurrent service as the 
Director of the Test Resource Management Center. 

* Developing performance criteria: The Reform Act requires the Deputy 
Assistant Secretaries, in coordination with an official designated by 
the Secretary of Defense, who was the Director of the Performance 
Assessments and Root Cause Analysis, to jointly issue guidance on the 
development and tracking of detailed measurable performance criteria 
for major defense acquisition programs. In response to this 
requirement, the Deputy Assistant Secretaries, in cooperation with the 
Director of Performance Assessments and Root Cause Analysis, have 
agreed that each office shall develop guidance within their respective 
functional areas in coordination with each other and openly share the 
data and findings of those performance criteria in conducting their 
oversight. 

The Deputy Assistant Secretary for Systems Engineering developed a set 
of time-based metrics to assess each program's ability to execute its 
system engineering plans and address risks the office had identified 
in prior reviews. The metrics measure program cost, schedule, 
staffing, reliability, availability and maintainability, software, 
integration, performance and manufacturing, and are to be incorporated 
into each program's systems engineering plan and evaluated at various 
points in the development process. Criteria developed by the Deputy 
Assistant Secretary for Developmental Test and Evaluation focus on 
early acquisition lifecycle activities to ensure that sound 
developmental testing planning is performed from the beginning of 
development. Other criteria measure program results and are meant to 
provide an objective foundation to assess a program's subsequent 
developmental testing performance as it approaches the production 
decision and the assessment of operational test readiness. The office 
plans to pilot test the metrics on six programs,[Footnote 13] with a 
goal of rolling them out to other programs by the end of 2011. 

* Changing leadership of Test Resource Management Center: Effective 
April 1, 2011, the Principal Deputy Under Secretary of Defense for 
Acquisition, Technology and Logistics designated the Deputy Assistant 
Secretary of Defense for Developmental Test and Evaluation for 
concurrent service as the Director of the Test Resource Management 
Center. DOD had not acted upon this optional Reform Act provision last 
year. Both offices will continue to be managed separately and report 
to different authorities--developmental test and evaluation office 
activities will be reported to the Assistant Secretary of Defense for 
Research and Engineering and Test Resource Management Center 
activities will be reported directly to the Under Secretary of Defense 
for Acquisition, Technology and Logistics.[Footnote 14] 

According to the Deputy Assistant Secretary of Defense for 
Developmental Test and Evaluation, the Principal Deputy Under 
Secretary for Acquisition, Technology and Logistics asked him to 
complete a study in the near future to identify efficiencies that can 
be obtained by merging some of the offices' activities. A study was 
started in July 2011. The Deputy Assistant Secretary for Developmental 
Test and Evaluation indicated that there could be some limitations on 
his ability to streamline management and reporting activities or shift 
resources between the organizations because the Test Resource 
Management Center is designated by statute to be a field activity and 
the organizations are funded separately. DOD has not studied the 
possible legal ramifications of combining the offices. 

In their fiscal year 2010 joint annual report to the Congress, the 
Deputy Assistant Secretaries also identified several focus areas for 
improvement. For example, the systems engineering office plans to 
reestablish the DOD Software Working Group to improve DOD's capability 
to address systemic software program issues. Among other things, the 
developmental test and evaluation office wants to develop a 
responsible test organization model that the services would use to 
designate the lead government test organization responsible for 
overseeing and/or conducting the developmental test and evaluation for 
an acquisition program. In addition, the office wants to issue a 
policy requiring programs to prioritize use of government capabilities 
and to provide a cost-benefit analysis when they decide to provide 
funding to use and/or develop test capabilities at prime contractor 
sites. 

In a departure from the fiscal year 2009 joint annual report,[Footnote 
15] the fiscal year 2010 report did not contain a discussion of the 
extent to which weapon acquisition programs are fulfilling their 
systems engineering or test and evaluation master plans.[Footnote 16] 
Further, the fiscal year 2010 report did not include a discussion of 
test and evaluation waivers or deviations that programs have received. 
Instead, the report identified the type of reviews or engagements both 
of the offices participated in for various programs. The Deputy 
Assistant Secretaries stated that they did not provide the information 
in the fiscal year 2010 report because they were directed to 
streamline the report. In May 2011, the Senate Armed Services 
Committee requested DOD to supplement the fiscal year 2010 report with 
this information as required by the Reform Act. DOD has not yet 
provided the information. 

The Deputy Assistant Secretaries believe they have had a positive 
influence on weapon acquisition programs over the past 2 years during 
milestone reviews with senior department leaders and through 
recommendations to program offices. Although the Assistant Secretary 
of Defense for Research and Engineering represents both systems 
engineering and developmental test positions at Defense Acquisition 
Board meetings, the Deputy Assistant Secretaries said that either 
they, or designees, also attend the meetings. They said that they have 
been asked by the Principal Deputy for Acquisition, Technology and 
Logistics to provide direct input about weapon acquisition programs at 
some of these meetings, including discussions on the Gray Eagle 
unmanned aircraft system, Ohio Replacement submarine, and the Joint 
Strike Fighter aircraft programs. The Deputy Assistant Secretaries 
also said that program offices are making changes based on the 
recommendations made by their offices during regular program 
assessments and technical reviews. 

Organizational Challenges Remain: 

While the Deputy Assistant Secretaries have made progress implementing 
the Reform Act requirements, we identified several organizational 
challenges that could limit their effectiveness. A summary of each of 
these challenges is presented below. 

* Reliance on contractor employees: The Deputy Assistant Secretaries 
rely heavily on contractors to help perform office activities. In 
fiscal year 2010, for example, nearly 85 percent of the staff in the 
systems engineering office and 67 percent of the staff in the 
developmental test and evaluation office were contractors. Both Deputy 
Assistant Secretaries would like to have a larger proportion of 
government employees because they believe it is important to maintain 
a core cadre of people with the required institutional knowledge and 
skills to support current and future program office needs. However, 
they are not optimistic about their chances of getting additional 
government employees because of a civilian hiring freeze. 

* Developmental Test and Evaluation Office influence: Current and 
former DOD test and evaluation officials continue to believe the 
developmental test and evaluation office could be more effective in 
its oversight role with the proper influence. For example, they 
pointed out that the office's primary avenue for voicing concerns 
about weapon acquisition programs to senior leaders is at overarching 
integrated product team meetings that take place in preparation for 
Defense Acquisition Board meetings. The integrated product team leader 
[Footnote 17] ultimately decides which organizations will get to 
present issues at the Defense Acquisition Board meetings. Current 
testing officials told us that in some cases developmental testing 
issues do not make the Defense Acquisition Board meeting agendas, 
which is a concern to officials who believe the developmental test and 
evaluation office should provide independent assessments of weapon 
acquisition programs directly to senior leaders. In addition, 
officials said that the deputy directors who attend the overarching 
meetings are not at the senior executive level like other meeting 
attendees, which officials said in some cases has reduced their 
relative influence during these meetings. To ensure developmental 
testing information, such as the assessment of operational test 
readiness, receives appropriate consideration by senior leaders, the 
Defense Science Board recommended in 2008 that the office report 
developmental testing issues directly to the Deputy Under Secretary of 
Defense for Acquisition and Technology. Currently, the office reports 
through an intermediary--the Assistant Secretary of Defense for 
Research and Engineering. 

Even though the systems engineering office reports through the same 
reporting channel as the developmental test and evaluation office, the 
Deputy Assistant Secretary for Systems Engineering believes his office 
has the appropriate amount of influence. This is because the systems 
engineering office's primary emphasis is on assisting program managers 
in the development of their systems engineering plans. In contrast, 
the Deputy Assistant Secretary for Developmental Test and Evaluation 
believes his office should put about equal effort into assisting and 
assessing program office activities. 

* Developmental Test and Evaluation Office resources: Information 
provided by the developmental test and evaluation office shows the 
office can not provide full coverage of its portfolio of about 250 
acquisition programs given its current workforce.[Footnote 18] For 
example, in fiscal year 2010 when the office had 30 people, 89 
programs (36 percent) did not receive any support, including the Air 
and Missile Radar program and sub elements of the Early Infantry 
Brigade Combat Team program, such as the Small Unmanned Ground 
Vehicles and Unmanned Aircraft System. 

Although staffing has doubled between fiscal years 2010 and 2011, the 
office still has not been able to support all the programs the Deputy 
Assistant Secretary for Developmental Test and Evaluation believes it 
should. The Deputy Assistant Secretary for Developmental Test and 
Evaluation said the office has had to be selective in using its 
resources and he has introduced a "triage" strategy for dealing with 
the overload of programs relative to the office's workforce. This 
strategy includes dropping virtually all programs below Acquisition 
Category I from the developmental testing oversight list and 
eliminating oversight of some major automated information systems. For 
the most part, the office focuses its efforts on major defense 
acquisition programs between milestone B (development start) and 
milestone C (the production decision) in order to retain sufficient 
depth with programs in development. It is providing minimal coverage 
to programs prior to the start of development, which is the most 
opportune time to influence a program's acquisition strategy. Based on 
the "triage" strategy, the Deputy Assistant Secretary for 
Developmental Test and Evaluation believes only about half of the 
current portfolio of about 250 programs would receive the level of 
support he believes is needed with a staff of 63 people. On the other 
hand, the Deputy Assistant Secretary for Systems Engineering believes 
his office, which is supposed to have 163 people in fiscal year 2012, 
will have enough staff to oversee its portfolio of 120 programs. 

While current and former DOD testing officials provided reasons for 
increasing the size of the developmental test and evaluation office, 
they could not specify the appropriate size for the office, as they 
indicated the issue has not been thoroughly analyzed. Officials 
familiar with the establishment of the office told us that three 
staffing scenarios were considered prior to the office being 
established--a high, medium, and low staffing scenario--but they 
indicated that no detailed analysis was done to support any of the 
scenarios. Under the high staffing scenario, the developmental test 
and evaluation office would have had 250 people, which would have 
matched the size of the office of the Director of Operational Test and 
Evaluation. The medium level of staffing, 120-150 people, was based on 
the size of a legacy developmental testing organization and the low 
level, which called for 90 people, was based on an assumption about 
the number of programs each person would be responsible for 
overseeing. As shown earlier in table 1, the staffing goal is 70 
people, which is fewer than the lowest staffing scenario considered. 

Former testing officials believe an opportunity exists to both 
increase the developmental testing office's influence and address 
resource concerns by merging Test Resource Management Center and 
developmental testing office activities. They believe this would give 
the Deputy Assistant Secretary for Developmental Test and Evaluation 
the most flexibility in how to allocate resources. They also pointed 
out that in the early 1990s, oversight of all developmental test and 
evaluation activities had been under one organization that reported 
directly to the Under Secretary of Defense for Acquisition.[Footnote 
19] 

Workforce Challenges Could Affect Services' Systems Engineering and 
Developmental Testing Efforts: 

The military services have been increasing the number of people in 
their systems engineering and test and evaluation career fields, but 
challenges exist that could impede future workforce growth plans as 
well as testing at the ranges, if not properly addressed. The services 
planned to increase their systems engineering and test and evaluation 
career fields by about 5,000 people (14 percent) and about 300 people 
(4 percent), respectively, between fiscal years 2009 and 2015 through 
hiring actions and by insourcing contractor positions. These increases 
are part of DOD's overall efforts to increase the number of people in 
acquisition career fields. The services have increased the systems 
engineering career field by about half of their projections and 
exceeded their planned growth for the test and evaluation career field 
through the end of fiscal year 2010. However, budget cuts and a 
clarification in DOD's insourcing approach may make hiring civilians 
more challenging in the future. The services' developmental test 
ranges are also experiencing declining budgets, as the fiscal year 
2012 budget includes cuts of nearly $1.2 billion over the next 5 years 
to support accounts that pay for overhead costs. The services plan to 
cut range personnel in response to the budget reductions, which could 
offset some of the workforce gains they have already achieved. They 
have not yet determined how the cuts will be allocated across the 
ranges or the impact they will have on meeting program office needs. 
Further budget cuts are possible based on the recent debt ceiling 
agreement, but details are unknown. Currently, the services lack 
common performance metrics that would assist in making funding 
decisions. 

Services Have Increased Their Systems Engineering and Test and 
Evaluation Career Fields: 

The services planned to increase their acquisition systems engineering 
and test and evaluation career fields by 14 percent and 4 percent, 
respectively, between fiscal years 2009 and 2015 through hiring and 
insourcing actions. This would increase the overall systems 
engineering career field by about 5,000 people from about 35,000 to 
40,000 people and the test and evaluation career field by almost 300 
people from about 7,400 to 7,700 people. These new positions would be 
located at service headquarters, program offices, and test range 
locations. It should be noted that insourcing actions alone do not 
result in real growth to the collective number of civilians, military, 
and contractors performing an activity because it only involves the 
transfer of positions from contractors to the government. Hiring 
additional civilian employees, on the other hand, would result in 
growth, assuming the contractor and military workforces remain stable. 

The following table shows the baseline, goal, and current number of 
civilian and military personnel performing systems engineering and 
test and evaluation activities for each of the services at the end of 
fiscal year 2010, as well as the percentage of the growth goal target 
achieved. 

Table 3: Actual and Planned Military Service Systems Engineering and 
Test and Evaluation Career Field Growth: 

Systems engineering career field: 

Air Force: 
Baseline[A]: Systems engineering career field: 6,380; 
Growth goal by the end of fiscal year 2015[B]: 7,059; 
Current workforce as of September 30, 2010: 7,575; 
Percentage of growth goal target achieved as of September 30, 2010: 
107%. 

Army: 
Baseline[A]: Systems engineering career field: 10,615; 
Growth goal by the end of fiscal year 2015[B]: 12,076; 
Current workforce as of September 30, 2010: 10,938; 
Percentage of growth goal target achieved as of September 30, 2010: 
91%. 

Navy: 
Baseline[A]: Systems engineering career field: 17,961; 
Growth goal by the end of fiscal year 2015[B]: 20,870; 
Current workforce as of September 30, 2010: 19,012; 
Percentage of growth goal target achieved as of September 30, 2010: 
91%. 

Totals: 
Baseline[A]: Systems engineering career field: 34,956; 
Growth goal by the end of fiscal year 2015[B]: 40,005; 
Current workforce as of September 30, 2010: 37,525; 
Percentage of growth goal target achieved as of September 30, 2010: 
94%. 

Test and evaluation career field: 

Air Force: 
Baseline[A]: Systems engineering career field: 2,622; 
Growth goal by the end of fiscal year 2015[B]: 2,566; 
Current workforce as of September 30, 2010: 2,840; 
Percentage of growth goal target achieved as of September 30, 2010: 
111%. 

Army: 
Baseline[A]: Systems engineering career field: 2,135; 
Growth goal by the end of fiscal year 2015[B]: 2,297; 
Current workforce as of September 30, 2010: 2,211; 
Percentage of growth goal target achieved as of September 30, 2010: 
96%. 

Navy: 
Baseline[A]: Systems engineering career field: 2,652; 
Growth goal by the end of fiscal year 2015[B]: 2,829; 
Current workforce as of September 30, 2010: 2,977; 
Percentage of growth goal target achieved as of September 30, 2010: 
105%. 

Totals: 
Baseline[A]: Systems engineering career field: 7,409; 
Growth goal by the end of fiscal year 2015[B]: 7,692; 
Current workforce as of September 30, 2010: 8,028; 
Percentage of growth goal target achieved as of September 30, 2010: 
104%. 

Source: Military service workforce numbers presented in DOD's April 
2010 Strategic Human Capital Plan Update and military service budget 
estimate submissions. 

Note: Workforce numbers include civilian and military positions. Navy 
systems engineering workforce also includes science and technology 
positions, which officials estimate to be less than 2 percent of the 
total. 

[A] Air Force and Army baseline is the end of fiscal year 2008 and the 
Navy baseline is the end of fiscal year 2009. 

[B] Air Force and Army projections are based on the fiscal year 2010 
budget estimate submission and the Navy projections are based on the 
fiscal year 2011 budget estimate submission. 

[End of table] 

Through the end of fiscal year 2010, the services have made 
significant progress towards increasing the two career fields. For 
example, the services increased the systems engineering career field 
by about 2,600 people, about half of the projected growth. 
Collectively, they have achieved 94 percent of the growth goal target 
planned for that career field. The services achieved 104 percent of 
the growth goal target planned for the test and evaluation career 
field by adding over 600 people. Information provided by the Army and 
Navy shows about 74 percent of their systems engineering career field 
growth was through new hires and the remaining 26 percent was through 
insourcing.[Footnote 20] About 86 percent of their test and evaluation 
career field growth was through new hires and the remaining 14 percent 
was through insourcing. 

Budget Cuts and Other Challenges Could Affect Overall Workforce Growth: 

Recently proposed budget changes would result in modifications to the 
services' workforce growth plans for both acquisition career fields. 
[Footnote 21] The services still plan to increase the systems 
engineering career field between fiscal years 2011 and 2015, but with 
about 800 positions less than originally planned. As a result, career 
field growth would be 10 percent instead of 14 percent. On the other 
hand, the services plan to hire 400 more people than they originally 
planned for the test and evaluation career field, despite the fact 
that the overall number of DOD civilians is frozen at fiscal year 2010 
levels. This would result in a 6 percent growth to the career field 
instead of 4 percent. 

Service officials stated that achieving additional career field 
growth, however, could be difficult because of a recent clarification 
in DOD's insourcing policy. According to a March 2011 memorandum 
issued jointly by the Under Secretary of Defense for Acquisition, 
Technology and Logistics and the Under Secretary of Defense 
Comptroller/Chief Financial Officer, a case-by-case approach will be 
used for additional insourcing of acquisition functions based on 
critical need, whether a function is inherently governmental, and the 
benefit demonstrated by a cost-benefit analysis. The memorandum also 
states that additional insourcing must be supported by current budget 
levels. In cases where added insourcing would breach the existing 
civilian ceilings, the proposal and associated justification must be 
provided to the Director of Human Capital Initiatives and then the 
proposal will be reviewed by the two Under Secretaries of Defense 
issuing the memorandum and approved by the Deputy Secretary of 
Defense. Of the nearly 2,000 people the Army and Navy now plan to hire 
for the systems engineering career field between fiscal years 2011 and 
2015, 96 percent is to be achieved through insourcing. And about 40 
percent of the over 200 people they plan to hire for the test and 
evaluation career field are to be through insourcing. 

Although the services have increased their test and evaluation staff, 
each of the 12 test ranges we visited experienced a mixture of 
recruiting, hiring, training, or retention challenges. For example, 
Pacific Missile Range Facility officials stated the range's location 
on the island of Kauai, Hawaii, has made it difficult to recruit 
personnel due to a lower pay grade structure and higher costs of 
living compared to other test ranges. Range officials stated that it 
can take months to hire new employees, forcing many qualified 
applicants to seek employment opportunities elsewhere. Test ranges 
have begun using acquisition workforce expedited hiring authority, 
which allows DOD components to streamline the process for making 
offers to qualified acquisition personnel. However, ranges have 
interpreted this authority differently. Service test and evaluation 
executives said they would clarify the policy to the ranges based upon 
our observations to ensure the policies are fully understood. Most of 
the test range officials we spoke with also had concerns about the 
timeliness and quality of the Defense Acquisition University's 
training classes.[Footnote 22] The Defense Acquisition University is 
coordinating with the services to address the increased demand for 
acquisition training. Finally, some test ranges are having difficulty 
retaining engineers or some software specialties because the private 
sector can pay them a higher salary. Officials at Aberdeen Test 
Center, Maryland, are concerned that many of their employees will take 
higher paying jobs with organizations that are moving into their area 
as the result of a Base Realignment and Closure decision. 

Impact of Proposed Budget Cuts on Ranges Has Not Been Determined: 

The services' developmental test range budgets are being reduced, but 
the full impact of the cuts has yet to be determined. The fiscal year 
2012 President's Budget includes cuts of nearly $1.2 billion (17 
percent) through fiscal year 2015 to the ranges' institutional funding 
accounts that fund operational and overhead expenses such as 
personnel, facilities, and equipment costs.[Footnote 23] According to 
service officials, the budget cuts are a result of direction they 
received late in the fiscal year 2012 budget cycle to keep their 
civilian workforce at the same level as they had at the end of fiscal 
year 2010. Research, development, test and evaluation accounts that 
fund range testing activities absorbed a large portion of the cuts. 

As shown in figure 1, the services had already planned to reduce their 
MRTFB research, development, test and evaluation funding between 
fiscal years 2012 and 2015. Although range budgets reached a peak in 
fiscal year 2011, budget estimates for that year projected funding 
decreases in subsequent years. According to service officials these 
reductions were expected due to expected workload reductions, savings 
associated with implementing efficiency initiatives, and a civilian 
pay freeze. The fiscal year 2012 budget contains additional, more 
significant decreases beyond those forecasted in the previous year's 
budget. In total, this budget provides $5.7 billion for the ranges 
between fiscal years 2012 and 2015--a 17 percent reduction from the 
$6.9 billion forecast in the fiscal year 2011 budget. Most notably, 
the Army and Air Force range budgets were each cut over $100 million 
in fiscal year 2012 alone. 

Figure 1: MRTFB Operation & Support Funding for Fiscal Years 2006-2015: 

[Refer to PDF for image: multiple line graph] 

Fiscal year 2012 Fiscal year 2011 

Fiscal year: 2006; 
Fiscal year 2012: $1.625 billion; 
Fiscal year 2011: $1.625 billion. 

Fiscal year: 2007; 
Fiscal year 2012: $1.704 billion; 
Fiscal year 2011: $1.704 billion. 

Fiscal year: 2008; 
Fiscal year 2012: $1.691 billion; 
Fiscal year 2011: $1.691 billion. 

Fiscal year: 2009; 
Fiscal year 2012: $1.725 billion; 
Fiscal year 2011: $1.715 billion. 

Fiscal year: 2010; 
Fiscal year 2012: $1.725 billion; 
Fiscal year 2011: $1.741 billion. 

Fiscal year: 2011; 
Fiscal year 2012: $1.780 billion; 
Fiscal year 2011: $1.780 billion. 

Fiscal year: 2012; 
Fiscal year 2012: $1.478 billion; 
Fiscal year 2011: $1.723 billion. 

Fiscal year: 2013; 
Fiscal year 2012: $1.422 billion; 
Fiscal year 2011: $1.712 billion. 

Fiscal year: 2014; 
Fiscal year 2012: $1.435 billion; 
Fiscal year 2011: $1.735 billion. 

Fiscal year: 2015; 
Fiscal year 2012: $1.429 billion; 
Fiscal year 2011: $1.765 billion. 

Source: GAO presentation of the military information for the fiscal 
year 2011 and fiscal year 2012 President's Budget. 

[End of figure] 

The Chairman of the Senate Armed Services Committee recently expressed 
concern that these proposed reductions threaten to seriously undermine 
the implementation of the Reform Act's requirement to rebuild DOD's 
systems engineering and developmental testing organizations and 
requested that the proposed cuts be reviewed. The Army and Air Force 
are in the process of reviewing these cuts and determining where they 
will be made. To minimize the impact on these activities, they are 
examining potential reductions to their overhead or administrative 
staff at headquarters and field locations. According to service 
headquarters officials, if DOD's proposed budget cuts between fiscal 
years 2012 and 2015 remain intact, they may have to cut some of their 
developmental test capabilities as well as personnel who conduct 
tests. This could offset some of the test and evaluation career field 
gains already achieved over the past 2 years. Test officials said that 
reductions in test personnel could limit the amount of testing 
performed on weapon acquisition programs, which could increase the 
risk associated with those programs and/or result in an extension of a 
program's test schedule. Final decisions on where to take the cuts 
have not yet been made. Therefore, we could not determine the impact 
funding cuts would have on the ranges' ability to meet program office 
testing needs. 

Range officials indicated that prior to these proposed cuts they had 
difficulty maintaining their test capabilities. The National Defense 
Authorization Act for Fiscal Year 2003[Footnote 24] required the 
Secretary of Defense by fiscal year 2006 to establish the funding 
objective of ensuring that the overhead and institutional costs of the 
MRTFB ranges were fully funded through the major test and evaluation 
investment accounts and to ensure that DOD customers were charged not 
more than the direct costs of testing. The law also required DOD to 
establish the Test Resource Management Center and required that the 
director of that organization certify whether the services' proposed 
budgets for test and evaluation activities are adequate. To comply 
with this law, the services increased their range operating budgets by 
over 50 percent in fiscal year 2006. Although MRTFB funding was fairly 
stable between fiscal years 2006 and 2011,[Footnote 25] range 
officials said they have had difficulties maintaining their test 
capabilities because the infrastructure is aging and operating costs 
for expenses like utilities and fuel have grown at a higher rate than 
their overall funding. As a result, officials said they have had to 
move money from their range modernization accounts to fund operating 
costs, only fix things that are broken or in emergency status, or fund 
capability upgrades over several years instead of a shorter period of 
time. According to range officials, these challenges are less of a 
problem for non-MRTFB ranges like the Redstone Test Center and other 
test/laboratory activities funded through working capital fund 
accounts, where customers pay direct and overhead costs.[Footnote 26] 

Service officials discussed several strategies or a combination of 
those strategies that the services could use to lessen the impact of 
budget cuts and funding concerns. One strategy is that the ranges may 
have to curtail certain testing, mothball or close test facilities, or 
consolidate test capabilities. Another option could be to move certain 
test capabilities out of the MRTFB management structure, thereby 
allowing ranges to charge customers the full cost of testing. 
Additionally, on the basis of our range observations, service 
officials said more specific guidance for interpreting financial 
management regulations on what constitutes direct and overhead charges 
for MRTFB operations could be developed to clarify and standardize the 
types of costs that can be passed on to customers. Service test and 
evaluation executives and the Deputy Assistant Secretary for 
Developmental Test and Evaluation believe the ranges' current policy 
interpretation is too restrictive, inconsistent across the MRTFB, and 
constrains service options as they strive to sustain MRTFB 
capabilities. The Test Resource Management Center recently established 
a team to study the issue. 

Services Lack Common Range Performance Measures to Aid Decision Making: 

The services have not implemented common range performance measures 
that would help them justify funding and assist them in making 
workforce decisions, how best to allocate funding, or make difficult 
decisions about mothballing, closing, or consolidating test 
capabilities, if necessary. Although ranges collect performance data 
relevant to their operations, these indicators may not be useful in 
making higher-level infrastructure decisions that cut across several 
ranges. According to service officials, it is very difficult to 
develop a common set of performance measures because of the uniqueness 
of each range and its variable capacity. We have found that 
performance measures can assist managers in making decisions about 
future strategies, planning and budgeting, identifying priorities, and 
allocating resources.[Footnote 27] 

Some efforts are under way to provide decision makers more 
information, but they are still in process and are not yet approved or 
implemented. The Test Resource Management Center has sponsored an 
effort to develop a comprehensive set of range metrics. According to 
the services, early efforts were not successful and were not well 
received. While the Air Force plans to use metrics resulting from the 
Test Resource Management Center's effort, the Army is evaluating the 
development of a readiness reporting system and a workload model for 
its ranges that could provide a better basis for investment or 
divestment decisions. The Navy is also in the process of developing 
metrics for its MRTFB test capabilities on the systems' condition, 
capacity, competency, and importance. Once developed, these metrics 
are expected to assist decision makers in directing future investment 
funding and to ensure test capabilities are adequate to support 
programs. 

Conclusions: 

The Reform Act points to the need for DOD to develop more robust 
systems engineering and developmental test and evaluation 
capabilities. DOD established new organizations and the services 
developed resource plans in order to increase emphasis on these 
activities. From an organizational standpoint, the offices of the 
Deputy Assistant Secretaries for Systems Engineering and Developmental 
Test and Evaluation are providing assistance to program managers in 
developing acquisition plans and providing oversight of program 
efforts for the Under Secretary of Defense for Acquisition, Technology 
and Logistics and the Congress. However, the developmental testing 
office is not as robust or efficient as it could be in part due to 
resource and organizational constraints. In addition, while the 
military services have made significant progress to date in increasing 
their systems engineering and developmental testing workforce 
capabilities, planned workforce reductions could offset these gains. 

It is incumbent upon DOD to provide the most effective systems 
engineering and developmental testing capability it can afford. It 
should have a sound analytical basis for establishing and resourcing 
that capability. However, it is not clear that DOD is yet at this 
point, especially for developmental testing. DOD has not conducted 
analysis on the right size of the developmental test and evaluation 
office or captured data that would reinforce range funding and 
workforce actions, or suggest needed adjustments. Additionally, the 
Deputy Assistant Secretary for Developmental Test and Evaluation 
believes that there may also be a statutory provision that limits his 
ability to achieve efficiencies and address office challenges. To the 
extent DOD cannot provide adequate systems engineering and 
developmental testing support to its weapons portfolio, the risks of 
executing the portfolio within cost and schedule are increased. 

Recommendations for Executive Action: 

We recommend that the Secretary of Defense take the following two 
actions: 

* assess the resources and influence needed by the developmental test 
and evaluation office to assist and oversee defense acquisition 
programs, including: 

- the number of defense acquisition programs that can be supported by 
different developmental test and evaluation office staffing levels, 
including specifying the total number of personnel, the mix of 
government and contractor employees, and the number of senior 
executive service personnel needed for each of these staffing levels; 

- whether the Test Resource Management Center and the office of the 
Deputy Assistant Secretary for Developmental Test and Evaluation 
should be combined or resources shifted between organizations to more 
effectively support the activities of both organizations and if so, 
identify for Congress any statutory revisions that would be necessary; 
and: 

- the proper reporting channel, taking into account the decision on 
whether or not to combine the organizations, the statutory oversight 
requirements, and the level of influence needed to oversee and assess 
program office developmental testing and service budgeting activities. 

* develop a plan to implement the results of the assessment. 

We also recommend that the Secretary of Defense, with input from the 
military services, take the following two actions: 

* develop metrics to assess the MRTFB test capabilities (expanding to 
DOD non-MRTFB, and non-DOD government test facilities once an approved 
set of metrics are in place supporting the MRTFB), justify funding, 
and assist in making decisions on the right-sizing of personnel, how 
best to allocate funding, or make future decisions on whether to 
mothball, shut down, or consolidate test facilities. These efforts 
should be coordinated with the Test Resource Management Center. 

* report the impact budget cuts reflected in the fiscal year 2012 
budget, as well as the insourcing policy clarification, will have on 
their (1) total workforce (civilians, military, and contractors) that 
support both of these activities and (2) ability to meet program 
office systems engineering and developmental test and evaluation needs. 

Matter for Congressional Consideration: 

Contingent upon the results of DOD's assessment, the Congress may want 
to consider revising any applicable statutory provisions necessary to 
allow for DOD to combine or shift resources between the Test Resource 
Management Center and the office of the Deputy Assistant Secretary for 
Developmental Test and Evaluation. 

Agency Comments and Our Evaluation: 

DOD provided us with written comments on a draft of this report. DOD 
concurred with two recommendations and partially concurred with two 
others. DOD offered suggested wording changes to the recommendations 
where it partially agreed to offer greater clarity. We agreed with the 
suggested changes and reworded our recommendations accordingly. DOD's 
comments appear in appendix II. DOD also provided technical comments, 
which we incorporated as appropriate in the report. 

In its response, DOD noted that the Deputy Assistant Secretary for 
Developmental Test and Evaluation has directed a study to assess the 
resources and influence needed by the developmental test and 
evaluation office to assist and oversee defense acquisition programs. 
The Deputy Assistant Secretary for Development Test and Evaluation 
will develop a plan to implement actionable recommendations from the 
results of the assessment, with the approval of the Under Secretary of 
Defense for Acquisition, Technology and Logistics. The Deputy 
Assistant Secretary for Developmental Test and Evaluation also plans 
to establish a working group, with participation from the military 
services, to develop metrics to assess MRTFB test capabilities, 
justify funding, and assist in making decisions on human capital and 
test facilities management. Finally, the Deputy Assistant Secretaries 
for Systems Engineering and Developmental Test and Evaluation plan to 
identify any impacts to the state of the workforce due to funding 
modifications or DOD's workforce policy updates in their joint annual 
report to the Congress. 

We are sending copies of this report to the Secretary of Defense, the 
Director of the Office of Management and Budget, and interested 
congressional committees. We will also make copies available at no 
charge on the GAO web site at [hyperlink,http://www.gao.gov]. 

If you have any questions about this report or need additional 
information, please contact me at (202) 512-4841 or sullivanm@gao.gov. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. Key contributors 
to this report are listed in appendix III. 

Signed by: 

Michael J. Sullivan: 
Director: 
Acquisition and Sourcing Management: 

[End of section] 

Appendix I: Scope and Methodology: 

This report examines the military services' systems engineering and 
developmental testing workforce capabilities and the Department of 
Defense's (DOD) efforts to implement Weapon Systems Acquisition Reform 
Act of 2009 (Reform Act) requirements.[Footnote 28] Specifically, we 
examined (1) the progress DOD has made in implementing the Reform 
Act's systems engineering and developmental testing requirements and 
(2) whether there are challenges at the military service level that 
could affect their systems engineering and developmental testing 
activities. 

To measure DOD's progress in implementing Reform Act requirements 
related to systems engineering and developmental testing, we 
interviewed officials and collected pertinent documents from the 
offices of the Deputy Assistant Secretary of Defense for Systems 
Engineering and Developmental Test and Evaluation. Specifically, we 
collected information on their efforts to develop additional policy 
and guidance; review and approve acquisition planning documents of 
major acquisition programs; monitor and review activities of major 
acquisition programs; and develop guidance for the development of 
performance metrics to use on weapon acquisition programs. We also 
reviewed their staffing plans and questioned the Deputy Assistant 
Secretaries, as well as former DOD developmental testing experts on 
whether each of the offices has the necessary amount of resources and 
influence to fulfill their missions. 

In order to determine whether there are challenges at the military 
service level that could affect future systems engineering and 
developmental testing activities, we looked at planned and actual 
workforce growth and developmental test range activities, and 
specifically at the following. 

* We collected and compared the military services' original workforce 
growth projections with their current workforce projections for the 
systems engineering and test and evaluation career fields. For this 
report, when we use the term systems engineering career field, we are 
referring to the Systems Planning, Research Development, and 
Engineering--Systems Engineering and Program Systems Engineering 
career fields. The Air Force and Army's original plans were based on 
their fiscal year 2008 budget estimate submissions[Footnote 29] and 
the Navy's was based on its fiscal year 2009 budget estimate 
submission. Current plans for each of the services were based on their 
fiscal year 2012 budget estimate submissions. Workforce data covered 
the fiscal years 2009-2015 timeframe for both the original and current 
plans. We interviewed officials within the Office of the Secretary of 
Defense and military services to determine the underlying causes for 
variances between the two plans and to determine how the Secretary of 
Defense's cost efficiency measures are affecting new hiring and 
insourcing plans. 

Navy officials stated that they used fiscal year 2009 as their 
baseline because before that time, information from various commands 
was not centralized and they could not verify the accuracy of the 
numbers. Navy officials stated that they have confidence in the 
validity of the numbers that support the fiscal year 2009 baseline 
because factors that affect workforce counts, such as recodes, 
decodes, reassignments, and attrition, are treated consistently. 

* We compared the services' baseline workforce data with end of fiscal 
year 2010 workforce numbers to determine how much growth occurred in 
the two career fields over the past 2 fiscal years. The Air Force's 
and Army's workforce baseline was the end of fiscal year 2008 and the 
Navy's was the end of fiscal year 2009. We assessed the reliability of 
these data by interviewing knowledgeable officials in the Air Force, 
Army, and Navy about the processes they use to ensure both the 
integrity and reliability of their manpower workforce databases used 
to track acquisition personnel. We determined that the data were 
sufficiently reliable for the purposes of this report. 

* We conducted site visits to 12 ranges and facilities--11 of these 
are designated as part of the Major Range and Test Facility Base 
(MRTFB) and one is a non-MRTFB range--to determine challenges 
affecting DOD's ability to conduct developmental testing. We focused 
our discussions on a broad range of topics, including funding, 
workforce, range facilities and instrumentation, encroachment, 
contractor duplication of DOD test facilities, and test resource 
capacity and demand. We selected four ranges per service based on 
geographical diversity, level of funding, type of testing, and 
recommendations from the Office of the Secretary of Defense and the 
services. The test ranges we visited were: Aberdeen Test Center, 
Maryland; Air Force Flight Test Center, California; Arnold Engineering 
Development Center, Tennessee; Atlantic Undersea Test and Evaluation 
Center, Bahamas; 46th Test Group, New Mexico; 46th Test Wing, Florida; 
High Energy Laser Systems Test Facility, New Mexico; Naval Air Warfare 
Center - Aircraft Division, Maryland; Naval Air Warfare Center - 
Weapons Division, California; Pacific Missile Range Facility, Hawaii; 
Redstone Test Center, Alabama; and White Sands Missile Range, New 
Mexico. 

* Finally, we compared developmental test range funding included in 
the President's Budget Future Year's Defense Plan for fiscal year 2011 
and 2012. We discussed differences between the two plans and how 
decreases in the fiscal year 2012 plan would affect developmental 
testing activities with officials from the Test Resource Management 
Center, the military services, and the office of the Deputy Assistant 
Secretary of Defense for Developmental Test and Evaluation. We also 
discussed with these officials the progress DOD and the services have 
made in developing metrics that could be used to make workforce and 
investment decisions. 

We conducted this performance audit from September 2010 to September 
2011 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Comments from the Department of Defense: 

Assistant Secretary Of Defense: 
Research And Engineering: 
3030 Defense Pentagon: 
Washington, DC 20301-3030: 

September 9, 2011: 

Mr. Michael Sullivan: 
Director, Acquisition and Sourcing Management: 
U.S. Government Accountability Office: 
441 G Street, N.W. 
Washington, DC 20548: 

Dear Mr. Sullivan: 

This is the Department of Defense (DoD) response to the GAO Draft 
Report 11-806, "Weapons Acquisition Reform: Actions Needed to Address 
Systems Engineering and Developmental Testing Challenges," dated 
August 5, 2011, (GAO Code 120936). Detailed comments on the report 
recommendations are enclosed. Detailed comments on factual information 
within the body of the report have been forwarded to the GAO action 
officer separately. 

The Department appreciates the opportunity to respond to your draft 
report and looks forward to working with you as we continue to ensure 
a strong and capable Defense acquisition capability. 

Sincerely, 

Signed by: 

Zachary J. Lemnios: 

Enclosure: As stated: 

[End of letter] 

GAO Draft Report Dated August 5, 2011: 
GAO-11-806 (GAO CODE 120936): 

"Weapons Acquisition Reform: Actions Needed To Address Systems 
Engineering And Developmental Testing Challenges" 

Department Of Defense Comments To The GAO Recommendations: 

Recommendation 1: The GAO recommends that the Secretary of Defense 
assess the resources and influence needed by the developmental test 
and evaluation office to assist and oversee defense acquisition 
programs, including: 

* the number of defense acquisition programs that can be supported by 
different developmental test and evaluation office staffing levels, 
including specifying the total number of personnel, the mix of 
government and contractor employees, and the number of senior 
executive service personnel needed for each of these staffing levels; 

* whether the Test Resource Management Center and the office of the 
Deputy Assistant Secretary for Developmental Test and Evaluation 
should be combined or resources shifted between organizations to 
support developmental testing activities and if so, identify for 
Congress any statutory revisions that would be necessary; and; 

* the proper reporting channel, taking into account the decision on 
whether or not to combine the organizations, the statutory oversight 
requirements, and the level of influence needed to oversee and assess 
program office developmental testing and service budgeting activities. 

DoD Response: Partially Concur. As written, the language is overly 
restrictive. The second sub-bullet recommends assessing "whether the 
Test Resource Management Center and the office of the Deputy Assistant 
Secretary for Developmental Test and Evaluation should be combined or 
resources shifted between organizations to support developmental 
testing activities..." DoD recommends amending the language to "... 
whether the Test Resource Management Center and the office of the 
Deputy Assistant Secretary for Developmental Test and Evaluation 
should be combined or resources shifted between organizations to more 
effectively support the activities of both organizations..." 

Rationale: The Under Secretary of Defense for Acquisition, Technology 
and Logistics tasked the Deputy Assistant Secretary of Defense for 
Developmental Test and Evaluation to review the two organizations to 
identify additional efficiencies, develop alternatives and to report 
the findings to the Under Secretary of Defense for Acquisition, 
Technology and Logistics for review and decision. 

The Deputy Assistant Secretary of Defense for Developmental Test and 
Evaluation has directed a study to assess the resources and influence 
needed by the developmental test and evaluation office to assist and 
oversee defense acquisition programs. 

Recommendation 2: The GAO recommends that the Secretary of Defense 
develop a plan to implement the results of the assessment. 

DoD Response: Concur. With the Under Secretary of Defense for 
Acquisition, Technology and Logistics approval, the Deputy Assistant 
Secretary of Defense for Developmental Test and Evaluation, serving 
concurrently as the Director of the Test Resource Management Center, 
will develop a plan to implement actionable recommendations from the 
results of the assessment. 

Recommendation 3: The GAO recommends that the Secretary of Defense, 
with input from the military services, develop metrics to assess 
development test capabilities, justify funding, and assist in making 
decisions on where to cut personnel, how best to allocate funding, or 
make future decisions on whether to mothball, shut down, or 
consolidate test facilities. These efforts should be coordinated with 
the Test Resource Management Center. 

DoD Response: Partially Concur. As written, the scope of the 
recommendation is too broad. This recommendation states, "GAO 
recommends that the Secretary of Defense, with input from the military 
Services, develop metrics to assess development test capabilities, 
justify funding ..." DoD recommends amending the language to "... 
develop metrics to assess the Major Range and Test Facility Base 
(MRTFB) test capabilities (expanding to DoD non-MRTFB, and non-
DoD government test facilities once an approved set of metrics are in 
place supporting the MRTFB), justify funding ..." 

Rationale: The term "development test capabilities" encompasses a wide 
range of DoD MRTFB, DoD non-MRTFB, and non-DoD government test 
facilities. The primary focus of the "Impacts of Proposed Budget Cuts 
on Ranges" paragraphs on pages 18-22 of the draft GAO report is 
clearly the MRTFB. 

Also as written, the focus is about reducing instead of "right-sizing" 
test capabilities. Recommend change from "... assist in making 
decisions on where to cut personnel..." to "assist in making decisions 
on the "right-sizing" of personnel." 

The Deputy Assistant Secretary of Defense for Developmental Test and 
Evaluation, serving concurrently as the Director of the Test Resource 
Management Center, will establish a working group, with participation 
from the military services, to develop metrics to assess MRTFB test 
capabilities, justify funding, and assist in making decisions on human 
capital and test facilities management. 

Recommendation 4: The GAO recommends that the Secretary of Defense, 
with input from the military services, report the impact budget cuts 
reflected in the fiscal year 2012 budget, as well as the insourcing 
policy clarification, will have on their (1) total workforce 
(civilians, military, and contractors) that support both of these 
activities and (2) ability to meet program office systems engineering 
and developmental test and evaluation needs. 

DoD RESPONSE: Concur. Each year the Deputy Assistant Secretary of 
Defense for Developmental Test & Evaluation (DT&E) and the Deputy 
Assistant Secretary of Defense for Systems Engineering (SE), with 
inputs from the military services, submit an assessment of the 
organization and capabilities of the Department of Defense for systems 
engineering, development planning, and developmental test and 
evaluation, to include the workforce, in the DT&E and SE Joint Annual 
Report to Congress, as directed by the Weapon Systems Acquisition
Reform Act of 2009. Any impacts to the state of the workforce due to 
funding modifications or Department workforce policy updates will be 
reflected in this report. 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Michael J. Sullivan, (202) 512-4841 or sullivanm@gao.gov: 

Staff Acknowledgments: 

In addition to the contact name above, Bruce Thomas, Assistant 
Director; Cheryl Andrew; Rae Ann Sapp; Keith Hudson; Laura Greifner; 
and Marie Ahearn made key contributions to this report. 

[End of section] 

Footnotes: 

[1] Pub. L. No. 111-23. 

[2] GAO, Defense Acquisitions: Assessments of Selected Weapons 
Programs, [hyperlink, http://www.gao.gov/products/GAO-11-233SP] 
(Washington, D.C.: Mar. 29, 2011). 

[3] Research and development costs grew over the two year period by an 
estimated $15 billion, while procurement grew by an estimated $121 
billion. Of the $135 billion total acquisition cost increase, $70 
billion cannot be attributed to quantity changes. 

[4] GAO, Defense Acquisitions: DOD Needs to Develop Performance 
Criteria to Gauge Impact of Reform Act Changes and Address Workforce 
Issues, [hyperlink, http://www.gao.gov/products/GAO-10-774] 
(Washington, D.C.: July 29, 2010). 

[5] DOD defines the core acquisition workforce as all personnel whose 
responsibilities are predominantly acquisition and assigned to 
positions designated as acquisition. 

[6] Total estimated costs include research, development, test and 
evaluation, procurement, military construction, and acquisition 
operations and maintenance funding. 

[7] Report of the Defense Science Board Task Force on Developmental 
Test and Evaluation (Washington, D.C.: May 2008). 

[8] National Academy of Sciences, Pre-Milestone A and Early-Phase 
Systems Engineering: A Retrospective Review and Benefits for Air Force 
Future Systems Acquisition (Washington, D.C.: 2008). 

[9] Pub. L. No. 110-181 § 852. 

[10] DOD officials said the decision to increase the acquisition 
workforce was in response to Pub. L. No. 110-181 § 852 and 
Presidential Memorandum, "Government Contracting," 74 Fed. Reg. 9,755 
(Mar. 6, 2009). 

[11] The Major Range and Test Facility Base is a set of test 
installations, facilities, and ranges which are regarded as "national 
assets." These assets are sized, operated, and maintained primarily 
for DOD test and evaluation missions. However, the facilities and 
ranges are also available to commercial and other users on a 
reimbursable basis. Oversight of the facilities and ranges is 
performed by the Test Resource Management Center. 

[12] DOD, Developmental Test and Evaluation and Systems Engineering FY 
2010 Annual Report (Washington, D.C.: Mar. 23, 2011). 

[13] The pilot test programs are the Integrated Air and Missile 
Defense, MQ-4C Unmanned Aircraft System Broad Area Maritime 
Surveillance, RQ-4A/B Unmanned Aircraft System Global Hawk, Joint Land 
Attack Cruise Missile Defense Elevated Netted Sensor System, Patriot 
Advanced Capability-3, and the MQ-9 Unmanned Aircraft System Reaper. 

[14] The Bob Stump National Defense Authorization Act for Fiscal Year 
2003 provided for the establishment of the Test Resource Management 
Center as a field activity that would report directly to the Under 
Secretary of Defense for Acquisition, Technology and Logistics. The 
Reform Act established the Director of Developmental Test and 
Evaluation within the Office of the Secretary of Defense, but did not 
dictate how that official's office should be aligned. The Under 
Secretary of Defense for Acquisition, Technology and Logistics decided 
to place the office under the Director of Defense, Research and 
Engineering, which has since been renamed the Assistant Secretary of 
Defense for Research and Engineering. 

[15] DOD, Developmental Test and Evaluation and Systems Engineering FY 
2009 Annual Report (Washington, D.C.: Mar. 2010). 

[16] The Reform Act requires the joint annual report to Congress to 
include (1) a discussion of the extent to which the major defense 
acquisition programs are fulfilling the objectives of their systems 
engineering master plans and developmental test and evaluation plans; 
(2) a discussion of the waivers of and deviations from requirements in 
test and evaluation master plans, systems engineering plans and other 
testing requirements, any concerns raised by such waivers or 
deviations, and the actions taken to address such concerns; and (3) an 
assessment of the organization and capabilities of DOD for systems 
engineering, development planning, and developmental test and 
evaluation with respect to such programs. 

[17] Depending on the acquisition program, the integrated product team 
leader could be the Deputy Assistant Secretary of Defense for 
Strategic and Tactical Systems, the Deputy Assistant Secretary of 
Defense for Space and Intelligence, the Deputy Assistant Secretary of 
Defense for C3, Space and Spectrum, or the Overarching Integrated 
Product Team Leader for Defense Business Systems (Office of the Deputy 
Chief Management Officer). 

[18] The portfolio includes a combination of Acquisition Category 1D, 
Acquisition Category 1C, Major Automated Information Systems, and 
other programs that are specified as special interest by the Under 
Secretary of Defense for Acquisition, Technology and Logistics or on 
the Deputy Assistant Secretaries' oversight list. Acquisition Category 
I programs are major defense acquisition programs estimated by the 
Under Secretary of Defense for Acquisition, Technology and Logistics 
to require eventual expenditure for research, development, test and 
evaluation of more than $365 million in fiscal year 2000 constant 
dollars or, for procurement, of more than $2.19 billion in fiscal year 
2000 constant dollars, or those designated by the Milestone Decision 
Authority to be special interest programs. For Acquisition Category ID 
programs, the milestone decision authority is the Under Secretary for 
Acquisition, Technology and Logistics. The "D" refers to the Defense 
Acquisition Board, which advises the Under Secretary at major decision 
points. For Acquisition Category 1C programs, the milestone decision 
authority is the DOD component head or, if delegated, the DOD 
component acquisition executive. The "C" refers to component. 

[19] The office of the Under Secretary of Defense for Acquisition is 
now the office of the Under Secretary of Defense for Acquisition, 
Technology and Logistics. 

[20] The Air Force could not provide a breakout of the number of 
positions it plans to insource between fiscal years 2010 and 2015. The 
Air Force only maintains information on its broad plans to insource, 
as they allow their major commands to determine the specific positions 
that are insourced. 

[21] Under the Budget Control Act of 2011, budgetary resources may be 
further reduced, but details about the effect, if any, on workforce 
growth plans are unknown at this time. Pub. L. No. 112-25, § 101. 

[22] According to DOD, each acquisition, technology, and logistics 
position, meaning positions designated to be acquisition positions 
based on the Defense Acquisition Workforce Improvement Act, has 
certification-level requirements. When an individual is placed in one 
of these positions, the DOD component is required to either document 
the determination that the individual has satisfied appropriate 
certification and assignment-specific training requirements, or 
establish a plan for the individual to meet the requirements within 24 
months of placement or other established period. If an individual does 
not meet position requirements within established time frames, a 
waiver must be obtained according to applicable procedures for the 
individual to remain in the position. Department of Defense 
Instruction 5000.66, Operation of the Defense Acquisition, Technology, 
and Logistics Workforce Education, Training, and Career Development 
Program at E2.4.1.2. (Dec. 21, 2005). 

[23] Under the Budget Control Act of 2011, budgetary resources may be 
further reduced, but details about the effect, if any, on workforce 
growth plans are unknown at this time. Pub. L. No. 112-25, § 101. 

[24] Bob Stump National Defense Authorization Act for Fiscal Year 
2003, Pub. L. No.107-314, § 232 (2002). 

[25] The Director of the Test Resource Management Center did not 
certify the Army's budget twice during this period. As a result, the 
Army added additional funding to its major test and evaluation 
investment account to address concerns. The Director of the Test 
Resource Management Center has not yet certified any of the services' 
fiscal year 2012 proposed test and evaluation budgets. 

[26] Charging customers for overhead costs does not reduce DOD's 
budget needs for overhead activities. Rather, it shifts the burden of 
paying for these costs from ranges to customers. 

[27] GAO, Managing for Results: Enhancing Agency Use of Performance 
Information for Management Decision Making, [hyperlink, 
http://www.gao.gov/products/GAO-05-927] (Washington, D.C.: Sept. 9, 
2005). 

[28] Pub. L. No. 111-23. 

[29] The Future Years Defense Program, a DOD centralized report 
consisting of thousands of program elements, provides information on 
DOD's current and planned outyear budget requests and is one of the 
principal tools available to help inform DOD and the Congress about 
resource data relating to these challenging trade-offs. The reporting 
requirement for it is codified at 10 U.S.C. § 221. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: