This is the accessible text file for GAO report number GAO-10-594R 
entitled 'Preliminary Observations on the Department of Defense's 
Counternarcotics Performance Measurement System' which was released on 
April 30, 2010. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

GAO-10-594R: 

United States Government Accountability Office: 
Washington, DC 20548: 

April 30, 2010: 

Congressional Committees: 

Subject: Preliminary Observations on the Department of Defense's 
Counternarcotics Performance Measurement System: 

The Department of Defense (DOD) leads detection and monitoring of 
aerial and maritime transit of illegal drugs into the United States in 
support of law enforcement agencies. Additionally, DOD's 
counternarcotics activities include sharing information with U.S. and 
foreign agencies, as well as helping foreign countries build their 
counternarcotics capacity. In support of these activities, Congress 
provided just over $6.1 billion to DOD's Counternarcotics Central 
Transfer Account from fiscal year 2005 through 2010. 

In November 2005, GAO recommended that DOD, in conjunction with other 
agencies performing counternarcotics activities, develop and 
coordinate counternarcotics performance measures. GAO has found that 
measuring performance allows organizations to track the progress they 
are making toward their goals and provides managers a basis for making 
key decisions to improve programs and results. In May 2007, the Office 
of National Drug Control Policy (ONDCP) released a circular that 
required agencies, including DOD, to submit a report on their 
performance-related information for counternarcotics activities, 
including how the agency's performance measures are used in the 
management of the program. 

The National Defense Authorization Act for Fiscal Year 2010 (Pub. L. 
No. 111-84, § 1016) mandated that GAO report on the performance 
evaluation system used by DOD to assess its counternarcotics 
activities. To satisfy this mandate, in April 2010, we briefed 
congressional staff from the defense committees. This letter 
summarizes the preliminary observations conveyed during those 
briefings. The briefing slides used to provide the briefing are 
attached as enclosure I. 

We plan to issue the final results of our evaluation of DOD's 
counternarcotics performance evaluation system in a report to be 
released in the summer of 2010 that will focus on the extent to which: 
(1) DOD's counternarcotics performance measures enable DOD to track 
progress toward achieving its goals and (2) DOD uses performance 
information to support its counternarcotics mission. 

In summary, the briefing provided to congressional staff noted that: 

* DOD has developed performance measures for its counternarcotics 
activities. DOD has also developed a database to collect performance 
information, including measures, targets, and results. However, these 
performance measures lack a number of the attributes, such as being 
clearly stated and having measurable targets, which GAO considers key 
to successful performance measures. 

* DOD is currently reassessing its performance measures for its 
counternarcotics activities. DOD hired a contractor to assist the 
department in revising its guidance for the development of 
counternarcotics performance measures. DOD officials, however, noted 
that the department still faces challenges measuring the performance 
of its counternarcotics activities. These challenges include creating 
performance measures that assess program outcomes; ensuring adequate 
resources, such as expertise in performance management, are available 
to develop performance measures; and ensuring that the data used to 
measure DOD performance is reliable. 

* It is unclear the extent to which DOD uses the performance 
information it collects through its database to manage its 
counternarcotics activities. Previous work by GAO has found that the 
full benefit of collecting performance information is realized only 
when managers use this information to inform key decisions to improve 
programs and results, such as identifying corrective actions, 
allocating resources, and sharing best practices for program 
implementation. However, officials at U.S. Central Command, U.S. 
Southern Command, and the Joint Interagency Task Force-South told us 
they rarely use the database to identify performance problems or 
manage counternarcotics activities. Rather, they tend to manage 
programs using measures and data not submitted to the database. We 
plan to examine this issue in more detail in our report to be released 
in the summer of 2010. 

We conducted this performance audit from December 2009 to April 2010, 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

We provided drafts of this report to DOD and ONDCP for comment. DOD 
provided no comments. ONDCP provided technical comments, which we have 
incorporated where appropriate. 

We are sending copies of this report to interested congressional 
committees, the Secretary of Defense, and the Director of the Office 
of National Drug Control Policy. The report also is available at no 
charge on GAO's Web site at [hyperlink, http://www.gao.gov]. 

If you or your staff members have any questions about this 
information, please contact me at (202) 512-4268 or fordj@gao.gov. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. GAO staff who 
made major contributions to this report are listed in enclosure II. 

Signed by: 

Jess T. Ford: 
Director, International Affairs and Trade: 

Enclosures: 

List of Congressional Committees: 

The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate: 

The Honorable Daniel K. Inouye:
Chairman:
The Honorable Thad Cochran:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
United States Senate: 

The Honorable Ike Skelton:
Chairman:
The Honorable Howard P. McKeon:
Ranking Member:
Committee on Armed Services:
House of Representatives: 

The Honorable Norman D. Dicks:
Chairman:
The Honorable:
Ranking Member C.W. Bill Young:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives: 

[End of section] 

Enclosure I: Briefing for Congressional Committees: 

Preliminary Observations on Department of Defense's Counternarcotics
Performance Measurement System: 

A Briefing for Congressional Committees: 

April 2010: 

Table of Contents: 
* Objectives, Scope, and Methodology: 
* Background: 
* Observations on Department of Defense's (DOD) Counternarcotics 
Performance Measures: 
* Preliminary Observations on DOD's Use of Performance Information to 
Support Its Counternarcotics Mission: 
* Agency Comments and Our Evaluation: 

Objectives, Scope, and Methodology: 

The National Defense Authorization Act of 2010 (Pub. L. No. 111-84, § 
1016) mandated that GAO report on the performance evaluation system 
used by the DOD to assess its counternarcotics activities. 

This briefing contains information on DOD's counternarcotics 
performance evaluation system, including the following: 

* the extent to which DOD's counternarcotics performance measures 
enable DOD to track progress toward achieving its goals, and; 

* preliminary observations on DOD's use of performance
information to support its counternarcotics mission. 

To address these objectives we: 

* Interviewed officials from the Office of National Drug Control 
Policy (ONDCP) and DOD components including the following: 

- Office of the Deputy Assistant Secretary of Defense for 
Counternarcotics and Global Threats (DASD-CN), 

- U.S. Central Command (CENTCOM), 

- U.S. Southern Command (SOUTHCOM), 

- Joint Interagency Task Force-South (JIATF-S), and, 

- DOD Inspector General (DOD-IG). 

To address these objectives we: 

* reviewed and analyzed DOD strategy and budget documents, as well as 
DOD guidance on performance measures, and, 

* evaluated a generalizable random sample of DOD's unclassified fiscal 
year 2009 counternarcotics performance measures (71 of 171 measures) 
to assess the extent to which these measures adhered to GAO criteria 
on the key attributes of performance measures (see table 1).
[Footnote 1] 

We conducted this performance audit from December 2009 to April 2010, 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

After conducting additional audit work, we plan to issue the final 
results of our evaluation of DOD's counternarcotics performance 
measurement system in a report to be released in the summer of 2010. 

[End of section] 

Background: 

The DOD leads detection and monitoring of aerial and maritime transit 
of illegal drugs into the United States in support of law enforcement 
agencies. 

To accomplish this mission, DOD has identified three goals: 

* detecting and monitoring drug trafficking, 

* sharing information with U.S. and foreign agencies, and, 

* helping foreign countries build their counternarcotics capacity. 

In support of these activities, Congress provided just over $6.1 
billion to DOD's Counternarcotics Central Transfer Account from fiscal 
year 2005 to fiscal year 2010. 

In May 2007, ONDCP issued a drug control accounting circular. Section 
7 of the circular requires that National Drug Control Program 
agencies, including DOD, submit to the Director of ONDCP a performance 
summary report including performance measures, targets, and results. 

ONDCP cited 21 U.S. Code § 1703(d)(7), § 1705(c)(4), § 1705 (a)(2)(xi) 
as the legal authorities that require ONDCP to monitor the 
implementation of the National Drug Control Strategy, develop a 
performance measurement system, and assess the effectiveness of 
achieving federal National Drug Control Strategy goals. 

DASD-CN reports that it is the focal point for DOD counternarcotics 
efforts and that it will ensure the department develops and implements 
a focused counternarcotics program with clear priorities and measured 
results. 

Programs, Resources, and Assessments, a division within DASDCN, is the 
lead office for the development of counternarcotics resources and 
plans. Among other activities, this office: 

* directs and manages the planning, programming, and budgeting system 
of the DOD counternarcotics program, and, 

* collects quarterly operational statistics to evaluate the 
effectiveness of DOD's contribution to meeting the goals of the 
National Drug Control Strategy. 

[End of section] 

DOD Has Developed Performance Measures for Its Counternarcotics 
Activities: 

In response to ONDCP guidance, DOD developed and submitted to ONDCP a 
set of performance measures for its fiscal year 2007 counternarcotics 
activities. Subsequently, DOD revised its performance measures for 
fiscal years 2008 and 2009 and submitted a report on these measures to 
ONDCP.[Footnote 2] 

DOD established a database to collect performance information from its 
components, including performance measures, targets, and results. In 
June 2008, DOD issued guidance with instructions for entering 
counternarcotics performance information into the database. 

In fiscal year 2009, DOD reported it had 285 performance measures for 
its counternarcotics activities. Of those, there were 171 unclassified 
performance measures related to DOD's mission of supporting law 
enforcement agencies. 

GAO Has Identified Nine Key Attributes of Successful Performance 
Measures: 

Table 1: GAO's Key Attributes of Successful Performance Measures: 

Attribute: Linkage; 
Definition: Measure is aligned with division and agencywide goals and 
mission and clearly communicated throughout the organization. 
Potentially adverse effects of not meeting attribute: Behaviors and 
incentives created by measures do not support achieving division or 
agencywide goals or mission. 

Attribute: Clarity; 
Definition: Measure is clearly stated, and the name and definition are 
consistent with the methodology used to calculate it. 
Potentially adverse effects of not meeting attribute: Data could be 
confusing and misleading to users. 

Attribute: Measurable target; 
Definition: Measure has a numerical goal. 
Potentially adverse effects of not meeting attribute: Cannot tell 
whether performance is meeting expectations. 

Attribute: Objectivity; 
Definition: Measure is reasonably free from significant bias or 
manipulation. 
Potentially adverse effects of not meeting attribute: Performance 
assessments may be systematically over- or understated. 

Attribute: Reliability; 
Definition: Measure produces the same result under similar conditions. 
Potentially adverse effects of not meeting attribute: Reported 
performance data is inconsistent and adds uncertainty. 

Attribute: Core program activities; 
Definition: Measures cover the activities that an entity is expected 
to perform to support the intent of the program. 
Potentially adverse effects of not meeting attribute: Not enough 
information available in core program areas to managers and 
stakeholders. 

Attribute: Limited overlap; 
Definition: Measure should provide new information beyond that 
provided by other measures. 
Potentially adverse effects of not meeting attribute: Manager may have 
to sort through redundant, costly information that does not add value.
	
Attribute: Balance; 
Definition: Balance exists when a suite of measures ensures that an 
organization's various priorities are covered. 
Potentially adverse effects of not meeting attribute: Lack of balance 
could create skewed incentives when measures overemphasize some goals. 

Attribute: Government-wide priorities; 
Definition: Each measure should cover a priority such as quality, 
timeliness, and cost of service. 
Potentially adverse effects of not meeting attribute: A program's 
overall success is at risk if all priorities are not addressed. 
		
Source: GAO-03-143. 

[End of table] 

DOD's Performance Measures Lack Key Attributes of Successful Measures: 

DOD and ONDCP officials acknowledged that there are weaknesses in the 
performance measures in DOD's counternarcotics database. 

GAO's analysis of a sample of DOD's unclassified fiscal year 2009 
performance measures shows that key attributes were not always present 
in the measures. 

* Two of the attributes, balance and limited overlap, did not appear 
to be present in the set of measures. 

* The other seven attributes of successful performance measures were 
present to varying degrees (see figure 1). 

Our preliminary observations suggest that DOD's classified fiscal year 
2009 performance measures may have similar weaknesses. 

Figure 1: DOD's Fiscal Year 2009 Unclassified Counternarcotics 
Performance Measures Relative to Seven Attributes of Successful 
Performance Measures: 

[Refer to PDF for image: horizontal bar graph] 

Key attributes of performance measures: 

Reliability: 
Percentage of DOD counternarcotics performance measures that exhibited 
key attribute: approximately 40%. 

Government-wide priorities: 
Percentage of DOD counternarcotics performance measures that exhibited 
key attribute: approximately 42%. 

Objectivity: 
Percentage of DOD counternarcotics performance measures that exhibited 
key attribute: approximately 60%. 

Clarity: 
Percentage of DOD counternarcotics performance measures that exhibited 
key attribute: approximately 63%. 

Measurable target: 
Percentage of DOD counternarcotics performance measures that exhibited 
key attribute: approximately 65%. 

Linkage: 
Percentage of DOD counternarcotics performance measures that exhibited 
key attribute: approximately 98%. 

Core program activities: 
Percentage of DOD counternarcotics performance measures that exhibited 
key attribute: approximately 100%. 

Source: GAO analysis of DOD performance measures. 

Note: GAO randomly sampled 71 of DOD's 171 counternarcotics 
performance measures for fiscal year 2009 that were not classified and 
were associated with DOD's goals of detection and monitoring, sharing 
information, and building capacity of partner nations. The resulting 
estimates, reported above, are subject to a maximum margin of error of 
+/- 10 percentage points. 

[End of figure] 

DOD Is Working To Improve Its Performance Measures, but Some 
Challenges Remain: 

DOD officials have acknowledged that the department's counternarcotics 
performance measures need to be revised. DOD hired a contractor to 
assist it in revising its guidance for the development of 
counternarcotics performance measures. Revised guidance, dated January 
25, 2010, exists but is not yet finalized. 

CENTCOM and SOUTHCOM officials told us that the revised
guidance provides better direction than previous guidance. 

We plan to comment on DOD's revised guidance in our summer 2010 
product. 

DOD officials, however, noted that the department still faces 
challenges measuring the performance of its counternarcotics 
activities. These challenges include the following: 

* creating outcome-based performance measures given DOD's supporting 
role in counternarcotics activities, 

* ensuring adequate resources, such as expertise in performance 
management, are available to develop performance measures, and, 

* ensuring data used to measure DOD performance is reliable. 

GAO Has Identified Use of Performance Information As Key To Improving 
Management: 

Figure 2: Uses of Performance Information: 

[Refer to PDF for image: illustration] 

Uses of Performance Data: 

* Identify and share effective approaches; 

* Identify problems and take corrective action; 

* Develop strategy and allocate resources; 

Lead to: 

Improved management decision making. 

Source: GAO. 

Note: GAO has also previously reported that recognizing and rewarding 
individual performance is a use of performance data. However, this use 
was not applicable to DOD's counternaroctics performance measures, as 
they relate only to program activities. 

[End of figure] 

[End of section] 

Preliminary Observations: 

DOD's Use of Performance Information to Support Its Counternarcotics 
Mission: 

Identify problems and take corrective action: CENTCOM, SOUTHCOM, and 
JIATF-S officials told us they rarely use the DASD-CN database to 
identify performance problems or manage counternarcotics activities. 
Rather, they tend to manage programs using information not submitted 
to the DASD-CN database. For example, CENTCOM officials told us 
information obtained in weekly program meetings, not data sent to the 
DASD-CN database, is most often used to help them identify problems 
and make program adjustments. Further, it is unclear the extent to 
which DASD-CN uses information in the database to oversee 
counternarcotics activities. 

Develop strategy and allocate resources: DASD-CN officials told us 
that they do not currently link performance information directly to 
budget allocation decisions. 

Identify and share effective approaches: While DOD creates reports for 
ONDCP, DASD-CN officials told us they do not produce additional 
reports for the department. 

[End of section] 

Agency Comments and Our Evaluation: 

We provided drafts of this report to DOD and ONDCP for comment. DOD 
provided no comments. ONDCP provided technical comments, which we have 
incorporated where appropriate. 

[End of Enclosure I] 

Enclosure II: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Jess Ford, Director, International Affairs and Trade, (202) 512-4268 
or fordj@gao.gov: 

Staff Acknowledgments: 

In additional to the individuals named above, Juan Gobel, Assistant 
Director; Joe Carney; Elizabeth Curda; Martin de Alteriis; Mark 
Dowling; Justin Fisher; Richard Geiger; Eileen Larence; Jeff Miller; 
Christopher Mulkins; John Pendleton; Elizabeth Repko; and Mark Speight 
made key contributions to this report. 

[End of Enclosure I] 

Footnotes: 

[1] Prior GAO work identified nine attributes of successful 
performance measures. See GAO, Tax Administration: IRS Needs to 
Further Refine Its Tax Filing Season Performance Measures, [hyperlink, 
http://www.gao.gov/products/GA0-03-143] (Washington, D.C.: Nov. 22, 
2002). 

[2] According to ONDCP, DOD's reports for fiscal years 2007, 2008, and 
2009 did not fulfill the requirements of ONDCP's drug control 
accounting circular because the reports were not authenticated by the 
DOD-IG. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: