GAO-10-835, Drug Control: DOD Needs to Improve Its Performance Measurement System to Better Manage and Oversee Its Counternarcotics Activities


This is the accessible text file for GAO report number GAO-10-835 
entitled 'Drug Control: DOD Needs to Improve Its Performance 
Measurement System to Better Manage and Oversee Its Counternarcotics 
Activities' which was released on July 21, 2010. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Committees: 

United States Government Accountability Office: 
GAO: 

July 2010: 

Drug Control: 

DOD Needs to Improve Its Performance Measurement System to Better 
Manage and Oversee Its Counternarcotics Activities: 

GAO-10-835: 

GAO Highlights: 

Highlights of GAO-10-835, a report to congressional committees. 

Why GAO Did This Study: 

The Department of Defense (DOD) leads detection and monitoring of 
aerial and maritime transit of illegal drugs into the United States in 
support of law enforcement agencies. DOD reported resources of more 
than $1.5 billion for fiscal year 2010 to support its counternarcotics 
activities. 

Congress mandated GAO report on DOD’s counternarcotics performance 
measurement system. Specifically, this report addresses the extent to 
which (1) DOD’s counternarcotics performance measurement system 
enables DOD to track progress and (2) DOD uses performance information 
from its counternarcotics performance measurement system to manage its 
activities. GAO analyzed relevant DOD performance and budget 
documents, and discussed these efforts with officials from DOD and the 
Office of National Drug Control Policy (ONDCP). 

What GAO Found: 

DOD does not have an effective performance measurement system to track 
the progress of its counternarcotics activities; however, it continues 
efforts to improve the system. GAO has previously reported that 
measuring performance provides managers a basis for making fact-based 
decisions. DOD has established performance measures for its 
counternarcotics activities and a database to collect performance 
information, including measures, targets, and results. However, these 
measures lack a number of attributes, such as being clearly stated and 
objective, which GAO considers key to successful performance measures. 
In May 2010, DOD issued new guidance for its counternarcotics 
performance measurement system. However, DOD officials noted the 
department will face challenges implementing the guidance. These 
challenges include creating performance measures that assess program 
outcomes and ensuring adequate resources, such as expertise in 
performance management, are available to develop measures. 

DOD rarely uses the information in its performance measurement system 
to manage its counternarcotics activities and has applied few 
practices to facilitate its use. GAO has found that the full benefit 
of collecting performance information is realized only when managers 
use it to inform key decisions. However, DOD officials responsible for 
counternarcotics activities throughout the department told us they 
rarely use data submitted to the system to manage activities. Rather, 
they tend to manage programs using data not submitted to the system, 
such as information obtained in weekly program meetings regarding the 
cost and timeliness of projects. Moreover, officials responsible for 
oversight of DOD’s activities stated they use the system to develop 
reports for ONDCP, but not to allocate resources. While DOD has 
applied some practices to facilitate the use of the performance 
information in its system, it does not utilize certain key practices 
identified by GAO, such as frequently and effectively communicating 
performance information. Absent an effective performance management 
system, DOD lacks critical information to use to improve the 
management and oversight of its counternarcotics activities. 

Figure: DOD’s Performance Measurement System as Compared to GAO-
Identified Steps: 

[Refer to PDF for image: illustration] 

GAO-identified steps in an effective performance measurement system: 

Measure performance: 
* establish performance measures; 
* collect data. 

Use information obtained from performance measures to: 
* inform key decisions; 
* improve programs and results. 

DOD’s counternarcotics performance measurement system: 

DOD has developed performance measures and a database to collect data: 
* measures lack key attributes of successful performance measures. 

DOD infrequently uses information from its performance measurement 
system to: 
* identify problems or manage activities; 
* develop strategy or allocate resources; 
* identify and share effective approaches. 

Source: GAO; Corel Draw (logos). 

[End of figure] 

What GAO Recommends: 

GAO recommends that the Secretary of Defense take steps to improve 
DOD’s counternarcotics performance measurement system by (1) revising 
its performance measures and (2) applying practices to better 
facilitate the use of performance data to manage its counternarcotics 
activities. DOD concurred with GAO’s recommendations. 

View [hyperlink, http://www.gao.gov/products/GAO-10-835] or key 
components. For more information, contact Jess Ford at (202) 512-4268 
or fordj@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

DOD Has Not Developed a System to Effectively Track the Progress of 
Its Counternarcotics Activities, but Continues to Work to Improve Its 
Efforts: 

DOD Rarely Uses the Performance Information Contained in Its 
Performance Measurement System to Manage Its Counternarcotics 
Activities and Has Applied Few Practices to Facilitate Its Use: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Comments from the Department of Defense: 

Appendix III: Contact and Staff Acknowledgments: 

Tables: 

Table 1: DOD Resources in Support of Its Counternarcotics Activities, 
Fiscal Years 2005-2010: 

Table 2: DOD Goals, Objectives, and Example Performance Measures 
Related to Its Counternarcotics Mission to Support U.S. Agencies and 
Foreign Partners: 

Table 3: GAO's Key Attributes of Successful Performance Measures: 

Table 4: Examples of Data Sources Other than DOD's Counternarcotics 
Performance Measurement System Used by DOD Components to Manage 
Counternarcotics Activities: 

Table 5: Status of DOD Efforts to Apply Practices to Facilitate Use of 
Performance Information in Its Counternarcotics Performance 
Measurement System and Reported Challenges, as of June 2010: 

Figure: 

Figure 1: Percentages of DOD's Fiscal Year 2009 Counternarcotics 
Performance Measures Exhibiting Six Key Attributes: 

Abbreviations: 

AFRICOM: U.S. Africa Command: 

CENTCOM: U.S. Central Command: 

DASD-CN>: Office of the Deputy Assistant Secretary of Defense- 
Counternarcotics and Global Threats: 

DOD: Department of Defense: 

DOD-IG: Department of Defense Inspector General: 

EUCOM: U.S. European Command: 

JIATF-S: Joint Interagency Task Force-South: 

JIATF-W: Joint Interagency Task Force-West: 

NORTHCOM: U.S. Northern Command: 

ONDCP: Office of National Drug Control Policy: 

SOUTHCOM: U.S. Southern Command: 

[End of section] 

United States Government Accountability Office:
Washington, DC 20548: 

July 21, 2010: 

Congressional Committees: 

The global drug trade threatens U.S. national security by weakening 
the rule of law in affected countries, financing the activities of 
global and regional terrorists, and contributing to dangers such as 
weapons trafficking. The Department of Defense (DOD) leads detection 
and monitoring of aerial and maritime transit of illegal drugs into 
the United States in support of law enforcement agencies. 
Additionally, DOD's counternarcotics activities include sharing 
information with U.S. and foreign agencies, as well as helping foreign 
countries build their counternarcotics capacity. DOD reported 
resources of more than $1.5 billion for fiscal year 2010 in support of 
these activities. 

The National Defense Authorization Act for Fiscal Year 2010 mandated 
that GAO report on the performance measurement system used by DOD to 
assess its counternarcotics activities.[Footnote 1] We have previously 
reported that performance measurement systems used by results-oriented 
agencies include steps to measure performance to gauge progress and 
use the information obtained to make key management decisions. 
[Footnote 2] In April 2010 we briefed congressional staff from the 
defense committees on our preliminary observations regarding DOD's 
counternarcotics performance measurement system.[Footnote 3] This 
report contains the final results of our evaluation. Specifically, we 
address the extent to which (1) DOD's counternarcotics performance 
measurement system enables DOD to track progress and (2) DOD uses 
performance information from its counternarcotics performance 
measurement system to manage its activities. 

To address these objectives, we analyzed DOD strategy, budget, and 
performance documents, as well as DOD and Office of National Drug 
Control Policy (ONDCP) guidance on performance measures. Further, we 
discussed DOD's counternarcotics performance measurement system and 
its use of performance information with officials from ONDCP and DOD 
components including the Office of the Deputy Assistant Secretary of 
Defense for Counternarcotics and Global Threats (DASD-CN>), U.S. 
Africa Command (AFRICOM), U.S. Central Command (CENTCOM), U.S. 
European Command (EUCOM), U.S. Northern Command (NORTHCOM), U.S. 
Southern Command (SOUTHCOM), the Joint Interagency Task Force-South 
(JIATF-S), the Joint Interagency Task Force-West (JIATF-W), and the 
DOD Inspector General (DOD-IG). We evaluated a generalizable random 
sample of DOD's fiscal year 2009 counternarcotics performance measures 
(115 of 239 measures) to assess the extent to which these measures 
adhered to GAO criteria on the key attributes of successful 
performance measures. We also analyzed the extent to which DOD applies 
key management practices identified by GAO to facilitate the use of 
performance information from its counternarcotics performance 
measurement system. Moreover, we visited CENTCOM, SOUTHCOM, and JIATF-
S to examine DOD's use of performance data to support its 
counternarcotics mission. (See appendix I for a complete description 
of our scope and methodology.) 

We conducted this performance audit from December 2009 to July 2010 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

Background: 

DOD Counternarcotics Strategy and Activities: 

According to DOD's Counternarcotics Strategy developed in fiscal year 
2009, the department seeks to disrupt the market for illegal drugs by 
helping local, state, federal, and foreign government agencies address 
the drug trade and narcotics-related terrorism.[Footnote 4] DOD 
achieves this mission through three goals--detecting and monitoring 
drug trafficking, sharing information on illegal drugs with U.S. and 
foreign government agencies, and building the counternarcotics 
capacity of U.S. and foreign partners. 

DASD-CN>, with oversight from the Under Secretary of Defense for 
Policy, exercises management and oversight of DOD's counternarcotics 
activities and performance measurement system. DASD-CN>'s 
responsibilities include ensuring DOD develops and implements a 
counternarcotics program with clear priorities and measured results. 
Programs, Resources, and Assessments, a division within DASD-CN>, is 
the lead office for the development of counternarcotics resources and 
plans. Among other activities, this office directs and manages the 
planning, programming, and budgeting system of the DOD 
counternarcotics program and is responsible for updating and 
disseminating guidance on DOD's counternarcotics performance 
measurement system. 

DOD's counternarcotics activities are implemented through DOD's 
combatant commands, military departments, and defense agencies. 
[Footnote 5] According to DOD, these organizations provide assets, 
such as aircraft and patrol ships, military personnel, and other 
assistance, to support U.S. law enforcement agencies and foreign 
security forces in countering narcotics trafficking. 

In support of DOD's counternarcotics activities, DOD reported 
resources totaling approximately $7.7 billion from fiscal year 2005 to 
fiscal year 2010, including more than $6.1 billion appropriated to its 
Counternarcotics Central Transfer Account and more than $1.5 billion 
in supplemental appropriations (see table 1). 

Table 1: DOD Resources in Support of Its Counternarcotics Activities, 
Fiscal Years 2005-2010 (Dollars in millions): 

Counternarcotics Central Transfer Account[A]: 
FY 2005: $905.8; 
FY 2006: $936.1; 
FY 2007: $1,075.2; 
FY 2008: $984.8; 
FY 2009: $1,096.7; 
FY 2010: $1,158.2; 
Total: $6,156.9. 

Supplemental appropriations[A,B]: 
FY 2005: $242.0; 
FY 2006: $86.9; 
FY 2007: $202.7; 
FY 2008: $328.0; 
FY 2009: $300.4; 
FY 2010: $369.9; 
Total: $1,529.9. 

Total: 
FY 2005: $1,147.8; 
FY 2006: $1,022.9; 
FY 2007: $1,277.8; 
FY 2008: $1,312.8; 
FY 2009: $1,397.2; 
FY 2010: $1,528.2; 
Total: $7,686.7. 

Source: GAO analysis of DOD data. 

Note: Totals may not add due to rounding. 

[A] DOD funding resources in support of its counternarcotics 
activities are annually reported as part of the National Drug Control 
Strategy Budget Summary Documents. For fiscal years 2005-2010, these 
documents list DOD resources for its Counternaroctics Central Transfer 
Account and for supplemental appropriations. 

[B] According to DOD, it rolls over unobligated supplemental funding 
into the next fiscal year; therefore, the supplemental totals listed 
here do not match the total supplemental funding appropriated for that 
year. 

[End of table] 

Of these resources, DOD estimated that approximately $4.2 billion were 
in support of its international counternarcotics activities from 
fiscal years 2005-2010. 

Previous GAO Reporting and Legislation Related to DOD's 
Counternarcotics Performance Measures: 

DOD efforts to develop performance measures for its counternarcotics 
activities are long-standing. We reported in December 1999[Footnote 6] 
that DOD had not developed a set of performance measures to assess the 
impact of its counternarcotics operations, but had undertaken initial 
steps to develop such measures. In January 2002[Footnote 7] and 
November 2005,[Footnote 8] we found that DOD was in the process of 
developing performance measures focused on its role of detecting and 
monitoring the trafficking of illegal drugs into the United States. In 
November 2005 we recommended that DOD, in conjunction with other 
agencies performing counternarcotics activities, develop and 
coordinate counternarcotics performance measures. 

In December 2006 Congress directed ONDCP--the organization that 
establishes U.S. counternarcotics goals and coordinates the federal 
budget to combat drugs--to produce an annual report describing the 
national drug control performance measurement system that identifies 
the activities of national drug control program agencies, including 
DOD. In May 2007 ONDCP issued guidance requiring DOD and other 
national drug control program agencies to annually submit to the 
Director of ONDCP a performance summary report including performance 
measures, targets, and results.[Footnote 9] In addition, ONDCP 
officials stated that they have recommended improvements to DOD's 
performance measures, both in correspondence and in meetings with DOD 
staff. 

DOD Has Not Developed a System to Effectively Track the Progress of 
Its Counternarcotics Activities, but Continues to Work to Improve Its 
Efforts: 

DOD does not have an effective system for tracking the progress of its 
counternarcotics activities; however, it continues efforts to improve 
the system. We have found that measuring performance provides managers 
a basis for making fact-based decisions. DOD has established 
performance measures for its counternarcotics activities and a 
database to collect performance information. However, these measures 
lack a number of attributes which we consider key to successful 
performance measures and, therefore, do not provide a clear indication 
of DOD's progress toward its counternarcotics goals. Recognizing the 
need to update and improve its measures, in May 2010, DOD issued new 
guidance for its counternarcotics performance measurement system. 
However, DOD officials noted the department will faces challenges 
implementing the guidance. 

DOD Has Developed Performance Measures and a Database for Its 
Counternarcotics Activities: 

We have previously reported that effective performance measurement 
systems include steps to measure performance, such as establishing 
performance measures and collecting data.[Footnote 10] In response to 
ONDCP's 2007 guidance, DOD developed performance measures for its 
fiscal year 2007 counternarcotics activities and established a 
centralized database within its performance measurement system to 
collect data on those performance measures.[Footnote 11] The 
counternarcotics performance measurement system database, maintained 
by DASD-CN>, requires DOD components to submit performance 
information at specified intervals during the fiscal year, such as 
results for performance measures, the mechanisms used to collect 
results data, and future performance targets. For fiscal year 2009, 
DOD guidance required that all projects funded by its Counternarcotics 
Central Transfer Account have a performance measure. As a result, DOD 
reported it had 285 performance measures for its fiscal year 2009 
counternarcotics activities. Of those, 239 were performance measures 
related to DOD's mission of supporting U.S. agencies and foreign 
partners in countering narcotics trafficking. (See table 2 for 
examples of DOD's counternarcotics performance measures.) 

Table 2: DOD Goals, Objectives, and Example Performance Measures 
Related to Its Counternarcotics Mission to Support U.S. Agencies and 
Foreign Partners: 

Goal: Detect and monitor illegal drug traffic; 
Related objective: 
* Detect and monitor illegal drug trafficking using DOD and contractor 
provided air and maritime assets; 
Example performance measure[A]: 
* On-station ship days (includes U.S. and allied); 

Related objective: 
* Detect and monitor illegal drug trafficking using DOD radar systems; 
Example performance measure[A]: 
* System mission capability rate (expressed as a percentage); 

Related objective: 
* Support detection and monitoring activities by providing operational 
facilities; 
Example performance measure[A]: 
* OPBAT[B] functionality (providing the system required to fuel 
resident helicopters) of fueling system capability; 

Related objective: 
* Provide integrated command and control, voice and data 
communications, and connectivity in support of interdiction operations; 
Example performance measure[A]: 
* Number of sensors integrated and providing reliable and dependable 
radar to JIATF-S and/or host nations. 

Goal: Share information on illegal drugs and technology support with 
U.S. and foreign government agencies; 
Related objective: 
* Manage intelligence collection against counternarcotics targets; 
Example performance measure[A]: 
* Percentage of tasked CN[C] missions flown; 

Related objective: 
* Provide analysis and reporting in response to strategic and tactical 
requirements; 
Example performance measure[A]: 
* Number of formal intelligence products provided to or on behalf of 
law enforcement agencies or other U.S. agencies; 

Related objective: 
* Share information with U.S. and partner nation security and law 
enforcement; 
Example performance measure[A]: 

Related objective: 
* Provide collection and analysis training to DOD, U.S., and partner 
nation law enforcement personnel; 
Example performance measure[A]: 
* Number of partner nation law enforcement agencies engaged; 

Related objective: 
* Develop and deploy technology that disrupts the flow of illegal 
drugs; 
Example performance measure[A]: 
* Number of attendees to Basic Counterdrug Intelligence Course. 

Goal: Build the counternarcotics capacity of U.S. and foreign partners; 

Related objective: 
* Provide training and support to U.S. law enforcement personnel 
conducting counternarcotics related activities; 
Example performance measure[A]: 
* Number of trained military working dog teams trained; 

Related objective: 
* Provide training and equipment to partner nation forces; 
Example performance measure[A]: 
* Percent of inland waterways controlled by Colombian Marine Corps 
forces. 

Related objective:
* Provide infrastructure projects in support of partner nation forces; 
Example performance measure[A]: 
* Number of infrastructure projects in support of training 
requirements. 

Related objective: 
* Provide support to partner nation forces; 
Example performance measure[A]: 
* Percentage of positive to negative media references from non-U.S. 
media sources for a calendar year. 

Source: DOD's counternarcotics performance measurement system's 
database. 

[A] The example performance measures are reprinted as they appear in 
DOD's counternarcotics performance measurement system's database. 

[B] OPBAT is an acronym used by DOD meaning "Operation Bahamas Turks 
and Caicos." 

[C] CN is an acronym used by DOD meaning "counternarcotics." 

[End of table] 

DOD's Fiscal Year 2009 Counternarcotics Performance Measures Exhibit 
Some, but Not All, Key Attributes of Successful Performance Measures: 

DOD's current set of counternarcotics performance measures varies in 
the degree to which it exhibits key attributes of successful 
performance measures. Prior GAO work has identified nine attributes of 
successful performance measures.[Footnote 12] Table 3 shows the nine 
attributes, their definitions, and the potentially adverse 
consequences of not having the attributes. 

Table 3: GAO's Key Attributes of Successful Performance Measures: 

Key attributes evaluated by reviewing performance measures as a set: 

Attribute: Core program activities; 
Definition: Key attributes evaluated by reviewing performance measures 
as a set: Measures cover the activities that an entity is expected to 
perform to support the intent of the program; 
Potentially adverse consequences of not meeting attribute: Key 
attributes evaluated by reviewing performance measures as a set: Not 
enough information available in core program areas to managers and 
stakeholders. 

Attribute: Balance; 
Definition: Key attributes evaluated by reviewing performance measures 
as a set: Balance exists when a suite of measures ensures that an 
organization's various priorities are covered; 
Potentially adverse consequences of not meeting attribute: Key 
attributes evaluated by reviewing performance measures as a set: Lack 
of balance could create skewed incentives when measures overemphasize 
some goals. 

Attribute: Limited overlap; 
Definition: Key attributes evaluated by reviewing performance measures 
as a set: Measure should provide new information beyond that provided 
by other measures; 
Potentially adverse consequences of not meeting attribute: Key 
attributes evaluated by reviewing performance measures as a set: 
Managers may have to sort through redundant, costly information that 
does not add value. 

Key attributes evaluated by reviewing performance measures 
individually: 

Attribute: Linkage; 
Definition: Key attributes evaluated by reviewing performance measures 
as a set: Measure is aligned with division and agencywide goals and 
mission and clearly communicated throughout the organization; 
Potentially adverse consequences of not meeting attribute: Key 
attributes evaluated by reviewing performance measures as a set: 
Behaviors and incentives created by measures do not support achieving 
division or agencywide goals or mission. 

Attribute: Governmentwide priorities; 
Definition: Key attributes evaluated by reviewing performance measures 
as a set: Each measure should cover a priority, such as quality, 
timeliness, and cost of service; 
Potentially adverse consequences of not meeting attribute: Key 
attributes evaluated by reviewing performance measures as a set: A 
program's overall success is at risk if all priorities are not 
addressed. 

Attribute: Reliability; 
Definition: Key attributes evaluated by reviewing performance measures 
as a set: Measure produces the same result under similar conditions; 
Potentially adverse consequences of not meeting attribute: Key 
attributes evaluated by reviewing performance measures as a set: 
Reported performance data is inconsistent and adds uncertainty. 

Attribute: Objectivity; 
Definition: Key attributes evaluated by reviewing performance measures 
as a set: Measure is reasonably free from significant bias or 
manipulation; 
Potentially adverse consequences of not meeting attribute: Key 
attributes evaluated by reviewing performance measures as a set: 
Performance assessments may be systematically over-or understated. 

Attribute: Clarity; 
Definition: Key attributes evaluated by reviewing performance measures 
as a set: Measure is clearly stated, and the name and definition are 
consistent with the methodology used to calculate it; 
Potentially adverse consequences of not meeting attribute: Key 
attributes evaluated by reviewing performance measures as a set: Data 
could be confusing and misleading to users. 

Attribute: Measurable target; 
Definition: Key attributes evaluated by reviewing performance measures 
as a set: Measure has a numerical goal; 
Potentially adverse consequences of not meeting attribute: Key 
attributes evaluated by reviewing performance measures as a set: 
Cannot tell whether performance is meeting expectations. 

Source: GAO. 

[End of table] 

Our analysis found that DOD's counternarcotics performance measures 
lack several of the key attributes of successful performance measures. 
Based on our analysis of a generalizable sample of DOD's fiscal year 
2009 performance measures,[Footnote 13] we found the attributes of 
core program activities and linkage were generally present, but other 
attributes such as balance and limited overlap were missing, and 
attributes including governmentwide priorities, reliability, 
objectivity, clarity, and measurable targets were present in varying 
degrees. 

We found that the attribute of core program activities was identified 
in the set of measures, while balance and limited overlap did not 
appear to be present. 

* Core program activities. We estimate that all of DOD's 
counternarcotics performance measures cover the department's core 
program activities. We have previously reported that core program 
activities are the activities that an entity is expected to perform to 
support the intent of the program, and that performance measures 
should be scoped to evaluate those activities. For the measures we 
reviewed, DOD divides its core counternarcotics activities across its 
3 goals and 13 objectives (see table 2). In our analysis, we found at 
least one performance measure covering each of DOD's counternarcotics 
objectives. Therefore, we determined that DOD's core program 
activities were covered. 

* Balance. DOD's set of performance measures lack balance. We have 
previously reported that balance exists when a set of measures ensures 
that an organization's various priorities are covered. According to 
DOD, performance measures best cover its priorities when five 
measurable aspects of performance, as defined by DOD--input, process, 
output, outcome, and impact--are present in its performance measures. 
As an example, "number of attendees to basic counterdrug intelligence 
course" is, in our determination, a measure of output, as it measures 
the services provided by DOD. We estimate 93 percent of DOD's fiscal 
year 2009 performance measures are input, process, or output measures, 
while 6 percent are outcome measures and 0 percent are impact 
measures.[Footnote 14] Therefore, given that DOD's set of measures is 
highly skewed towards input, process, and output measures and contains 
no impact measures, we determined that the set is not balanced by 
DOD's criteria. Performance measurement efforts that lack balance 
overemphasize certain aspects of performance at the expense of others, 
and may keep DOD from understanding the effectiveness of its overall 
mission and goals. 

* Limited overlap. We determined there to be overlap among DOD's 
performance measures. We found instances where the measures and their 
results appeared to overlap with other measures and results. When we 
spoke with DASD-CN> officials concerning this, they stated that the 
set of measures could be conveyed using fewer, more accurate measures. 
We have reported that each performance measure in a set should provide 
additional information beyond that provided by other measures. When an 
agency has overlapping measures, it can create unnecessary or 
duplicate information, which does not benefit program management. 

Of the remaining six attributes of successful performance measures, 
only one attribute--linkage--was present in almost all of the 
measures, while the other five attributes--governmentwide priorities, 
reliability, objectivity, clarity, and measurable targets--appeared in 
varying degrees (see figure 1). 

Figure 1: Percentages of DOD's Fiscal Year 2009 Counternarcotics 
Performance Measures Exhibiting Six Key Attributes: 

[Refer to PDF for image: horizontal bar graph] 

Key attributes of performance measure: Governmentwide priorities; 
Estimated percentage of DOD performance measures exhibiting key 
attribute: 41%. 

Key attributes of performance measure: Reliability; 
Estimated percentage of DOD performance measures exhibiting key 
attribute: 46%. 

Key attributes of performance measure: Objectivity; 
Estimated percentage of DOD performance measures exhibiting key 
attribute: 59%. 

Key attributes of performance measure: Clarity; 
Estimated percentage of DOD performance measures exhibiting key 
attribute: 65%. 

Key attributes of performance measure: Measureable target; 
Estimated percentage of DOD performance measures exhibiting key 
attribute: 66%. 

Key attributes of performance measure: Linkage; 
Estimated percentage of DOD performance measures exhibiting key 
attribute: 99%. 

Source: GAO analysis of DOD performance measures. 

[End of figure] 

DOD's counternarcotics performance measures demonstrate linkage. We 
estimate that 99 percent of DOD's measures are linked to agencywide 
goals and mission. DOD's counternarcotics performance measurement 
system database requires that for each performance measure entered 
into the database, a goal and related objective of DOD's 
counternarcotics mission be identified. Our analysis found that in all 
but one instance, linkage between DOD's goals and performance measures 
is easily identified. 

However, DOD's counternarcotics performance measures did not fully 
satisfy five attributes. 

* Governmentwide priorities. We estimate that 41 percent of the 
measures we analyzed cover a broader governmentwide priority, such as 
quality, timeliness, efficiency, cost of service, or outcome. We 
determined, for example, that the governmentwide priority of "quality" 
was reflected in the measure "number of sensors integrated and 
providing reliable and dependable radar data to JIATF-S and/or host 
nations," because it measures the reliability and dependability of 
detection services. In the majority of the instances, however, 
measures did not address a governmentwide priority. For example, the 
measure "number of trained military working dog teams trained" was 
determined not to cover a governmentwide priority because it does not 
measure the quality or efficiency of training provided. When measures 
fail to cover governmentwide priorities managers may not be able to 
balance priorities to ensure the overall success of the program. 

* Reliability. We estimate that 46 percent of DOD's performance 
measures have data collection methods indicated in the database that 
generally appear reliable. Reliability refers to whether a measure is 
designed to collect data or calculate results such that the measure 
would be likely to produce the same results if applied repeatedly to 
the same situation. For each entry in the database, users are directed 
to enter, among other information, one performance measure and its 
associated methodology, target, and result. However, in numerous 
instances the system contained multiple performance measures entered 
into fields that should contain only one measure. Such entries could 
result in errors of collecting, maintaining, processing, or reporting 
the data. Additionally, some measures did not provide enough 
information on data collection methods or performance targets to 
assure reliability. For example, a measure in the database states 
"continuous U.S. Navy ship presence in the SOUTHCOM area of 
responsibility." The performance target listed for this measure is 
"3.5," but to what 3.5 refers--such as days, number of ships, or 
percentage points--is not explained. Moreover, the methodology in the 
database for this measure is entered as "not applicable." Therefore, 
the measure's methodology does not provide insight into how DOD could 
measure whether or not it reached its target of 3.5. As a result, we 
determined that this measure did not have data collection methods to 
gather reliable results. We have previously reported that if errors 
occur in the collection of data or the calculation of their results, 
it may affect conclusions about the extent to which performance goals 
have been achieved. 

* Objectivity. We estimate that 59 percent of DOD's performance 
measures for its counternarcotics activities are objective. We have 
previously reported that to be objective, measures should indicate 
specifically what is to be observed, in which population or 
conditions, and in what time frame, and be free of opinion and 
judgment. We estimate that 41 percent of DOD's measures are not 
objective and could therefore face issues of bias or manipulation. For 
example, a measure in the database is, "percent of inland waterways 
controlled by Colombian Marine Corps forces." For this measure, no 
criteria for "controlled" is provided and it is not clear how the 
Colombian government reports the percentage of waterways under its 
control and over what time frame this control will occur. 

* Clarity. We estimate that 65 percent of DOD's performance measures 
exhibit the attribute of clarity. A measure achieves clarity when it 
is clearly stated and the name and definition are consistent with the 
methodology used for calculating the measure. However, we estimate 
that 35 percent of DOD's measures are not clearly stated. For example, 
one of DOD's measures linked to the objective of sharing information 
with U.S. and partner nations is "identify and establish methodology 
for implementation." For this measure, no associated methodology is 
identified, and it is unclear what is being implemented. We have 
previously reported that a measure that is not clearly stated can 
confuse users and cause managers or other stakeholders to think that 
performance was better or worse than it actually was. 

* Measurable target. We estimate that 66 percent of DOD's measures 
have measurable targets. Where appropriate, performance goals and 
measures should have quantifiable, numerical targets or other 
measurable values. Some of DOD's measures, however, lacked such 
targets. For example, one performance measure identified its target as 
"targets developed by the local commander." As it is not quantifiable, 
this target does not allow officials to easily assess whether goals 
were achieved because comparisons cannot be made between projected 
performance and actual results. 

DOD Is Working To Improve Its Counternarcotics Performance Measures, 
but Implementation Challenges Exist: 

DOD officials have acknowledged that weaknesses exist in the 
department's current set of counternarcotics performance measures. In 
May 2010 DOD issued revised guidance for its counternarcotics 
performance measurement system to guide users in establishing 
performance measures that more accurately capture the quantitative and 
qualitative achievements of DOD's activities. To do this, the guidance 
states that performance measures should be, among other attributes, 
useful for management and clearly stated. The guidance describes 
different types of performance measures that can be used to monitor 
DOD's contribution to its strategic counternarcotics goals, such as 
those that measure DOD's efficiency, capability, and effectiveness at 
performing its activities. Additionally, according to the guidance, 
DOD components should provide evidence of the quality and reliability 
of the data used to measure performance. 

However, DOD officials noted four specific challenges that the 
department faces in developing performance measures consistent with 
its revised guidance. 

* Creating performance measures that assess program outcomes. Some DOD 
officials noted that, because DOD acts as a support agency to partner 
nations and other law enforcement entities--and the actual 
interdiction of drugs is conducted by other entities--measuring the 
outcome of DOD's performance is difficult. While developing outcome 
measures can be challenging, we have found that an agency's 
performance measures should reflect a range of priorities, including 
outcomes. Moreover, we have found that methods to measure program 
outcomes do exist. For example, agencies have applied a range of 
strategies to develop outcome measures for their program, such as 
developing measures of satisfaction based upon surveys of 
customers.[Footnote 15] In addition, officials from EUCOM, AFRICOM, 
and JIATF-S stated that while developing outcome performance measures 
can be difficult, developing such measures for support activities is 
possible and is done at other federal agencies. For example, EUCOM 
indicated it could track the outcome of the support it provides to 
partner nations by tracking the annual percentage increase in 
interdictions and arrests related to illicit trafficking. 
Additionally, JIATF-W[Footnote 16] indicated that it conducts 
quarterly command assessments of current programs, which focus on 
aligning resources provided by JIATF-W to the outcomes of its law 
enforcement partners. 

* Implementing revisions in a timely manner. DOD officials noted that 
implementing revisions to the department's performance measures in a 
timely fashion will be difficult given that such revisions are 
resource and time intensive. Further, while including dates for 
submission, DOD's revised guidance does not clearly specify a time 
frame by which DOD components should revise the counternarcotics 
performance measures that are to be submitted to the database. We have 
previously reported that establishing timetables for the development 
of performance measures can create a sense of urgency that assists in 
the effort being taken more seriously. DASD-CN> officials noted that 
time frames by which DOD's measures would be revised are being 
discussed. However, these officials do not expect new performance 
measures to be established in fiscal year 2010, and said that fiscal 
year 2011 would be the earliest year of full implementation of the 
guidance. 

* Ensuring adequate resources are available. DOD officials noted that 
ensuring adequate resources--such as expertise and training in 
performance management--are available to develop performance measures 
at both DASD-CN> and the combatant commands will be a challenge. 
These officials noted that DOD employees tasked with developing 
performance measures and tracking the progress towards achieving goals 
are not sufficiently trained to design and monitor outcome performance 
measures. We have previously reported that access to trained staff 
assists agencies in their development of performance measures. 
[Footnote 17] 

* Ensuring reliable data. DOD officials noted that ensuring data used 
to measure DOD performance are reliable is challenging. To measure the 
performance of its counternarcotics activities DOD officials told us 
they rely heavily on external sources of data, such as U.S. law 
enforcement agencies and foreign government officials. This challenge 
can pose issues for DOD regarding data verification and ensuring 
proper information is recorded for performance measures. 

DOD Rarely Uses the Performance Information Contained in Its 
Performance Measurement System to Manage Its Counternarcotics 
Activities and Has Applied Few Practices to Facilitate Its Use: 

DOD makes limited use of its performance measurement system to manage 
its counternarcotics activities and has applied few practices to 
facilitate its use. We have found that the full benefit of collecting 
performance information is realized only when managers use the 
information to inform key decisions.[Footnote 18] While DOD has 
applied some practices to facilitate the use of the performance 
information in its system, it does not utilize certain key practices, 
such as frequently and effectively communicating performance 
information. Absent an effective performance management system, DOD 
lacks critical information to use to improve the management and 
oversight of its counternarcotics activities. 

Agencies Can Use Performance Information to Manage for Results: 

We have previously reported that, in addition to measuring 
performance, effective performance measurement systems include steps 
to use information obtained from performance measures to make 
decisions that improve programs and results.[Footnote 19] We 
identified several ways in which agencies can use performance 
information to manage for results, including using data to (1) 
identify problems and take corrective actions, (2) develop strategy 
and allocate resources, and (3) identify and share effective 
approaches. 

DOD Submits Performance Reports to ONDCP, But Makes Limited Use of the 
Information in Its Performance Measurement System to Manage and 
Oversee Its Counternarcotics Activities: 

DOD officials representing DASD-CN>, AFRICOM, CENTCOM, EUCOM, 
NORTHCOM, SOUTCOM, JIATF-S, and JIATF-W told us they rarely use 
information from DOD's counternarcotics performance measurement system 
to manage counternarcotics activities. Specifically, they rarely use 
the system to: 

* Identify problems and take corrective actions. Agencies can use 
performance information to identify problems or weaknesses in 
programs, to try to identify factors causing the problems, and to 
modify a service or process to try to address problems. DOD officials 
representing DASD-CN> and SOUTHCOM told us that they currently make 
limited use of the performance information in DOD's performance 
measurement system to manage counternarcotics activities. Officials 
from DASD-CN> stated that they use data from the performance 
measurement system to produce reports for ONDCP, which may include 
information identifying problems in the implementation of DOD's 
counternarcotics activities. However, in reviewing these documents, we 
found that the reports did not include a clear assessment of DOD's 
overall progress toward its counternarcotics goals. For instance, the 
report submitted to ONDCP for fiscal year 2009 contained detailed 
information on 6 of DOD's 285 counternarcotics performance measures, 
but did not clearly explain why the results of these 6 measures would 
be critical to the success of DOD's counternarcotics program.[Footnote 
20] Moreover, according to ONDCP, DOD's reports for fiscal years 2007, 
2008, and 2009 did not fulfill the requirements of ONDCP's guidance 
because the reports were not authenticated by the DOD-IG.[Footnote 21] 

Further, officials from AFRICOM, CENTCOM, EUCOM, NORTHCOM, JIATF-S, 
and JIATF-W told us they do not use the DOD's performance measurement 
system to manage counternarcotics activities. While these officials 
indicated that they submitted performance information to the system's 
database as required by DOD guidance, they stated they tend to manage 
programs using information not submitted to the system (see table 4). 
For example, CENTCOM officials told us information obtained in weekly 
program meetings regarding the timeliness and cost of counternarcotics 
projects, not data sent to the system's database, is most often used 
to help them identify problems and make program adjustments. 

Table 4: Examples of Data Sources Other than DOD's Counternarcotics 
Performance Measurement System Used by DOD Components to Manage 
Counternarcotics Activities: 

DOD component: AFRICOM; 
Examples of other data sources used: Information obtained from site 
visits and U.S and foreign partners. For instance, an AFRICOM official 
told us the command obtained information on the inoperability of 
detection equipment installed in Ghana through site visits. 

DOD component: CENTCOM; 
Examples of other data sources used: Information obtained from 
contractors, site visits, and U.S. law enforcement and foreign 
partners. For example, CENTCOM officials told us they obtain 
information during weekly program meetings with contractors and 
program managers. 

DOD component: EUCOM; 
Examples of other data sources used: Information obtained from site 
visits and U.S and foreign partners. For instance, EUCOM officials 
told us they engage with U.S. law enforcement liaisons to obtain 
information on counternarcotics activities, such as seizures, arrests, 
and closed investigations. 

DOD component: NORTHCOM; 
Examples of other data sources used: Information obtained from site 
visits, U.S. law enforcement and foreign partners. For example, 
NORTHCOM officials told us they obtain information on detection and 
monitoring of drug traffic from the Mexican Navy. 

DOD component: SOUTHCOM; 
Examples of other data sources used: Information obtained from 
contractors, site visits, and U.S. law enforcement and foreign 
partners. For example, SOUTHCOM officials told us they obtain 
information from their foreign partners, such as Colombia and Peru. 

DOD component: JIATF-S; 
Examples of other data sources used: Information from detection and 
monitoring activities. For example, JIATF-S manages activities using 
information stored in databases tracking the effectiveness of 
detection and monitoring activities. 

DOD component: JIATF-W; 
Examples of other data sources used: Information obtained from U.S. 
law enforcement and foreign partners. For example, JIATF-W officials 
told us they obtain information during quarterly command reviews in 
which law enforcement outcomes of JIATF-W activities to build partner 
capacity and share information are discussed. 

Source: GAO analysis of DOD information. 

[End of table] 

Recognizing the need improve the information in the system's database, 
officials from DASD-CN> told us that for fiscal year 2011 they are 
working with DOD components to integrate performance information into 
the system's database that can be more useful for decision making. 
Officials from several combatant commands stated they could integrate 
performance information obtained from outside sources into the 
counternarcotics performance measurement system. Officials from JIATF- 
S, for example, told us they collect and analyze a variety of data on 
counternarcotics activities that they do not input into DOD's 
counternarcotics performance measurement system. On a daily basis, 
JIATF-S collects information on "cases"--that is, boats or planes 
suspected of illegal trafficking. In addition to tracking the number 
of cases, JIATF-S compiles information as to whether or not a 
particular case was targeted, detected, or monitored, and whether or 
not those actions resulted in interdictions or seizures of illegal 
drugs. By compiling this information, officials at JIATF-S told us 
they can better identify program outcomes, areas in which their 
efforts are successful, and ways to take corrective actions. 

* Develop strategy and allocate resources. Agencies can use 
performance information to make decisions that affect future 
strategies, planning, and budgeting, and allocating resources. DASD-
CN>'s role includes both defining the strategic goals and managing 
the budgeting system of the DOD counternarcotics program. DOD's 
counternarcotics guidance states that information from the 
counternarcotics performance measurement system will inform strategic 
counternarcotics plans, but it does not clearly state how the system 
will be used to inform decisions to allocate resources. Moreover, 
officials from DASD-CN> told us that the office does not currently 
link performance information from the counternarcotics performance 
measurement system's database directly to budget allocation decisions. 
In addition, our analysis of DOD's fiscal year 2011 Drug Interdiction 
and Counterdrug Activities Budget Estimates--which provides details on 
DOD's fiscal year 2011 budget request for its counternarcotics 
activities--identified no clear link between budget allocation 
decisions and performance information in the system's database. DOD 
officials told us they plan to incorporate performance information 
from the counternarcotics performance measurement system into future 
budget requests provided to Congress. 

* Identify and share effective approaches. We have reported that high- 
performing organizations can use performance information to identify 
and increase the use of program approaches that are working well. 
According to DOD's counternarcotics performance measurement system 
guidance, DASD-CN> will use performance information submitted to the 
system's database to compile reports for ONDCP, which DASD-CN> has 
done. However, DASD-CN> officials told us they do not currently use 
the system to produce reports for DOD components, which could assist 
in identifying and sharing effective approaches between DOD's 
components. While indicating performance reports could be a useful 
tool, officials from several DOD components told us they had not 
received such reports from DASD-CN>. DOD's May 2010 guidance does 
not state whether the system will be used to produce such reports in 
the future. 

DOD Has Applied Few Practices to Facilitate the Use of Its 
Counternarcotics Performance Measurement System: 

We have found that agencies can adopt practices that can facilitate 
the use of performance data[Footnote 22]. These include (1) 
demonstrating management commitment to results-oriented management; 
(2) aligning agencywide goals, objectives, and measures; (3) improving 
the usefulness of performance data to better meet management's needs; 
(4) developing agency capacity to effectively use performance 
information; and (5) communicating performance information within the 
agency frequently and effectively. 

As part of its role overseeing DOD's counternarcotics activities, DASD-
CN> manages the DOD counternarcotics performance measurement system. 
DASD-CN> applies some practices to facilitate the use of its 
counternarcotics performance measurement system. For example, DASD- 
CN> has recently taken steps to demonstrate management commitment by 
issuing revised guidance emphasizing the development of improved 
performance measures and, according to DASD-CN> officials, 
conducting working groups with some DOD components[Footnote 23] to 
assist them in revising performance measures. Moreover, DASD-CN> 
officials told us they are taking steps to increase staffing to better 
oversee the performance measurement system. We have found that the 
commitment of agency managers to result-oriented management is 
critical to increased use of performance information for policy and 
program decisions. Further, DASD-CN> has created a results framework 
that aligns agencywide goals, objectives, and performance measures for 
its counternarcotics activities. As we have previously reported, such 
an alignment increases the usefulness of the performance information 
collected by decision makers at each level, and reinforces the 
connection between strategic goals and the day-to-day activities of 
managers and staff. 

However, DASD-CN> has not applied certain key practices to 
facilitate the use of data, such as improving the usefulness of 
performance information in its performance measurement system, 
developing agency capacity to use performance information, and 
communicating performance information frequently and effectively. 
Furthermore, DOD officials told us they face challenges using DOD's 
performance measurement system to manage their activities due to (1) 
the limited utility of the performance measures and data currently in 
DOD's counternarcotics database, (2) insufficient capacity to collect 
and use performance information, and (3) infrequent communication from 
DASD-CN> regarding performance information submitted to the 
database. For instance, DOD's guidance emphasizes the development of 
performance measures that are, among other attributes, useful for 
management and supported by credible data. However, DOD officials from 
several combatant commands told us that the performance measures and 
targets currently in the system are of limited utility[Footnote 24] 
and will need to be revised. Moreover, officials from several DOD 
components emphasized the need to build additional capacity to use 
performance data, such as receiving training on how to revise 
performance standards and measures. We have found that the practice of 
building analytical capacity to use performance information--both in 
terms of staff trained to do analysis and availability of research and 
evaluation resources--is critical to an agency using performance 
information in a meaningful way. Finally, DOD components told us that 
they received little feedback or direction from DASD-CN> regarding 
performance information they submitted to the system. We have 
previously reported that improving the communication of performance 
information among staff and stakeholders can facilitate the use of 
performance information in key management activities. For more 
information see table 5. 

Table 5: Status of DOD Efforts to Apply Practices to Facilitate Use of 
Performance Information in Its Counternarcotics Performance 
Measurement System and Reported Challenges, as of June 2010: 

Key practice: Demonstrating management commitment; 
Examples of practice: Agency managers can demonstrate commitment to 
results-oriented management through leading and involving staff from 
different levels in regular performance review meetings to discuss 
progress made toward achieving results; 
DOD efforts to apply practice: 
* DASD-CN> has held working groups with some DOD components to 
discuss the development of performance measures; 
* DOD revised guidance for its counternarcotics performance 
measurement system as of May 2010 with information on the development 
of performance measures; 
* DASD-CN> told us they are increasing staff to oversee the 
counternarcotics performance measurement system; 
Reported challenges: Limited feedback and direction from DASD-CN> 
regarding performance information submitted to the database. 

Key practice: Aligning goals and measures; 
Examples of practice: Agencies can encourage greater use of 
performance information by aligning program performance measures with 
goals and day-to-day activities; 
DOD efforts to apply practice: 
* Most DOD performance measures clearly link to agency goals and 
objectives; 
Reported challenges: None reported. 

Key practice: Improving the usefulness of performance information; 
Examples of practice: To ensure performance information meets users' 
needs, agencies can implement practices such as using an assessment 
tool to document the intended use of a measure, assess the information 
and system in which data are kept, and identify any limitations in 
data; 
DOD efforts to apply practice: 
* DOD's revised guidance for its counternarcotics performance 
measurement system as of May 2010 emphasizes standards of data quality; 
Reported challenges: Limited utility of the performance measures and 
data currently in DOD's counternarcotics database. 

Key practice: Developing agency capacity; 
Examples of practice: Agencies can build analytical capacity to use 
performance information by providing training to staff on setting 
performance standards and measures, analyzing data, and using 
information to revise standards and measures; as well as by providing 
staff access to technical resources and evaluation support staff; 
DOD efforts to apply practice: 
* DASD-CN> has held working groups and training sessions with some 
DOD components on the development of performance measures; 
* Some DOD components have hired staff to assist in the development of 
performance measures; 
Reported challenges: Some DOD components suggest additional training 
on topics, such as analyzing performance data, and using information 
to revise measures is needed due to limited capacity to collect and 
use performance information. 

Key practice: Communicating performance information; 
Examples of practice: To enhance communication among staff and 
stakeholders, agencies can provide performance updates through regular 
e-mail; distribute performance review meeting minutes; or use visual 
tools such as poster displays, performance score cards, or agency 
intranet sites to share performance information; 
DOD efforts to apply practice: 
* DASD-CN> has held working groups with some DOD components to 
discuss the development of performance measures; 
Reported challenges: Limited feedback and direction from DASD-CN> 
regarding performance information submitted to the database. 

Source: GAO analysis of DOD information. 

[End of table] 

Conclusions: 

DOD reported more than $1.5 billion in fiscal year 2010 for its 
counternarcotics activities, but has not yet developed an effective 
performance measurement system to readily inform progress toward the 
achievement of its counternarcotics goals. We have previously reported 
that performance measurement systems include steps to measure 
performance to gauge progress and use the information obtained to make 
key management decisions. DOD acknowledges weaknesses in its 
performance measurement system and has taken steps to improve the 
system, such as revising its guidance for the development of 
performance measures and holding working groups with DOD components. 
However, its current set of measures lack key attributes of successful 
performance measures, such as balance, objectivity, and reliability. 
Moreover, DOD infrequently uses the information presently in its 
counternarcotics performance measurement system and has yet to fully 
apply key practices to facilitate its use. Absent an effective 
performance measurement system, DOD lacks critical performance 
information to use to improve its management decisions, eliminate 
wasteful or unproductive efforts, and conduct oversight of its 
activities. 

Recommendations for Executive Action: 

To improve DOD's performance measurement system to manage and oversee 
its counternarcotics activities, we recommend that the Secretary of 
Defense take the following two actions: 

1. To address weaknesses identified in DOD's counternarcotics 
performance measurement system, we recommend that the Secretary of 
Defense direct the Deputy Assistant Secretary for Counternarcotics and 
Global Threats to review the department's performance measures for 
counternarcotics activities and revise the measures, as appropriate, 
to include the key attributes of successful performance measures 
previously identified by GAO. 

2. To address factors associated with the limited use of DOD's 
counternarcotics performance measurement system, we recommend that the 
Secretary of Defense direct the Deputy Assistant Secretary for 
Counternarcotics and Global Threats to apply practices that GAO has 
identified to facilitate the use of performance data. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to DOD and ONDCP for their review 
and comment. We received written comments from DOD, which are 
reprinted in appendix II. DOD concurred with our recommendations, and 
stated it has developed and begun to implement a plan to improve the 
quality and usefulness of its counternarcotics performance measurement 
system. ONDCP did not provide written comments. 

We received technical comments from DOD and ONDCP, which we have 
incorporated where appropriate. 

We are sending copies of this report to interested congressional 
committees, the Secretary of Defense, and the Director of the Office 
of National Drug Control Policy. In addition, the report will be 
available at no charge on the GAO Web site at [hyperlink, 
http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-4268 or fordj@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made major contributions 
to this report are listed in appendix III. 

Signed by: 

Jess T. Ford: 
Director, International Affairs and Trade: 

List of Congressional Committees: 

The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate: 

The Honorable Daniel K. Inouye:
Chairman:
The Honorable Thad Cochran:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
United States Senate: 

The Honorable Ike Skelton:
Chairman:
The Honorable Howard P. McKeon:
Ranking Member:
Committee on Armed Services:
House of Representatives: 

The Honorable Norman D. Dicks:
Chairman:
The Honorable C.W. Bill Young:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

Section 1016 of the National Defense Authorization Act for Fiscal Year 
2010 directed GAO to report on the Department of Defense's (DOD) 
performance measurement system used to assess its counternarcotics 
activities. In response to this mandate, we examined the extent to 
which (1) DOD's counternarcotics performance measurement system 
enables DOD to track progress and (2) DOD uses performance information 
from its counternarcotics performance measurement system to manage its 
activities. 

Our work focused on the efforts of DOD to develop an effective 
counternarcotics performance measurement system. Within DOD, we spoke 
with officials from several relevant components involved in the 
management, oversight, and implementation of DOD's counternarcotics 
activities, including the Office of the Deputy Assistant Secretary of 
Defense for Counternarcotics and Global Threats (DASD-CN>), U.S. 
Africa Command (AFRICOM), U.S. Central Command (CENTCOM), U.S. 
European Command (EUCOM), U.S. Northern Command (NORTHCOM), U.S. 
Southern Command (SOUTHCOM), the Joint Interagency Task Force-South 
(JIATF-S), the Joint Interagency Task Force-West (JIATF-W), and the 
DOD Inspector General (DOD-IG). We also discussed DOD efforts with 
officials from the Office of National Drug Control Policy (ONDCP), the 
organization that establishes U.S. counternarcotics goals and 
coordinates the federal budget to combat drugs. 

To examine the extent to which DOD's counternarcotics performance 
measurement system enables the department to track its progress we 
analyzed DOD strategy, budget, and performance documents, such as 
DOD's Counternarcotics Strategy, Drug Interdiction and Counterdrug 
Activities Budget Estimates, and Performance Summary Reports. We 
reviewed relevant DOD and ONDCP guidance on performance measures, such 
as DOD's Standard Operating Procedures for the Counternarcotics 
Performance Metrics System and ONDCP's Drug Control Accounting 
circular. Further, we evaluated a generalizable random sample of DOD's 
fiscal year 2009 counternarcotics performance measures (115 of 239 
measures) to assess the extent to which these measures adhered to GAO 
criteria on the key attributes of successful performance measures. 
Because we followed a probability procedure based on random 
selections, our sample is only one of a large number of samples that 
we might have drawn. Since each sample could have provided different 
estimates, we express our confidence in the precision of our 
particular sample's results at a 95 percent confidence interval (e.g., 
plus or minus 6 percentage points). This is the interval that would 
contain the actual population value for 95 percent of the samples we 
could have drawn. To evaluate the sample, two analysts independently 
assessed each of the performance measures against nine attributes of 
successful performance measures identified by GAO.[Footnote 25] Those 
analysts then met to discuss and resolve any differences in the 
results of their analyses. A supervisor then reviewed and approved the 
final results of the analysis. In conducting this analysis, we 
analyzed information contained in DOD's counternarcotics performance 
measurement system database and spoke with DOD officials responsible 
for managing counternarcotics activities and entering information into 
the database. We did not, however, review supporting documentation 
referenced but not included in the system's database, nor did we 
assess other databases that might exist at the DOD component level. We 
also discussed DOD's performance measures with cognizant officials 
from ONDCP and several DOD components, including DASD-CN>, AFRICOM, 
CENTCOM, EUCOM, NORTHCOM, SOUTHCOM, JIATF-S, JIATF-W, and the DOD-IG. 

To evaluate the extent to which DOD uses performance information from 
its counternarcotics performance measurement system to support its 
mission, we held discussions with officials from DOD components-- 
including DASD-CN>, AFRICOM, CENTCOM, EUCOM, NORTHCOM, SOUTHCOM, 
JIATF-S, and JIATF-W--to determine the ways in which these components 
use information from DOD's system, as well as other sources of 
performance information. We also examined DOD's Performance Summary 
Reports and fiscal year 2011 Drug Interdiction and Counterdrug 
Activities Budget Estimates to assess the extent to which these 
materials reported that DOD used performance information from its 
counternarcotics performance measurement system database. Further, we 
analyzed the extent to which DOD applies key management practices 
previously identified by GAO[Footnote 26] to facilitate the use of 
performance information from its counternarcotics performance 
measurement system. We also traveled to Tampa, Miami, and Key West, 
Florida where we visited CENTCOM, SOUTHCOM, and JIATF-S. In these 
visits, we met with DOD officials responsible for management and 
implementation of counternarcotics activities to discuss DOD's use of 
performance data to support its counternarcotics mission. 

To determine the completeness and consistency of DOD funding data, we 
compiled and compared data from DOD with information from cognizant 
U.S. agency officials in Washington, D.C. We also compared the funding 
data with budget summary reports from the ONDCP to corroborate their 
accuracy. Although we did not audit the funding data and are not 
expressing an opinion on them, based on our examination of the 
documents received and our discussions with cognizant agency 
officials, we concluded that the funding data we obtained were 
sufficiently reliable for the purposes of this report. 

We conducted this performance audit from December 2009 to July 2010 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Comments from the Department of Defense: 

Assistant Secretary Of Defense: 
Special Operations, Low-Intensity Conflict & Interdependent 
Capabilities: 
2500 Defense Pentagon: 
Washington, DC 20301-2500: 

July 13, 2010: 

Mr. Jess T. Ford: 
International Affairs and Trade: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Attached is the Department of Defense response to the Government
Accountability Office (GAO) draft report, GAO-10-835, 'DRUG CONTROL: DOD
Needs to Improve Its Performance Measurement System to Better Manage 
and Oversee Its Counternarcotics Activities,' dated July 21, 2010 (GAO 
Code 320743). 

My office has worked diligently to establish a performance metric 
system for the Department of Defense counternarcotics (CN) efforts as 
required under the Office of National Drug Control Policy's (ONDCP) 
Circular "Drug Control Accounting." While it continues to be a work in 
progress, I am confident that the progress being made will result in 
the DoD CN performance metric program providing information that is 
useful and informative to both strategic decision making and 
operational tactics. The attached provides the DoD response to the two 
issues reported by GAO. 

My point of contact for this action is Mrs. Silvia Serban, at 703-614-
8847. 

Signed by: 

Michael Vickers: 

[End of letter] 

GAO Draft Report — Dated July 22,2010: 
GAO-10-835 (GAO Code 320743): 

"Drug Control: DoD Needs to Improve Its Performance Measurement System
to Better Manage and Oversee Its Counternarcotics Activities" 

Department Of Defense Comments To The GAO Recommendations (Draft 8 
July 2010): 

Recommendation 1: To address weaknesses identified in DoD's 
counternarcotics performance measurement system, the GAO recommends 
that the Secretary of Defense direct the Deputy Assistant Secretary 
for Counternarcotics and Global Threats to review the department's 
performance measures for counternarcotics activities and revise the 
measures, as appropriate, to include the key attributes of successful 
performance measures previously identified by GAO. 

DoD Response: Concur. Beginning in June 2009, the Deputy Assistant 
Secretary for Counternarcotics and Global Threats (DASD-CN>) 
identified the need to improve its performance measurement system for 
the Department's counternarcotics program. The DASD-CN> initiated an 
effort to review the current process and guidance for performance 
measurement and developed a two-year transition plan to improve the 
quality and usefulness of the program's performance measurement 
system. The first steps of the review included an examination of the 
current counternarcotics strategy, prior audits by the DoD Inspector 
General, feedback received from the Office of National Drug Control 
Policy (ONDCP), and input from the counternarcotics program managers 
within the combatant commands. The 285 existing performance 
measurements submitted during FY 2008 were individually reviewed along 
with the corresponding project code budget justifications. Each 
performance measurement was objectively analyzed and critiqued based 
on the following criteria: the direct applicability of the stated 
measure; the measure's objectivity; the usefulness of the measure for 
management; the practicality of the measure; the attributable link 
between the measure and its related goal; the timeliness of the data 
collection; and the adequateness of the stated measure to capture the 
activity. 

After the initial review was completed, an assessment report of the 
measures and the overall performance measurement system was compiled. 
From this initial assessment, DoD identified the following areas for 
revision to improve the performance measurement system: 

1. Establish a CN strategic results framework that depicts the casual 
logic cascading from the ONDCP National Drug Control Strategy through 
the DASDCN> strategic goats and objectives, to the individual 
theater CN strategies in place at each combatant command. This 
strategic framework captures the enabling roles among illicit drug 
trafficking disruption, interdiction and apprehension participants and 
presents a comprehensive cause and effect framework that clarifies 
relationships among CN activities and expected results. 

2. Distill the current number of performance indicators to a more 
manageable size built around a uniform and consistent set of 
performance dimensions to increase program understanding and 
accessibility, and lay the foundation for comparing performance across 
similar project code investments. 

3. Establish meaningful performance targets that coalesce with the 
Department's annual planning, program, budgeting, and execution 
timeline. 

4. Expand the counternarcotics performance metric guidance to 
institutionalize the performance metric system and provide 
counternarcotics program managers with informational tools to improve 
the collection and quality of data. 

The DASD-CN> has begun to revise the current performance measures, 
as appropriate, to include the key attributes of successful 
performance measures previously identified by GAO. On May 18, 2010, 
the DASD-CN> issued standard operating procedures (SOPs) for the 
counternarcotics performance metric system. The SOPs provide 
guidelines and instructions to be used in the documentation of 
performance for any counternarcotics activity funded by the 
Department's Central Transfer Account (CTA). As part of the two-yeas 
transition strategy, the SOPs and resulting FY 2010 revisions to the 
counternarcotics performance measurements are focused on creating 
performance measures that display key attributes of successful 
performance measures. In FY 2011, performance indicators will 
incorporate theater level CN strategies and campaign plans to expand 
the performance outcomes achieved within each of the combatant 
commands. Following the release of the SOPs, the DASD-CN> has 
facilitated workshops to assist the combatant commands with the 
composition of a theatre results framework and the revision of current 
performance measurements. Work to improve the performance metrics and 
the overall system for performance measurement is ongoing at the time 
of this report. 

Recommendation 2: To address factors associated with the limited use 
of DoD's counternarcotics performance measurement system, the GAO 
recommends that the Secretary of Defense direct the Deputy Assistant 
Secretary for Counternarcotics and Global Threats to apply practices 
that GAO has identified to facilitate the use of performance data. 

DOD Response: Concur. The efforts of the DASD-CN> to improve the DoD 
counternarcotics performance measurement system are focused on 
capturing information that is useful and informative to both CN 
strategy and operational tactics. The DASDCN> has launched an effort 
to revise the current DoD Counternarcotics Strategy to establish more 
precise goals and robust objectives for the Department's 
counternarcotics program. Furthermore, a results framework has been 
created that aligns goals, objectives, activities, and performance 
measures for each combatant command's counternarcotics activities. 
This alignment increases the usefulness of the performance information 
collected and reinforces the connection between strategic goals and 
tactical activities. 

In FY 2011, performance measurement will incorporate CN theater 
strategies by combatant commands to produce both theater specific 
outcome and output data, thereby assisting counternarcotics program 
managers to identify issues or trends and make immediate adjustments 
as appropriate. The DASD-CN> has expanded the use of performance 
measurements in the Program Objectives Memorandum (POM) process to 
make sure that budget justifications and resource allocations are 
informed by objective performance data. Access to performance data 
will also be improved though the planned use of new technologies that 
will allow counternarcotics program managers to access financial data 
and its corresponding performance measurement information. The Office 
of Counternarcotics and Global Threats also plans to conduct more 
frequent and periodic performance measurement reviews with 
counternarcotics program managers to ensure more information 
management and oversight of the counternarcotics activities. 

[End of section] 

Appendix III: Contact and Staff Acknowledgments: 

Contact: 

Jess Ford, Director, International Affairs and Trade, (202) 512-4268 
or fordj@gao.gov: 

Staff Acknowledgments: 

In addition to the individual named above, Juan Gobel, Assistant 
Director; Elizabeth Curda; Martin de Alteriis; Karen Deans; Mark 
Dowling; Justin Fisher; Richard Geiger; Eileen Larence; Marie Mak; 
Christopher Mulkins; John Pendleton; Elizabeth Repko; and Mark Speight 
made key contributions to this report. 

[End of section] 

Footnotes: 

[1] The National Defense Authorization Act for Fiscal Year 2010, Pub. 
L. No. 111-84 § 1016. 

[2] GAO, Government Reform: Goal Setting and Results, [hyperlink, 
http://www.gao.gov/products/GAO/AIMD/GGD-95-130R] (Washington, D.C.: 
Mar. 27, 1995) and Executive Guide: Effectively Implementing the 
Government Performance and Results Act, GAO/GGD-96-118 (Washington, 
D.C.: June 1996). 

[3] GAO, Preliminary Observations on the Department of Defense's 
Counternarcotics Performance Measurement System, [hyperlink, 
http://www.gao.gov/products/GAO-10-594R] (Washington, D.C.: Apr. 30, 
2010). 

[4] In addition to helping U.S. and foreign agencies address the drug 
trade, the DOD Counternarcotics Strategy also seeks to maintain DOD 
readiness through drug demand reduction programs. As the mandate to 
GAO contained in the National Defense Authorization Act for 2010 
focused on DOD's international counternarcotics activities, this 
report does not contain information on DOD's demand reduction programs. 

[5] DOD defines a combatant command as a military command with 
geographic or functional responsibilities, such as SOUTHCOM or U.S. 
Strategic Command. Military departments include the U.S. Army, Navy, 
and Air Force. Defense agencies, such as the Defense Intelligence 
Agency, perform selected support and service functions on a department-
wide basis. 

[6] GAO, Drug Control: Assets DOD Contributes to Reducing the Illegal 
Drug Supply Have Declined, [hyperlink, 
http://www.gao.gov/products/GAO/NSIAD-00-9] (Washington, D.C.: Dec. 
21, 1999). 

[7] GAO, Drug Control: Difficulties in Measuring Costs and Results of 
Transit Zone Interdiction Efforts, [hyperlink, 
http://www.gao.gov/products/GAO-02-13] (Washington, D.C.: Jan. 25, 
2002). 

[8] GAO, Drug Control: Agencies Need to Plan for Likely Declines in 
Drug Interdiction Assets, and Develop Better Performance Measures for 
Transit Zone Operations, [hyperlink, 
http://www.gao.gov/products/GAO-06-200] (Washington, D.C.: Nov. 15, 
2005). 

[9] ONDCP, ONDCP Circular: Drug Control Accounting, (May 1, 2007). 
Section 7 of the circular contains the requirements of a performance 
summary report. 

[10] [hyperlink, http://www.gao.gov/products/GAO/AIMD-GGD-95-130R] and 
[hyperlink, http://www.gao.gov/products/GAO/GGD-96-118]. 

[11] While some components previously developed performance measures 
to monitor counternarcotics activities, fiscal year 2007 was the first 
year for which DOD centrally collected counternarcotics performance 
measures in its counternarcotics performance measurement system 
database. 

[12] GAO, Tax Administration: IRS Needs to Further Refine Its Tax 
Filing Season Performance Measures, [hyperlink, 
http://www.gao.gov/products/GAO-03-143] (Washington, D.C.: Nov. 22, 
2002). 

[13] We randomly sampled 115 of DOD's 239 counternarcotics performance 
measures for fiscal year 2009 that were associated with DOD's goals of 
detection and monitoring, sharing information, and building capacity 
of partner nations. The resulting estimates are subject to a maximum 
margin of error of plus or minus 6 percentage points. 

[14] We could not determine which of the 5 measurable aspects of 
performance were present for 1 of the 115 measures in the sample 
because the measure did not contain enough information for a thorough 
analysis. As a result, the sum of the percentages does not equal 100. 

[15] GAO, Managing for Results: Analytic Challenges in Measuring 
Performance, [hyperlink, 
http://www.gao.gov/products/GAO/HEHS/GGD-97-138] (Washington, D.C.: 
May 30, 1997). 

[16] JIATF-W is a taskforce of U.S. Pacific Command with a mission to 
combat drug-related transnational organized crime. 

[17] [hyperlink, http://www.gao.gov/products/GAO/HEHS/GGD-97-138]. 

[18] GAO, Managing for Results: Enhancing Agency Use of Performance 
Information for Management Decision Making, [hyperlink, 
http://www.gao.gov/products/GAO-05-927] (Washington, D.C.: Sept. 9, 
2005). 

[19] [hyperlink, http://www.gao.gov/products/GAO/GGD-96-118]. 

[20] We have previously reported that, according to the Government 
Performance and Results Act (GPRA), performance reports should contain 
elements such as describing whether or not agency performance goals 
have been met and discussing performance measures that are most 
significant to the success of a program. See GAO, Results-Oriented 
Government: GPRA Has Established a Solid Foundation for Achieving 
Greater Results, [hyperlink, http://www.gao.gov/products/GAO-04-38] 
(Washington, D.C.: Mar. 10, 2004). 

[21] In May 2008, the DOD-IG released a review of the department's 
fiscal year 2007 performance reporting, see DOD-IG, Independent 
Auditor's Report on the FY 2007 Performance Summary Report for DOD 
National Drug Control Program Activities, D-2008-085 (May 2, 2008). In 
the report, DOD-IG stated that due to delays in receiving material, it 
was not able to express an opinion as to whether the report conformed 
to ONDCP guidance. DOD-IG has not produced reports authenticating 
DOD's fiscal year 2008 or 2009 counternarcotics performance measures. 

[22] [hyperlink, http://www.gao.gov/products/GAO-05-927]. 

[23] As of June 2010, DASD-CN> told us they had conducted working 
groups with CENTCOM and EUCOM, and had scheduled working groups with 
AFRICOM, NORTHCOM, and SOUTHCOM. 

[24] We have previously reported that to be useful, performance 
information must meet users' needs for completeness, accuracy, 
consistency, timeliness, validity, and ease of use. Other attributes 
that affect the usefulness of information include, but are not limited 
to, relevance, credibility, and accessibility. 

[25] [hyperlink, http://www.gao.gov/products/GAO-03-143]. 

[26] [hyperlink, http://www.gao.gov/products/GAO-05-927]. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: