This is the accessible text file for GAO report number GAO-06-270 
entitled 'Federal Contact Centers: Mechanism for Sharing Metrics and 
Oversight Practices along with Improved Data Needed' which was released 
on March 10, 2006. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Requesters: 

United States Government Accountability Office: 

GAO: 

February 2006: 

Federal Contact Centers: 

Mechanism for Sharing Metrics and Oversight Practices along with 
Improved Data Needed: 

GAO-06-270: 

GAO Highlights: 

Highlights of GAO-06-270, a report to congressional requesters: 

Why GAO Did This Study: 

Federal agencies have increasingly relied on contact centers—centers 
handling inquiries via multiple channels such as telephone, Web page, e-
mail, and postal mail—as a key means of communicating with the public. 
Many of these centers are contractor-operated. Concerns exist about the 
accuracy of responses provided through contractor-operated centers. 

This report examines (1) the extent to which the contract terms and 
oversight practices for contact centers at selected agencies emphasize 
the importance of providing accurate information to the public, and (2) 
whether guidance for the operation of contact centers and basic 
information needed to provide general oversight exist. GAO reviewed one 
contractor-operated contact center at each of six agencies: the Centers 
for Disease Control and Prevention (CDC), General Services 
Administration (GSA), U.S. Postal Service (USPS), and the Departments 
of Defense, Labor, and Education (DOD, DOL, and Education). 

What GAO Found: 

The contracts and oversight practices for the contact centers of the 
six agencies reviewed, which handle millions of inquiries annually, 
varied significantly regarding the emphasis they placed on providing 
accurate information to the public. Although federal policy for 
disseminating information to the public specifically emphasizes 
accuracy, only four of the six agencies include accuracy as a 
performance metric in their contracts. With respect to oversight, only 
two of the six agencies used all four of the accuracy-related oversight 
practices we identified—regular knowledge database reviews, regular 
contact monitoring, postcontact customer satisfaction surveys, and 
validation of contractor reports. Although each agency used some form 
of oversight to assess the accuracy of the information provided by its 
contact center, each agency differed regarding how it implemented these 
practices. 

There is no governmentwide guidance or standards for operating contact 
centers—including guidance on specifying accuracy as a contract 
performance metric or as a key focus for oversight. Some agencies 
indicated that had federal guidance been available, it would have 
helped them establish performance indicators and develop oversight 
policies and practices. Recognizing the need for operational standards 
for contact centers, an interagency working group recently proposed 
draft guidelines to OMB and other federal agencies, but OMB has no 
plans to issue these guidelines or any standards for use by agencies. 
Additionally, until recently the federal government had not collected 
data on the universe of federal contact centers. OMB and GSA attempted 
to collect data on the number, types, and costs of federal contact 
centers in 2004, but the data collected were incomplete. In addition, 
no governmentwide procurement information was reported to the Federal 
Procurement Data System (FPDS) in fiscal years 2000 through 2004 using 
the reporting code for telephone call centers, which OMB said is the 
appropriate code for contact centers. The five agencies we reviewed 
that report data to FPDS used a variety of different codes, some 
because they believe that the telephone call center code is too narrow 
to cover the services of their multichannel contact centers. 

Agency Oversight Practices Used for Ensuring Accuracy of Information: 

[See Table 4] 

What GAO Recommends: 

GAO makes recommendations to the Office of Management and Budget (OMB) 
and GSA to improve the sharing of metrics and oversight practices for 
contact centers, and to improve data reporting on contracts for these 
services. OMB and GSA concurred with our recommendations. 

www.gao.gov/cgi-bin/getrpt?GAO-06-270. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact William T. Woods at (202) 
512-8214 or woodsw@gao.gov. 

Contents: 

Letter: 

Results in Brief: 

Background: 

Emphasis on Providing Accurate Information to the Public Varies: 

Governmentwide Guidance and Information on Contact Centers Do Not 
Exist: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: Contact Center Profiles for Six Agencies Reviewed: 

Appendix III: Comments from the Department of Defense: 

Appendix IV: Comments from the General Services Administration: 

Appendix V: Comments from the Department of Health & Human Services: 

Appendix VI: GAO Contact and Staff Acknowledgments: 

Appendix VII: Related GAO Products: 

Tables: 

Table 1: Agency Contact Centers Reviewed: 

Table 2: Summary of Key Performance Metrics in Agency Contracts: 

Table 3: Description of Oversight Practices for Ensuring Accuracy of 
Information: 

Table 4: Agency Oversight Practices Used for Ensuring Accuracy of 
Information: 

Table 5: NAICS Codes Used for Contact Center Contracts Reviewed: 

Table 6: Contact Center Profiles for Agencies Reviewed: 

Abbreviations: 

CDC: Centers for Disease Control and Prevention: 

DOD: Department of Defense: 

DOL: Department of Labor: 

FPDS: Federal Procurement Data System: 

GSA: General Services Administration: 

NAICS: North American Industry Classification System: 

OFPP: Office of Federal Procurement Policy: 

OMB: Office of Management and Budget: 

TMA: TriCare Management Activity: 

USPS: U.S. Postal Service: 

[End of section] 

United States Government Accountability Office: 

Washington, DC 20548: 

February 8, 2006: 

The Honorable Henry A. Waxman: 
Ranking Member: 
Committee on Government Reform: 
House of Representatives: 

The Honorable Edolphus Towns: 
Ranking Member: 
Subcommittee on Government Management, Finance, and Accountability: 
Committee on Government Reform: 
House of Representatives: 

Each year millions of citizens contact the government for information 
critical to their health, finances, or general well-being. In recent 
years, federal agencies have increasingly relied on contact centers-- 
centers handling inquiries via multiple channels such as telephone, Web 
page, e-mail, and postal mail--as an important means of communicating 
with the general public. This was demonstrated most recently during the 
response to Hurricane Katrina. One of the principal ways the federal 
government delivers results, according to the Office of Management and 
Budget (OMB), is by providing timely and accurate information to 
citizens.[Footnote 1] Improving access to high-quality government 
information and services is an important focus for OMB's current 
electronic government initiative, which is an element of the 
President's Management Agenda. Information provided to the public can 
be evaluated by various measures, including accuracy, timeliness, 
completeness, courtesy, and overall customer service satisfaction. OMB 
guidance specifies accuracy as one of the basic elements of quality for 
information provided by agencies to the public.[Footnote 2] Incorrect 
information provided in a fast and courteous manner may be satisfying 
initially, but in the long run, it may be of little use or could be 
detrimental to the individual. 

Many agencies have contracted with private sector firms to operate 
contact centers. Recent GAO reports identified high error rates in the 
accuracy of policy-oriented responses given to the public through one 
contractor-operated contact center.[Footnote 3] In this context, you 
asked us to review how agencies are overseeing their contractor- 
operated contact centers to ensure that they provide accurate 
information to the public. Specifically, we (1) identified the extent 
to which the contract terms and oversight practices for selected 
agencies' contractor-operated contact centers emphasize the importance 
of providing accurate information to the public, and (2) determined 
whether guidance for the operation of contact centers and basic 
information needed to provide general oversight exist. 

To identify contract terms and oversight practices for contact centers, 
we selected one contact center at each of six agencies. We selected 
centers that handle over 1 million inquiries annually and provide 
information to citizens that could significantly affect their finances, 
health, or safety.[Footnote 4] Table 1 lists the contact centers we 
selected for our review and the main public inquiry issues they handle. 

Table 1: Agency Contact Centers Reviewed: 

Agency: Department of Defense (DOD) TriCare Management Activity (TMA) 
North region[A]; 
Contact center: Healthnet's contact center; 
Public inquiry issues handled: Medical benefit coverage, enrollments, 
and claims processing. 

Agency: Department of Education (Education); 
Contact center: Federal Student Aid Information Center; 
Public inquiry issues handled: Aid application status and loan issues. 

Agency: Department of Health and Human Services' Centers for Disease 
Control and Prevention (CDC); 
Contact center: CDC INFO contact center; 
Public inquiry issues handled: Disease prevention, detection, and 
outbreak control information. 

Agency: Department of Labor (DOL); 
Contact center: National Contact Center; 
Public inquiry issues handled: Job issues, workplace safety, pension 
and health benefits. 

Agency: General Services Administration (GSA); 
Contact center: National Contact Center; 
Public inquiry issues handled: General information and referrals for 
any government program; orders for government publications. 

Agency: U.S. Postal Service (USPS); 
Contact center: National Contact Center; 
Public inquiry issues handled: Mail delivery and shipping issues. 

Source: GAO. 

[A] DOD TMA is divided into three separate contract service regions--
North, South, and West. 

[End of table] 

For each of these centers, we interviewed agency officials and reviewed 
the contract terms and performance measures, as well as the policies, 
procedures, reports, and tools used to oversee and evaluate the 
contractors' operation of the contact centers. In addition, we visited 
four contractor-operated centers to observe their operations and 
quality control procedures. We did not independently assess the 
operations of the contact centers or evaluate the effectiveness of the 
contractors' quality control processes. We identified industry 
practices for ensuring the accuracy of information provided by contact 
centers and interviewed representatives from two major contact center 
industry groups. 

To determine the guidance provided to agencies and the information 
gathered by the federal government about contact centers, we 
interviewed OMB and GSA officials and reviewed OMB guidance, the 
results of OMB's request for information from federal agencies about 
their centers, GSA's baseline survey of a sample of agencies' centers, 
and data from the Federal Procurement Data System (FPDS). We also 
reviewed the report of the GSA-sponsored interagency working group--the 
Citizen Service Levels Interagency Committee--that developed potential 
standards for federal contact centers. Appendix I includes a more 
detailed discussion of our scope and methodology, and appendix II 
contains additional details on the six contact centers we reviewed. Our 
work was conducted from February through November 2005 in accordance 
with generally accepted government auditing standards. 

Results in Brief: 

The contracts and oversight practices six agencies used for their 
contractor-operated contact centers varied significantly regarding the 
emphasis they placed on providing accurate information to the public. 
Although federal policy for disseminating information to the public 
specifically emphasizes accuracy, only four of the six agencies' 
contracts explicitly include accuracy as a performance metric. Each 
agency we reviewed used some form of oversight to assess the accuracy 
of the information provided by its contact center, but only two of the 
six agencies used all four of the accuracy-related oversight practices 
we identified, such as contact monitoring and independent review of 
contractor performance reports. Each agency differed regarding how it 
implemented the various practices. For example, some agencies monitor 
calls only on an ad hoc basis, whereas others use a more structured 
system of monitoring to ensure accuracy. This variance was due to a 
number of factors, such as differences among the agencies in staffing 
levels, funding, and the use of guidance specific to the agency. 

There is no governmentwide guidance or standards for operating contact 
centers--including guidance on including accuracy as a contract 
performance metric or as a focus for oversight. Some agencies indicated 
that had federal guidance been available, it would have helped them 
establish performance indicators for service quality, identify the most 
effective contract types and provisions, and develop oversight policies 
and practices. Recognizing the need for operational standards, an 
interagency working group recently proposed draft guidelines to federal 
agencies and OMB. Officials at OMB have reviewed the proposed 
guidelines but do not plan to issue any governmentwide guidance at this 
time; they believe agencies can refer to the proposed guidelines for 
guidance. Additionally, until recently the federal government has not 
collected data on the universe of federal contact centers that could be 
used in oversight of their operations. In 2004, OMB and GSA attempted 
to collect data on the number, types, and costs of federal contact 
centers. The data collected are incomplete, however, because OMB did 
not follow up with nonresponding agencies and the survey GSA used to 
collect the data had methodological flaws. In addition, each of the 
agencies we reviewed that reported information to FPDS chose a 
different reporting code for its contact center. None of the agencies 
used the code that OMB says covers contact center services because at 
least some of the agencies believe the code is limited to telephone 
services. OMB is considering issuing a clarification to the description 
of the code to explain that the code includes not just telephones, but 
also Web sites, e-mails, facsimiles, and so forth, in its next update 
to the North American Industry Classification System (NAICS) manual. 

This report contains recommendations to OMB and GSA that are intended 
to promote the sharing of metrics and oversight practices for contact 
center operations and to ensure that governmentwide data gathered on 
contact centers provide reliable information. OMB and three of the six 
agencies whose centers we reviewed provided comments on a draft of this 
report. OMB, CDC, and GSA concurred with our recommendations. DOD did 
not concur with our draft report because it believed that we did not 
describe all the efforts taken to ensure the accuracy of its contact 
center information. We recognize that DOD takes steps, through its 
contractor and otherwise, to ensure the accuracy of its contact center 
information. Nevertheless, we found no specific performance metric in 
the contract itself regarding accuracy, and we believe our report 
correctly depicts DOD's use of the oversight practices we identified. 

Background: 

In 2001, the President announced his management agenda for making the 
government more focused on citizens and results, which included 
expanding Electronic Government (E-Government). The President's E- 
Government Strategy identified several governmentwide initiatives with 
a goal of eliminating redundant systems and significantly improving the 
government's quality of customer service for citizens and businesses. 
The expected results of the E-Government initiative include providing 
high-quality customer service regardless of whether a citizen contacts 
an agency by phone, in person, or on the worldwide Web. The E- 
Government Act of 2002 codified the President's E-Government 
initiatives and expanded OMB's leadership role by establishing the 
Office of E-Government and Information Technology within OMB.[Footnote 
5] The act also requires that agencies comply with OMB E-Guidance. One 
of the 24 presidential E-Government initiatives is developing and 
deploying governmentwide citizen customer service using industry best 
practices that will provide citizens with timely, consistent responses 
about government information and services via e-mail, telephone, 
Internet, and publications. 

By congressional direction, OMB also is responsible for establishing 
and issuing governmentwide guidelines to federal agencies for ensuring 
the quality of the information disseminated to the public.[Footnote 6] 
In response to this direction, OMB issued guidance to agencies in 
February 2002 that defined the quality of information to include 
accuracy as one of its fundamental elements and directed agencies to 
develop procedures for reviewing and substantiating the quality of 
their information before dissemination. [Footnote 7] 

Contact centers are one method agencies use to disseminate information 
to the public. In the past, public inquiries to the government were 
often made by telephone and thus federal agencies began establishing 
call centers. With evolving technology, citizen inquiries to the 
government now come through various channels such as e-mails, Web-based 
forms, facsimiles, Web chat rooms, and traditional postal mail. As a 
result, agencies have established multichannel contact centers to 
handle these inquiries.[Footnote 8] Contact centers rely on automated 
and live telephone response systems, Web site technologies, and trained 
customer service representatives to provide information to the public. 
For contractor-operated contact centers, the agency typically provides 
either scripted responses or the content from which the contractor 
creates its own scripted responses. The scripts are used for the 
prerecorded telephone response systems, Web pages, and preformatted 
responses given by the customer service representatives. Contact 
centers are staffed in tiers by generalist or specialist 
representatives or a combination of both. Usually, Tier 1 staff handle 
general information inquiries and direct more complex or personal 
issues to specialized Tier 2 or Tier 3 staff or to the agency's subject 
matter experts. 

One method for obtaining information on the contact centers that are 
operated by contractors on behalf of the government is to review data 
from FPDS. FPDS is used to report individual procurement transactions, 
which include the industrial classification of the goods and services 
procured by the federal government. FPDS was implemented by OMB's 
Office of Federal Procurement Policy (OFPP) in 1978 in response to the 
Office of Federal Procurement Policy Act of 1974 requirement of 
establishing a system for collecting and developing information about 
federal procurement contracts.[Footnote 9] Since 1982, GSA has 
administered FPDS on OFPP's behalf. In 2003, the system was revised and 
is now called FPDS-Next Generation.[Footnote 10] A wide range of users, 
including those with the executive and legislative branches, rely on 
FPDS data for information on agency contracting actions, governmentwide 
procurement trends, and achievement of goals related to small business. 

Emphasis on Providing Accurate Information to the Public Varies: 

The six agencies we reviewed emphasized accuracy of contact center 
information to varying degrees through the quality assurance mechanisms 
of their contracts and various oversight practices. Four of the six 
included a specific metric to measure contractor performance related to 
providing accurate information to the public, but only one of the six 
used all four of the oversight practices we identified--such as 
actively monitoring contacts--to ensure that accurate information is 
provided to the public. 

Most Agencies' Contract Metrics Include Accuracy, but Some Did Not: 

Each of the six agencies we reviewed specified key performance metrics 
that its contractor is required to meet. These performance metrics 
define the minimum level of quality acceptable to the agency and 
provide the basis against which the contractor is to be evaluated. We 
found that four of the six agencies' contracts included accuracy of 
information in one or more of the key performance metrics. The 
remaining two agencies did not have specific metrics that addressed the 
need to provide accurate information to the public. Table 2 summarizes 
the key performance metrics specified in the contracts we reviewed and 
indicates through shading those that specifically address providing 
accurate information. 

Table 2: Summary of Key Performance Metrics in Agency Contracts: 

Agency: CDC; 
Key performance metrics: 
* Quality of service--accuracy and completeness of response and 
customer service skills; 
* Customer satisfaction--point value based on monthly average of survey 
respondents providing favorable rating; 
* Cost management--efficiency of operational costs; 
* Business performance--ability to collaborate with other related 
contractors. 

Agency: DOD-TMA; 
Key performance metrics: 
* Busy signals--required percentage of calls received without busy 
signal; 
* Hold time--limit to the number of seconds of hold time during the 
call. 

Agency: DOL; 
Key performance metrics: 
* Quality of service--accuracy of response and customer service skills; 
* Staff occupancy--Minimum percentage of time that customer service 
representatives are actively handling inquiries; 
* First contact resolution--required percentage of contacts resolved 
during first contact; 
* Service level--percentage of calls answered within specified number 
of seconds; 
* Average speed of answer--required percentage of calls answered within 
specific number of seconds; 
* Abandonment rate--minimum percentage of calls that disconnect before 
being handled by the automated system or a customer service 
representative; 
* All trunks busy--minimum percentage of time all telephone trunk lines 
are busy; 
* Percentage of calls completed in Interactive Voice Response system--
minimum percentage of calls handled by the automated response system; 
* Customer satisfaction--percentage of survey respondents providing 
favorable rating; 
* Hold time--limit to the number of seconds of hold time while waiting 
in call queue. 

Agency: Education; 
Key performance metrics: 
* Accuracy of agent answer on calls/e-mails/online live help/facsimile--
required percentage without errors; 
* Customer satisfaction--percentage of survey respondents providing 
favorable rating; 
* Average speed of answer-- required percentage of calls answered 
within specific number of seconds while maintaining minimal abandonment 
rate. 

Agency: GSA; 
Key performance metrics: 
* Accuracy of agent answer on calls/e-mails/postal/facsimile--required 
percentage without errors; 
* Knowledge base audits--number of knowledge pieces reviewed that meet 
accuracy checks; 
* Customer satisfaction--percentage of survey respondents providing 
favorable rating on automated surveys at end of call, outbound call 
surveys, and e-mail surveys; required percentage increase in the number 
of survey respondents providing favorable ratings each quarter; 
* Average speed of answer--required percentage of calls answered within 
specific number of seconds and e-mails/postal/faxes/orders responded to 
within specific number of business days; 
* Service availability--required percentage of time that services are 
available to the public. 

Agency: USPS; 
Key performance metrics: 
* First contact resolution-- required percentage of inquiries resolved 
during first contact (call/e- mail/postal/fax); 
* Service level--required percentage of calls answered within specific 
number of seconds; 
* Customer satisfaction-- percentage of survey respondents providing 
favorable rating on automated surveys at end of call, outbound call 
surveys, and e-mail surveys. 

Source: GAO analysis of agencies' contracts. 

Note: Shaded metrics are those that specifically address accuracy of 
information. 

[End of table] 

Agencies' Contractor Oversight Practices Vary: 

The Federal Acquisition Regulation requires agencies to perform and 
document oversight of their contractors' performance to ensure the 
government receives high-quality services as specified in the 
contract.[Footnote 11] Agency oversight provides quality assurance 
independent of the contractors' own quality control processes. Although 
each agency employed some oversight practices, only two of the six 
agencies we reviewed used all four of the oversight practices we 
identified for ensuring that accurate information is provided to the 
public. Each agency emphasized accuracy of information to varying 
degrees within its practices. 

On the basis of our review of industry contact center practices and the 
practices employed by the agencies considered leaders in government 
contact centers, we identified four agency oversight practices related 
to ensuring that accurate information is provided to the public via a 
contractor-operated contact center.[Footnote 12] Table 3 describes the 
four accuracy-related oversight practices. 

Table 3: Description of Oversight Practices for Ensuring Accuracy of 
Information: 

Oversight practice: Regular knowledge database management; 
Description: Regular review of the information used by customer service 
representatives to respond to inquiries, known in the industry as the 
"knowledge database;" "regular" is defined as occurring at least 
annually. 

Oversight practice: Regular contact monitoring; 
Description: Regular reviews of the information provided in calls or e-
mails to evaluate how well the customer service representatives handled 
the inquiry; the evaluation typically uses a score sheet that allows 
the reviewer to rate the customer service representatives in multiple 
areas, such as courtesy, accuracy of information provided, timeliness, 
completeness, and so forth; "regular" is defined as occurring at least 
on a weekly basis. 

Oversight practice: Postcontact customer satisfaction surveys; 
Description: Surveys asking if the individual was satisfied with the 
service received from the contact center; survey questions ask about 
the individual's opinions, including, to a limited extent, whether the 
information received was accurate; "postcontact" surveys refer to those 
initiated by the agency subsequent to the initial contact and do not 
include self-selected surveys available at the end of a telephone 
conversation. 

Oversight practice: Validation of contractor reports; 
Description: Validating the data provided in contractor-prepared 
reports to ensure accuracy; these reports, which could be provided 
daily, weekly, monthly, or quarterly, include operational information 
such as the center's workload volumes, transaction handling times, and 
results of the contractor's contact monitoring. 

Source: GAO analysis based on review of industry practices and 
discussions with officials from the agencies we reviewed. 

[End of table] 

The first two practices, knowledge database management and agency 
contact monitoring, provide direct oversight regarding accuracy of 
information, because they focus on detecting inaccuracies in the source 
information used to provide responses to the public and in the actual 
responses provided by the customer service representatives. The 
remaining two practices, customer satisfaction surveys and validation 
of contractor-prepared reports, are more indirect methods of ensuring 
accuracy in that they review customers' reactions to the information 
provided and independent agency corroboration of the contractor's 
reporting on its own quality procedures. The agencies we reviewed 
varied with respect to how they implemented these practices. This 
variance was due to a number of factors, such as differences among the 
agencies in staffing levels, funding, and the use of guidance specific 
to the agency. Table 4 shows the extent to which the six agencies we 
reviewed employ each of the accuracy-related oversight practices. 

Table 4: Agency Oversight Practices Used for Ensuring Accuracy of 
Information: 

Agency: CDC; 
Agency oversight practices: Regular knowledge database review: Planned; 
Agency oversight practices: Regular contact monitoring: Daily by third-
party contractor; 
Agency oversight practices: Postcontact customer satisfaction surveys: 
Planned; 
Agency oversight practices: Validation of contractor reports: By third-
party contractor. 

Agency: DOD-TMA North; 
Agency oversight practices: Postcontact customer satisfaction surveys: 
Yes; 

Agency: DOL; 
Agency oversight practices: Regular knowledge database review: 
Continuous basis; 
Agency oversight practices: Regular contact monitoring: Weekly. 

Agency: Education; 
Agency oversight practices: Regular knowledge database review: 
Annually; 
Agency oversight practices: Regular contact monitoring: Weekly. 

Agency: GSA; 
Agency oversight practices: Regular knowledge database review: 
Continuous basis; 
Agency oversight practices: Regular contact monitoring: Weekly; 
Agency oversight practices: Postcontact customer satisfaction surveys: 
On publication orders. 

Agency: USPS; 
Agency oversight practices: Regular knowledge database review: 
Continuous basis; 
Agency oversight practices: Regular contact monitoring: Weekly; 
Agency oversight practices: Postcontact customer satisfaction surveys: 
Yes; 
Agency oversight practices: Validation of contractor reports: With 
third-party contractor assistance. 

Source: GAO analysis. 

[End of table] 

Oversight Practice 1: Regular Knowledge Database Reviews: 

Most of the agencies we reviewed had a structured process for ensuring 
accurate information is maintained in the knowledge database. DOL, 
Education, GSA, and USPS approve contractor-developed information that 
is created based on government-provided materials. These agencies then 
perform periodic reviews of the information in the knowledge database. 
CDC currently prepares all scripted responses and Web site information, 
which the contractor is required to use, and plans to implement annual 
reviews of the knowledge database, starting at the first anniversary of 
operation in February 2006. TMA allows the contractor to develop 
information based on material TMA provides, but does not review the 
information used by the contractor to respond to public inquires. DOD 
said that TMA relies on the expertise and skills of its contractor to 
provide the required services. 

Oversight Practice 2: Regular Agency Contact Monitoring: 

Almost all of the agencies we reviewed perform regular monitoring of 
the contractor's responses to the public to help assess whether 
accurate information is provided. CDC, DOL, Education, GSA, and USPS 
each monitor a number of contacts on a regular basis, although accuracy 
of information is addressed to varying degrees in the score sheets. For 
example, accuracy is clearly weighted as an important aspect of the 
call in CDC's score sheet. Therefore if an inaccurate answer is 
provided, the contractor "fails" for that call and the customer service 
representative is counseled. On the other hand, Education's score sheet 
does not clearly weight accuracy of information. Education and its 
contractor staff could not explain how providing inaccurate information 
on a call would be indicated on the monitoring score sheet. In addition 
to giving different weights to accuracy, the five agencies also vary in 
terms of the frequency with which they monitor their contacts. 
Education and USPS each employ one full-time staff to monitor a 
selection of the contact centers' contacts. CDC has a third-party 
contractor monitor the contact center on a daily basis and uses this 
assessment in the determination of the contractor's award fee. GSA 
staff monitor a sample of calls on a weekly basis and started 
performing quarterly audits of the contractor's monitoring efforts in 
November 2005. The sixth center, TMA, only monitors calls on an ad hoc 
basis when officials visit the contact center. 

Oversight Practice 3: Postcontact Customer Satisfaction Surveys: 

Three of the agencies we reviewed conduct customer satisfaction surveys 
subsequent to the initial contact from an individual. GSA, TMA, and 
USPS conduct customer satisfaction surveys, which ask, to limited 
degrees, questions that address the accuracy of information provided. 
While providing some level of insight regarding accuracy, customer 
surveys may not always provide a valid basis for oversight of the 
accuracy of information, since they usually ask the individual's 
opinion on the service provided. If the survey is conducted too closely 
to the time of the inquiry, the individual may not have had time to act 
upon the information to know whether it is accurate or not. CDC plans 
to implement three types of postcontact customer satisfaction surveys 
through a third-party contractor beginning in June 2006. DOL does not 
conduct postcontact surveys because it does not maintain personal 
information on the individuals that contact the agency. 

Oversight Practice 4: Validation of Contractor Reports: 

Three of the six agencies we reviewed take steps to validate the 
information in the contractor-prepared reports related to contact 
center performance. These reports generally include some aspects 
related to accuracy of information provided to the public, such as the 
contractor's results of its monitoring of contacts. CDC and USPS 
validate to some degree the reports provided by the contractor. GSA 
conducts quarterly audits of its contractor's supporting data. Although 
DOL, Education, and TMA review their contractor reports, they rely upon 
the reports without validation. According to GAO's standards for 
internal control in the federal government, good internal control 
practices require that agencies validate the performance reports 
provided by the contractor to ensure the information is valid.[Footnote 
13] 

Governmentwide Guidance and Information on Contact Centers Do Not 
Exist: 

The federal government does not have comprehensive, centralized 
guidance for operating a contact center or for overseeing a contractor- 
operated center. Although operation and oversight of contact centers 
are the responsibility of individual agencies, GSA, in consultation 
with OMB, determined that governmentwide standards would be useful. GSA 
sponsored an interagency committee that recently provided draft 
guidelines for operating federal contact centers to OMB and other 
federal agencies. However, OMB told us it does not plan to issue any 
governmentwide guidance based on the committee's recommended guidelines 
at this time, because OMB has not identified the operation of contact 
centers as an area of concern. Furthermore, until recently, no 
governmentwide information specific to contact centers has been 
collected. Initial attempts to gather governmentwide information about 
the number and type of activities that agencies use to provide public 
information proved to be inadequate for providing a comprehensive 
governmentwide view of contact centers. In addition, officials from the 
agencies we reviewed told us that no industry classification code in 
FPDS currently covers the full range of services provided by a contact 
center. 

Limited Federal Guidance for Operating and Overseeing Contractor- 
Operated Contact Centers: 

In its 2004 report on the electronic government initiative, OMB 
highlighted the importance of delivering timely and accurate 
information to the public and stated that there are opportunities to 
apply existing and emerging best practices to achieve increases in 
productivity and delivery of services and information.[Footnote 14] To 
date, however, OMB has established only limited guidance on preferred 
practices at contact centers. The only OMB guidance[Footnote 15] we 
found that specifically related to contact centers is focused on the 
use of performance-based contracting for such services.[Footnote 16] 
This guidance is dated and limited in its coverage and does not provide 
guidance on performance metrics for contact centers or oversight 
practices. 

Because of the need for governmentwide standards for operating contact 
centers, GSA in consultation with OMB, took the initiative to form an 
interagency working group to propose guidelines to OMB and other 
federal agencies. Formed in March 2005, the Citizen Service Levels 
Interagency Committee is composed of 58 contact service representatives 
from 33 executive branch agencies. In addition to relying on their 
knowledge in running contact centers, the committee also had a 
contractor perform two studies to provide insight on citizens' 
expectations when contacting government agencies for information and 
current industry metrics, benchmarks, and best practices for operating 
contact centers. The committee submitted a report with 37 proposed 
standards for operating contact centers to OMB in September 2005, 
including four standards specifically related to ensuring accuracy. The 
committee plans to continue to work on additional contact center issues 
and to help agencies implement any contact center standards that OMB 
might endorse. In October 2005, OMB officials stated that they had 
reviewed the committee's report but did not plan to issue any 
governmentwide guidance based on the committee's recommended guidelines 
at this time, because OMB has not identified the operation of contact 
centers as an area of concern. OMB stated further that if agencies need 
additional guidance in developing their standards, they can refer to 
the committee's report. 

The agencies we reviewed each performed independent research to develop 
their contracts and formulate a management strategy for operating their 
contact centers. Performing independent research resulted in 
duplication of efforts across agencies, using limited resources and 
taking valuable time. For example, to develop guidance, the Department 
of Education performed market research, worked with a contractor on 
customer services and related standards, and studied industry best 
practices on a limited basis. Similarly, CDC sent out a request for 
information to industry to gain insight on the technology available for 
operating contact centers before it developed its contract. CDC then 
performed market research, reviewed industry practices, and visited 
other government contact centers, such as that of the Social Security 
Administration and the Centers for Medicare and Medicaid Services, to 
learn about the practices of government-and contractor-operated 
centers. 

In 2004, GSA created a multiple-award contract called FirstContact to 
assist agencies in contracting for contact center services. Under this 
multiple-award contract, agencies can issue a task order to any of five 
preapproved contractors to operate a contact center. Using FirstContact 
will minimize the time and effort required of agencies to locate a 
contractor to manage their centers. To date, the Department of Homeland 
Security's Federal Emergency Management Agency, GSA, and the Department 
of Health and Human Services have placed six task orders against this 
contract, and three other agencies are looking to place orders as well. 
For example, the Federal Emergency Management Agency recently used this 
contract vehicle to quickly provide contact center services for the 
influx of calls and applications for government assistance in the 
aftermath of Hurricane Katrina. Although agencies now have this 
multiple-award contract as a mechanism to assist them in contracting 
for contact center service, they still must develop specific 
performance metrics and oversight practices specific to their center. 

Governmentwide Data on Contact Centers Do Not Exist: 

Given the lack of governmentwide information on the activities that 
provide information to the public, OMB made an initial attempt to 
collect such data in 2004. For this effort, OMB issued a data request 
to all executive branch agencies to obtain basic data, such as the 
contact center name, volume of contacts, and whether performance and 
cost metrics are collected, for every activity that provides 
information directly to the public. OMB normally uses data requests as 
a census tool to acquire a snapshot of the current budget environment 
of the government. Agencies responded by self-identifying over 1,800 
activities that currently provide information to citizens using various 
communication channels such as telephone, e-mail, and Internet Web 
sites. The individual activities identified ranged in size from a 
couple of employees who answer telephone calls as part of their duties 
to contact centers with staffs of several hundred employees who handle 
millions of inquiries through several channels. Of the 1,800 activities 
identified, over 500 categorized themselves as contact centers. Since 
it was making a nonstandard data request, OMB performed little follow- 
up on nonresponding agencies and did not verify reported results. We 
noted that some large agencies, such as DOD, did not report any 
activities that provide information to the public. 

In an effort to expand on the information collected through the OMB 
data request, GSA surveyed 360 activities--approximately a quarter of 
those who responded to the data request--to develop a baseline snapshot 
of governmentwide activities providing information to the public. 
However, GSA's survey methodology was flawed because the agency 
selected its sample from an incomplete universe, had a low survey 
response rate, and did not perform a nonresponder analysis. Thus, the 
survey results did not provide a representative view of activities 
across the government. GSA plans to conduct a follow-up survey of 
government activities in 2007. 

While OMB and GSA information regarding the universe of federal contact 
centers is incomplete, another potential source of information on those 
contact centers that are contracted out by the government is the FPDS. 
Since its inception in 1978, the FPDS has served as the governmentwide 
system for collecting federal procurement data. Five of the six 
agencies we reviewed, however, each used a different code to report 
their contact center procurement actions to FPDS.[Footnote 17] 
Officials from agencies we reviewed told us that no current North 
American Industry Classification System (NAICS) code covers the full 
range of services provided by a contact center.[Footnote 18] Table 5 
lists the NAICS codes used by the five agencies reporting data to FPDS. 

Table 5: NAICS Codes Used for Contact Center Contracts Reviewed: 

Agency: CDC; 
NAICS code: 518210; 
NAICS definition: Data processing, hosting, and related services. 

Agency: DOD-TMA; 
NAICS code: 524114; 
NAICS definition: Direct health and medical insurance carriers. 

Agency: DOL; 
NAICS code: 541519; 
NAICS definition: Other computer- related services. 

Agency: Education; 
NAICS code: 541512; 
NAICS definition: Computer systems design services. 

Agency: GSA; 
NAICS code: 519190; 
NAICS definition: All other information services. 

Source: GAO analysis. 

[End of table] 

The five agencies that reported to FPDS chose different NAICS codes for 
different reasons. Officials from DOL and GSA stated that they chose 
alternative NAICS codes because the definition provided for telephone 
call center does not cover all of the activities handled in a contact 
center. Education and DOD-TMA officials explained that they chose NAICS 
codes that encompassed the main work of the contract, since the contact 
center is only a portion of the work in a contract for a larger 
program. CDC chose its NAICS code based upon the information technology 
services required for creating its contact center. 

No governmentwide procurement information was reported to FPDS using 
the NAICS codes for telephone call centers in fiscal years 2000 through 
2004. This category of NAICS codes--56142--is defined as establishments 
primarily engaged in answering telephone calls and relaying messages or 
in telemarketing activities. Although officials from three of the 
agencies we reviewed expressed the opinion that the definition for 
telephone call centers is too narrow to encompass all the work 
performed by a contact center, OMB told us that the telephone call 
center code is the correct code to use. Specifically, OMB stated that 
the subcode of 561422--telemarketing bureaus--was written with the 
intent to cover all the functions of a contact center. OMB is 
considering issuing a clarification to the description of the 56142 
codes to explain that these codes include more than telephones--such as 
Web sites, e-mails, facsimiles, and so forth--in its next update to the 
NAICS manual in 2012. 

Conclusions: 

Providing timely and accurate information is a key result area for the 
federal government. Federal agencies that use contractor-operated 
contact centers to meet the public's demand for information assume the 
burden of assuring that the information provided by the contractors is 
accurate. While the agencies we reviewed have taken a variety of steps 
to ensure that their contractor-operated contact centers address 
accuracy, at some agencies accuracy clearly does not have the same 
priority as other objectives, such as timeliness. Although agencies 
need flexibility in meeting the needs of the individuals that contact 
them, they also can benefit from the experience gained by their peers 
operating other government contact centers. Short of mandating specific 
guidance, increased sharing among agencies of successful practices for 
managing contact centers may help improve their abilities to write and 
oversee contracts for these centers and may avoid needless duplication 
of effort. The guidelines proposed by the Citizen Service Levels 
Interagency Committee are a step in this direction. However, whether 
this effort will result in information sharing across agencies is 
uncertain. Leveraging knowledge gained by other agencies through the 
sharing of effective practices could be enhanced by governmentwide 
leadership. OMB's leadership of the electronic government initiative, 
its role in guiding agency dissemination of public information, as well 
as its procurement policy role, put it in an ideal position to 
facilitate the exchange of information among agencies to ensure 
effective oversight of contractors in meeting the public's need for 
timely and accurate information. 

While OMB and GSA have taken initial steps to enhance the oversight of 
federal contact centers by gathering some information on the universe 
of these centers, it is not clear whether the data collected provide 
enough information for governmentwide oversight of contact center 
operations or whether GSA's planned data collection efforts will do so 
either. With additional reliable information, OMB may be able to more 
quickly identify and act on emerging problems and opportunities. In 
addition, FPDS can also be more effective in identifying the number of 
contracts and dollars obligated for contact centers across the 
government, but only if the agencies consistently use the appropriate 
NAICS code for these services. 

Recommendations for Executive Action: 

To facilitate the sharing of sound oversight practices for the 
operation of contact centers, to help ensure that providing accurate 
information to the public by contact centers is a priority outcome, and 
to improve the quality of information gathered about these centers, we 
recommend that the Director of the Office of Management and Budget take 
the following actions: 

* Building on efforts begun by the GSA-sponsored interagency committee, 
work with agencies to develop a mechanism for sharing performance 
metrics and oversight practices for contact centers. Continued efforts 
should stress that providing accurate information to the public needs 
to be a key factor in the oversight of federal contact centers. 

* Take steps to ensure consistent reporting on contact centers by 
developing an industry category or specific code definition in NAICS 
that encompasses all the services provided by contact centers or by 
providing further instruction to agencies regarding the appropriate 
NAICS code to use for contact centers. 

To improve the quality of information about federal contact centers, we 
recommend that the Administrator of General Services take the following 
action: 

* Ensure that further efforts to develop governmentwide data on contact 
center operations--such as the survey planned for next year--employ 
sound methodologies to ensure that the resulting information is 
representative of the activities across the government. 

Agency Comments and Our Evaluation: 

We requested comments on a draft of this report from the Office of 
Management and Budget and each of the six agencies we reviewed-- 
Department of Defense, Department of Education, Department of Health 
and Human Services, Department of Labor, General Services 
Administration, and U.S. Postal Service. The Office of Management and 
Budget provided oral comments in which they concurred with our findings 
and recommendations. The Department of Defense, Department of Health 
and Human Services, and the General Services Administration provided 
written comments that are reproduced in appendices III, IV, and V, 
respectively. OMB and most of the agencies also provided technical 
comments, which we incorporated as appropriate. 

The Department of Health and Human Services and the General Services 
Administration also concurred with our findings and recommendations. 
The Department of Defense did not concur with our draft report because 
it believes the report does not fully reflect all the metrics and 
practices DOD and its contractor use to ensure the accuracy of 
information provided to TRICARE beneficiaries. In its comments, DOD 
emphasizes that its approach to contracting for contact center 
operations relies on the contractor to use industry standards for 
ensuring information accuracy. DOD states that standards exist in its 
contract related to the accuracy of information provided by telephone. 
DOD also cites additional metrics it uses for monitoring contractor 
performance. In addition, DOD requires the contractor to have a quality 
management program which must be validated by a nationally recognized 
third-party organization. DOD points out that it receives monthly 
briefings on the operation of the contractor's quality management 
program and observes call center operations during site visits. 
Finally, DOD explained that it monitors the expertise and skills of the 
contractor staff that perform the knowledge management function. 

We recognize that DOD has decided to use what it calls the "audit the 
auditor" approach to quality assurance. It was not our objective, 
however, to assess the merits of any particular approach to ensuring 
quality, but rather to determine the extent to which contract terms and 
agency oversight practices emphasize the importance of providing 
accurate information to the public. In this regard, while the 
contractor may use specific standards for accuracy in its quality 
management program, we found no specific metric related to accuracy in 
the TRICARE contact center contract itself or in the additional metrics 
cited in DOD's comments. For the most part, the additional quality 
control activities listed by the Department are those of its 
contractor, not oversight activities performed by the agency, which was 
the focus of our review. While independent validation of the 
contractor's quality control program helps to ensure the contractor has 
a quality process in place for monitoring its responses to the public, 
this does not substitute for DOD oversight activities such as 
validating the contractor's reports of its monitoring efforts. In 
addition, while DOD performs site visits to oversee the contractor's 
operations, it does so only on an ad hoc basis. Based on DOD's 
comments, we added additional language to the report regarding DOD's 
approach to knowledge management. 

As arranged with your offices, unless you publicly announce its 
contents earlier, we plan no further distribution of this report until 
30 days after the date of this report. We will then send copies of this 
report to the Director of the Office of Management and Budget, the 
Administrator of General Services, the Postmaster General, and the 
Secretaries of the Department of Defense, Department of Education, 
Department of Health and Human Services, and Department of Labor. We 
will also make copies available to others upon request. Contact points 
for our Office of Congressional Relations and Public Affairs may be 
found on the last page of this report. This report is available at no 
charge on GAO's Web site at http://www.gao.gov. 

If you or your staff have any questions about this report, please call 
me at (202) 512-4841. An additional GAO contact and staff who made 
contributions to this report are listed in appendix VI. 

Signed by: 

William T. Woods: 
Director, Acquisition and Sourcing Management: 

[End of section] 

Appendix I: Scope and Methodology: 

To assess the guidance provided to federal agencies and the information 
gathered by the federal government about contact centers, we conducted 
interviews with Office of Management and Budget (OMB) and General 
Services Administration (GSA) officials, and reviewed related guidance 
and results of their initial data collection efforts. We researched and 
discussed with OMB the absence of guidance related to the operation and 
oversight of contact centers. We also discussed the results of OMB's 
2004 request for information to federal agencies that asked for self- 
identification of any activities that provide information to the 
public. In addition, we reviewed and discussed the results of GSA's 
survey of a sample of agency activities that responded to OMB's 
request. We did not assess the validity of the data gathered by OMB and 
GSA. However, a GAO methodologist reviewed the GSA survey methodology 
and identified its weaknesses. In addition, we monitored the progress 
of the GSA-sponsored working group--the Citizen Service Levels 
Interagency Committee--as it developed and recommended standards to OMB 
for federal contact centers. We did not assess the committee's 
recommendations as a whole, but rather reviewed how accuracy of 
information was addressed within its proposed standards. We reviewed 
data from the Federal Procurement Data System for the past 5 fiscal 
years to determine if any contract actions were reported using the code 
for telephone call center services.[Footnote 19] 

To describe federal agencies' efforts to ensure accurate information is 
provided to the public by contractor-operated centers, we reviewed the 
contract terms and oversight activities for one center at each of six 
agencies. We selected centers that handle over 1 million inquiries 
annually and provide information to citizens that could significantly 
affect their finances, health, or safety.[Footnote 20] The contact 
centers selected for our review are: 

* Department of Defense TriCare Management Activity (TMA) North region-
-Healthnet's contact center: provides general and personalized medical 
benefit and coverage information and processes enrollments and claims 
for military families in the North region;[Footnote 21] 

* Department of Education (Education)--Federal Student Aid Information 
Center: provides general information about applications and loan issues 
and personalized information on the status of applications and loans to 
the public and academic community; 

* Department of Health and Human Services' Centers for Disease Control 
and Prevention (CDC)--CDC INFO contact center: provides information 
about health and safety issues--including prevention, detection, and 
outbreak control--to the public and medical professionals; 

* Department of Labor (DOL)--National Contact Center: provides general 
information and referrals regarding job issues, workplace safety, and 
pension and health benefits to the public and employers; 

* General Services Administration--National Contact Center: provides 
general information and referrals related to any agency or government 
program; and: 

* U.S. Postal Service (USPS)--National Contact Center: provides general 
and individualized information on mail delivery and shipping issues to 
the public and businesses. 

To complete our review, we interviewed management and staff responsible 
for oversight of the contractor-operated contact center at each agency. 
We reviewed the performance metrics specified in the agency's contract 
as well as the related reports used to oversee and evaluate the 
contractors' operation of the contact centers. In addition to 
conducting discussions with the agencies, we visited four contractor- 
operated centers to observe their operations and quality control 
procedures. Specifically, we visited locations for the GSA center 
operated by ICT Group, CDC and Education centers operated by Pearson 
Government Solutions, and the DOL center operated by Datatrac 
Information Services.[Footnote 22] At each center we interviewed 
management and customer service representatives regarding the oversight 
practices used to monitor the accuracy of information. We did not test 
the contractors' internal control procedures or validate any data from 
their sample reports. 

We identified industry practices for ensuring the accuracy of 
information provided by contact centers, interviewed representatives 
from two major contact center industry groups--the Society of Consumer 
Affairs Professionals and the Incoming Calls Management Institute--and 
attended the 2005 Government Customer Support Conference. In addition, 
we reviewed prior GAO reports concerning contact centers. We also 
discussed contact center issues with other GAO teams that were 
currently reviewing or had recently reviewed other federal contact 
centers. Our work was conducted from February through November 2005 in 
accordance with generally accepted government auditing standards. 

[End of section] 

Appendix II: Contact Center Profiles for Six Agencies Reviewed: 

Table 6: Contact Center Profiles for Agencies Reviewed: 

Agency: Contact center; 
Centers for Disease Control and Prevention: CDC INFO; 
Department of Defense TriCare Management Activity--North: Healthnet's 
contact center; 
Department of Labor: National Contact Center; 
Department of Education: Federal Student Aid Information Center; 
General Services Administration: National Contact Center; 
U.S. Postal Service: National Contact Center. 

Agency: Purpose of public inquiries; 
Centers for Disease Control and Prevention: Medical disease issues--
including prevention, detection, and outbreak control; 
Department of Defense TriCare Management Activity--North: Medical 
benefits and coverage issues, enrollment, and claims processing; 
Department of Labor: Job loss support, wage information, workplace 
safety, pension and health benefits; fulfillment of orders for posters, 
fact sheets, and handbooks; 
Department of Education: Federal Student Aid application and loan 
issues; 
General Services Administration: Any citizen issues for any agency in 
the government; fulfillment orders for government publications; 
U.S. Postal Service: Mail delivery and shipping information. 

Agency: Methods of contact handled; 
Centers for Disease Control and Prevention: Telephone, e-mail; 
Department of Defense TriCare Management Activity--North: Telephone, e-
mail; 
Department of Labor: Telephone, e-mail; 
Department of Education: Telephone, e-mail, facsimile, Web site, online 
live help; 
General Services Administration: Telephone, e-mail, Web site; 
U.S. Postal Service: Telephone, e-mail, facsimile, Web sites. 

Agency: Main hours of operation; 
Centers for Disease Control and Prevention: Daily 24 hours/day; 
Department of Defense TriCare Management Activity--North: Daily 7:00 
a.m. to 7:00 p.m. (eastern time); 
Department of Labor: Monday-Friday 8:00 a.m. to 8:00 p.m. (eastern 
time)[A]; 
Department of Education: Monday-Friday 8:00 a.m. to midnight; Saturday 
9:00 a.m. to 6:00 p.m. (eastern)[A]; 
General Services Administration: Monday-Friday 8:00 a.m. to 8:00 p.m. 
(eastern)[A]; 
U.S. Postal Service: Monday-Friday 8:00 a.m. to 8:30 p.m; Saturday 8:00 
a.m. to 6:00 p.m. (eastern time). 

Agency: Languages supported (beyond English); 
Centers for Disease Control and Prevention: Spanish; 
Department of Defense TriCare Management Activity--North: Spanish; 
Department of Labor: Spanish and 140 additional languages via 
translation service; 
Department of Education: Spanish; 
General Services Administration: Spanish[B]; 
U.S. Postal Service: Spanish. 

Agency: Fiscal year 2005 call volume; 
Centers for Disease Control and Prevention: 314,749; from February to 
September 2005 (estimated at 2.7 million calls when fully 
operational)[C]; 
Department of Defense TriCare Management Activity--North: 5,236,793; 
Department of Labor: 1,347,476; 
Department of Education: 5,780,158; 
General Services Administration: 1,700,000; 
U.S. Postal Service: 64,001,962. 

Agency: Fiscal year 2005 e-mail volume; 
Centers for Disease Control and Prevention: 834; from February to 
September 2005 (estimated at 18,000 e-mails when fully operational)[C]; 
Department of Defense TriCare Management Activity--North: 56,160; 
Department of Labor: 71,765; 
Department of Education: 488,158; 
General Services Administration: 107,000; 
U.S. Postal Service: 1,041,210. 

Agency: Contractor; 
Centers for Disease Control and Prevention: Pearson Government 
Solutions; 
Department of Defense TriCare Management Activity--North: Healthnet 
Federal Services; 
Department of Labor: Datatrac Information Services; 
Department of Education: Pearson Government Solutions; 
General Services Administration: ICT Group; 
U.S. Postal Service: Convergys. 

Agency: Date of award; 
Centers for Disease Control and Prevention: September 1, 2004; 
Department of Defense TriCare Management Activity--North: September 1, 
2003; 
Department of Labor: September 30, 2001; 
Department of Education: February 1, 2005; 
General Services Administration: November 28, 2004; 
U.S. Postal Service: January 27, 2003. 

Agency: Period of performance; 
Centers for Disease Control and Prevention: 2-year base plus 5 1-year 
options; 
Department of Defense TriCare Management Activity--North: 7-month base 
plus 5 1-year options; 
Department of Labor: 1-year base plus 4 1-year options; 
Department of Education: 1-year base plus 9 1-year options; 
General Services Administration: 1-year base plus 4 1-year options; 
U.S. Postal Service: 4-year base plus 6 1-year options. 

Agency: Total value of contract at award (in millions of dollars); 
Centers for Disease Control and Prevention: $73.7; 
Department of Defense TriCare Management Activity--North: $2,298; (the 
contact center is only a portion of this contract)[D]; 
Department of Labor: $23; 
Department of Education: $80.1; (the contact center is only a portion 
of this contract)[D]; 
General Services Administration: $29; 
U.S. Postal Service: $254.6 (for 4-year base only). 

Agency: Payment structure; 
Centers for Disease Control and Prevention: Cost plus award fee; 
Department of Defense TriCare Management Activity-
-North: Firm fixed price plus award fee; 
Department of Labor: Time and materials; 
Department of Education: Firm fixed price plus award fee; 
General Services Administration: Firm fixed price; 
U.S. Postal Service: Firm fixed price plus award fee. 

Agency: Number of customer service representatives; 
Centers for Disease Control and Prevention: 50-100; 
Department of Defense TriCare Management Activity--North: 600-650; 
Department of Labor: 50-100; 
Department of Education: 400-450; 
General Services Administration: 50-100; 
U.S. Postal Service: 1,200-1,300. 

Agency: Number of locations; 
Centers for Disease Control and Prevention: 3; 
Department of Defense TriCare Management Activity--North: 6; 
Department of Labor: 2; 
Department of Education: 3; 
General Services Administration: 1; 
U.S. Postal Service: 3. 

Source: GAO analysis. 

[A] DOL provides service 24 hours a day for the Occupational Safety and 
Health Administration toll-free number and provided service 24 hours a 
day during hurricane relief efforts; Education extends its hours during 
student aid application season; GSA provides service 24 hours a day 
under emergency situations. 

[B] Although only Spanish is provided at this time, GSA's contract 
allows for the provision of additional languages. 

[C] The CDC contact center is in its second year of operation and is 
consolidating the work for 40 different toll-free numbers over a total 
period of 4 years. 

[D] The contact center is a portion of a larger service contract. The 
value shown here is for the entire contract, as the agency could not 
provide a breakdown of the cost for the contact center alone. 

[End of table] 

[End of section] 

Appendix III: Comments from the Department of Defense: 

THE ASSISTANT SECRETARY OF DEFENSE: 
HEALTH AFFAIRS: 
1200 DEFENSE PENTAGON: 
WASHINGTON, DC 20301-1200: 

JAN 24 2005: 

William T. Woods: 
Director, Acquisition and Sourcing Management: 
U.S. Government Accountability Office: 
441 G. Street, N.W.: 
Washington, DC 20548: 

Dear Mr. Woods: 

This is the Department of Defense (DoD) response to the Government 
Accountability Office (GAO) Draft Report, "FEDERAL CONTACT CENTERS: 
Mechanism for Sharing Metrics and Oversight Practices Along with 
Improved Data Needed," dated December 21, 2005, GAO Code 120407/GAO-06- 
270). 

Thank you for the opportunity to review and comment on the draft 
report. Access to healthcare is one of my highest priorities. Ensuring 
our beneficiaries are able to contact our call centers to address 
questions and receive accurate and timely information is vitally 
important to the success of our TRICARE program. This report provides 
invaluable information on bow other agencies support their contact 
centers and ensure the accuracy of information that might be useful in 
how we operate our centers. 

I want to take this opportunity to clarify the contracting approach TMA 
uses, as well as the Department's approach to performance management 
from both a contractual and operational perspective. 

Overall, I do not concur with the report's findings and 
recommendations. The data contained in the report portrays that TMA is 
not concerned with nor takes efforts to ensure the accuracy of 
information provided to our beneficiaries. In developing our new 
TRICARE contracts (T-NEX) we utilized state-of-the-art contracting 
techniques to obtain best practices of our contractors to provide 
services and healthcare. Further, your report suggests that TMA only 
utilizes two performance metrics. This is incorrect. We use many 
additional measures (also noted in your assessment of other agencies) 
such as first contact resolution, average speed of answer, abandonment 
rate, hold time, staffing ratios, service level, and customer 
satisfaction. Therefore, your comparison between the six agencies in 
the report and TMA is inaccurate. Our contact centers are highly 
effective according to ratings we receive from our beneficiaries, and 
according to our measures. Additional information addressing this is 
enclosed. 

My points of contact are Ms. Christine VanCleave at (303) 676-3639 or 
Mr. Gunther Zimmerman at (703) 681-3492. 

Sincerely, 

Signed by: 

William Winkenwerder, Jr., 

Enclosure: As stated: 

GAO DRAFT REPORT -DATED December 21, 2005 GAO CODE 120407/GAO-06-270: 

"FEDERAL CONTACT CENTERS: Mechanism for Sharing Metrics and Oversight 
Practices Along with Improved Data Needed" 

DEPARTMENT OF DEFENSE COMMENTS: 

GENERAL: 

Following acquisition reforms principles, in 2003 the TRICARE 
Management Activity (TMA) awarded managed care support contracts that 
reflected best commercial practices and performance outcomes. This 
represented a divergence from our historical approach where the 
Government mandated processes, procedures and standards and incurred 
the much higher costs of these Government specified provisions. 
Acquisition reform principles required the Government to assess risk 
and only specify Government processes, procedures and standards where 
industry standards are not acceptable. This does not mean that 
standards do not exist. It simply means that the Government is 
accepting of industry-proposed standards when these standards are 
expected to achieve the Government's specified outcome of the contract. 

TMA, the Surgeons General, and the Assistant Secretary of Defense for 
Health Affairs determined that one of the five objectives of our 
healthcare delivery contracts was to achieve "beneficiary satisfaction 
at the highest level possible throughout the period of performance.." 
With this contract objective defined, a risk assessment of every 
activity occurring through the contracts was conducted to determine 
where there was a high risk of failure associated with commercial best 
practices. In relation to our call center requirements, the 
Government's experts determine that commercial "hold times," "blockage 
rates," and the commercial use of "automated response units" and call 
routing equipment represent a high risk of failure related to 
maintaining beneficiary satisfaction at the highest possible level. 
Because of the risk associated with commercial best practices, the 
Government mandated standards for these functions. The same risk 
assessment was conducted related to an information accuracy standard. 
The Government team determined that industry could not afford errors in 
their commercial sector from either a customer satisfaction or 
liability standpoint and, as such, assessed the risk associated with 
accepting commercial standards for accuracy as low. In the single 
contract considered by the GAO, the quality control program mandated by 
the TRICARE Operations Manual, Chapter l, Section 4, paragraph 4.2. 
[NOTE 1] specifically requires standards and monitoring of the accuracy 
of telephone based responses. 

NOTE: 

[1] The contract, Section C, paragraph C-3.1. incorporates the TRICARE 
Operations Manual as "an integral part of this contract." 

Health Net Federal Services (HNFS) complied with this requirement and 
proposed a 95% standard. Contractually, this standard requires HNFS to 
"establish and operate an internal quality management/quality 
improvement program covering every aspect of the contractor's 
operation, both clinically and administratively." (See contract 
provision C-7.25.) [NOTE 2] This approach allows the Government to 
focus its surveillance efforts on the operationally high risk areas, 
monitor areas of reasonable risk, and be aware of areas with minimal 
risk. At present telephone accuracy is an area assessed as reasonable 
risk. IMA assesses contract performance by mandating internal sampling 
and requiring monthly monitoring. During on-site visits, TMA samples 
the work of HNFS quality management department to validate the accuracy 
of reported information and the implementation of corrective actions. 
This approach effectively utilizes the skills of Government personnel 
who must focus on the outcome of the call, including accuracy, 
timeliness, tone, and follow-up. Each of these individual components is 
effectively monitored through a contractor operated Quality 
Management/Quality Improvement (QM/QI) process that is validated by the 
Government and a nationally recognized body such as International 
Standards Organization (ISO). 

TMA also uses independent sources for performance monitoring. HNFS is 
required, by contract, to obtain ISO certification of their quality 
management program. This independent certification provides the 
Government with additional assurance that the quality management 
processes proposed are utilized, that these processes measure intended 
objectives, and that results are accurately reported. HNFS has obtained 
this certification and continues to manage based on this certification 
criteria. A second ISO 9001:2000 surveillance audit of the HNFS 
Business Management System is tentatively scheduled for January 2006. 
The assessment will include site visits to HNFS corporate headquarters 
in Rancho Cordova CA, Hampton VA and San Diego CA Medical Management 
offices, the Mid Atlantic Sub-regional Headquarters in Norfolk VA, and 
three of the sub- region's TRICAR , Service Centers (Kenner Army Health 
Clinic, Seymour Johnson Air Force Base Clinic, and Cherry Point Naval 
Hospital. 

The TRICARE Regional Office (TRO) Director and staff personally observe 
call center operations when visiting HNFS corporate offices, and 
receive monthly performance briefings on HNFS quality management 
programs. The HNFS call center has an extensive training program for 
new associates, and an ongoing monitoring program for all associates 
that includes evaluation of recorded calls and "chair-side" sessions 
evaluating the accuracy of information in real time. All calls are 
recorded and reviewed for accuracy within a few days. If erroneous 
information is observed, the beneficiary is contacted and correct 
information communicated. HNFS has processes in place to ensure that 
only the latest "approved" information is accessible to customer 
agents. 

NOTE: 

[2] The Internal Quality Management/Quality Improvement Program was a 
distinguishing feature in the evaluation of offerors. 

In summary, standards do exist in TRICARE contracts related to the 
accuracy of information provided by telephone. These standards are 
assessed on a routine basis (one percent of all calls) and reported 
monthly. Telephone operations are observed and monitored during field 
visits, and HNFS provides monthly information briefings discussing 
quality management performance. TMA uses the audit the auditor approach 
to validate the accuracy of the information provided monthly and 
require independent ISO certification of the processes employed. This 
approach allows for the implementation of cost effective industry 
standards and avoids placing the Government in the position of managing 
and monitoring individual contractor employees in favor of managing the 
outcome of the contracts. 

* Page 6. Most Agencies Contract Metrics Include Accuracy, but Some Did 
Not. The GAO states that four of the six agencies' contracts included 
accuracy of information in one more of the key performance metrics. The 
remaining two agencies (to include TMA) did not have specific metrics 
that addressed the need to provide accurate information to the public. 
TMA Comment: The perception created by this narrative is misleading. 
TMA utilizes a "best practice" methodology in negotiating contracts in 
which we rely upon the contractor to meet numerous standards in 
supporting our requirements. No mention of this approach is cited in 
the paragraph. 

Page 6. Table 2: Summary of Key Performance Metrics in Agency 
Contracts. The table identifies two metrics -busy signals and hold 
times. TMA Comment: The table does not identify the "best practice" 
approach utilized in our contracts, nor the many additional metrics 
that we routinely monitor. 

Page 9. Table 4: Agency Oversight Practices Used for Ensuring Accuracy 
of Information. The table identifies oversight practices utilized by 
the six agencies reviewed. TMA Comment: For TMA, only one oversight 
practice is displayed - post contact customer satisfaction survey. 
Again, the display of facts in this manner creates an inaccurate 
portrayal that TMA is not concerned about the accuracy of information 
provided to beneficiaries. 

Page 9. Oversight Practice 1: Regular Knowledge Database Reviews. GAO 
indicates that the other agencies maintain a structure process for 
ensuring accurate information is maintained and provide scripted 
responses and web site information which the contractors use in 
responding to beneficiaries. GAO states that TMA allows the contractor 
to develop information based on material TMA provides, but does not 
review the information used by the contractor to respond to public 
inquiries. TMA Comment: As identified earlier, by our use of "best 
practices" we monitor the expertise and skills of the contactor to 
provide the services we require. 

[End of section] 

Appendix IV: Comments from the General Services Administration: 

GSA: 

GSA Administrator: 

January 26, 2006: 

The Honorable David M. Walker: 
Comptroller General of the United States: 
Government Accountability Office: 
Washington DC 20548: 

Dear Mr. Walker: 

The General Services Administration (GSA) appreciates the opportunity 
to review your draft report, "Federal Contact Centers, Mechanism for 
Sharing Metrics and Oversight Practices Along with Improved Data 
Needed" (GAO-06-270). GSA has nearly 40 years of experience managing 
programs that respond to public inquiries and is a recognized source of 
assistance and expertise in this field. In fiscal year 2005 alone, we 
facilitated the response to 8.5 million calls and emails from citizens. 
Through the Presidential E-Gov initiative, USA Services, we have 
provided leadership governmentwide to help agencies improve their 
response to citizens. 

The following are our comments to your draft report: 

GAO draft report: The chart on page 9 indicates that GSA's National 
Contact Center does not include "accuracy of information" in two of the 
four key performance metrics used to evaluate contractor performance. 

GSA Response: By December of 2005, GSA had put into effect activities 
in each of the four areas. When your audit team visited GSA's National 
Contact Center in June 2005, we had just begun to implement the terms 
and conditions of a new contract with a new contractor. The chart below 
shows the difference between what your team found when they reviewed 
activities in June and the current status. 

Table 4: Agency Oversight Practices Used for Ensuring Accuracy of 
Information: 

[See Table 4] 

Specific actions that we have taken include the following. 

Regular knowledge database review: GSA staff now reviews the National 
Contact Center's database on a daily basis and has begun to conduct 
quarterly audits, completing its first audit in December 2005. Although 
the Government provided the version of the knowledge database used by 
the previous contractor, the current contractor is responsible for its 
further development and maintenance. They accomplish this by reviewing 
the nature of telephone inquiries and the on-line use of the knowledge 
base. 

Regular contact monitoring: In addition to our regular contact 
monitoring, GSA is conducting a quarterly audit of the contractor's 
quality control efforts. The first audit was completed in November 
2005. 

Post contact customer satisfaction surveys: Although GSA's ability to 
conduct post-contact customer satisfaction surveys of callers on its 
information lines is limited by our not collecting information on 
callers (in compliance with the letter and spirit of the Privacy Act), 
we do conduct such surveys of our customers who order publications. 
These surveys, performed under the auspices of the American Customer 
Satisfaction Index, do include a review of the accuracy of the 
transactions and therefore assist us in our evaluation of the 
contractor's efforts. 

Validation of contractor reports: We have started to perform validation 
of contractor reports in two ways. One is a review of the quantity 
measurements by our ability to review reports from MCI and track the 
daily quantity of incoming e-mail. We validate their quality 
assessments by regularly scheduled (at least monthly) calibration 
sessions. These sessions assure that the contractor and the Government 
independently apply the same standards to public inquiries. As 
mentioned above, we also perform quarterly audits. 

GAO report: Page 12, "Although agencies now have this multiple-award 
contract as a mechanism to assist them in contracting for contact 
center service, they still must develop specific performance metrics 
and oversight practices specific to their center." 

GSA response: We are proud that USA Services, a Presidential E-Gov 
initiative managed by GSA, has provided governmentwide leadership by 
establishing the Citizen Service Levels Interagency Committee (CSLIC). 
USA Services established this group in February 2005 and it has now 
completed recommendations to improve agency responses to citizens' 
email and telephone inquiries. These recommendations are enclosed and 
also available online at http://usaservices.gov/cslic.htm. The work of 
CSLIC is ongoing. For the remainder of fiscal year 2006, four 
subcommittees will be working toward certain deliverables. The items 
below are just one example from each subcommittee: 

1) Implementing CSLIC's Recommendations: 
A template to help agencies determine their compliance with CSLIC 
recommendations; 

2) Online Resources: 
A website for the Federal contact center community; 

3) Measuring Customer Satisfaction: 
A library of customer satisfaction measurement methods, and: 

4) Drafting Additional Recommendations: 
CSLIC has already improved its original report, to stress the 
importance of accurate information being delivered to citizens. CSLIC 
plans to draft additional recommendations related to, for example, 
serving people with disabilities. 

GAO draft report: Page 15, "..we recommend that the Administrator of 
General Services.. ensure that the further efforts to develop 
governmentwide data on contact center operations - such as the survey 
planned for next year - employ sound methodologies so that the 
resulting information is representative of the activities across the 
government." 

GSA Response: We concur with your recommendation for GSA executive 
action and have already begun to develop a governmentwide assessment of 
citizen service activities. We are working with the Office of 
Management and Budget (OMB) to ensure a comprehensive gathering of 
data. If the resources are available, our survey methodology will pay 
particular attention to the number of agencies sampled and to the 
analysis of non-responding agencies. 

Information about USA Services can be found online at 
http://usaservices.gov. If you have any questions about the information 
that we have provided in this response, please feel free to contact Mr. 
Stuart Willoughby, Director, USA Services, E-Gov Program Office, at 
stuart.willoughby@gsa.gov or at (202) 501-9121. 

We appreciate the opportunity to respond to GAO draft report GAO-06-270 
and look forward to having our comments reflected in the final report. 

Sincerely, 

Signed by: 

David L. Bibb: 
Acting Administrator: 

Enclosure: 

[End of section] 

Appendix V: Comments from the Department of Health & Human Services: 

DEPARTMENT OF HEALTH & HUMAN SERVICES: 
Office of Inspector General: 

Washington, D.C. 20201: 

JAN 24 2006: 

Mr. William T. Woods: 
Director: 
Acquisition & Sourcing Management: 
U.S. Government Accountability Office: 
Washington, DC 20548: 

Dear Mr. Woods: 

Enclosed are the Department's comments on the U.S. Government 
Accountability Office's (GAO's) draft report entitled, "FEDERAL CONTACT 
CENTERS: Mechanism for Sharing Metrics and Oversight Practices Along 
with Improved Data Needed" (GAO-06-270). These comments represent the 
tentative position of the Department and are subject to reevaluation 
when the final version of this report is received. 

The Department appreciates the opportunity to comment on this draft 
report before its publication. 

Sincerely, 

Signed by: 

Daniel R. Levinson: 
Inspector General: 

Enclosure: 

The Office of Inspector General (OIG) is transmitting the Department's 
response to this draft report in our capacity as the Department's 
designated focal point and coordinator for U.S. Government 
Accountability Office reports. OIG has not conducted an independent 
assessment of these comments and therefore expresses no opinion on 
them. 

COMMENTS OF THE DEPARTMENT OF HEALTH AND HUMAN SERVICES ON THE U.S. 
GOVERNMENT ACCOUNTABILITY OFFICE'S (GAO) DRAFT REPORT ENTITLED, 
"FEDERAL CONTACT CENTERS: MECHANISM FOR SHARING METRICS AND OVERSIGHT 
PRACTICES ALONG WITH IMPROVED DATA NEEDED" (GAO-06-270): 

The Department of Health and Human Services (HHS) appreciates the 
opportunity to comment on the Government Accountability Office's (GAO) 
draft report. 

HHS's Centers for Disease Control and Prevention's (CDC) consolidated 
contact center, CDC-INFO, began operation in February 2005 with a 4- 
year phased implementation. Contact volumes to date include 
approximately 700,000 telephone calls and 10,000 e-mail inquiries. CDC 
staff constantly reevaluate not only content and content accuracy as 
new topics are added but also revisit standard operating procedures to 
ensure that all contacts are handled appropriately. In addition, 
sampling strategies are adjusted to ensure that valid samples are 
reviewed and evaluated. HHS and CDC consider not only accuracy but also 
quality and completeness of service to be equally important indicators, 
and CDC is diligent in its efforts to achieve these goals. 

[End of section] 

Appendix VI: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

John K. Needham (202) 512-5274: 

Acknowledgments: 

In addition to the contact named above, Ruth Eli DeVan, William 
McPhail, Jean Lee, David Schilling, Nyankor Matthews, Robert Swierczek, 
John Krump, Monica Wolford, and Karen O'Conor made key contributions to 
this report. 

[End of section] 

Appendix VII: Related GAO Products: 

Improvements Needed to the Federal Procurement Data System-Next 
Generation. GAO-05-960R. Washington, D.C.: September 27, 2005. 

Social Security Administration: Additional Actions Needed in Ongoing 
Efforts to Improve 800-Number Service. GAO-05-735. Washington, D.C.: 
August 8, 2005. 

Immigration Services: Better Contracting Practices Needed at Call 
Centers. GAO-05-526. Washington, D.C.: June 30, 2005. 

Federal Thrift Savings Plan: Customer Service Practices Adopted by 
Private Sector Plan Managers Should be Considered. GAO-05-38. 
Washington, D.C.: January 18, 2005. 

Medicare: Accuracy of Responses from the 1-800-MEDICARE Help Line 
Should Be Improved. GAO-05-130. Washington, D.C.: December 8, 2004. 

Medicare: Call Centers Need to Improve Responses to Policy-Oriented 
Questions from Providers. GAO-04-669. Washington, D.C.: July 16, 2004. 

Reliability of Federal Procurement Data, GAO-04-295R. Washington, D.C.: 
December 30, 2003. 

Medicare: Communications with Physicians Can Be Improved. GAO-02-249. 
Washington, D.C.: February 27, 2002. 

IRS Telephone Assistance: Limited Progress and Missed Opportunities to 
Analyze Performance in the 2001 Filing Season. GAO-02-212. Washington, 
D.C.: December 7, 2001. 

IRS Telephone Assistance: Quality of Service Mixed in the 2000 Filing 
Season and below IRS' Long-Term Goal. GAO-01-189. Washington, D.C.: 
April 6, 2001. 

IRS Telephone Assistance: Opportunities to Improve Human Capital 
Management. GAO-01-144. Washington, D.C.: January 30, 2001. 

Customer Service: Human Capital Management at Selected Public and 
Private Call Centers. GAO/GGD-00-161. Washington, D.C.: August 22, 
2000. 

Social Security Administration: Information on Monitoring 800 Number 
Telephone Calls. GAO/HEHS-98-56R. Washington, D.C.: December 8, 1997. 

Social Security Administration: More Cost-Effective Approaches Exist to 
Further Improve 800-Number Service. GAO/HEHS-97-79. Washington, D.C.: 
June 11, 1997. 

FOOTNOTES 

[1] Office of Management and Budget, Expanding E-Government: Partnering 
for a Results-Oriented Government, (Washington, D.C.: Dec. 2004). 

[2] Office of Management and Budget, Guidelines for Ensuring and 
Maximizing the Quality, Objectivity, Utility, and Integrity of 
Information Disseminated by Federal Agencies, Federal Register, Vol. 
67, No. 36 (Feb. 22, 2002). 

[3] GAO, Medicare: Accuracy of Responses from the 1-800-MEDICARE Help 
Line Should Be Improved, GAO-05-130 (Washington, D.C.: Dec. 8, 2004); 
GAO, Medicare: Call Centers Need to Improve Responses to Policy- 
Oriented Questions from Providers, GAO-04-669 (Washington, D.C.: Jul. 
16, 2004). 

[4] One agency we selected for our review--the Centers for Disease 
Control and Prevention--is in its first year of a 4-year consolidation 
of workloads. The contact center is projected to receive an estimated 
2.7 million inquiries annually when it is fully operational. 

[5] E-Government Act of 2002, Public Law 107-347, Section 101 (December 
17, 2002). The U.S. Postal Service considers itself exempt from this 
act. 

[6] Consolidated Appropriations Act 2001, Public Law 106-554, Section 
515 (December 21, 2000). 

[7] Federal Register, volume 67, number 36, pages 8452-8460, February 
22, 2002. 

[8] See appendix IV for a list of GAO reports concerning call and 
contact centers. 

[9] Office of Federal Procurement Policy Act of 1974, Public Law 93-400 
(August 30,1974). 

[10] GAO, Improvements Needed to the Federal Procurement Data System- 
Next Generation, GAO-05-960R, (Washington, D.C.: Sept. 27, 2005). 

[11] Federal Acquisition Regulation, at Sections 37.602-2 and 46.104 
(July 2005). 

[12] Some of the agencies we reviewed indicated that training is also a 
key part of ensuring accuracy. However, since training was provided as 
part of the contractors' quality control processes, we did not consider 
it an oversight practice. 

[13] GAO, Standards for Internal Control in the Federal Government, 
GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999); Internal Control 
Management and Evaluation Tool, GAO-01-1008G (Washington, D.C.: August 
2001). 

[14] Office of Management and Budget, Expanding E-Government: 
Partnering for a Results-Oriented Government, (Washington, D.C.: 
December 2004). 

[15] Office of Federal Procurement Policy, Performance-Based Concepts 
for Telephone Call Center Contracting, (Washington, D.C.: 1997). 

[16] Performance-based contracts specify the desired outcomes and allow 
the contractors to determine how best to achieve those outcomes, rather 
than prescribe the methods contractors should use. Performance-based 
contracts can encourage contractors to be innovative and to find cost- 
effective ways of delivering services. 

[17] USPS does not report to FPDS. 

[18] The NAICS code is used to classify the industry type for the 
product or service being purchased. It replaced the Standard Industrial 
Code. 

[19] Despite the shortcomings of FPDS noted in previous GAO reports, 
using FPDS did not adversely affect our work. GAO, Improvements Needed 
to the Federal Procurement Data System-Next Generation, GAO-05-960R 
(Washington, D.C.: Sept. 27, 2005), and Reliability of Federal 
Procurement Data, GAO-04-295R (Washington, D.C.: Dec. 30, 2003). 

[20] One agency we selected for our review--Centers for Disease Control 
and Prevention--is in its first year of a 5-year consolidation of 
workloads. The contact center is projected to receive an estimated 2.7 
million inquiries annually when it is fully operational. 

[21] DOD TMA is divided into three separate contract service areas--
North, South, and West. 

[22] The contractors consider the locations of their call centers to be 
proprietary information. 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, 

NelliganJ@gao.gov 

(202) 512-4800 

U.S. Government Accountability Office, 

441 G Street NW, Room 7149 

Washington, D.C. 20548: