This is the accessible text file for GAO report number GAO-02-329 
entitled 'Desktop Outsourcing: Positive Results Reported, but Analyses 
Could Be Strengthened' which was released on March 29, 2002. 

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products’ accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the 
printed version. The portable document format (PDF) file is an exact 
electronic replica of the printed version. We welcome your feedback. 
Please E-mail your comments regarding the contents or accessibility 
features of this document to Webmaster@gao.gov. 

United States General Accounting Office: 
GAO: 

Report to the Chairman, Subcommittee on Technology and Procurement 
Policy, Committee on Government Reform, House of Representatives. 

March 2002: 

Desktop Outsourcing: 

Positive Results Reported, but Analyses Could Be Strengthened: 

GAO-02-329: 

GAO Highlights: 

Highlights of GAO-02-329, a report to the Chairman, Subcommittee on 
Technology and Procurement Policy, Committee on Government Reform, 
House of Representatives. 

Why GAO Did This Study: 

Since 1997, federal agencies have been engaging in an information-
technology outsourcing approach for acquiring services in support of 
their desktop computing environment from a single source, generally 
known as "seat management." Among its objectives, GAO was asked to 
determine whether six agencies involved in seat management have 
realized expected costs and benefits and to identify lessons learned. 
GAO selected agencies that used various contract vehicles and had at 
least 1 year of experience with the contract. 

What GAO Found: 

Although the six agencies reviewed reported various positive results 
from implementing seat management, GAO could not determine whether 
they were achieving expected costs and benefits because they did not 
perform sufficient up-front analyses or routinely monitor actual 
results. All were tracking elements of contractor performance, such as 
user satisfaction. Although not a substitute for determining whether 
the benefits of seat management outweigh its costs and risks, this 
tracking provides an indication of whether expected services are being 
provided. 

Positive results that these agencies reported can be categorized into 
four general areas: improving information technology management (e.g., 
using a standard technology environment to eliminate incompatible 
hardware and software and improve information sharing across an 
agency); improving end-user support, such as help-desk support; 
enhancing mission support (e.g., staff were freed from desktop 
management duties to perform other mission-related duties); and more 
timely upgrading of technology. 

However, these agencies performed limited or, in some cases, no 
analyses of expected costs and benefits before implementing seat 
management and did not routinely monitor all actual costs or benefits. 
These agencies and other organizations consider up-front analyses and 
subsequent program management critical practices for a successful 
implementation and have identified various lessons learned from their 
experiences that would benefit other agencies considering future seat 
management investments (see list below). 

Lessons Learned: 

Agencies can reduce the risk of an unsuccessful implementation by: 

* obtaining agency commitment, especially by top management; 

* completing thorough up-front preparation and planning activities; 

* carefully managing solicitation and contract award activities; 

* developing strong program and contract management activities, 
including monitoring contractor performance; 

* developing partnerships between agencies and the seat management 
contractors in which they work toward establishing and achieving 
common goals; and; 

* establishing effective and continual communication within the 
agency, with the seat management contractor, and among contractors 
working on related activities. 

What GAO Recommends: 

So that the six agencies reviewed can determine the extent to which 
their current seat management programs are achieving positive results, 
GAO recommends that they monitor actual seat management costs and 
benefits. Also, to help ensure that future seat management investments 
of these six agencies are justified, GAO recommends that when 
considering such investments, these agencies analyze expected costs 
and benefits and, to the extent feasible, implement the lessons 
learned in this report. 

Of the six agencies that commented on a draft of this report, three 
agreed with its findings or recommendations, two did not indicate 
whether they agreed or disagreed, and one supported many of the 
findings, but disagreed with portions of the report. 

This is a test for developing highlights for a GAO report. This full 
report, including GAO's objectives, scope, methodology, and analysis 
is available at [hyperlink, http://www.gao.gov/products/GAO-02-329]. 
For additional information about this report, contact David McClure 
(202-512-6257). To provide comments on this test highlights, contact 
Keith Fultz (202-512-3200) or e-mail HighlightsTest@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Rationales for Adopting Seat Management Varied: 

Agencies Reported Positive Seat Management Results, but Costs and 
Benefits Were Not Adequately Assessed and Monitored: 

Risks Were Identified, but Analyses Were Incomplete or Not Timely: 

Lessons Learned Can Help Other Agencies More Effectively Implement 
Seat Management: 

Conclusions: 

Recommendations: 

Agency Comments and Our Evaluation: 

Appendixes: 

Appendix I: Information on Seat Management Contracts as of December 
31, 2001, Awarded under the NASA ODIN Master Contract: 

Appendix II: Information on Seat Management Contracts as of December 
31, 2001, Awarded under the GSA Seat Management Master Contract: 

Appendix III: Scope and Methodology: 

Appendix IV: Comments from the Department of the Treasury's 
Departmental Offices: 

Appendix V: Comments from the Bureau of Alcohol, Tobacco and Firearms: 

Appendix VI: Comments from the Peace Corps: 

Appendix VII: Comments from the Department of Defense: 

Appendix VIII: Comments from the General Services Administration: 

Appendix IX: Comments from the National Aeronautics and Space 
Administration: 
GAO Comments: 

Appendix X: GAO Contact and Staff Acknowledgments: 
GAO Contact: 
Acknowledgments: 

Table: 

Table 1: Types of Seat Management Services and Cost Data for Agencies 
We Reviewed: 

Abbreviations: 

ATF: Bureau of Alcohol, Tobacco and Firearms: 

CIO: chief information officer: 

CMS: Centers for Medicare and Medicaid Services: 

DLA: Defense Logistics Agency: 

FSS: Federal Supply Service: 

GSA: General Services Administration: 

IT: information technology: 

NASA: National Aeronautics and Space Administration: 

ODIN: Outsourcing Desktop Initiative for NASA: 

OMB: Office of Management and Budget: 

[End of section] 

United States General Accounting Office: 
Washington, D.C. 20548: 

March 29, 2002: 

The Honorable Tom Davis: 
Chairman, Subcommittee on Technology and Procurement Policy: 
Committee on Government Reform: 
House of Representatives: 

Dear Mr. Chairman: 

This letter responds to your request that we review federal agencies' 
efforts to engage in an information technology (IT) outsourcing 
[Footnote 1] alternative for acquiring distributed computing services 
(typically those pertaining to desktop management) known as "seat 
management." Generally speaking, under seat management, contractor-
owned desktop and other computing hardware, software, and related 
services are bundled and provided on the basis of a fixed price per 
unit (or seat). Since the Department of the Treasury's Bureau of 
Alcohol, Tobacco and Firearms (ATF) pioneered the use of seat 
management in the federal government in 1997, more than a dozen other 
federal entities have contracted for seat management services, and 
still others are considering this approach. 

Because more agencies have begun to use the seat management concept, 
you expressed interest in identifying how well this approach has 
worked at different entities. Accordingly, our objectives were to (1) 
determine agencies' rationales for awarding seat management 
contracts,[Footnote 2] (2) assess whether estimated costs and benefits 
have been achieved, (3) ascertain how well agencies have managed the 
risks associated with seat management, and (4) identify lessons 
learned. 

In addition to the primary objectives, you also asked that we identify 
the agencies using the National Aeronautics and Space Administration's 
(NASA) and the General Services Administration's (GSA) seat-management 
governmentwide acquisition contracts. For the agencies using these 
contracts, you asked that we identify (1) the types of services for 
which the agencies contracted and (2) the expected costs and benefits 
associated with these services. We are providing this information in 
appendixes I and II for the NASA and GSA governmentwide seat 
management contracts, respectively. 

To determine agencies' rationales for using seat management, estimated 
and actual costs and benefits, and risk management analyses, we 
reviewed how seat management was implemented at six agencies: NASA, 
the Centers for Medicare and Medicaid Services (CMS), Treasury's 
Departmental Offices and ATF, the Peace Corps, and the Defense 
Logistics Agency (DLA). We chose these agencies because they (1) used 
a variety of contracting vehicles, (2) had seat management contracts 
that had been awarded more than 1 year before our review, and (3) had 
at least 500 seats. 

We reviewed the agencies' seat management contracts, analyses of costs 
and benefits, processes to monitor contractor performance, risk 
analyses, and other related documents. To obtain information on 
lessons learned, we interviewed officials from the six agencies and 
reviewed applicable documentation. We also interviewed officials from 
private-sector organizations, including the seat management 
contractors for the six agencies in our review, and reviewed research 
on seat management and distributed computer outsourcing published by 
private research firms. In addition, we interviewed and obtained 
applicable lessons learned and other documentation from GSA because it 
(1) manages one of the governmentwide seat management contracts and 
(2) abandoned an attempt to implement seat management internally in 
late 2001. Appendix III provides more information on our scope and 
methodology. 

Results in Brief: 

No single overarching reason emerged regarding why agencies decided to 
adopt seat management to address their distributed computing needs. 
Instead, the six agencies in our review adopted seat management for a 
variety of reasons. The most common rationales pertained to improving 
IT management, improving user support and productivity, and obtaining 
or upgrading agency IT. While acknowledging the importance of cost 
considerations in making the seat management decisions, agency 
officials generally stated that lowering the cost of their existing 
services was not their primary reason for choosing this approach. 

All agencies in our review reported that their seat management 
approaches had achieved positive results, such as improving IT 
management (e.g., using one standard technology environment to 
eliminate incompatible hardware and software and improve information 
sharing across an agency) and improving end-user support, such as help-
desk support. However, we could not determine whether any of the 
agencies were achieving expected costs and benefits because they did 
not (1) perform sufficient up-front analyses of their baseline and 
projected costs and benefits and (2) routinely monitor all actual seat 
management costs and benefits. To their credit, all six agencies 
tracked contractor performance against specific performance metrics in 
their contracts in areas such as service delivery and availability. 
While this type of tracking can be an indicator of whether certain 
types of goals are being achieved, such as improved user satisfaction, 
such metrics do not fully address whether the overall cost and benefit 
goals of seat management programs are being met. 

Like any important IT effort, it is critical that agencies consider 
the risks associated with seat management before adopting this 
approach, as noted by the Office of Management and Budget (OMB) and 
our guidance. Four of the agencies in our review identified risks 
associated with seat management, such as possible cost overruns, 
schedule delays, or contractor performance problems. However, of these 
four agencies, none ranked their risks in order of priority, and only 
one identified actions to mitigate risks before implementing seat 
management. In addition, two agencies did not perform an analysis of 
risks at all. One area in which agencies effectively addressed a 
critical risk was by executing contracts that contained adequate 
clauses to protect the government in areas such as quality assurance 
and termination rights. 

By applying the lessons learned by agencies that have implemented seat 
management initiatives, agencies considering this approach could more 
effectively plan their activities and reduce the risk of encountering 
problems experienced by others. Agencies and seat management 
contractors have identified important lessons learned from their 
implementation experiences, namely that, 

* agency commitment is crucial, especially by top management; 

* thorough up-front preparation and planning activities must be 
completed; 

* solicitation and contract award activities should be carefully 
managed; 

* program and contract management activities, including monitoring 
contractor performance, are key; 

* partnerships between agencies and the seat management contractors, 
in which they work toward establishing and achieving common goals, are 
critical; and; 

* effective and continual communication within the agency, with the 
seat management contractor, and among contractors working on related 
activities, is important. 

So that agencies can determine the extent to which their current seat 
management programs have achieved positive results, we are 
recommending that each of the six agencies in our review monitor all 
actual seat management costs and benefits. Also, to ensure that the 
future seat management investments of these agencies are justified, we 
are recommending that they establish a baseline for current costs; 
perform an analysis of expected costs, benefits, and risks; and 
consider implementing the lessons learned identified in this report. 

In providing written comments on a draft of this report, three 
agencies agreed with our findings or recommendations; two did not 
indicate whether they agreed or disagreed; and NASA, while supporting 
many of the findings, disagreed with portions of the report. 
Specifically, NASA did not agree with our assessment that its up-front 
cost analysis was not sufficient, that it did not track its internal 
seat management costs, and that it did not adequately track benefits. 
As discussed in this report, (1) an analysis conducted by a consulting 
firm concluded that NASA lacked a comprehensive baseline of its pre-
seat-management costs; (2) NASA tracked contractor costs and the 
number of internal staff involved with seat management; however, the 
agency did not fully track other internal costs, such as overhead; and 
(3) NASA tracked contractor performance, which addresses certain 
expected benefits, but it did not fully monitor others, such as 
potential increases in user efficiency and productivity. As such, NASA 
lacks a full picture and analysis of its seat management program. 

Background: 

Although there is no generally accepted definition of seat management, 
at its core, it involves using a performance-based contract[Footnote 
3] to obtain equipment, software, and services from a private-sector 
firm to meet an agency's distributed computing requirements (typically 
pertaining to desktop equipment). Agencies are charged on a per-seat 
basis for the services provided,[Footnote 4] but what constitutes a 
"seat" can vary substantially in terms of the type of equipment and 
type and level of service provided. For example, a seat can consist of 
a single type of service sold at a fixed price per unit or as a bundle 
of different services which, taken together, are also sold at a fixed 
price. These different views of seat management account, in part, for 
the different contracting vehicles for seat management in the 
government. Nevertheless, in its purest form, seat management turns 
personal computer resources into a utility or commodity in which the 
customer purchases the right to use the vendor's equipment and 
resources. The vendor remains the owner of the equipment[Footnote 5] 
and is ultimately responsible for its upkeep. 

According to Gartner, Inc., a leading private research firm, 1996 to 
1998 saw a growth trend in publicly reported desktop-management 
outsourcing contracts in the private sector.[Footnote 6] Leading 
organizations have identified both business- and technology-based 
reasons for outsourcing IT services. Business-based reasons included 
being able to (1) focus resources on core business competencies by 
transferring responsibility for IT services to an external provider 
and (2) respond more quickly to business and industry changes by 
leveraging the experience of an external provider. Technology-based 
reasons for outsourcing included (1) gaining quicker access to 
technology skills that are in high demand, (2) acquiring the 
flexibility to grow and shrink high-technology skills as needed, (3) 
gaining access to enhanced hardware and software, (4) acquiring the 
ability to refresh hardware and software as needed, (5) aggregating 
the demand for IT resources from across the organization, and (6) 
achieving a more standardized IT environment. 

A similar growth in IT service contracts occurred in the federal 
government. Specifically, the government's purchases of IT services 
have increased from $3.7 billion in fiscal year 1990 to about $13.4 
billion in fiscal year 2000. In the case of distributed computing 
services outsourcing, ATF is generally acknowledged as the pioneering 
federal agency in implementing seat management since it initially 
outsourced its desktop and network services in late 1997. In mid-1998, 
NASA and GSA awarded governmentwide seat-management indefinite-
delivery, indefinite-quantity contracts. First, in June 1998, NASA 
awarded contracts to seven prime contractors under its Outsourcing 
Desktop Initiative for NASA (ODIN) governmentwide contract. The next 
month, GSA awarded contracts to eight prime contractors under its GSA 
seat management contract. As of December 31, 2001, two agencies were 
using the NASA ODIN contract, and eight agencies were using the GSA 
seat management contract (see appendix I and II for additional 
information about these agencies). In addition, some agencies are 
using GSA's Federal Supply Service (FSS) Schedule 70 contracts' to 
obtain seat management services or have developed and managed their 
own seat management contracts. 

Table 1 illustrates the different seat management approaches taken by 
the six agencies in our review, categorized by the contract vehicle 
used by each. 

Table 1: Types of Seat Management Services and Cost Data for Agencies 
We Reviewed: 

GSA's seat management master contract: 

Contract/Agency: Peace Corps; 
Date of contract/award[A]: April 2000; 
Contract term: 3-year base period, with two 1-year option periods; 
Number and types of seat management services acquired, as of July 31, 
2001: 683 general-purpose desktop seats; 171 portable computer seats; 
31 server seats; 127 network printer seats; 39 communication device 
seats; Additional services include off-site tape storage and 
application program development support; 
Reported actual contract costs, as of July 31, 2001: $7,564,396 
Comments: Under the terms of the master GSA seat management contract, 
infrastructure management (e.g., network management); user support 
(e.g., help desks); and asset management services are bundled as a 
part of a seat. 

Contract/Agency: Treasury's Departmental Offices; 
Date of contract/award[A]: June 1999; 
Contract term: 1-year base period, with nine 1-year option periods; 
First two options exercised; 
Number and types of seat management services acquired, as of July 31, 
2001: 1,642 desktop seats; 700 portable computer seats (other 
equipment, such as servers, printers, and routers, are included in the 
price of these seats); Additional services include voice/telephone 
administration, Web-site support, training, fax support, personal 
digital assistant support, and the establishment of a classified 
network; 
Reported actual contract costs, as of July 31, 2001: $26,678,415; 
Comments: Same as above. 

NASA's ODIN master contract: 

Contract/Agency: NASA's Kennedy Space Center[B]; 
Date of contract/award[A]: November 1998; 
Contract term: 3 years; 3-year follow-on contract also awarded to 
incumbent contractor[C]; 
Number and types of seat management services acquired, as of July 31, 
2001: 3,013 general-purpose desktop computing seats; 421 scientific 
and engineering computing seats; 364 maintenance-only seats; 293 
network-attached device seats (a desktop unit acquired from non-ODIN 
sources attached to the network); 57 World Wide Web seats; 109 file 
storage seats; 17,615 telephone service seats; 48 remote 
communications seats. Additional services include asset management; 
help-desk services; catalog orders (e.g., printers); special orders 
(e.g., toner for network printers); and infrastructure upgrades (e.g., 
upgraded telephone system); 
Reported actual contract costs, as of July 31, 2001: $30,151,512; 
Comments: Under the terms of the NASA ODIN master contract, agencies 
may acquire one or more categories of service (such as desktop 
hardware and software) without other types of services (such as local- 
or wide-area-network services). Some associated services, such as user 
support and infrastructure management, are provided as a part of these 
seats.
		
Contract/Agency: CMS; 
Date of contract/award[A]: June 1999; 
Contract term: 3 years; 
Number and types of seat management services acquired, as of July 31, 
2001: 5,100 general-purpose desktop computer seats; 325 portable 
computer seats. Additional services include asset management; catalog 
orders; special orders (e.g., engineering services); and 
infrastructure upgrades (e.g., gateway servers); 
Reported actual contract costs, as of July 31, 2001: $20,462,675; 
Comments: Same as above. 

GSA's FSS Schedule 70 contracts: 

Contract/Agency: ATF; 
Date of contract/award[A]: April 2001; 
Contract term: 3 years; 
Number and types of seat management services acquired, as of July 31, 
2001: 6,000 desktop/laptop computer seats 300 server seats; 1,143 
printer/scanner seats; 1,275 nonseat annual server, printer, and 
scanner maintenance support. Additional services include centralized 
management, help-desk support, desk side support for key personnel, 
order tracking and support, network infrastructure, and installation 
services (which includes asset management and order tracking); 
Reported actual contract costs, as of July 31, 2001: $62,070,575; 
Comments: ATF awarded its first seat management contract on October 9, 
1997. After this contract expired, ATF awarded a second 3-year 
contract to the incumbent contractor in April 2001. The contract costs 
are from the date of the original contract.
	
Contract/Agency: DLA[D]; 
Date of contract/award[A]: April 2000; 
Contract term: 1-year base period, with four 1-year options[E]; First 
option exercised; 
Number and types of seat management services acquired, as of July 31, 
2001: 2,102 desktop computer seats 251 laptop computer seats; 1,523 
printer seats; 68 server seats; 4 plotter seats; 64 personal digital 
assistant seats. Additional services include local-area-network 
administration and operations; technical support; desktop and server 
maintenance; asset and configuration management; installations, moves, 
additions, and changes; network monitoring; monthly service-level 
reporting; and help-desk support. 	
Reported actual contract costs, as of July 31, 2001: $6,781,493; 
Comments: DLA has maintained ownership of the assets acquired through 
its seat management contract. 
				
[A] This is the date of the resulting delivery order, task order, or 
blanket purchase agreement signed by the agency. 

[B] NASA implemented seat management at 10 different sites. Each site 
had a different mix of seats and services. Since this table is for 
illustrative purposes, we used the Kennedy Space Center as an example 
of a NASA seat implementation site because we visited this site. See 
appendix I for further information on the other 9 NASA seat 
implementation sites. 

[C] Instead of optional years, NASA's contract authorizes the award of 
sole-source follow-on contracts in which a new delivery order is 
negotiated and signed with the incumbent contractor. 

[D] At the time of our review, DLA had implemented seat management 
only at its headquarters complex. 

[E] DLA's base period was from October 1, 2000, through September 30, 
2001, following a transition period from the date of contract award 
through September 30, 2000. 

Source: GAO, based on documents from each of the agencies. We did not 
verify this information. 

[End of table] 

Rationales for Adopting Seat Management Varied: 

Each agency in our review cited unique reasons for implementing seat 
management. However, the most common ones fit into three categories: 
(1) improving IT management, (2) improving end-user support and 
productivity, and (3) obtaining new technology or upgrading current 
technology. In addition, although lower cost was not the primary 
reason that agencies chose the seat management approach, cost issues 
were a consideration. 

* Improving IT management. Five of the six agencies reported that part 
of their rationale for implementing a seat management approach was to 
enhance management of IT resources, which included improvements to 
areas such as standardization, asset management, and IT human capital. 
For example, the Peace Corps implemented seat management, in part, to 
improve and standardize its IT environment—which consisted of diverse, 
old, and incompatible computing equipment—and to obtain a reliable 
inventory of its IT assets. As another example, NASA wanted to shift 
IT asset management responsibilities for its diverse enterprises to 
contractors and use government personnel that were performing desktop 
support for other mission-related work. 

* Improving end-user support and productivity. All six agencies 
reported that at least part of their rationale for implementing seat 
management was to improve desktop management from a user perspective. 
Agency officials stated that they wanted to improve user satisfaction 
and service levels or optimize service delivery by using commercial 
best practices. For example, DLA implemented seat management to 
address the operational impairment of the users at its headquarters 
complex due to problems with its distributed computing environment, 
including frequent and extended local-area-network downtime and remote 
access problems. Treasury's Departmental Offices implemented seat 
management, in part, to combine several services, such as help-desk 
support and maintenance, so that users could have a single point of 
contact rather than having to deal with multiple vendors. 

* Obtaining new technology or upgrading current technology. Four 
agencies reported that part of their rationale for implementing seat 
management was to obtain technology, upgrade current software and 
hardware, and/or provide for periodic technology refreshment. For 
example, ATF wanted to provide its special agents, field inspectors, 
and support staff with personal computers and laptops because they 
could not communicate vital information with each other unless they 
used unsecured cellular or public telephones. Furthermore, ATF wanted 
to institute a technology refreshment program to upgrade or replace 
its equipment every 3 years. Another example is CMS, which implemented 
seat management in part to update its systems and to ensure that its 
equipment was compliant with Year 2000 requirements. 

Reducing Costs Was Not the Primary Objective: 

Although half the agencies said that reducing costs was one of their 
seat management objectives and acknowledged its importance, agency 
officials noted that it was not the primary objective. The other 
agencies cited reasons other than costs for implementing seat 
management. For example, DLA officials indicated that the critical 
issue for their agency was improving its headquarters computing 
environment, which, among other problems, had frequent downtimes. In 
addition, Peace Corps officials stated that the agency did not choose 
seat management to reduce the agency's budget requests associated with 
distributed computing and did not expect that it would. Although cost 
reduction may not have been a primary objective, other cost-related 
considerations, such as the ability to better predict and manage their 
costs, were cited by all of the agencies as reasons for implementing 
seat management. 

The organizations in our November 2001 IT services outsourcing study 
also did not identify reducing the overall costs of the existing IT 
service as a reason for their decision to implement an outsourcing 
solution.[Footnote 8] However, their reasons did include some cost 
considerations, namely, to reduce capital investments and to better 
predict operating costs by contracting for IT services using a 
standard unit of measure. For example, under seat management, services 
are priced on a per-seat basis. 

Agencies Reported Positive Seat Management Results, but Costs and 
Benefits Were Not Adequately Assessed and Monitored: 

Although every agency in our review reported positive results from 
implementing seat management, we could not determine whether they were 
achieving expected costs and benefits because the agencies had not (1) 
conducted adequate pre-seat-management analyses and (2) tracked all 
actual costs and benefits. Specifically, agencies generally did not 
sufficiently analyze their baseline and projected costs and benefits 
up front and monitor actual implementation results. Without such 
critical information, an agency is not positioned to make well-
informed decisions about seat management options or able to 
convincingly demonstrate real results. Although the six agencies were 
not monitoring overall seat management benefits, all were tracking 
elements of contractor performance. While it can be an important 
indicator of results, tracking contractor performance is not a 
substitute for tracking whether the overall cost and benefit goals of 
the seat management program are being achieved. 

Agencies Reported a Variety of Seat Management Results: 

The six agencies in our review reported that their seat management 
initiatives had achieved positive results. These results relate to a 
wide variety of areas that can be categorized into four general areas: 
improving IT management, improving end-user support, enhancing mission 
support, and upgrading technology. Many of these reported 
accomplishments directly correlate with the reasons that agencies 
chose to implement seat management. 

* Improving IT management. Five agencies noted improvements in areas 
such as asset management, security, standardization and 
interoperability, and/or planning for periodic technology 
refreshments. For example, the Peace Corps reported that before 
implementing seat management, the agency had a wide variety of 
incompatible hardware and software that made it difficult for 
employees to easily share information, and that managing its assets 
was very difficult. According to Peace Corps officials, seat 
management allowed the agency to implement a standard technology 
environment and, as a result, information is much more easily shared 
across the agency. Various NASA organizations also reported a myriad 
of IT management improvements, such as improved consistency and 
currency of operating systems and applications; the automated 
distribution of software, including computer virus protection; a 
better understanding of the entity's IT inventory, which resulted in 
the removal of obsolete equipment; and improved software license 
management. 

* Improving end-user support. All six agencies reported that end-user 
support has improved since they implemented seat management. In 
particular, they generally reported improvement in the quality and 
timeliness of help-desk support. At DLA, for example, before seat 
management, customer dissatisfaction with its headquarters IT help-
desk services was widespread. DLA seat management officials believe 
that, on the basis of the results of customer surveys, the quality and 
timeliness of the help-desk support improved substantially after seat 
management was implemented. At NASA, officials in the Office of Space 
Flight reported that implementing seat management has resulted in more 
consistent high-quality service to all users. 

* Enhancing mission support. All six agencies stated that seat 
management has enhanced their mission support and productivity 
because, for example, the staff were assigned to other duties or were 
freed from desktop management duties. In one case, the Treasury 
assistant director responsible for seat management said that after 
implementing seat management, fewer agency employees were needed to 
support IT operations for the Departmental Offices. Therefore, 
Treasury was able to reassign six staff to other organizations within 
the department to help develop new applications software and work on 
security issues. Similarly, NASA's Office of Space Flight reported 
that seat management helped improve the agency's asset management 
services by reducing the amount of work that agency staff have to 
perform to maintain annual inventories of government-owned property 
and auditing of contractors' property control procedures. 

* Upgrading of technology. Four agencies reported that they upgraded 
their technology under seat management in a timely manner. For 
example, using the seat management contract, ATF provided over 4,000 
personal and laptop computers to field agents, among others, in about 
6 months. CMS reported that seat management enabled it to deploy Year 
2000 compliant equipment and software throughout the agency in a 
little more than 3 months. 

Agencies Lacked Facts on Whether Expected Costs and Benefits Were 
Being Achieved: 

We could not determine whether the agencies were achieving expected 
costs and benefits because they generally did not (1) sufficiently 
analyze their baseline and projected costs and benefits up front and 
(2) routinely monitor all actual seat management costs and benefits. 
Like any other major IT investment, seat management initiatives should 
be supported by a well-developed business case that evaluates the 
expected returns against the costs. An explicit understanding of the 
expected costs and benefits up front provides the basis for a sound 
financial and strategic decision and creates a baseline against which 
managers and executives can measure progress. According to OMB 
guidance, an analysis of the expected costs and benefits of IT 
investments, as well as the costs and benefits of alternatives, should 
be included in the justification for major investments.[Footnote 9] 
Similarly, our guidance on IT investment management calls for agencies 
to identify the expected costs and benefits of proposed investments. 
[Footnote 10] In addition, our research on leading commercial 
practices for acquiring IT services found that the optimizing and 
baselining of existing internal IT processes provide an organization 
with the information to make a sourcing decision.[Footnote 11] 

Although all six agencies analyzed their expected seat management 
costs and/or benefits or baselined their existing environment, these 
analyses were missing critical elements. Specifically, 

* NASA and CMS estimated the costs and benefits that they expected to 
achieve with seat management, including an analysis of alternatives. 
However, internal agency costs, such as those associated with program 
and contract management, were not included in their estimates. 
Moreover, NASA and CMS's analyses did not include a thorough or 
reliable baseline of the costs associated with their pre-seat-
management computing environments. For example, in a November 2001 
postimplementation report, a private-sector firm concluded that it was 
not possible to determine with any degree of confidence whether NASA 
had saved money because of the lack of a comprehensive baseline. 
[Footnote 12] 

* While Treasury's Departmental Offices estimated the costs of seat 
management and an alternative, it did not estimate the benefits for 
either approach. In addition, the cost estimate was developed about a 
week before the contract award and was based on the winning bidder's 
proposal. 

* Although DLA and the Peace Corps performed analyses of their preseat-
management costs, they did not estimate the expected costs or benefits 
of seat management or any alternatives. In addition, the Peace Corps' 
analysis was not a true baseline for the seat management program that 
the agency implemented because it included (and did not distinguish 
between) the costs for both domestic and foreign operations, whereas 
it implemented seat management only for its domestic operations. 

* ATF did not estimate the costs and benefits of seat management or 
alternatives before implementing this approach in 1997. However, in 
2000, when the contract was under consideration for renewal and the 
agency was reviewing its possible choices, a cost study was performed 
that identified the agency's total cost of ownership and service 
levels under seat management and compared them with peer 
organizations. However, this study did not include an analysis of non-
seatmanagemet[Footnote 13] alternatives or expected benefits. 

The agencies cited various reasons for not completing a thorough 
analysis of their existing environment and expected costs and benefits 
of seat management and alternatives, including that they lacked time 
or resources to complete the analysis, that upper-level management had 
mandated that outsourcing and a performance-based contract be used, 
and that budgetary savings was not a goal of the seat management 
program. However, such an approach is risky. Together with OMB, we 
have long held that the analysis of costs and benefits is critical to 
IT investment decision-making. As we noted in our 1997 IT investment 
guide, an IT investment process cannot operate effectively without 
accurate, reliable, and up-to-date data on project costs and benefits. 
[Footnote 14] In addition, it is important that agencies consider the 
criticality of distributed computing to their mission, not just the 
size of the investment, when determining the necessary 
comprehensiveness of the analyses of costs and benefits. 

Once an IT investment is implemented, OMB Circular A-130 and our IT 
investment management guide[Footnote 15] recommend that agencies 
validate whether estimated costs and benefits are being achieved. In 
an outsourcing arrangement, to obtain a true picture of the cost of 
the investment, it is important to consider both the cost of the 
contract as well as the agency's internal costs to administer the 
program and contract. Each of the six agencies tracked contractor 
costs. However, while they provided us with estimates of their 
internal costs for managing the seat management contract and program 
(in some cases just providing salaries and benefits for a given 
month), agencies in our review did not routinely monitor all internal 
agency costs associated with seat management.[Footnote 16] Such costs 
can be substantial. For example, NASA's Kennedy Space Center reported 
that its salaries and benefits to manage the Office of Space Flight's 
four seat management implementations in July 2001 were about $146,000. 
If projected over the life of the original 3-year contract,[Footnote 
17] the center's salaries and benefit costs to manage this initiative 
would be about $5.3 million. 

None of the agencies routinely monitored the actual overall benefits 
of their seat management programs. This includes the two agencies 
(NASA and CMS) that had estimated the quantifiable benefits that they 
had expected to achieve. For example, CMS estimated that it would have 
achieved quantifiable benefits of about $22 million between fiscal 
years 1999 and 2001 in increased staff productivity and the avoidance 
of costs associated with, for example, asset theft. However, CMS has 
not validated whether these benefits have been achieved. The agencies 
had a variety of reasons for not monitoring actual benefits, including 
that they lacked resources or that, because they did not baseline 
their costs before seat management, they did not have a basis to 
perform a comparison. Since agencies' rationales for adopting seat 
management were generally to meet certain programmatic objectives and 
not to achieve cost savings, it is especially important that they 
monitor benefits to ensure that their seat management programs are 
meeting these objectives. 

Although the six agencies were not tracking overall seat management 
benefits, all of them were measuring and tracking some aspects of 
contractor performance, typically by reviewing contractor-provided 
data periodically (e.g., monthly or quarterly). The agencies have 
specific performance metrics within their contracts to measure 
contractor performance, generally including measures related to 
service delivery and availability and user satisfaction or help-desk 
performance. The agencies use these measures as the basis for 
approving vendor payments, awarding bonuses or other incentives, 
and/or applying penalties.[Footnote 18] For example, one agency 
assessed its contractor's performance against 24 service-level 
agreements,[Footnote 19] such as server availability and customer 
satisfaction, to determine whether penalties were warranted (this 
agency's contract does not provide for incentives or bonuses). For 
fiscal year 2001, this agency assessed its contractor penalties 
totaling $30,295. In another example, NASA uses "retainage" pools in 
which a certain percentage (the amount of which is determined by each 
center's contract) of the contractor's monthly seat price is retained 
by the government and disbursed periodically on the basis of a review 
of the contractor's performance. One center retained about $893,000 
between December 1998 and June 2001, of which it disbursed about 
$509,000. 

Agency seat management contracts sometimes did not include contractor 
performance measures pertaining to all business goals of the seat 
management program. For example, one of the Peace Corps' performance 
goals in its September 2000 strategic plan is to pursue efforts to cut 
costs and improve agency productivity. Although the implementation of 
seat management is a part of the evaluation criteria associated with 
this goal, the agency's seat-management performance metrics plan does 
not include a performance measure for cost reductions or improvements 
to agency productivity. In another case, a private-sector firm's 
review of NASA's ODIN program noted that the agency's contracts 
address only portions of its program objectives. The firm recommended 
that NASA initiate a full review of its metrics. 

Although critical to ensuring that the agency is obtaining 
contractually required services and an indicator that certain types of 
benefits are, or are not, being achieved (e.g., improved user 
satisfaction), measuring and tracking contractor performance does not 
take the place of tracking the overall benefits of seat management 
programs. For example, contractor performance metrics do not measure 
many of the types of benefits that agencies believe they are achieving 
or wanted to achieve, such as increased end-user productivity, 
improved mission support through the transfer of agency IT staff to 
more critical tasks, and improved standardization. 

Some agencies have recognized the need to review their actual seat 
management costs and benefits and have obtained, or plan to obtain, 
independent assessments of at least some aspects of their seat
management program. For example, 

* NASA contracted with a private-sector firm to perform a 
postimplementation business case assessment. In November 2001, this 
firm reported that (1) over 90 percent of NASA management officials 
surveyed indicated that ODIN provided some benefit in areas such as 
improved service and standardization and (2) it was impossible to 
determine whether NASA is saving money because of the lack of a 
comprehensive pre-seat-management baseline, although there was some 
evidence that the agency had achieved savings due, at least in part, 
to seat management. 

* In late 2000, CMS conducted a pilot total-cost-of-ownership study of 
its post-seat-management distributed computing environment using a 
private-sector methodology and software tool. CMS limited its data 
collection to readily available information and did not attempt to 
conduct a full-scale study. 

* DLA plans to have a contractor complete a postimplementation total-
cost-of-ownership study, which evaluates the current distributed 
computing environment of an organization, and user satisfaction 
benchmark survey in the spring. 

* In January, Treasury signed a contract for the completion of a total-
cost-of-ownership seat management study that is expected to be 
completed in June 2002. In responding to a draft of this report, 
Treasury's Departmental Offices stated that this study is expected to 
(1) review current costs and associated benefits and (2) allow the 
department to predict the expected costs of services for the future 
years of the contract and identify specific benefits expected to be 
derived from these costs. 

* Peace Corps officials stated that the agency intends to issue a 
contract for a total-cost-of-ownership review by the end of this 
fiscal year to help it identify seat management results. In addition, 
the Peace Corps contracted for an independent review of user 
satisfaction, and it expects to receive the final report pertaining to 
this review by the end of March. In commenting on a draft of this 
report, the Peace Corps stated that it expects that these analyses 
will address agency productivity and cost-reduction performance 
indicators. 

For postimplementation reviews to be most effective, it is critical 
that agencies have an established baseline before implementation so 
that there could be a basis of comparison for determining progress. 
However, because most of the six agencies did not have a full or 
reliable baseline of their costs before implementing seat management, 
it would be difficult to validate whether any cost savings had been 
achieved. For example, the CMS seat management program manager told us 
that it would be difficult to determine whether expected cost savings 
are being achieved because the agency lacks comparable information on 
its costs before implementing seat management. 

Risks Were Identified, but Analyses Were Incomplete or Not Timely: 

Although four agencies identified a variety of risks, these analyses 
were incomplete or not performed in a timely manner. OMB and our 
guidance note that agencies should address potential risks, such as 
investment, organizational, funding, and technical risks, when 
considering new IT investments.[Footnote 20] Not identifying risks or 
developing strategies to resolve them can be problematic since most IT 
investments require a constant focus on interim results and successful 
risk management strategies to effectively address existing or emerging 
factors that influence implementation results. Nevertheless, of the 
four agencies that identified seat management risks, none ranked their 
risks and only one identified actions to mitigate risks before 
implementing seat management. In addition, two agencies did not assess 
risks at all. One important risk area associated with outsourcing—
ensuring that contracts contain clauses that protect the government's 
interests—was adequately addressed by the six agencies for (1) quality 
assurance, (2) termination rights, and (3) the government's rights to 
supplied hardware and software at the end of contract performance. 

Of the six agencies we reviewed, four identified risks associated with 
their seat management implementations. In particular, consistent with 
OMB and our guidance, which specifies a variety of risks for agencies 
to consider,[Footnote 21] these four agencies identified risks 
relating to costs, time frames for implementation, contractor 
performance, and technical issues as well as other risks. Examples of 
risks identified in these categories are discussed below: 

* Costs. All four agencies identified cost as a risk. For example, 
Treasury determined that its cost risk was low because multiple 
vendors would be bidding. 

* Time frames for implementation. Three agencies identified possible 
delays as a risk. For example, CMS identified implementation as a risk 
that it would mitigate by developing milestones and providing contract 
incentives. 

* Contractor performance. Three agencies identified contractor 
performance issues as a risk area. For example, NASA identified 
contractor performance as a manageable risk, but was concerned that if 
it were to become a problem, mission effectiveness might be 
compromised because the agency would have limited staff available to 
correct the deficiencies. The agency planned to address this risk by, 
for example, limiting the contract to 3 years and having a pool of 
vendors from which to choose. 

* Technical issues. Three agencies identified technical issues as a 
risk area. For example, CMS was concerned about the suitability of the 
hardware and software to be provided by its seat management 
contractor. CMS addressed this risk by requiring testing of equipment 
proposed by firms competing for the seat-management contract award, 
requiring postaward testing of initial equipment installations, and 
using an independent contractor to test the capabilities of systems 
proposed for the refreshment cycle. 

* Other risks. Two agencies identified other risks. For example, NASA 
identified the need for coordination between ODIN-related desktop and 
intracenter communication architectures and other major agency 
initiatives, including its Integrated Financial Management Program, 
and vendors providing non-ODIN computing and communication services. 
NASA's plans for addressing these risks included extensive internal 
communication and the development of program interface agreements. 

Even though four agencies identified the risks associated with seat 
management, these analyses were incomplete and/or not timely. 
Specifically, 

* None of the analyses ranked risks in order of their potential 
impacts on agency operations. OMB's guidance provides examples of how 
risks can be characterized by the likelihood of their occurrence and 
the severity of potential consequences and then ranked according to 
their importance.[Footnote 22] 

* Only CMS identified actions, before implementing seat management, to 
mitigate risks. NASA also identified actions to mitigate risks, but 
this analysis was conducted over a year after award of the ODIN master 
contract. The other two agencies did not identify planned mitigation 
actions. OMB's Capital Programming Guide states that agencies should 
determine how best to mitigate the impact of each risk that they 
identify.[Footnote 23] Developing preventive measures and 
countermeasures to successfully deal with problems as they develop is 
critical. 

Two of the agencies in our review, ATF and the Peace Corps, did not 
assess their seat management risks at all. According to ATF officials, 
the agency did not assess the risks of its seat management initiative 
because it was ordering equipment and services from the FSS schedule 
and chose to rely on maintaining a close relationship with its 
contractor to ensure quality services. However, this approach is not a 
substitute for a well-thought-out assessment of risks that includes 
mechanisms to mitigate identified risks. As for the Peace Corps, 
agency officials stated that they lacked the resources to conduct a 
risk assessment. 

Because seat management relies on the use of contractors to perform 
critical IT services, it is essential that the contract contain 
certain clauses that reduce the government's risks. In particular, the 
Federal Acquisition Regulation requires that contracting officers 
include in contracts, including seat management contracts, appropriate 
quality requirements. The type and extent of contract quality 
requirements needed depend on the particular acquisition and may range 
from inspection at the time of acceptance to requiring that the 
contractor implement a comprehensive program for controlling quality. 
In addition, seat management contracts, as government contracts, 
should include provisions for termination for the convenience of the 
government and termination for default. Finally, seat management 
contracts should include some provision for the disposition of 
contractor-supplied property (hardware and software) at the end of 
contract performance to ensure agency operations continue 
uninterrupted. 

All of the agencies' contracts adequately addressed the issues of 
quality assurance, termination rights, and rights to supplied hardware 
and software at the end of contract performance.[Footnote 24] In 
general, the contracts making use of the GSA and NASA master contracts—
the Peace Corps, Treasury's Departmental Offices, NASA, and CMS—tended 
to be more comprehensive in dealing with these issues than the 
contracts making use of GSA schedule contracts—DLA and ATE The GSA and 
NASA master contracts were more likely to include specific provisions, 
especially regarding the disposition of hardware and software, than 
the GSA schedule contracts, which more typically relied on standard 
government contract clauses. 

Lessons Learned Can Help Other Agencies More Effectively Implement 
Seat Management: 

By incorporating lessons learned from those who have implemented seat 
management, agencies considering this approach could more effectively 
plan their activities and reduce the risk that they will encounter 
problems experienced by others. Moreover, incorporating lessons 
learned from peers who have engaged in similar sourcing decisions is a 
leading commercial practice for acquiring IT services. The numerous 
lessons identified by agencies, their contractors, and private-sector 
research firms generally fall into six categories: (1) agency 
commitment, (2) preparation and planning, (3) solicitation and 
contract award, (4) program and contract management, (5) 
agency/contractor partnership, and (6) communication. These lessons 
are generally consistent with our report issued last year on the 
leading commercial practices for acquiring IT services.[Footnote 25] 
In addition, these lessons are often interrelated and build on one 
another. For example, if an agency's preparation and planning are 
lacking, developing an effective solicitation and contract award 
process could be difficult. 

Moving to outsourcing solutions such as seat management can involve a 
cultural change for government organizations because it may require a 
change to an agency's operating model, such as using a contractor to 
provide IT services using a performance-based contract. In the past, 
an agency's IT services may have been provided by government staff, in 
which case it is important that the agency consider the impact of 
outsourcing on these employees. In addition, the agency may not be 
familiar with outsourcing or the use of performance-based contracts, 
in which the customer agency specifies the outcome or result it 
desires and allows the vendor to decide how best to achieve the 
desired outcome. As a result, it is crucial that agencies demonstrate 
a commitment to change. Without such a commitment, success is more 
difficult to ensure. For example, according to the GSA Federal 
Technology Service's chief information officer (CIO), GSA's decision 
to implement seat management was driven, in part, by the desire to 
show the agency's support for its seat-management governmentwide 
acquisition contract.[Footnote 26] He asserted that GSA's seat 
management implementation did not succeed because the agency's culture 
did not support the change, and seat management was not implemented in 
a consistent manner throughout the agency. GSA's FSS CIO provided a 
similar assessment of the agency's seat management implementation. 

One way that an agency can demonstrate a commitment to seat management 
is through the involvement of top agency officials. Our wide-ranging 
work on federal management issues has shown that perhaps the single 
most important element of successful management improvement 
initiatives is the demonstrated commitment of top leaders to change. 
[Footnote 27] Moreover, our research has shown that executive 
leadership is a critical success factor for outsourcing IT services in 
the commercial world. The applicability of this principle to seat 
management was echoed by federal agencies and private-sector 
organizations alike. For example, (1) Treasury's seat management 
program manager stated that for seat management to succeed, agency 
executives need to understand and support it; (2) NASA's ODIN delivery 
order contracting officer representative at the Goddard Space Flight 
Center noted that outsourcing is extremely difficult unless there is a 
clear mandate from the top of the organization; (3) a NASA 
consultant's report stressed that unequivocal senior management 
support of ODIN is vital to its success; and (4) a member of the 
Industry Advisory Council's Managed Services Shared Interest Group 
[Footnote 28] stated that top agency officials, including the CIO, 
chief operating officer, and chief financial officer must "buy in" to 
seat management and be committed and involved. 

To address the culture change sometimes imposed on an agency 
implementing seat management, the Managed Services Shared Interest 
Group and GSA's Federal Technology Service noted that change 
management processes can help address this issue. In addition, our 
research of leading practices associated with the outsourcing of IT 
services provided examples of actions that entities can take to 
demonstrate executive leadership, namely the following: 

* keep the entire organization informed throughout the outsourcing 
initiative; 

* conduct regular peer-to-peer meetings at each level in the 
organization; 

* secure key executive support before eliciting organizational 
support; and; 

* establish a communications team to promote the idea of outsourcing. 

Preparation and Planning Are Critical: 

Success in implementing seat management rests in large part on 
thorough preparation and planning both before and after the contract 
is awarded. For preaward planning, it is vital that an organization 
optimize and baseline its internal IT processes to provide it with 
essential information to make a sourcing decision.[Footnote 29] 
Another important preaward planning process is developing a business 
case that evaluates the expected returns against the costs and focuses 
on strategic objectives that are agreed on by key stakeholders. 
Moreover, it is critical that agencies gather as much data as possible 
regarding, for example, inventories, transaction volumes, and service 
requirements. Finally, according to Gartner, the most positive desktop 
outsourcing outcomes should be expected when the outsourcing 
organization's desktop environment is already under control (including 
having good standards, established processes, and predictable service 
levels) and effort is expended to define and maintain an effective 
relationship between the affected parties.[Footnote 30] 

Without such preaward preparation and planning, agencies could 
encounter significant problems during implementation. For example, 
Peace Corps officials reported that the agency had file conversion 
problems during its seat management transition because it did not know 
the size and number of files involved and greatly underestimated the 
time that it would take to complete the conversion. Peace Corps seat-
management contractor officials estimated that solving the file 
conversion problem required the vendor to add five staff members over 
a 2-month period. One of the contractor officials added that the 
vendor did not pass these costs on to the Peace Corps because the 
vendor wanted to demonstrate its commitment to the contract. 

Postcontract award planning is equally important. In particular, 
agencies in our review, as well as private-sector organizations, 
[Footnote 31] repeatedly cited the importance of the transition period 
between awarding the contract and reaching a "steady state." In 
addition, GSA's seat-management program office noted that one of the 
most difficult aspects for both the agency and the contractor is 
learning the existing infrastructure during the transition period. 
Officials from two agencies stated that they should have allowed for 
more time to complete the transition period. One of these officials 
asserted that the agency should have doubled the amount of time for 
transition because the 3-month period that it had used did not allow 
adequate time to set up help-desk procedures or define how to monitor 
the network. The seat management contractor for this agency also 
acknowledged that the transition period was ill-defined and did not 
adequately lay out the parallel operations with the incumbent 
contractor. Continuous planning after the transition period is also 
important. For example, officials at Treasury and NASA noted the 
importance of planning ahead for technology refreshments and future IT 
requirements, respectively. 

Examples of practices that agencies and/or private-sector 
organizations implemented, or suggested should be implemented, in the 
planning area are as follows: 

* Solidly track and measure the existing IT organization. A leading 
research firm, the Giga Information Group, Inc., suggested that an 
entity employ solid tracking and measurement of its IT organization 
before outsourcing because only then would it be able to determine 
whether the arrangement has been successful.[Footnote 32] In addition, 
this firm noted that constructing payments and incentives accurately 
is extremely difficult, if not impossible, unless the entity has 
already established measurements for the functions being outsourced. 

* Consider using a total-cost-of-ownership study. Officials from the 
Managed Services Shared Interest Group, the Peace Corps, DLA, and GSA 
cited the value of total-cost-of-ownership studies, which evaluate the 
current distributed computing environment of the organization. For 
example, a member of the shared interest group stated that meaningful 
data must be collected to put the agency's current environment and its 
associated costs in perspective. Officials from DLA and the Peace 
Corps also suggested outsourcing the total-cost-of-ownership studies. 
For example, DLA officials believed that the agency's use of an 
objective consulting firm to help develop a total-cost-of-ownership 
study and other analyses worked well because the agency was able to 
take advantage of the firm's seat management expertise. 

* Develop a transition plan. Among the most important lessons learned 
identified by a GSA-supported, seat-management lessons-learned 
workshop was to develop a detailed project/transition plan. In 
addition, a leading commercial practice for the outsourcing of IT 
services is developing a transition plan, which can help make the move 
from internal to external IT services a smooth one. The clear 
definition of responsibilities and the careful consideration of 
employees' needs matched against the organization's needs enable both 
the client and the provider to focus on responsibilities that they can 
execute successfully.[Footnote 33] In addition, officials from DLA 
suggested that agencies establish a governance structure for 
monitoring contractor performance before the transition period begins. 

* Implement the program in phases. ATF and NASA's Office of Space 
Flight suggested that seat management be implemented in phases. For 
example, ATF's CIO touted the agency's development of an installation 
"cookbook" that was used as a repeatable process for other locations 
and refined as seat management was implemented at these other 
locations. NASA's Office of Space Flight suggested that an 
organization new to seat management begin with core requirements and 
add new services after the contractor has stabilized operations. 

Solicitation and Contract Award Should Be Carefully Managed: 

Agencies emphasized the importance of carefully managing the contract 
solicitation and award process. This involves using a solicitation and 
award process that (1) includes a solicitation document that is based 
on the goals and requirements of the agency; (2) encourages 
contractors to bid; (3) effectively evaluates the suitability of 
bidding contractors; (4) results in a winning vendor that best suits 
the organization's needs; and (5) produces a contract with fair 
pricing and with specific, relevant, and measurable performance 
requirements geared toward outcomes. In particular, at the beginning 
of the process, it is important that the agency understand its needs 
and convey this in the solicitation document. For example, according 
to Gartner, to develop valid pricing estimates, the contractor needs 
as much information as possible regarding hardware and software 
inventories, current transaction volumes, expected levels of service, 
and customization requirements.[Footnote 34] 

Gartner also encourages organizations to use the vendor selection 
process to gain an understanding of the experience of the bidding 
contractors and their relationships with subcontractors and to match 
the contractors' services to user requirements and the specific 
account team assigned.[Footnote 35] If this is not done, an agency may 
overestimate the contractor's experience and capabilities. For 
example, officials from one agency told us that because the winning 
bidder was a "world class organization," it assumed that the 
contractor had "world class" expertise in areas such as testing. The 
agency later found that this was not always the case. 

Once the winning bidder is chosen, it is important to negotiate a 
contract that provides a solid foundation for the working relationship 
between the agency and its contractor by setting the expectations of 
service levels, delivery of essential services, and continuous 
improvement. This includes establishing appropriate performance 
metrics to measure and reward or penalize contractor performance. 
Defining appropriate metrics and related penalties and incentives can 
be challenging. For example, NASA and Treasury have reassessed or are 
in the process of reassessing the performance metrics in their seat 
management contracts because of concerns regarding their adequacy. In 
particular, four NASA organizations reported that the metrics in the 
ODIN contract were inadequate, incomplete, and/or did not provide 
sufficient incentive to the contractor. One organization stated that 
the metrics were complex and cumbersome, whereas another noted that 
metrics applicable to e-mail and other enterprise services were not 
adequate to ensure acceptable service performance and adequate 
capacity for service growth. NASA has recognized these problems and 
has made changes in its follow-on seat management contracts pertaining 
to the payment provisions associated with achieving the performance 
metrics. However, the private-sector firm that performed a 
postimplementation review of the NASA ODIN program recommended that 
the agency take additional action, including initiating a full review 
of the metrics in the contract. 

The following are examples of practices that agencies and/or private-
sector organizations implemented, or suggested should be implemented, 
in the solicitation and contract award area: 

* Provide potential bidders with critical requirements information. 
Agencies and private-sector organizations made various suggestions 
related to providing information to prospective bidders. For example, 
the Peace Corps, DLA, and a contractor stated that issuing draft 
requirements to obtain comments and questions proved useful. Moreover, 
this contractor stated that the agency's decision to provide the 
service levels that specified the required levels and types of service 
to the bidders before awarding the contract allowed them to structure 
and price their offers to satisfy these levels. In another case, the 
Peace Corps and its contractor both agreed that the preaward site 
visits that the agency sponsored helped potential vendors better 
understand the environment in which they would be working. 

* Consider requiring oral presentations. DLA and the Peace Corps noted 
that requiring competing contractors to provide oral presentations 
clarified the bidders' written material and allowed the agency to 
better understand the bidders' management approach, respectively. DLA 
also videotaped these sessions, which it later used to confirm 
contractor representations. 

* Use relevant performance metrics and related incentives and 
penalties in contracts. Agencies and private-sector organizations 
alike reported that the structuring of the performance metrics and 
related incentives and penalties in the contract was critical. For 
example, the Managed Services Shared Interest Group stated that the 
performance requirements should define the work in measurable, mission-
related terms and that the performance standards should be tied to the 
requirements. The Giga Information Group provided the following 
examples of areas in which bonus payments can be structured in desktop 
outsourcing: (1) procurement, in which the contractor is provided 
incentives to reduce the purchase price of equipment that are tied to 
its overall repair history to avoid situations in which inferior 
equipment is chosen; (2) uptime or meantime between failure, in which 
incentive payments are tied directly to service costs or service 
quality; and (3) service-level improvement, in which incentives are 
used to motivate the contractor to invest in methodologies, 
technologies, and processes that improve the services provided to the 
end user without substantially increasing the associated costs. 
[Footnote 36] In addition, the Giga Information Group noted that 
companies should consider penalties structured around these same 
areas. Our November 2001 report on the outsourcing of IT services also 
provides specific examples of leading commercial practices related to 
performance requirements.[Footnote 37] 

* Consider incentives related to the transition period. The master 
ODIN contract provides for the payment of bonuses related to 
contractor performance during the transition period. Officials from 
another agency that did not include transition bonuses in its seat 
management contract stated that they wished they had included an 
incentive related to the transition period in its contract. 

Strong Program and Contract Management Is Key to Success: 

Since the agency is ultimately responsible for ensuring that services 
are provided and users' needs are met, a key to the successful 
implementation of seat management is program and contract management. 
While much of this responsibility is defined by the terms of the 
contract, market conditions may change, and an arrangement that was 
once advantageous may become less so over time. Therefore, it is 
important for an organization to monitor service levels internally as 
well as to maintain an external view of the performance of other 
providers in its peer group. Gartner recommends having a person or 
group manage the contractor in a way that fits seamlessly into the 
overall IT service model.[Footnote 38] Specifically, according to 
Gartner, designated staff members, including representatives from IT 
operations, business units, and procurement, need to be assigned to 
manage the contractor and establish service-level metrics. The need 
for dedicated federal personnel to manage the program and contractor 
was also cited as important by NASA, GSA, ATF, and DLA officials. Also 
important is that an agency implement a process to validate the 
information being provided by the contractor to ensure that it is 
accurate. For example, the contracting officer for the Department of 
State's seat management contract stated that she rejected an inventory 
report issued by the agency's contractor because it contained names of 
nonexistent users and duplicate records. 

The necessity for strong program and contract management was 
demonstrated by GSA's decision not to continue with its seat 
management implementation. In this case, according to the Federal 
Technology Service's CIO, GSA attempted to manage its seat management 
program centrally while also allowing each of its services and field 
locations to develop different service-level agreements and define 
drastically different desktop requirements. As a result, according to 
GSA and contractor officials, the contract terms did not match the 
actual agency implementation. In addition, GSA did not adhere to the 
implementation schedule or the service-level mix it had agreed upon 
with its contractor. Moreover, even though the contractor was 
responsible for asset management under the GSA seat management 
contract, the agency's Federal Technology Service's CIO stated that 
GSA had difficulty obtaining accurate information from the contractor 
in terms of the number, type, and costs of these assets when arranging 
to transfer ownership of these assets from the contractor to GSA. 
According to the project manager for the GSA contractor, this was due, 
at least in part, to actions by users, such as moving equipment 
without authorization. 

Examples of practices that agencies and/or private-sector 
organizations implemented, or suggested should be implemented, in the 
program and contract management area are as follows: 

* Benchmark contractor performance. NASA and CMS used an independent 
contractor to perform market surveys quarterly or biannually and to 
test original equipment manufacturers' products in order to benchmark 
the IT equipment proposed by the seat management contractor. In 
addition, DLA plans to annually benchmark its users' satisfaction 
level through the use of surveys. Agency use of benchmarking is 
consistent with our research of leading commercial practices for 
outsourcing of IT services, which found that periodic benchmarking 
allows an organization to ascertain whether it is still obtaining good 
value from its provider. 

* Use service-level agreements. The use of service-level agreements 
was cited as a critical practice by both agencies and contractors. For 
example, a contractor noted that its service-level agreements with 
Treasury's Departmental Offices were a critical success factor because 
they established a specific understanding between the contractor and 
the agency in which customer expectations were realistic and in 
concert with the IT and support services the contractor must deliver. 
In addition, Treasury, two contractors, and the Managed Services 
Shared Interest Group noted that it is important to periodically 
review the service-level agreements to ensure that they are still 
appropriate. 

* Ensure that services are provided. In January 2001, NASA's Office of 
Space Flight issued a plan outlining a strategy for managing its four 
ODIN contracts, including (1) observing contractor processes and 
procedures; (2) sampling items for review (such as desktop hardware 
and software and trouble tickets); (3) conducting audits using 
checklists; and (4) assessing contractor-generated data (such as asset 
tracking and performance-related data). 

Agency/Contractor Partnerships Are Vital to Success: 

Agencies and contractors often noted that it is vital that seat 
management be approached as a partnership, which can be realized by 
working to establish and achieve common goals. In addition, they cited 
the importance of establishing an environment of mutual trust so that 
issues and potential conflicts can be resolved more easily. For 
example, CMS emphasized the importance of an agency's developing a 
strong working relationship with its seat management contractor that 
involves mutual trust, noting that flexibility and shared goals are 
keys to success. The need for agency/contractor partnerships is 
consistent with our research of commercial practices, which found that 
aligning client and provider objectives in a partnership is key to 
building consensus and is imperative to establishing early trust among 
all stakeholders. For this alignment to occur, the client and provider 
must work together to establish common project goals beyond objectives 
stated in the request for proposal. Both sides must recognize and 
understand each other's underlying motives and strive to achieve 
established expectations. Developing a productive agency/contractor 
relationship is not always easy. For example, even though the ODIN 
master contract has been in place since mid-1998, in November 2001, a 
NASA consultant noted that a true partnership between the agency and 
its contractors had not been realized, although some progress had been 
made. NASA has since established a working group that comprises both 
agency and contractor representatives to address this matter. 

The following are examples of practices that agencies and/or private-
sector organizations implemented, or suggested should be implemented, 
to develop and nurture agency/contractor partnerships: 

* Develop a trusting relationship with the seat management contractor. 
A NASA consultant's recommendation to the Goddard Space Flight Center 
was that the center and its vendor build institutional and personal 
relationships and develop a formal conflict-resolution process using 
government and contractor staff. In responding to a draft of this 
report, NASA noted that the Goddard Space Flight Center had 
established a partnering arrangement with its seat management 
contractor and developed a conflict resolution process. 

* Consider including the contractor in agency IT planning. One agency 
suggested including the contractor in strategic and tactical planning, 
such as enabling the contractor to provide input into future standards 
and policies. 

* Hold agency/contractor meetings. A seat management contractor 
recommended the approach taken by one agency of holding periodic off-
site meetings to discuss issues and establish mutually agreed-upon 
priorities. 

Establishing Effective Communication Among Various Entities Is 
Critical: 

When the IT service provider is outside of the entity, as it is in the 
case of seat management, disconnects between organizations are more 
likely; thus, processes to facilitate good communication are critical. 
Indeed, most of the entities we reviewed cited communication among the 
seat management contractor, the agency program office, users, and/or 
other agency contractors as critical to the success of seat 
management. For example, (1) the Managed Services Shared Interest 
Group stated that stakeholders and executives should be kept informed 
of progress "early and often," (2) various NASA centers and two 
contractors emphasized the need to provide proactive user outreach and 
manage user expectations, and (3) three agencies and one contractor 
noted the importance of communication among the various contractors 
that an agency might use for related services. Continuous 
communication throughout the seat-management life cycle is also 
important. For example, according to Gartner, communication is a 
critical part of both the evaluation and implementation processes of 
seat management.[Footnote 39] 

Without effective communication, an agency's seat management 
initiative can encounter problems. For example, GSA's Federal 
Technology Service's CIO noted that the agency did not take enough 
steps to market the seat management program throughout the agency; 
consequently, users of desktop services neither understood nor liked 
the changes that this approach entailed. Inadequate user outreach was 
also cited as a problem at NASA. For example, a consultant to the 
Goddard Space Flight Center noted that customer satisfaction requires 
a high degree of openness and cooperation among ODIN representatives, 
the ODIN project office, and the vendor, which was not occurring. 
Indeed, the consultant found that among users there was a perception 
that "no one listens or cares." 

Examples of practices that agencies and/or private-sector 
organizations implemented, or suggested should be implemented, in the 
communications area are as follows: 

* Market seat management within the agency. Various approaches to 
marketing seat management to users were cited, including the 
sponsorship of "town hall meetings" and technology days or the 
distribution of written materials. In terms of the type of information 
to be provided to users, GSA's seat management program office noted 
that it is important that users be educated about the benefits of seat 
management, the changes they may expect, and the procedures for using 
the new service. 

* Consider the seat management contractor's relationship to other 
service providers. A NASA consultant recommended that the Goddard 
Space Flight Center identify all service providers that interface with 
or whose activities or responsibilities overlap with the seat 
management contractor and define how they will work together. 
Similarly, four agencies, a contractor, and the participants in a GSA-
supported seat-management lessons learned workshop suggested that the 
seat management contractor establish formal agreements with other 
agency contractors. The contractor noted that such an agreement is 
especially important when the other agency contractor can affect the 
seat management vendor's ability to meet the service-level agreements. 
In addition, within about 3 months of contract award, CMS signed a 
memorandum of understanding with its seat management contractor and 
facilities contractor that set forth the responsibilities of all three 
organizations. 

Conclusions: 

The agencies we reviewed implemented seat management for a variety of 
reasons, including to (1) improve their IT management, (2) improve end-
user support and productivity, and (3) obtain new or upgrade current 
technology. In addition, these agencies reported a variety of 
accomplishments resulting from implementing seat management, such as 
improved asset management and end-user support. However, they have not 
performed the analyses necessary to validate the overall results of 
this approach. Specifically, the agencies performed limited or, in 
some cases, no analyses of costs and benefits before implementing seat 
management and have not routinely monitored all actual costs or 
benefits. As a result, these agencies lack vital data to demonstrate 
actual investment results. Without these data, it is difficult to 
determine whether the benefits of seat management outweigh its costs 
and risks. Moreover, this lack of monitoring could impair the 
agencies' ability to justify and implement future seat management 
investments. These agencies, and others considering future seat 
management investments, could benefit from the myriad lessons learned 
by organizations that have implemented seat management, such as the 
need for thorough preparation and planning, agency commitment, program 
and contract management, and continual communication. By applying the 
lessons learned in these critical areas and others, agencies 
considering seat management could more effectively plan their 
activities and reduce the risks associated with implementing such a 
relatively new concept. 

Recommendations: 

To determine to what extent their current seat management programs 
have achieved positive results, we recommend that the secretary of the 
treasury; administrators for the National Aeronautics and Space 
Administration and Centers for Medicare and Medicaid Services; and 
directors of the Peace Corps, Bureau of Alcohol, Tobacco and Firearms, 
and Defense Logistics Agency each routinely monitor all actual seat 
management costs and benefits. 

To provide for adequate justification of any future seat management 
investments, we recommend that the secretary of the treasury; 
administrators for the National Aeronautics and Space Administration 
and Centers for Medicare and Medicaid Services; and directors of the 
Peace Corps, Bureau of Alcohol, Tobacco and Firearms, and Defense 
Logistics Agency each ensure that existing federal policy and guidance 
for information technology investments be followed when considering 
investments in information-technology-service outsourcing. 
Specifically, for future seat management investments, we recommend 
that these agencies: 

* baseline the current costs of the service being outsourced, 
including the cost of internal agency operations; 

* perform an analysis of expected costs and benefits; 

* perform an analysis of risks, including developing plans to mitigate 
risks identified; 

* monitor actual costs and benefits as a basis for results 
accountability; and; 

* implement, to the extent feasible, the lessons learned that were 
identified in this report. 

Agency Comments and Our Evaluation: 

We received written comments on a draft of this report from Treasury's 
Departmental Offices and ATF, the Peace Corps, the Department of 
Defense, GSA, and NASA. Three agencies agreed with the findings or 
recommendations in the report, two did not indicate whether they 
agreed or disagreed, and NASA supported many of the findings but 
disagreed with portions of the report. We also requested comments from 
the Centers for Medicare and Medicaid Services, but none were provided. 

The comments provided by Treasury's Departmental Offices and ATF, the 
Peace Corps, Defense, and GSA varied in scope and detail. Specifically, 

* Treasury's acting director, Customer Service Infrastructure and 
Operations, Office of the Chief Information Officer, stated that 
Treasury's Departmental Offices has implemented, or is in the process 
of implementing, our recommendations. Treasury also offered clarifying 
comments that we incorporated into the report, as appropriate. The 
comments from Treasury are reproduced in appendix IV. 

* The director of ATF stated that its Office of Science and Technology 
had reviewed the report and had no comments at this time. ATF's 
written response is reproduced in appendix V. 

* The director of the Peace Corps did not address whether the agency 
agreed or disagreed with the findings or recommendations in the 
report, but offered clarifying comments that we incorporated into the 
report, as appropriate. The comments from the Peace Corps are 
reproduced in appendix VI. 

* Defense's deputy assistant secretary of defense (deputy CIO) stated 
that the department generally concurs with the recommendations in the 
report. Defense also included technical corrections that we 
incorporated into the report, as appropriate. The comments from 
Defense are reproduced in appendix VII. 

* GSA's chief information officer concurred with the findings in the 
report. The comments from GSA are reproduced in appendix VIII. 

Although NASA supported many of the findings, it disagreed with 
portions of the report. Specifically, NASA did not agree with our 
assessment that (1) its up-front cost analysis was not sufficient; (2) 
it did not track its internal seat management costs, citing its 
tracking of the full-time-equivalents associated with seat management; 
and (3) it did not adequately track benefits. 

We disagree with these NASA comments. First, the problems with NASA's 
up-front cost analysis were cited by the contractor the agency 
employed to conduct a postimplementation review. The report prepared 
by the contractor asserted that it was impossible to determine whether 
the agency is saving money with seat management because of the lack of 
a comprehensive pre-seat-management baseline. Second, while important, 
monitoring of full-time-equivalents does not provide the agency with a 
complete picture of internal costs associated with the implementation 
of seat management. As NASA officials acknowledged during the exit 
conference, the agency does not track the full costs of seat 
management, which would include internal cost items such as overhead 
and salaries and benefits. Finally, NASA's efforts to track program 
benefits are not complete. Specifically, while NASA's quarterly 
reports and postimplementation review address some of the agency's 
expected seat management benefits, other expected benefits, such as 
potential improved staff productivity and efficiency, were not 
addressed. NASA also provided technical comments that we have 
incorporated in this report, as appropriate. NASA's written comments, 
along with our responses, are reproduced in appendix IX. 

As agreed with your office, unless you publicly announce its contents 
earlier, we plan no further distribution of this report for 30 days 
from the date of this letter. At that time, we will send copies to the 
chairman and ranking minority member, Senate Committee on Governmental 
Affairs; chairman and ranking minority member, House Committee on 
Government Reform, ranking minority member, Subcommittee on Technology 
and Procurement Policy, House Committee on Government Reform; and 
other interested congressional committees. We are also sending copies 
to the secretary of the treasury; administrators for the National 
Aeronautics and Space Administration and Centers for Medicare and 
Medicaid Services; and directors of the Peace Corps, Bureau of 
Alcohol, Tobacco and Firearms, Defense Logistics Agency, and the 
Office of Management and Budget; and other interested parties. We will 
also make copies available to others upon request. 

If you have any questions on matters discussed in this report, please 
contact me at (202) 512-6257 or by e-mail at mcclured@gao.gov. Other 
contacts and key contributors to this report are listed in appendix X. 

Sincerely yours, 

Signed by: 

David L. McClure: 
Director, Information Technology Management: 

[End of section] 

Appendix I: Information on Seat Management Contracts as of December 
31, 2001, Awarded under the NASA ODIN Master Contract: 

Dollars in thousands: 

Agency/Location: National Aeronautics and Space Administration's 
(NASA)/Ames Research Center; 
Services identified in the contracts[A]: Application/Database, file 
storage, general-purpose desktop, maintenance only, scientific and 
engineering desktop, and World Wide Web seats; 
Reported estimated contract amount[B]: $16,014; 
Estimated benefits: Estimated benefits are the same for all contracts 
awarded by NASA's centers and are described in the original business 
case for Outsourcing Desktop Initiative for NASA (ODIN), as follows: 
* divesting the day-to-day management of noncore information 
technology (IT) functions;
* improved management of assets and their configuration;
* improved technology refreshment;
* improved interoperability;
* improved standardization;
* a consistent agencywide desktop strategy;
* simplified procurement, contractor, and budget management processes;
* transfer risk from the government to the commercial sector;
* potential increases in user efficiency and productivity;
* better use of civil service personnel previously employed in desktop 
support; and
* access to greater information technology expertise. 

Agency/Location: NASA/Dryden Research Center; 	
Services identified in the contracts[A]: Application/Database, file 
storage, general-purpose desktop, maintenance only, scientific and 
engineering desktop, and World Wide Web seats; 
Reported estimated contract amount[B]: $21,216; 
Estimated benefits: Same as above. 

Agency/Location: NASA/Glenn Research Center; 	
Services identified in the contracts[A]: Application/Database, file 
storage, general-purpose desktop, maintenance only, scientific and 
engineering desktop, and World Wide Web seats; 
Reported estimated contract amount[B]: $44,074; 
Estimated benefits: Same as above.
	
Agency/Location: NASA/Langley Research Center; 	
Services identified in the contracts[A]: Application/Database, 
cellular phone, facsimile, file storage, general-purpose desktop, 
local-area network, local video, maintenance only, network-attached 
device, remote communications, scientific and engineering desktop, 
telephone, and World Wide Web seats; 
Reported estimated contract amount[B]: $51,083; 
Estimated benefits: Same as above. 

Agency/Location: NASA/Goddard Space Flight Center; 
Services identified in the contracts[A]: Application/Database, 
facsimile, file storage, general-purpose desktop, local-area network, 
scientific and engineering computer, maintenance only, network 
attached device, and World Wide Web and remote communications seats; 
Reported estimated contract amount[B]: $20,005[C]; 
Estimated benefits: Same as above.

Agency/Location: NASA/Headquarters; 
Services identified in the contracts[A]: Administrative radio, 
application/database, cellular phone, computational server, facsimile, 
file storage, general-purpose desktop, local-area network, local 
video, maintenance only, meeting place conferencing, public address, 
remote communications, scientific and engineering desktop, telephone, 
and World Wide Web seats; 
Reported estimated contract amount[B]: $20,190 
Estimated benefits: Same as above.
	
Agency/Location: NASA/Johnson Space Center; 
Services identified in the contracts[A]: Application/Database, 
cellular phone, facsimile, general-purpose desktop, local video, 
remote communications, scientific and engineering desktop, telephone, 
and World Wide Web seats; 
Reported estimated contract amount[B]: $180,721; 
Estimated benefits: Same as above. 

Agency/Location: NASA/Kennedy Space Center; 
Services identified in the contracts[A]: Application/Database, 
cellular phone, facsimile, general-purpose desktop, local video, 
remote communications, scientific and engineering desktop, telephone, 
and World Wide Web seats; 
Reported estimated contract amount[B]: $65,170; 
Estimated benefits: Same as above. 
		
Agency/Location: NASA/Marshall Space Flight Center; 
Services identified in the contracts[A]: Application/Database, 
cellular phone, facsimile, general-purpose desktop, local video, 
remote communications, scientific and engineering desktop, telephone, 
and World Wide Web seats; 
Reported estimated contract amount[B]: $100,187; 
Estimated benefits: Same as above. 

Agency/Location: NASA/Stennis Space Center; 
Services identified in the contracts[A]: Application/Database, 
cellular phone, facsimile, general-purpose desktop, local video, 
remote communications, scientific and engineering desktop, telephone, 
and World Wide Web seats; 
Reported estimated contract amount[B]: $30,396; 
Estimated benefits: Same as above. 
	
Agency/Location: Centers for Medicare and Medicaid Services (CMS); 
Services identified in the contracts[A]: General-purpose desktop 
seats, asset management, and engineering services; 
Reported estimated contract amount[B]: $25,647; 
Estimated benefits: 
* Avoidance of costs incurred from theft or loss of desktop computing 
assets.
* Reduced costs of license compliance.
* Avoidance of costs for duplicate computer equipment.
* Cost savings over purchased equipment.
* Cost savings from consolidation of contracts.
* Improved user productivity.
* Improved responsiveness at the user level.
* Increased end-user satisfaction.
* Compliance with Year 2000 requirements.
* Improved mission effectiveness.
* Reduced risks associated with Year 2000 readiness.
* Improved morale.
* Redirected staffing responsibilities. 

[A] In addition to the individual categories of seat services acquired 
by each of the NASA implementing entities and CMS, the NASA ODIN 
contract allows for acquiring other hardware, software, and services 
through a catalog service offered by the seat management contractors 
as well as a special order process. 

[B] We asked the agencies to provide us with the total estimated costs 
through the completion date of the current delivery order. 

[C] The reported estimated contract amount for the Goddard Space 
Flight Center reflects the amount estimated through the end of its 
original delivery order. As of January 30, Goddard had not signed a 
follow-on delivery order. According to the Goddard delivery-order 
contracting officer, until this follow-on delivery order is signed, 
Goddard has agreed to pay the contractor the amount of the monthly 
invoice (generally about $600,000) plus 30 percent. 

Source: GAO, based on information provided by the agencies. We did not 
verify this information. 

[End of table] 

[End of section] 

Appendix II: Information on Seat Management Contracts as of December 
31, 2001, Awarded under the GSA Seat Management Master Contract: 

Dollars in thousands: 
Agency: Department of the Treasury’s Departmental Offices; 
Services identified in the contracts[A]: Asset management, commercial 
information services management, desktop/portable computers, help 
desk, local- and wide-area network
servers, network printers, and voice/telephone administration; 
Reported estimated contract amount[B]: $114,732; 
Estimated benefits: 
* Ensure the continued reliability and availability of existing and
planned IT systems, equipment, services, and data. 
* Provide for access to private-sector technical expertise, staff
support, management experience, and corporate capabilities. 
* Acquire an operational IT platform that results in a positive
return on investment. 
* Provide for a single point of contact for IT operations.
* Increase user satisfaction with IT solutions. 

Agency: U.S. Air Force Special Operations Forces/Aeronautical System 
Center System Program Office at Wright-Patterson Air Force Base; 
Services identified in the contracts[A]: Asset management, 
desktop/portable computer, help desk, local- and wide-area network 
servers, network printing, peripheral device, and
software (including custom applications) services; 
Reported estimated contract amount[B]: $7,266; 
Estimated benefits: 
* Ability to establish and sustain effective and efficient life-cycle
support for IT operations. 
* Single point of contact. 
* Access to private-sector expertise regarding the latest available 
technologies.
* Migration to a self-contained network.
* Paperless acquisition office environment.
* Cost savings from potential future expansion of seat management.
* Ability of staff to focus efforts on mission needs. 

Agency: Department of State/Office of Foreign Building Operations; 
Services identified in the contracts[A]: Asset management, classified 
network, desktop/portable computers, local-area-network servers, and 
network printers; 
Reported estimated contract amount[B]: $48,139; 
Estimated benefits: 
* Ability to establish and sustain effective and efficient managed
life-cycle support of the office’s distributed computing environment 
with a single point of contact. 

Agency: Federal Highway Administration; 
Services identified in the contracts[A]: Asset management, 
desktop/portable computer, help desk, local-area-network server, 
network printing, programming, software/software
maintenance, and infrastructure management services; 
Reported estimated contract amount[B]: $150,000; 
Estimated benefits: 
* Ability to establish and sustain effective and efficient managed
life-cycle support of a pilot project on distributed computing
with a single point of contact.
* Achieving high-quality, reliable, and responsive levels of service.
* More easily achieving agencywide technology transfers through an 
integrated and uniform processing environment.
* Obtaining technology upgrades at regular intervals.
* Reduced asset maintenance and management responsibilities.
* Achieving greater contracting efficiency.
* Consolidation of diverse product and support service requirements.
* More efficient use of resources resulting from a consolidated 
internal infrastructure and support environment. 

Agency: U.S. Army/Center for Substance Abuse Programs; 
Services identified in the contracts[A]: Asset management, 
desktop/portable computer, help desk, Internet, local-area-network 
servers, network printing, and World Wide Web
management services; 
Reported estimated contract amount[B]: $3,058; 
Estimated benefits: 
* Implementation of comprehensive and integrated desktop and
local-area-network management services.
* Freeing agency personnel from the burden of maintaining and
updating desktop resources, services, and training so that they
can refocus their efforts on the core mission.
* Having a single point of contact.
* Achieving the goal of acquiring distributed computing services
seamlessly and efficiently. 

Agency: Department of Housing and Urban Development/Office of 
Inspector General; 
Services identified in the contracts[A]: Asset management, 
desktop/portable computers, help desk, local- and wide-area-network 
servers, network printers, and virtual private network
services; 
Reported estimated contract amount[B]: $48,300; 
Estimated benefits: 
* Obtain a secure network.
* Acquire applications to support the agency’s audit and investigation 
missions.
* Easier technology transfers through an integrated and uniform 
processing environment.
* Technology upgrades at regular intervals so the agency can benefit 
from new and improved technological improvements.
* Freedom from maintaining and managing capital assets and refocusing 
staff on the office’s mission.
* Greater contracting efficiency through consolidation of diverse
requirements.
* More efficient use of resources resulting from a consolidated 
internal infrastructure and support environment. 

Agency: Peace Corps; 
Services identified in the contracts[A]: Asset management, 
desktop/portable computer, help desk, local- and wide-area-network 
servers, network printing, off-site storage, and training services; 
Reported estimated contract amount[B]: $30,045; 
Estimated benefits: 
* Migration to a different technology environment.
* Easier technology transfers through an integrated and uniform 
processing environment.
* Technology upgrades at regular intervals.
* Freedom from having to maintain and manage capital assets.
* Greater contracting efficiency and a decrease in contract management 
functions by consolidating diverse product and support service 
requirements.
* More efficient use of resources through a consolidated internal 
infrastructure and support environment. 

Agency: Nuclear Regulatory Commission; 
Services identified in the contracts[A]: Integrated infrastructure 
management, asset management, help desk, maintenance, 
development/integration, and contingency operations; 
Reported estimated contract amount[B]: $80,288; 
Estimated benefits: 
* Desktop and network refreshment to reduce maintenance costs and 
maintain a high level of service availability.
* Streamlined integrated desktop refreshment strategy.
* A single focal point for distributed computing support.
* Obtain a single operating system.
* Enhanced help-desk support through access to subject-matter experts.
* Defined and managed level of service delivery.
* High level of customer satisfaction.
* Quicker and more efficient access for the purchase of peripheral 
equipment and software through the use of an online catalog.
* Cost avoidance. 

Note: Two other organizations—the General Services Administration 
(GSA) and the District of Columbia Housing Authority—issued task 
orders under the GSA seat-management master contract, which they let 
expire. 

[A] In addition to the bundled seat management services shown, the GSA 
seat-management master contract allows agencies to purchase catalog 
orders. 

[B] We asked the agencies to provide us with the total estimated costs 
through the completion date of the current task order. Accordingly, 
the amounts provided assumed that all option years would be exercised. 

Source: GAO, based on information provided by the agencies. We did not 
verify this information. 

[End of table] 

[End of section] 

Appendix III: Scope and Methodology: 

To evaluate agencies' approaches to the seat-management outsourcing 
concept, we selected six agencies for review. Specifically, as agreed 
with your office, we chose two agencies each that were using the 
General Services Administration's (GSA) seat management contract, 
NASA's ODIN contract, and GSA's Federal Supply Service (FSS) Schedule 
70 contracts. In selecting the agencies, we chose those whose seat 
management contract had been awarded more than 1 year before our 
review to ensure that the agencies had time to adjust to the use of 
seat management and whose contract included at least 500 seats. 
Accordingly, we selected the Department of the Treasury's Departmental 
Offices and the Peace Corps, which use the GSA seat-management master 
contract; NASA and CMS, which use the NASA ODIN master contract; and 
the Defense Logistics Agency (DLA) and Treasury's Bureau of Alcohol, 
Tobacco and Firearms (ATF), which use GSA's FSS Schedule 70 contracts. 

To determine these agencies' rationales for using the seat management 
alternative, whether they achieved estimated costs and benefits, and 
how well they managed associated risks, we reviewed and analyzed 
documents related to each agency's decision to implement seat 
management and how it managed the program. For example, we reviewed 
business cases; program and implementation plans; and analyses of 
costs, benefits, and risks. We also assessed whether these documents 
followed Office of Management and Budget (OMB)[Footnote 40] and our 
[Footnote 41] guidelines. In addition, to determine whether the six 
agencies had implemented contracts that protect the government, we 
evaluated their seat management contracts to determine whether they 
adequately addressed the issues of (1) quality assurance, (2) 
termination rights, and (3) rights to supplied hardware and software 
at the end of contract performance. We also interviewed agency 
officials, including chief information officers (CIO) and seat-
management program and contract officials. 

To identify lessons learned, we reviewed applicable documentation and 
interviewed officials from the six agencies and their contractors, and 
reviewed applicable external and internal studies. We also conducted 
interviews with GSA officials, including the agency CIO and the CIOs 
for FSS and the Federal Technology Service, and the program manager 
for the governmentwide seat management contract. We also reviewed 
documents relating to GSA's decision not to exercise the option to 
extend its seat management contract. In addition, we obtained lessons 
learned from other organizations, such as the Industry Advisory 
Council's Managed Services Shared Interest Group and leading private 
research firms Gartner, Inc., and the Giga Information Group, Inc. 

To identify the agencies using the GSA and NASA governmentwide seat 
management contracts, we obtained a list of these agencies from GSA 
and NASA. We contacted each of these agencies to obtain the services 
being provided, the estimated contract amounts, and estimated 
benefits. We also reviewed contracts and other documents provided by 
each of the agencies but did not verify the data provided. 

We performed our work at the headquarters offices of ATF, GSA, NASA, 
the Peace Corps, and the Department of the Treasury located in 
Washington, D.C.; at DLA headquarters in Ft. Belvoir, Va.; at CMS 
headquarters in Baltimore, Md.; at the GSA offices located in Falls 
Church, Va.; and at NASA's Kennedy Space Center in Cape Canaveral, 
Fla. We conducted our review between April 2001 and January 2002 in 
accordance with generally accepted government auditing standards. 

[End of section] 

Appendix IV: Comments from the Department of the Treasury's 
Departmental Offices: 

Department Of The Treasury: 
Washington, D.C. 20220: 

February 28, 2002: 

Mr. David L. McClure: 
Director: 
Information Technology Management Issues: 
General Accounting Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Mr. McClure: 

The Department of the Treasury, Departmental Offices (DO), is pleased 
to provide an agency response and comments to the General Accounting 
Office (GAO) draft report, GAO-02-329, entitled, "Information 
Technology: Desktop Outsourcing Management Approaches, Costs, 
Benefits, and Risks Vary." 

If you have any further questions on DO's response, please contact 
Rory Schultz on (202) 622-2829, or via e-mail at 
rory.schultz@do.treas.gov. 

Sincerely, 

Signed by: 

Patrick N. Hargett: 
Acting Director: 
Customer Service Infrastructure and Operations: 
Office of the Chief Information Officer: 

Enclosure: 

[End of letter] 

Department of the Treasury Departmental Offices: 
Response/Comments to GAO-02-329: 

The Department of the Treasury, Departmental Offices (DO), has 
reviewed the Draft Report of the General Accounting Office (GAO) 
regarding Desktop Outsourcing Management and offers the following 
response to GAO's recommendations: 

Treasury DO has been one of the leaders in the areas of outsourcing 
desktop management, specifically in using the General Services 
Administration (GSA) Seat Management Government Wide Acquisition 
Contract (GWAC). After GSA, Treasury DO was the next agency to
successfully implement a Seat Management contract in June 1999. 
Treasury DO continues to refine its contract and service levels 
provided to our customers, as well as add services to the base 
contract. 

The GAO report on outsourcing desktop management includes specific 
recommendations which are quoted below: 

"To determine to what extent their current seat management programs 
have achieved positive results, we recommend that the Secretary of the 
Treasury, administrators for the National Aeronautics and Space 
Administration and Center for Medicare and Medicaid Services, and 
directors of the Peace Corps, Bureau of Alcohol, Tobacco, and 
Firearms, and Defense Logistics Agency each routinely monitor all 
actual seat management costs and benefits. 

"To provide for adequate justification of any future seat management 
investments, we recommend that the Secretary of the Treasury, 
administrators for the National Aeronautics and Space Administration 
and Center for Medicare and Medicaid Services, and directors of the 
Peace Corps, Bureau of Alcohol, Tobacco, and Firearms, and Defense 
Logistics Agency each ensure that existing federal policy and guidance 
for information technology investments be followed when considering 
investments in information technology-service outsourcing. 
Specifically we recommend that these agencies: 

* Baseline the current costs of the service being outsourced, 
including the cost of internal agency operations; 

* Perform an analysis of expected costs and benefits; 

* Perform an analysis of risks, including developing plans to mitigate 
risks identified; 

* Monitor actual costs and benefits as a basis for results 
accountability; and; 

* Implement, to the extent feasible, the lessons learned identified in 
this report." 

A number of these recommendations have been addressed by Treasury DO 
during its ongoing process of self-review and evaluation of the 
contract. In January 2002, we identified funding to perform a Total 
Cost of Ownership (TCO) study that includes an Outsourcing Financial 
Assessment (OFA), as well as a Distributed Computing Assessment (DCA). 
A contract was awarded to the Gartner Group, IT industry leader in 
research, and one of the sources cited by the GAO in its report as a 
contributor. Work began on this effort in February 2002 and is 
expected to continue through June 2002. 

As part of the TCO study, Treasury DO will receive a baseline of all 
seat costs, including "soft" or internal agency costs. Due to time 
constraints in awarding the original SEAT Management contract, we were 
not able to perform the TCO function prior to award. This current TCO 
study will allow Treasury DO to now establish a new baseline for all 
future requests for additional services and will allow us to compare 
our seat costs with similar organizations within government and 
private industry. Treasury DO intends to periodically review the TCO 
of the contract over its life cycle by awarding follow-on contracts to 
industry experts, such as Gartner. 

The OFA and DCA components of the TCO study will allow Treasury DO to 
predict expected costs of services for the future years of the 
contract. Such an analysis will allow us to also identify specific 
benefits to be derived from these projected costs, as well as review 
current costs and associated benefits derived therefrom. The periodic 
TCO reviews will allow us to keep these analyses current and in line 
with customer expectations. 

Treasury DO considered Seat Management to be a low risk endeavor prior 
to contract award. Experience over the past three years has verified 
that this assumption was correct. Nonetheless, we recognize the fact 
that an ongoing assessment of potential risks is crucial to 
maintaining a responsive contract and protecting the government's 
interests. The Seat Management Program Office (SMPO) has formalized 
contract review by conducting weekly meetings with contracting 
officials, the program office, and the contractor to ensure that risk 
remains at an acceptable level. In addition, the SMPO has a formalized 
monthly program review that includes senior representatives from all 
major policy offices, CIO management, and the Office of Information 
Systems Security (OISS). These vehicles allow us to continue to keep 
any identified risk low and acceptable. The SMPO is working closely 
with OISS to finalize a formal risk identification and mitigation 
strategy that will be included as part of a DO Information Systems 
Security Manual. 

Treasury DO agrees with the recommendation that actual costs and 
benefits should be monitored on a regular basis. The current 
methodology for this review is the weekly program office meeting and 
the monthly program review by senior management. In addition, we are 
reviewing other IT industry and government best practices, such as the 
balanced scorecard to improve our accountability. We will implement 
such best practices where applicable within our program. 

Treasury DO is one of the leaders in implementing lessons learned in 
the outsourcing environment. Many of the lessons learned cited in the 
GAO report are, in fact, a result of our experiences with Seat 
Management. We maintain regular contact with GSA and other agencies
that have implemented Seat Management and are continually applying 
lessons learned to our program to make it more responsive to our 
customers and accountable to our stakeholders. 

In summary, Treasury DO has implemented, or is in the process of 
implementing, all of GAO's recommendations for continued successful 
execution of our outsourcing contract. We are committed to a continual 
process of review and improvement of services to our customers. We 
remain one of the leaders in the federal government in implementing 
Seat Management. 

[End of section] 

Appendix V: Comments from the Bureau of Alcohol, Tobacco and Firearms: 

Department Of The Treasury: 
Director: 
Bureau Of Alcohol, Tobacco And Firearms: 
Washington, D.C. 20226: 

March 6, 2002: 
	
302000:KTS 7200: 

Mr. David L. McClure: 
Director: 
Information Technology Management Issues: 
United States General Accounting Office: 
441 G Street, N.W. 
Washington, DC 20548: 

Dear Mr. McClure: 

This is in response to your request for written or oral comments on 
the proposed report entitled "Information Technology: Desktop 
Outsourcing Management Approaches, Cost, Benefits, and Risk Vary" (GA0-
02-329). The Office of Science and Technology has reviewed the 
document and has no comments at this time. We appreciate the 
opportunity to review the draft report before it is issued in final 
form. 

If you have any questions or need additional information, please do 
not hesitate to contact Ms. Kimberly T. Sanchez, Division Systems 
Operations Officer at (202) 927-7870. 

Sincerely yours, 

Signed by: 

Bradley A. Buckles: 
Director: 

[End of section] 

Appendix VI: Comments from the Peace Corps: 

The Director Of The Peace Corps: 
Washington, D.C. 

March 1, 2002: 

Mr. David L. McClure: 
Director: 
Information Technology Management Issues: 
U.S. General Accounting Office: 
Washington, DC 20548: 

Dear Mr. McClure: 

Thank you for the opportunity to review and comment on the General 
Accounting Office's (GAO) draft report on Information Technology 
entitled: Desktop Outsourcing Management Approaches,
Costs, Benefits and Risks Vary. The Peace Corps welcomes the 
opportunity to contribute to this review of the government-wide 
approach to this relatively new mechanism for supporting a distributed 
computing environment. We hope that the feedback and exchange of 
information proves beneficial through sharing of lessons learned and 
improving the use of managed services. 

Our interest in providing this response is to ensure that the 
information in this study is complete and accurate in its depiction of 
the Peace Corps approach concerning the move to a managed services 
contract. Further, the Peace Corps is pleased to offer its perspective 
on the report's chief purpose, i.e., to answer the question "How well 
is seat management working?". In the case of the Peace Corps, we 
believe the answer to be "very well." The benefits have been very 
positive, with a high level of customer satisfaction and substantial 
gains in employee productivity. 

Accordingly, the following points reflect our clarifying comments on 
statements contained in the draft GAO report: 

1) Reference page 10: "In addition, Peace Corps officials stated that 
the agency did not choose seat management to reduce the agency's 
budget requests associated with distributed computing and did not 
expect that it would." 

The Peace Corps did not anticipate reduction in its budget associated 
with distributed computing because we saw seat management as an 
upgrade. We knew that buying more timely and comprehensive services 
and equipment would cost more in this one budget category. The Peace
Corps was certain, however, that there would be a reduction in soft 
costs and increases in productivity across the agency based on the 
assessment of these costs through a Total Cost of Ownership (TCO) 
Study. We believe this to be true. 

2) Reference page 14: "The agencies cited various reasons for not 
completing a thorough analysis of existing and expected costs and 
benefits of seat management and alternatives, including that they 
lacked time to complete the analysis...." 

The Peace Corps did not complete a more comprehensive analysis because 
it had insufficient fiscal and staff resources. The Peace Corps 
anticipated and defined requirements, assessed costs based on pre-
award research, and budgeted accordingly. Most importantly, the Peace 
Corps was certain of improved service delivery and reduction of soft 
costs relative to its existing environment. And, in fact, savings were 
realized. With additional resources, more analysis could have been 
accomplished either in-house or through outsourcing and the amount of 
savings could be better quantified and documented. 

3) Reference page 17: "Agency seat management contracts sometimes did 
not include contractor performance measures pertaining to all business 
goals of the seat management program. For example, one of the Peace 
Corps' performance goals in its September 2000 strategic plan is to 
pursue efforts to cut costs and improve agency productivity. Although 
the implementation of seat management is a part of the evaluation 
criteria associated with this goal, the agency's seat management 
performance metrics plan does not include a performance measure for 
cost reductions or improvements to agency productivity." 

Again, although the quantitative analysis has not been completed, we 
believe soft costs are down and productivity is up. The Peace Corps 
believes that performance metrics used to evaluate contractor 
performance do not inherently correlate with improved agency 
productivity and cost reductions. However, the performance metrics do 
speak to system availability and responsiveness of the help desk 
functions. Further, we believe that indicators relative to both agency 
productivity and cost reductions will be documented in our Customer 
Satisfaction Survey currently in progress and in the TCO study, which 
will be undertaken later this summer. 

4) Reference page 17: "For example, contractor performance metrics do 
not measure many of the types of benefits agencies believe that they 
are achieving or wanted to achieve, such as increased end-user 
productivity, improved mission support through transfer of agency 
staff to more critical tasks, and improved standardization." 

The Peace Corps acknowledges that the metrics do not speak to the 
agency strategic plan. However, implementation of seat management has 
afforded the Peace Corps improved mission support as follows: Staffing 
related to domestic infrastructure support was transferred to overseas 
IT support functions; the agency has standardized software and 
hardware; asset management is greatly improved; and under seat 
management IT security far exceeds that which existed in our previous 
environment. 

5) Reference page 21: "Two of the agencies in our review, ATF and the 
Peace Corps, did not assess their seat management risks at all... As 
for the Peace Corps, agency officials stated that they lacked the 
staff to conduct a risk assessment." 

The Peace Corps agrees that risk assessment is important. But even 
more important is the fact that seat management has reduced soft costs 
and increased productivity. The Peace Corps believes that in the 
course of discussions with the GAO we actually stated that we lacked 
the "resources" at the time of award to conduct a formal risk 
assessment. Resources have been planned for in our current strategic 
planning and budget process where risk management is integrated into 
the IT security plan. 

6) Reference page 25: "For example, Peace Corps officials reported 
that it had file conversion problems during its seat management 
transition because it did not know the size and number of files 
involved and greatly underestimated the time it would take to complete 
the conversion." 

The Peace Corps would like to clarify this statement. Because all 
files were stored locally on individual hard drives and not on the 
network it was difficult to fully capture the number and size of 
graphic, video, and audio files. We agree that while very time 
consuming and labor intensive, it would have been greatly beneficial 
to capture this information. 

In closing, let me take this opportunity to thank the GAO team for 
their desire to fully understand our unique operations and for their 
constructive comments, which will be helpful to the new Peace Corps 
management team. If you have any further questions regarding these 
comments please contact Judy Van Rest at (202) 692-1102. 

Sincerely, 

Signed by: 

Gaddi Vasquez: 
Director: 

[End of section] 

Appendix VII: Comments from the Department of Defense: 

Department Of Defense: 
Chief Information Officer: 
6000 Defense Pentagon: 
Washington, DC 20301-6000: 

March 7, 2002: 

Mr. David L. McClure: 
Director, Information Technology Management Issues: 
U.S. General Accounting Office: 
Washington, D.C. 20548: 

Dear Mr. McClure: 

This is the Department of Defense (DoD) response to the GAO draft 
report, "Information Technology: Desktop Outsourcing Management 
Approaches, Costs, Benefits, and Risks Vary" dated February 5, 2002 
(GAO Code 310414). 

The Department has reviewed the draft report as it addresses on going 
seat management efforts at the Headquarters, Defense Logistics Agency 
(DLA), and generally concurs with the recommendations. The 
Department's specific responses to the recommendations are found in 
Attachment 1. 

In addition, we request that your report be modified to reflect that 
your review specifically focused on implementation of seat management 
at DLA's Headquarters Complex in Fort Belvoir, VA. Without this 
change, it could be easily misconstrued that your review focused on 
seat management at the DLA enterprise level, rather than on this one 
segment of DLA. The suggested modifications are found in Attachment 2. 
Please note that DLA fully intends to integrate the improvements 
contained in your recommendations into its process of implementing 
seat management throughout DLA. 

We appreciate the opportunity to comment on your report. The 
Department's point of contact for this effort is Leo Milanowski, who 
can be reached at (703) 602-0980 X115 or by e-mail: 
leo.milanowski@osd.mil. 

Signed by: 

Priscilla E. Guthrie: 
Deputy Assistant Secretary of Defense (Deputy CIO): 

2 Attachments: As stated: 

[End of letter] 

Attachment 1: GAO Draft Report Dated February 5, 2002
(GAO Code 310414): 

"Information Technology: Desktop Outsourcing Management Approaches, 
Costs, Benefits, And Risks Vary" 

Department Of Defense Comments To The GAO Recommendations: 

Recommendation 1: The GAO recommended that the Defense Logistics Agency
routinely monitor all actual seat management costs and benefits to 
determine to what extent its current seat management programs have 
achieved positive results. (p. 36/GAO Draft Report) 

DLA Response: Concur — DOD has added information on internal costs 
specific to the seat management contract to the monthly review process 
that monitors contractor costs and performance. 

Recommendation 2: To provide for adequate justification of any future 
seat management investments, the GAO recommended that the Defense 
Logistics Agency ensure that existing federal policy and guidance for 
information technology investments be followed when considering 
investments in information-technology-service outsourcing. 
Specifically, for future seat management investments, the GAO 
recommended that the Defense Logistics Agency: 

* baseline the current costs of the service being outsourced, 
including the cost of internal agency operations; 

* perform an analysis of expected costs and benefits; 

* perform an analysis of risks, including developing plans to mitigate 
risks identified; 

* monitor actual costs and benefits as a basis for results 
accountability; and; 

* implement, to the extent feasible, the lessons learned identified in 
this GAO report. (pp. 3637/GAO Draft Report) 

DLA Response: Concur — DLA will continue to baseline all service costs 
when considering additional implementations of Seat Management. In 
addition, DLA will make an analysis of expected costs and benefits a 
part of the Seat Management expansion process. This process will
also include procedures for risk analysis and risk mitigation plan 
development, and the addition of data representing the actual costs 
and benefits of the new Seat Management implementation to
the monthly review currently used to monitor Agency IT performance. 
The expansion process will be completed prior to any further Seat 
Management implementations, and will include consideration of all 
lessons learned, including those identified in the GAO report. 

Attachment 2: GAO Draft Report Dated February 5, 2002 (GAO Code 
310414): 

"Information Technology: Desktop Outsourcing Management Approaches, 
Costs, Benefits, And Risks Vary" 

Specific technical recommendations are as follows: 

1. Page 9, Paragraph 3, Sentence 3. Recommend changing "...DLA 
implemented seat management to address the operational impairment of 
its users..." to "...DLA implemented seat management to address the 
operational impairment of its Headquarters Complex users..." 
Justification: The statement as written in the GAO report implies that 
seat management was implemented to address Agency-wide computing 
environment problems. The initial implementation of seat management 
within DLA effectively addressed problems that were limited to the DLA 
Headquarters Complex in Fort Belvoir, VA. The recommended change more 
accurately describes the problems as Headquarters Complex issues. 

2. Page 10, Paragraph 2, Sentence 3. Recommend changing "...the 
critical issue for their agency was improving its distributed 
computing environment, which among other problems had frequent 
downtimes..." to "...the critical issue for their agency was improving 
its Headquarters computing environment, which among other problems had 
frequent downtimes..." 
Justification: The statement as written in the GAO report implies that 
the critical issue involved problems and frequent downtimes across the 
Agency-wide distributed computing environment. Although seat 
management may indeed be used to improve the Agency-wide distributed 
computing environment with future implementations, the critical issue 
driving the initial implementation of seat management in the 
Headquarters Complex involved problems and frequent downtimes at that 
location. The recommended change clarifies the scope of the problems 
driving DLA's initial seat management implementation. 

3. Page 12, Paragraph 2, Sentence 3. Recommend changing "...before 
seat management, customer dissatisfaction with its IT help desk 
services was widespread..." to "...before seat management, customer 
dissatisfaction with its Headquarters IT help desk services was 
widespread..."
Justification: The statement as written in the GAO report implies that 
customers Agency-wide were dissatisfied with their IT help desk 
services. Although DLA may move to a single Agency-wide TT help desk 
in the future, the implementation of seat management addressed the 
dissatisfaction of Headquarters Complex TT help desk users. The 
recommended change clarifies the scope of the IT help desk 
dissatisfaction addressed by seat management. 

[End of section] 

Appendix VIII: Comments from the General Services Administration: 

GSA: 
GSA Office of the Chief Information Officer: 
U.S. General Services Administration: 
1800 F Street, NW
Washington, DC, 20405-0002: 
[hyperlink, http://www.gsa.gov] 
	
March 5, 2002: 

Mr. David L. McClure: 
Director, Information Technology Issues: 
General Accounting Office: 
441 G Street NW: 
Room 4075: 
Washington, DC 20548: 

Dear Mr. McClure: 

This letter is in response to your draft report titled, "Information 
Technology: Desktop Outsourcing Management Approaches, Costs, 
Benefits, and Risks Vary," GAO-02-329. The report was received in my 
office on Wednesday, February 6, 2002 via electronic mail. 

My staff has reviewed the draft report cited above. They have 
coordinated this response with the other principals relevant to this 
report. I concur with the findings as stipulated in your report. We do 
note one correction in spelling that should be made in the paragraph 
at the top of page 22 of the draft. Please correct the following 
sentence to read: "Finally, seat management contracts (not contacts) 
should...." 

Thank you for the opportunity to review and comment on this draft 
report. Should you or members of your staff have any questions, please 
contact L. Diane Savoy, Director, Office of Policy and Plans, on (202) 
501-3535. 

Sincerely, 

Signed by: 

Susan Chu, for: 

Michael W. Carleton: 
Chief Information Officer: 

[End of section] 

Appendix IX: Comments from the National Aeronautics and Space 
Administration: 

Note: GAO comments supplementing those in the report text appear at 
the end of this appendix. 

National Aeronautics and Space Administration: 
Office of the Administrator: 
Washington, DC 20546-0001: 
	
February 28, 2002: 

Mr. David L. McClure: 
Director, Information Technology Team: 
U.S. General Accounting Office: 
441 G Street, NW: 
Washington, DC 20548-0002: 

Dear Mr. McClure: 

This letter is in response to your request for comments on the GAO 
report GAO-02-329 dealing with desktop outsourcing management 
approaches, costs, benefits, and risks. As a pioneer in the 
outsourcing of desktop computing at NASA, we applaud GAO in their 
thoughtful analysis in this area. We support many of the findings in 
the report. After 3 years of outsourcing experience, NASA continues to 
believe that the benefits of outsourcing have met many of our original 
goals. 

NASA believes that GAO's finding that NASA did not perform sufficient 
upfront cost analysis is not accurate. NASA is further troubled by the 
methodology for evaluating effectiveness using cost as the main basis 
for GAO's conclusion. Base-lining the cost of NASA's Outsourcing 
Desktop Initiative for NASA (ODIN) seat management initiative was 
unusually difficult because of the very heterogeneous nature of the 
pre-ODIN desktop inventory. NASA's desktop environment is extremely 
diverse because of its various range of R&D missions. The conclusion 
by GAO that NASA did not perform a full cost analysis is incorrect. 
NASA performed a full business case analysis that included a cost 
analysis before the ODIN acquisition phase. A re-examination of 
achieved benefits from ODIN was performed 3 years after the 
acquisition phase and the analysis concluded that the current costs 
for ODIN seats are a good value compared to industry standards. [See 
comment 1] 

Another issue raised by GAO is that NASA has not been tracking its 
internal cost. This is incorrect. Besides ODIN services and contracts 
and procurement costs, NASA also tracks the number of internal FTE's 
associated with management of the various seat types. GAO was provided 
a copy NASA's submission to OMB of its Exhibit 300 Capital Asset 
Planning Report and the NASA Performance Plan which documents those 
internal FTE values. [See comment 2] 

GAO has the impression that NASA is not tracking benefits. Actually, 
NASA tracks benefits both qualitatively and quantitatively as shown in 
ODIN's post-implementation review. The CIO's quarterly status reports 
provide evidence of benefits tracking. [See comment 3] 

Detailed comments on the GAO report are as follows: 

1. Table 1, pages 7 and 8: For the KSC and CMS delivery orders, the 
comment, "Under the terms of the master NASA ODIN contract, each type 
of service is sold separately." is misleading. All the services 
required to support a seat are bundled in the fixed monthly seat 
price. These services include asset management, help desk services, 
software upgrades, network services including management, hardware 
refreshment, etc. Upgrades to the infrastructure are typically paid 
for as a separate line item as are catalog items, though the cost of 
providing a catalog is probably bundled into the seat prices. [See 
comment 4] 

2. Page 16, First full paragraph, first 2 sentences: The NASA ODIN 
Program Manager reports benefits realized and status to the NASA CIO 
quarterly. This information is based on monthly reports submitted to 
the PM by the Center ODIN Project Managers. [See comment 5] 

3. Page 33, Agency/Contractor Partnerships are Vital to Success: It 
should be noted that the report states that in November of 2001, a 
NASA consultant pointed out there was no true partnership between the 
Agency and the ODIN contractors. On the next page, the report quotes a 
NASA consultant's recommendation that Goddard Space Flight Center 
(GSFC) and its vendor begin to build institutional and personal 
relationships. They also quote the same consultant as saying GSFC and 
its vendor develop a conflict resolution process. That consultant's 
recommendation came from a December 2000 report. Nowhere does the GAO 
report mention that GSFC went into a partnering arrangement with its 
ODIN contractor during the summer of 2000, and at that time developed 
a conflict resolution process. At the last ODIN workshop, GSFC led, 
and continues to lead the Agency in the partnering effort. [See 
comment 6] 

Should you or your staff wish to discuss this report, please Richard 
Brozen, Deputy CIO for Management, at 358-2240. 

Cordially yours, 

Signed by: 

Lee Holcomb: 
Chief Information Officer: 

cc: 

AA/C. Stadd: 
AB/M. Christensen: 
AI/D. Mulville: 

The following are GAO's comments on the National Aeronautics and Space 
Administration's letter dated February 28, 2002. 

1. We disagree that our characterization of NASA's up-front cost 
analysis is not accurate. For reasons discussed in this report, we do 
not agree that NASA performed sufficient up-front cost analysis. A 
thorough understanding of existing agency IT assets and functions is 
critical to an agency's ability to select appropriately among 
alternative ways to meet its distributed computing needs, and to 
ensure the effective management of contracts awarded to acquire these 
services. While we acknowledge that performing such an analysis can be 
challenging, the consequences of not doing so are nontrivial. This is 
clearly illustrated by the inability of a contractor employed by NASA 
to determine whether the agency is saving money with seat management 
because of the lack of a comprehensive pre-seat-management baseline. 
Also, as discussed in this report and our IT investment management 
guidance to agencies,[Footnote 42] we agree that cost represents only 
one element of a complete preinvestment analysis. Other considerations 
include an evaluation of expected benefits and risks. 

2. We disagree that our characterization of NASA's tracking of 
internal seat costs is incorrect. We note in this report that NASA 
tracks contractor costs and the number of staff associated with the 
management of its seat management program. While important, the 
monitoring of full-time-equivalents does not provide the agency with a 
complete picture of internal costs associated with the implementation 
of seat management. Moreover, as NASA officials acknowledged during 
the exit conference, the agency does not track the full internal costs 
of seat management, which would include cost items such as overhead 
and salaries and benefits. This is critical since the internal costs 
to manage seat management can be substantial. For example, NASA's 
Kennedy Space Center reported that its salaries and benefits to manage 
the Office of Space Flight's four seat-management implementations in 
July 2001 were about $146,000. 

3. We do not agree that NASA routinely monitors the actual overall 
benefits of its seat management program. Tracking of actual benefits 
is a critical aspect of accountability in that it answers the question 
of whether expected cost savings and other qualitative and 
quantitative benefits are being achieved. NASA's seat management 
business case documented 11 program goals, but neither the agency's 
quarterly reporting process nor the ODIN postimplementation review 
fully addresses whether these goals have been achieved. In particular, 
the performance measures included in the quarterly reports focus on 
certain contractor performance metrics (e.g., service delivery and 
availability) and the average seat costs for general-purpose desktop 
computing seats. As we noted in this report, while this type of 
tracking can be an indicator of whether certain types of benefits are 
being achieved, such metrics do not fully address whether the overall 
costs and benefits of the seat management program are being met. 
NASA's quarterly reports do not contain performance measures that 
address estimated benefits such as potential increases in user 
efficiency and productivity and improved staff focus on mission-
related activities. With respect to NASA's postimplementation review, 
the contractor that performed the review addressed whether NASA 
achieved certain types of benefits, such as cost savings and 
standardization, but did not address other expected program benefits, 
such as potential increases in user productivity and efficiency. 
Nevertheless, we agree that this postimplementation review was a good 
first step, which, if followed up by subsequent reviews that address 
all expected benefits of the NASA seat management program, would 
address our concerns in this area. 

4. We modified our report to clarify the terms of the NASA master 
contract. 

5. See comment 3. 

6. We modified our report to recognize NASA's initiatives. 

[End of section] 

Appendix X: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Linda J. Lambert, (202) 512-9556. 

Acknowledgments: 

Harold Brumm, Peggy A. Hegg, James C. Houtz, Barbarol J. James, and 
Frank Maguire made key contributions to this report. 

[End of section] 

Footnotes: 

[1] IT outsourcing describes the activities associated with acquiring 
IT services from one or more external providers. During outsourcing, a 
client organization transfers responsibility for one or more IT 
services to one or more external providers. This responsibility is 
executed by controlling and managing the processes, people, and 
technology associated with these services. 

[2] The agencies in our review used various governmentwide contracts 
to implement seat management. As such, they generally did not enter 
into contracts with the seat management vendors; instead, they used 
task orders, delivery orders, or blanket purchase agreements, which 
were placed against existing contracts. For the purposes of this 
report, however, we will refer to such acquisition documents as 
contracts. 

[3] Performance-based service contracting is a process in which the 
customer agency specifies the outcome or result it desires and leaves 
it to the vendor to decide how best to achieve the desired outcome. 

[4] In some cases, agencies used their seat management contract to 
obtain other IT services that were not charged on a per-seat basis. 

[5] In at least one instance, an agency chose to retain ownership of 
the assets obtained through its seat management contract. 

[6] Gartner, Inc., Trends in Desktop Management Outsourcing Contracts, 
1996-1998 (March 8, 1999). 

[7] Under the schedule program, GSA enters into indefinite-delivery, 
indefinite-quantity contracts with commercial firms to provide 
commercial goods and services governmentwide. Schedule 70 contains 
contracts related to general-purpose commercial IT equipment, 
software, and services. 

[8] U.S. General Accounting Office, Information Technology: Leading 
Commercial Practices for Outsourcing of Services, [hyperlink, 
http://www.gao.gov/products/GAO-02-214] (Washington, D.C.: November 
30, 2001). 

[9] Office of Management and Budget (OMB), Circular A-130, Management 
of Federal Information Resources (November 30, 2000) and Circular A-
11, Part 3, Planning, Budgeting, and Acquisition of Capital Assets 
(July 2001). These circulars define capital assets as including IT 
that is used by the federal government and has an estimated useful 
life of 2 years or more. Such assets may be acquired in different 
ways, including through lease/purchase or other capital leases 
regardless of whether title has passed to the federal government or 
through an operating lease for an asset with an estimated useful life 
of 2 years or more. 

[10] U.S. General Accounting Office, Information Technology Investment 
Management: A Framework for Assessing and Improving Process Maturity, 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-10.1.23], Exposure 
Draft (Washington, D.C.: May 2000). 

[11] [hyperlink, http://www.gao.gov/products/GA0-02-214]. 

[12] Kelly, Anderson & Associates, ODIN Program, The Outsourcing 
Desktop Initiative for NASA: Post Implementation Business Case 
Assessment (November 2001). 

[13] The analysis included an assessment of using the GSA seat-
management contract alternative. 

[14] U.S. General Accounting Office, Assessing Risks and Returns: A 
Guide for Evaluating Federal Agencies' IT Investment Decision-making, 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-10.1.13] (Washington 
D.C.: February 1997). 

[15] [hyperlink, http://www.gao.gov/products/GAO/AIMD-10.1.23]. 

[16] During exit conferences with the agencies, most acknowledged that 
they did not monitor all internal agency costs but stated that they 
could identify the number of staff associated with the management of 
their seat management programs. In addition, DLA provided us with 
several months' worth of costs, which included categories such as 
salaries and benefits, travel, rent, communication, and utilities. 
However, these costs were for the entire Technology Services and 
Infrastructure Support Directorate, of which the seat management 
office is only a part. A DLA official stated that the agency could 
extract the seat management costs from these amounts, but these data 
are not presented to management at that level of detail. 

[17] In June 2001, NASA signed a 3-year follow-on contract with the 
incumbent Kennedy Space Center seat management contractor, with a 
period of performance beginning December 1, 2001. 

[18] Three of the agencies in our review have awarded incentives or 
applied penalties to their contractors. In addition, ATF has not 
finalized its service-level agreements with its seat management 
contractor. Agency officials stated that they are monitoring 
contractor performance against these interim agreements but have not 
finalized the penalties and incentives that will be associated with 
the final service-level agreements. 

[19] Service-level agreements define the agency's expectations and are 
used to track and measure a contractor's performance. 

[20] Office of Management and Budget, Circular A-11, Part 3 
Supplement, Capital Programming Guide, and GAO's IT investment 
management guidance, [hyperlink, 
http://www.gao.gov/products/GAO/AIMD-10.1.23], and [hyperlink, 
http://www.gao.gov/products/GAO/AIMD-10.1.13]. 

[21] See footnote 20. 

[22] Office of Management and Budget, Evaluating Information 
Technology Investments: A Practical Guide (November 1, 1995) and 
Circular A-11. One ranking process included assigning a score ranging 
from 1 to 10 to each risk and then multiplying each score by a 
percentage weight reflecting the relative importance of each factor in 
the agency's decision processes. 

[23] Mitigation approaches include transferring, avoiding, reducing, 
assuming, and sharing the potential for an event that could produce an 
adverse consequence, or for the adverse consequence itself. 

[24] The DLA seat management contract did not specifically address 
disposition of contractor-supplied hardware and software. DLA advised 
us that it bought and maintained ownership of all contract assets. 

[25] [hyperlink, http://www.gao.gov/products/GAO-02-214]. 

[26] In the exit conference with GSA, agency officials noted that 
while the timing of the decision to implement seat management may have 
been driven by the desire to show the agency's support for the 
concept, the agency also implemented seat management to achieve other 
goals, such as increased standardization and decreased costs per seat. 

[27] U.S. General Accounting Office, Management Reform: Elements of 
Successful Improvement Initiatives, [hyperlink, 
http://www.gao.gov/products/GAO/T-GGD-00-26] (Washington, D.C.: 
October 15, 1999). 

[28] The Industry Advisory Council is an organization of IT 
professionals representing more than 270 companies nationwide that 
provide products and services to the public. The mission of its 
Managed Services Shared Interest Group is to facilitate information 
flow; foster dialog about best practices; and influence the future of 
managed services (which includes seat management) to the mutual 
benefit of government, industry, and end users. 

[29] [hyperlink, http://www.gao.gov/products/GAO-02-214]. 

[30] Gartner, Inc., Desktop Management Outsourcing: Do's and Don'ts, 
Research Note DF-14-0931 (October 2, 2001). 

[31] The private-sector organizations referred to in this section 
include the seat management contractors for the six agencies and GSA; 
Gartner, and Giga Information Group, Inc. (private research firms); 
and members of the Managed Services Shared Interest Group. 

[32] Giga Information Group, Inc., Payment and Incentives for 
Outsourcing Management (July 27, 2000). 

[33] In cases in which the seat management contractor is assuming 
responsibility for functions previously performed by federal 
employees, it is especially important that the organization address 
internal staff issues. This can include, when appropriate and 
consistent with organizational objectives, (1) encouraging the 
transition of staff to the seat management contractor; (2) assisting 
employees who do not want to transfer in finding other jobs, either 
within the organization or at another organization; and (3) developing 
employee retention programs to keep key people. 

[34] Gartner, Inc., Is Your Organization Ready for Seat Management?, 
Research Note #DF-09-2729 (October 4, 1999). 

[35] Gartner, Inc., Seat Management: Look Beyond the Prime Vendors, 
Research Note #DF-07-9093 (May 10, 1999). 

[36] See footnote 32. 

[37] [hyperlink, http://www.gao.gov/products/GAO-02-214]. 

[38] See footnote 30. 

[39] See footnote 34. 

[40] Office of Management and Budget, Circular A-130, Management of 
Federal Information Resources (November 30, 2000); Circular A-11, Part 
3, Planning, Budgeting, and Acquisition of Capital Assets (July 2001); 
and Evaluating Information Technology Investments: A Practical Guide 
(November 1, 1995). 

[41] U.S. General Accounting Office, Information Technology Investment 
Management: A Framework for Assessing and Improving Process Maturity, 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-10.1.23], Exposure 
Draft (Washington, D.C.: May 2000) and Assessing Risks and Returns: A 
Guide for Evaluating Federal Agencies' IT Investment Decision-making, 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-10.1.13] (Washington, 
D.C.: February 1997). 

[42] U.S. General Accounting Office, Information Technology Investment 
Management: A Framework for Assessing and Improving Process Maturity, 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-10.1.23], Exposure 
Draft (Washington, D.C.: May 2000) and Assessing Risks and Returns: A 
Guide for Evaluating Federal Agencies' IT Investment Decision-making, 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-10.1.13] (Washington, 
D.C.: February 1997). 

[End of section] 

GAO’s Mission: 

The General Accounting Office, the investigative arm of Congress, 
exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO’s commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO’s Web site (www.gao.gov) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as “Today’s Reports,” on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select “Subscribe to daily E-mail alert for newly 
released products” under the GAO Reports heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. General Accounting Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

To Report Fraud, Waste, and Abuse in Federal Programs: 
Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, NelliganJ@gao.gov: 
(202) 512-4800: 
U.S. General Accounting Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: