This is the accessible text file for GAO report number GAO-03-328 
entitled 'Electronic Procurement: Business Strategy Needed for GSA's 
Advantage System' which was released on March 21, 2003.



This text file was formatted by the U.S. General Accounting Office 

(GAO) to be accessible to users with visual impairments, as part of a 

longer term project to improve GAO products’ accessibility. Every 

attempt has been made to maintain the structural and data integrity of 

the original printed product. Accessibility features, such as text 

descriptions of tables, consecutively numbered footnotes placed at the 

end of the file, and the text of agency comment letters, are provided 

but may not exactly duplicate the presentation or format of the printed 

version. The portable document format (PDF) file is an exact electronic 

replica of the printed version. We welcome your feedback. Please E-mail 

your comments regarding the contents or accessibility features of this 

document to Webmaster@gao.gov.



GAO: 



Report to the Chairman, Committee on Government Reform, U.S. House of 

Representatives:



February 2003: 



Electronic Procurement: 



Business Strategy Needed for GSA’s Advantage System: 



GAO-03-328:



GAO Highlights: 



Highlights of GAO-03-328, a report to the Chairman, Committee on 

Government Reform, U.S. House of Representatives:



February 2003: 



Electronic Procurement: 



Business Strategy Needed for GSA’s Advantage System: 



Why GAO Did This Study: 



The Congress has been seeking to increase federal agency purchases of 

equipment and services on line.  The General Services Administration’s 

(GSA) Advantage Internet-based ordering system is meant for conducting 

market research and ordering all types of products and services on 
line. 

About $84 million has already been invested in this endeavor. GAO was 

asked to assess how effectively GSA has managed this investment. 



What GAO Found: 



GSA Advantage has had only limited success as an on-line market 
research 

and ordering tool.  Market research has been limited primarily to off-
the-

shelf office products, and sales through Advantage have never exceeded 
one–

half of 1 percent of overall schedule sales.  Because of initial design 

limitations, Advantage has not been effective in acquiring complex 
products 

and services, particularly information technology services that make up 

most of the growth in schedule sales.  Recent enhancements may 
facilitate 

use of the system for buying complex services, but vendor participation 

has not been strong.  For example, for almost half the requests for 
quotes, 

agencies received no responses from vendors.  Advantage does not 
contain 

all schedule products and services, as required, and some schedule 
vendors 

complain about the cost and complexity involved in participating.  



While GSA has modified the Advantage system to provide better and more 

information on products and services, the system is unlikely to succeed 

without a business strategy.  To date, GSA has not yet answered basic 

questions surrounding Advantage, including (1) if it is technically 
feasible 

to make purchases of all products and services on line given the range 
of 

activities involved with making more complex acquisitions, (2) if so, 

whether Advantage is the right system to do this, and (3) if not, 
whether

Advantage could be used for smaller, simpler purchases and as an 
information 

source for making larger, more complicated acquisitions.  



Moreover, GSA has not thoroughly analyzed other alternatives to 
Advantage 

or developed effective measures to assess return on investment and user 

satisfaction. A business strategy would require GSA to not only 
reexamine 

its objectives for on-line purchasing and various solutions, but also 
develop 

measures appropriate for gauging success.  As such, it would help GSA 
assess 

whether continued investment in Advantage is worthwhile and make 
modifications 

to the system that will substantially enhance its performance.



Figure: 



[See PDF for image]



[End of figure]



What GAO Recommends: 



GAO recommends that GSA develop a business strategy that reexamines its 

objectives for on-line purchasing of products and services, assesses 
what 

solutions are most appropriate for meeting these objectives, and 
develops 

measures for assessing success.  GSA concurs with this report’s 
recommendations.



www.gao.gov/cgi-bin/getrpt?GAO-03-328.



To view the full report, including the scope and methodology, click on 
the 

link above.For more information, contact William Woods (202) 512-4841, 

WoodsW@gao.gov



contents: 



Letter:



Results in Brief: 



Background: 



Agencies Use Advantage More for Market Research than for Ordering, and 
Mostly 

for Standard Office Products: 



Accomodating Buys of Complex Services Has Been Problematic: 



Lack of Full Vendor Participation Due to Concerns About Cost and Use:



Performance Reflects the Lack of a Business Strategy:



Conclusion:



Recommendations:



Agency Comments:



Scope and Methodology:



Appendix:



Comments from the General Services Administration:



Figure: 



Figure 1: Sales under FSS Schedule Contracts--Fiscal Years 1998 to 
2002:



This is a work of the U.S. Government and is not subject to copyright 

protection in the United States. It may be reproduced and distributed 
in its

entirety without further permission from GAO. It may contain 
copyrighted 

graphics, images or other materials. Permission from the copyright 
holder

may be necessary should you wish to reproduce copyrighted material 

separately from GAO’s product.



February 19, 2003:



The Honorable Tom Davis:



Chairman:



Committee on Government Reform:



House of Representatives:



Dear Mr. Chairman:



Federal agencies ordered about $22 billion worth of products and 

services from commercial firms during fiscal year 2002 using the 

General Services Administration’s (GSA) Multiple Award Schedules 

program. To help federal agencies use the program more efficiently, GSA 

developed a system called GSA Advantage, which is an Internet-based 

system for conducting market research and ordering schedule products 

and services on line. Because of concerns about the effectiveness of 

GSA Advantage, you asked that we review the performance of the system. 

In response to your request, we assessed how effective Advantage has 

been as an on-line system for federal agencies to shop for and order 

schedule products and services.



Results in Brief:



GSA Advantage has had only limited success as an on-line market 

research and ordering tool. Market research has been limited primarily 

to off-the-shelf office products, and sales through Advantage have 

never exceeded one-half of 1 percent of overall schedule sales. Because 

of initial design limitations, Advantage has not been an effective 

means for acquiring complex products and services, particularly 

information technology services that make up most of the growth in 

schedule sales. Recent enhancements may facilitate use of the system 

for buying complex services, but vendor participation has not been 

strong. For example, for almost half the requests for quotes, agencies 

received no responses from vendors. Advantage does not contain all 

schedule products and services, as required, and some schedule vendors 

complain about the cost and complexity involved in participating.



The limited success of Advantage reflects the lack of a business 

strategy for the system. Such a strategy normally would examine such 

issues as what type of schedule products and services is best suited 

for on-line market research and ordering, whether Advantage is the best 

system to accomplish these functions in a manner consistent with the 

needs of customer agencies and the capabilities of the vendors, and how 

the performance of that system could be measured. After developing such 

a strategy, GSA would have greater assurance that its system is the 

most feasible solution to accomplish its objectives and would have a 

sound basis for setting policies for Advantage. We are recommending 

that GSA develop a comprehensive business strategy for the Advantage 

system.



Background:



GSA’s mission includes providing policy leadership and expertise in the 

acquisition of products and services for use by federal agencies. To 

help carry out its mission, GSA’s Federal Supply Service (FSS) assists 

federal agencies in acquiring supplies --such as furniture, computers, 

tools, and equipment --and a variety of services through its commercial 

acquisition business line. This GSA business line gives agencies access 

to several million commercial products and services available either 

through schedule contracts or as stock items from a GSA warehouse.



GSA launched Advantage in October 1995. The system was envisioned as an 

on-line system for agencies to order stock items from GSA’s warehouses 

with a government payment card. Initially, Advantage offered only stock 

items. In February 1996, Congress passed the Clinger-Cohen Act, which 

required the Administrator of GSA to develop a system capable of 

providing governmentwide, on-line Internet access to information on all 

FSS schedule products and services. GSA decided that Advantage, 

although in its infancy, was the only Internet-based, on-line ordering 

system that potentially could meet the requirements of the 

act,[Footnote 1] and began using Advantage to satisfy this requirement. 

GSA began adding schedule services to Advantage in February 2000.



Despite efforts to include more products and services on Advantage, 

only 35 percent of all vendors participated in the system at the 

beginning of fiscal year 2001. GSA therefore issued a mandate that all 

schedule vendors participate in Advantage and offer all their products 

and services through it. For off-the-shelf products, vendors typically 

upload three types of information: product price, features, and 

performance. In some cases, they offer quantity discounts. For 

services, vendors typically upload the labor rates contained in their 

schedule contracts, along with other information.



GSA has added various functions and enhancements to Advantage during 

the last 7 years, such as reference files to increase the amount of 

information available to customer agencies. In total, GSA has spent 

about $84 million to develop, implement, and maintain Advantage.



Agencies Use Advantage More for Market Research than for Ordering, and 

Mostly for Standard Office Products:



Customer agencies do a great deal more comparative shopping than 

ordering through Advantage. During a 7-month period in fiscal year 

2002, comparative shopping on Advantage ranged between 27,000 and 

50,000 site searches each day.[Footnote 2] Most of this market research 

involved office products and equipment. For example, the top searched-

for products included office supplies like binders, pens, copier paper, 

and envelopes, and office equipment such as laptops, fax machines, 

chairs, and shredders. Searches for services were far less frequent 

than for products, and most often were for training, security, 

maintenance, and consulting.



Sales through Advantage represent a very small portion of total 

schedule sales. As shown in figure 1, overall schedule sales have grown 

considerably in the past 5 years--from about $8.5 billion to almost $22 

billion. Sales, however, have never exceeded one-half of 1 percent of 

total schedule sales. For fiscal year 2002, for example, there were 

129,370 schedule orders through Advantage for a total of $81,579,970, 

representing 0.37 percent of total schedule sales. Most sales through 

Advantage were for standard office products. For example, for fiscal 

year 2002, the top 10 schedule items in order of sales frequency 

included file folders, self-stick pads, multiuse paper, double pocket 

portfolio, ink jet toner, copy paper, binder clips, and three-ring 

binders. Our review of Advantage orders from 1997 through 2002 shows a 

similar pattern of purchases of standard office products. According to 

GSA, schedule sales through Advantage have averaged about $500 per 

order.[Footnote 3]



Figure 1: Sales under FSS Schedule Contracts--Fiscal years 1998-2002:



[See PDF for image]



Source: GSA:



Note: Numbers on top of columns are the percentages of schedule sales 

through Advantage:



[End of figure]



Accommodating Buys of Complex Services Has Been Problematic:



As shown in figure 1, most of the growth in overall schedule sales has 

been in spending for information technology (IT) under GSA’s schedule 

70 contracts. Most of this growth in IT spending has been for IT 

services, which now account for about 65 percent of IT spending. Yet an 

inherent limitation in using Advantage is that the system was not 

designed for making complex purchases, such as procurements of IT 

services.



The information available on Advantage regarding complex services 

typically consists of general descriptions of labor categories, such as 

a program manager or network analyst, and loaded labor rates.[Footnote 

4] In a prior report, we pointed out that this information alone is not 

sufficient to enable agencies using GSA’s schedules to select the best 

value provider of IT services.[Footnote 5] Rather, agencies must 

consider the number of hours and mix of labor categories proposed by 

the schedule contractors to perform specific tasks. Special ordering 

procedures issued by GSA require agencies acquiring complex services to 

obtain this information by providing a statement of work to at least 

three schedule contractors asking for either fixed or ceiling prices 

for the work specified.



In 2001, GSA added a capability to Advantage, known as E-buy, which 

enables agencies to provide a statement of work to vendors and request 

responses on line. This modification provides a means to share 

information between customer agencies and vendors on a range of 

products and services, including the more complex IT services. During 

fiscal year 2002, the first full fiscal year of operation, agencies 

used E-buy to place a total of 64 orders. By comparison, customers 

placed an average of about 1,240 non-E-buy orders each day through 

Advantage during the same period.



Due to the limited initial use of E-buy, GSA revised the function in 

July 2002 to make it more user-friendly. During the next 2 months of 

operation (July-August 2002), agency usage of the revised system 

increased, but there were other implementation problems. About 12 

percent of requests for quotations (RFQ) were sent to fewer than three 

vendors, as required. For each RFQ issued, agencies received an average 

of only 1.3 responses. For almost half the RFQs (about 46 percent), the 

agencies received no responsive quote from any vendor.



GSA officials have expressed disappointment in the low usage of E-buy 

in its first year of operation, and they are concerned about the lack 

of vendor responses following revisions to the system. The officials 

also pointed out, however, that Advantage and its E-buy enhancement 

should improve agency buyers’ market research capabilities.



Full Vendor Participation Lacking Due to Concerns About Cost and Use:



Despite GSA’s mandatory vendor participation policy, not all vendors 

list all their products and services on Advantage. Out of approximately 

9,800 schedule contracts, about 1,200 contracts (12 percent) were not 

on Advantage as of September 2002. Even some vendors who supply off-

the-shelf products - which are most likely to be ordered on line - 

decline to participate. For example, one vendor who sells scales and 

butcher supplies is not on Advantage because the vendor’s primary 

customers do not use the system. Another vendor indicated that it 

uploaded only certain products from its catalog; it has not uploaded 

the services it offers because its principal service customers do not 

use Advantage. Other vendors report that it can cost several thousand 

dollars to upload data on Advantage, particularly for complex products 

and services. Such expenditures do not make sense, they say, if 

customers are not using Advantage to make purchases.



An underlying reason why some vendors are reluctant to list all of 

their complex products and services on Advantage is because the system 

does not contain sufficient descriptive information to permit agencies 

to make informed buying decisions. To address this issue, GSA added a 

capability called e-catalog reference files, which allows vendors to 

supplement the information about their products or services by 

providing links to their own Web pages. Although this capability 

provides more information about vendor products and services, the 

descriptions vary widely in quality.



Performance Reflects the Lack of a Business Strategy:



GSA has not conducted the type of analysis central to developing a 

comprehensive business strategy for Advantage. Specifically, GSA has 

not analyzed customer requirements, identified the products and 

services most likely to be purchased on-line by agency customers, 

assessed the technological feasibility of meeting customer requirements 

through an on-line system, or considered alternatives to Advantage. In 

addition, GSA has not established performance goals for Advantage or a 

system for measuring results. Each of these is an important element of 

a comprehensive business strategy.



GSA officials acknowledge the lack of a business strategy for 

Advantage. According to these officials, GSA started Advantage before 

performing the kind of rigorous analysis it would undertake before 

initiating a major system today. They recognized that the benefit of 

doing a business strategy was that it would have enabled GSA management 

to answer two important questions: (1) how much and what kind of 

products and services could be expected to be purchased by federal 

customers on-line, and (2) whether Advantage is the right system to 

facilitate such on-line purchases.



Conclusion:



GSA has spent about $84 million on Advantage, but it has not developed 

a comprehensive business strategy that would justify that level of 

investment. In the absence of a business strategy, GSA has made changes 

to enhance the system, but is yet to target what products and services 

are best suited for on-line market research and ordering. GSA has made 

these modifications without integrating them into an overall strategy. 

A rigorously developed business strategy, including performance 

measures, would help GSA assess whether its continued investment in 

Advantage is worthwhile and, if so, what additional changes to enhance 

its performance may be needed.



Recommendation:



We recommend that the Administrator of General Services develop a 

comprehensive business strategy that identifies GSA’s objectives for an 

on-line market research and ordering system, the desired capabilities 

of that system, the resources required to implement the system, and the 

measures to track performance. After developing the business strategy, 

GSA should determine whether Advantage is the most appropriate system 

for achieving its objectives.



Agency Comments:



GSA concurs with our recommendation. GSA’s comments appear in Appendix 

I.



Scope and Methodology:



To address our objective, we interviewed GSA and industry officials, 

analyzed electronic files of past market research and on-line sales, 

attended a GSA training course on how vendors electronically transfer 

information about their products and services into Advantage, and 

received various demonstrations of Advantage operations and marketing 

techniques. We also visited GSA’s two largest acquisition centers 

(Arlington, Virginia and Fort Worth, Texas) to learn more about GSA’s 

efforts to address vendor concerns and encourage vendor participation 

in Advantage. We conducted our review in accordance with generally 

accepted government auditing standards from March through December 

2002.



Unless you publicly announce the contents of this report earlier, we 

plan no further distribution of it until 30 days from the report date. 

At that time, we will send copies to other interested congressional 

committees; the Administrator of General Services; and the Director, 

Office of Management and Budget. We will also make copies available to 

others upon request. In addition, the report will be available at no 

charge on the GAO Web site at http://www.gao.gov.



Please contact me at (202) 512-8214 or Ralph Dawn at (202) 512-4544 if 

you have any questions regarding this report. Major contributors to 

this report were John E. Clary, Ronald J. Salo, James B. Smoak, and 

Ralph O. White.



Sincerely yours,



William T. Woods:



Director:



Acquisition and Sourcing Management:



Signed by William T. Woods:



Appendix:



Comments From the General Services Administration:



GSA:



GSA Administrator:



February 5, 2003:



The Honorable David M. Walker Comptroller General:



of the United States General Accounting Office Washington, DC 20548:



Dear Mr. Walker:



Thank you for providing us with the opportunity to comment on the 

General Accounting Office (GAO) draft report entitled “Electronic 

Procurement: Business Strategy Needed for GSA Advantage Program” (GAO-

03-328), dated January 16, 2003. The draft recommends that the General 

Services Administration (GSA) develop a comprehensive business strategy 

that identifies GSA’s objectives for an on-line market research and 

ordering system, the desired capabilities of that system, the resources 

required to implement the system, and measures to track performance. 

GSA concurs with the draft report’s recommendation.



GSA’s Federal Supply Service (FSS) is currently conducting an FSS-wide 

enterprise IT architecture study during FY 2003. GSA recognizes that it 

is time to perform a rigorous analysis that will document the business 

case for our future decisions, and in fact, we are in the process of 

developing a strategic business plan for GSA Advantage as part of that 

effort.



If you have any additional questions or need further assistance, please 

have a member of your staff contact Mr. Ralph Boldt, Branch Chief, 

Audit Followup and Evaluation Branch, on (202) 501-3094.



Sincerely, 



Stephen A. Perry, Administrator:



Signed by Stephen A. Perry:



cc: William T. Woods, Director:



Acquisition and Sourcing Management United States General Accounting 

Office Washington, DC 20548:



U.S. General Services Administration 1800 F Street, NW:



Washington, DC 20405-0002 Telephone: (202) 501-0800 Fax: (202) 219-1243 

www.gsa.gov:



FOOTNOTES



[1] The Clinger-Cohen Act called for the economic and efficient 

procurement of information technology and other commercial items using 

a system that provides on-line access to price, features, and 

performance information on all products and services found in FSS 

schedules, and that provides on-line ordering capability. Although the 

system continues, the statutory requirement for the system was repealed 

by Section 825(c)(1) of the Bob Stump National Defense Authorization 

Act for Fiscal Year 2003, Pub. L. No. 107-314, 116 Stat. 2616.







[2] This calculation does not include holidays and weekends.



[3] Advantage sales also include purchases of stock items from GSA’s 

two remaining warehouses. GSA’s stock program that comprises about 55 

percent of the Advantage sales (in fiscal year 2002) has had a 4-year 

decline in sales overall due to a changing customer base and the 

increased use of credit cards throughout the government. In fact, FSS 

has closed all but two warehouses and transitioned from a government-

owned and operated supply distribution network to a system of alliances 

with private companies to provide the goods and services. Without 

alternative strategies such as a virtual store through Advantage that 

would allow GSA to provide the products to the agencies using direct 

delivery rather storing and shipping them from a warehouse, agency 

officials told us the stock program sales might continue to decline.



[4] Loaded labor rates include salary, fringe benefits, overhead, and 

profit.



[5] See U.S. General Accounting Office, Contract Management: Not 

Following Procedures Undermines Best Pricing Under GSA’s Schedule, GAO-

01-125 (Washington D.C.: Nov. 28, 2000).



GAO’s Mission:



The General Accounting Office, the investigative arm of Congress, 

exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability 

of the federal government for the American people. GAO examines the use 

of public funds; evaluates federal programs and policies; and provides 

analyses, recommendations, and other assistance to help Congress make 

informed oversight, policy, and funding decisions. GAO’s commitment to 

good government is reflected in its core values of accountability, 

integrity, and reliability.



Obtaining Copies of GAO Reports and Testimony:



The fastest and easiest way to obtain copies of GAO documents at no 

cost is through the Internet. GAO’s Web site ( www.gao.gov ) contains 

abstracts and full-text files of current reports and testimony and an 

expanding archive of older products. The Web site features a search 

engine to help you locate documents using key words and phrases. You 

can print these documents in their entirety, including charts and other 

graphics.



Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its 

Web site daily. The list contains links to the full-text document 

files. To have GAO e-mail this list to you every afternoon, go to 

www.gao.gov and select “Subscribe to daily E-mail alert for newly 

released products” under the GAO Reports heading.



Order by Mail or Phone:



The first copy of each printed report is free. Additional copies are $2 

each. A check or money order should be made out to the Superintendent 

of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. 

Orders should be sent to:



U.S. General Accounting Office



441 G Street NW,



Room LM Washington,



D.C. 20548:



To order by Phone: 	



	Voice: (202) 512-6000:



	TDD: (202) 512-2537:



	Fax: (202) 512-6061:



To Report Fraud, Waste, and Abuse in Federal Programs:



Contact:



Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov



Automated answering system: (800) 424-5454 or (202) 512-7470:



Public Affairs:



Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.



General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.



20548: