This is the accessible text file for GAO report number GAO-07-268 
entitled 'Telecommunications: GSA Has Accumulated Adequate Funding for 
Transition to New Contracts but Needs Cost Estimation Policy' which was 
released on March 26, 2007. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Ranking Member, Committee on Oversight and Government 
Reform, House of Representatives: 

United States Government Accountability Office: 

GAO: 

February 2007: 

Telecommunications: 

GSA Has Accumulated Adequate Funding for Transition to New Contracts 
but Needs Cost Estimation Policy: 

GAO-07-268: 

GAO Highlights: 

Highlights of GAO-07-268, a report to the Ranking Member, Committee on 
Oversight and Government Reform, House of Representatives 

Why GAO Did This Study: 

The General Services Administration (GSA) and its customer agencies are 
preparing to transition new governmentwide telecommunications contracts 
known as the Networx program. GSA estimated the costs for which it is 
responsible to be $151.5 million. This report addresses (1) the 
soundness of the analysis GSA used to derive the estimate of funding 
that would be required for the transition and (2) whether GSA will have 
accumulated adequate funding to pay for transition costs. In performing 
this work, GAO reviewed cost estimation best practices, analyzed 
relevant GSA documents, and performed an uncertainty analysis on GSA’s 
estimate. 

What GAO Found: 

GSA did not use sound analysis when estimating the amount of funding 
needed to meet its transition-related commitments. Specifically, its 
analysis was not sufficiently accurate, comprehensive, documented, or 
validated. A primary weakness is that the estimate is largely based an 
assumption—known as the transition traffic factor—that 76 percent of 
the services provided under the current contracts would be moved to a 
different provider under the Networx contracts. However, according to 
program officials, this assumption is intentionally conservative and 
represents a worst-case scenario that is unlikely to occur. 
Additionally, GSA may have double counted a cost and did not update its 
analysis to reflect a nearly 2 year delay. Finally, GSA did not 
document significant assumptions and data sources used in its analysis, 
or validate it. These weaknesses can be attributed in part to the lack 
of a cost estimation policy that reflects best practices. While GSA’s 
intentionally conservative approach minimizes the risk that it would 
have inadequate funds to pay for committed transition costs, it 
increases the risk that GSA will retain excess funds that could be used 
for other purposes. 

GSA has accumulated adequate funding to support its anticipated 
transition costs. As of fiscal year-end 2006, GSA had approximately 
$142 million in a transition reserve. GAO analysis of the estimate 
indicates it is unlikely that GSA will need more than it has already 
accumulated to fund the transition. Specifically, the $142 million 
already retained will be adequate to cover anticipated costs 96 percent 
of the time. The recent merger of two GSA funds gives the agency 
additional flexibility that reduces its need to accumulate the entire 
$151.5 million it estimated would be needed (see table). With Networx 
contracts scheduled to be awarded starting in March 2007, GSA will soon 
have the information necessary to reassess the main assumption 
underlying its estimate—the transition traffic factor—and address the 
weaknesses GAO identified. Once this has been accomplished, GSA can 
reevaluate the funding needed to meet anticipated commitments. 

Table: GSA's Transition Cost Estimate: 

Cost element: GSA contractor support; 
Estimated cost: $35.0. 

Cost element: Certain agency transition costs; 
Estimated cost: 116.5. 

Cost element: Total; 
Estimated cost: $151.5. 

Source: GSA. 

[End of table] 

What GAO Recommends: 

GAO recommends that the GSA Administrator establish a cost estimation 
policy that reflects best practices. In addition, GAO recommends that 
the Administrator revise the transition cost estimate using best 
practices after the award of the Networx contracts and reassess the 
funding needed to meet its commitments. GSA concurred with these 
recommendations but questioned the way GAO characterized the soundness 
of GSA’s analysis and whether this transition was comparable with the 
previous one. GAO clarified its characterization of GSA’s analysis. 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-268]. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Linda Koontz at (202) 512-
6240 or koontzl@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

GSA's Analysis Was Not Sound: 

GSA Has Accumulated Adequate Funding for the Transition: 

Conclusions: 

Recommendations: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Uncertainty Analysis for GSA's Transition Estimate: 

Appendix III: Comments from the GSA Administrator: 

Appendix IV: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: GSA Transition Cost Estimate: 

Table 2: Cumulative Effect of Quantifiable Weaknesses: 

Table 3: Risk Adjusted Factors: 

Figures: 

Figure 1: Estimated Cost of Transition Using Various Percents of 
Transition Traffic: 

Figure 2: Confidence That Certain Amounts of Funding Will Account for 
Actual Costs: 

Abbreviations: 

GSA: General Services Administration: 

IMC: Interagency Management Council: 

IT: Information Technology: 

United States Government Accountability Office: 
Washington, DC 20548: 

February 23, 2007: 

The Honorable Tom Davis: 
Ranking Member: 
Committee on Oversight and Government Reform: 
House of Representatives: 

Dear Mr. Davis: 

As innovations in telecommunications services continue to transform the 
way the federal government conducts business, the General Services 
Administration's (GSA) governmentwide telecommunications acquisition 
programs offer federal agencies the opportunity to apply innovative 
services and solutions to their operations. With the current set of 
governmentwide telecommunications contracts approaching expiration, GSA 
and its customer agencies will have to transition the services acquired 
under these contracts to their replacements, known collectively as 
Networx. 

GSA will incur program management costs associated with planning and 
executing this transition. It has also made a commitment to absorb 
certain agency transition costs. To ensure it would have the funds 
necessary to pay for these costs, GSA estimated that it would need to 
set aside approximately $151.5 million. 

This report responds to your request that we determine (1) the 
soundness of the analysis GSA used to derive the estimate of funding 
that would be required for the transition and (2) whether GSA will have 
accumulated adequate funding to pay for its transition management 
costs. To accomplish the first objective, we conducted a search of over 
250 documents from both government and industry[Footnote 1] for 
examples of best practices in the field of cost estimation and 
identified common characteristics among them. We determined that high- 
quality, reliable estimates should be accurate, comprehensive, well- 
documented, and validated. Using these characteristics, we analyzed 
transition estimate documentation developed by GSA, documentation 
provided by GSA on the previous transition, the Networx Request for 
Proposals, and other relevant documents. To accomplish the second 
objective, we analyzed financial and operational documents from related 
GSA programs. In addition, we interviewed GSA program officials about 
both objectives and conducted an analysis of GSA's estimate to examine 
the effects of varying the main cost driver in the estimate. We 
conducted our work at GSA's Washington, D.C., area headquarters between 
June 2006 and January 2007 in accordance with generally accepted 
government auditing standards. A detailed discussion of our objectives, 
scope, and methodology can be found in appendix I. 

Results in Brief: 

GSA did not use sound analysis when estimating the amount of funding 
needed to meet its transition-related commitments. Specifically, its 
analysis was not sufficiently accurate, comprehensive, documented, or 
validated. A primary weakness is that the estimate is largely based on 
the assumption that 76 percent of the services provided under the 
current contracts will transition to a different provider under the 
Networx contracts. However, according to program officials, this 
assumption is intentionally conservative and represents a worst-case 
scenario that is unlikely to occur. Additionally, GSA may have double- 
counted a cost, did not update its analysis to reflect a nearly 2-year 
delay, and did not adequately document or validate its analysis. The 
weaknesses we identified can be attributed in part to the lack of a 
policy requiring cost estimates to be developed using best practices. 
Without such a policy, GSA's future cost estimates could exhibit 
similar weaknesses, increasing the risk that it will retain excess 
funds that could be reallocated for other purposes. 

GSA has accumulated adequate funding to support its anticipated 
transition costs. As of fiscal year-end 2006, GSA had approximately 
$142 million in a transition reserve to be used to pay for its costs 
associated with the transition. Our analysis of the estimate indicates 
that it is unlikely that GSA will need to accumulate the entire $151.5 
million it estimated and that the funds it has already accumulated 
should be sufficient to fund the transition. Specifically, the $142 
million already retained will be adequate to cover anticipated costs 96 
percent of the time. Further, the recent merger of two GSA funds 
increases the agency's flexibility and could provide additional money, 
if needed. With Networx contracts scheduled to be awarded starting in 
March 2007, GSA will soon have the information necessary to reassess 
the main cost driver underlying its estimate and address the weaknesses 
we identified. Once it has a current, accurate estimate, GSA can 
reevaluate the funding needed to meet anticipated commitments. 

To ensure that future cost estimates by GSA are sound and can be used 
as a reliable basis for decisions, we recommend that the GSA 
Administrator establish an agencywide policy requiring that cost 
estimates be developed using best practices. In addition, we recommend 
that the Administrator revise the transition cost estimate using best 
practices after the award of contracts under the Networx program and, 
if feasible, reallocate any excess funds for other purposes or return 
them to the Treasury. 

In written comments on a draft of this report, the GSA Administrator 
concurred with our recommendations and emphasized the importance of 
supporting a successful governmentwide telecommunications transition. 
Her comments also state that GSA's transition cost estimate was within 
2 percent of our independent analysis using the same assumptions. 
However, while we identified only $3 million in quantifiable errors 
(roughly 2 percent of GSA's total estimate), we also determined that if 
the extent of transitioning services is significantly lower than GSA's 
intentionally conservative assumption, actual costs could be more than 
$110 million less than GSA's estimate. 

In addition, the Administrator presented two main objections to our 
draft. First, she questioned whether our reported findings were 
balanced given the facts and results presented. We clarified our report 
to ensure that our findings better reflect the information we discuss. 
Second, she stated that we incorrectly suggested comparability between 
the pending Networx transition and the prior transition. We maintain 
that the two transitions are comparable, particularly because GSA's 
analysis is based, in part, on the results of the previous transition. 
Appendix III provides the full text of GSA's comments. GSA also 
provided technical comments that have been incorporated in our report 
as appropriate. 

Background: 

As part of its mission of providing federal agencies with acquisition 
services and solutions at best value, GSA's technology programs offer 
agencies options to acquire needed telecommunications services. An 
option chosen by more than 135 agencies is the FTS2001 program, which 
consists of two large governmentwide telecommunications contracts--one 
awarded to Sprint[Footnote 2] in December 1998 and the other to 
MCI[Footnote 3] in January 1999--and FTS2001 crossover 
contracts.[Footnote 4] 

GSA is planning to replace the FTS2001 contracts, FTS2001 crossover 
contracts, and separate wireless contracts with a new set of contracts. 
Collectively known as the Networx program, these new contracts are to 
provide governmentwide telecommunications services through two 
indefinite-delivery/indefinite-quantity acquisitions--Networx Universal 
and Networx Enterprise. The Universal acquisition is expected to 
satisfy the requirements for a full range of national and international 
network services and, according to GSA, to ensure the continuity of 
broad-ranging services with global geographic coverage rendered under 
expiring contracts. The Enterprise acquisition is expected to offer 
agencies leading-edge services and solutions with less extensive 
geographic and service requirements than Universal. The services 
required in these contracts focus on Internet-based offerings and 
related security and management services. 

GSA expects the transition to begin when Networx Universal awards are 
made in March 2007 and to continue until fiscal year 2010. Because the 
FTS2001 contracts with Sprint Nextel Corporation and Verizon Business 
expired in December 2006 and January 2007, respectively, GSA and the 
incumbent vendors negotiated separate sole-source contracts that 
essentially extend the terms of the FTS2001 contracts for 42 
months.[Footnote 5] These sole-source contracts were awarded to ensure 
uninterrupted service and allow agencies adequate time to complete the 
transition to Networx. 

GSA is working with representatives of federal agencies to prepare for 
the upcoming transition to Networx, both directly and through the 
Interagency Management Council (IMC), a group of senior federal 
information resource officials who advise GSA on issues related to 
telecommunications contracts. The IMC has worked with GSA to document 
lessons learned from the transition to FTS2001 that began in the late 
1990s, GSA's most recent governmentwide telecommunications transition. 
One important lesson learned was that GSA and agency plans for funding 
transition expenses should be determined early to allow agencies to 
gauge the impact of transition expenses on their budgets. Specifically, 
the lessons-learned document recommended that guidelines be established 
to allow agencies to complete the financial planning required to ensure 
that the resources needed for transition were available. This led the 
IMC and GSA to develop a Taxonomy and Allocation of Transition Costs 
document that identified which Networx transition costs would be borne 
by GSA and which would be borne by transitioning agencies. 

This taxonomy document indicated that GSA will incur or reimburse 
agencies, including: 

* GSA contractor support costs to, for example, aid in planning for the 
transition and oversight of Networx contractors. GSA officials also 
indicated that contractor support will be used to develop a methodology 
for tracking transition progress and the establishment of a transition 
coordination center; and: 

* certain costs incurred by agencies during transition. 

In 2004, following the development of the taxonomy document, GSA 
generated an estimate of its costs for the transition. GSA's 
methodology for its estimate was to: 

* develop assumptions for the estimate, 

* define and develop a baseline for the estimate based on experiences 
and lessons learned from the previous transition, 

* determine the network growth as well as the total business volume 
projections for fiscal year 2006 based on historical traffic and cost 
trends, 

* define the transition cost elements, 

* define and develop a formula for calculating an estimate for each 
cost element, and: 

* design and develop the estimated transition cost model for 
sensitivity analysis. 

Using this methodology, GSA estimated that it would need a total of 
$151.5 million for a 30-month transition, most of which would be used 
to reimburse agencies' transition costs. Table 1 below details the 
transition costs GSA expects to pay. 

Table 1: GSA Transition Cost Estimate: 

Cost element: GSA contractor support; 
Estimated cost: $35.0. 

Cost element: Certain agency transition costs; 
Estimated cost: 116.5. 

Cost element: Total; 
Estimated cost: $151.5. 

Source: GSA. 

[End of table] 

GSA planned to pay for its costs and the reimbursement of certain 
agency costs using its Information Technology (IT) Fund.[Footnote 6] 
The IT Fund was a full-cost recovery revolving fund[Footnote 7] whereby 
GSA fully recovered all costs of its technology programs and its 
operations via estimated fee rates. The fees, which are charged to 
agencies for the use of GSA contracts cover the direct costs of its 
operations--such as the development and management of contract 
vehicles--and indirect costs associated with its headquarters, such as 
support for the Offices of the Chief Information Officer and the Chief 
Financial Officer. The IT Fund allowed GSA to stabilize rates for its 
services when expenses varied and was used to provide funding for the 
previous transition to FTS2001. The IT Fund also contained a working 
capital reserve that was used to offset losses due to fluctuations in 
business volumes and other unexpected contingencies. At each fiscal 
year-end, the uncommitted balance of funds remaining was to be 
transferred to the general fund of the treasury as miscellaneous 
receipts. 

Recently, a new law changed the structure of the IT Fund and GSA's 
organization. On October 6, 2006, the General Services Administration 
Modernization Act[Footnote 8] combined the IT Fund with another GSA 
revolving fund--the General Supply Fund--to create the Acquisition 
Services Fund. This legislation, which also merged GSA's technology and 
supply programs,[Footnote 9] made all capital assets and balances 
remaining in the IT and General Supply Funds available for the purposes 
of the Acquisition Services Fund. According to GSA, the Federal 
Acquisition Service will increase agency savings, enhance GSA's 
capability to meet customer requirements for excellence, and improve 
internal efficiencies. 

GSA's Analysis Was Not Sound: 

The analysis GSA used to derive its estimate of $151.5 million was not 
sound because it was not sufficiently accurate, comprehensive, 
documented, or validated. The analysis was not sufficiently accurate 
because it is largely based on the assumption that agencies will 
transition 76 percent of the services acquired under the current 
FTS2001 contracts to a different provider under Networx--an 
intentionally conservative scenario that GSA program officials believe 
is unlikely to occur. Further, the analysis has not been updated after 
a nearly 2-year delay in the contract award. While GSA appears to have 
included all pertinent costs, it may have double-counted a cost, 
calling into question the comprehensiveness of its analysis. In 
addition, GSA did not document significant assumptions and data. 
Finally, GSA's analysis was not validated by an independent cost 
estimate nor was an uncertainty analysis performed that would allow GSA 
to quantify the level of confidence it has in its estimate. These 
weaknesses can be attributed in part to the lack of a cost estimation 
policy that would help to ensure that such estimates are developed 
using best practices. While an intentionally conservative approach 
minimizes the risk that GSA would have inadequate funds to pay for 
committed transition costs, it increases the risk that GSA will retain 
excess funds that could be used for other purposes. 

GSA's Analysis Was Not Sufficiently Accurate: 

Estimates are accurate when they are not overly conservative, based on 
an assessment of the most likely costs, and adjusted properly for 
inflation. Best practices further dictate that as schedules change, 
cost estimates should be revised to provide management with insight 
into the current program status, effective control of the program, and 
the ability to balance resources and the budget. 

The analysis GSA used to derive the estimate, however, was not 
sufficiently accurate. First, the estimate's main cost driver is based 
on the assumption that agencies will transition 76 percent of the 
services acquired under the current FTS2001 contracts to a different 
provider under Networx--60 percent due to agencies being forced to 
change providers and 16 percent due to voluntary changes. However, 
according to program officials, the 76 percent "transition traffic 
factor" is intentionally conservative and represents a worst-case 
scenario that is unlikely to occur. 

The 76 percent transition traffic factor is also overly conservative 
when compared with the previous transition. Then, approximately 60 
percent of services were shifted from an incumbent to a different 
provider under FTS2001, the bulk being forced to change providers when 
an incumbent providing more than half of the services (AT&T) was not 
awarded an FTS2001 contract. This forced shift happened in part because 
GSA limited the FTS2001 awards to only two vendors. In contrast, GSA 
has placed no such limit on the number of Networx vendors, and 
therefore, all incumbent FTS2001 vendors could potentially be awarded a 
Networx contract. Further, because the most-used FTS2001 incumbent 
(Verizon Business) provides approximately 50 percent of the services 
under the current FTS2001 contracts, the assumption of a 60 percent 
forced shift would only be realized if no awards were made to Verizon 
Business and at least one other incumbent. Finally, GSA's assumption of 
a 16 percent voluntary shift is significantly higher than the 3 percent 
that voluntarily changed providers during the previous transition. 

Program officials could not identify any basis for the assumption that 
voluntary changes will reach 16 percent. However, the officials stated 
that the percentage of voluntary changes could be higher because 
agencies will likely have more options when selecting vendors and 
because of improved guidance on federal requirements regarding the fair 
opportunity process, which is intended to give each awardee an equal 
opportunity to compete for agencies' telecommunications services 
requirements based on agency-established selection criteria. 

Second, the analysis has not been updated to reflect schedule changes. 
When the estimate was developed in 2004, GSA expected to award the 
first Networx Universal contracts in mid-2005. Since that time, delays 
have pushed the time frame back almost 2 years. Universal awards are 
now expected in March 2007. GSA's analysis has not been adjusted to 
reflect additional inflation during this period, which could increase 
the estimates total by as much as $9 million. 

Finally, GSA has not revised its analysis to reflect an assessment of 
most likely costs using currently available information. The analysis 
assumes that reimbursable agency transition costs would be greater than 
during the previous transition due to growth in the volume of services 
ordered. While GSA estimated that service levels would grow 60 percent 
over the previous transition by the time of contract award, as of 
August 2006, service levels had actually grown by 55.9 percent and are 
now expected to remain stable. The overestimation of service growth 
added approximately $1.7 million to the estimate. 

GSA's Analysis May Not Be Comprehensive: 

Estimates are comprehensive when their level of detail ensures that all 
pertinent costs are included and no costs are double-counted. It is 
important to ensure the completeness, consistency, and realism of the 
information contained in the cost estimate. 

GSA appears to have included all pertinent costs in its analysis; 
however, according to officials, a specific agency cost valued at $4.3 
million may have been double-counted. For several of the transition 
costs estimated, GSA based its calculations on the actual charges 
incurred during the previous transition. However, officials indicated 
that a specific agency cost that is estimated separately in its current 
estimate may have already been included in a more general category of 
its previous costs. Despite this uncertainty, GSA calculated the 
separate total for this specific agency cost based on current service 
levels and added it to its estimate. As a result, it is likely this 
specific agency cost is double-counted and therefore overstated. 

GSA's Analysis Was Not Adequately Documented: 

Cost estimates are well-documented when they can be easily repeated or 
updated and can be traced to original sources through auditing. 
Rigorous documentation increases the credibility of an estimate and 
helps support an organization's decision-making process. The 
documentation should explicitly identify the primary methods, 
calculations, results, rationales or assumptions, and sources of the 
data used to generate each cost element. 

GSA provided us documentation of its methodology, the calculations it 
used to derive each cost element, results, and many of the previous 
transition costs.[Footnote 10] However, it did not document significant 
assumptions. Specifically, GSA did not document the rationale behind 
its 76 percent transition traffic factor or why it used a 30-month time 
period for the transition--two key assumptions of its analysis. 

GSA also did not provide documentation of certain data sources. 
Specifically, program officials could not provide supporting data used 
to estimate an agency transition cost valued at $4.7 million. In 
addition, GSA could not document the data sources used to estimate 
costs for contractor support in planning and implementing the 
transition. While many costs in its estimate are based on the charges 
incurred during the previous transition, GSA officials stated that it 
was not appropriate to use previous costs as a basis for the contractor 
cost element. These officials explained that unlike the previous 
transition, GSA would not provide agencies with on-site contractor 
support. Officials made this decision, in part, because the 2½ years of 
transition planning that has occurred to date is expected to result in 
better preparation by agencies and the ability for them to facilitate 
their transitions without direct assistance from GSA or its 
contractors. Instead of basing their projection of contractor costs on 
prior charges, program officials told us that GSA management decided 
that contractor support costs should not exceed $35 million. Program 
officials could not provide any data or analysis to support this 
decision. 

GSA's Analysis Was Not Validated: 

Estimates are adequately validated when they have been cross-checked 
with an independent cost estimate, and when a level of uncertainty 
associated with the estimate is identified. An independent cost 
estimate provides the estimator with an unbiased test of the 
reasonableness of the estimate and reduces the cost risk associated 
with the project by demonstrating that alternate methods generate 
similar results. In performing an uncertainty analysis, an entity 
examines the effects of varying multiple elements and, as a result, is 
able to express a level of confidence in its estimate. 

GSA did not validate its analysis against an independent cost estimate 
or perform an uncertainty analysis. GSA program officials could not 
provide a rationale of why these activities were not performed. 

Quantifiable Weaknesses Are Small, but Effect of Estimate's 
Intentionally Conservative Assumption Could Be Significant: 

The cumulative effect of the quantifiable weaknesses we identified in 
GSA's analysis is relatively small, resulting in an underestimation of 
$3 million, or roughly 2 percent of the total $151.5 million estimate 
(as shown in table 2). The underestimation of costs, related to 
inflation during the extended delay in making award, was offset by the 
overestimation of service growth and the possible double-counting of a 
specific agency transition cost. 

Table 2: Cumulative Effect of Quantifiable Weaknesses: 

Dollars in millions. 

Issue identified: The estimate has not been adjusted to reflect 
schedule changes; 
Aspect of estimate affected: Calculation of inflation; 
Approximate effect: +$9.0. 

Issue identified: The estimate does not reflect an assessment of most 
likely costs due to currently available information; 
Aspect of estimate affected: Actual growth in services; 
Approximate effect: -1.7. 

Issue identified: The estimate may have double-counted a cost; 
Aspect of estimate affected: Agency transition costs; 
Approximate effect: -4.3. 

Issue identified: Cumulative effect;
Aspect of estimate affected: [Empty]; 
Approximate effect: +$3.0. 

Source: GAO analysis of GSA information. 

[End of table] 

In contrast, if the actual level of services transitioning to a 
different vendor is lower than the 76 percent transition traffic factor 
assumed by GSA, the effect will be greater. Because this factor is the 
primary cost driver in the estimate, significant changes in 
transitioning traffic result in similarly significant changes in total 
costs. This effect can be illustrated using GSA's transition cost 
estimate model (see fig. 1). For example, if a transition traffic 
factor of 66 percent is used (GSA's assumption of 16 percent for a 
voluntary transition plus a forced shift of 50 percent that would occur 
if Verizon Business is not awarded a Networx contract), the total 
estimated cost of the transition falls to about $136 million. 
Similarly, if all incumbents are awarded Networx contracts and only 
GSA's assumption of a 16 percent voluntary transition occurs, the total 
estimated cost of the transition is reduced to approximately $40 
million. GSA officials explained that they used a risk-averse 
transition traffic factor to minimize the possibility of 
underestimating costs. 

Figure 1: Estimated Cost of Transition Using Various Percents of 
Transition Traffic: 

[See PDF for image] 

Source: GSA cost estimate model. 

[End of figure] 

GSA Does Not Have a Policy to Ensure That Cost Estimates Are Sound: 

The weaknesses in GSA's analysis can be attributed in part to the lack 
of a policy requiring cost estimates to be developed using best 
practices. Officials from the Offices of the Chief Acquisition Officer, 
the Chief Information Officer, and the program office's Controller 
confirmed that GSA does not have centralized policy or guidance on cost 
estimation. Instead, the officials that prepared the estimate stated 
that they based their work loosely on best practices learned as a 
result of past experiences and were comfortable with the estimate. 
Officials believe there is no reason to revise their estimate to 
address the issues we raised in this report because their purpose in 
producing it was to minimize the risk of underestimating costs. While 
GSA's approach minimized the risk of having inadequate funds to fulfill 
its commitments, it increased the risk that GSA will retain excess 
funds that could be used for other purposes. 

GSA Has Accumulated Adequate Funding for the Transition: 

GSA has accumulated adequate funding to support its anticipated 
commitments related to the Networx transition. As of fiscal year-end 
2006, GSA accumulated approximately $142 million in a reserve dedicated 
to costs associated with the transition from FTS2001 to Networx. The 
uncertainty analysis we performed on GSA's cost estimate indicates that 
the funding GSA has already accumulated will most likely be adequate to 
pay for expected transition costs. By varying the transition traffic 
factor, our analysis indicates that the $142 million already retained 
should be adequate to cover expected expenses 96 percent of the time. 
Figure 2 illustrates the probability that a particular level of funding 
will be adequate to account for the total actual costs incurred. For 
example, the $151.5 million estimate represents a confidence level of 
100 percent: there is almost no chance that costs will exceed the 
estimate. The full detail of our methodology is detailed in appendix 
II. 

Figure 2: Confidence That Certain Amounts of Funding Will Account for 
Actual Costs: 

[See PDF for image] 

Source: GAO analysis of GSA data. 

[End of figure] 

The merger of the IT and General Supply Funds provides GSA with 
additional flexibility, further reducing the need to retain the entire 
amount of the estimate. As discussed, legislation[Footnote 11] combined 
the two funds into an Acquisition Services Fund, making their capital 
assets and balances available for the purposes of the new Federal 
Acquisition Service. The legislation also allows the Federal 
Acquisition Service to establish a reserve to retain surplus revenues 
from GSA's technology and service programs specifically for the purpose 
of offsetting losses and other unexpected contingencies. Further, at 
the end of each year, the statute requires GSA to return to the 
Treasury any funds not expended or held in a working capital reserve. 
During fiscal year 2006, GSA's technology programs experienced losses, 
but its supply programs reported overall positive earnings of 
approximately $126 million. If operations continue in this fashion, 
excess revenue will be available in the combined fund to offset losses 
or account for contingencies, such as the Networx transition, be 
retained within the Acquisition Services Fund, or be returned to the 
Treasury. 

With Networx contracts scheduled to be awarded starting in March 2007, 
GSA will soon be in a position to reassess its main assumption, the 
transition traffic factor, and the resulting level of funding needed to 
meet anticipated commitments. Unless GSA revises its estimate, it risks 
unnecessarily retaining funds that could be reallocated to other agency 
priorities or returned to the Treasury. 

Conclusions: 

While GSA achieved its goal of minimizing the risk that it would have 
inadequate funds to pay its transition commitments, the analysis used 
to develop the transition cost estimate was not sound because it was 
not sufficiently accurate, comprehensive, documented, or validated. The 
weaknesses can be attributed in part to a lack of a policy at GSA to 
ensure that such estimates are developed using best practices and make 
the best use of agency resources. While the quantifiable effect of the 
weaknesses in accuracy and comprehensiveness is small, the effect of 
potential inaccuracies resulting from the intentional use of an overly 
conservative assumption about a main cost driver could be more 
significant. Without the use of a cost estimation policy that reflects 
best practices, GSA could continue to produce similarly unsound 
estimates, increasing the risk that it will unnecessarily retain funds 
that could be reallocated for other purposes. 

Despite the weaknesses in its analysis, GSA has accumulated adequate 
funding to support its anticipated commitments related to the Networx 
transition. Our analysis indicates that it is highly unlikely that GSA 
will need more than the $142 million it has already accumulated. In 
addition, the merger of two revolving funds gives it increased 
flexibility in meeting costs. Once Networx contracts are awarded 
beginning in March 2007, GSA will be able to forecast the number of 
forced transitions more accurately, and, if necessary, reduce the 
amount of funding it plans to accumulate in the future or free already 
accumulated funds for other purposes. This reassessment will also be an 
opportunity for GSA to address the other weaknesses in its analysis 
that resulted from its deviation from best practices. 

Recommendations: 

To improve GSA's program management, we are making two recommendations. 
First, to ensure that future cost estimates are sound and can be used 
as a reliable basis for decisions, we recommend that the GSA 
Administrator establish a policy for cost estimation efforts at GSA. 
Specifically, this policy should reflect best practices by requiring 
that estimates are: 

* accurate (not overly conservative, based on an assessment of the most 
likely costs, and adjusted properly for inflation); 

* comprehensive (their level of detail ensures that all pertinent costs 
are included and no costs are double-counted); 

* well-documented (can be easily repeated or updated and can be traced 
to original sources through auditing); and: 

* validated (they have been cross-checked with an independent cost 
estimate and a level of uncertainty associated with the estimate has 
been identified.) 

Second, to ensure the most efficient use of federal funds, we recommend 
that the Administrator revise the transition cost estimate following 
the award of contracts under the Networx program. Specifically, this 
revision should reflect best practices, include a more precise 
transition traffic factor, and address the overestimation of service 
growth, the possible double-counting of a nonrecurring charge, and the 
effects of inflation during the extended delay in making awards. If the 
results of this new estimate indicate that the full $151.5 million is 
not needed to reasonably support the transition effort, GSA should 
reallocate any excess funds for other purposes allowable within the 
Acquisition Services Fund or return them to the Treasury. 

Agency Comments and Our Evaluation: 

In written comments on a draft of this report, the GSA Administrator 
concurred with our recommendations and emphasized the importance of 
supporting a successful governmentwide telecommunications transition. 
To address our recommendations, she stated that GSA will issue improved 
policy guidance on cost estimating and review and adjust the cost 
estimate as additional information becomes available. 

The Administrator also commented that GSA's transition cost estimate 
was within 2 percent of our analysis using the same assumptions. 
However, while we identified only $3 million in quantifiable errors 
(roughly 2 percent of GSA's total estimate), our report also states 
that if the extent of transitioning services is significantly lower 
than GSA's intentionally conservative assumption, the actual costs of 
the transition could be considerably less than GSA's estimate. 
Specifically, if all incumbents are awarded Networx contracts and only 
GSA's assumption of a 16 percent voluntary transition occurs, the total 
cost of the transition could be reduced to approximately $40 million-- 
over $110 million less than GSA's estimate. 

In addition, the Administrator stated that GSA does not concur with the 
entirety of the draft report. She presented two main objections. First, 
she questioned whether our reported findings were balanced given the 
facts and results presented. We clarified our report to ensure that our 
findings better reflect the information we discussed. Second, she 
raised a concern that we incorrectly suggested comparability between 
the pending Networx transition and the prior transition. We maintain 
that the two transitions are comparable, particularly because GSA's 
analysis is based, in part, on the results of the previous transition. 
Specifically, GSA's analysis relied on experiences and lessons learned 
from the previous transition to establish costs for the current 
estimate. 

Appendix III provides the full text of GSA's comments. GSA also 
provided technical comments that have been incorporated in this report, 
as appropriate. 

As agreed with your office, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies of this report 
to the GSA Administrator and interested congressional committees. We 
will also make copies available to others upon request. In addition, 
the report will be available at no charge on the GAO Web site at 
http://www.gao.gov. 

Should you or your offices have any questions about matters discussed 
in this report, please contact me at (202) 512-6240 or by e-mail at 
koontzl@gao.gov. Contact points for our Offices of Congressional 
Relations and Public Affairs may be found on the last page of this 
report. GAO staff who made major contributions to this report are 
listed in appendix IV. 

Sincerely, 

Signed by: 

Linda D. Koontz; 
Director, Information Management Issues: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

Our objectives were to determine (1) the soundness of the analysis the 
General Services Administration (GSA) used to derive the estimate of 
funding that would be required for the transition and (2) whether GSA 
will have accumulated adequate funding to pay for its transition 
management costs. 

To determine the soundness of GSA's analysis, we conducted an intensive 
search of over 250 source documents of both government and industry 
literature for examples of best practices in the field of cost 
estimation. This included literature on cost estimation from the 
Society of Cost Estimating and Analysis, the Department of the U.S. 
Army, the Office of the Secretary of Defense, the National Aeronautics 
and Space Administration, and the Department of Energy. This indicated 
that high quality, reliable cost estimates are: 

* accurate (not overly conservative, based on an assessment of the most 
likely costs, and adjusted properly for inflation); 

* comprehensive (their level of detail ensures that all pertinent costs 
are included and no costs are double-counted); 

* well-documented (can be easily repeated or updated and can be traced 
to original sources through auditing); and: 

* validated (they have been cross-checked with an independent cost 
estimate and a level of uncertainty associated with the estimate has 
been identified). 

To determine the extent to which GSA followed these practices, we 
analyzed documentation supporting the transition estimate, 
documentation provided by GSA on the previous transition estimate, and 
the Networx Request for Proposals. We also interviewed GSA Networx 
program managers and attended a GSA sponsored transition conference and 
meetings of the Interagency Management Council Transition Working 
Group.[Footnote 12] 

To address whether GSA has accumulated adequate funding to pay for its 
transition management costs, we obtained and analyzed Cost and Capital 
Requirements Plans for the Information Technology Fund submitted to the 
Office of Management and Budget and legislation for GSA's Information 
Technology, General Supply, and Acquisition Services Funds. In 
addition, to determine the extent of funding held in GSA's related 
accounts, we analyzed financial statements for GSA's technology and 
service programs. To verify the reliability of these records, we 
obtained and analyzed the results of the most recent GSA financial 
audits and audit reports from GSA's Inspector General, and we 
interviewed GSA's independent financial auditor regarding the quality 
control procedures in place. The independent auditor did not 
specifically review the dollar amounts in the Information Technology or 
General Supply Funds for accuracy but did test the controls in place 
for compliance with laws and regulations. This auditor stated that 
there were no reportable findings associated with either fund, and it 
was reasonable to assume that these accounts were fairly stated. As a 
result, we determined the data were sufficiently reliable for the 
purposes of this report. We also interviewed officials from the Office 
of Management and Budget and officials from GSA's technology and 
service programs, Office of the Chief Acquisition Officer, Office of 
the Chief Financial Officer, and Office of the Chief Information 
Officer. 

To assess the adequacy of the level of funding already accumulated by 
GSA, we performed an uncertainty analysis for GSA's estimate using a 
Monte Carlo simulation.[Footnote 13] A Monte Carlo simulation provides 
a perspective on the potential variability of the cost estimate should 
the facts, circumstances, or assumptions change. We chose to vary only 
the transition traffic factor because it is the main driver of the 
costs in GSA's estimate. To carry out this simulation, we identified a 
minimum, maximum, and median value for the transition traffic factor 
based on information received from GSA. 

We conducted our work between May 2006 and January 2007 in accordance 
with generally accepted government auditing standards. 

[End of section] 

Appendix II: Uncertainty Analysis for GSA's Transition Estimate: 

An uncertainty analysis provides decision makers with a perspective on 
the potential variability of the estimate should the facts, 
circumstances, and assumptions change. By examining the effects of 
varying the estimates elements, a degree of uncertainty about the 
estimate can be expressed, possibly as an estimated range or qualified 
by some factor of confidence. For example, an estimate that produces a 
100 percent confidence level indicates that, based on the methodology 
used to create that estimate, there is almost no chance that costs will 
exceed the estimate. 

We performed our uncertainty analysis on GSA's estimate using a Monte 
Carlo simulation.[Footnote 14] We chose to vary only the transition 
traffic factor because it is the main driver of the costs in GSA's 
estimate. To carry out this analysis, we identified a minimum, maximum, 
and median value for the transition traffic factor, based on 
information received from GSA. 

The transition traffic factor represents the possible percentage of 
services under FTS2001 that may transition to a different provider 
under the Networx contracts. For this factor, we chose a minimum of 3 
percent, which represents the voluntary shift to a different vendor 
that occurred during GSA's previous transition to FTS2001. For the 
maximum, we used 76 percent, as this value was chosen by GSA officials 
as a worst-case scenario in an effort to mitigate the risks of 
underestimating costs. The median of these two numbers is 39.5 percent. 
Table 3 shows the variations of each factor used in our uncertainty 
analysis. 

Table 3: Risk Adjusted Factors: 

Transition traffic factor; 
GSA estimate: 76%; 
Minimum: 3%; 
Median: 39.5%; 
Maximum: 76%. 

Source: GAO analysis of GSA data. 

[End of table] 

[End of section] 

Appendix III: Comments from the GSA Administrator: 

GSA Administrator: 
U.S. General Services Administration: 
1800 F Street, NW: 
Washington, DC 20405-002: 
Telephone: (202)501-0800: 
Fax: (202)-219-1243: 
www.gsa.gov: 

February 12, 2007: 

The Honorable David M. Walker: 
Comptroller General of the United States: 
Government Accountability Office: 
441 G Street NW: 
Washington, DC 20548: 

Dear Mr. Walker: -: 

Thank you for the opportunity to comment on the Government 
Accountability Office (GAO) draft report, "GSA Has Accumulated Adequate 
Funding for Transition to New Contracts but Needs Cost Estimation 
Policy" (GAO-07-268). I am pleased with GAO's finding that the General 
Services Administration (GSA) has assured the availability of adequate 
funding to support a successful Networx program transition. As you 
know, Networx is a very important Governmentwide telecommunications 
acquisition program that will allow GSA to continue to offer our 
Federal agency customers innovative services and solutions to support 
and improve the efficiency and effectiveness of their operations. 
Networx, with its substantial number of new services and features, will 
also allow agencies to operate with greater security while supporting 
the President's Management Agenda and such Presidential mandates as 
Internet Protocol Version 6 (IPv6). 

The availability of funds to support the Networx transition is 
enormously important to GSA and our customer agencies. As noted in your 
report, our paramount consideration is the ability for GSA to guarantee 
that transition funding will be available to all agencies, consistent 
with the provisions of the GSA/Interagency Management Council Taxonomy 
Document, under all possible circumstances. I appreciate your 
independent review and GAO's observation that our transition cost 
estimate, while conservative by design, was within two percent of GAO's 
estimation using the same assumptions. This confirms that GSA met the 
important goal of ensuring adequate funding for the Networx transition. 

I concur with both recommendations in the report. As you noted, one of 
the weaknesses inherent in our initial estimate is the uncertainty 
associated with many variables that will impact the cost of the 
transition to Networx. As GSA oversees and manages contract 
implementation activities, the transition cost estimate will be 
reviewed and adjustments will be made as additional information becomes 
available. GSA will also issue improved policy guidance on cost 
estimating. 

Despite the overall positive findings in your draft report, there are 
several observations in it that I believe either inaccurately represent 
the facts or indicate an incomplete understanding of conditions 
associated with the Networx transition. For example, in some instances 
the wording used to characterize the reported findings is not balanced 
with the facts presented and exaggerates their significance, given the 
results of your own independent analysis. In others, assumptions are 
made that! believe incorrectly suggest comparability between pending 
Networx transition and prior contract transition efforts. For these 
reasons, !cannot concur with the entirety of the draft report. My 
specific concerns are detailed in the enclosure. 

As always. I welcome an open dialogue with you and your staff on 
oversight activities. Your efforts are valuable in helping us improve 
the quality of the programs through which we deliver so many important 
services to our customers. 

Cordially, 

Signed by:  

Lurita Doan: 
Administrator: 

Enclosure: 

cc: Linda D. Koontz, Director: 
Information Management Issues: 
Government Accountability Office: 
441 G Street NW: 
Washington, DC 20548: 

[End of section] 

Appendix IV: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Linda D. Koontz (202) 512-6240 or koontzl@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, James R. Sweetman, Jr., 
Assistant Director; Jamey A. Collins; Neil J. Doherty; Jennifer K. 
Echard; Wilfred B. Holloway; Ethan J. Iczkovitz; Frank Maguire; Karen 
A. Richey; Glenn D. Slocum; and Amos A. Tevelow made key contributions 
to this report. 

FOOTNOTES 

[1] These documents included published literature on cost estimation 
from the Society of Cost Estimating and Analysis, the Department of the 
U.S. Army, the Office of the Secretary of Defense, the National 
Aeronautics and Space Administration, and the Department of Energy. 

[2] Sprint Corporation merged with Nextel Communications, Inc., to form 
Sprint Nextel Corporation in August 2005. 

[3] MCI merged with Verizon to form Verizon Business in January 2006. 

[4] In August 2001, GSA allowed contractors that had been awarded local 
telecommunications contracts in selected metropolitan areas, through 
GSA's Metropolitan Area Acquisition program, to offer long-distance 
services on the FTS2001 contracts. This process is termed "crossover." 

[5] GSA negotiated sole-source contracts for a 24-month base period 
with three 6-month optional periods, for a total of 42 months. 

[6] The IT Fund was authorized by the Paperwork Reduction 
Reauthorization Act of 1986, Public Law 99-500 and 99-691; 40 U.S.C. 
322. 

[7] A revolving fund is established by Congress to finance a cycle of 
businesslike operations through amounts received by the fund. A 
revolving fund charges for the sale of products or services and uses 
the proceeds to finance its spending, usually on a self-sustaining 
basis. Instead of recording the collections in receipt accounts, the 
budget records the collections and the outlays of revolving funds in 
the same account. A revolving fund is a form of permanent 
appropriation. 

[8] General Services Administration Modernization Act, Pub. L. No. 109- 
313, 120 Stat. 1734 (2006). 

[9] GSA's supply programs provide agencies a source for commercial 
products and services such as office supplies, vehicle purchasing, 
travel, and furniture. 

[10] For example, GSA provided data on contractor support costs 
incurred during the previous transition. 

[11] Public Law No. 109-313. 

[12] This included documentation available on a Transition Manager Web 
site established by GSA, such as presentations, meeting minutes, and 
FTS2001 lessons learned. 

[13] A Monte Carlo simulation allows the model's parameters to vary 
according to their associated probability distribution. The result is a 
set of estimated probabilities of achieving alternative outcomes, given 
the uncertainty in the underlying parameters. 

[14] See appendix I, footnote 2, for a description of Monte Carlo 
simulation. 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts 
newly released reports, testimony, and correspondence on its Web site. 
To have GAO e-mail you a list of newly posted products every afternoon, 
go to www.gao.gov and select "Subscribe to Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 441 G Street NW, Room LM 
Washington, D.C. 20548: 

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 
512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. 
Government Accountability Office, 441 G Street NW, Room 7125 
Washington, D.C. 20548: 

Public Affairs: 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, D.C. 20548: