This is the accessible text file for GAO report number GAO-11-6 
entitled 'Secure Border Initiative: DHS Needs to Strengthen Management 
and Oversight of Its Prime Contractor' which was released on October 
18, 2010. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Requesters: 

United States Government Accountability Office:
GAO: 

October 2010: 

Secure Border Initiative: 

DHS Needs to Strengthen Management and Oversight of Its Prime 
Contractor: 

GAO-11-6: 

GAO Highlights: 

Highlights of GAO-11-6, a report to congressional requesters. 

Why GAO Did This Study: 

The Department of Homeland Security’s (DHS) Secure Border Initiative 
Network (SBInet) is to place surveillance systems along our nation’s 
borders and provide Border Patrol command centers with the imagery and 
related tools and information needed to detect breaches and make agent 
deployment decisions. To deliver SBInet, DHS has relied heavily on its 
prime contractor. Because of the importance of effective contractor 
management and oversight to SBInet, GAO was asked to determine the 
extent to which DHS (1) defined and implemented effective controls for 
managing and overseeing the prime contractor and (2) effectively 
monitored the contractor’s progress in meeting cost and schedule 
expectations. To do this, GAO analyzed key program documentation 
against relevant guidance and best practices, and interviewed program 
officials. 

What GAO Found: 

DHS has largely defined but has not adequately implemented the full 
range of controls that is reflected in relevant guidance and related 
best practices and is needed to effectively manage and oversee its 
SBInet prime contractor. To the department’s credit, it has defined a 
number of key policies and procedures for verifying and accepting 
contract deliverables and conducting technical reviews, such as the 
criteria that need to be met before commencing and concluding a 
critical design review. Moreover, it has implemented some of these 
defined practices, such as those associated with training key 
contractor management and oversight officials. However, DHS has not 
effectively implemented other controls. For example, it has not 
adequately documented its review of contract deliverables and 
communicated to the prime contractor the basis for rejecting certain 
deliverables. Further, it has not ensured that key documentation 
satisfied relevant criteria before concluding key technical reviews. 
These weaknesses can be attributed in part to limitations in the 
defined verification and acceptance deliverable process, a program 
office decision to exclude certain deliverables from the process, and 
insufficient time to review technical review documentation. All told, 
DHS has not effectively managed and overseen its SBInet prime 
contractor, thus resulting in costly rework and contributing to SBInet’
s well-chronicled history of not delivering promised capabilities and 
benefits on time and within budget. 

DHS has not effectively monitored the SBInet prime contractor’s 
progress in meeting cost and schedule expectations. While DHS has used 
earned value management (EVM), which is a proven management approach 
for understanding program status and identifying early warning signs 
of impending schedule delays and cost overruns, it has not ensured 
that its contractor has effectively implemented EVM. In particular, 
DHS has not ensured that validated performance baselines (the 
estimated value of planned work against which performance is measured) 
were timely, complete, and accurate. For example, the contractor was 
allowed to perform work on task orders for several months without a 
validated baseline in place. Further, not all work to be performed was 
included in the baselines that were eventually established, and the 
schedules for completing this work did not substantially comply with 
six of the eight key practices that relevant guidance states are 
important to having a reliable schedule. Also, DHS regularly received 
incomplete and anomalous EVM data from the prime contractor, which it 
had to rely on to measure progress and project the time and cost to 
complete the program. As a result, DHS has not been able to gain 
meaningful and proactive insight into potential cost and schedule 
performance shortfalls, and thus take corrective actions to avoid 
shortfalls in the future. Program officials attributed these 
weaknesses in part to the instability in the scope of the work to be 
performed, and the contractor’s use of estimated, rather than actual, 
costs for subcontractor work and the subsequent adjustments that are 
made when actual costs are received. This inability has contributed to 
the program’s failure to live up to expectations and to it costing 
more and taking longer than was necessary. 

What GAO Recommends: 

GAO is making recommendations to DHS aimed at revising and 
implementing policies and procedures related to contractor 
deliverables and technical reviews, and improving EVM baselines and 
data. DHS agreed with GAO’s recommendations and described actions to 
address them, but took exception with selected findings and 
conclusions regarding EVM implementation. GAO stands by these findings 
and conclusions for reasons discussed in the report. 

View [hyperlink, http://www.gao.gov/products/GAO-11-6] or key 
components. For more information, contact Randolph C. Hite at (202) 
512-3439 or hiter@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

DHS Has Largely Defined but Has Not Effectively Implemented the Full 
Range of Needed Contractor Management and Oversight Controls: 

DHS Has Not Effectively Monitored the Contractor's Progress in Meeting 
Cost and Schedule Expectations: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Comments from the Department of Homeland Security: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Summary of SBInet Task Orders as of May 2010: 

Table 2: Percentage of Performance Measurement Baseline Work Tasks 
Categorized as Level-of-Effort for Six Baselines: 

Figures: 

Figure 1: Partial High-Level Depiction of SBI Program Organization: 

Figure 2: Partial High-Level Depiction of SBI Contracting Division 
Organization: 

Figure 3: Summary of Contract Deliverables That Did and Did Not Follow 
the Defined Process: 

Figure 4: Summary of Time Frames With and Without Validated 
Performance Baselines from June 2008 through February 2010: 

Figure 5: Summary of IBR Schedules' Satisfaction of Schedule 
Estimating Practices: 

Figure 6: Percentage of Work Breakdown Structure Elements with EVM 
Data Anomalies by Month for Each of the Four Task Orders: 

Figure 7: Total Number of Data Anomalies and Number of Explained 
Anomalies in the Monthly EVM Reports across the Four Task Orders: 

Abbreviations: 

ADTO: Arizona Deployment Task Order: 

AJO-1: Ajo Border Patrol Station: 

ANSI: American National Standards Institute: 

CBP: U.S. Customs and Border Protection: 

CDR: Critical Design Review: 

CMMI®: Capability Maturity Model Integration®: 

COP: common operating picture: 

COTR: contracting officer's technical representative: 

C3I: command, control, communications, and intelligence: 

DCMA: Defense Contract Management Agency: 

DTO: Design Task Order: 

DHS: Department of Homeland Security: 

EVM: earned value management: 

FAC-C: Federal Acquisition Certification in Contracting: 

IBR: integrated baseline review: 

IDIQ: Indefinite Delivery/Indefinite Quantity: 

IV&V: independent verification and validation: 

JPMR: Joint Program Management Review: 

NOC/SOC: Network Operations Center/Security Operations Center: 

SEP: Systems Engineering Plan: 

SBI: Secure Border Initiative: 

SBInet: Secure Border Initiative Network: 

SEI: Software Engineering Institute: 

SPO: SBInet System Program Office: 

STO: System Task Order: 

TUS-1: Tucson Border Patrol Station: 

WBS: work breakdown structure: 

[End of section] 

United States Government Accountability Office:
Washington, DC 20548: 

October 18, 2010: 

The Honorable Bennie G. Thompson: 
Chairman: 
Committee on Homeland Security: 
House of Representatives: 

The Honorable Christopher P. Carney: 
Chairman: 
The Honorable Gus M. Bilirakis: 
Ranking Member Subcommittee on Management, Investigations, and 
Oversight: 
Committee on Homeland Security: 
House of Representatives: 

The Honorable Mike Rogers: 
Ranking Member: 
Subcommittee on Emergency Communications, Preparedness, and Response: 
Committee on Homeland Security: 
House of Representatives: 

To enhance border security and reduce illegal immigration, the 
Department of Homeland Security (DHS) launched its multiyear, 
multibillion dollar Secure Border Initiative (SBI) program in November 
2005. Through SBI, DHS intends to enhance surveillance technologies, 
increase staffing levels, enforce immigration laws, and improve the 
physical infrastructure along our borders. Within SBI, the Secure 
Border Initiative Network (SBInet) is a multibillion-dollar program 
that involves the acquisition, development, integration, deployment, 
and operation and maintenance of surveillance technologies to create a 
"virtual fence" along the border as well as command, control, 
communications, and intelligence (C3I) technologies to create a 
picture of the border in command centers and vehicles. 

DHS's strategy is to deliver SBInet capabilities incrementally. In 
doing so, the department has relied heavily on its prime contractor-- 
the Boeing Company. Because of the importance of effective contractor 
management and oversight to the successful deployment and operation of 
SBInet capabilities, you asked us to determine to what extent DHS (1) 
has defined and implemented effective controls for managing and 
overseeing the SBInet prime contractor and (2) is effectively 
monitoring the prime contractor's progress in meeting cost and 
schedule expectations. 

To accomplish our objectives, we focused on DHS's management and 
oversight of four key task orders associated with designing, 
developing, and deploying the first increment of SBInet, known as 
Block 1, to its initial deployment sites in Arizona--the Tucson Border 
Patrol Station (TUS-1) and the Ajo Border Patrol Station (AJO-1). In 
doing so, we reviewed DHS and program guidance, policies, and plans 
for managing and overseeing the contractor; task orders and associated 
modifications; contract deliverable review comments; contractor 
schedules; the work breakdown structure governing all task orders; 
integrated baseline review documents for each task order; and task 
order contract cost performance reports. We also analyzed a 
nonprobability, random sample of 28 contract deliverables; a 
nonprobability, random sample of 8 technical reviews conducted with 
the contractor; and a nonprobability sample of 11 action items 
identified during program management reviews.[Footnote 1] In addition, 
we interviewed program officials about contractor management and 
oversight activities, such as verifying and accepting contract 
deliverables and conducting technical and management reviews. We also 
interviewed program officials about the prime contractor's cost and 
schedule performance. We then compared this information to relevant 
guidance and leading industry practices on contractor management and 
oversight, schedule estimating, and earned value management (EVM), 
[Footnote 2] such as the Software Engineering Institute's (SEI) 
Capability Maturity Model Integration® (CMMI®) for Acquisition, GAO's 
Cost Estimating and Assessment Guide, and the American National 
Standards Institute (ANSI) standard.[Footnote 3] 

We conducted this performance audit from June 2009 to October 2010 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. Further 
details of our objectives, scope, and methodology are in appendix I. 

Background: 

Managed by U.S. Customs and Border Protection (CBP), SBInet is to 
strengthen DHS's ability to detect, identify, classify, track, and 
respond to illegal breaches at and between ports of entry. It includes 
the acquisition, development, integration, deployment, and operations 
and maintenance of a mix of surveillance technologies, such as 
cameras, radars, sensors, and C3I technologies. Surveillance 
technologies include unattended ground sensors that can detect heat 
and vibrations associated with foot traffic and metal associated with 
vehicles, radar mounted on fixed towers that can detect movement, and 
cameras mounted on fixed towers that can be used to identify and 
classify items of interest detected and tracked by ground sensors and 
radar. These technologies are generally commercial off-the-shelf 
products. C3I technologies include customized software development to 
produce a common operating picture (COP)--a uniform presentation of 
activities within specific areas along the border--as well as the use 
of CBP network capabilities. Together, the surveillance technologies 
are to gather information along the border and transmit this 
information to COP terminals located in command centers, which are to 
assemble it to provide CBP agents with border situational awareness, 
to, among other things, enhance agents' tactical decisionmaking 
regarding potential apprehensions. 

Since fiscal year 2006, DHS has received about $4.4 billion in 
appropriations for SBI, including about $2.5 billion for physical 
fencing and related infrastructure, about $1.5 billion for virtual 
fencing (e.g., surveillance systems) and related infrastructure (e.g., 
towers), and about $300 million for program management.[Footnote 4] As 
of May 2010, DHS had obligated about $1.02 billion for SBInet. 

Program and Acquisition Organizational Structure: 

The SBI Program Management Office, which is organizationally within 
CBP, is responsible for managing key acquisition functions associated 
with SBInet, including prime contractor management and oversight. It 
is organized into four components: the SBInet System Program Office 
(SPO), Operational Integration Division, Business Operations Division, 
and Systems Engineering Division.[Footnote 5] The SPO is responsible 
for such key contractor oversight activities as verifying and 
accepting contract deliverables and conducting contractor management 
and technical reviews. In addition, the Business Operations Division 
has primary responsibility for monitoring the prime contractor's cost 
and schedule performance. As of May 2010, the SBI Program Management 
Office was staffed with 194 people--105 government employees, 78 
contractor staff, and 11 detailees. (See figure 1 for a partial SBI 
program organization chart.) 

Figure 1: Partial High-Level Depiction of SBI Program Organization: 

[Refer to PDF for image: organization chart] 

Top level: 
Program Management Office: 
* Executive Director; 
* Deputy Executive Director (reports to Executive Director). 

Second level, reporting to Executive Director: 
* Operational Integration Division; 
* Business Operations Division; 
* Systems Engineering Division. 

Second level, reporting to Executive Director: 
SBInet System Program Office: 
* Executive Director; 
* Deputy Director (reports to Executive Director). 

Third level, reporting to Deputy Director: 
* Systems Solution Division; 
* Functional Engineering and Environmental Resources Division; 
* Southwest Border Deployment Division; 
* Northern Border Division. 

Source: GAO analysis based on DHS data. 

[End of figure] 

In addition, CBP has engaged the Defense Contract Management Agency 
to, among other things, perform surveillance of the prime contractor's 
EVM, systems engineering, hardware and software quality assurance, and 
risk management. 

Further, the SBI Contracting Division, which is organizationally 
within the CBP Office of Administration Procurement Directorate's 
Enterprise Contracting Office, is responsible for performing contract 
administration activities, such as maintaining the contract file and 
notifying the contractor in writing of whether deliverables are 
accepted or rejected. The SBI Contracting Division is organized into 
three branches: SBI Contracting, SBInet System Contracting, and SBInet 
Deployment Contracting. As of May 2010, the SBI Contracting Division 
was staffed with 22 people--18 government employees and 4 contractor 
staff. (See figure 2 for a partial SBI Contracting Division 
organization chart.) 

Figure 2: Partial High-Level Depiction of SBI Contracting Division 
Organization: 

[Refer to PDF for image: organization chart] 

Top level: 
* Office of Administration Procurement Directorate: 
- Executive Director. 

Second level, reporting to Office of Administration Procurement 
Directorate: 
* Enterprise Contracting Office: 
- Deputy Executive Director. 

Third level, reporting to Enterprise Contracting Office: 
* SBI Contracting Division; 
- Division Director. 

Fourth level, reporting to SBI Contracting Division: 
* SBI Contracting Branch; 
* SBInet System Contracting Branch; 
* SBInet Deployment Contracting Branch. 

Source: GAO analysis based on DHS data. 

[End of figure] 

Overview of SBInet Acquisition Strategy and Program Status: 

In September 2006, CBP awarded an Indefinite Delivery/Indefinite 
Quantity (IDIQ)[Footnote 6] prime contract to the Boeing Company. The 
contract was for 3 base years with three additional 1-year options for 
designing, producing, testing, deploying, and sustaining SBI. In 
September 2009, CBP exercised the first option year. Under the prime 
contract, CBP has issued 11 task orders that relate to SBInet, 
covering for example, COP design and development, system deployment, 
and system maintenance and logistics support. As of May 2010, 5 of the 
11 task orders were complete and 6 were ongoing. (See table 1 for a 
summary of the SBInet task orders.) 

Table 1: Summary of SBInet Task Orders as of May 2010: 

Completed task orders: 

Task order: Program Management; 
Description: Completed task orders: Mission engineering, facilities 
and infrastructure, systems engineering, test and evaluation, and 
program management services; 
Period of performance: Completed task orders: Sept. 2006-Apr. 2008; 
Approximate contract value: $146.9 million; 
Approximate contract obligation: $146.9 million; 
Contract type: Cost-plus-fixed-fee[A]. 

Task order: Project 28; 
Description: Completed task orders: Prototype along 28 miles of the 
border in the Tucson sector; 
Period of performance: Completed task orders: Oct. 2006-Feb. 2008; 
Approximate contract value: $20.7 million; 
Approximate contract obligation: $20.7 million; 
Contract type: Firm-fixed-price. 

Task order: Project 28 Contractor Maintenance Logistics and Support; 
Description: Completed task orders: Project 28 operational maintenance 
and logistics support; 
Period of performance: Completed task orders: Dec. 2007-Dec. 2009; 
Approximate contract value: $10.6 million; 
Approximate contract obligation: $10.6 million; 
Contract type: Cost-plus-fixed-fee. 

Task order: Design for Buffalo Sector; 
Description: Completed task orders: SBInet design of remote video 
surveillance system capability for the Buffalo sector; 
Period of performance: Completed task orders: Feb. 2009-July 2009; 
Approximate contract value: $0.6 million; 
Approximate contract obligation: $0.6 million; 
Contract type: Firm-fixed-price[B]. 

Task order: System; 
Description: Completed task orders: Program management and systems 
engineering activities required to integrate all task orders; 
Period of performance: Completed task orders: Apr. 2008-May 2010; 
Approximate contract value: $222.1 million; 
Approximate contract obligation: $215.9 million; 
Contract type: Cost-plus-award-fee. 

Ongoing task orders: 

Task order: Command, Control, Communications, and Intelligence (C3I) 
Common Operating Picture (COP); 
Description: Completed task orders: Design, development, 
demonstration, and operations and maintenance of a functional C3I/COP 
system; 
Period of performance: Completed task orders: Dec. 2007-June 2010; 
Approximate contract value: $70.5 million[C]; 
Approximate contract obligation: $79.3 million; 
Contract type: Cost-plus-award-fee/cost-plus-fixed-fee/firm-fixed-
price[D]. 

Task order: Design; 
Description: Completed task orders: Design of deployment solution, 
environmental clearance support, and other location-related work for 
the Tucson sector; 
Period of performance: Completed task orders: Aug. 2007-July 2010; 
Approximate contract value: $115.0 million; 
Approximate contract obligation: $115.0 million; 
Contract type: Cost-plus-fixed-fee. 

Task order: Arizona Deployment; 
Description: Completed task orders: Deployment to two sites covering 
approximately 53 miles of the southwest border in the Tucson sector; 
Period of performance: Completed task orders: June 2008-May 2011; 
Approximate contract value: 148.5; 
Approximate contract obligation: 148.5; 
Contract type: Cost-plus-incentive-fee/cost-plus-award-fee[E]. 

Task order: Integrated Logistics Support; 
Description: Completed task orders: Maintenance logistics and 
operational support; 
Period of performance: Completed task orders: Aug. 2008-Sept. 2010; 
Approximate contract value: $61.6 million; 
Approximate contract obligation: $61.6 million; 
Contract type: Cost-plus-fixed-fee[F]. 

Task order: Northern Border Project; 
Description: Completed task orders: Design, installation, and 
deployment of surveillance capabilities in the Detroit and Buffalo 
sectors; 
Period of performance: Completed task orders: Mar. 2009-Dec. 2010; 
Approximate contract value: $22.7 million; 
Approximate contract obligation: $22.7 million; 
Contract type: Fixed-price[B]. 

Task order: Systems Support; 
Description: Completed task orders: Program management and systems 
engineering activities required to integrate all task orders; 
C3I/COP maintenance; 
Period of performance: Completed task orders: May 2010-Mar. 2011; 
Approximate contract value: $28.6 million; 
Approximate contract obligation: $28.6 million; 
Contract type: Cost-plus-award-fee. 

Total: 
Approximate contract value: $847.80 million; 
Approximate contract obligation: $850.40 million. 

Source: GAO analysis of DHS data. 

Note: "Fixed-price" types of contracts provide for a firm price or, in 
appropriate cases, an adjustable price; "firm-fixed-price" contracts 
provide for a price not subject to adjustment on the basis of the 
contractor's experience in performing the contract; "cost-plus- 
incentive-fee" contracts provide for the reimbursement of allowable 
costs plus an initially negotiated fee, to be adjusted later based on 
the relationship of total allowable costs to total target costs; "cost-
plus-award-fee" contracts provide for the reimbursement of allowable 
costs plus a base fee fixed at the contract's inception (which may be 
zero) and an award amount that the government determines to be 
sufficient to motivate excellence in performance; and "cost-plus-fixed-
fee" contracts provide for the reimbursement of allowable costs plus a 
negotiated fee fixed at the inception of the contract. 

[A] The initial contract type of the task order was a cost-plus-award- 
fee. A final award fee determination did not take place because of 
scope and schedule changes. In lieu of the final award fee 
determination, the contract type was changed to a cost-plus-fixed-fee. 

[B] The travel component of the task order is cost reimbursable. 

[C] According to SBI Contracting Division officials, the contract 
value with the cost overruns is $79.3 million. 

[D] The initial contract type of the task order was cost-plus-award- 
fee. On July 31, 2009, additional development work was definitized as 
a cost-plus-fixed-fee structure. Further, on September 30, 2009, the 
software operations and maintenance component of the task order was 
changed to a firm-fixed-price structure. 

[E] The initial contract type of the task order was a cost-plus- 
incentive-fee. On November 20, 2009, the performance and schedule 
incentives component of the task order was changed to a cost-plus- 
award-fee. The cost incentives component remains a cost-plus-incentive-
fee structure. 

[F] The initial contract type of the task order was cost-plus- 
incentive-fee. On November 6, 2009, future work under the task order 
was changed to a cost-plus-fixed-fee structure. 

[End of table] 

Through the task orders, CBP's strategy is to deliver SBInet 
capabilities incrementally. To accomplish this, the program office 
adopted an evolutionary system life cycle management approach in which 
system capabilities are to be delivered to designated locations in a 
series of discrete subsets of system functional and performance 
capabilities that are referred to as blocks. The first block (Block 1) 
includes the purchase of commercially available surveillance systems, 
development of customized COP systems and software, and use of 
existing CBP communications and network capabilities. According to 
program officials, as of July 2010, the Block 1 design was deployed to 
TUS-1 and was being deployed to AJO-1, both of which are located in 
CBP's Tucson Sector of the southwest border. Together, these two 
deployments cover 53 miles of the 1,989-mile-long southern border. 
Also, according to program officials, as of July 2010, TUS-1 and AJO-1 
were to be accepted in September 2010 and December 2010, respectively. 

In January 2010, the Secretary of Homeland Security ordered an 
assessment of the SBI program. In addition, on March 16, 2010, the 
Secretary froze fiscal year 2010 funding for any work on SBInet beyond 
TUS-1 and AJO-1 until the assessment is completed. Also at that time, 
the Secretary redirected $35 million that had been allocated to SBInet 
Block 1 deployment to other tested and commercially-available 
technologies, such as truck-mounted cameras and radar, called Mobile 
Surveillance Systems, to meet near-term needs.[Footnote 7] According 
to the SBI Executive Director, the department's assessment would 
consist of a comprehensive and science-based assessment to determine 
if there are alternatives to SBInet that may more efficiently, 
effectively, and economically meet U.S. border security needs. 
Further, the Executive Director stated that if the assessment suggests 
that the SBInet capabilities are worth the cost, DHS will extend its 
deployment to sites beyond TUS-1 and AJO-1. However, if the assessment 
suggests that alternative technology options represent the best 
balance of capability and cost-effectiveness, DHS will redirect 
resources to these other options. According to program officials, the 
initial phase of the assessment, which addresses the Arizona border, 
was completed in July 2010, and the results are currently being 
reviewed by senior DHS management. Officials were unable to provide a 
date for completion of the review. 

Use of Acquisition Management Disciplines Is Important to Programs 
Like SBInet: 

Our research and evaluations of information technology programs have 
shown that delivering promised system capabilities and benefits on 
time and within budget largely depends on the extent to which key 
acquisition management disciplines are employed. These disciplines 
include, among others, requirements management, risk management, and 
test management. As is discussed in the following section, we have 
previously reported on the extent to which these disciplines have been 
employed on SBInet. 

Contractor management and oversight, which is the focus of this 
report, is another acquisition management discipline. Among other 
things, this discipline helps to ensure that the contractor performs 
the requirements of the contract and the government receives the 
service or product intended. DHS acquisition policies and guidance, 
along with other relevant guidance, recognize the importance of these 
management and oversight disciplines. As we have previously reported, 
not employing them increases the risk that system acquisitions will 
not perform as intended and will require expensive and time-consuming 
rework.[Footnote 8] 

GAO Has Previously Reported on Numerous SBInet Acquisition Management 
Weaknesses and Risks: 

Since 2007, we have identified a range of management weaknesses and 
risks facing SBInet, and we have made a number of recommendations to 
address them that DHS has largely agreed with, and to varying degrees, 
taken actions to address. For example, in February 2007, we reported 
that DHS had not fully defined activities, milestones, and costs for 
implementing the program or demonstrated how program activities would 
further the strategic goals and objectives of SBI.[Footnote 9] 
Further, we reported that the program's schedule contained a high 
level of concurrency among related tasks and activities, which 
introduced considerable risk. Accordingly, we recommended that DHS 
define explicit and measurable commitments relative to, among other 
things, program capabilities, schedules, and costs, and re-examine the 
level of concurrency in the schedule and adjust the acquisition 
strategy appropriately. 

In September 2008, we reported that a number of program management 
weaknesses put SBInet at risk of not performing as intended and taking 
longer and costing more to deliver than was necessary.[Footnote 10] 
Specifically, we reported the following: 

* Important aspects of SBInet were ambiguous and in a continued state 
of flux, making it unclear and uncertain what technology capabilities 
were to be delivered when. For example, the scope and timing of 
planned SBInet deployments and capabilities had continued to change 
since the program began and remained unclear. Further, the SPO did not 
have an approved integrated master schedule to guide the execution of 
the program, and our assimilation of available information indicated 
that key milestones continued to slip. This schedule-related risk was 
exacerbated by the continuous change in and the absence of a clear 
definition of the approach used to define, develop, acquire, test, and 
deploy SBInet. Accordingly, we concluded that the absence of clarity 
and stability in these key aspects of SBInet impaired the ability of 
Congress to oversee the program and hold DHS accountable for results, 
and it hampered DHS's ability to measure the program's progress. 

* SBInet requirements had not been effectively defined and managed. 
While the SPO had issued guidance that defined key practices 
associated with effectively developing and managing requirements, the 
guidance was developed after several key activities had been 
completed. In the absence of this guidance, the SPO had not 
effectively performed key requirements development and management 
practices, such as ensuring that different levels of requirements were 
properly aligned. As a result, we concluded that the risk of SBInet 
not meeting mission needs and performing as intended was increased, as 
were the chances of the system needing expensive and time-consuming 
rework. 

* SBInet testing was not being effectively managed. For example, the 
program office had not tested the individual system components to be 
deployed to initial locations, even though the contractor had 
initiated integration testing of these components. Further, its test 
management strategy did not contain, among other things, a clear 
definition of testing roles and responsibilities or sufficient detail 
to effectively guide planning for specific test events. 

To address these issues, we recommended that DHS assess and disclose 
the risks associated with its planned SBInet development, testing, and 
deployment activities and that it address the system deployment, 
requirements management, and testing weaknesses that we had 
identified. DHS largely agreed to implement our recommendations. 

In September 2009, we reported that SBInet had continued to experience 
delays.[Footnote 11] For example, deployment to the entire southwest 
border had slipped from early 2009 to 2016, and final acceptance of 
TUS-1 and AJO-1 had slipped from November 2009 and March 2010 to 
December 2009 and June 2010, respectively. We did not make additional 
recommendations at that time. 

In January 2010, we reported that SBInet testing was not being 
effectively managed.[Footnote 12] Specifically, we reported the 
following: 

* Test plans, cases, and procedures for the most recent test events 
were not defined in accordance with important elements of relevant 
guidance. Further, a large percentage of the test cases[Footnote 13] 
for these events were changed extemporaneously during execution. While 
some of the changes were minor, others were more significant, such as 
rewriting entire procedures and changing the mapping of requirements 
to test cases. Moreover, these changes to procedures were not made in 
accordance with documented quality assurance processes, but rather 
were based on an undocumented understanding that program officials 
said they established with the contractor. Compounding the number and 
significance of changes were questions raised by the SPO and a support 
contractor about the appropriateness of some changes, noting that some 
changes made to system qualification test cases and procedures 
appeared to be designed to pass the test instead of being designed to 
qualify the system. 

* About 1,300 SBInet defects had been found from March 2008 through 
July 2009, with the number of new defects identified during this time 
generally increasing faster than the number being fixed--a trend that 
is not indicative of a system that is maturing and ready for 
deployment. While the full magnitude of these unresolved defects was 
unclear because the majority were not assigned a priority for 
resolution, some of the defects that had been found were significant. 
Although DHS reported that these defects had been resolved, they had 
nevertheless caused program delays, and related problems had surfaced 
that continued to impact the program's schedule. 

In light of these weaknesses, we recommended that DHS (1) revise the 
program's overall test plan; (2) ensure that test schedules, plans, 
cases, and procedures are adequately reviewed and approved consistent 
with the revised test plan; (3) ensure that sufficient time is 
provided for reviewing and approving test documents prior to beginning 
a given test event; and (4) triage the full inventory of unresolved 
system problems, including identified user concerns, and periodically 
report on their status to CBP and DHS leadership. DHS fully agreed 
with the last three recommendations and partially agreed with the 
first. 

In May 2010, we raised further concerns about the program and called 
for DHS to reconsider its planned investment in SBInet.[Footnote 14] 
Specifically, we reported the following: 

* DHS had defined the scope of the first incremental block of SBInet 
capabilities that it intended to deploy and make operational; however, 
these capabilities and the number of geographic locations to which 
they were to be deployed continued to shrink. For example, the 
geographic "footprint" of the initially deployed capability has been 
reduced from three border sectors spanning about 655 miles to two 
sectors spanning about 387 miles. 

* DHS had not developed a reliable integrated master schedule for 
delivering the first block of SBInet. Specifically, the schedule did 
not sufficiently comply with seven of nine key practices that relevant 
guidance states are important to having a reliable schedule. For 
example, the schedule did not adequately capture all necessary 
activities, assign resources to them, and reflect schedule risks. As a 
result, it was unclear when the first block was to be completed, and 
continued delays were likely. 

* DHS had not demonstrated the cost-effectiveness of Block 1. In 
particular, it had not reliably estimated the costs of this block over 
its entire life cycle. Further, DHS had yet to identify expected 
quantifiable and qualitative benefits from this block and analyze them 
relative to costs. Without a meaningful understanding of SBInet costs 
and benefits, DHS lacks an adequate basis for knowing whether the 
initial system solution is cost effective. 

* DHS had not acquired the initial SBInet block in accordance with key 
life cycle management processes. While processes associated with, 
among other things, requirements development and management and risk 
management, were adequately defined they were not adequately 
implemented. For example, key risks had not been captured in the risk 
management repository and thus had not been proactively mitigated. As 
a result, DHS is at increased risk of delivering a system that does 
not perform as intended. 

We concluded that it remains unclear whether the department's pursuit 
of SBInet is a cost-effective course of action, and if it is, that it 
will produce expected results on time and within budget. Accordingly, 
we recommended that DHS (1) limit near-term investment in the first 
incremental block of SBInet, (2) economically justify any longer-term 
investment in SBInet, and (3) improve key program management 
disciplines. DHS largely agreed with our recommendations, and noted 
ongoing and planned actions to address each of them. 

In June 2010, we reported several technical, cost, and schedule 
challenges facing SBInet, such as problems with radar functionality, 
and noted that the program office was staffed substantially below 
planned levels for government positions.[Footnote 15] We did not make 
any additional recommendations at that time. 

DHS Has Largely Defined but Has Not Effectively Implemented the Full 
Range of Needed Contractor Management and Oversight Controls: 

Federal regulations and relevant guidance recognize effective 
contractor management and oversight as a key acquisition management 
discipline. In addition, they describe a number of practices 
associated with it, including training persons who are responsible for 
contractor management and oversight, verifying and deciding whether to 
accept contract deliverables, conducting technical reviews with the 
contractor to ensure that products and services will satisfy user 
needs, and holding management reviews with the contractor to monitor 
contractor performance.[Footnote 16] 

To DHS's credit, it has largely defined key practices aimed at 
employing each of these contractor management and oversight controls. 
Moreover, it has implemented some of them, such as training key 
contractor management and oversight officials and holding management 
reviews with the prime contractor. However, it has not effectively 
implemented others, such as documenting contract deliverable reviews 
and using entry and exit criteria when conducting technical reviews. 
Reasons for these weaknesses include limitations in the defined 
verification and acceptance deliverable process and a SPO decision to 
exclude some deliverables from the process, and insufficient time to 
review technical review documentation. Without employing the full 
range of practices needed to effectively manage and oversee the prime 
contractor, DHS is limited in its ability to know whether the 
contractor is meeting performance and product expectations. Moreover, 
it increases the chances that SBInet will not function as intended and 
will take more time and resources than necessary to deliver, which we 
have previously reported is the case for Block 1. 

SBInet Contractor Management and Oversight Officials Have Been 
Properly Trained: 

Training supports the successful performance of relevant activities 
and tasks by helping to ensure that the responsible people have the 
necessary skills and expertise to perform the tasks. According to 
relevant guidance, organizations should define training expectations 
and should ensure that these expectations are met for individuals 
responsible for contractor management and oversight.[Footnote 17] 

DHS's acquisition management directives define training requirements 
for, among others, program managers, contracting officers, and 
contracting officer's technical representatives (COTR).[Footnote 18] 
Specifically: 

* Program managers must be certified at a level commensurate with 
their acquisition responsibilities. For a Level 1 information 
technology program,[Footnote 19] like SBInet, the designated program 
manager must be certified at a program management Level 3.[Footnote 20] 

* Contracting officers must be certified at the appropriate level to 
support their respective warrant authority.[Footnote 21] 

* COTRs must be trained and certified within 60 days of appointment to 
a contract or task order. The minimum mandatory requirements are the 
completion of 40 hours of COTR training and a 1-hour procurement 
ethics training course. 

For SBInet, CBP has ensured that people in each of these key positions 
have been trained in accordance with DHS requirements. Specifically, 
the two program managers associated with the development and 
deployment of SBInet Block 1--the SBI Executive Director and the SPO 
Executive Director--were issued training certifications that qualify 
each as an Acquisition Professional. Further, each contracting officer 
responsible for executing actions on the Arizona Deployment Task Order 
(ADTO), the Design Task Order (DTO), the C3I/COP task order, and the 
System Task Order (STO) between June 2008 and February 2010 was 
certified commensurate with his or her respective warrant authority. 
In addition, for the same time period, each COTR assigned to each of 
the four task orders received DHS-issued certificates indicating that 
each had met the minimum training requirements before being assigned 
to a task order. 

According to CBP officials, DHS leadership has made contractor 
management and oversight training a high priority, which helped to 
ensure that key officials were trained. By doing so, CBP has 
established one of the key controls to help ensure that prime 
contractor products and services meet expectations. 

DHS Has Not Effectively Verified and Accepted All Contract 
Deliverables: 

Effectively implementing a well-defined process for verifying and 
accepting prime contractor deliverables is vital to SBInet's success. 
DHS has a defined process for verifying and accepting contract 
deliverables, but this process does not ensure that deliverable 
reviews are sufficiently documented. Further, while the SPO has 
followed key aspects of this process, it has not effectively 
documented its review of certain deliverables and has not effectively 
communicated to the prime contractor the basis for rejecting all 
deliverables. Reasons for not doing so include limitations in the 
defined verification and acceptance process and a SPO decision to 
exclude some deliverables from the process. Without documenting all 
its reviews and effectively communicating the review results to the 
contractor, the SPO has increased the chances of accepting 
deliverables that do not meet requirements and having the contractor 
repeat work to correct deliverable problems. 

CBP Has a Defined Process for Verifying and Accepting SBInet Contract 
Deliverables: 

The purpose of contractor deliverable verification and acceptance is 
to ensure that contractor-provided products and services meet 
specified requirements and otherwise satisfy the terms of the 
contract. According to relevant guidance, organizations should have 
written policies and procedures for verifying and accepting 
deliverables that, among other things, (1) assign roles and 
responsibilities for performing verification and acceptance tasks and 
(2) provide for conducting and documenting deliverable reviews and for 
effectively communicating to the contractor deliverable acceptance and 
rejection decisions.[Footnote 22] 

To its credit, CBP has defined policies and procedures for verifying 
and accepting SBInet deliverables. Specifically, it issued its 
Deliverable Review and Approval Process in July 2007, which specifies 
how the SPO is to receive, review, and respond to all contract 
deliverables. Among other things, this guide assigns verification and 
acceptance roles and responsibilities to key program management 
positions. For example, it assigns the project manager[Footnote 23] 
responsibility for overseeing the review process and determining the 
acceptability of the deliverables, designates reviewers[Footnote 24] 
responsibilities for examining the deliverable, and assigns the 
contracting officer responsibility for communicating the decision to 
accept or reject the deliverable to the contractor. In addition, it 
provides for conducting deliverable reviews, which according to 
program officials, involves comparing the deliverable to the 
requirements enumerated in the applicable task order statement of work 
(typically within the Data Item Description). The process further 
specifies that the decision to accept or reject the deliverable is to 
be communicated in writing to the contractor. 

However, the process does not state that the results of all reviews 
are to be documented. Instead, its states that a deliverable review 
comment form is to be prepared only when deficiencies or problems 
exist. If the deliverable is acceptable, the form does not need to be 
prepared. Program officials could not explain why review documentation 
was not required for acceptable deliverables. As a result, and as 
discussed in the next section of this report, the SPO cannot 
demonstrate its basis for accepting a number of deliverables, which in 
turn has increased the risk of, and actually resulted in, deliverables 
being accepted that do not meet requirements. 

CBP Has Not Fully Followed its Defined Process for Verifying and 
Accepting SBInet Deliverables: 

The SPO followed its process for verifying and accepting SBInet-
related deliverables about 62 percent of the time, based on the 29 
deliverables that we reviewed.[Footnote 25] Specifically, the process 
was fully followed for 18 of the deliverables: (1) 6 that were 
accepted without documented review comments, (2) 5 that were accepted 
with documented review comments, and (3) 7 that were rejected with 
documented review comments. In addition, the acceptance or rejection 
of all 18 deliverables was communicated in writing to the contractor. 
For example, the ADTO Security Plan Addendum was delivered to the SPO 
in August 2008. The SPO reviewed the plan and documented its review 
comments, which included a determination that the plan did not address 
all required items specified in the task order's Data Item 
Description. The CBP contracting officer subsequently notified the 
contractor in writing that the plan was rejected for this and other 
reasons. In February 2009, the contractor resubmitted the plan, the 
SPO reviewed the plan and documented its comments, and the contracting 
officer again notified the contractor in writing that the plan was 
rejected. The contractor resubmitted the plan in May 2009, and program 
officials documented their review of the deliverable. The contracting 
officer subsequently communicated the deliverable's acceptance in 
writing to the contractor. (Figure 3 summarizes the number of contract 
deliverables that did and did not follow the defined process.) 

Figure 3: Summary of Contract Deliverables That Did and Did Not Follow 
the Defined Process: 

[Refer to PDF for image: 3 pie-charts] 

Deliverables following process: 18; 
Accepted with comments and with letter: 5; 
Accepted without comments and with letter: 6; 
Rejected with comments and with letter: 7. 

Deliverables not following process: 11; 
Accepted without comments and without letter: 5; 
Rejected without comments and without letter: 3; 
Decision not made, no comments, and no letter: 3. 

Source: GAO analysis of DHS data. 

[End of figure] 

The remaining 11 deliverables, however, did not fully adhere to the 
defined process. Of these, five were accepted without any documented 
review comments and without communicating the acceptance in writing to 
the contractor. The following are examples of these five and reasons 
for not adhering to the process. 

* Three of the five deliverables related to the C3I/COP task order did 
not require government approval. However, the Deliverable Review and 
Approval Process document does not provide for such treatment of these 
deliverables, and thus this practice is not in accordance with the 
process. Program officials told us that they have since recognized 
that this was a poor practice, and they have modified the task order 
to now require approval of all C3I/COP task order deliverables. 

* One of the five deliverables relates to the STO and is for the C2 
Component Qualification Test package, which includes, among other 
things, the test plan and test cases and procedures for conducting the 
C2 qualification test event. In this case, the SPO accepted the test 
plan because, according to program officials, several days had passed 
since the deliverable was received, and they had not received any 
comments from the reviewers. They said that they therefore accepted 
the deliverable on the basis of not receiving any review comments to 
the contrary, but did not notify the contractor in writing of the 
acceptance. 

The 11 deliverables also include 3 that were rejected without 
documented review comments and without the rejection being 
communicated to the contractor in writing. The following are examples 
of these three and reasons for not adhering to the process are 
discussed below. 

* One of the three deliverables relates to the C3I/COP task order and 
is for the Network Operations Center/Security Operations Center (NOC/ 
SOC)[Footnote 26] Test Plan/Procedures/Description. According to 
program officials, the contractor did not submit the plan on time, 
thus requiring them to review it during the readiness review.[Footnote 
27] Based on this review, the plan was rejected, which was 
communicated verbally to the contractor during the review. Despite 
rejecting the plan, the program office began testing the NOC/SOC 
component on the day of the review, without a revised plan being 
submitted, reviewed, and approved. According to program officials, 
this occurred in part because of insufficient time and resources to 
review contractor test-related deliverables. 

* Another one of the three deliverables also relates to the C3I/COP 
task order, and is for the Release 0.5 Software Test Plan/Procedures/ 
Description. According to program officials, the contractor submitted 
the plan late. The program office rejected the plan and provided oral 
comments during a teleconference prior to the review. Nevertheless, 
the test event again occurred without a revised plan being submitted, 
reviewed, and accepted. According to program officials, this was also 
due to insufficient time and resources to review the test plan. In 
this case, the plan was approved in late April 2009, which was 5 
months after the test event was conducted. 

The 11 deliverables also include 3 for which a decision to accept or 
reject the deliverable was not made. See the following examples: 

* One of the three relates to the C3I/COP task order, and is for the 
NOC/SOC Interface Control Document. For this deliverable, review 
comments were not documented and no written communication with the 
contractor occurred. The deliverable was subsequently submitted three 
times and ultimately accepted. However, program officials could not 
explain whether the initial deliverable was accepted or rejected, or 
why the deliverable was submitted multiple times. 

* Another one of these relates to STO, and is for the Dynamic Object- 
Oriented Requirements System.[Footnote 28] For this deliverable, 
review comments were not documented, but CBP communicated in writing 
to the contractor that it was withholding comment on this submission 
of the deliverable and was to provide a consolidated set of comments 
on the subsequent submission. Subsequently, the contractor resubmitted 
the deliverable, and because it was accepted, the review was not 
documented. The contracting officer communicated the deliverable's 
acceptance in writing to the contractor. 

By not effectively verifying and accepting contractor deliverables, 
the SPO cannot ensure that the deliverables will satisfy stated 
requirements, thus increasing the risk of costly and time-consuming 
rework. For example, we recently reported that contractor-delivered 
test plans were poorly defined and resulted in problems during 
testing. In particular, NOC/SOC testing was hampered by requirements 
incorrectly mapped to test cases, did not provide for testing all 
requirements, and required significant extemporaneous changes to test 
cases during the test events.[Footnote 29] As a result of the testing 
problems, the SPO had to conduct multiple test events. 

Key Contract Technical Reviews Have Not Been Adequately Conducted: 

Technical reviews are performed throughout the project life cycle to 
confirm that products and services being produced by the contractor 
provide the desired capability and ultimately satisfy user needs. To 
its credit, DHS has defined a process for conducting technical 
reviews, but it has not effectively implemented it. In particular, the 
SPO did not ensure that all key documentation was reviewed and 
relevant criteria were satisfied before concluding key technical 
reviews. Program officials attributed these limitations to the 
program's aggressive schedule, which resulted in insufficient time to 
review relevant documentation. Concluding technical reviews without 
adequate justification has resulted in schedule delays and costly 
rework. 

CBP Has a Defined Process for Conducting Technical Reviews: 

According to relevant guidance, organizations should have written 
policies and procedures for conducting technical reviews that, among 
other things, (1) assign roles and responsibilities for performing the 
specific technical review tasks and (2) establish entry and exit 
criteria to determine the readiness of the technical solution to 
proceed to the technical review and to demonstrate and confirm 
completion of required accomplishments.[Footnote 30] 

To its credit, DHS has policies and guidance for conducting technical 
reviews. Specifically, DHS's Systems Engineering Life Cycle outlines 
the key reviews to be performed as well as how these reviews are 
aligned with the department's governance process.[Footnote 31] In 
addition, DHS guidance defines expectations for technical review exit 
criteria, stating that compliance with exit criteria is based upon the 
satisfaction of the content of the criteria, and not upon only the 
delivery of specified documents. 

To augment DHS policy and guidance, the SBInet Systems Engineering 
Plan (SEP), dated November 2008, identifies and describes the 
technical reviews to be conducted. They include, for example: 

* Requirements Review, which is to ensure that requirements have been 
completely and properly identified and are understood by the SPO and 
the contractor. Documentation associated with this review is to 
include, among other things, a requirements traceability matrix (i.e., 
a tool for demonstrating that component-level requirements are 
traceable to higher-level system level requirements). 

* Critical Design Review (CDR), which is to (1) demonstrate that the 
designs are complete and baselined and (2) ensure that the solution is 
ready for fabrication, coding, assembly, and integration. 
Documentation for this review is to include, among other things, (1) 
baselined requirements, (2) interface descriptions, and (3) identified 
risks and mitigation plans. 

* Test Readiness Review, which is to assess the readiness of the 
system solution to begin formal testing. Documentation for this review 
is to include, among other things, (1) test plans that include test 
cases and procedures and (2) a traceability matrix that maps each 
requirement to be tested to a corresponding test case. 

In addition, the SBInet SEP describes high-level roles and 
responsibilities for performing these reviews, and establishes entry 
and exit criteria for each. For example, it states that the SPO 
program manager and the chief engineer are responsible for leading the 
reviews. Further, the SEP defines entry and exit criteria for the CDR. 
For example: 

* Entry. System-level requirements should be traceable to component- 
level requirements; system internal and external interfaces should be 
defined. 

* Exit. Design baseline should be established and balanced across 
cost, schedule, performance, and risk considerations over the 
investment's life cycle; system risks should be identified and 
mitigation plans should be in place. 

Contractor Technical Reviews Have Not Been Performed Adequately: 

The SPO did not follow the defined process for conducting technical 
reviews. Instead, program officials told us that they used the 
requirements defined in the respective task orders to guide each 
review. However, the task orders do not define entry and exit 
criteria. Rather, they list a set of documents that the contractor is 
to provide and the SPO is to review. For example, for the Block 1 CDR, 
the relevant task order requires that the contractor deliver, among 
other documents, (1) baselined component and system requirements, (2) 
interface descriptions (i.e., descriptions of the data to be exchanged 
and the protocols used to exchange the data), and (3) all identified 
risks and mitigation plans for those risks. However, the task orders 
do not associate these documents with either entry or exit criteria, 
and they do not specify characteristics or qualities that the 
documents are to satisfy. 

Without explicit entry and exit criteria, the basis for beginning and 
ending the technical reviews is unclear, thus increasing the risk that 
a program will be allowed to proceed and begin the next phase of 
development before it is ready to do so. In fact, this risk was 
realized for SBInet. Technical reviews were concluded without adequate 
justification, which ultimately resulted in problems that required 
additional time and resources to fix. For example: 

* NOC/SOC Requirements Review. At this review, the contractor did not 
deliver a requirements traceability matrix, as required by the 
relevant task order, until almost a month after the review was 
completed. Nonetheless, program officials stated that they concluded 
the review in June 2008, without knowing whether the applicable higher-
level system requirements were fully satisfied. 

* Block 1 CDR. For this review, the contractor delivered (1) the 
baselined component and system requirements, (2) the interface 
descriptions, and (3) all identified risks and mitigation plans for 
those risks. 

However, these deliverables did not demonstrate that all component- 
level requirements were baselined and interface descriptions were 
understood. As we previously reported, baselined requirements 
associated with the NOC/SOC were not adequately defined at the time of 
the CDR, as evidenced by the fact that they were significantly changed 
2 months later.[Footnote 32] Program officials stated that while they 
knew that requirements were not adequately baselined at this review, 
they believed that the interface requirements were understood well 
enough to begin system development. However, this was also not the 
case. Specifically, 39 of 90 NOC/SOC interface requirements were 
removed from the baseline, and 2 new interface requirements were added 
after CDR. 

Further, all relevant risks were not identified, and not all 
identified risks had mitigation plans. Specifically, 7 of 31 
identified risks did not have mitigation plans, including risks 
associated with poorly established requirements traceability and 
inadequately defined requirements for integration suppliers. Moreover, 
the risks identified were as of May 2008, prior to the beginning of 
CDR, and did not include four risks identified between June and 
October 2008, when CDR was concluded. For example, a risk associated 
with the instability of the C3I/COP software was not addressed during 
CDR. 

Without properly baselined requirements (including interfaces) and 
proactive mitigation of known risks, system performance shortfalls are 
likely. To illustrate, we previously reported that ambiguities in 
requirements actually forced testers to rewrite test steps during 
execution based on interpretations of what they thought the 
requirements meant, and they required the SPO to incur the time and 
expense of conducting multiple events to test NOC/SOC requirements. 
[Footnote 33] 

* NOC/SOC Component Qualification Test Readiness Review. The SPO did 
not ensure that a well-defined test plan was in place, to include, 
among other things, test cases and procedures and a traceability 
matrix that maps each requirement to be tested to a corresponding test 
case. Specifically, the contractor delivered the test plan on the day 
of the review, rather than 10 days prior to the review, as required by 
the relevant task order. Nevertheless, the SPO concluded the review 
based on its review of the plan during the test readiness review. In 
this regard, we previously reported problems with the NOC/SOC test 
plan, noting that the plan mapped 28 out of 100 requirements to 
incorrect test cases. Program officials attributed the test plan 
limitations to, among other things, insufficient time and resources to 
review the deliverables. 

The SBInet independent verification and validation (IV&V) contractor 
[Footnote 34] also identified weaknesses within technical reviews. 
Specifically, the IV&V contractor reported that the SPO was not 
provided with documentation, including the test plan, early enough for 
the NOC/SOC test readiness review to allow sufficient time for review. 
Moreover, in December 2009, the program identified technical oversight 
of technical review milestones as a major risk to the cost, schedule, 
and performance goals of the program. According to program officials, 
they are developing a technical review manual that is to supplement 
the SEP and provide detailed guidance for conducting technical 
reviews. In commenting on a draft of this report, DHS stated that it 
plans to complete and implement its technical review guide by December 
2010. 

Program officials attributed the limitations with the technical 
reviews, in large part, to the program's aggressive schedule, which 
resulted in insufficient time to review relevant documentation. As 
discussed above, these limitations have resulted in schedule delays 
and costly rework. 

Contractor Management Reviews Were Conducted as Planned, but Were 
Limited by Unreliable Contractor Performance Data: 

Management reviews help to ensure that the contractor's interpretation 
and implementation of the requirements are consistent with those of 
the program office. According to relevant guidance, organizations 
should have written policies and procedures for conducting management 
reviews that, among other things, (1) involve relevant stakeholders; 
(2) assign roles and responsibilities for performing management review 
tasks; (3) communicate project status information, including cost and 
schedule information, and risks; and (4) identify, document, and track 
action items to closure.[Footnote 35] 

CBP policy also recognizes the importance of these reviews by 
requiring the conduct of management reviews.[Footnote 36] Further, the 
SBInet Program Management Plan identifies the types of management 
reviews that are to be conducted with the contractor. For SBInet, the 
primary management review is known as the Joint Program Management 
Review (JPMR). The plan also identifies, for example, the stakeholders 
that are to participate in the reviews, including the program manager, 
project managers, program control staff, and the risk management team; 
and it specifies the topics that are to be discussed at the reviews, 
such as project status, cost and schedule performance, and risks. 

To its credit, the program office has held monthly JPMRs, and these 
reviews include the prime contractor, the SBI Executive Director, the 
SBInet Program Manager, program control staff, risk management team, 
border patrol agents, DHS representatives, and the Defense Contract 
Management Agency (DCMA). Further, review documentation shows that 
JPMRs addressed cost and schedule performance to include EVM 
performance data, project milestones status and potential delays, and 
risks. Review documentation also shows action items were identified, 
documented, and tracked to closure, as part of these reviews. For 
example, during the May 2009 JPMR, an action to review the risk 
management process--including roles, responsibilities, and decision 
authority--was identified, and this action was documented in Boeing's 
Management Emphasis Tracking database.[Footnote 37] To address and 
close the action item, the program subsequently completed a review of 
the SBInet risk management process and tool, including reviewing 
lessons learned from other programs. The results of the review were 
presented during a February 2010 briefing, and the action item was 
closed. 

Effectively conducting management reviews has helped to provide 
program leadership with an understanding of the contractor's progress 
and the program's exposure to risks so that appropriate corrective 
action can be taken and the chances of delivering a system solution 
that meets mission needs within budget are enhanced. However, as 
discussed in the next section, the EVM performance data presented at 
these management reviews were not reliable, thus rendering those 
reviews, at best, limited in the extent to which they disclosed the 
true status of the program. 

DHS Has Not Effectively Monitored the Contractor's Progress in Meeting 
Cost and Schedule Expectations: 

Measuring and reporting progress against cost and schedule 
expectations (i.e., baselines) is a vital element of effective 
contractor management and oversight. As noted earlier, EVM provides a 
proven means for measuring progress against cost and schedule 
commitments and thereby identifying potential cost overruns and 
schedule delays early, when the impact can be minimized. 

However, DHS has not ensured that its prime contractor's EVM system, 
which was certified as meeting relevant standards, has been 
effectively implemented on SBInet. In particular, it has not ensured 
that performance measurement baselines were validated in a timely 
manner, that established baselines were complete and realistic, and 
that contractor-provided cost and schedule data were reliable. Reasons 
cited by program officials for these weaknesses include the 
instability in the scope of the work to be performed, an unexpected 
temporary stop in Block 1 design and deployment work when SBInet 
funding was redirected, and the contractor's use of estimated, rather 
than actual, costs for subcontractor work, which are subsequently 
adjusted when actual costs are received. Without effectively 
implementing EVM, DHS has not been positioned to identify potential 
cost and schedule problems early, and thus has not been able to take 
timely actions to correct problems and avoid program schedule delays 
and cost increases. 

Contractor EVM System Has Been Certified: 

In August 2005, the Office of Management and Budget issued guidance 
[Footnote 38] that, among other things, directs agencies to ensure 
that EVM systems are compliant with the American National Standards 
Institute (ANSI) standard.[Footnote 39] The ANSI standard consists of 
32 guidelines associated with a sound EVM system that are intended to 
ensure that data are reliable and can be used for informed decision-
making. 

The program office relies on the prime contractor's EVM system to 
provide cost and schedule performance data. This system was certified 
in April 2005 by DCMA as being compliant with the ANSI standard. DCMA 
certified the contractor's EVM system again in February 2009. 

Notwithstanding these certifications, DCMA identified a number of 
issues with the contractor's implementation of its EVM system. 
[Footnote 40] In particular, in January 2010, DCMA reported that the 
SBInet prime contractor's implementation of EVM was not consistent 
with all of the 32 ANSI guidelines. Specifically, DCMA identified 
concerns with the quality of scheduling and reporting, and the 
identification of significant differences between planned and actual 
cost and schedule performance, as well as reasons for those 
differences. 

Program Office Has Not Established Timely Performance Baselines: 

According to relevant guidance, the performance measurement baseline, 
which is the foundation of an EVM system and the estimated cumulative 
value of planned work, serves as the value against which performance 
is measured for the life of the program or task order.[Footnote 41] As 
such, it should be established as early as possible after contract or 
task order award, or whenever a major contract modification or 
baseline change occurs. DHS guidance further states that a baseline 
should be validated within 90 days of the contract or task order 
award.[Footnote 42] 

However, the program office validated a performance measurement 
baseline within 90 days for only two of the six baselines that we 
reviewed (see figure 4). For the other four, the length of time to 
establish a validated baseline ranged from 5 to 10 months. For 
example, the program office issued the ADTO in June 2008, and it did 
not establish a validated baseline until 10 months later in April 
2009. Similarly, in February 2009, the program office modified the 
scope of the STO and extended the period of performance, but it did 
not validate the revised baseline to include the additional scope and 
time until 7 months later in September 2009. Figure 4 summarizes the 
periods of time during which earned value was, and was not, measured 
against a validated baseline. 

Figure 4: Summary of Time Frames With and Without Validated 
Performance Baselines from June 2008 through February 2010: 

[Refer to PDF for image: illustration] 

Design Task Order: 

June 2008 and earlier through March, 2009: Earned value data was not 
measured against a government-validated baseline. 

March 2009 through August 2009: (IBR 3/09) Earned value data was 
measured against a government-validated baseline. 

August 2009 through December 2009: Earned value data was not measured 
against a government-validated baseline. 

December 2009 and beyond: Earned value data was measured against a 
government-validated baseline. 

C3I/COP: 

IBR (2/08): through March 2009: Earned value data was measured against 
a government-validated baseline. 

System Task Order: 

IBR (5/08): through March 2009: Earned value data was measured against 
a government-validated baseline. 

March 2009 through September 2009: Earned value data was not measured 
against a government-validated baseline. 

IBR (9/09): September 2009 and beyond: Earned value data was measured 
against a government-validated baseline. 

Arizona Deployment Task Order: 

Contract award: June 2008 through April 2009: Earned value data was 
not measured against a government-validated baseline. 

IBR (4/09): April 2009 and beyond: Earned value data was measured 
against a government-validated baseline. 

Source: GAO analysis of SBInet data. 

Notes: Generally, the government established baselines through the 
period of performance for the task order at the time of the Integrated 
Baseline Review (IBR). The government had not completed all work for 
the DTO and STO within the original period of performance, and as 
such, extended the period of performance, and in some cases, 
established a new baseline. 

According to program officials, they held an IBR for the C3I/COP task 
order in March 2009. However, they did not provide sufficient 
documentation to support this statement. 

[End of figure] 

According to program officials, the delays in validating performance 
baselines were due to instability in the work to be performed, and the 
need to temporarily stop Block 1 design and deployment work between 
September 2008 and January 2009 because of DHS's decision to redirect 
funds from SBInet to the physical infrastructure.[Footnote 43] Without 
validated baselines, DHS was not positioned to identify potential cost 
and schedule problems early and to take timely corrective actions to 
mitigate those problems. 

DHS Has Not Established Reliable Performance Measurement Baselines: 

An integrated baseline review (IBR) is used to validate the 
performance measurement baseline. This review is intended to verify 
that the baseline is realistic and ensure that the contractor and the 
government mutually understand scope, schedule, and risks for a given 
task order before a substantial amount of work is performed. According 
to relevant guidance, establishing a complete and realistic 
performance measurement baseline includes (1) assigning responsibility 
for managing, tracking, and reporting earned value data for work 
performed; (2) estimating needed resources (i.e., budgets and staff), 
including management reserve, for performing assigned tasks; (3) 
defining a product-oriented description of all work to be performed; 
[Footnote 44] (4) scheduling all work in a time-phased sequence that 
reflects the duration of the program's activities; and (5) 
establishing objective performance measures for each task.[Footnote 45] 

In validating the performance measurement baselines for the four task 
orders that we reviewed, the program office implemented two of the 
above elements, but it did not implement the other three. 
Specifically, for each of the six baselines associated with the task 
orders, the program office (1) assigned responsibility for managing, 
tracking, and reporting earned value data associated with each work 
breakdown structure element and (2) estimated a time-phased budget, 
including the anticipated staff needed, for each work breakdown 
structure element, and established a management reserve. However, as 
discussed in the following section, the program office did not (1) 
define a product-oriented description of all work to be performed, (2) 
reliably estimate schedule baselines, and (3) adequately measure 
earned value performance. 

Program officials attribute these limitations in establishing 
comprehensive baselines to instability in the nature of the work to be 
performed and the prime contractor's method for determining 
subcontractor performance. Nevertheless, without complete and 
realistic baselines, the SPO has been hampered in its ability to 
conduct meaningful measurement and oversight of the prime contractor's 
status and progress, as well as holding the contractor accountable for 
results. More importantly, the lack of meaningful measurement and 
oversight has contributed to program cost overruns and schedule delays. 

Work Breakdown Structure Was Not Product Oriented and Did Not Include 
All Work: 

According to relevant guidance, a work breakdown structure 
deconstructs a program's end product into successively smaller levels 
until the work is subdivided to a level suitable for management 
control.[Footnote 46] Further, a work breakdown structure should be 
product oriented and include all work to be performed. 

The work breakdown structure that was used to define each of the task 
order baselines was not product oriented. Instead, it was defined in 
terms of functions that span multiple system products, such as systems 
engineering, system test and evaluation, and program management. 

Additionally, the work breakdown structure did not reflect all work to 
be performed. Specifically, for four of the six performance 
measurement baselines, the work breakdown structure did not include 
all work described in the corresponding task order's statement of 
work. For example, the work breakdown structure used to define the May 
2008 STO baseline did not include the work associated with identifying 
and selecting components that meet system requirements and program 
security. Similarly, DCMA reported in June 2008 that the work 
breakdown structure included in this baseline did not account for all 
work identified in the system task order. 

Estimated Baseline Schedules Were Not Reliable: 

A reliable schedule provides a road map for systematic execution of a 
program and the means by which to gauge progress, identify and address 
potential problems, and promote accountability. Our research has 
identified nine best practices associated with developing and 
maintaining a reliable schedule: (1) capturing all activities, (2) 
sequencing all activities, (3) assigning resources to all activities, 
(4) establishing the duration of all activities, (5) integrating 
activities horizontally and vertically, (6) establishing the critical 
path[Footnote 47] for all activities, (7) identifying reasonable 
"float"[Footnote 48] between activities, (8) conducting a schedule 
risk analysis, and (9) updating the schedule using logic and 
durations.[Footnote 49] 

The six task order baselines were not reliable because they 
substantially complied with only two of the eight key schedule 
estimating practices, and they did not comply with, or only partially 
or minimally complied with, the remaining six practices.[Footnote 50] 
(See figure 5 for a summary of the extent to which each of the 
baseline schedules met each of the eight practices.) 

Figure 5: Summary of IBR Schedules' Satisfaction of Schedule 
Estimating Practices: 

[Refer to PDF for image: illustrated table] 

Schedule practice: Capturing all activities: 
Task order schedule: 
C3I/COP February 2008: Minimally met; 
STO May 2008: Minimally met; 
DTO March 2009: Partially met; 
ADTO April 2009: Minimally met; 
STO September 2009: Minimally met; 
DTO December 2009: Minimally met. 

Schedule practice: Sequencing all activities: 
Task order schedule: 
C3I/COP February 2008: Substantially met; 
STO May 2008: Substantially met; 
DTO March 2009: Substantially met; 
ADTO April 2009: Substantially met; 
STO September 2009: Substantially met; 
DTO December 2009: Substantially met. 

Schedule practice: Assigning resources to all activities: 
Task order schedule: 
C3I/COP February 2008: Minimally met; 
STO May 2008: Minimally met; 
DTO March 2009: Minimally met; 
ADTO April 2009: Partially met; 
STO September 2009: Minimally met; 
DTO December 2009: Partially met. 

Schedule practice: Establishing the duration of all activities: 
Task order schedule: 
C3I/COP February 2008: Substantially met; 
STO May 2008: Substantially met; 
DTO March 2009: Substantially met; 
ADTO April 2009: Substantially met; 
STO September 2009: Substantially met; 
DTO December 2009: Substantially met. 

Schedule practice: Integrating activities horizontally and vertically: 
Task order schedule: 
C3I/COP February 2008: Partially met; 
STO May 2008: Partially met; 
DTO March 2009: Partially met; 
ADTO April 2009: Partially met; 
STO September 2009: Partially met; 
DTO December 2009: Partially met. 

Schedule practice: Establishing the critical path for all activities: 
Task order schedule: 
C3I/COP February 2008: Partially met; 
STO May 2008: Partially met; 
DTO March 2009: Partially met; 
ADTO April 2009: Partially met; 
STO September 2009: Partially met; 
DTO December 2009: Partially met. 

Schedule practice: Identifying reasonable float between activities: 
Task order schedule: 
C3I/COP February 2008: 
STO May 2008: Partially met; 
DTO March 2009: Minimally met; 
ADTO April 2009: Partially met; 
STO September 2009: Partially met; 
DTO December 2009: Partially met. 

Schedule practice: Conducting a schedule risk analysis: 
Task order schedule: 
C3I/COP February 2008: Not met; 
STO May 2008: Not met; 
DTO March 2009: Not met; 
ADTO April 2009: Not met; 
STO September 2009: Not met; 
DTO December 2009: Not met. 

Met = DHS provided complete evidence that satisfied the criteria. 

Substantially Met = DHS provided evidence that satisfied more than one-
half of the criteria. 

Partially Met = DHS provided evidence that satisfied about one-half of 
the criteria. 

Minimally Met = DHS provided evidence that satisfied less than one-
half of the criteria. 

Not Met = DHS provided no evidence that satisfied any of the criteria. 

Source: GAO analysis of DHS data. 

[End of figure] 

* Capturing all activities. The six schedules did not capture all 
activities defined in the task order baseline. Specifically, five of 
the six schedules did not reflect the work to be performed across the 
four task orders (i.e., integrated master schedule). Further, as 
previously mentioned, four of six work breakdown structures were 
missing elements defined in the respective task order statements of 
work. Moreover, two of the six schedules did not reflect all work that 
was defined in the work breakdown structure. For example, the December 
2009 DTO schedule omitted efforts associated with design work for TUS- 
1 and AJO-1. 

* Sequencing all activities. The six schedules substantially met this 
practice. Each of the schedules identified almost all of the 
predecessor and successor activities. However, each contained improper 
predecessor and successor relationships. For example, the May 2008 STO 
baseline included 52 of 538 activities (about 10 percent) with 
improper predecessor and successor relationships. Additionally, many 
activities in four of the schedules were constrained by "start no 
earlier than" dates. For example, as previously reported, the 
September 2009 baseline schedule contained 403 of 1,512 activities 
(about 27 percent) with "start no earlier than" constraints, which 
means that these activities are not allowed to start earlier than 
their assigned dates, even if their respective predecessor activities 
have been completed. 

* Assigning resources to all activities. Two of the six schedules 
partially met this practice. Specifically, two schedules included 
resources; however, those resources were allocated to less than 15 
percent of the activities identified in each schedule. Moreover, the 
remaining four schedules did not include estimated resources. Instead, 
resources for all six schedules were maintained separately as part of 
the contractor's earned value system and only available to DHS upon 
request. 

* Establishing the duration of all activities. Each of the six 
baseline schedules substantially met this practice. Specifically, each 
schedule established the duration of key activities and included 
baseline start and end dates for most of the activities. Further, 
reasonable durations were established for the majority of the 
activities in the schedules, meaning that the durations established 
were less than 44 days. Nevertheless, each of the schedules included 
activities that were not of short duration, that is, more than 44 
days. For example, the April 2009 ADTO baseline included 29 of 1,009 
activities with durations ranging from 45 days to 352 days. 

* Integrating activities horizontally and vertically. Each of the 
schedules partially met this practice. As mentioned previously, the 
six schedules did not capture all activities defined in the task order 
baseline. Further, four of six work breakdown structures were missing 
elements defined in respective task order statements of work. 
Additionally, five of six schedules did not reflect the work performed 
across the four task orders (i.e., integrated master schedule), and 
each had improper predecessor and successor relationships. 

* Establishing the critical path for all activities. Each of the six 
schedules partially met this practice. Specifically, four of six work 
breakdown structures were missing elements defined in the respective 
task order statements of work. Additionally, four of the six schedules 
were missing predecessor and successor activities, and each of the 
schedules included improper predecessor and successor relationships. 
Further, five of the six schedules did not reflect the work to be 
performed across the four task orders (i.e., integrated master 
schedule). Unless all activities are included and properly linked, it 
is not possible to generate a true critical path. 

* Identifying reasonable float between activities. Each of the 
schedules identified float; however, the amount of float was 
excessive. For example, the February 2008 C3I/COP task order baseline 
included 259 of 294 activities (about 88 percent) with float greater 
than 100 days and 189 of the 259 (about 73 percent) with float in 
excess of 200 days. 

* Conducting a schedule risk analysis. DHS did not conduct a risk 
analysis of any of the schedules. 

Earned Value Performance Was Not Adequately Measured: 

According to the ANSI standard for EVM systems,[Footnote 51] only work 
for which measurement is impractical may be classified as "level-of- 
effort."[Footnote 52] Our research shows that if more than 15 percent 
of a program's budget is measured using level-of-effort, then that 
amount should be scrutinized because it does not allow schedule 
performance to be measured (i.e., performance equals planned work). 
[Footnote 53] 

However, the six baselines had between 34 and 85 percent of the 
baseline dollar value categorized as level-of-effort, including four 
with more than 50 percent (see table 2). Moreover, for five of the six 
baselines, program documentation showed that the program office did 
not identify any action items during the respective IBRs related to 
the high use of level-of-effort. 

According to program officials, the STO, which categorized between 70 
and 85 percent of the baseline dollar value as level-of-effort, 
includes many program management activities (e.g., cost, schedule, and 
subcontractor management). Nevertheless, they recognized that the 
level-of-effort for this task order was high, and in November 2009, 
they directed the contractor to minimize the use of level-of-effort 
for STO. According to program officials, the high level-of-effort was 
due, in part, to the prime contractor's use of this measurement for 
subcontractor work. In November 2009, DCMA stated that the SPO's use 
of level-of-effort activities was high, noting that this could be 
masking true contractor performance. 

Table 2: Percentage of Performance Measurement Baseline Work Tasks 
Categorized as Level-of-Effort for Six Baselines: 

Task order IBR: C3I/COP task order February 2008; 
Level-of-effort percentage: 46 percent. 

Task order IBR: STO May 2008; 
Level-of-effort percentage: 85 percent. 

Task order IBR: DTO March 2009; 
Level-of-effort percentage: 53 percent. 

Task order IBR: ADTO April 2009; 
Level-of-effort percentage: 56 percent. 

Task order IBR: STO September 2009; 
Level-of-effort percentage: 70 percent. 

Task order IBR: DTO December 2009; 
Level-of-effort percentage: 34 percent. 

Source: GAO analysis of DHS data. 

[End of table] 

Unreliable EVM Data Limit DHS's Ability to Measure Contractor 
Performance: 

If performed properly, EVM can provide an objective means for 
measuring program status and forecasting potential program cost 
overruns and schedule slippages so that timely action can be taken to 
minimize their impact. To do so, however, the underlying EVM data must 
be reliable, meaning that they are complete and accurate and all data 
anomalies are explained. 

In the case of SBInet, the EVM data provided by the prime contractor 
for the 21-month period ending in February 2010 have not been 
reliable, as evidenced by numerous and unexplained anomalies in 
monthly EVM reports. Reasons for the anomalies include the 
contractor's use of estimated, rather than actual, costs for 
subcontractor work, which are subsequently adjusted when actual costs 
are received. Without reliable performance data, the true status of 
the SBInet program is unclear, thus limiting the SPO's ability to 
identify potential cost and schedule shortfalls. 

EVM Data Can Provide Objective Insight into Program Cost and Schedule 
Performance: 

EVM is a proven program measurement approach that, if implemented 
appropriately, can create a meaningful and coherent understanding of a 
program's true health and status. As a result, the use of EVM can 
alert decision makers to potential program problems sooner than is 
possible by using actual versus planned expenditure alone, and thereby 
reduce the chance and magnitude of program cost overruns and schedule 
slippages. 

Simply stated, EVM measures the value of completed work in a given 
period (i.e., earned value) against (1) the actual cost of work 
completed for that period (i.e., actual cost) and (2) the value of the 
work that is expected to be completed for that period (i.e., planned 
value). Differences in these values are referred to as cost and 
schedule variances, respectively. 

* Cost variances compare the value of the work completed with the 
actual cost of the work performed. For example, if a contractor 
completed $5 million worth of work and the work actually cost $6.7 
million, there would be a negative $1.7 million cost variance. 

* Schedule variances are also measured in dollars, but they compare 
the value of the work completed with the value of the work that was 
expected to be completed. For example, if a contractor completed $5 
million worth of work at the end of the month but was expected to 
complete $10 million worth of work, there would be a negative $5 
million schedule variance. 

Positive variances indicate that activities are costing less or are 
being completed ahead of schedule. Negative variances indicate 
activities are costing more or are falling behind schedule. To 
determine both cost and schedule variances, all three values are 
necessary. 

SBInet EVM Data Have Not Been Reliable: 

According to relevant guidance, EVM data should be valid and free from 
unexplained anomalies (e.g., missing or negative values) because they 
can limit program management's ability to identify potential cost and 
schedule shortfalls.[Footnote 54] Therefore, anomalies should be 
minimized for each of the three values--earned value, planned value, 
and actual cost. Moreover, all anomalies should be identified, and the 
reason for each should be fully explained in the monthly EVM reports. 
To do less limits the completeness and accuracy of these values, and 
thus makes the resulting variance determinations unreliable. While an 
industry standard for what constitutes an acceptable volume of 
anomalies does not exist, EVM experts in the public and private 
sectors that we interviewed stated that the occurrence of EVM data 
anomalies should be rare. Of these experts, some agreed that an 
anomaly should occur in no more than 5 percent of the work breakdown 
structure elements for a given contract or task order, while some of 
these advocated an occurrence percentage of no more than 1-2 percent. 

However, the EVM data that the prime contractor delivered to the SPO 
from June 2008 through February 2010 (21 months) contained numerous, 
unexplained anomalies. Specifically, the monthly EVM reports for all 
four task orders that we reviewed showed one or more anomalies (e.g., 
missing or negative values for earned value, planned value, and actual 
cost) in each of the months that had a validated performance 
measurement baseline. More specifically, the average number of work 
breakdown structure elements across the four task orders that had data 
anomalies during this 21-month period ranged from 11 percent to 41 
percent. For the C3I/COP task order in particular, the monthly 
percentage of work breakdown structure elements with anomalies ranged 
between 25 and 67 percent over the 21 months. (See figure 6 for the 
percentage of work breakdown structure elements with anomalies by 
month for each of the four task orders.) 

Figure 6: Percentage of Work Breakdown Structure Elements with EVM 
Data Anomalies by Month for Each of the Four Task Orders: 

[Refer to PDF for image: vertical bar graph] 

2008: 

June: 
Arizona Deployment: 0; 
Design: 0; 
C3I/COP: 33%; 
System: 7%. 

July: 
Arizona Deployment: 0; 
Design: 0; 
C3I/COP: 33%; 
System: 4%. 

August: 
Arizona Deployment: 0; 
Design: 0; 
C3I/COP: 33%; 
System: 13%. 

September: 
Arizona Deployment: 0; 
Design: 0; 
C3I/COP: 42%; 
System: 22%. 

October: 
Arizona Deployment: 0; 
Design: 0; 
C3I/COP: 42%; 
System: 18%. 

November: 
Arizona Deployment: 0; 
Design: 0; 
C3I/COP: 58%; 
System: 9%. 

December: 
Arizona Deployment: 0; 
Design: 0; 
C3I/COP: 67%; 
System: 16%. 

2009: 

January: 
Arizona Deployment: 0; 
Design: 0; 
C3I/COP: 33%; 
System: 13%; 

February: 
Arizona Deployment: 0; 
Design: 0; 
C3I/COP: 25%; 
System: 2. 

March: 
Arizona Deployment: 0; 
Design: 0; 
C3I/COP: 0; 
System: 0. 

April: 
Arizona Deployment: 0; 
Design: 14%; 
C3I/COP: 0; 
System: 0. 

May: 
Arizona Deployment: 11	
Design: 10	
C3I/COP: 0; 
System: 0. 

June: 
Arizona Deployment: 7%; 
Design: 10%; 
C3I/COP: 0; 
System: 0. 

July: 
Arizona Deployment: 11%; 
Design: 10%; 
C3I/COP: 0; 
System: 0. 

August: 
Arizona Deployment: 15%; 
Design: 0; 
C3I/COP: 0; 
System: 0. 

September: 
Arizona Deployment: 22%; 
Design: 0; 
C3I/COP: 0; 
System: 0. 

October: 
Arizona Deployment: 11%; 
Design: 0; 
C3I/COP: 0; 
System: 24%. 

November: 
Arizona Deployment: 19%; 
Design: 0; 
C3I/COP: 0; 
System: 29%. 

December: 
Arizona Deployment: 41%; 
Design: 0; 
C3I/COP: 0; 
System: 64%. 

2010: 

January: 
Arizona Deployment: 41%; 
Design: 10%; 
C3I/COP: 0; 
System: 22%. 

February: 
Arizona Deployment: 56%; 
Design: 14%; 
C3I/COP: 0; 
System: 20%. 

Source: GAO analysis of DHS data. 

Note: We only analyzed EVM reports for those months with a validated 
baseline for any of the four task orders. There were no validated 
baselines for any of the four task orders in March 2009. 

[End of figure] 

The October 2009 STO monthly EVM report illustrates how the anomalies 
can distort the contractor's performance. According to this report, 
about $13,000 worth of work was planned to be completed on integration 
and management, approximately $13,000 worth of work was performed, and 
actual costs were about negative $550,000. Thus, the report 
erroneously suggests that the contractor performed $13,000 worth of 
work, and actually saved about $550,000 in doing so. Similarly, the 
September 2009 ADTO monthly report showed that about $200,000 worth of 
work was planned to be completed on tower sites and infrastructure, 
and $25,000 worth of work was performed, but that no costs were 
incurred, suggesting that the work was performed for free. 

Exacerbating the large percentage of monthly data anomalies across the 
four task orders is the fact that in most cases the reasons for the 
anomalies were not explained in the monthly EVM variance analysis 
reports. Specifically, about 79 percent of all anomalies across all 
four task orders during the 21-month period were not explained. In 
particular, 82 of 119 (or about 69 percent) of all data anomalies for 
the STO task order were not explained in the monthly reports, and none 
of the anomalies were explained for DTO. (See figure 7 for the total 
number of data anomalies and the number that were explained in the 
monthly reports across the four task orders.) 

Figure 7: Total Number of Data Anomalies and Number of Explained 
Anomalies in the Monthly EVM Reports across the Four Task Orders: 

[Refer to PDF for image: vertical bar graph] 

2008: 

June: 
Total EVM data anomalies: 7; 
Total EVM data anomalies with explanations: 0. 

July: 
Total EVM data anomalies: 6; 
Total EVM data anomalies with explanations: 0. 

August: 
Total EVM data anomalies: 10; 
Total EVM data anomalies with explanations: 3. 

September: 
Total EVM data anomalies: 15; 
Total EVM data anomalies with explanations: 4. 

October: 
Total EVM data anomalies: 13; 
Total EVM data anomalies with explanations: 2. 

November: 
Total EVM data anomalies: 11; 
Total EVM data anomalies with explanations: 2. 

December: 
Total EVM data anomalies: 15; 
Total EVM data anomalies with explanations: 1. 

2009: 

January: 
Total EVM data anomalies: 10; 
Total EVM data anomalies with explanations: 2. 

February: 
Total EVM data anomalies: 4; 
Total EVM data anomalies with explanations: 2. 

March: 
Total EVM data anomalies: 0; 
Total EVM data anomalies with explanations: 0. 

April: 
Total EVM data anomalies: 3; 
Total EVM data anomalies with explanations: 0. 

May: 
Total EVM data anomalies: 5; 
Total EVM data anomalies with explanations: 1. 

June: 
Total EVM data anomalies: 4; 
Total EVM data anomalies with explanations: 1. 

July: 
Total EVM data anomalies: 5; 
Total EVM data anomalies with explanations: 1. 

August: 
Total EVM data anomalies: 4; 
Total EVM data anomalies with explanations: 1. 

September: 
Total EVM data anomalies: 6; 
Total EVM data anomalies with explanations: 1. 

October: 
Total EVM data anomalies: 14; 
Total EVM data anomalies with explanations: 2. 

November: 
Total EVM data anomalies: 18; 
Total EVM data anomalies with explanations: 0. 

December: 
Total EVM data anomalies: 40; 
Total EVM data anomalies with explanations: 26. 

2010: 

January: 
Total EVM data anomalies: 23; 
Total EVM data anomalies with explanations: 1. 

February: 
Total EVM data anomalies: 27; 
Total EVM data anomalies with explanations: 1. 

Source: GAO analysis of DHS data. 

Note: As mentioned previously, there were no validated baselines for 
any of the four task orders in March 2009. 

[End of figure] 

Program officials acknowledged problems with the EVM data and stated 
that they meet with the prime contractor each month to discuss the EVM 
reports, including the reliability of the data. According to program 
officials, limitations in the EVM data are due, in part, to the 
contractor's use of estimated, rather than actual, costs for 
subcontractor work, which are subsequently adjusted when actual costs 
are received. Officials further stated that they have been working 
with the contractor to reduce the volume of unexplained anomalies, and 
they believe that the reliability of the data has improved since 
February 2010. However, program officials did not provide any 
documentation to support this statement. Without reliable EVM data, 
the program office is unable to identify actual cost and schedule 
shortfalls, which along with the other contractor tracking and 
oversight weaknesses discussed in this report, has limited its ability 
to effectively minimize program cost increases and schedule delays. 

Conclusions: 

Effective management and oversight of a program's prime contractor is 
essential to successfully acquiring and deploying a system like 
SBInet. Integral to accomplishing this is defining and implementing a 
range of contractor management and oversight controls (e.g., processes 
and practices) that reflect relevant federal guidance and best 
practices. To do less increases the chances that contractor-delivered 
products and services will not satisfy stated requirements and will 
not meet customer expectations. The result is incurring the additional 
time and expense to redo or rework contractor deliverables, accepting 
products and services that do not perform as intended and do not meet 
mission needs, or both. 

Overall, DHS has not done an effective job of managing and overseeing 
its prime contractor, including monitoring the contractor's 
performance. DHS has largely defined key management and oversight 
processes and practices that it should have followed, and it 
implemented a number of these processes and practices. However, 
several key management and oversight controls were not adequately 
defined, and essential controls were not implemented. Most 
significantly, DHS did not adequately document deliverable reviews and 
communicate the basis for rejecting certain deliverables in writing to 
the contractor, which contributed to deliverables that did not live up 
to expectations and necessitated rework and caused later problems. 
Further, technical reviews were not grounded in explicit criteria for 
determining when reviews should begin and conclude, which also 
contributed to contract deliverables requiring costly and time-
consuming rework. In addition, the cost and schedule baselines for 
measuring the contractor's performance were frequently validated too 
late and without sufficient accuracy and completeness to provide a 
meaningful basis for understanding performance, which precluded DHS 
from taking timely action to correct unfavorable results and trends. 
Compounding these serious baseline limitations was contractor-provided 
data about actual performance that were replete with unexplained 
anomalies, thus rendering the data unfit for effective contractor 
management and oversight. Notwithstanding of a number of contractor 
management and oversight definition and implementation efforts that 
DHS executed well, such as defining key processes and practices and 
training key staff, these above-cited weaknesses collectively mean 
that DHS's management and oversight of its prime contractor has been a 
major contributor to the SBInet program's well-chronicled history of 
not delivering promised system capabilities on time and on budget. 

These limitations can be attributed to a number of factors, including 
gaps in how certain processes and practices were defined, as well as 
not enforcing other processes and practices that were defined and 
applicable and not taking sufficient time to review deliverables that 
were submitted late. The limitations can be further attributed to the 
fact that SBInet has from its outset lacked clear definition and 
stability, and thus experienced continuous change in scope and 
direction--an issue that we have previously reported and made 
recommendations to address. Collectively, these factors have helped to 
create a contractor management and oversight environment, which, when 
combined with the many other acquisition management weaknesses that we 
have previously reported about and made recommendations to address, 
have produced a program that to date has not been successful, and if 
not corrected, can become worse. 

Recommendations for Executive Action: 

To improve DHS management and oversight of the SBInet prime 
contractor, we recommend that the Secretary of Homeland Security 
direct the Commissioner of the U.S. Customs and Border Protection to 
have the SBI Executive Director, in collaboration with the SBInet 
Program Director, take the following four actions: 

* Revise and implement, as applicable, contractor deliverable review 
processes and practices to ensure that (1) contractor deliverables are 
thoroughly reviewed and are not constrained by late contractor 
deliverables and imposed milestones, (2) the reviews are sufficiently 
documented, and (3) the acceptance or the rejection of each contractor 
deliverable is communicated in writing to the contractor, to include 
explicit explanations of the basis for any rejections. 

* Ensure that applicable entry and exit criteria for each technical 
review are used and satisfied before initiating and concluding, 
respectively, a given review. 

* Establish and validate timely, complete, and accurate performance 
measurement baselines for each new task order or major modification of 
an existing task order, as appropriate, to include, but not be limited 
to, ensuring that (1) the work breakdown structure includes all work 
to be performed, (2) baseline schedules reflect the key schedule 
estimating practices discussed in this report, and (3) level-of-effort 
performance measurement in excess of 15 percent is scrutinized, 
justified, and minimized. 

* Ensure that all anomalies in contractor-delivered monthly earned 
value management reports are identified and explained, and report 
periodically to DHS acquisition leadership on relevant trends in the 
number of unexplained anomalies. 

Because we have already made recommendations in prior reports to 
address the other management and oversight weaknesses discussed in 
this report, such as those related to requirements management, risk 
management, and Systems Engineering Plan implementation, we are not 
making any additional recommendations at this time. 

Agency Comments and Our Evaluation: 

In written comments on a draft of this report, signed by the Director, 
Departmental GAO/OIG Liaison Office and reprinted in appendix II, DHS 
agreed with our four recommendations and described actions under way 
or planned, which we summarize below, to address them. 

* With respect to our recommendation to revise and implement the 
contractor deliverable review process, DHS stated that it is updating 
the process to require written documentation of each review and the 
communication to the contractor of review results. 

* With respect to our recommendation to ensure that entry and exit 
criteria are used to initiate and conclude each technical review, DHS 
stated that it has established an SBI Systems Engineering Directorate 
to focus on technical oversight of the acquisition process, adding 
that the Directorate is developing a technical review guide that 
describes in detail the review process and the relevant entry and exit 
criteria for each technical review. 

* With respect to our recommendation to establish and validate timely, 
complete, and accurate performance measurement baselines, DHS stated 
that it is mindful of the need to establish and maintain current 
performance baselines, and to plan and implement baseline updates as 
completely and promptly as practicable, which it indicated is done 
through IBRs. DHS also noted that while scheduling practices remain a 
challenge, it continues to make improvements to its process, including 
implementing scheduling tools and templates. 

* With respect to our recommendation to identify and explain all 
anomalies in monthly EVM reports, and to report periodically relevant 
trends to DHS acquisition leadership, DHS acknowledged the need to 
correctly document anomalies in the monthly EVM reports, and stated 
that it is working with DCMA to improve contractor quality control 
issues and the content of the monthly EVM reports. It also stated that 
it is augmenting the reports with routine conversations between 
contractor and project management staff. The department also committed 
to advising the appropriate acquisition leaders through established 
reporting and oversight opportunities as issues arise with contractor 
performance or reporting. 

Notwithstanding its agreement with our recommendations, the department 
also commented that it took exception to selected findings and 
conclusions regarding the program's implementation of EVM. A summary 
of DHS's comments and our responses are provided below. 

* The department stated that it took exception to our finding that it 
did not ensure performance measurement baselines were validated in a 
timely manner, and said that it was not accurate to conclude that the 
lack of validated baselines precluded the program office from 
identifying cost and schedule problems and taking corrective action. 
In support of these positions, the department made the following three 
points, which our response addresses. 

First, the department stated that the SBInet program office delayed 
formal IBRs until it had finalized negotiated modifications to the 
task orders, and in doing so, was able to complete an IBR within 90 
days of each major task order modification. We do not question whether 
the program office held IBRs within 90 days of final negotiation of 
major task order modifications. Our point is that DHS did not validate 
task order performance measurement baselines (i.e., hold IBRs) within 
90 days of task order award, which is what DHS guidance states should 
occur. As our report states, the program office only met this 90-day 
threshold for two of the six baselines that we reviewed. Further, the 
length of time to validate the performance baselines for the four task 
orders far exceeded 90 days (5 to 10 months), during which time DHS 
reports show that significant work was performed and millions of 
dollars were expended. In fact, the DHS reports show that most of the 
planned work for some of these task orders had already been performed 
by the time the IBR was held and the baseline was validated. As we 
state in our report, and DHS acknowledged in its comments, the purpose 
of an IBR is to verify that the performance baseline is realistic and 
that the scope, schedule, and risks are mutually understood by the 
contractor and the government before a substantial amount of work is 
performed. 

Second, DHS commented that the program office maintained what it 
referred to as "interim" performance measurement baselines during the 
period of major program scope, schedule, and budget changes. We 
acknowledge that in some cases the program office had these "interim" 
baselines. However, these baselines are the contractor-provided 
baselines, meaning that the program office and the contractor had not 
mutually agreed to the scope, schedule, and risks associated with the 
work to be performed. Moreover, for two of the task orders, the 
program office did not have an "interim" baseline, even though the 
contractor performed significant work under these task orders. 
[Footnote 55] 

Third, the department stated that program leadership reviewed the 
contractor's technical and financial performance information relative 
to performance measurement baselines and implemented management 
actions as needed. We do not question whether program leadership 
reviewed contractor-provided performance information or whether 
actions to address problems may have been taken. However, our report 
does conclude, as is discussed later in this section, that the 
combination of the EVM weaknesses that our report cites, to include 
unreliable performance baselines and contractor-provided performance 
data, did not allow the program office to identify performance 
problems early and to take timely actions to avoid the well-documented 
schedule delays and cost increases that the program has experienced. 

* The department expressed two concerns with how it said our report 
characterized and quantified EVM anomalies. 

First, the department stated that our report failed to distinguish 
between factual errors and legitimate monthly accounting adjustments. 
We agree that our report does not distinguish between the two types of 
anomalies, and would add that this was intentional because making the 
distinction was not relevant to our finding. Specifically, our finding 
is that the reasons for the numerous anomalies were not explained in 
the monthly EVM variance analysis reports, therefore making the true 
status of the program unclear. 

Second, DHS stated that we incorrectly concluded that both errors and 
adjustments are problematic, distort cost performance, and limit 
management insight. In response, we did not conclude that all errors 
and adjustments have these impacts, but rather that the lack of 
explanation associated with such a large volume of anomalies made the 
true status of the program unclear, thus limiting the program office's 
ability to identify actual cost and schedule shortfalls, which is 
certainly problematic. Further, our report cites examples of cost 
performance data that provide a distorted picture of actual 
performance vis-à-vis expectations. Accordingly, the correct 
characterization of the report's conclusion concerning the reliability 
of EVM data is that the lack of explanation of the numerous anomalies 
in monthly reports is problematic, provides a distorted picture of 
cost performance, and limits management insight. To this very point, 
DHS acknowledged in its comments the importance of explaining the 
reason for anomalies in the monthly variance reports, regardless of 
whether they are due to factual errors or accounting adjustments. 

* The department stated that it took exception to our conclusion that 
the program office's lack of validated baselines in particular, and 
EVM shortcomings in general, contributed to cost and schedule growth 
and made it unable to identify cost and schedule problems early and 
take corrective actions to avoid them. In response, we did not 
conclude that the lack of validated baselines alone had either of 
these impacts. However, we did conclude that the collection of EVM 
weaknesses discussed in our report, to include untimely validated 
baselines, incomplete and unreliable baselines, and unreliable 
performance data, together precluded the program office from 
identifying problems early and taking corrective action needed to 
avoid the program's well-chronicled history of schedule delays and 
cost increases. In support of this conclusion, we state in the report, 
for example, that the performance measurement baselines that we 
reviewed understated the cost and time necessary to complete the work 
because they did not capture all work in the task orders' statements 
of work and because they were not grounded in a range of scheduling 
best practices. Given that cost and schedule growth is a function of 
the baseline against which actual cost and schedule performance is 
measured, it follows logically that an understated baseline would 
produce actual cost overruns and schedule delays. In addition, we 
would note that beyond these EVM shortcomings, our report also 
recognizes other contract tracking and oversight, test management, and 
requirements management weaknesses that have collectively contributed 
to the program's cost, schedule, and performance shortfalls. 

In addition to the above points, DHS provided technical comments, 
which we have incorporated in the report as appropriate. 

We are sending copies of this report to the Chairmen and Ranking 
Members of the Senate and House Appropriations Committees and other 
Senate and House committees and subcommittees that have authorization 
and oversight responsibilities for homeland security. We will also 
send copies to the Secretary of Homeland Security, the Commissioner of 
U.S. Customs and Border Protection, and the Director of the Office of 
Management and Budget. In addition, the report will be available at no 
cost on the GAO Web site at [hyperlink, http://www.gao.gov]. 

Should you or your offices have any questions on matters discussed in 
this report, please contact me at (202) 512-3439 or at hiter@gao.gov. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. Key contributors 
to this report are listed in appendix III. 

Signed by: 

Randolph C. Hite: 
Director, Information Technology Architecture and Systems Issues: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

Our objectives were to determine the extent to which the Department of 
Homeland Security (DHS) (1) has defined and implemented effective 
controls for managing and overseeing the Secure Border Initiative 
Network (SBInet) prime contractor and (2) is effectively monitoring 
the prime contractor's progress in meeting cost and schedule 
expectations. To accomplish our objectives, we focused on four key 
task orders--the Arizona Deployment Task Order, the Design Task Order, 
the System Task Order, and the Command, Control, Communications, and 
Intelligence/Common Operating Picture Task Order--that are integral to 
the design, development, and deployment of the first increment of 
SBInet, known as Block 1. We also focused on the SBInet System Program 
Office's (SPO) contracting and oversight activities that occurred from 
June 2008 through February 2010. 

To determine the extent to which DHS has defined and implemented 
effective controls for managing and overseeing the SBInet prime 
contractor, we focused on training, verifying and accepting contract 
deliverables, conducting technical reviews, and conducting management 
reviews. We chose these four because each is important to tracking and 
overseeing contractor performance for the four task orders. 

* Training. We compared relevant DHS contractor management and 
oversight training requirements to the program managers', contracting 
officers', and contracting officer's technical representatives' 
training certifications. 

* Verifying and accepting contract deliverables. We compared U.S. 
Customs and Border Protection's (CBP) process for verifying and 
accepting contract deliverables to leading industry practices[Footnote 
56] to identify any variances. We then assessed a nonprobability, 
random sample of 28 contract deliverables that the SPO identified as 
being delivered between June 1, 2008, and August 31, 2009.[Footnote 
57] We also judgmentally selected one additional review that was 
conducted between September 1, 2009, and February 28, 2010.[Footnote 
58] For each of the 29 deliverables, we reviewed relevant 
documentation, such as the contract deliverables, review comment 
forms, and documented communications with the prime contractor 
indicating acceptance or rejection of the deliverable, and compared 
them to the CBP process and leading industry practices to determine 
what, if any, deviations existed. 

* Technical reviews. We compared relevant DHS and CBP guidance and 
entry and exit criteria in the task order Data Item Descriptions to 
leading industry practices to identify any variances. We assessed a 
nonprobability, random sample of technical reviews. Specifically, we 
assessed a technical review from each of the eight unique combinations 
of task orders and review types held between June 1, 2008, and August 
31, 2009. We also judgmentally selected one additional review that was 
conducted between September 1, 2009, and February 28, 2010.[Footnote 
59] For each of the nine reviews, we compared the package of 
documentation prepared for and used during these reviews to the 
criteria defined in the relevant task orders to determine the extent 
to which the reviews satisfied the criteria. 

* Management reviews. We compared relevant CBP guidance to leading 
industry practices to identify any variances. We then compared 
relevant documentation prepared for and used during monthly joint 
program management reviews to determine the extent to which the 
reviews addressed cost, schedule, and program risks. We also assessed 
a nonprobability sample of 11 action items that were identified during 
the reviews held from October 2008 through October 2009,[Footnote 60] 
and assessed relevant documentation to determine the extent to which 
they were tracked to closure. 

To determine the extent to which DHS is effectively monitoring the 
prime contractor's progress in meeting cost and schedule expectations, 
we focused on the program's implementation of earned value management 
(EVM) because it was the tool used to monitor the contractor's cost 
and schedule performance. Specifically, we analyzed the six 
performance measurement baselines, and the associated integrated 
baseline review documentation, such as briefings, the work breakdown 
structure (WBS) governing all task orders, task order statements of 
work, schedules, monthly contract performance reports, control account 
plans, and responsibility assignment matrixes. In doing so, we 
compared this documentation to EVM and scheduling best practices as 
identified in our Cost Estimating and Assessment Guide.[Footnote 61] 
Specifically, for each of the six baselines: 

* We reviewed control account plans and responsibility assignment 
matrixes to determine the period of performance and scope of work for 
each baseline, compared the work described in the respective task 
order statements of work to the work described in the responsibility 
assignment matrix, and reviewed the control account plans to determine 
the extent to which the level-of-effort measurement method was to 
measure contractor performance. 

* We analyzed the schedule presented at each baseline review against 
eight key schedule estimating practices in our Cost Estimating and 
Assessment Guide.[Footnote 62] In doing so, we used commercially 
available software tools to determine whether each schedule, for 
example, included all critical activities, a logical sequence of 
activities, and reasonable activity durations. Further, we 
characterized the extent to which the schedule met each of the 
practices as either not met, minimally met, partially met, 
substantially met, or met. 

* We analyzed the contract performance reports for each of the four 
task orders for each month that there was a validated baseline. 
Specifically, we identified instances of the following: (1) negative 
planned value, earned value, or actual cost; (2) planned value and 
earned value without actual cost; (3) earned value and actual cost 
without planned value; (4) actual cost without planned value or earned 
value; (5) earned value without planned value and actual cost; (6) 
inconsistencies between the estimated cost at completion and the 
planned cost at completion; (7) actual cost exceeding estimated cost 
at completion; and (8) planned or earned values exceeding planned cost 
at completion. To determine the number of anomalies, we identified 
each WBS element that had one or more of the above anomalies. Then, we 
identified the number of WBS elements at the beginning and the end of 
the baseline period of performance, and calculated the average number 
of WBS elements. We used this to determine the percentage of WBS 
elements with anomalies for each task order and for each month for 
which there was a validated baseline. 

* To support our work across this objective, we interviewed officials 
from the Department of Defense's Defense Contract Management Agency 
(DCMA), which provides contractor oversight services to the SPO, 
including oversight of EVM implementation, and prime contractor 
officials. We also reviewed DCMA monthly status reports and corrective 
action reports. 

For both objectives, we interviewed program officials to obtain 
clarification on the practices, and to determine the reasons for any 
deviations. 

To assess the reliability of the data that we used to support the 
findings in this report, we reviewed relevant program documentation to 
substantiate evidence obtained through interviews with knowledgeable 
agency officials, where available. We determined that the data used in 
this report are sufficiently reliable. We have also made appropriate 
attribution indicating the sources of the data. 

We performed our work at the CBP headquarters and prime contractor 
facilities in the Washington, D.C., metropolitan area and with DCMA 
officials from Huntsville, Alabama. We conducted this performance 
audit from June 2009 to October 2010 in accordance with generally 
accepted government auditing standards. Those standards require that 
we plan and perform the audit to obtain sufficient, appropriate 
evidence to provide a reasonable basis for our findings and 
conclusions based on our audit objectives. We believe that the 
evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

[End of section] 

Appendix II: Comments from the Department of Homeland Security: 

Department of Homeland Security: 
Washington, DC 25528: 

September 23, 2010: 

Mr. Randolph C. Hite: 
Director, Information Technology Architecture and Systems Issues: 
U.S. Government Accountability Office: 
Washington, D.C. 20548: 

Dear Mr. Hite: 

Thank you for the opportunity to review and offer comment on the 
Government Accountability Office (GAO) draft report entitled, "Secure 
Border Initiative: DHS Needs to Strengthen Management and Oversight of 
its Prime Contractor," GAO-11-6, dated October 2010. GAO was asked to 
determine the extent to which the Department of Homeland Security 
(OHS) (I) defined and implemented effective controls for managing and 
overseeing the SBInet prime contractor and (2) effectively monitored 
the prime contractor's progress in meeting cost and schedule 
expectations, To do this, GAO analyzed key performance documentation 
against relevant guidance and industry best practices, and interviewed 
program officials. 

GAO acknowledges that OHS has largely defined key management and 
oversight processes and procedures for verifying and accepting 
contract deliverables and conducting technical reviews. However, the 
draft report identifies measures that the U.S. Customs and Border 
Protection (CBP) SBInet can take to further strengthen contractor 
management and oversight. 

GAO made four recommendations to improve DHS management and oversight 
of the SBInet prime contractor. CBP concurs with the four 
recommendations. However, not withstanding our concurrence, we take 
exception to several assertions regarding aspects of the reliability 
of the Secure Border Initiative (SB1) program's Earned Value 
Management (EVM) practices, and government oversight of contractor 
activities. The comments below address our specific concerns. 

First, GAO asserts the program office did not ensure EVM performance 
measurement baselines (PMB) were "validated" in a timely manner, and 
concludes that the program office's EVM shortcomings led to cost and 
schedule growth in the program. We firmly take exception to these 
assertions. 

As background to many of the EVM findings and recommendations, GAO 
acknowledges throughout the report the significant amount of program 
change occurring with the SBInet program in late 2008-2009. These 
positive changes followed a major program re-baselining (Acquisition 
Program Baseline) with the Department as well as SBInet final system 
design updates after Critical Design Review. The program baseline 
change drove major scope, schedule, and budget changes across all of 
the Boeing task orders and associated performance measurement 
baselines. 

GAO concluded that without a "validated" baseline through the conduct 
of an Integrated Baseline Review (IBR), the program office was unable 
to identify cost and schedule problems early and to take timely 
corrective actions to mitigate those problems. The report also 
concluded that the program's lack of a "validated" baseline 
contributed to cost overruns and schedule delays. We believe these 
conclusions are inaccurate, and the causal assertion is not supported 
by any analysis presented in the report. Of significance is the fact 
that the program office maintained interim performance measurement 
baselines throughout the transition period with the best available 
task plan information, as required by EVM best practices. Program 
leadership also reviewed in various forum, the contractor's technical 
and financial performance information relative to the established PMBs 
and implemented rigorous management actions as needed. The program 
office delayed formal IBR(s) until completing final, negotiated task 
order modifications, and in each case the IBR was completed within the 
timeframe specified in DEIS directives (less than 90 days after the 
major contract modification). Certainly during a period of significant 
program changes. earned value management is challenged, but the 
program office believes prudent and compliant EVM measures were 
executed. 

GAO documented other EVM implementation concerns reported by the 
program office, specifically the prime contractor's over-reliance on 
the "level of effort" (LoE) EVM measure, and recurring quality control 
lapses, or "anomalies," contained in the contractor's monthly reports. 
For the LoE concern, the program office and the prime contractor 
initiated several actions to improve task order work planning by 
creating discreet work packages, enabling improved performance 
management and issue identification. However, given the stage of the 
program today, we will continue to see high levels of effort in the 
task order baselines due to maintaining sizeable contractor program 
management "overhead" (typically LoE), while transitioning from 
construction and testing completion into a sustaining system and 
software engineering mode (typically LoE), as well as performing a 
variety of system maintenance functions (also typically LoE). 

Secondly. we are concerned with GAO's characterization and 
quantification of "anomalies." The SBI program office demands accurate 
contractor performance reports, and is continuously engaged with the 
contractor to fix underlying problems. The draft report fails to 
distinguish between factual errors versus legitimate monthly 
accounting adjustments. For example: 

* Errors, and the resultant corrections, occur when the contractor 
collects and reports costs for one activity when the costs actually 
apply to another activity, and might include incorrect contract charge 
numbers issued to subcontractors or other simple administrative 
mistakes. Errors distort cost performance status however the SBI 
program office and the contractor promptly identify and take 
corrective actions to correct identified problems and improve quality 
controls for final monthly reports. 

* Adjustments routinely occur when the prime contractor reports 
estimated costs for a subcontractor (while awaiting a subcontractor to 
submit an invoice), and then in a subsequent report, the prime 
reconciles previously reported estimated costs with the final, 
invoiced actual costs. Adjustments may also be necessary to reflect 
contractor rate changes. These adjustments improve the accuracy and 
reliability of the cumulative cost performance data, yet
may appear anomalous for the current month. The prime contractor's 
adjustments are completed in accordance with the contractor's approved 
system guide and established best practices. 

The GAO report combines both errors and adjustments when quantifying 
"anomalies" and incorrectly concludes that both errors and adjustments 
are problematic, distorting cost performance, and limits management 
insight. 

Regardless of error corrections or adjustments, the contractor is 
required to document such actions in the monthly cost performance 
report (Format 5). Over the past eight months, the contractor 
implemented quality control measures, and has significantly improved 
the process for monthly reporting of both errors and anomalies as well 
as significant accounting adjustments. 

The recommendations and CBP's corrective actions to address the 
recommendation are described below. 

Recommendation 1: Revise and implement, as applicable, contractor 
deliverable review processes and practices to ensure that (I) 
contactor deliverables are thoroughly reviewed and are not constrained 
by late contractor deliverables and imposed milestones, (2) the 
reviews are sufficiently documented, and (3) the acceptance or the 
rejection of each contractor deliverable is communicated in writing to 
the contractor, to include explicit explanations of the basis for any 
rejections. 

CBP Response: CBP concurs with the recommendation. Since the draft 
report was issued, we have taken several major steps to improve the 
SBInet program management structure, capabilities, procedures, and 
processes. This includes significant improvement in SBInet execution 
of the Contract Deliverable Requirements List (CDRL) review process. 
The program has commenced reviewing and updating the current CDRL, 
review policy. The updated CDRL policy will document the process and 
procedures for program technical reviews, and written documentation to 
the contractor regarding results of the review to include reviewer 
comment forms and formal communication of the acceptance, conditional 
acceptance, or rejection of the contract deliverable. In implementing 
the updated CDRL policy, contract deliverable review participants will 
be trained to ensure that reviewers are technically qualified and 
possess the procedural knowledge to effectively review and provide 
concise and thorough, review comments and that the process is fully 
aligned with the approved CDRL review policy. 

Due Date: Policy update expected to be completed by December 2010. 

Recommendation 2: Ensure that applicable entry and exit criteria for 
each technical review are used and satisfied before initiating and 
concluding, respectively, a given review. 

CLIP Response: CBP concurs with the recommendation. We have taken 
several major steps to improve the SBInet program management 
structure, capabilities, procedures, and processes. This has led to a 
more rigorous application of review entry and exit criteria. In 
addition, the SBI Systems Engineering (SE) Directorate was established 
to ensure that systems engineering capabilities are focused on 
providing the technical oversight required to support knowledge-based 
decision making throughout the acquisition process. The SE Directorate 
is incorporating the DIN Engineering Life Cycle best practices into a 
Technical Review Guide (TRG) that describes in detail the SE technical 
review process and the relevant entry and exit criteria for each 
technical review. The draft TRG is currently under review and when 
implemented will serve as the roadmap used by the SBI System Program 
Office to ensure that all relevant entry and exit criteria are 
understood and satisfied in a manner consistent with programmatic 
cost, schedule, and performance risk considerations. 

Due Date: Technical Review Guide expected to be completed and 
implemented by December 2010. 

Recommendation 3: Establish and validate timely, complete, and 
accurate performance measurement baselines for each new task order or 
major modification of an existing task order, as appropriate, to 
include, but not be limited to, ensuring that (1) the work breakdown 
structure includes all work to be performed; (2) baseline schedules 
reflect the key schedule estimating practices discussed in this 
report; and (3) level-of-effort performance measurement in excess of 
15 percent is scrutinized, justified, and minimized. 

CRP Response: CBP concurs with the recommendation. The SBI program 
office is mindful of the need to establish and maintain current 
Performance Measurement Baselines (PMB), and to plan and implement 
baseline updates as completely and promptly as practicable. For new 
task orders and significant modifications. this is done through 
Integrated Baseline Reviews (IBRs). The purpose of the IBR is to 
confirm that the PMB covers the scope of work; the work is 
realistically and accurately scheduled; the proper mix of resources 
and budgets are assigned; the appropriate management processes are in 
place; and the risks associated with the PMB are identified, 
documented, and managed. Prior to an IBR, the program office routinely 
evaluates the control accounts, and the integrated PMB, for 
consistency, traceability, and completeness relative to the work 
breakdown structure (WBS), to include a cross-check of the WBS back to 
the control accounts. Also, the program office documents any adverse 
findings and resolves such with the prime contractor as part of the 
1BR proceedings. As described above, the program office scrutinizes, 
and with the prime contractor, minimizes level of effort tasks in the 
PMB. 

Scheduling practices remain a challenge, and as reported in our 
response to prior GAO reviews, we continue to upgrade our processes. 
The program office is revising our scheduling standards to reflect 
procedural improvements for schedule development, management and 
surveillance. We have created and put into practice the use of 26 
diagnostic schedule filters to analyze the health of our schedules. 
The filters fully leverage GAO's scheduling best practices. We have 
also implemented improvements to our practices for developing 
Integrated Master Plan/Integrated Master Schedule that utilizes newly 
revised planning and schedule templates based on the OHS Systems 
Engineering Lifecycle. 

Due Date: Ongoing corrective actions are in place, and the program 
office continues to assess and manage contractor progress and 
improvements. 

Recommendation 4: Ensure that all anomalies in contractor-delivered 
monthly earned value management reports are identified and explained, 
and report periodically to DHS acquisition leadership on relevant 
trends in the number of unexplained anomalies. 

CRP Response: CBP concurs with the recommendation. The monthly 
contract performance report "anomalies" (which include a variety of 
legitimate monthly updates to actual costs, rate changes, etc.) need 
to be documented correctly each month. The SBI program office shared 
with the GAO significant, recent activities and progress with 
associated corrective actions: 

* Fixing prime contractor systemic and quality control issues, with 
the assistance of the Defense Contract Management Agency, through 
active participation in program EVM surveillance; and; 

* Improving discipline and content of Format 5 reporting narrative, 
augmented with routine management discussions with contractor and 
government project management staff. 

Our focus remains to ensure we are able to determine program status 
relative to well-defined performance baselines. As issues arise with 
either contractor performance or reporting, we will advise the 
appropriate acquisition leaders through established reporting and 
oversight opportunities. 

Due Date: Ongoing; corrective actions are in place, and the program 
office continues to assess and manage contractor progress and 
improvements. 

Sincerely, 

Signed by: 

Jerald E. Levine: 
Director: 
Departmental GAO/OIG Liaison Office: 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Randolph C. Hite, (202) 512-3439 or hiter@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, Deborah A. Davis (Assistant 
Director), Tisha Derricotte, Neil Doherty, Kaelin Kuhn, Lee McCracken, 
Jamelyn Payan, Karen Richey, Matt Snyder, Sushmita Srikanth, Stacey 
Steele, and Matthew Strain made key contributions to this report. 

[End of section] 

Footnotes: 

[1] We also selected one additional contract deliverable and one 
additional technical review for review. 

[2] EVM is a project management approach that, if implemented 
appropriately, provides objective reports of project status, produces 
early warning signs of impending schedule delays and cost overruns, 
and provides unbiased estimates of a program's total costs. 

[3] SEI, CMMI® for Acquisition, Version 1.2 (Pittsburgh, Pa., November 
2007); and GAO, GAO Cost Estimating and Assessment Guide: Best 
Practices for Developing and Managing Capital Program Costs, 
[hyperlink, http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: 
March 2009). Also, recognizing the importance of ensuring quality 
earned value data, ANSI and the Electronic Industries Alliance (EIA) 
jointly established a national standard for EVM systems in May 1998 
(ANSI/EIA-748-A-1998). This standard, commonly called the ANSI 
standard, is comprised of guidelines on how to establish a sound EVM 
system. This document was updated in July 2007 and is referred to as 
ANSI/EIA-748-B. 

[4] The remaining $126 million was for upgrading CBP 
telecommunications links. 

[5] The physical infrastructure (e.g., physical fencing) portion of 
SBI is managed on a day-to-day basis by CBP's Office of Finance 
Facilities Management and Engineering division. 

[6] IDIQ contracts are used when the government cannot predetermine, 
above a specified minimum, the precise quantities of supplies or 
services that it will require during the contract period. Under IDIQ 
contracts, the government can issue delivery orders (for supplies) and 
task orders (for services). 

[7] The $35 million was part of American Recovery and Reinvestment Act 
of 2009 funding. Pub. L. No. 111-5, 123 Stat. 115, 162 (Feb. 17, 2009). 

[8] See for example, GAO, Secure Border Initiative: DHS Needs to 
Reconsider Its Proposed Investment in Key Technology Program, 
[hyperlink, http://www.gao.gov/products/GAO-10-340] (Washington, D.C.: 
May 5, 2010); Secure Border Initiative: DHS Needs to Address Testing 
and Performance Limitations That Place Key Technology Program at Risk, 
[hyperlink, http://www.gao.gov/products/GAO-10-158] (Washington, D.C.: 
Jan. 29, 2010); and DOD Business Systems Modernization: Important 
Management Controls Being Implemented on Major Navy Program, but 
Improvements Needed in Key Areas, [hyperlink, 
http://www.gao.gov/products/GAO-08-896] (Washington, D.C.: Sept. 8, 
2008). 

[9] GAO, Secure Border Initiative: SBInet Expenditure Plan Needs to 
Better Support Oversight and Accountability, [hyperlink, 
http://www.gao.gov/products/GAO-07-309] (Washington, D.C.: Feb. 15, 
2007). 

[10] GAO, Secure Border Initiative: DHS Needs to Address Significant 
Risks in Delivering Key Technology Investment, [hyperlink, 
http://www.gao.gov/products/GAO-08-1086] (Washington, D.C.: Sept. 22, 
2008). 

[11] GAO, Secure Border Initiative: Technology Deployment Delays 
Persist and the Impact of Border Fencing Has Not Been Addressed, 
[hyperlink, http://www.gao.gov/products/GAO-09-896] (Washington, D.C.: 
Sept. 9, 2009). 

[12] [hyperlink, http://www.gao.gov/products/GAO-10-158]. 

[13] A test case is a set of test inputs, execution conditions, and 
expected results developed for a particular objective, such as to 
verify compliance with a specific requirement. 

[14] [hyperlink, http://www.gao.gov/products/GAO-10-340]. 

[15] GAO, Department of Homeland Security: Assessments of Selected 
Complex Acquisitions, [hyperlink, 
http://www.gao.gov/products/GAO-10-588SP] (Washington, D.C.: June 30, 
2010). 

[16] See, for example, Federal Acquisition Regulation, part 46, 
quality assurance; and SEI, CMMI® for Acquisition. 

[17] SEI, CMMI® for Acquisition. 

[18] Training requirements for key officials are defined in three DHS 
policies: (1) Acquisition Certification Requirements for Program 
Manager, Management Directive (MD) 0782 (May 26, 2004); (2) 
Contracting Officer Warrant Program (Apr. 16, 2007); and (3) COTR 
Certification, Appointment & Responsibilities, MD 0780.1 (Dec. 20, 
2004). 

[19] DHS has categorized major information technology investments as 
Levels 1, 2, and 3. A Level 1 major investment is defined in DHS 
Acquisition Directive 102-01 as program at or more than $1 billion in 
life cycle costs. 

[20] DHS's Acquisition Certification Requirements for Program Manager, 
MD 0782 (May 26, 2004) establishes three levels of certification: 
Level 1 establishes the foundation for managing increasingly complex 
investments or portfolios, Level 2 includes knowledge of strategic and 
capital planning, and Level 3 represents mastery of the knowledge and 
skills associated with the complexities of managing DHS's most 
strategic and sophisticated acquisitions. 

[21] For warrant authority up to $100,000, the contracting officer 
must have a Federal Acquisition Certification in Contracting (FAC-C) 
Program Level I certification; for warrant authority up to $25 
million, the contracting officer must have a FAC-C Level II 
certification; and for warrant authority more than $25 million, 
including unlimited warrant authority, the contracting officer must 
have a FAC-C Level III certification. 

[22] SEI, CMMI® for Acquisition. 

[23] A project manager is responsible for each of the task orders. 
These project managers have overall responsibility and accountability 
for achieving the task order's scope, cost, schedule, and technical 
objectives. 

[24] According to the SPO, subject matter experts review the 
deliverables based on the contractual requirements that are outlined 
in the Data Item Descriptions, as well as their own expertise and 
knowledge about the deliverables. 

[25] These 29 deliverables actually resulted in 46 separate 
submissions because, for example, an initial submission was rejected 
and had to be resubmitted. 

[26] The NOC/SOC monitors networked equipment, provides alerts for 
network problems, protects equipment from network-based attacks, and 
provides user authentication. 

[27] According to the C3I/COP task order statement of work, the plan 
was to be submitted 10 days before the test readiness review. However, 
the contractor submitted the plan the day of this review. 

[28] This system is the program office's requirements management 
database. 

[29] [hyperlink, http://www.gao.gov/products/GAO-10-158]. 

[30] SEI, CMMI® for Acquisition. 

[31] DHS, Acquisition Instruction/Guidebook, 102-01-001, Appendix B, 
Systems Engineering Life Cycle, Interim Version 1.9 (Nov. 7, 2008). 

[32] [hyperlink, http://www.gao.gov/products/GAO-10-340]. 

[33] [hyperlink, http://www.gao.gov/products/GAO-10-158]. 

[34] In 2008, the SPO contracted with an IV&V agent to, among other 
things, review the program's test documentation, execution, and 
related processes. Generally, the purpose of IV&V is to independently 
determine the extent to which program processes and products meet 
quality standards. The use of IV&V is recognized as an effective 
practice for large and complex system development and acquisition 
programs, like SBInet. 

[35] SEI, CMMI® for Acquisition. 

[36] CBP, System Life Cycle Handbook, Version 1.2 (Sept. 30, 2008). 

[37] This database is Boeing-managed and tracks SBInet action items 
throughout their lifecycle. The database captures such information as 
individual action assignments, status and updates, and relevant 
comments. 

[38] Office of Management and Budget Memorandum, M-05-23 (Aug. 4, 
2005). 

[39] ANSI/EIA-748-B. 

[40] As mentioned previously, DCMA conducts periodic surveillance 
reviews of the prime contractor's EVM system implementation. 

[41] [hyperlink, http://www.gao.gov/products/GAO-09-3SP], 227-229, and 
Department of Defense, Earned Value Management Implementation Guide 
(October 2006). 

[42] DHS, Department of Homeland Security Acquisition Manual, Section 
3034.202 Integrated Baseline Reviews (October 2009). Integrated 
baseline reviews are intended to verify that the baseline is realistic 
and the scope, schedule, and risks are mutually understood by the 
contractor and the government. 

[43] Physical infrastructure includes physical fencing and roads. 

[44] The scope of the work is generally defined in a work breakdown 
structure, which is a hierarchical structure that shows how work tasks 
relate to one another as well as to the overall end product. 

[45] [hyperlink, http://www.gao.gov/products/GAO-09-3SP], 215-231. 

[46] [hyperlink, http://www.gao.gov/products/GAO-09-3SP], 215. 

[47] The critical path represents the chain of dependent activities 
with the longest total duration in the schedule. 

[48] Float is the amount of time a task can slip before affecting the 
critical path. 

[49] [hyperlink, http://www.gao.gov/products/GAO-09-3SP], 218-224. 

[50] We did not assess the ninth practice because the schedules that 
were reviewed at the IBRs were the initial schedules for each task 
order, and as such, were not updated. 

[51] ANSI/EIA-748-B. 

[52] Level-of-effort is unmeasured effort of a general or supportive 
nature which does not produce definitive end products (e.g., program 
administration). 

[53] [hyperlink, http://www.gao.gov/products/GAO-09-3SP], 226. 

[54] [hyperlink, http://www.gao.gov/products/GAO-09-3SP], 226. 

[55] The program office did not have a baseline for ADTO and DTO, from 
July 2008 through October 2008 and from June 2008 through January 
2009, respectively. 

[56] Software Engineering Institute, Capability Maturity Model 
Integration® for Acquisition, Version 1.2 (Pittsburgh, Pa., November 
2007). 

[57] We conducted a nonprobability sample because we determined that 
the SPO's contract deliverable log was not sufficiently reliable for 
purposes of identifying all required deliverables during our time 
frames, and thus we did not select a sample that could be projected to 
the total population. 

[58] We selected one additional deliverable that was associated with 
the technical review noted below to provide a more current assessment 
of the program's adherence to its process. 

[59] We selected one additional technical review based on the 
combinations of task orders and review types in our existing sample to 
provide a more current assessment of the program's implementation of 
established criteria. 

[60] We selected one action item for each month in which a joint 
program management review was held and action items were documented. 

[61] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for 
Developing and Managing Capital Program Costs, [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 
2009), 215 steps 1-6. 

[62] We did not assess the ninth practice--updating the schedule using 
logic and durations--because the schedules that were reviewed at the 
integrated baseline reviews were the initial schedules for each task 
order, and as such, were not updated. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: