This is the accessible text file for GAO report number GAO-07-565 
entitled 'Information Technology: Immigration and Customs Enforcement 
Needs to Fully Address Significant Infrastructure Modernization Program 
Management Weaknesses' which was released on April 30, 2007. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Committees: 

April 2007: 

Information Technology: 

Immigration and Customs Enforcement Needs to Fully Address Significant 
Infrastructure Modernization Program Management Weaknesses: 

GAO-07-565: 

GAO Highlights: 

Highlights of GAO-07-565, a report to congressional committees 

Why GAO Did This Study: 

The Department of Homeland Security (DHS) fiscal year 2006 
appropriations act provided $40.15 million for the Immigration and 
Customs Enforcement’s (ICE) program to modernize its information 
technology (IT) infrastructure. As mandated by the appropriations act, 
the department is to develop and submit for approval an expenditure 
plan for the program, referred to as “Atlas,” that satisfies certain 
legislative conditions, including a review by GAO. In performing its 
review of the Atlas plan, GAO (1) determined whether the plan satisfies 
certain legislative conditions and (2) provided other observations 
about the plan and management of the program. To do this, GAO analyzed 
the fiscal year 2006 Atlas expenditure plan and supporting documents 
against the legislative conditions, federal requirements, and related 
best practices. GAO also interviewed relevant DHS officials. 

What GAO Found: 

The fiscal year 2006 Atlas expenditure plan, in combination with 
related program documentation and program officials’ statements, 
satisfies or partially satisfies the legislative conditions set forth 
by Congress (see table). 

Table: DHS's Satisfaction of Legislative Conditions: 

Legislative conditions: 1. Expenditure plan must meet the capital 
planning and investment control review requirements established by the 
Office of Management and Budget (OMB); Status: partially satisfies. 

Legislative conditions: 2. DHS must ensure Atlas is compliant with 
DHS's enterprise architecture; Status: partially satisfies. 

Legislative conditions: 3. The plan must comply with the acquisition 
rules, requirements, guidelines, and systems acquisition management 
practices of the federal government; Status: partially satisfies. 

Legislative conditions: 4. The plan must include a certification by 
DHS's chief information officer that an independent verification and 
validation agent is currently under contract for the project; Status: 
partially satisfies. 

Legislative conditions: 5. The plan must be reviewed and approved by 
the DHS Investment Review Board, the Secretary of Homeland Security, 
and OMB; Status: satisfies. 

Source: GAO. 

[End of table] 

This satisfaction, however, is based on plans and commitments that 
provide for meeting these conditions rather than on completed actions 
to satisfy them. For example, to address the legislative condition 
related to capital planning and investment control review requirements, 
the program plans to, among other things, update its cost-benefit 
analysis in September 2007 to reflect emerging requirements and other 
program changes and to complete a privacy impact assessment by April 
2007. In addition, the program is in the process of defining how it 
plans to use its independent verification and validation agent. 

GAO also observed that DHS has not implemented key system management 
practices. Specifically, 
* rigorous practices are not being fully adhered to in developing and 
managing system requirements, 
* key contract management and oversight controls have not been fully 
implemented, 
* planned risk management practices have yet to be implemented, and; 
* performance management practices that are critical to measuring 
progress against program goals are still being implemented. 

Thus, much still needs to be accomplished to minimize the risks 
associated with the program’s capacity to deliver promised IT 
infrastructure capabilities and benefits on time and within budget. 
Given that hundreds of millions of dollars are to be invested and the 
program is critical to supporting the ICE mission, it is essential that 
DHS follow through on its commitments to build the capability to 
effectively manage the program. Proceeding without it introduces 
unnecessary risks to the program and potentially jeopardizes its 
viability for future investment. 

What GAO Recommends: 

GAO is recommending that DHS minimize Atlas program risks by fully 
adhering to requirements development and management practices, 
implementing key contract management and oversight practices, 
completing planned risk management activities, and implementing 
critical performance management practices. DHS concurred with GAO’s 
recommendations and described actions planned and under way to 
implement them. 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-565]. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Randolph C. Hite at (202) 
512-3439 or hiter@gao.gov. 

[End of section] 

Contents: 

Letter: 

Compliance with Legislative Conditions: 

Observations on the Expenditure Plan and Management of Atlas: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendixes: 

Appendix I: Briefing to the Staffs of the Subcommittees on Homeland 
Security, Senate and House Committees on Appropriations: 

Appendix II: Comments from the Department of Homeland Security: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Abbreviations: 

CCE: Common Computing Environment: 

CIO: chief information officer: 

CMMI: Capability Maturity Model Integrated: 

COTR: contracting officer's technical representative: 

DHS: Department of Homeland Security: 

EA: enterprise architecture: 

ICE: Immigration and Customs Enforcement: 

IT: information technology: 

IV&V: independent verification and validation: 

OMB: Office of Management and Budget: 

SACMM: Software Acquisition Capability Maturity Model: 

SEI: Software Engineering Institute: 

CCE: Common Computing Environment: 

CIO: chief information officer: 

CMMI: Capability Maturity Model Integrated: 

COTR: contracting officer's technical representative: 

DHS: Department of Homeland Security: 

EA: enterprise architecture: 

ICE: Immigration and Customs Enforcement: 

IT: information technology: 

IV&V: independent verification and validation: 

OMB: Office of Management and Budget: 

SACMM: Software Acquisition Capability Maturity Model: 

April 27, 2007: 

The Honorable Robert C. Byrd: 
Chairman: 
The Honorable Thad Cochran: 
Ranking Member: 
Subcommittee on Homeland Security: 
Committee on Appropriations: 
United States Senate: 

The Honorable David E. Price: 
Chairman: 
The Honorable Harold Rogers: 
Ranking Member: 
Subcommittee on Homeland Security: 
Committee on Appropriations: 
House of Representatives: 

The 2006 Department of Homeland Security (DHS) appropriations 
act[Footnote 1] provided $40.15 million for Immigration and Customs 
Enforcement's (ICE) program to modernize its information technology 
(IT) infrastructure. The goals of the program--which consists of seven 
related IT projects and is referred to by ICE as the "Atlas" program-- 
include improving information sharing and collaboration, strengthening 
information security, and improving workforce productivity. The act 
provides that none of the funds may be obligated until the department 
develops a plan for how the funds are to be spent that satisfies 
certain legislative conditions. The conditions included, among other 
things, having GAO review the plan. On November 9, 2006, DHS submitted 
its fiscal year 2006 expenditure plan to the Senate and House 
Appropriations Subcommittees on Homeland Security. Pursuant to the act, 
we reviewed the plan. Our objectives were to (1) determine whether the 
plan satisfies legislative conditions specified in the act and (2) 
provide other observations about the expenditure plan and the 
department's management of the Atlas program. 

On February 2, 2007, we provided our review results via a briefing to 
the staffs of the Senate and House Appropriations Subcommittees on 
Homeland Security. This report summarizes our findings, conclusions, 
and recommendations from that briefing. The full briefing, including 
our scope and methodology, is reprinted in appendix I. 

Compliance with Legislative Conditions: 

DHS's fiscal year 2006 Atlas expenditure plan, including related 
program documentation and program officials' stated commitments, 
satisfies or partially satisfies key aspects of (1) meeting the capital 
planning and investment control review requirements of the Office of 
Management and Budget (OMB);[Footnote 2] (2) complying with the DHS 
enterprise architecture;[Footnote 3] (3) complying with acquisition 
rules, requirements, guidelines, and systems acquisition management 
practices of the federal government; (4) including certification by the 
department's chief information officer (CIO) that an independent 
verification and validation agent is currently under contract for the 
project; and (5) having the plan reviewed and approved by the 
department's Investment Review Board, the Secretary of Homeland 
Security, and OMB. 

Regarding the fourth legislative condition, the department certified 
that an independent verification and validation agent is under contract 
for Atlas, and the department is in the process of defining how it 
plans to use its agent, but unresolved issues remain. For example, we 
have previously reported[Footnote 4] that the independence of a 
verification and validation agent is critical to providing management 
with objective insight into program processes and associated work 
products. We also reported[Footnote 5] that to be effective, the 
verification and validation function should be performed by an entity 
that is independent of the processes and products that are being 
reviewed. When the agent engaged to provide the service is a 
contractor, a practice to help ensure independence is to include in the 
agent's contract a provision that prohibits the agent from soliciting, 
proposing, or being awarded program work other than the independent 
verification and validation services and products. However, the 
contract that Atlas awarded to its agent did not include such a 
provision. 

Observations on the Expenditure Plan and Management of Atlas: 

Our observations address program efforts regarding (1) requirements 
development and management, (2) contract management and oversight, (3) 
risk management, and (4) performance management. An overview of these 
observations follows: 

* Rigorous practices are not being fully adhered to in developing and 
managing system requirements. On its three key projects,[Footnote 6] 
the program has not fully implemented federal IT guidance, relevant 
best practices, and agency policies and procedures (i.e., ICE System 
Lifecycle Management Handbook) that provide for effective requirements 
development and management activities. For example, on its Common 
Computing Environment project, while the project established a policy 
to guide project efforts in developing and managing requirements, it 
did not among other things: 

- document stakeholder comments as part of the requirements elicitation 
process and review and obtain stakeholder approval on requirements 
until approximately 1 year after deploying system capabilities; 

- follow a disciplined change management process for several years 
(between 2003 and 2006) after the project was initiated in March 2003 
(during this time, requirements were introduced and implemented in an 
ad hoc fashion); and: 

- develop supporting analysis showing traceability among requirements, 
systems design, and the contract. 

* Key contract management and oversight practices are not fully 
implemented. The program has not fully implemented practices that (1) 
the Software Engineering Institute's CMMI®[Footnote 7] has identified 
as essential to effectively managing and overseeing contracts that 
support IT projects and (2) the program has largely established in its 
existing policies and procedures. Specifically, the program has 
implemented two key practices, but has not fully implemented four 
others. For example, the program has assigned responsibility and 
authority for performing contract management and oversight, and has 
trained staff performing contract management and oversight. However, it 
has not, among other things, fully documented each contract--including 
clearly identifying the work to be performed and acceptance criteria-- 
and verified and accepted deliverables. 

* Key risk management practices have yet to be implemented. We 
previously reported[Footnote 8] that the program had developed a risk 
management plan and process to guide program office staff in managing 
risks throughout each project's life cycle and had recently begun to 
implement the risk management process. Since then, the program has 
taken additional steps to implement risk management, such as hiring a 
risk management coordinator and conducting quarterly program management 
meetings to review the program's progress in managing risks. However, 
the program has not fully implemented key practices essential to 
effectively managing project risks. For example, it does not have a 
transparent, documented, and traceable process for inventorying and 
resolving risks. Further, while the risk management plan defines a 
process and associated practices for developing risk mitigation 
strategies, the program has not fully implemented them. In addition, 
although the risk management plan calls for developing a process for 
elevating risks to management's attention, the program has not 
developed and implemented such a process. 

* Performance management practices are still being implemented and 
available data show mixed results and may not be reliable. We 
reported[Footnote 9] in July 2006 that Atlas had begun taking steps to 
implement performance goals and measures. Since then, the program has 
taken additional steps to define and implement them. For example, the 
program identified 13 goals--for 5 of its 7 projects--that it has been 
using to measure progress during fiscal year 2006. However, it has yet 
to fully implement and institutionalize its performance goals and 
measures. For example, the program has not yet developed performance 
goals for 2 projects. 

In addition, progress against performance goals, as reported by the 
program, show mixed results. For example, of the 12 goals, the program 
has met 3 of them. Further, underlying data used by the program to 
measure performance may not be reliable in all cases. For example, on 1 
goal, the program uses project cost and schedule data from the 2005 
Atlas business case in measuring progress against the goal, even though 
these data are out of date (e.g., they do not reflect project cost and 
schedule slippages that have occurred since 2005). 

Conclusions: 

The fiscal year 2006 Atlas expenditure plan, in combination with 
related program documentation and program officials' statements, 
satisfies or partially satisfies the legislative conditions set forth 
by Congress. However, this satisfaction is based on plans and 
commitments that provide for meeting these conditions, rather than on 
completed actions to satisfy the conditions. In addition, while steps 
are being initiated that are intended to address significant program 
management weaknesses, a number of improvements, including those 
recommended in our past reports, have yet to be implemented. 

Further, the program has not fully achieved many performance goals that 
it set out to accomplish over the past year. The above factors continue 
to put the program at risk and call for heightened and sustained 
management attention to expeditiously address and resolve the issues. 
Thus, there is much that still needs to be accomplished to minimize the 
risks associated with the program's capacity to deliver promised IT 
infrastructure capabilities and benefits on time and within budget. 
This includes demonstrating better progress against established 
performance goals in the coming year. 

Given that hundreds of millions of dollars are to be invested and the 
program is critical to supporting the ICE mission, it is essential that 
DHS follow through on its commitments to build the capacity to 
effectively manage the program. Proceeding without this capacity 
introduces unnecessary risks to the program and potentially jeopardizes 
its viability for future investment. 

Recommendations for Executive Action: 

To minimize risks to the Atlas program, we recommend that the Secretary 
of Homeland Security direct the Assistant Secretary for Immigration and 
Customs Enforcement to ensure that ICE follows through on its 
commitments to implement effective management controls and capabilities 
by implementing the following five recommendations: 

* Employ practices essential to ensuring that the Atlas program's 
independent verification and validation agent is and remains 
independent, including incorporating requirements in future contracting 
actions, such as the renegotiation or recompetition of the current 
independent verification and validation contract, which will prohibit 
the agent from soliciting, proposing, or being awarded program work 
other than providing independent verification and validation services 
and products. 

* Fully adhere to requirements development and management practices, 
including those specified in ICE's policies and procedures. This should 
also include having the Atlas program manager develop a process 
improvement plan for all of the projects that is consistent with the 
ICE System Lifecycle Management Handbook and provide for making the 
program manager and project managers responsible and accountable for 
rigorously adhering to the requirements in the handbook. 

* Fully implement key contract management and oversight practices, 
including those specified in ICE's policies and procedures. This should 
also include ensuring that the Atlas program manager, working with the 
program's contracting officer's technical representative, follow 
through on planned efforts to strengthen the program's compliance with 
these practices by (1) revising the acquisition plan by May 2007; (2) 
revising the ICE System Lifecycle Management Handbook by June 2007 to 
incorporate key contract management activities such as contract 
tracking and oversight; and (3) preparing statements of work (beginning 
in February 2007) for future contracts to clearly define Atlas work 
statements and acceptance criteria. 

* Complete implementation of planned risk management activities. This 
should include (1) updating the risk inventory to include risks for all 
projects and risk areas and (2) fully implementing and 
institutionalizing procedures for reporting to management on the 
existence and status of risks and progress in implementing mitigation 
strategies. 

* Improve program performance management, including developing 
performance goals for projects that do not have goals and reporting on 
their progress in the fiscal year 2007 expenditure plan. Further, the 
program should assess that the data being used to measure progress are 
reliable, complete, and accurate. 

Agency Comments and Our Evaluation: 

In DHS's written comments on a draft of this report, signed by the 
Director, Departmental GAO/Office of Inspector General Liaison Office, 
the department concurred with all of our recommendations and described 
actions planned and under way to implement them. The department also 
orally provided technical comments updating Atlas obligation amounts 
and the milestone for revising the acquisition plan. We incorporated 
these comments in the report and briefing as appropriate. DHS's written 
comments are reprinted in appendix II. 

We are sending copies of this report to the Chairman and Ranking 
Members of other Senate and House committees and subcommittees that 
have authorization and oversight responsibilities for homeland 
security. We are also sending copies to the Secretary of Homeland 
Security, the Assistant Secretary for Immigration and Customs 
Enforcement, and the Director of the Office of Management and Budget. 
Copies of this report will also be available at no charge on our Web 
site at [Hyperlink, http://www.gao.gov.]. 

Should you or your offices have any questions on matters discussed in 
this report, please contact me at (202) 512-3439 or at hiter@gao.gov. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. Key contributors 
to this report are listed in appendix III. 

Signed by: 

Randolph C. Hite: 
Director, Information Technology Architecture and Systems Issues: 

[End of section] 

Appendix I: Briefing to the Staffs of the Subcommittees on Homeland 
Security, Senate and House Committees on Appropriations: 

Information Technology: Immigration and Customs Enforcement Needs to 
Fully Address Significant Infrastructure Modernization Program 
Management Weaknesses: 

Briefing to the staffs of the: 

Subcommittees on Homeland Security: 

Senate and House Committees on Appropriations: 

February 2, 2007: 

Introduction: 

Objectives, Scope, and Methodology: 

Results in Brief: 

Background: 

Results: 

* Legislative Conditions: 

* Observations: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments: 

Attachment I: Detailed Scope and Methodology: 

Attachment II: GAO Analysis of Requirements Management: 

Introduction: 

The Department of Homeland Security's (DHS) Bureau of Immigration and 
Customs Enforcement (ICE)[Footnote 10] is responsible for enforcing 
immigration, border security, trade, and other laws by, among other 
means, investigating and collecting intelligence on individuals and 
groups that act to violate these laws. ICE is also responsible for 
protecting federal facilities. 

Atlas is the ICE program developed to modernize the bureau's 
information technology (IT) infrastructure, which includes the hardware 
(e.g., servers, routers, storage devices, communication lines) and 
system software (e.g., database management and operating systems and 
network management) that provide an environment for operating and 
maintaining software applications. 

According to ICE, the goals of the Atlas program include improving 
information sharing, strengthening information security, and improving 
workforce productivity. 

The fiscal year 2006 Department of Homeland Security Appropriations 
Act[Footnote 11] appropriated $40.15 million for Atlas[Footnote 12] and 
stated that none of the funds may be obligated until the department 
submits for approval to the Senate and House Committees on 
Appropriations a plan for how the funds are to be spent that satisfies 
the following legislative conditions: 

meets the capital planning and investment control review requirements 
established by the Office of Management and Budget (OMB), including OMB 
Circular A-11, part 7[Footnote 13];

complies with DHS's information systems enterprise architecture; 

complies with the acquisition rules, requirements, guidelines, and 
systems acquisition management practices of the federal government; 

includes certification by the DHS chief information officer (CIO) that 
an independent verification and validation agent is currently under 
contract for the project; 

is reviewed and approved by DHS's Investment Review Board[Footnote 14], 
the Secretary of Homeland Security, and OMB; and: 

is reviewed by GAO. 

DHS submitted its fiscal year 2006 expenditure plan for $39.75[Footnote 
15] million on November 9, 2006, to the Senate and House Appropriations 
Subcommittees on Homeland Security. 

Objectives, Scope, and Methodology: 

As agreed, our objectives were to: 

determine whether the Atlas fiscal year 2006 expenditure plan satisfies 
the legislative conditions and: 

provide other observations about the expenditure plan and DHS's 
management of the Atlas program. 

To address our first objective, we analyzed the fiscal year 2006 Atlas 
expenditure plan and supporting documents against legislative 
conditions and relevant federal requirements and related best practices 
to determine whether the conditions were being met. 

To address our second objective, we evaluated supporting documentation 
and interviewed relevant officials to determine progress in 
establishing program management capabilities in areas such as 
requirements development and management, contract management and 
oversight, risk management, and performance management. In assessing 
requirements management and contract management and oversight, we 
focused on three projects-Common Computing Environment, Integration, 
and ICE Mission Information-that collectively account for approximately 
75 percent of the funds provided to the program and that the Atlas 
program manager identified as being the most critical to the program's 
success and ICE's mission. We conducted our work at DHS and ICE 
headquarters in Washington, D.C., from July 2006 through January 2007 
in accordance with generally accepted government auditing standards. 
Details of our scope and methodology are provided in attachment 1. 

Results in Brief: 

Objective 1: Satisfaction of Legislative Conditions: 

1; 
Legislative conditions: Meets the capital planning and investment 
control review requirements established by OMB,including OMB Circular A-
11, part 7; 
Status: partially satisfies[Footnote 16]. 

2; 
Legislative conditions: Complies with the DHS information systems 
enterprise architecture; 
Status: partially satisfies. 

3; 
Legislative conditions: Complies with the acquisition rules, 
requirements, guidelines, and systems acquisition management practices 
of the federal government; 
Status: partially satisfies. 

4; 
Legislative conditions: Includes a certification by the chief 
information officer of the Department of Homeland Security that an 
independent verification and validation agent is currently under 
contract for the project; 
Status: partially satisfies. 

5; 
Legislative conditions: Is reviewed and approved by the DHS Investment 
Review Board, Secretary of Homeland security, and OMB; 
Status: satisfies[17]. 

6; 
Legislative conditions: Is reviewed by GAO; 
Status: satisfies. 

Source: GAO. 

[End of table]  

Objective 2: Other Observations: 

DHS has not implemented key system management practices essential to 
establishing an effective program management control environment. 
Specifically, in managing the Atlas program, the department has not: 

fully adhered to rigorous practices called for in developing and 
managing system requirements, 

implemented key contract management and oversight controls, 

completed implementation of planned risk management practices, or: 

implemented performance management practices critical to measuring 
progress against program goals. 

These shortfalls are attributable in part to a number of factors, 
including the Atlas program's expedited schedule to deploy modernized 
IT infrastructure capabilities to users in the wake of the events of 
September 11, 2001, and in the haste of doing so, not following these 
practices or related ICE policies and procedures. Until these practices 
are fully implemented, the program is at an increased risk of not being 
able to deliver modern IT infrastructure capabilities on time and 
within budget. Program officials acknowledge the shortfalls and have 
efforts planned or under way to address some, but not all, of the 
missing practices. 

Accordingly, we are making recommendations to the Secretary of Homeland 
Security to strengthen the Atlas program by implementing and 
institutionalizing these key system management practices, thereby 
helping to establish an effective program management control 
environment. In commenting on a draft of this briefing, ICE officials, 
including the deputy chief information officer who is also the Atlas 
program manager, agreed with our results. 

Background ICE: 

ICE was formed as a component agency of DHS in 2003 when the law 
enforcement functions of the Department of Justice's former Immigration 
and Naturalization Service and Department of the Treasury's former 
Customs Service and other agencies were merged into DHS. 

The ICE mission is to ensure the security of the American people and 
homeland by, among other means, investigating violators of and 
enforcing the nation's immigration and customs laws; policing and 
securing federal buildings and other facilities; and collecting, 
analyzing, and disseminating intelligence to assist in these endeavors. 
Headed by the Assistant Secretary of Immigration and Customs 
Enforcement, ICE has approximately 15,000 employees in more than 400 
offices in the United States and in other countries. 

The Atlas program was started by the Immigration and Naturalization 
Service in 2002. Responsibility for the program was transferred to ICE 
in 2003. The figure on the following slide is a simplified version of 
the current ICE and Atlas organizational placements within DHS. 

Background: ICE and Atlas Organizational Placement: 

[See PDF for image] 

Source: GAO analysis of DHS data. 

[End of figure] 

Background: Program Impetus and Goals: 

Impetus for Program: 

According to ICE officials, Atlas was initiated to address information 
sharing and security limitations within the former Immigration and 
Naturalization Service caused by, for example, 

obsolete hardware/software; 

incompatible, noninteroperable information systems; and: 

system security weaknesses. 

These officials stated that these challenges were exacerbated by the 
formation of ICE because the organizations merged into ICE had 
different hardware/software environments (e.g., multiple e-mail 
systems) and different missions and customer needs. 

Program Goals: 

The stated goals of Atlas are, among other things, to: 

promote information sharing and collaboration, 

strengthen information security, and: 

enhance workforce productivity. 

Background Atlas Projects: 

Atlas Consists of Seven Projects: 

Project: Common Computing Environment; 
Description: * Deploy compatible desktops, servers, and standard office 
automation applications;  
* Deploy a common e-mail platform;  
* Integrate ICE with a shared departmental active directory structure. 

Project: Integration; 
Description: * Migrate from the ICE network to the DHS OneNetwork 
infrastructure; 
* Provide streaming video;  
* Refresh local area network cabling and electronics and replace legacy 
firewalls;  
* Refresh the wide area network and local area network communications 
hardware at the overseas ICE attaché offices;  
* Provide network support for all applications and provide converged 
communications among all system users. 

Project: ICE Mission Information; 
Description: * Provide enterprise data warehousing capabilities and 
integrate related ICE business processes; 
* Organize information and support user access so ICE users can find 
relevant, timely, and reliable information. 
* Improve information searching and indexing capabilities and integrate 
legacy applications with service-oriented techniques. 

Project: Information Assurance; 
Description: * Institute strong information, system/ application 
platform, network, and computer security measures; 

Project: Architecture Engineering; 
Description: * Provide state-of-the-art engineering facilities and 
tools to ensure new technologies align to the DHS enterprise 
architecture (EA).  

Project: Data Center Migration; 
Description: * Migrate ICE hardware and applications from the 
Department of Justice data centers to the common DHS data center; 
* Ensure compliance with the DHS EA. 

Project: Transformation Planning; 
Description: * Oversee and coordinate the various Atlas projects; 
* Ensure compliance with the DHS EA; 
* Ensure adherence to project cost, schedule, and performance goals. 

Source: GAO analysis of DHS data. 

[End of table] 

Figure: 

[See PDF for image] 

Source: GAO analysis based on DHS appropriation laws. 

[A] Counterterrorism funding came from Pub. L. No. 107-117, Jan. 10, 
2002; H.R. Rep. 107-350, Dec. 19, 2001. 

[B] Pub. L. No. 108-90, Oct. 1, 2003. 

[C] Pub. L. No. 108-334, Oct. 18, 2004. 

[D] DHS's fiscal year 2006 appropriations (Pub. L. No. 109-90, Oct. 18, 
2005) provided $40.15 million for Atlas. In December 2005, this amount 
was reduced to $39.7 million via a 1 percent governmentwide rescission 
in Pub. L. No. 109-148, Division B, Dec. 30, 2005. [E] pub. L. No. 109-
295, Oct. 4, 2006. The law mandates that DHS also submit an expenditure 
plan prior to obligation of these funds. 

[End of figure] 

ICE reports that it has obligated $84.7 million of the $113.0 million 
available for Atlas in fiscal years 2002, 2003, 2004, and 2005. 

Table: Atlas funds obligated (by project) for fiscal years 2002-2005 
(as of December 31, 2006): 

Project (in order presented in the plan): Common Computing Environment; 
Obligations (in millions): FY 2002: $0; 
Obligations (in millions): FY 2003: $10.5; 
Obligations (in millions): FY 2004: $3.2; 
Obligations (in millions): FY 2005: $4.3; 
Total: $18.0. 

Project (in order presented in the plan): Integration; 
Obligations (in millions): FY 2002: $6.1; 
Obligations (in millions): FY 2003: $20.4; 
Obligations (in millions): FY 2004: $2.5; 
Obligations (in millions): FY 2005: $0.5; 
Total: $29.5. 

Project (in order presented in the plan): ICE Mission Information; 
Obligations (in millions): FY 2002: $1.3; 
Obligations (in millions): FY 2003: $4.6; 
Obligations (in millions): FY 2004: $0.8; 
Obligations (in millions): FY 2005: $4.0; 
Total: $10.7. 

Project (in order presented in the plan): Information Assurance; 
Obligations (in millions): FY 2002: $10.2; 
Obligations (in millions): FY 2003: $9.6; 
Obligations (in millions): FY 2004: $0; 
Obligations (in millions): FY 2005: $0.6; 
Total: $20.4. 

Project (in order presented in the plan): Architecture Engineering; 
Obligations (in millions): FY 2002: $0; 
Obligations (in millions): FY 2003: $0; 
Obligations (in millions): FY 2004: $0; 
Obligations (in millions): FY 2005: $0.7; 
Total: $0.7. 

Project (in order presented in the plan): Data Center Migration; 
Obligations (in millions): FY 2002: $0; 
Obligations (in millions): FY 2003: $0; 
Obligations (in millions): FY 2004: $0; 
Obligations (in millions): FY 2005: $0; 
Total: $0. 

Project (in order presented in the plan): Transformation planning; 
Obligations (in millions): FY 2002: $0; 
Obligations (in millions): FY 2003: $0.9; 
Obligations (in millions): FY 2004: $3.3; 
Obligations (in millions): FY 2005: $1.2; 
Total: $5.4. 

Total; 
Obligations (in millions): FY 2002: $17.6; 
Obligations (in millions): FY 2003: $46.0; 
Obligations (in millions): FY 2004: $9.8; 
Obligations (in millions): FY 2005: $11.3; 
Total: $84.7. 

Source: GAO analysis of DHS data. 

[End of table] 

Background: Planned Use of Fiscal Year 2006 Funding: 

According to the fiscal year 2006 expenditure plan, the $39.75 million 
is to be spent on the projects as follows. 

Table: Distribution of funding for Atlas projects (in order presented 
in the plan): 

Project: Common Computing Environment; 
Funding (in millions): $23.88; 
Planned use: 
* Complete acquisition of hardware refresh; 
* Deploy active directory and e-mail exchange system. 

Project: Integration; 
Funding (in millions): $2.30; 
Planned use: 
* Replace outdated communications electronics at foreign attaches to 
ensure compatibility with current ICE Net WAN and LAN infrastructure 
standards; 
* Complete acquisition of streaming video for field sites. 

Project: ICE Mission Information; 
Funding (in millions): $5.84; 
Planned use: 
* Complete acquisition of technology to efficiency integrate all 
related ICE business processes; 
* Complete 60 percent of acquisition of technology to support data 
warehousing capabilities. 

Project: Information Assurance; 
Funding (in millions): $0.92; 
Planned use: 
* Acquire IT security specialists to design and develop single sign-on 
and audit log capabilities. 

Project: Architecture Engineering; 
Funding (in millions): $0; 
Planned use: 
* No use of fiscal year 2006 funds identified. 

Project: Data Center Migration; 
Funding (in millions): $0.89; 
Planned use: 
* Plan for the migration of ICE hardware and applications from two 
Department of Justice data centers to a common DHS data center. 

Project: Transformation Planning; 
Funding (in millions): $5.92; 
Planned use: 
* Provide $2.3 million to continue performing and planning program 
management activities, including maintaining the Atlas business case; 
* Provide $3.6 million in management reserve for unanticipated program 
costs or to fund priority emerging IT modernization requirements. 

Total: $39.75. 

Source: GAO analysis of DHS data. 

[End of table] 

Objective 1 Results Legislative Conditions: 

The Atlas expenditure plan satisfies or partially satisfies each of the 
legislative conditions. 

Condition 1 partially satisfied. The expenditure plan, including 
related program documentation and statements from the program manager, 
partially satisfies the capital planning and investment control review 
requirements established by OMB, including Circular A-11, part 7, which 
establishes policy for planning, budgeting, acquisition, and management 
of federal capital assets. 

Examples of our analysis are included in the following table. As the 
table shows, not all OMB requirements have been satisfied, but oral 
commitments have been made for doing so. Given that ICE has reportedly 
already invested $75.4 million on Atlas projects and plans to invest 
another $39.7 million this year, it is important for ICE to follow 
through on these commitments. 

Objective 1 Results Legislative Conditions: 

GAO Analysis of Atlas Compliance with Legislative Conditions. 

Examples of OMB Circular A- 11 requirements: Indicate whether the 
investment was approved by an investment review committee; 
Results of our analysis: The plan was approved on May 11, 2006, by 
DHS’s Deputy Secretary, who chairs the DHS Investment Review Board. 

Examples of OMB Circular A- 11 requirements: Provide justification and 
describe acquisition strategy; 
Results of our analysis: A business case, including a cost-benefit 
analysis, was issued in December 2005 to provide economic justification 
for the program. It estimated the program’s life cycle costs to be 
approximately $1 billion through the year 2024. The program plans to 
issue an updated cost-benefit analysis in September 2007 to reflect 
emerging requirements and other program changes (e.g., cost and 
schedule slippages). In addition, the expenditure plan and supporting 
documents (e.g., fiscal year 2006 Atlas budget submission to OMB known 
as an exhibit 300) provide aspects of a high-level acquisition 
strategy, such as identifying the ICE contracts that are being used and 
are to be used to acquire hardware and software products and program 
support services. An acquisition plan was approved in February 2006 
that includes a statement of need and the capabilities to be delivered 
through these contracts. To help ensure compliance with contract 
criteria and Atlas program objectives, a contracting officer’s 
technical representative (COTR) was hired in November 2006. The 
program’s efforts to ensure compliance with contract management and 
oversight practices are more fully discussed in the Observations 
section of this briefing. 

Examples of OMB Circular A- 11 requirements: Summarize life cycle costs 
and cost-benefit analysis, including the return on investment; 
Results of our analysis: The December 2005 cost-benefit analysis 
provides costs and benefits for the life cycle of Atlas, which is 
through the year 2024. The analysis also included an estimated return 
on investment for three alternative approaches. As we previously 
reported[Footnote 18], the analysis was in large part consistent with 
OMB and best practices guidance, but it did omit key practices such as 
key mission requirements. According to the program manager, the 
analysis is considered to be a “living document” and the program plans 
to issue an update in September 2007 that, among other things, 
addresses these requirements. 

Examples of OMB Circular A- 11 requirements: Provide performance goals 
and measures; 
Results of our analysis: The expenditure plan and supporting 
documentation (e.g., prior expenditure plans) identify 13 proposed 
goals and measures, but two projects do not have measures. Proposed 
measures have been drafted for these projects, but target dates for 
their completion have not been identified by the program. In addition, 
as part of the December 2005 Atlas business case, the program mapped 
Atlas’s goals to ICE’s mission and goals. Our analysis of Atlas’s 
performance management is discussed in more detail in the Observations 
section of this briefing. 

Examples of OMB Circular A- 11 requirements: Address security and 
privacy; 
Results of our analysis: The plan and supporting documentation state 
the importance of security and privacy and provide high-level 
information on intended security measures, including one proposed 
project—information assurance—that is intended to implement an ICE 
security program. The plan allocates $0.9 million to this project in 
addition to the $1.9 million provided in the fiscal year 2005 plan. In 
addition, the program issued an Atlas system security plan, as well as 
a security test and evaluation plan, in April 2006. The program also 
recently issued an updated system security plan and security test and 
evaluation plan in December 2006. ICE also developed a draft Atlas 
privacy impact assessment in August 2005, submitted a revised 
assessment in August 2006 to the DHS privacy office, and plans to 
complete and submit the final Atlas privacy impact assessment by April 
2007. 

Examples of OMB Circular A- 11 requirements: Provide for managing risk; 
Results of our analysis: The program issued a risk management plan in 
January 2006. This plan provides guidance for identifying, analyzing, 
and resolving program risks before they occur. In addition, a risk 
management coordinator was hired in August 2006 to help identify and 
monitor risks. According to the program manager, this official recently 
vacated the position, and the program is currently in the process of 
hiring a new coordinator. (At our Feb. 1, 2007, exit meeting with DHS 
and ICE officials, the program manager told us a coordinator had been 
hired on Jan. 22, 2007.) Further, while the program procured an 
automated tool in May 2006 to help complete the inventory and manage 
risks, it does not have, among other things, a complete inventory of 
all program risks. Our complete findings related to risk management are 
addressed in the Observations section of this briefing. 

Source: GAO analysis of DHS data. 

[End of table] 

Condition 2 partially satisfied. The plan, including related program 
documentation and DHS officials' statements, partially satisfies the 
condition that the department ensures Atlas is compliant with DHS's 
enterprise architecture (EA). 

An EA provides a clear and comprehensive picture of an organization's 
operations and its supporting systems and infrastructure. It is an 
essential tool for effectively and efficiently engineering business 
processes and for implementing and evolving supporting systems in a 
manner that maximizes interoperability, minimizes overlap and 
duplication, and optimizes performance. We have worked with Congress, 
OMB, and the federal Chief Information Officers Council to highlight 
the importance of architectures for both organizational transformation 
and IT management. An important element of EA management is ensuring 
that IT investments are compliant with the EA, including basing such 
assessments on documented analysis. 

The DHS Enterprise Architecture Board, supported by the Enterprise 
Architecture Center of Excellence,[Footnote 19] is responsible for 
ensuring that projects demonstrate adequate technical and strategic 
compliance with the department's EA. To this end, the board conducts 
compliance reviews at key decision points in an investment's life 
cycle. Specifically, DHS guidance[Footnote 20] directs the board prior 
to an investment's acquisition milestone (referred to by DHS as key 
decision point 2) to assess the investment against the transition 
strategy, data architecture, application component, and technology 
architecture. 

We previously reported[Footnote 21] in May 2005 that the Atlas program 
manager requested that the center assess the program's compliance with 
the EA. The center staff compared Atlas to version 2.0 of the DHS EA 
and reported the results of its assessment to the board, stating that 
Atlas was in compliance with the DHS EA. In July 2005, the board 
approved this compliance determination. 

Subsequently, we reported[Footnote 22] that ICE'S compliance 
determination was not based on a documented analysis mapping Atlas's 
infrastructure to the DHS EA. Specifically, the department did not have 
a documented methodology for evaluating programs for compliance with 
the DHS EA, other than relying on the expertise of the Center of 
Excellence members. 

Since the July 2005 compliance determination, DHS issued version 2006 
of its EA.[footnote 23] In addition, Atlas has undertaken an effort to 
define an architecture framework and technical environment for a data 
warehouse aimed at providing reporting and analytical capabilities to 
investigative and enforcement personnel. Despite these changes, no new 
compliance reviews have been conducted. According to the Atlas program 
manager, a compliance review was not required during this time as the 
program had not yet reached a phase requiring it. The program manager 
stated the department has scheduled an EA compliance review in March 
2007. 

In addition, program officials told us that they are taking other steps 
aimed at ensuring compliance. These steps include: 

participating in the DHS Enterprise Architecture Center of Excellence 
and the Data Management Working Group; these groups are currently 
tasked with developing the DHS EA and the DHS Data Reference Model, 

participating in DHS EA initiatives, such as serving on the Service 
Oriented Architecture Tactical Working Group,[Footnote 24] which is to 
provide the framework for departmentwide service alignment and 
interoperability efforts, and: 

establishing an Atlas Interoperability Lab-under the Architecture 
Engineering project-to among other things, help ensure the program's 
projects are compliant with the EA. 

These steps notwithstanding, until DHS demonstrates, through verifiable 
documentation and a methodologically based analysis, that ICE is 
aligned with the DHS enterprise architecture, the program will remain 
at risk of being defined and implemented in a way that does not support 
optimal departmentwide operations, performance, and achievement of 
strategic goals and outcomes. 

Condition 3 partially satisfied. The plan, including related program 
documentation and statements from the Atlas program manager, partially 
satisfies the condition to comply with the acquisition rules, 
requirements, guidelines, and systems acquisition management practices 
of the federal government. These practices provide a management 
framework based on the use of rigorous and disciplined processes for 
planning, managing, and controlling the acquisition of IT resources, 
including: 

acquisition planning, which ensures, among other things, that 
reasonable plans, milestones, and schedules are developed and that all 
aspects of the acquisition effort are included in these plans; 

procurement, which involves making sure that (1) a request for 
proposals delineating a project's requirements is prepared and (2) 
consistent with relevant acquisition laws and regulations, a contractor 
is selected that can cost-effectively satisfy these requirements; 

requirements development and management, which includes establishing 
and maintaining a common and unambiguous definition of requirements 
among the acquisition team, the system users, and the development 
contractor; 

project management, which provides for management of the activities 
within the project office and supporting contractors to ensure a 
timely, efficient, and cost-effective acquisition; 

contract tracking and oversight, which ensures that the development 
contractor performs according to the terms of the contract; needed 
contract changes are identified, negotiated, and incorporated into the 
contract; and contractor performance issues are identified early, when 
they are easier to address; and: 

evaluation, which determines whether the acquired products and services 
satisfy contract requirements before acceptance. 

These acquisition management processes are also embodied in published 
best practices models, such as the Software Engineering Institute's 
Software Acquisition Capability Maturity Model[Footnote 25] 

Examples of our analysis of ICE performance of these processes and 
practices are shown in the following table. They show that not all 
aspects of the processes and practices have been implemented, but that 
oral commitments have been made for doing so. 

Table: GAO Analysis of Atlas with Legislative Conditions. 

Example of practices: Acquisition Planning; Ensures that reasonable 
plans, milestones, and schedules are developed and that all aspects of 
the acquisition effort are included in these plans; 
Results of our analysis: The expenditure plan and supporting documents 
(e.g., fiscal year 2006 Atlas budget submission to OMB known as an 
exhibit 300) provide aspects of a high-level acquisition strategy, such 
as identifying the ICE contracts that are being used and are to be used 
to acquire products and program support services. An acquisition plan 
was issued in February 2006, and it includes a statement of the 
capabilities to be delivered through these contracts;  
To manage and oversee the contracts, including helping ensure 
compliance with contract performance criteria and Atlas program 
objectives, the program hired a COTR in November 2006. In addition, the 
business case issued in December 2005 provides details on alternatives, 
cost, and schedule. The program plans to issue an updated business case 
and cost-benefit analysis by September 2007 to reflect emerging 
requirements and other program changes (e.g., project cost and schedule 
slippages). In addition, Atlas is also updating and revising its 
January 2006 acquisition program baseline, which is to include cost and 
schedule baselines for its projects, with the goal of finalizing it by 
March 2007; 
According to the program manager, Atlas is currently following the 
February 2005 ICE System Lifecycle Management Handbook.[Footnote 26]  
The handbook addresses a number of key process areas such as project 
management and requirements development and management; however, it 
does not address certain key acquisition management activities such as 
solicitation and contract tracking and oversight. According to the 
Atlas program manager, he acknowledges this problem and plans to update 
the handbook by June 2007 to include the missing acquisition management 
activities. This area is more fully discussed in the Observations 
section of this briefing.

Example of practices: Project Management; Provides for the management 
of activities within the project office and supporting contractors to 
ensure a timely, efficient, and cost-effective acquisition; 
Results of our analysis: ICE has established a program office with 
responsibility for managing the acquisition, deployment, operation, and 
sustainment of Atlas. The Atlas program management office is to be 
allocated funding of $2.3 million via the Transformation Planning 
project in the expenditure plan. The current staffing of the program 
office consists of a program manager, who is also the deputy chief 
information officer; a deputy program manager; a manager for each of 
the seven Atlas projects; and other personnel (e.g., a business analyst 
or a communications specialist). According to the program manager, the 
program office is now fully operational. In addition, a project plan 
for each of the projects was approved by the program manager in the 
latter part of 2006. 

Source: GAO analysis of DHS data. 

[End of table] 

Condition 4 partially satisfied. DHS partially satisfies the 
legislative condition requiring that the plan include a certification 
by the department's CIO that an IV&V agent is currently under contract 
for the project. 

In an October 25, 2006, letter to ICE's CIO, DHS's Deputy CIO certified 
that an IV&V agent is under contract for Atlas. However, there are 
unresolved issues with the Atlas program's efforts in this area. First, 
in the Deputy CIO's letter, he requested additional information and 
made recommendations regarding the IV&V agent, asking for a response by 
January 31, 2007. Specifically, this official: 

requested a copy of the IV&V contract and the agent's work plan in 
order to validate, among other things, whether planned IV&V activities 
are in accordance with industry quality standards and whether the agent 
has sufficient staff and: 

recommended that the Atlas program (1) review whether the information 
assurance and security experience of key IV&V personnel was sufficient 
and (2) appoint a full-time COTR before the end of the first quarter of 
2007. 

The program manager stated that he is currently drafting the response 
on behalf of ICE's CIO with the goal of transmitting it to DHS by the 
first or second week of February 2007. 

Second, as we have previously reported[Footnote 27], the independence 
of an IV&V agent is critical to providing management with objective 
insight into program processes and associated work products. We also 
reported that to be effective, the verification and validation function 
is to be performed by an entity that is independent of the processes 
and products that are being reviewed. When the agent engaged to provide 
the service is a contractor, a practice to help ensure independence is 
to include in the agent's contract a provision that prohibits the agent 
from soliciting, proposing, or being awarded program work other than 
the IV&V services and products. 

The contract that Atlas awarded to its IV&V agent (dated Sept. 13, 
2006) does not include such a provision. Instead, the contract states 
that the contractor will notify the agency of any potential conflicts 
of interest in performing work under the contract. The Atlas program 
manager told us he did not believe incorporating the contractual 
provision called for by the key practice was necessary because the 
agent's only line of work is performing IV&V activities, and he agreed 
to provide documentation to support this. He also said that in the 
event the agent solicited, proposed, or is awarded other work on the 
Atlas program, he would replace the agent with one that was 
independent. However, the program manager did not provide documentation 
to IV&V being the agent's sole line of business. In addition, having to 
replace an agent could delay program activities and is not consistent 
with the key practice to avoid such problems. Consequently, until the 
practice to ensure independence is included in the agent's contract, 
there is an increased risk that the agent's independence could become 
impaired, resulting in the program being unable to rely on the agent's 
work. 

Condition 5 satisfied. DHS and OMB satisfied the legislative condition 
requiring that the plan be reviewed and approved by the DHS Investment 
Review Board, Secretary of Homeland Security, and OMB. 

The DHS Deputy Secretary, who chairs the DHS Investment Review Board, 
approved the plan on May 11, 2006. 

OMB approved the plan on September 14, 2006. 

Condition 6 satisfied. GAO satisfied the condition that it review the 
plan. 

Our review was completed in January 2007. 

Objective 2 Results: 

Observation 1: Requirements Management: 

Observation 1: Atlas did not fully adhere to rigorous requirements 
development and management practices: 

Federal IT management guidance and relevant best practices[Footnote 28] 
recognize the importance of effectively developing and managing system 
requirements. According to this criteria, well-defined requirements are 
important because they establish agreement among the various 
stakeholders on what the system is to do, how well it is to do it, and 
how it is to interact with other systems. Effective requirements 
development and management involves, among other things: 

establishing a requirements management policy; 

eliciting desired or required system capabilities from users and 
translating them into system requirements and obtaining approval from 
all stakeholders before deploying system capabilities; 

establishing a system concept of operations, which describes the 
business process to be supported and how users will interact with the 
proposed system, and defines system requirements; 

ensuring that changes to requirements are controlled and approved via a 
disciplined change control process as the system is developed and 
implemented; and: 

maintaining bidirectional traceability, meaning that a given 
requirement can be traced backward to systems documentation and forward 
to the appropriate contract vehicle and system components that will 
satisfy the requirement and the test that will verify that it is 
satisfied. 

As we reported in July 2006,[Footnote 29] ICE incorporated these 
practices in its ICE System Lifecycle Management Handbook,[Footnote 30] 
which was issued in February 2005. However, the program did not fully 
follow these practices on its three key projects: Common Computing 
Environment (CCE), Integration, and ICE Mission Information. For 
example, on CCE, while the project established the ICE System Lifecycle 
Management Handbook as the policy to guide project efforts in 
developing and managing requirements, it did not: 

document stakeholder comments as part of the requirements elicitation 
process and review and obtain stakeholder approval on requirements 
until approximately 1 year after deploying system capabilities; 

develop a concept of operations until several years after the project 
had been initiated, specifically, the project was initiated in March 
2003 and began deploying system capabilities in December 2005, but the 
concept of operations was not developed until June 2006; 

follow a disciplined change management process for several years 
(between 2003 and 2006) after the project was initiated in March 2003 
(during this time, requirements were introduced and implemented in an 
ad hoc fashion); and: 

develop supporting analysis showing traceability between requirements, 
system design, and the contract. 

Our assessment of the extent to which the program complied with these 
practices on the three projects is shown in attachment II. 

According to ICE officials, including the program and project managers, 
the reason that the Atlas projects did not fully adhere to these 
practices is due to the agency's expedited schedule to deploy 
modernized IT infrastructure capabilities to users in the wake of the 
events of September 11, 2001. Specifically, the officials stated that 
in their haste to deliver capabilities, the project manager skipped 
certain system management life cycle practices such as these. 

The program manager acknowledged that requirements management needs to 
be improved and stated that key steps are being taken to ensure the 
projects closely follow the practices specified in the ICE System 
Lifecycle Management Handbook. He cited recent efforts by the CCE 
project manager to follow such practices on work recently initiated to 
refresh hardware, such as desktops and laptop computers across the 
agency. For example, in January 2007, the project conducted a 
workstation requirements analysis survey that was sent out to ICE users 
to identify requirements regarding the current workstation being used 
and issues that needed to be addressed in order to perform functions 
associated with their jobs. In accordance with the handbook, project 
staff drafted requirements, which they plan to finalize and approve by 
February 2007. While these are steps in the right direction on this 
particular project, the program manager was not able to provide us with 
a process improvement plan on how the program going forward would 
ensure compliance with requirements development and management 
practices across all of the projects. 

Consequently, until Atlas fully implements effective requirements 
development and management practices programwide, it faces the 
increased likelihood that its projects will not meet user needs, 
operate as intended, or be delivered on time and within budget. In 
addition, there is evidence that this has already occurred. For 
example, on CCE, the delivery of its initial operating capability (the 
date when it was to begin providing common e-mail capabilities to ICE 
users) was delayed from September 2005 to December 2005. According to 
the project manager, this delay is attributable to, among other things, 
the project not following these practices. Specifically, the project 
manager told us that the project experienced numerous changes to 
requirements when it was transferred from the Department of Justice to 
DHS and that these changes were not documented well. As a result, there 
was extensive system rework to address missing requirements, which 
caused schedule delays. 

Objective 2 Results: Observation 2: Contract Management and Oversight: 

Observation 2: Key contract management and oversight practices are not 
fully implemented: 

The Software Engineering Institute's CMMI[Footnote 31] identifies best 
practices that are essential to effectively managing and overseeing 
contracts that support IT projects. These best practices include: 

establishing and maintaining a plan for managing and overseeing 
contracts, 

assigning responsibility and authority for performing contract 
management and oversight, 

training staff performing contract management and oversight activities, 

documenting each contract, including the statement of work and 
acceptance criteria, 

verifying and accepting contractor-provided products and services 
(i.e., deliverables), and; 

conducting reviews with contractors to ensure that cost and schedule 
commitments are being met and risks are being managed. 

As we reported[footnote 32] in June 2006, the Atlas program has largely 
established policies and procedures for these practices. While the 
program has implemented two of the practices, it has not fully 
implemented the other four. Specifically, our analysis of the program's 
performance of these practices, including efforts planned and underway, 
are shown in the following table. 

Implementation of Contract Management and Oversight Practices for Atlas 
Program. 

1; 
CMMI practice: Establish and maintain a plan for managing and 
overseeing contracts; 
Implemented: no; 
GAO's assessment of compliance: The program office issued a plan in 
February 2006 (referred to as an acquisition plan by program 
officials), but it does not describe contract management and oversight 
processes. Instead, the plan simply states that Atlas is to have a COTR 
work with program and contracting officials to administer the 
contracts. In November 2006, the DHS Acquisition Policy and Oversight 
Office, which is responsible for reviewing and approving the 
acquisition plan, stated that the plan lacked details on contract 
management and oversight and recommended that Atlas develop, among 
other things, a contract management and oversight plan and incorporate 
it into the acquisition plan. In addition, we reported in July 2006 
that the ICE System Lifecycle Management Handbook, which is used to 
help manage the Atlas program, does not address certain key acquisition 
management activities, such as contract tracking and 
oversight.[Footnote 33]  Atlas’s program manager stated that the 
acquisition plan and the handbook are being revised to include contract 
management and oversight activities. The acquisition plan is to be 
revised by March 2007 and the handbook by June 2007. 

2; 
CMMI practice: Assign responsibility and authority for performing 
contract management and oversight; 
Implemented: yes; 
GAO's assessment of compliance: The program office hired a COTR in 
November 2006, and ICE assigned the individual the responsibility and 
authority for Atlas contract management and oversight. The program 
office also provided one contractor staff member to assist the COTR. 
The contractor’s responsibilities include helping develop statements of 
work, delivery schedules, and inspection and acceptance procedures. 
Further, each of the project managers has been tasked by the program 
manager to assist the COTR in identifying requirements and providing 
assistance in determining whether contractor deliverables meet 
requirements. 

3; 
CMMI practice: Train staff performing contract management and oversight 
activities; 
Implemented: yes;  
GAO's assessment of compliance: The program’s COTR has extensive 
contract oversight training and experience. For example, during 2005 
and 2006, the official took 120 hours of training on topics such as 
systems acquisition management, COTR responsibilities, the use of 
government purchase cards, and contracting officer’s representative 
mentoring. Further, this official’s prior experience includes 5 years 
of managing and overseeing contracts at the Department of Defense, at a 
civilian agency, and in private industry. Also, all but one of the 
Atlas project managers received contract management and oversight 
training in October 2006. The remaining project manager is to receive 
the training by June 2007. 

4; 
CMMI practice: Document each contract, including clearly identifying 
the work to be performed and acceptance criteria; 
Implemented: no; 
GAO's assessment of compliance: On its three key projects—namely CCE, 
Integration, and the ICE Mission Information projects—the program did 
not satisfy this practice. Specifically, the contracts for these three 
projects did not clearly identify the work to be performed and the 
criteria for accepting contractor deliverables. Instead, the contracts 
identified the work to be performed and the acceptance criteria in 
general terms. For example, the contract for deploying CCE (a standard 
e-mail system agencywide) states that the contractor is responsible for 
providing deployment services, but does not identify that deploying a 
standard e-mail system is one of those services. Similarly, regarding 
acceptance criteria, the CCE contract states that the contractor will 
provide monthly status reports, but does not identify how the program 
will determine whether the deliverable is acceptable, such as 
describing tests to be performed.
Program officials, including the program manager and the COTR, 
acknowledged that the statement of work and the acceptance criteria are 
not clearly defined, and that this makes contract administration 
difficult. They added that this difficulty extended to the other Atlas 
contracts because the program managed these contracts in the same 
manner as the other three. To address the problem, the program manager 
stated that he has tasked the COTR to work with project managers in 
developing clear statements of work and acceptance criteria for new 
contracts that are to be issued by late February 2007 to replace 
existing Atlas contracts that will begin to expire at that time. 

5; 
CMMI practice: Verify and accept the deliverables; 
Implemented: partially; 
GAO's assessment of compliance: The COTR’s responsibilities include 
verifying and accepting contract deliverables. On the contracts of the 
three projects reviewed, the COTR, in assessing whether to accept 
deliverables, did not compare them with specific criteria in the 
contract because none existed (see the previous slide). Instead, the 
COTR, in consultation with the staff from the involved projects and ICE 
offices of budget and acquisition management, reviewed each 
contractor’s invoices to assess whether it appeared what the contractor 
was charging was reasonable for the deliverables provided. If consensus 
was reached among these parties, then the COTR would approve the 
invoices for payment. 

6; 
CMMI practice: Conduct reviews with contractors to ensure cost and 
schedule commitments are being met and risks are managed; 
Implemented: partially; 
GAO's assessment of compliance: The program manager stated that he and 
the ICE CIO conduct reviews with program contractors on a quarterly 
basis to determine whether cost and schedule commitments are being met 
and to review project risks. However, he could not provide 
documentation that the meetings occurred due to the program not 
documenting the meetings. In addition, the contractors provide reports 
to program officials on their progress in meeting commitments and to 
identify risks related to each project. However, according to the 
program manager, assessing meaningful contractor progress from these 
reports is difficult without clearly defined work-to-be-performed 
statements and acceptance criteria. 

Source: GAO analysis of DHS data. 

[End of table] 

The Atlas program manager stated that the program did not fully adhere 
to these practices due to it not having until recently a COTR to manage 
and ensure compliance with contract management and oversight 
requirements. He also stated that to strengthen the program's 
compliance with such practices, he has tasked the COTR with (1) 
revising the acquisition plan (by March 2007) to provide detailed 
guidance on how Atlas will manage and oversee contracts, (2) revising 
the ICE System Lifecycle Management Handbook by June 2007 to 
incorporate key contract management activities such as contract 
tracking and oversight, and (3) preparing statements of work (beginning 
in February 2007) for future contracts to clearly define Atlas work 
statements and acceptance criteria. While these are steps in the right 
direction, they are works in progress and, as such, have not been 
completed. 

Until these steps and the other practices are fully implemented and 
institutionalized, Atlas faces the risk that it will not deliver 
required capabilities and promised benefits on time and within budget. 
It also makes the program vulnerable to contract mismanagement. 

Objective 2 Results: Observation 3: Risk Management: 

Observation 3: Key risk management practices have yet to be 
implemented: 

Federal guidance and related best practices[Footnote 34] identify key 
practices essential for effectively managing risks that may impact an 
IT project. They include, among other things: 

identifying and prioritizing risks as to their probability of 
occurrence and impact, as well as documenting them in an inventory; 

developing and implementing the appropriate risk mitigation strategies; 
and: 

reporting to management on the existence and status of risks and 
progress in implementing mitigation strategies. 

In July 2006, we reported[Footnote 35] that the program had developed a 
risk management plan and process to guide program office staff in 
managing risks throughout each project's life cycle. We also reported 
that although the program had recently begun to implement the risk 
management process, a complete inventory of risks had not been 
developed. Since then, the program has taken additional steps to 
implement risk management. For example, the program hired a risk 
management coordinator to maintain and update the risk management plan, 
facilitate risk assessments, track efforts to reduce risks to 
acceptable levels, and develop and conduct risk management training. In 
addition, the program has conducted periodic risk management meetings 
and quarterly program management meetings to review the program's 
progress in managing risks. 

Further, the program conducted risk management training with project 
managers on October 24, 2006. The training included guidance on, among 
other things, adhering to key risk management practices such as 
preparing risk assessments and developing risk mitigation strategies. 

However, the program has yet to implement other key practices. First, 
it does not have a transparent, documented, and traceable process for 
inventorying and resolving risks. For example, the January 23, 2006, 
inventory identified three risks related to data/information, security, 
and privacy. These risks were previously reported as having a medium 
probability of occurrence and a low impact. However, the January 23, 
2007, inventory did not include these risks. Program officials 
explained that the discrepancy with the number of risks in the 
inventory is that the program periodically closes risks and removes 
them from the inventory. However, these officials provided 
documentation on only one risk that had been removed from the 
inventory, and it did not include a clear rationale for why the risk 
had been removed. In addition, the officials were unable to provide 
documentation showing that the other two risks had been closed and 
removed and the justification for doing so. 

Second, while the risk management plan defines a process and associated 
practices for developing risk mitigation strategies, the program has 
not fully implemented them. Specifically, the plan specifies that 
mitigation strategies are to include: 

a rationale for mitigation strategy chosen; 

an explanation of the impact on program performance; 

a proposed schedule showing milestones for initiation, significant risk 
mitigation activities, and completion; and: 

the official responsible for implementing and tracking mitigation 
activities. 

Out of the 61 risk mitigation strategies documented in the January 23, 
2007, inventory: 

all included a rationale for mitigation strategy chosen; 

13 did not include an explanation of the impact on program performance; 

none included a proposed schedule showing milestones for initiation, 
significant risk mitigation activities, and completion; and: 

all identified officials responsible for implementing and tracking 
mitigation activities. 

Third, although the risk management plan calls for developing a process 
for elevating risks to management's attention, the program has not 
developed and implemented such a process. According to the program 
manager, in lieu of a defined escalation process, program officials do 
make ICE's deputy CIO (and sometimes the CIO) aware of program risks as 
part of quarterly program management meetings, but acknowledged that, 
for the most part, such reviews-and thus the escalation of risks-occur 
in an ad hoc fashion and are not documented. 

The program manager stated that the reason for the current state of the 
program's risk management was that the process is new and that it will 
take time for the risk program to fully mature. He added that the 
program is in the process of addressing these weaknesses by, for 
example, revising the risk management plan to include procedures for 
(1) reporting to management on the existence and status of risks and 
(2) closing risks. More recently, on January 25, 2007, the program 
manager provided us with the revised plan; it calls for the risk 
management coordinator to, among other things, establish processes for 
elevating and reporting to management risks that are likely to occur 
and have a high impact. Further, at our February 1, 2007, exit meeting 
with program officials, they provided a document, approved on January 
25, 2007, defining their processes for elevating risk. While these are 
steps in the right direction, the program manager acknowledged that 
program staff were just beginning to implement the processes. 

Until the Atlas program fully implements and institutionalizes risk 
management, there is increased probability that program risks will not 
be proactively mitigated and, thus, will become actual program cost, 
schedule, and performance shortfalls. 

Objective 2 Results: Observation 4: Performance Management: 

Observation 4: Atlas program still implementing performance management 
practices essential to measuring progress against commitments: 

As we have previously reported,[Footnote 36] to enable Atlas program 
management to measure progress and make well-informed investment 
decisions, it is important for the program to develop and implement 
rigorous performance management practices that include, among other 
things, properly aligned goals and anticipated achievements that are 
defined in measurable terms. In our September 2005 report,[Footnote 37] 
we recommended that the Atlas program develop and implement an 
effective performance management system that includes such goals and 
measures. More recently, we reported[Footnote 38] in July 2006 that 
Atlas had begun taking steps to implement performance goals and 
measures, but had not yet completed defining and implementing them. For 
example, the program had developed goals for four of its seven projects 
but had not done so for its three other projects, which were: 

Transformation Planning, 

Architecture Engineering, and: 

Data Center Migration. 

We also reported that the program had developed measures to gauge 
progress on the goals, but had not yet begun to track their progress. 

Since then, the program has taken additional steps to define and 
implement performance goals and measures. For example, the program 
identified 13 goals for five of its seven projects that it has been 
using to measure progress during fiscal year 2006. Twelve of these 
goals either were expected to be finished by fiscal year 2006 or were 
expected to have some interim measure completed during fiscal year 
2006. The expenditure plan identified an additional goal (goal 12) that 
did not have an interim measure, but rather was expected to be finished 
by December 2006. 

In addition, the program now reports it is measuring progress against 
these performance goals by, for example, using them to discuss each 
project's status and progress at quarterly program management reviews. 

The following table provides our analysis of the program's performance 
goals against its reported progress (as of Sept. 30, 2006). 

Table: Atlas Project Progress in Meeting Fiscal Year 2006 Goals. 

Project: Overall Program Management/Transformation Planning; 
Goal: Goal 1: 100 percent of Atlas project activities that are managed 
with less than a 10 percent variance in cost and schedule per fiscal 
year; 
Did Atlas meet goal?: no; 
If no, what is progress against goal?: 93 percent. 

Project: Overall Program Management/Transformation Planning; 
Goal: Goal 2: 30 percent reduction in ICE's operations and maintenance 
contribution for network services; 
Did Atlas meet goal?: yes; 
If no, what is progress against goal?: not applicable. 

Project: Overall Program Management/Transformation Planning; 
Goal: Goal 3: The enterprise project management tool, primavera, was 
scheduled for installation, configuration, and deployment in third 
quarter of fiscal year 2006; 
Did Atlas meet goal?: no; 
If no, what is progress against goal?: deployed in 4th quarter of FY 
2006. 

Project: Overall Program management/Transformation Planning; 
Goal: Goal 4: Performance management processes and balanced scorecard 
to be developed and implemented in fourth quarter of fiscal year 2006; 
Did Atlas meet goal?: no; 
If no, what is progress against goal?: *. 

Project: Common Computing Environment; 
Goal: Goal 5: 100 percent of ICE personnel using the DHS standard 
Microsoft Outlook E-mail platform; 
Did Atlas meet goal?: no; 
If no, what is progress against goal?: 95 percent. 

Project: Common Computing Environment; 
Goal: Goal 6: 33 percent progress in implementing a refresh of ICE 
personnel desktop equipment; 
Did Atlas meet goal?: no; 
If no, what is progress against goal?: 20 percent. 

Project: Common Computing Environment; 
Goal: Goal 7: 100 percent of ICE office sites achieving workforce 
productivity upgrades through implementation of ICE e-mail standard, 
backend server upgrades, and ICENet connections; 
Did Atlas meet goal?: no; 
If no, what is progress against goal?: 95 percent. 

Project: Integration; 
Goal: Goal 8: 100 percent of ICE sites using DHS standard high-speed 
network circuits; 
Did Atlas meet goal?: no; 
If no, what is progress against goal?: 73 percent. 

Project: Integration; 
Goal: Goal 9: 100 percent of ICE office sites achieving workforce 
productivity upgrades through implementation of ICE e-mail standard, 
backend server upgrades, and ICENet connections; 
Did Atlas meet goal?: no; 
If no, what is progress against goal?: 73 percent. 

Project: ICE Mission Information; 
Goal: Goal 10: 14 percent of investigative and enforcement personnel 
with access to decision support system data marts; 
Did Atlas meet goal?: yes; 
If no, what is progress against goal?: not applicable. 

Project: ICE Mission Information; 
Goal: Goal 11: 8 percent increase in ICE investigative and enforcement 
systems incorporated into ICE decision support system consolidated data 
marts; 
Did Atlas meet goal?: yes; 
If no, what is progress against goal?: not applicable. 

Project: ICE Mission Information; 
Goal: Goal 12: The ICE Mission Information project will implement a 
technology model in the first-quarter of FY 2007 for the integration of 
systems and information throughout ICE within the framework of the DHS 
EA; 
Did Atlas meet goal?: Not applicable (target goal set for FY 2007, but 
not for FY 2006); 
If no, what is progress against goal?: **. 

Project: Information Assurance; 
Goal: Goal 13: 30 percent of ICE users employing single sign-on 
capability for systems access; 
Did Atlas meet goal?: no; 
If no, what is progress against goal?: 0 percent. 

Source: GAO analysis of DHS data. 

* While the program did not meet this goal in the fourth quarter of FY 
2006 as planned, program officials told us that the program has efforts 
underway to reach this goal, but did not provide a date for when it 
would be accomplished. 

** The program reported that it met this goal in fiscal year 2007, as 
planned. 

[End of table] 

However, while the program has taken additional steps, it has yet to 
fully implement and institutionalize its performance goals and 
measures. Specifically, the program has not yet developed goals for 
these two projects-Architecture Engineering and Data Center Migration. 
According to program officials, they have developed draft goals for 
these projects and are in the process of having them reviewed by 
management, but officials did not provide a date for when the goals are 
to be approved and implemented. 

In addition, the progress on the performance goals, as reported by the 
program, shows mixed results. As shown in the chart, of the 12 goals, 
the program has met only 3 of them. Of the remaining 9, Atlas was close 
to meeting its fiscal year 2006 targets. For example, on goal 5, the 
fiscal year 2006 target was 100 percent of ICE personnel using the DHS 
standard Microsoft Outlook e-mail platform, and the program reported 95 
percent of users have been migrated. Also, on goal 8, the fiscal year 
2006 target was 100 percent of ICE sites using DHS standard high speed 
network circuits, and the program reported 73 percent of ICE sites are 
using the circuits. 

Further, underlying data used by the program to measure performance may 
not be reliable in all cases. For example, on goal 1, the program uses 
project cost and schedule data from the 2005 Atlas business case in 
measuring progress against the goal, including determining whether 
variances have occurred. However, these data are out of date (i.e., 
they do not reflect project cost and schedule slippages that have 
occurred since 2005) and are currently being revised as part of a 
larger effort to update the entire business case to reflect overall 
program changes. 

In addition, the program relies on its project managers to develop 
estimates of where their projects should be (with respect to schedule) 
given funds used to date. The program official responsible for 
determining progress against goal 1 told us that this is a key weakness 
of the performance management process because it allows project 
managers to report on their progress without the program having a means 
to verify and validate their assessments. The program manager 
acknowledged that having and using reliable information to support 
measurement against goals is a challenge across all of the program 
measures, and the program plans to address the problem by, among other 
things, updating the business case and supporting cost and schedule 
analyses (which are to be issued in September 2007) and by implementing 
and using an enterprise project management tool (planned to start in 
October 2007) that calculates progress against funds used for each 
project. While these are steps in the right direction, it also means it 
will be months before the program has data it can rely on to measure 
performance. 

According to the program manager, the program's limited progress in 
implementing performance management practices is attributable in large 
part to (1) the program's evolving of its performance goals as the 
program evolves; (2) performance management being new to the program, 
and it will take time for the program's processes to fully mature; and 
(3) the enterprise project management tool's taking longer to implement 
than originally planned. According to the program manager, the program 
recently purchased the tool and plans to begin using it for all 
projects beginning in October 2007. Nonetheless, until the program 
completes implementation and institutionalization of its performance 
management capabilities, including well-defined goals and measures, its 
managers will not have the necessary information for measuring progress 
and making well-informed investment decisions. 

Conclusions: 

The fiscal year 2006 Atlas expenditure plan, in combination with 
related program documentation and program officials' statements, 
satisfies or partially satisfies the legislative conditions set forth 
by Congress. However, this satisfaction is based on plans and 
commitments that provide for meeting these conditions rather than on 
completed actions to satisfy the conditions. In addition, while steps 
are being initiated that are intended to address significant program 
management weaknesses, a number of improvements, including those 
recommended in our past reports, have yet to be implemented. Further, 
the program has not fully achieved many performance goals that it set 
out to accomplish over the past year. These factors continue to put the 
program at risk and call for heightened and sustained management 
attention to expeditiously address and resolve the issues. 

Thus, there is much that still needs to be accomplished to minimize the 
risks associated with the program's capacity to deliver promised IT 
infrastructure capabilities and benefits on time and within budget. 
This includes demonstrating better progress against established 
performance goals in the coming year. Given that hundreds of millions 
of dollars are to be invested and the program is critical to supporting 
the ICE mission, it is essential that DHS follow through on its 
commitments to build the capacity to effectively manage the program. 
Proceeding without this capacity introduces unnecessary risks to the 
program and potentially jeopardizes its viability for future 
investment. 

Recommendations for Executive Action: 

To minimize risks to the Atlas program, we recommend that the Secretary 
of Homeland Security direct the Assistant Secretary for Immigration and 
Customs Enforcement to ensure that ICE follows through on its 
commitments to implement effective management controls and capabilities 
by implementing the following five recommendations: 

* Employing practices essential to ensuring that the Atlas program's 
IV&V agent is and remains independent, including incorporating 
requirements in future contracting actions such as the renegotiation or 
recompetition of the current independent verification and validation 
contract, which will prohibit the agent from soliciting, proposing, or 
being awarded program work other than providing independent 
verification and validation services and products. 

* Fully adhering to requirements development and management practices, 
including those specified in ICE's policies and procedures. This should 
also include having the Atlas program manager develop a process 
improvement plan for all of the projects that is consistent with the 
ICE System Lifecycle Management Handbook and provide for making the 
program manager and project managers responsible and accountable for 
rigorously adhering to the requirements in the handbook. 

* Fully implementing key contract management and oversight practices, 
including those specified in ICE's policies and procedures. This should 
also include ensuring that the Atlas program manager, working with the 
program's COTR, follow through on planned efforts to strengthen the 
program's compliance with these practices by (1) revising the 
acquisition plan by March 2007; (2) revising the ICE System Lifecycle 
Management Handbook by June 2007 to incorporate key contract management 
activities such as contract tracking and oversight; and (3) preparing 
statements of work (beginning in February 2007) for future contracts to 
clearly define Atlas work statements and acceptance criteria. 

Completing implementation of planned risk management activities. This 
should include (1) fully implementing and institutionalizing procedures 
for reporting to management on the existence and status of risks and 
progress in implementing mitigation strategies and (2) updating the 
risk inventory to include risks for all projects and risk areas. 

Improving program performance management, including developing 
performance goals for projects that do not have goals and reporting on 
their progress in the fiscal year 2007 expenditure plan. Further, the 
program should assess whether the data being used to measure progress 
are reliable, complete, and accurate. 

Agency Comments: 

In their February 1, 2007, oral comments on a draft of this briefing, 
ICE officials, including the Deputy CIO and Atlas program manager, 
agreed with our findings, conclusions, and recommendations. These 
officials also provided technical comments, which we incorporated in 
the briefing as appropriate. 

Attachment I: Detailed Scope and Methodology: 

To accomplish our objectives, we: 

analyzed the fiscal year 2006 Atlas expenditure plan and supporting 
documents against legislative conditions and other relevant federal 
requirements, guidance, and best practices to determine whether the 
conditions were met (in doing so, we considered the conditions met when 
the expenditure plan, including supporting program documentation and 
program officials' representations, either satisfied or provided for 
satisfying the conditions) and: 

evaluated supporting documentation and interviewed program and other 
involved ICE and DHS officials to determine progress in establishing 
capabilities in program management areas, such as acquisition planning, 
enterprise architecture, project management, requirements development 
and management, contract management and oversight, and risk management 
and performance management. 

In assessing the management of requirements development and contract 
management and oversight, we focused on three projects-Common Computing 
Environment, Integration, and ICE Mission Information-that collectively 
accounted for approximately 75 percent of the funds provided to the 
program and the Atlas program manager identified as being the most 
critical to the program's success and ICE's mission. For DHS and ICE 
data that we did not substantiate, we made appropriate attribution 
indicating the data source. We conducted our work at ICE and DHS 
headquarters in Washington, D.C., from November 2006 through January 
2007 in accordance with generally accepted government auditing 
standards. 

Attachment II: GAO Analysis of Requirements Management: 

Table: ICE Requirements Management Efforts Compared with Federal IT 
Management Guidance and Best Practices. 

Establish a requirements management policy. 

Project: CCE; 
Implemented: yes; 
GAO's assessment of compliance: In February 2005, the CCE project 
established the ICE System Lifecycle Management Handbook[Footnote 39]  
as the policy to guide project efforts in developing and managing 
requirements. The handbook requires requirements development and 
management activities, such as eliciting desired system capabilities 
from users and translating them into requirements and obtaining 
approval by all stakeholders. 

Project: Integration; 
Implemented: yes; 
GAO's assessment of compliance: In February 2005, the Integration 
project established the ICE System Lifecycle Management Handbook as the 
policy to guide project efforts in developing and managing 
requirements. 

Project: ICE Mission Information; 
Implemented: yes; 
GAO's assessment of compliance: In February 2005, the ICE Mission 
Information project established the ICE System Lifecycle Management 
Handbook as the policy to guide project efforts in developing and 
managing requirements. 

Source: GAO analysis of DHS data. 

Legend: Yes = established/implemented; No=not established/implemented; 
Partially = some, but not all, actions are implemented or actions are 
in progress of being implemented: 

[End of table] 

Table: ICE Requirements Management Efforts Compared with Federal IT 
Management Guidance and Best Practices 

Elicit desired or required system capabilities from users and translate 
them into system requirements and obtain approval by all stakeholders. 

Project: Common Computing Environment; 
Implemented: partially; 
GAO's assessment of compliance: In 2003, the CCE project elicited 
system capabilities from component and departmental users. Although the 
capabilities elicited from the users were not documented, project 
officials told us they translated the users’ input into system 
requirements, issuing a draft requirements document in June 2004. 
Although the draft requirements had not yet been reviewed and approved 
by system stakeholders, the project began deploying system capabilities 
in December 2005. Project officials told us that they planned to 
finalize the requirements document, including obtaining stakeholder 
approval, and in January 2007, they provided us with a finalized and 
approved version of the requirements document.  

Project: Integration; 
Implemented: partially; 
GAO's assessment of compliance: In 2003, Integration elicited system 
capabilities based on a directive from the DHS Deputy Secretary and 
departmental users. The capabilities were elicited from users at 
requirements workshops and documented. Project officials told us that 
they translated the users’ input into system requirements, issuing a 
draft requirements document on June 15, 2005. Although the draft 
requirements have not yet been reviewed and approved by system 
stakeholders, the project began deploying system capabilities in March 
2006. Project officials told us that they have obtained stakeholder 
approval on the requirements document; however, they could not provide 
documentation showing this had occurred. 

Project: ICE Mission Information; 
Implemented: partially; 
GAO's assessment of compliance: Divided into four phases, the ICE 
Mission Information project is currently in the process of eliciting 
user system capabilities for each of the respective phases. For the 
first phase, the project has elicited system capabilities from its 
users, but has not documented them. Project officials told us that they 
translated the user capabilities into system requirements, issuing a 
draft requirements document dated August 7, 2006. Project officials 
stated that they planned to finalize the requirements document, 
including obtaining system stakeholder approval, but could not provide 
a date by when this was to occur. For the other three phases, the 
project has not completed eliciting requirements and documenting 
capabilities, but project officials told us they planned to do so in 
accordance with the practices. However, they were not able to provide a 
plan and associated timetable for when they are to complete the 
practices. 

Develop a system of concept of operations. 

Project: CCE; 
Implemented: partially; 
GAO's assessment of compliance: CCE did not develop a concept of 
operations, which is integral to defining system requirements, until 3 
years after the project had been initiated. Specifically, the project 
was initiated in March 2003 and began deploying system capabilities in 
December 2005, and the concept of operations was developed in June 
2006. 

Project: Integration; 
Implemented: no; 
GAO's assessment of compliance: According to the Integration project 
manager, a concept of operations was developed. However, Integration 
was not able to provide documentation showing it had been developed. 

Project: ICE Mission Information; 
Implemented: partially; 
GAO's assessment of compliance: The project issued a draft concept of 
operations on October 12, 2006. The ICE Mission Information project 
manager told us that the project is in the process of finalizing the 
document but did not provide a date for when the document would be 
completed. 

Ensure that changes to requirements are controlled and approved via a 
disciplined change control process as the system is developed and 
implemented. 

Project: CCE; 
Implemented: partially; 
GAO's assessment of compliance: CCE did not follow a disciplined change 
management process for 3 years after the project was initiated in March 
2003; during this time, requirements were introduced and implemented in 
an ad hoc fashion. The program has taken steps to improve change 
management efforts by (1) developing an Atlas Configuration Management 
Plan on February 15, 2006; (2) developing a CCE change control board 
charter on November 1, 2006; and (3) developing a change process that 
provides the procedures for reviewing and approving CCE change 
requests. The project is currently in the process of implementing these 
changes. Project officials told us that the project recently began 
conducting CCE change control board meetings in September 2006. 
According to the change process, CCE is required to ensure that 
technical, cost, and schedule impacts are considered before approval is 
granted. Although program documentation showed the technical impact of 
the change, the cost and schedule impacts were not documented. 

Project: Integration; 
Implemented: partially; 
GAO's assessment of compliance: The configuration control board and 
change control process that Integration follows was developed by DHS’s 
Infrastructure Transformation Program, which is the departmental 
organization (commonly referred to as the “network steward” by DHS and 
ICE officials) responsible for ensuring that all DHS components migrate 
from their existing networks to the DHS network. However, Integration 
officials could not provide evidence of compliance with the network 
steward’s change control process. 

Project: ICE Mission Information; 
Implemented: partially; 
GAO's assessment of compliance: The project plans to control and 
approve changes via a change control process when the requirements are 
developed and finalized. The project plans to do so consistent with the 
guidance specified in the ICE System Lifecycle Management Handbook. As 
previously noted, project officials were not able to identify when the 
requirements would be finalized and approved. 

Maintain bidirectional traceability, meaning that a given requirement 
can be traced backward to systems documentation and forward to the 
appropriate contract vehicle and system components. 

Project: CCE; 
Implemented: partially; 
GAO's assessment of compliance: In December 2006, CCE developed a 
requirements traceability analysis document—commonly referred to as a 
matrix—to show bidirectional traceability between the requirements and 
system documentation. While the project identified requirements, it did 
not trace requirements to the system documentation, such as the system 
design and test plans. Additionally, the project did not trace 
requirements to the existing contract task order. 

Project: Integration; 
Implemented: partially; 
GAO's assessment of compliance: In June 2005, Integration developed a 
requirements traceability analysis document—commonly referred to as a 
matrix—to show bidirectional traceability between the requirements and 
system documentation. While the project identified requirements, it did 
not trace requirements to the system documentation, such as the system 
design and test plans. Additionally, the project did not trace 
requirements to the existing contract task order. 

Project: ICE Mission Information; 
Implemented: partially; 
GAO's assessment of compliance: The project plans to ensure 
traceability among requirements, system documentation, and the existing 
contract task order when the requirements are developed and finalized. 
The project intends to do so in accordance with the guidance specified 
in the ICE System Lifecycle Management Handbook. As previously noted, 
project officials were not able to identify when the requirements would 
be finalized and approved. 

Source: GAO analysis of DHS data. 

[End of table] 

[End of section] 

Appendix II: Comments from the Department of Homeland Security: 

U.S. Department of Homeland Security: 
Washington, DC 20528: 

April 5, 2007: 

Mr. Randolph C. Hite: 
Director: 
Information Technology Architecture and Systems Issues: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Mr. Hite: 

Thank you for the opportunity to review and comment on the Government 
Accountability Office's (GAO's) draft report GAO 07-565 entitled 
Information Technology: Immigration and Customs Enforcement Needs to 
Fully Address Significant Infrastructure Modernization Program 
Management Weaknesses. 

U.S. Immigration and Customs Enforcement (ICE) concurs with the draft 
report's five recommendations. The following describes the actions ICE 
has taken or plans to take to implement the recommendations. 

Recommendation 1: Employ practices essential to assuring that the Atlas 
program's independent verification and validation agent is and remains 
independent, including incorporating requirements in future contracting 
actions such as the renegotiation or recompetition of the current 
independent verification and validation contract, which will prohibit 
the agent from soliciting, proposing, or being awarded program work 
other than providing independent verification and validation services 
and products. 

Response: Future Atlas contract actions will prohibit independent 
verification and validation (IV&V) agents from soliciting, proposing, 
or being awarded non-IV&V Atlas work to prevent a conflict of interest 
situation. 

Recommendation 2: Fully adhere to requirements development and 
management practices, including those specified in ICE's policies and 
procedures. This should also include having the Atlas program manager 
develop a process improvement plan for all the projects that is 
consistent with the ICE System Lifecycle Management Handbook and 
provide for making the program manager and project managers responsible 
and accountable for rigorously adhering to the requirements in the 
handbook. 

Response: The Atlas processes are being refined to fully comply with 
Systems Lifecycle Management standards. By May 31, 2007, ICE 
anticipates the implementation of an improvement plan that will address 
requirement development and management processes, training, and the 
establishment of a Change Control Board to govern and manage impacts. 

Recommendation 3: Fully implement key contract management and oversight 
practices, including those specified in ICE's policies and procedures. 
This should also include ensuring that the Atlas program manager, 
working with the program's contracting officer's technical 
representative, follow through on planned efforts to strengthen the 
program's compliance with these practices by (1) revising the 
acquisition plan by May 2007, (2) revising the ICE System Lifecycle 
Management Handbook by June 2007 to incorporate key contract management 
activities such as contract tracking and oversight, and (3) preparing 
statements of work (beginning in February 2007) for future contracts to 
clearly define Atlas work statements and acceptance criteria. 

Response: In November 2006, a full-time contracting officer's technical 
representative joined the Atlas program to implement industry compliant 
contract management and oversight policies. Furthermore, the Atlas 
Acquisition Plan is currently being revised and is proceeding ahead of 
the May 2007 schedule. The Atlas program will make the newly instituted 
contract tracking and management oversight processes available for 
incorporation into the next revision of the ICE System Lifecycle 
Management Handbook scheduled for release in June 2007. The use of a 
standardized statement of work (SOW) template with sections for 
detailed requirements and acceptance criteria has been instituted for 
the creation of all Atlas SOWs. 

Recommendation 4: Complete implementation of planned risk management 
activities. This should include (1) updating the risk inventory to 
include risks for all projects and risk areas and (2) fully 
implementing and institutionalizing procedures for reporting to 
management on the existence and status of risks and progress in 
implementing mitigation strategies. 

Response: The Atlas risk management and escalation processes have been 
refined to ensure effective risk management and better communication of 
risk, and were deployed in March 2007. Furthermore, the risk inventory 
has been updated to re-incorporate all risks previously archived due to 
their closed status, to report risk as it relates to impact on cost, 
schedule and performance, to detail risk mitigation strategy, and track 
status of risks and progress. 

Recommendation 5: Improve program performance management, including 
developing performance goals for projects that do not have goals and 
reporting on their progress in the fiscal year 2007 expenditure plan. 
Further, the program should assess that the data being used to measure 
progress are reliable, complete, and accurate. 

Response: Performance goals have been developed for all Atlas projects. 
Standard procedures to include quarterly assessments have been deployed 
and are currently employed to ensure measurement accuracy. 

Thank you again for the opportunity to comment on this draft report and 
we look forward to working with you on future homeland security issues. 

Sincerely, 

Sincerely, 

Steven J. Pecinovsky: 
Director: 
Departmental GAO/OIG Liaison Office: 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Randolph C. Hite (202) 512-3439: 

Staff Acknowledgments: 

In addition to the individual named above, Gary Mountjoy, Assistant 
Director; Nancy Glover; James Houtz; Jennifer Mills; Freda Paintsil; 
Madhav Panwar; Karl Seifert; and Kelly Shaw made key contributions to 
this report. 

(310640): 

FOOTNOTES 

[1] Pub. L. No. 109-90, Oct. 18, 2005. 

[2] OMB Circular A-11, part 7, establishes policy for planning, 
budgeting, acquiring, and managing federal capital assets. 

[3] An enterprise architecture provides a clear and comprehensive 
picture of an organization's operations and its supporting systems and 
infrastructure. It is an essential tool for effectively engineering 
business processes and for implementing and evolving supporting systems 
in a manner that maximizes interoperability, minimizes overlap and 
duplication, and optimizes performance. 

[4] GAO, Homeland Security: Recommendations to Improve Management of 
Key Border Security Program Need to Be Implemented, GAO-06-296 
(Washington, D.C.: Feb. 14, 2006). 

[5] GAO-06-296. 

[6] These three key projects are Common Computing Environment, 
Integration, and ICE Mission Information. 

[7] CMMI is registered in the U.S. Patent and Trademark Office by 
Carnegie Mellon University. 

[8] GAO, Information Technology: Immigration and Customs Enforcement Is 
Beginning to Address Infrastructure Modernization Program Weaknesses 
but Key Improvements Still Needed, GAO-06-823 (Washington, D.C.: July 
27, 2006). 

[9] GAO-06-823. 

[10] ICE was formed from the former Immigration and Naturalization 
Service, U.S. Customs Service, and other entities. Atlas began in 2002 
under the former Immigration and Naturalization Service. 

[11] pub. L. No. 109-90, Oct. 18, 2005. 

[12] In the act, appropriations for Atlas are under the heading 
Automation Modernization for Immigration and Customs Enforcement. 

[13] OMB Circular A-11 establishes policy for planning, budgeting, 
acquisition, and management of federal capital assets. 

[14] The purpose of the Investment Review Board is to integrate capital 
planning and investment control, budgeting, and acquisition. It is also 
to ensure that spending on investments directly supports and furthers 
the mission and that this spending provides optimal benefits and 
capabilities to stakeholders and customers. 

[15] DHS's fiscal year 2006 appropriations (Pub. L. No. 109-90, Oct. 
18, 2005) provided $40.15 million for Atlas. In December 2005, this 
amount was reduced to $39.75 million via a 1 percent governmentwide 
rescission in Pub. L. No. 109-148, Division B, Dec. 30, 2005. 

[16] Partially satisfies=the plan, in combination with supporting 
documentation and stated commitments by program officials, either 
satisfies or provides for satisfying many, but not all, key aspects of 
the condition that we reviewed. 

[17] Satisfies=the plan, in combination with supporting documentation 
and stated commitments by program officials, either satisfies or 
provides for satisfying every aspect of the condition that we reviewed.

[18] GAO, Information Technology. Immigration and Customs Enforcement 
Is Beginning to Address Infrastructure Modernization Program Weaknesses 
but Key Improvements Still Needed, GAO-06-823 (Washington, D.C.: July 
27, 2006). 

[19] Areview group made up of subject matter experts that recommends EA 
compliance to the DHS Enterprise Architecture Board and ultimately to 
the DHS Investment Review Board. 

[20] Department of Homeland Security Enterprise Architecture Board: EAB 
Governance Process Guide, August 9, 2004, draft version 2.0. 

[21] GAO-06-823. 

[22] GAO-06-823. 

[23] On August 14, 2006, we reported that version 2006 of DHS's EA was 
at stage 2 of the Enterprise Architecture Management Maturity 
Framework, demonstrating that DHS has established the foundational 
commitments and capabilities needed to manage the development of the 
architecture. See GAO, Enterprise Architecture: Leadership Remains Key 
to Establishing and Leveraging Architectures for Organizational 
Transformation, GAO-06-831 (Washington, D.C.: Aug. 14, 2006). 

[24] The Tactical Working Group is responsible for developing 
strategies to address service oriented architecture design and 
operational issues. 

[25] Developed by Carnegie Mellon Software Engineering Institute (SEI), 
Software Acquisition Capability Maturity Model (SA-CMM' ) version 1.03 
(March 2002). 

[26] U.S. Immigration and Customs Enforcement: System Lifecycle 
Management Handbook, The Systems Development Lifecycle of Immigration 
and Customs Enforcement, February 2005, version 1.0. 

[27] GAO, Homeland Security: Recommendations to Improve Management of 
Key Border Security Program Need to Be Implemented, GAO-06-296 
(Washington, D.C.: Feb. 14, 2006). 

[28] See, for example, Software Engineering Institute, Capability 
Maturity Model Integrated (CMMI), version 1.1 (March 2002). 

[29] GAO-06-823. 

[30]  U. S. Immigration and Customs Enforcement: System Lifecycle 
Management, The Systems Development Lifecycle of Immigration and 
Customs Enforcement, version 1.0 (Washington, D.C.: February 2005). 

[31] CMMI is registered in the U.S. Patent and Trademark Office by 
Carnegie Mellon University. 

[32] GAO, Homeland Security: Contract Management and Oversight for 
Visitor and Immigrant Status Program Need to Be Strengthened, GAO-06- 
404, (Washington, D.C.: June 9, 2006). 

[33] GAO-06-823. 

[34] See, for example, Software Engineering Institute, Capability 
Maturity Model Integrated (CMMI), version 1.1 (March 2002). 

[35] GAO-06- 823. 

[36] GAO-06-823. 

[37] GAO, Information Technology. Management Improvements Needed on 
Immigration and Customs Enforcement's Infrastructure Modernization 
Program, GAO-05-805 (Washington, D.C.: Sept. 7, 2005). 

[38] GAO-06-823. 

[39] U.S. Immigration and Customs Enforcement. System Lifecycle 
Management, The Systems Development Lifecycle of Immigration and 
Customs Enforcement, version 1.0. (Washington, D.C.: February 2005). 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts 
newly released reports, testimony, and correspondence on its Web site. 
To have GAO e-mail you a list of newly posted products every afternoon, 
go to www.gao.gov and select "Subscribe to Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 441 G Street NW, Room LM 
Washington, D.C. 20548: 

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 
512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. 
Government Accountability Office, 441 G Street NW, Room 7125 
Washington, D.C. 20548: 

Public Affairs: 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, D.C. 20548: