Immigration Benefits System: Better Informed Decision Making Needed on Transformation Program
Highlights
What GAO Found
The U.S. Citizenship and Immigration Services' (USCIS) currently expects that its Transformation Program will cost up to $3.1 billion and be fully deployed no later than March 2019, which is an increase of approximately $1 billion and delay of over 4 years from its initial July 2011 baseline. In March 2012, the program began to significantly change its acquisition strategy to address various technical challenges (see table).
Key Changes to the Transformation Program's Acquisition Strategy
|
Key change |
Previous approach |
New approach |
|
Software development |
Waterfall: sequential phases, often with delivery years after start |
Agile: delivery in small, short increments |
|
Contracting approach |
Single contractor |
Multiple contractors with USCIS as system integrator |
|
Program architecture |
Wide use of proprietary commercial off the shelf products |
Wide use of open source software (i.e., publicly available) |
Source: GAO analysis of USCIS documentation. | GAO-15-415
These changes have significantly delayed the program's planned schedule, which in turn has had adverse effects on when USCIS expects to achieve cost savings, operational efficiencies, and other benefits. Among other things, USCIS has yet to achieve the goal of enhancing national security by authenticating users and integrating with external agency databases.
While the program's two key governance bodies have taken actions aligned with leading IT management practices, neither has used reliable information to make decisions and inform external reporting (see table). For example, one governing body's vote in March 2013 to migrate to a new architecture was based in part on savings that did not account for the added costs of merging data from the old architecture. The ability of USCIS, DHS, and Congress to effectively monitor program performance may be limited until these bodies more effectively use reliable information to inform their program evaluations.
Extent to Which Program Governance Bodies Met Leading Practices for Oversight
|
Leading practice |
Acquisition Review Board |
Executive Steering Committee |
|
Monitor project's performance and progress toward predefined cost and schedule |
◐ |
◐ |
|
Ensure that corrective actions are identified and assigned to the appropriate parties at the first sign of cost, schedule, or performance problems |
● |
● |
|
Ensure that corrective actions are tracked until the desired outcomes are achieved |
◐ |
● |
|
Rely on complete and accurate data to review performance against stated expectations |
○ |
○ |
● Implemented ◐ Partially implemented ○ Not implemented
Source: GAO analysis of USCIS documentation. | GAO-15-415
Why GAO Did This Study
Each year, the Department of Homeland Security's (DHS) USCIS processes millions of applications for persons seeking to study, work, visit, or live in the United States. USCIS has been working since 2005 to transform its outdated systems into an account-based system with electronic adjudication and case management tools that will allow applicants to apply and track the progress of their application online. In 2011, USCIS reported that this effort, called the Transformation Program, was to be completed no later than June 2014 at a cost of up to $2.1 billion.
Given the critical importance of the Transformation Program, GAO was asked to review it. This report (1) discusses the program's current status, including the impact of changes made, and (2) assesses the extent to which DHS and USCIS are executing effective program oversight and governance.
To do so, GAO reviewed DHS and USCIS documents, interviewed relevant officials, and compared program documentation and actions to DHS and USCIS policy and guidance and GAO and industry leading information technology practices.
Recommendations
GAO is making recommendations to DHS components and offices to improve governance and oversight of the Transformation Program. DHS agreed with the recommendations, but did not agree with GAO's evaluation of the impact of changes made to the acquisition strategy. GAO maintains its position on the impact of changes, as discussed in the report.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Homeland Security | To help ensure that progress made by the Transformation Program can be monitored against established and approved parameters, the Secretary of DHS should direct the department's Under Secretary for Management to re-baseline cost, schedule, and performance expectations for the remainder of the Transformation Program. |
In May 2015, the Department of Homeland Security provided an approved acquisition program baseline and supporting documentation, including the Acquisition Decision Memorandum signed by the Acting Deputy Under Secretary for Management. These documents demonstrate that the department has approved a re-baseline of cost, schedule, and performance expectations for the remainder of the Transformation Program. This re-baseline helps ensure that progress made by the program can be monitored against established and approved parameters.
|
| Department of Homeland Security | To improve Transformation Program governance, the Secretary of DHS should direct the Under Secretary for Management to ensure that the Acquisition Review Board is effectively monitoring the Transformation Program's performance and progress toward a predefined cost and schedule; ensuring that corrective actions are tracked until the desired outcomes are achieved; and relying on complete and accurate program data to review the performance of the Transformation Program against stated expectations. |
As of August 2019, the Department of Homeland Security (DHS) has demonstrated that it has taken sufficient steps to close this recommendation. In February 2016, the Office of Program Accountability and Risk Management (PARM), which serves as the Acquisition Review Board (ARB) executive secretariat and is to oversee DHS's acquisition portfolio, demonstrated that DHS developed a procedure to help ensure acquisition decision memorandum actions, including corrective actions, are tracked until the desired outcomes are achieved. In July 2016, we reported that ongoing data reported monthly by the program office was not always accurate. For example, with respect to schedule data, we reported that releases exceeded the projected schedule defined in the life cycle cost estimate and that updates to the schedule did not always present a full picture of the program. In October 2016, the program reported a breach of its committed schedule for an Acquisition Decision Event 2C decision and failure to achieve two Acquisition Program Baseline (APB) threshold schedule milestones. In response, in December 2016 the Deputy Under Secretary for Management (USM) and Chief Financial Officer decided that until approved by the ARB, the program must stop new planning and development work. In July 2018, the USM, in coordination with the ARB, decided that the program schedule breach was cleared and that planning and development for the remaining work was authorized to proceed. This decision included approval of an updated APB. Since July 2018, the ARB has been monitoring the program's progress toward the cost and schedule estimates agreed upon in the updated baseline as part of recurring reviews. For example, the U.S. Citizenship and Immigration Services Transformation program office provided evidence that, as of April 2019, the ARB continues to discuss cost, schedule, and operational performance metrics as part of the program's ARB meetings.
|
| Department of Homeland Security | To improve Transformation Program governance, the Secretary of DHS should direct the DHS Under Secretary for Management, in coordination with the Director of US Citizenship and Immigration Services, to ensure that the Executive Steering Committee is effectively monitoring the Transformation Program's performance and progress toward a predefined cost and schedule and relying on complete and accurate program data to review the performance of the Transformation Program against stated expectations. |
As of May 2019, DHS has demonstrated that it has taken sufficient steps to close this recommendation. More specifically, the U.S. Citizenship and Immigration Services (USCIS) Transformation program office provided evidence that, as of March 2019, the Executive Steering Committee (ESC) continues to discuss cost, schedule, and operational performance metrics as part of the program's ESC meetings. In July 2016, we reported that ongoing data reported monthly by the program office was not always accurate. For example, with respect to schedule data, we reported that releases exceeded the projected schedule defined in the life cycle cost estimate and that updates to the schedule did not always present a full picture of the program. In October 2016, the program reported a breach of its committed schedule for an Acquisition Decision Event 2C decision and failure to achieve two Acquisition Program Baseline (APB) threshold schedule milestones. In response, in December 2016 the Deputy Under Secretary for Management (USM) and Chief Financial Officer decided that until approved by the Acquisition Review Board (ARB), the program must stop new planning and development work. In July 2018, the USM, in coordination with the ARB, decided that the program schedule breach was cleared and that planning and development for the remaining work was authorized to proceed. This decision included approval of an updated APB. Since July 2018, the ESC has been monitoring the program's progress toward the cost and schedule estimates agreed upon in the updated baseline as part of recurring reviews.
|
| Department of Homeland Security | To help ensure that assessments prepared by the Office of the Chief Information Officer in support of the department's updates to the federal IT Dashboard more fully reflect the current status of the Transformation Program, the Secretary of DHS should direct the department's Chief Information Officer to use accurate and reliable information, such as operational assessments of the new architecture and cost and schedule parameters approved by the Under Secretary of Management. |
As of August 2019, the Department of Homeland Security (DHS) has demonstrated that it has taken sufficient steps to close this recommendation. In particular, in February 2016, the DHS Office of the Chief Information Officer (OCIO), in coordination with the Office of Program Accountability and Risk Management (PARM), had consolidated the department's Investment Management System and Next Generation Periodic Reporting System tools into a single enterprise information management and repository system named Investment Evaluation, Submission, and Tracking (INVEST). According to the department, this effort should improve the reliability of the metrics used by OCIO's Enterprise Business Management Office (EBMO), as well as the other line of business and component program offices, and ensure data integrity. The data reported in INVEST include cost, schedule, and operational performance metrics that are to align with the OMB's Information Technology (IT) Dashboard reporting requirements. Since June 2018, OCIO has monitored the Transformation Program's performance and progress in developing new software through the PARM-managed Acquisition Program Health Assessment (APHA) process, replacing its prior program health assessment process. OCIO submits the results of the full APHA for the program to the IT Dashboard. The APHA accounts for an assessment of 6 technical factors completed by OCIO along with cost, schedule, and performance factors assessed by other DHS offices.
|