DHS Financial Management: Better Use of Best Practices Could Help Manage System Modernization Project Risks
Highlights
What GAO Found
The Department of Homeland Security's (DHS) TRIO project represents a key effort to address long-standing financial management system deficiencies. During 2012 and 2013, the TRIO components—U.S. Coast Guard (Coast Guard), Transportation Security Administration (TSA), and Domestic Nuclear Detection Office (DNDO)—each completed an alternatives analysis (AA) to determine a preferred alternative for modernizing its financial management system. GAO found that DNDO's AA substantially met the four characteristics—well-documented, comprehensive, unbiased, and credible—that GAO previously identified for a reliable, high-quality analysis of alternatives (AOA) process. However, Coast Guard's and TSA's AAs did not fully or substantially meet three of these characteristics, and DHS guidance for conducting AAs did not substantially incorporate certain best practices, such as identifying significant risks and mitigation strategies and performing an independent review to help validate the AOA process. Based on these analyses and other factors, the TRIO components determined that migrating to a federal shared service provider (SSP) represented the best alternative, and in 2014, DHS selected the Department of the Interior's Interior Business Center (IBC) as the federal SSP for the project. However, because Coast Guard's and TSA's AAs did not fully or substantially reflect all of the characteristics noted above, they are at increased risk that the alternative selected may not achieve mission needs.
DHS also did not fully follow best practices for managing project risks related to its use of IBC on the TRIO project. Specifically, DHS followed three of seven risk management best practices, such as determining risk sources and categories and establishing a risk management strategy. However, it did not fully follow four best practices for defining risk parameters, identifying risks, developing risk mitigation plans, and implementing these plans largely because its guidance did not sufficiently address these best practices. For example, although DHS created joint teams with IBC and provided additional resources to IBC to help address risk mitigation concerns, it did not always develop sufficiently detailed risk mitigation plans that also included contingency plans for selected critical risks. As a result, although IBC's capacity and experience for migrating large agencies the size of Coast Guard and TSA was identified as a risk in July 2014, a contingency plan working group to address this concern was not established until January 2017. By not fully following risk management best practices, DHS is at increased risk that potential problems may not be identified or properly mitigated.
DHS, IBC, Office of Management and Budget (OMB), and other federal oversight agencies identified various challenges that have impacted the TRIO project and contributed to a 2-year delay in the implementation of Coast Guard's and TSA's modernized solutions. These challenges include the lack of sufficient resources, aggressive schedule, complex requirements, increased costs, and project management and communication concerns. To help address these challenges, DHS and IBC established review teams and have taken other steps to assess potential mitigating steps. In May 2017, DHS determined that migrating the solution from IBC to a DHS data center represented the best option and initiated discovery efforts to further assess this as its path forward for the TRIO project.
Why GAO Did This Study
To help address long-standing financial management system deficiencies, DHS initiated its TRIO project, which has focused on migrating three of its components to a modernized financial management system provided by IBC, an OMB-designated, federal SSP. House Report Number 3128 included a provision for GAO to assess the risks of DHS using IBC in connection with its modernization efforts.
This report examines (1) the extent to which DHS and the TRIO components followed best practices in analyzing alternatives, and the key factors, metrics, and processes used in their choice of a modernized financial management system; (2) the extent to which DHS managed the risks of using IBC for its TRIO project consistent with risk management best practices; and (3) the key factors and challenges that have impacted the TRIO project and DHS's plans for completing remaining key priorities. GAO interviewed key officials, reviewed relevant documents, and determined whether DHS followed best practices identified by GAO as necessary characteristics of a reliable, high-quality AOA process and other risk management best practices.
Recommendations
GAO recommends that DHS more fully follow best practices for conducting an AOA process and managing risks. DHS concurred with GAO's recommendations and described actions it will take, or has taken, in response.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Homeland Security | The DHS Under Secretary for Management should develop and implement effective processes and improve guidance to reasonably assure that future AAs fully follow AOA process best practices and reflect the four characteristics of a reliable, high-quality AOA process. (Recommendation 1) |
DHS concurred with this recommendation. DHS took several actions, which addressed the intent of our recommendation and were substantially completed by May 2021. Specifically, DHS improved its guidance related to alternatives analysis (AA) and analysis of alternatives (AOA) reviews by developing and implementing guidance to reasonably assure that future reviews follow best practices and reflect the four characteristics of reliable, high-quality processes. Specifically, DHS Issued a Lean AOA/AA Guidance in February 2019 for certain software development acquisitions. DHS addressed all other (full) software development acquisitions through various actions: (1) issued DHS Instruction 102-02-103 System Engineering Life Cycle in February 2021, which establishes a technical framework for DHS's system acquisition programs, including AOA/AA processes, and is based on GAO and industry recognized systems engineering best practices; (2) issued the Systems Engineering Life Cycle Guidebook in May 2021, to provide implementation guidance on how to conduct system engineering life cycle activities and technical reviews, including AOA/AA processes, and is based on government, industry, and academia best practices; and (3) issued assessment tools and a study-plan template, which are referred to in the various guidance and includes best practices related to the four characteristics of well documented, comprehensive, unbiased, and credible AOA/AA processes. DHS's improved guidance, supplemental tools, and template provide reasonable assurance that both the lean and full AOA/AAs follow best practices and reflect the four characteristics of a reliable AOA/AA process. If properly implemented, these procedures will help DHS improve its process of performing reliable and high-quality AOA/AA reviews associated with financial management system modernization projects.
|
| Department of Homeland Security | The DHS Under Secretary for Management should improve the Risk Management Planning Handbook and other relevant guidance for managing risks associated with financial management system modernization projects to fully incorporate risk management best practices, including (1) defining thresholds to facilitate review of performance metrics to determine when risks become unacceptable; (2) identifying and analyzing risks to include periodically reconsidering risk sources, documenting risks specifically related to the lack of sufficient, reliable cost and schedule information needed to help properly manage and oversee the project, and timely disposition of IV&V contractor-identified risks; (3) developing risk mitigation plans with specific risk-handling activities, the costs and benefits of implementing them, and contingency plans for selected critical risks; and (4) implementing risk mitigation plans to include establishing periods of performance for risk-handling activities and defining time intervals for updating and certifying the accuracy and completeness of information on risks in DHS's risk register. (Recommendation 2) |
DHS concurred with this recommendation. DHS took several actions, which were substantially completed by the end of fiscal year 2019 in response to our recommendation. DHS hired two programmatic risk management and subject matter experts. DHS's Program Accountability and Risk Management (PARM) office developed (1) a Risk Management Training Aide, (2) risk slides with instructions for acquisition review board briefs, and (3) a risk register spreadsheet. In addition, PARM updated DHS acquisition policy, developed and conducted risk management trainings, and held one-on-one meetings with risk managers at the component, portfolio, and program levels. PARM also established a process to review and provide feedback to officials responsible for programs on their risk management plans and risk registers, focusing on the completeness of the data. Additionally, PARM developed and provided a number of templates and tools to DHS components for use in their risk management programs, including: a Risk and Issue Template and a sample Risk Management Plan. Further, because PARM is responsible for DHS's overall acquisition governance process, DHS established a financial systems modernization Joint Program Management Office (JPMO) to lead and manage all aspects of financial systems modernization programs. Actions that JPMO has taken to address this recommendation include: (1) establishing risk parameters, which are thresholds for each category of cost, schedule, scope, and quality; (2) developing a process to identify and discuss components' analysis of risks and risk status through bi-weekly meetings; (3) requiring risk mitigation and contingency plans, which are tracked on the risk register; and (4) establishing a process for continuous tracking of the implementation of risk mitigation plans. The additional procedures described in these guidance documents, if properly implemented, and the creation of JPMO, meet the intent of this recommendation and should help DHS to improve its process of evaluating and managing risks associated with financial management system modernization projects.
|