Critical Infrastructure Protection:

DHS Could Strengthen the Management of the Regional Resiliency Assessment Program

GAO-13-616: Published: Jul 30, 2013. Publicly Released: Jul 30, 2013.

Additional Materials:


Stephen L. Caldwell
(202) 512-8777


Office of Public Affairs
(202) 512-4800

What GAO Found

The Department of Homeland Security (DHS) has developed nine criteria that consider various factors--including the willingness of various stakeholders, such as asset owners and operators, to participate and concentrations of high-risk critical infrastructure--when identifying possible locations for Regional Resiliency Assessment Program (RRAP) projects. According to DHS officials, final project selections are then made from a list of possible locations based on factors including geographic distribution and DHS priorities, among other considerations. However, it is unclear why some RRAP projects are recommended over others because DHS does not fully document why these decision are made. Federal internal control standards call for agencies to promptly record and clearly document transactions and significant events. Because DHS's selection process identifies a greater number of potential projects than DHS has the resources to perform, documenting why final selections are made would help ensure accountability, enabling DHS to provide evidence of its decision making.

DHS has worked with states to improve the process for conducting RRAP projects and is considering an approach for sharing resilience information with its critical infrastructure (CI) partners, including federal, state, local, and tribal officials. Since 2011, DHS has worked with states to improve the process for conducting RRAP projects, including more clearly defining the scope of projects. According to DHS officials, these efforts have been viewed favorably by states. DHS is currently considering an approach to more widely share resilience lessons learned with its CI partners, including a possible resiliency product or products that draw from completed RRAP projects. DHS officials stated that they engage CI partners in meetings and conferences where partners' resilience information needs are discussed and have been incorporating this input into their efforts to develop a resilience information sharing approach.

DHS has taken action to measure efforts to enhance security and resilience among facilities that participate in the RRAP, but faces challenges measuring results associated with RRAP projects. DHS performs security and vulnerability assessments at individual CI assets that participate in RRAPs projects as well as those that do not participate. Consistent with the National Infrastructure Protection Plan, DHS also performs periodic follow-ups among asset owners and operators that participate in these assessments with the intent of measuring their efforts to make enhancements arising out of these surveys and assessments. However, DHS does not measure how enhancements made at individual assets that participate in a RRAP project contribute to the overall results of the project. DHS officials stated that they face challenges measuring performance within and across RRAP projects because of the unique characteristics of each, including geographic diversity and differences among assets within projects. GAO recognizes that measuring performance within and among RRAP projects could be challenging, but DHS could better position itself to gain insights into projects' effects if it were to develop a mechanism to compare facilities that have participated in a RRAP project with those that have not, thus establishing building blocks for measuring its efforts to conduct RRAP projects. One approach could entail using DHS's assessment follow-up process to gather and analyze data to assess whether participation in a RRAP project influenced owners and operators to make related resilience enhancements.

Why GAO Did This Study

In October 2012, Hurricane Sandy caused widespread damage across multiple states. Further, threats to CI are not limited to natural disasters, as demonstrated by the terrorist attacks of September 11, 2001. In 2009, DHS initiated the RRAP, a voluntary program intended to assess regional resilience of CI. RRAP projects are to analyze a region's ability to adapt to changing conditions, and prepare for, withstand, and rapidly recover from disruptions.

GAO was asked to examine DHS's efforts to manage the program. GAO assessed the extent to which DHS (1) developed criteria for identifying RRAP project locations, (2) worked with states to conduct RRAP projects and share information with CI partners to promote resilience, and (3) is positioned to measure results associated with RRAP projects.

GAO reviewed applicable laws, DHS policies and procedures, and all 17 RRAP reports completed since the program inception in 2009. GAO also interviewed officials from 10 states with issued RRAP reports, DHS officials who conducted 20 RRAP projects from 2009 through 2012, and other federal officials representing nine departments and agencies involved in RRAP projects. While the results of the interviews are not generalizable, they provided insight.

What GAO Recommends

GAO recommends that DHS document final RRAP selections and develop a mechanism to measure whether RRAP participation influences facilities to make RRAP-related enhancements. DHS concurred with the recommendations.

For more information, contact Stephen L. Caldwell at (202) 512-8777 or

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: This recommendation was made in July 2013. In September 2013, DHS provided the following update. DHS agrees that modifications can be made to the nomination and selection process to better record why projects were or were not recommended to the Assistant Secretary for selection. Infrastructure Protection (IP) has taken immediate steps to implement recommended modifications for the ongoing Fiscal Year 2014 RRAP project selection process (note, this is well ahead of the initial plan to make such changes for the Fiscal Year 2015 selection process). As of August 2013, IP is already documenting justifications for recommending or not recommending projects for selection, and final IP leadership selections and non-selections will be documented by the end of September 2013. DHS subsequently provided a memorandum showing that it had begun to document decisions made with regard to RRAP project recommendations. The September 2013 memoranda showed the rationale for recommending particular projects which provided insights into why one project was recommended over another. This recommendation is closed as implemented.

    Recommendation: To help ensure that DHS is taking steps to strengthen the management of RRAP projects and the program in general, the Assistant Secretary for Infrastructure Protection, Department of Homeland Security, should document decisions made with regard to recommendations about individual RRAP projects to provide insights into why one project was recommended over another and assurance that recommendations among equally feasible proposals are defensible.

    Agency Affected: Department of Homeland Security: National Protection and Programs Directorate: Office of Infrastructure Protection: Assistant Secretary for Infrastructure Protection

  2. Status: Open

    Comments: This recommendation was made in July 2013. In January 2014, DHS's Protective Security Coordination Division (PSCD) examined five options for measuring resilience improvements at facilities resulting from RRAPs. PSCD decided upon an option to use current data from the facility follow-up process to determine statistically if individual facilities that received site visits during an RRAP implement options for consideration at a rate greater than that found following non-RRAP site visits based on the nationwide average within the appropriate sector. PSCD based this decision upon cost, the value proposition for additional follow-up engagements with individual facilities, and relevance to measuring the effectiveness of the overall RRAP. PSCD indicated it would begin employing this measurement in April 2014 with expected completion in September 2014. In response to a GAO inquiry regarding the status of this measure, in March 2015, PSCD contacted GAO with additional data comparing facility improvements for site visits at RRAP facilities with the rate of facility improvements that took place at facilities that did not participate in an RRAP. PSCD also provided information from the "RRAP Tracker" which on an annual basis measures the percent of RRAP projects that have implemented actions addressing an RRAP key finding. PSCD officials said that they decided to change the measure they use for the RRAP from the facility-based comparison to the RRAP-tracker measure. Officials said the comparison of facility improvement rates did not demonstrate the impact of the RRAP because the rates were focused on the facilities and their individual improvements, not the region and improvements to regional resiliency, which is the intention of the RRAP. Officials said the RRAP tracker, however, gets to improvements made and actions taken in response to RRAP report key findings. According to IP and PSCD officials, these actions will directly impact regional resiliency and are more appropriate indicators for the impact of the RRAP. In addition, the actions for this measure can take place within 3 years of delivery of the RRAP report and therefore includes RRAPs across multiple years, according to PSCD officials. In April 2015, PSCD provided RRAP tracker figures and sample RRAP support reports showing the specific actions taken that support the RRAP Tracker calculations. In May 2015, we requested additional evidence to tie the resilience improvements cited in the RRAP support reports to participation in the RRAP in order to close the recommendation. We told PSCD staff that this additional information was necessary to more completely link the findings in the RRAP reports to the actions taken by RRAP participants. PSCD staff said they would provide updated documentation on their efforts to satisfy the recommendation.

    Recommendation: To help ensure that DHS is taking steps to strengthen the management of RRAP projects and the program in general, the Assistant Secretary for Infrastructure Protection, Department of Homeland Security, should develop a mechanism to assess the extent to which individual projects influenced participants to make RRAP related enhancements, such as revising the security and vulnerability assessment follow-up tool to query facilities that participated in RRAP projects on the extent to which any resilience improvements made are due to participation in the RRAP.

    Agency Affected: Department of Homeland Security: National Protection and Programs Directorate: Office of Infrastructure Protection: Assistant Secretary for Infrastructure Protection


Explore the full database of GAO's Open Recommendations »

Jul 30, 2015

Jul 22, 2015

Jul 14, 2015

Jul 9, 2015

Jul 8, 2015

Jun 10, 2015

Jun 2, 2015

Looking for more? Browse all our products here