Quadrennial Homeland Security Review:

Enhanced Stakeholder Consultation and Use of Risk Information Could Strengthen Future Reviews

GAO-11-873: Published: Sep 15, 2011. Publicly Released: Oct 5, 2011.

Additional Materials:

Contact:

Diana C. Maurer
(202) 512-9627
maurerd@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The United States continues to face a range of evolving threats, such as the 2010 attempted attack on the nation's air cargo system, that underscore why homeland security planning efforts are crucial to the security of the nation. The Implementing Recommendations of the 9/11 Commission Act of 2007 required the Department of Homeland Security (DHS) to provide a comprehensive examination of the U.S. homeland security strategy every 4 years. In response, DHS issued its first Quadrennial Homeland Security Review (QHSR) report in February 2010 and a Bottom-Up Review (BUR) report in July 2010, to identify initiatives to implement the QHSR. As requested, this report addresses the extent to which DHS (1) consulted with stakeholders in developing the QHSR, (2) conducted a national risk assessment, and (3) developed priorities, plans, monitoring mechanisms, and performance measures for implementing the QHSR and BUR initiatives. GAO analyzed relevant statutes and DHS documents on the QHSR and BUR processes and, in response to a request for comments on the processes, received comments from 63 of the 85 federal and nonfederal stakeholders it contacted. Their responses are not generalizable, but provided perspectives on the processes.

DHS solicited input from various stakeholder groups in conducting the first QHSR, but DHS officials, stakeholders GAO contacted, and other reviewers of the QHSR noted concerns with time frames provided for stakeholder consultations and outreach to nonfederal stakeholders. DHS consulted with stakeholders--federal agencies; department and component officials; state, local, and tribal governments; the private sector; academics; and policy experts-- through various mechanisms, such as the solicitation of papers to help frame the QHSR and a web-based discussion forum. DHS and these stakeholders identified benefits from these consultations, such as DHS receiving varied perspectives. However, stakeholders also identified challenges in the consultation process. Sixteen of 63 stakeholders who provided comments to GAO noted concerns about the time frames for providing input into the QHSR or BUR. Nine DHS stakeholders, for example, responded that the limited time available for development of the QHSR did not allow DHS to have as deep an engagement with stakeholders. Further, 9 other stakeholders commented that DHS consultations with nonfederal stakeholders, such as state, local, and private sector entities, could be enhanced by including more of these stakeholders in QHSR consultations. In addition, reports on the QHSR by the National Academy of Public Administration, which administered DHS's web-based discussion forum, and a DHS advisory committee comprised of nonfederal representatives noted that DHS could provide more time and strengthen nonfederal outreach during stakeholder consultations. By providing more time for obtaining feedback and examining mechanisms to obtain nonfederal stakeholders' input, DHS could strengthen its management of stakeholder consultations and be better positioned to review and incorporate, as appropriate, stakeholders' input during future reviews. DHS identified threats confronting homeland security in the 2010 QHSR report, such as high-consequence weapons of mass destruction and illicit trafficking, but did not conduct a national risk assessment for the QHSR. DHS officials stated that at the time DHS conducted the QHSR, DHS did not have a well-developed methodology or the analytical resources to complete a national risk assessment that would include likelihood and consequence assessments--key elements of a national risk assessment. To develop an approach to national risk assessments, DHS created a study group as part of the QHSR process that developed a national risk assessment methodology. DHS officials plan to implement a national risk assessment in advance of the next QHSR, which DHS anticipates conducting in fiscal year 2013. DHS developed priorities, plans, monitoring mechanisms, and performance measures, but did not consider risk information in making its prioritization efforts. DHS considered various factors in identifying high-priority BUR initiatives for implementation in fiscal year 2012 but did not include risk information as one of these factors, as called for in GAO's prior work and DHS's risk management guidance, because of differences among the initiatives that made it difficult to compare risks across them, among other things. Consideration of risk information during future implementation efforts could help strengthen DHS's prioritization of mechanisms for implementing the QHSR, including assisting in determinations of which initiatives should be implemented in the short or longer term. GAO recommends that for future reviews, DHS provide the time needed for stakeholder consultations, explore options for consulting with nonfederal stakeholders, and examine how risk information could be considered in prioritizing QHSR initiatives. DHS concurred with our recommendations.

Recommendations for Executive Action

  1. Status: Closed - Not Implemented

    Comments: We found that DHS did not provide sufficient time for consultations with stakeholders during the 2010 Quadrennial Homeland Security Review (QHSR). As a result, we recommended that DHS provide more time for consulting with stakeholders to help ensure that the stakeholders are provided the time needed to review QHSR documents and provide input into the review and build this time into the department's project planning for the 2014 QHSR. During our review of the 2014 QHSR, we found that DHS did not implement the recommendation to provide more time for consulting with stakeholders during the review, based on what QHSR stakeholders told us. In a survey we conducted of 2014 QHSR stakeholders, we asked whether DHS provided a clear overview of the timeframes and stakeholder responsibilities during each of the four phases of development. Stakeholders varied in their responses from about fourteen percent reporting that they did not receive a clear overview of the timeframes and responsibilities in phase 1 to 34 percent in phase 4 of the QHSR process. Stakeholder narrative responses also indicate that these timeframes shifted and that overall time constraints persisted throughout the 2014 QHSR process. In addition, in our interviews with federal stakeholders, officials from five of nine federal agencies stated that involving the agencies earlier in the process could help alleviate compressed timeframes in the final review phase. DHS officials responsible for conducting stakeholder outreach expressed frustration with delays and noted that they were unable to move forward with some planned outreach activities. In addition, DHS SPAR officials acknowledged that engaging the stakeholder community early and regularly needs to be built into the planning for the review. As a result of time constraints, the 2014 QHSR process may not have benefited from the full participation of all relevant stakeholders and may have missed opportunities to incorporate their perspectives. Therefore, we consider this recommendation closed and not implemented.

    Recommendation: To strengthen DHS's planning, management, and execution of the next QHSR, the DHS Assistant Secretary for Policy should provide more time for consulting with stakeholders during the QHSR process to help ensure that stakeholders are provided the time needed to review QHSR documents and provide input into the review, and build this time into the department's project planning for the next QHSR.

    Agency Affected: Department of Homeland Security: Office of Policy

  2. Status: Closed - Implemented

    Comments: We found that DHS faced challenges in obtaining feedback from nonfederal stakeholders during the 2010 Quadrennial Homeland Security Review (QHSR). As a result, we recommended that DHS examine additional mechanisms for obtaining input from nonfederal stakeholders during the 2014 QHSR process, such as whether panels of state, local, and tribal government officials or components' existing advisory or other groups could be useful, and use them for obtaining nonfederal stakeholders' input, as appropriate. During our review of the 2014 QHSR, we found that DHS took steps to expand its outreach in 2014 QHSR process to elicit input from a broader set of stakeholders, including additional nonfederal entities such as state, local, and tribal governments, private organizations, and foreign governments. DHS stated that it relied primarily on its electronic tools to elicit input from nonfederal stakeholders. According to DHS officials, the electronic tools elicited the views of more than 2,000 registered participants and informed the analytical studies that ultimately shaped the final QHSR report. Given the volume and diversity of views among the stakeholders, DHS determined that using the electronic tool mechanism was the most efficient way to capture the majority of the input from the nonfederal entities for the 2014 QHSR review. Based on these actions, we consider this recommendation closed and implemented.

    Recommendation: To strengthen DHS's planning, management, and execution of the next QHSR, the DHS Assistant Secretary for Policy should examine additional mechanisms for obtaining input from nonfederal stakeholders during the QHSR process, such as whether panels of state, local, and tribal government officials or components' existing advisory or other groups could be useful, and use them for obtaining nonfederal stakeholders' input, as appropriate, during the next QHSR.

    Agency Affected: Department of Homeland Security: Office of Policy

  3. Status: Closed - Implemented

    Comments: We found that DHS identified threats confronting homeland security in the 2010 QHSR report, such as high-consequence weapons of mass destruction and illicit trafficking, but did not conduct a national risk assessment for the QHSR. We also found that DHS developed priorities, plans, monitoring mechanisms, and performance measures, but did not consider risk information in making its prioritization efforts. As a result, we recommended that DHS examine the extent to which risk information could be used as one input to prioritize QHSR implementing mechanisms. In June 2014, DHS published its second QHSR, which was informed by DHS's Homeland Security Strategic Environment Assessment--a risk characterization designed to provide a greater understanding of homeland security risks and to inform the selection of strategic priorities for the 2014 QHSR. In addition to informing the QHSR, DHS reported using the Homeland Security Strategic Environment Assessment to inform annual budgeting guidance, operational planning guidance, and the selection of annual strategic priorities. Ultimately, DHS's risk assessment provided context for the various QHSR and Strategic Plan implementation mechanisms; however, DHS was unable to demonstrate the extent to which this data was used to prioritize which initiatives or programs should be implemented in the short versus long term, or how to allocate resources to address different levels and types of risks. Nonetheless, DHS's consideration of risk information is consistent with our recommendation, and therefore we consider this recommendation closed as implemented.

    Recommendation: To strengthen DHS's planning, management, and execution of the next QHSR, the DHS Assistant Secretary for Policy should examine the extent to which risk information could be used as one input to prioritize QHSR implementing mechanisms, including reviewing the extent to which the mechanisms could include characteristics, such as defined outcomes, to allow for comparisons of the risks addressed by each mechanism. To the extent that DHS determines that risk information could be used, consider such information as one input into the decision-making process for prioritizing the QHSR implementation mechanisms.

    Agency Affected: Department of Homeland Security: Office of Policy

 

Explore the full database of GAO's Open Recommendations »

Aug 15, 2016

Jul 21, 2016

Jul 12, 2016

Jul 7, 2016

Jun 14, 2016

Jun 7, 2016

Looking for more? Browse all our products here