Homeland Security:

Guidance and Standards Are Needed for Measuring the Effectiveness of Agencies' Facility Protection Efforts

GAO-06-612: Published: May 31, 2006. Publicly Released: Jul 7, 2006.

Additional Materials:

Contact:

Mark L. Goldstein
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The need to better protect federal facilities, coupled with federal budget constraints and the increased scrutiny of homeland security funding and programs, has prompted the need for U.S. agencies to measure the performance of their facility protection efforts. In this environment, it is important for these agencies to ensure that investments in facility protection are providing adequate returns in terms of better protecting real property assets against terrorism. In addition, the U.S. government's national strategy, Presidential directive, and guidance on protecting critical infrastructures--including facilities--have identified the use of performance measurement as a key means of assessing the effectiveness of protection programs. Given that protection of critical infrastructures is an important issue for organizations outside of the federal government as well, it is beneficial to look to the experiences of these organizations to identify lessons learned. As such, our objectives for this review were (1) to identify examples of performance measures for facility protection being used by selected organizations outside of the federal government--including private-sector entities, state and local governments, and foreign governments, and (2) to determine the status of U.S. federal agencies' efforts to develop and use performance measures as part of their facility protection programs.

We found a range of examples of performance measures that organizations outside the U.S. government, including private-sector firms, state and local governments, and foreign government agencies, use to help improve the security of facilities, inform risk-management and resource-allocation decisions, and hold security officials and others in their organizations accountable for security performance. These included output measures, such as the average time to process background screenings, and outcome measures, such as the change in the total number of security incidents relating to thefts, vandalism, and acts of terrorism. Despite some organizations' use of these measures, less than one-quarter of the organizations we contacted had developed performance measures for facility protection, and there was widespread acknowledgement among the organizations that effectiveness in facility protection is challenging to measure. We found that some bureaus and services within three of the agencies we reviewed--DHS (for GSA properties), USPS, and Interior--are using output measures, and, to a lesser extent, outcome measures, while VA and some bureaus and services within the other three agencies are not. The agencies that have developed performance measures use them to evaluate and improve program effectiveness, make risk management decisions, and help ensure adequate protection at individual facilities. For example, within DHS, FPS has established an output-oriented performance measure to monitor the timely deployment of security enhancements such as x-ray machines. Such a measure provides a basis for FPS to compare planned versus actual performance. Several bureaus and services within USPS and Interior have developed methodologies to rank and monitor the relative risk ratings of their respective facilities over time--these ratings are then used as outcome measures for determining the change in the effectiveness of facility protection efforts. VA and the bureaus and services that did not have security performance measures generate data on ongoing protection activities, such as monitoring the numbers and types of security breaches at a given facility. This information could provide useful feedback about the agency's effectiveness in mitigating building security risks and therefore could be used for measuring performance. Although agencies have placed an emphasis on performance measurement and initiatives are under way, agency security officials said it has been challenging to measure the actual impact of various approaches on improving security and that resources for measurement initiatives have been scarce. Furthermore, while importance has been placed on performance measures in national homeland security policies and broad guidance exists for measuring the performance of critical infrastructure protection programs, agencies have not established specific guidance and standards for developing and using performance measures for facility protection programs in particular. This differs from the information technology security area, where agencies not only are required to measure performance, but also have detailed guidance and standards for developing and implementing performance measures. Without effective performance measurement data, especially data on program outcomes, decision makers may have insufficient information to evaluate whether the benefits of security investments justify their costs, to determine the effectiveness of security activities, to know the extent to which security enhancements have improved security or reduced federal facilities' vulnerability to acts of terrorism or other forms of violence, or to determine funding priorities within and across agencies.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In response to this report, the Interagency Security Committee issued guidance entitled "Use of Physical Security Performance Measures" in 2009. According to ISC, this policy requires all federal agencies to assess and document the effectiveness of their physical security programs through performance measurement and testing. This standard intends to provide guidance on how to establish and implement a comprehensive measurement and testing program.

    Recommendation: To ensure that useful information is available for making decisions about the allocation of resources for, and the effectiveness of investments in, the protection of federal facilities, the Secretary of Homeland Security should direct the Chair of the Interagency Security Committee (ISC), as part of ISC's efforts to support DHS in developing sector-specific performance measures for the security of federal government facilities, to establish guidance and standards, with input from ISC member agencies, for measuring performance in facility protection--with a particular focus on developing outcome measures.

    Agency Affected: Department of Homeland Security: Interagency Security Committee

  2. Status: Closed - Implemented

    Comments: We found a range of examples of performance measures that organizations outside the U.S. government--including private-sector entities, state and local governments, and foreign government agencies--have developed that, collectively, indicate whether facility protection efforts are achieving results. Without effective performance measurement data, decision makers may have insufficient information to evaluate whether their investments have improved security or reduced federal facilities vulnerability to acts of terrorism or other forms of violence. However, agencies faced challenges in further developing and using security performance measures, and there was no governmentwide guidance or standards on measuring facility protection performance to help federal agencies address these challenges. We recommended, in addition to developing such guidance, that the Interagency Security Committee (ISC) communicate its guidance and standards for measuring performance in federal government facility protection to relevant federal agencies. In response, ISC made its guidance and standards entitled, "Use of Physical Security Performance Measures" available on the Department of Homeland Security's website and the guidance intended for use by federal departments and agencies. As a result, this guidance (issued in 2009) provides a performance model that measures inputs and accomplishments, identifies the performance measurement cycle process, and provides examples of physical security performance metrics.

    Recommendation: To ensure that useful information is available for making decisions about the allocation of resources for, and the effectiveness of investments in, the protection of federal facilities, the Secretary of Homeland Security should direct the Chair of the ISC to communicate the established guidance and standards to the relevant federal agencies.

    Agency Affected: Department of Homeland Security: Interagency Security Committee

  3. Status: Closed - Implemented

    Comments: We found a range of examples of performance measures that organizations outside the U.S. government--including private-sector entities, state and local governments, and foreign government agencies--have developed that, collectively, indicate whether facility protection efforts are achieving results. Without effective performance measurement data, decision makers may have insufficient information to evaluate whether their investments have improved security or reduced federal facilities vulnerability to acts of terrorism or other forms of violence. However, agencies faced challenges in further developing and using security performance measures, and there was no governmentwide guidance or standards on measuring facility protection performance to help federal agencies address these challenges. We recommended, in addition to developing such guidance and standards, that the Interagency Security Committee (ISC) ensure that the guidance and standards are updated regularly. In response, ISC issued performance measurement guidance and standards in 2009, entitled, "Use of Physical Security Performance Measures." According to ISC, the guidance provides the foundation for a measurement program that is one of six key management practices the ISC is promoting within the federal physical security committee. In addition, ISC's identification of performance measurement as one of six key practices it is promoting is responsive to the spirit of GAO's recommendation that the guidance be regularly reviewed and updated. As a result, regular review and updating of the guidance should better ensure that agencies have a current performance model that measures inputs and accomplishments, identifies the performance measurement cycle process, and provides examples of physical security performance metrics.

    Recommendation: To ensure that useful information is available for making decisions about the allocation of resources for, and the effectiveness of investments in, the protection of federal facilities, the Secretary of Homeland Security should direct the Chair of the ISC to ensure that the guidance and standards are regularly reviewed and updated.

    Agency Affected: Department of Homeland Security: Interagency Security Committee

 

Explore the full database of GAO's Open Recommendations »

Dec 19, 2014

Dec 17, 2014

Nov 6, 2014

Oct 14, 2014

Sep 30, 2014

Sep 24, 2014

Sep 18, 2014

Looking for more? Browse all our products here