Coastal Zone Management: Opportunities Exist for NOAA to Enhance Its Use of Performance Information
Highlights
What GAO Found
During fiscal years 2008 through 2013, the 34 states participating in the National Oceanic and Atmospheric Administration's (NOAA) National Coastal Zone Management Program (CZMP) allocated nearly $400 million in CZMP funds for a variety of activities. States allocated this funding for activities spanning six broad focus areas based on goals outlined in the Coastal Zone Management Act. For example, states allocated about a quarter of their CZMP funding to the coastal habitat focus area, according to NOAA's analysis. Coastal habitat activities encompassed a variety of actions to protect, restore, or enhance coastal habitat areas, such as habitat mapping or restoration planning efforts of marsh habitats for fish and wildlife and enhanced recreational opportunities.
NOAA's two primary performance assessment tools—its CZMP performance measurement system and state program evaluations—have limitations, even with changes NOAA made since 2008, and NOAA makes limited use of the performance information it collects. Regarding the performance measurement system, NOAA has made changes such as taking steps intended to improve the reliability of data it collects. However, its current measurement system does not align with some key attributes of successful performance measures, including the following:
Balance: a balanced set of measures ensures that a program's various goals are covered. NOAA removed the coastal water quality focus area, one of six focus areas based on goals in the act, to streamline the performance measurement system. As a result, the system may not provide a complete picture of states' overall performance across all focus areas based on goals in the act.
Limited overlap: measures should produce new information beyond what is provided by other data sources . NOAA's system includes measures that overlap with financial data provided in cooperative agreements. By requiring states to submit financial data available through other sources, NOAA may be unnecessarily burdening states with data collection requirements.
NOAA plans to review and potentially revise its measurement system, but it has not documented the approach it plans to take, including how the measures will align with key attributes of successful performance measures. Regarding state program evaluations, in 2013, NOAA revised its process to conduct evaluations more efficiently, at a reduced cost. However, GAO identified a limitation in NOAA's method for sampling stakeholders to survey under its revised process that may result in the selection of stakeholders that do not span all six focus areas based on goals of the act. Finally, NOAA makes limited use of the performance information it collects from these tools. For example, since it began collecting performance measurement data in 2008, NOAA used the data once to report on accomplishments. NOAA recognizes the importance of using performance information to improve program implementation, but it has not documented a strategy for how it will use its performance information to manage the program. As a result, NOAA may not be realizing the full benefit of collecting performance information.
Why GAO Did This Study
The U.S. coast is home to more than half the U.S. population and integral to the nation's economy. Under the Coastal Zone Management Act, NOAA administers the CZMP, a federal-state partnership that encourages states to balance development with protection of coastal zones in exchange for federal financial assistance and other incentives. In 2008, GAO reviewed the CZMP and recommended improvements for CZMP performance assessment tools.
A fiscal year 2013 appropriations committee report mandated GAO to review NOAA's implementation of the act. This report examines (1) how states allocated CZMP funds awarded in fiscal years 2008 through 2013 and (2) how NOAA's primary performance assessment tools have changed since GAO's 2008 report and the extent to which NOAA uses performance information in managing the CZMP. GAO reviewed laws, guidance, and performance-related reports; analyzed CZMP funding data for fiscal years 2008-2013; and interviewed NOAA officials and a nongeneralizeable sample of officials from seven states selected for receiving the most fiscal year 2012 funding in each of NOAA's regions.
Recommendations
GAO recommends that NOAA document an approach to analyze and revise, as appropriate, its performance measures against key attributes, revise its process for selecting stakeholders to survey in its state program evaluations, and document a strategy for using the performance information it collects. NOAA concurred with the recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Commerce | To ensure that NOAA collects and uses meaningful performance information to help manage the CZMP, including continuing to improve its CZMP performance measurement system and its state program evaluations, the Secretary of Commerce should direct the Administrator of NOAA to develop a documented strategy to use the range of performance information the agency collects, as appropriate, to aid its management of the CZMP, such as to identify potential problems or weaknesses in the CZMP; set program priorities or strategies; or recognize and reward high-performing state programs. |
In December 2015, NOAA developed a strategy for using performance information which focused on the use of information for decision-making by NOAA and improving its communication and messaging. As of August 2016, NOAA had completed some of the actions identified in the strategy, including the development and dissemination of an annual national level performance progress report to help with program planning, among other actions.
|
| Department of Commerce | To ensure that NOAA collects and uses meaningful performance information to help manage the CZMP, including continuing to improve its CZMP performance measurement system and its state program evaluations, the Secretary of Commerce should direct the Administrator of NOAA to, as part of its intended review of the CZMP performance measurement system and in consideration of how it intends to use the performance information, document the approach it plans to take to analyze and revise, as appropriate, the performance measures, and in so doing ensure the analysis considers key attributes of successful performance measures, such as balance and limited overlap. |
In response to this recommendation, by November 2016, NOAA adopted the revisions proposed by a NOAA team tasked with analyzing and revising the performance measurement system and updated its guidance, accordingly. Specifically, the designated NOAA team used a decision matrix that evaluated existing measures against a series of 19 criteria, such as whether measures are useful to states and NOAA management decision-making, represent core program activities, or are redundant with other information NOAA collects. The team also considered input from NOAA management and state stakeholders in making the final recommendation to help streamline and improve the performance measurement system. By October 2016, states collected and began reporting performance measures data using the revised guidance and measures.
|
| Department of Commerce | To ensure that NOAA collects and uses meaningful performance information to help manage the CZMP, including continuing to improve its CZMP performance measurement system and its state program evaluations, the Secretary of Commerce should direct the Administrator of NOAA to revise the sampling methodology for selecting stakeholders to survey--included as part of its state program evaluation process--to ensure perspectives are gathered from stakeholders representing a variety of program goals and are collected in an objective manner. |
In February 2015, NOAA developed internal guidance on conducting stakeholder surveys that included methods for selecting survey participants representing all CZMP areas, and outlined its privacy policy to ensure confidentiality of survey participants that could assist in the collection of unbiased responses.
|