This is the accessible text file for GAO report number GAO-11-800 entitled 'Climate Monitoring: NOAA Can Improve Management of the U.S. Historical Climatology Network' which was released on September 30, 2011. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to the Ranking Member, Committee on Environment and Public Works, U.S. Senate: August 2011: Climate Monitoring: NOAA Can Improve Management of the U.S. Historical Climatology Network: GAO-11-800: GAO Highlights: Highlights of GAO-11-800, a report to the Ranking Member, Committee on Environment and Public Works, U.S. Senate. Why GAO Did This Study: The National Oceanic and Atmospheric Administration (NOAA) maintains a network of weather-monitoring stations known as the U.S. Historical Climatology Network (USHCN), which monitors the nation’s climate and analyzes long-term surface temperature trends. Recent reports have shown that some stations in the USHCN are not sited in accordance with NOAA’s standards, which state that temperature instruments should be located away from extensive paved surfaces or obstructions such as buildings and trees. GAO was asked to examine (1) how NOAA chose stations for the USHCN, (2) the extent to which these stations meet siting standards and other requirements, and (3) the extent to which NOAA tracks USHCN stations’ adherence to siting standards and other requirements and has established a policy for addressing nonadherence to siting standards. GAO reviewed data and documents, interviewed key NOAA officials, surveyed the 116 NOAA weather forecast offices responsible for managing stations in the USHCN, and visited 8 forecast offices. What GAO Found: In choosing USHCN stations from a larger set of existing weather- monitoring stations, NOAA placed a high priority on achieving a relatively uniform geographic distribution of stations across the contiguous 48 states. NOAA balanced geographic distribution with other factors, including a desire for a long history of temperature records, limited periods of missing data, and stability of a station’s location and other measurement conditions, since changes in such conditions can cause temperature shifts unrelated to climate trends. NOAA had to make certain exceptions, such as including many stations that had incomplete temperature records. In general, the extent to which the stations met NOAA's siting standards played a limited role in the designation process, in part because NOAA officials considered other factors, such as geographic distribution and a long history of records, to be more important. USHCN stations meet NOAA’s siting standards and management requirements to varying degrees. According to GAO’s survey of weather forecast offices, about 42 percent of the active stations in 2010 did not meet one or more of the siting standards. With regard to management requirements, GAO found that the weather forecast offices had generally but not always met the requirements to conduct annual station inspections and to update station records. NOAA officials told GAO that it is important to annually visit stations and keep records up to date, including siting conditions, so that NOAA and other users of the data know the conditions under which they were recorded. NOAA officials identified a variety of challenges that contribute to some stations not adhering to siting standards and management requirements, including the use of temperature-measuring equipment that is connected by a cable to an indoor readout device-—which can require installing equipment closer to buildings than specified in the siting standards. NOAA does not centrally track whether USHCN stations adhere to siting standards and the requirement to update station records, and it does not have an agencywide policy regarding stations that do not meet its siting standards. Performance management guidelines call for using performance information to assess program results. NOAA’s information systems, however, are not designed to centrally track whether stations in the USHCN meet its siting standards or the requirement to update station records. Without centrally available information, NOAA cannot easily measure the performance of the USHCN in meeting siting standards and management requirements. Furthermore, federal internal control standards call for agencies to document their policies and procedures to help managers achieve desired results. NOAA has not developed an agencywide policy, however, that clarifies for agency staff whether stations that do not adhere to siting standards should remain open because the continuity of the data is important, or should be moved or closed. As a result, weather forecast offices do not have a basis for making consistent decisions to address stations that do not meet the siting standards. What GAO Recommends: GAO recommends that NOAA enhance its information systems to centrally capture information useful in managing the USHCN and develop a policy on how to address stations that do not meet its siting standards. NOAA agreed with GAO’s recommendations. View [hyperlink, http://www.gao.gov/products/GAO-11-800] or key components. For more information, contact Anu K. Mittal at (202) 512- 3841 or mittala@gao.gov. [End of section] Contents: Letter: Background: USHCN Stations Were Selected on the Basis of Several Factors, but Siting Conditions Played a Limited Role: USHCN Stations Meet Siting Standards and Management Requirements to Varying Degrees: NWS Does Not Centrally Track Whether USHCN Stations Meet Siting Standards or Have a Policy for Addressing Stations That Do Not Meet the Standards: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Scope and Methodology: Appendix II: Other National Climate Networks: Appendix III: Comments from the Department of Commerce: Appendix IV: GAO Contact and Staff Acknowledgments: Table: Table 1: Differences in Selected Siting Standards of National Climate Networks: Figures: Figure 1: NOAA's Organizational Structure for Managing the USHCN: Figure 2: Percentage of Climate Divisions, and Number of Active USHCN Stations, as of April 2011: Figure 3: Active USHCN Stations in 2010 That Did Not Adhere to One or More NWS Siting Standards: Figure 4: Comparison of Conditions at Two USHCN Stations That Do Not Meet NWS Siting Standards: Figure 5: Primary Factors Contributing to USHCN Stations Not Adhering to NWS Siting Standards: Figure 6: A USHCN Station with Obstructions to Temperature-Measuring Equipment but No Obstructions Shown on the Station Record: Figure 7: U.S. Climate Reference Network Station: Abbreviations: NCDC: National Climatic Data Center: NOAA: National Oceanic and Atmospheric Administration: NWS: National Weather Service: USHCN: U.S. Historical Climatology Network: [End of section] United States Government Accountability Office: Washington, DC 20548: August 31, 2011: The Honorable James M. Inhofe: Ranking Member: Committee on Environment and Public Works: United States Senate: Dear Senator Inhofe: The National Oceanic and Atmospheric Administration's (NOAA) weather forecast offices maintain a network of thousands of weather-monitoring stations throughout the United States, some with temperature records stretching back well over a century. These weather-monitoring stations are equipped to measure meteorological conditions at ground level-- usually the daily maximum and minimum temperatures and 24-hour precipitation totals--to support weather forecasts and for the study of climate. In 1987 NOAA designated a subset of these weather- monitoring stations as the U.S. Historical Climatology Network (USHCN). The purpose of the USHCN is to monitor the nation's climate and, in particular, to analyze long-term surface temperature trends. On the basis of their analysis of data from a variety of sources, including temperature records from the USHCN, NOAA scientists have concluded that the average surface temperatures across the nation have warmed over the past century. The temperature measurements used in NOAA's analysis have been taken largely by trained volunteer observers at the weather-monitoring stations that are part of the USHCN. The stations are located at private residences, farms, and parks, as well as on the properties of universities, water treatment plants, airports, and other institutions. The volunteer observers generally record temperature measurements on a daily basis using equipment that NOAA has placed on their properties. NOAA has established standards for air temperature measurement, which state, among other things, that temperature instruments should be located away from extensive paved surfaces or obstructions such as buildings and trees. Siting conditions at stations can change over time, however, particularly for stations with many years of data, as volunteer observers make changes to their properties, urbanization occurs, or local land uses change. Recent reports have shown that some USHCN stations are not sited in accordance with NOAA's standards. According to NOAA officials, for many years the agency has recognized the need to improve confidence in its data on changes to the national climate and, as part of this effort, in 2001 took the initiative to establish a new national climate observation network, the U.S. Climate Reference Network. NOAA is also developing another new observation network designed to monitor regional climate trends. Unlike many of the stations in the USHCN, the monitoring stations for these new networks are generally located on government rather than private property and do not require a human observer to record temperature measurements. According to NOAA officials, however, the agency plans to continue relying on data from the USHCN to help characterize national and regional temperature trends. You asked us to examine issues related to the siting of USHCN stations. Specifically, we examined (1) how NOAA selected weather- monitoring stations for inclusion in the USHCN, (2) the extent to which such stations meet siting standards and management requirements for weather-monitoring stations, and (3) the extent to which NOAA tracks USHCN stations' adherence to siting standards and management requirements and has established a policy for addressing stations that do not adhere to siting standards. As we discussed with your office, we focused our examination of these issues on NOAA's management of the USHCN and did not assess the effect of stations not meeting siting standards on the reliability of NOAA's analysis of temperature trends. To examine NOAA's selection of USHCN stations, we reviewed NOAA documents describing the selection process, interviewed NOAA scientists, and analyzed NOAA data on all stations that constitute the USHCN. We analyzed the data to determine the extent to which USHCN stations meet the criteria NOAA scientists considered important for the purpose of monitoring long-term temperature trends, such as geographic distribution of stations. To assess the extent to which USHCN stations meet siting standards and management requirements for weather-monitoring stations, we surveyed all 116 NOAA weather forecast offices in the contiguous 48 states, which are responsible for managing these stations, and received a response rate of 100 percent.[Footnote 1] Our questionnaire included questions regarding adherence to siting standards, reasons for stations not adhering to the standards, and general management challenges. We visited a nonprobability sample of 8 weather forecast offices across the country to discuss these issues in greater depth, observe siting conditions at a total of 19 USHCN stations managed by the offices, and review weather forecast office files for network stations to examine adherence to general management requirements for weather-monitoring stations. The set of 19 USHCN stations we visited was also a nonprobability sample we selected to ensure that we observed a variety of siting conditions. To ensure geographic distribution in the weather forecast offices we visited, we selected 2 offices in each of the four NOAA regions in the contiguous United States, and we selected the specific offices we visited to ensure a range of sizes in terms of the offices' forecast areas. Because we used a nonprobability sample of weather forecast offices and USHCN stations to visit, the information we obtained from these visits cannot be generalized to other weather forecast offices or USHCN stations. The visits instead provided us with more in-depth information on the perspectives of various participants in the weather forecast offices about managing weather- monitoring stations and examples of station siting conditions. To evaluate the extent to which NOAA tracks USHCN stations' adherence to siting standards and management requirements and has established a policy for addressing stations that do not adhere to siting standards, we reviewed NOAA documents and interviewed NOAA officials responsible for managing weather-monitoring stations. Appendix I presents a more detailed description of our scope and methodology. We conducted this performance audit from August 2010 through August 2011, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: NOAA's National Weather Service (NWS) manages the approximately 11,000 weather-monitoring stations across the country that are part of the Cooperative Observer Program.[Footnote 2] Volunteer observers at the stations generally record daily maximum and minimum temperatures and 24-hour precipitation totals and submit the data to NWS over the telephone, by Internet, or by mail. The records for stations in the Cooperative Observer Program can stretch back well over a century, with some records predating the establishment of NWS in 1890.[Footnote 3] NWS uses data from these Cooperative Observer Program stations to support weather forecasts and warnings and other public service programs. The data are also used by others, including state climatologists, farmers, and resource planners such as energy providers that use weather information to anticipate and plan for varying levels of energy consumption. NOAA's National Climatic Data Center (NCDC)[Footnote 4] established the USHCN in 1987 by selecting a subset of weather-monitoring stations from the existing Cooperative Observer Program network of stations. The USHCN currently consists of 1,218 stations. NCDC has twice revised the makeup of stations that compose the USHCN--in 1996 and 2009-- primarily to extend the weather records of stations that have closed over time as volunteer observers have discontinued their service. To address this issue, NCDC added data from nearby stations with similar temperature trends that are continuing to gather and report data. In all, NCDC has added over 100 stations as of the latest revision in 2009. NCDC does not have a direct role in managing USHCN stations but relies on NWS's weather forecast offices throughout the contiguous United States to continue to manage the stations as part of the larger group of weather-monitoring stations in the Cooperative Observer Program. For example, NCDC relies on weather forecast offices to maintain records on the location of the stations and other conditions that can affect weather observations, including the types of equipment used to measure temperature and precipitation and the time of observation. NCDC uses USHCN data to assess and monitor climate variation and change, including to quantify national-and regional-scale temperature trends within the contiguous 48 states. On the basis of its analysis of USHCN data, NCDC estimates that the average surface temperature across the contiguous states has warmed by about 1.4 degrees Fahrenheit since 1895.[Footnote 5] NCDC's analysis has also identified areas of the country where temperatures have cooled or remained relatively stable. NCDC combines temperature records from the USHCN with temperature records from weather-monitoring stations around the world to analyze global temperature trends. This analysis has in turn been summarized in the assessment reports of the Intergovernmental Panel on Climate Change, an international body that reviews and assesses the most recent scientific and technical information produced worldwide relevant to the understanding of climate change. NWS headquarters establishes the policies, standards, and requirements for managing the Cooperative Observer Program, and weather forecast offices in six NWS regions (central, eastern, southern, western, Alaska, and Pacific) have responsibility for recruiting and training observers and installing and maintaining temperature-measuring equipment and rain gauges on observers' properties. NWS applies the same standards and requirements to all stations in its Cooperative Observer Program, including those in the USHCN. In particular, NWS has established siting standards for measuring air temperature to ensure uniformity in meeting national and international requirements for climate observation.[Footnote 6] The standards, which cover conditions in the immediate vicinity of the stations, specify that temperature- measuring instruments should: * not be sited on rooftops; * be installed over level terrain; * be installed at least 100 feet from any extensive concrete or paved surface; * be mounted 4 to 6 feet above the surface; and: * be no closer than four times the height of any nearby building, tree, fence, or similar obstruction. NWS guidelines state that implementation of these standards should be flexible and balanced with other factors, such as the availability of space. According to NWS, these siting standards are based in part on recommendations of the World Meteorological Organization, an agency of the United Nations that, among other things, coordinates the activities of member states to generate data and information on weather and climate in accordance with international standards. For example, according to World Meteorological Organization guidelines, the best sites for measuring air temperature are over level ground; freely exposed to sunshine and wind; and not shielded by or close to trees, buildings, and other obstructions.[Footnote 7] NWS has also established management requirements for weather- monitoring stations that call for inspections of stations and updates of station records to reflect any changes. The requirements for inspections call for a minimum of one inspection by weather forecast office officials per year and specify that during these inspections, the officials are to review observers' practices for taking weather measurements, check equipment and perform any needed repairs, and assess the conditions surrounding the station, among other things. The management requirements state that, even if there are no changes at a station, officials from weather forecast offices should update each station record at least once every 5 years. To provide a complete and permanent record of a station, NWS has designed an information system that weather forecast offices are to use to record the dates of inspections and update station records.[Footnote 8] Such records are used by NCDC and other researchers to help interpret weather records from a station and determine how factors such as station location and measurement instruments affect the weather records. Figure 1 depicts the roles of NWS and NCDC in managing the USHCN. Figure 1: NOAA's Organizational Structure for Managing the USHCN: [Refer to PDF for image: illustration] National Oceanic and Atmospheric Administration: * National Weather Service headquarters: Establishes policies, standards, and requirements for managing the Cooperative Observer Program: - Regional offices: Implement and ensure compliance with national policy; - Weather forecast offices: Manage weather-monitoring stations: National Weather Service headquarters: Manages: Cooperative Observer Program: Comprises about 11,000 weather stations located across the country. * National Environmental Satellite, Data, and Information Service: - National Climatic Data Center: Selects USHCN stations, analyzes national- and regional-scale temperature trends, and archives all station documentation. National Environmental Satellite, Data, and Information Service: Manages: USHCN: Includes a subset of 1,218 stations in the contiguous 48 states (within the Cooperative Observer Program). Source: GAO. [End of figure] USHCN Stations Were Selected on the Basis of Several Factors, but Siting Conditions Played a Limited Role: According to NCDC officials, achieving a relatively uniform geographic distribution across the contiguous 48 states was a high priority when selecting USHCN stations and was balanced with other factors, including how long stations had collected temperature records, limited periods of missing temperature data, and the stability of measurement conditions. According to NCDC officials, consideration of siting conditions in the immediate vicinity of stations played a limited role in both the initial selections in 1987 and when stations were added in 1996 and 2009 because they considered other factors, such as geographic distribution, to be more important to the analysis of long- term temperature trends. NCDC Selected USHCN Stations by Balancing Geographic Distribution with Other Factors: NCDC officials told us that in selecting stations for the USHCN, the agency placed a high priority on achieving geographic distribution across the contiguous 48 states, so that the network could help identify both national and regional warming and cooling trends. To achieve the geographic distribution needed to identify regional trends, according to agency officials, NCDC aimed to select a minimum of two stations from each of the 344 climate divisions across the country.[Footnote 9] NCDC officials acknowledged that they encountered difficulties achieving the desired geographic distribution in certain areas of western states--such as Nevada--that have a relatively low population density and thus fewer stations to choose from because of a lack of volunteers to serve as observers. As a result, according to NCDC officials, station density is slightly higher across the eastern states than in the western states. Our analysis of all 1,218 USHCN stations (including active stations and those that were inactive or closed) found that while NCDC generally met its aim of two stations per climate division, 14 percent of climate divisions had fewer than two stations.[Footnote 10] As of April 2011, 20 percent of climate divisions had fewer than two active stations (see figure 2). According to NCDC officials, the existing climate divisions are only one way to partition the nation's climate, and if divisions were being developed today, the climate divisions would differ in number and in the areas they cover. Figure 2: Percentage of Climate Divisions, and Number of Active USHCN Stations, as of April 2011: [Refer to PDF for image: pie-chart] More than two stations: 58%; Two stations: 22%; One station: 15%; No stations: 5%. Source: GAO analysis of USHCN data. [End of figure] According to NCDC officials and documents describing the process used to select USHCN stations in 1987 and to amend the list of stations in 1996 and 2009, the agency also sought stations that had temperature records dating back to the early 20th century, had limited periods of missing data, and had a limited number of station changes, but sometimes made exceptions to these factors. * Number of years of temperature records. In order to detect long-term temperature trends, NCDC aimed to select stations that had a long history of temperature records, ideally dating back to the early 20th century. In some cases, however, NCDC selected stations with a shorter history of temperature records than was ideal to ensure geographic distribution of stations across the contiguous 48 states, according to officials. NCDC officials also told us that they created composite stations to achieve a minimum record length when no stations in a particular geographic area had been collecting temperature records as long as they sought. According to NCDC officials, they create a composite station by combining data from one or two stations that have closed with data from an active station in the same area whose temperature records overlap in time with records from the closed station or stations and continue to the present. NCDC officials told us that they compare the stations' overlapping temperature trends before creating a composite to help ensure that the climates at the stations are similar. According to NCDC documents, the initial selection of USHCN stations in 1987 included 84 composite stations, and, as of the latest revision to the network in 2009, the number of composite stations had increased to 208, largely in response to station closures. Our analysis of the 1,218 stations that make up the USHCN as of the latest revision in 2009, including composite stations, found that NCDC has largely achieved its desired record length. Specifically, as of 2010, over 85 percent of the stations had a record length dating back more than 100 years, and another 14 percent had temperature record lengths of 76 to 100 years. Less than 1 percent of stations had record lengths of 75 or fewer years. * Extent of missing data. NCDC officials told us that they also attempted to select USHCN stations with limited periods of missing data but that they often had to select stations with incomplete temperature records, including stations that were missing data for multiple years, because few stations have complete records. For example, about half the data from the Little Falls Mill Street station, located in upstate New York, are missing. The station's record has data for a few years in the 19th century, but data in the intervening years are sparse, with frequent gaps in the middle of the 20th century, according to a 1990 NCDC report on the USHCN.[Footnote 11] According to NCDC, various factors result in missing data, such as periods when a volunteer observer is not available or when instruments malfunction and need to be repaired. Our analysis of temperature records shows that only 24 of the 1,218 USHCN stations (about 2 percent) have complete temperature data from the time they were established through 2010; the remaining 98 percent of stations are missing an average of 5 percent of temperature data. To generate uninterrupted temperature records, NCDC uses estimates for the missing data based on records from nearby stations in the larger set of Cooperative Observer Program weather-monitoring stations. For example, according to agency officials, NCDC used this process to fill in missing data for the Little Falls Mill Street station. According to NCDC officials, filling in missing data ensures that temperature records from all areas of the contiguous 48 states are represented when the agency uses the USHCN to identify national temperature trends. * Stability of measurement conditions. A final consideration in selecting USHCN stations was NCDC's desire to maximize the stability of measurement conditions--such as station location, type of temperature-measuring instrument, and time of day when observations were recorded--because such stability makes it easier to discern actual temperature trends at a station. NCDC officials told us that, like stability in other measurement conditions at USHCN stations, stability in siting conditions facilitates officials' ability to use temperature data to accurately identify long-term warming and cooling trends, even if those conditions do not meet NWS siting standards. Most stations with long temperature records, however, are likely to have undergone multiple changes in measurement conditions.[Footnote 12] For example, according to NCDC's records, the Reno, Nevada, USHCN station was originally located at an NWS weather forecast office before being moved in the mid-1930s to an airport and then again in the 1990s to another location at the same airport. According to NCDC, such changes in measurement conditions may cause a rise or drop in the temperatures recorded at stations, which could affect the temperature trends identified using the USHCN. For example, NCDC has studied the impact of a gradual change in the time that observers record temperature measurements from afternoon to morning observation times and concluded that the change has obscured the warming trend across the contiguous 48 states, which would otherwise have appeared more pronounced. NCDC officials told us that they use statistical methods to identify significant shifts in temperature data unrelated to actual trends in temperature and to adjust the data to remove such shifts. [Footnote 13] According to NCDC officials, all 1,218 USHCN stations have undergone at least one change in measurement conditions requiring such an adjustment, with an average per station of four to five changes. NCDC officials acknowledged that a greater degree of stability of measurement conditions than typically found at USHCN stations would be preferable. As a result, NCDC has established a new network of surface weather-monitoring stations specifically to monitor the nation's climate--the U.S Climate Reference Network--and is establishing a second one--the U.S. Regional Climate Reference Network (see appendix II). According to agency officials, they have developed criteria for selecting locations for stations in the new networks to help ensure a greater degree of station stability in comparison with the USHCN and reduce the need to identify and remove shifts in temperature records that are unrelated to actual warming or cooling. NCDC officials told us these new networks can be used to construct a continuous temperature record with the USHCN once the new networks have a sufficient period of overlap with the USHCN to allow for a comparison of temperature trends. Siting Conditions at Weather-Monitoring Stations Played a Limited Role in Selections for the USHCN: The extent to which stations met specific NWS siting standards played only a limited role in the initial selection of stations for the USHCN in 1987 and when the makeup of the USHCN was revised in 1996 and 2009, according to NCDC officials. NCDC officials told us they considered other factors, such as geographic distribution and a long history of temperature records, to be more important to their ability to analyze long-term temperature trends than strict adherence to NWS's siting standards. For example, NCDC has included in the USHCN the Central Park station in New York City, which has a temperature record dating to 1876 and has had limited moves, even though current information on the station shows it is encircled by trees. NCDC officials said that, in an effort to consider some information on siting as part of the process of selecting stations, they obtained recommendations from state climatologists and others with detailed knowledge of the siting conditions at stations in their states. NCDC officials told us that another reason siting conditions played a limited role in their initial selection of USHCN stations in 1987 was that NCDC had limited information about siting conditions at the time. NCDC officials said they generally did not visit stations to examine siting conditions, except for a few stations near their headquarters in Asheville, North Carolina, because it was not feasible to do so with so many stations distributed nationwide. In addition, according to NCDC officials, when they first considered stations for inclusion in the USHCN, they had more limited electronic access to station histories and information about siting conditions than they do today. NCDC officials also said that the station histories they did have may not have included all relevant siting information, such as proximity to obstructions. According to NCDC officials, they may have kept some weather- monitoring stations that do not meet specific NWS siting standards out of the USHCN by generally excluding many sites in large urban areas. For example, weather-monitoring stations located in large urban areas may be too close to extensive paved surfaces or obstructions to meet specific NWS siting standards. Nevertheless, individual stations were excluded because they were located in a large urban area, not because they did or did not meet a specific NWS siting standard. Similarly, NCDC officials told us that many stations with the longest records were not selected because NCDC considered the temperature records for these stations to have been affected by the stations' location in or adjacent to large urban areas. Nevertheless, the officials told us, NCDC made exceptions and selected some stations in or near large urban areas. According to NCDC's 1987 report on its initial designation of the USHCN, 70 percent of the selected stations were located in areas with populations of less than 10,000 in the 1980 census, and 90 percent were located in areas with populations of less than 50,000. [Footnote 14] USHCN Stations Meet Siting Standards and Management Requirements to Varying Degrees: According to our survey of NWS weather forecast offices, close to half of USHCN stations do not adhere to one or more siting standards. Weather forecast offices cited a variety of factors that contributed to stations not adhering to siting standards, such as the use of temperature-measuring equipment that limits NWS's ability to locate stations so that they adhere to the standards. With regard to management requirements for USHCN stations, we found that the weather forecast offices generally but not always met requirements to conduct annual inspections and update station records. Close to Half of USHCN Stations Do Not Meet NWS Siting Standards: The survey responses we received from weather forecast offices that manage stations included in the USHCN indicate that about 42 percent of the active stations in 2010 did not adhere to one or more of the NWS siting standards for air temperature measurement.[Footnote 15] This percentage is slightly higher than the percentage not meeting the standards in the larger set of Cooperative Observer Program stations in the contiguous 48 states, of which the USHCN is a part. Specifically, according to our survey responses, about 37 percent of the active Cooperative Observer Program stations in 2010 did not adhere to one or more of the standards.[Footnote 16] The two standards most commonly cited by weather forecast offices as unmet by USHCN stations were distance to obstructions, such as buildings and trees, and distance to extensive concrete or paved surfaces (see figure 3). According to weather forecast offices' survey responses, only a small fraction of the stations did not adhere to the other siting standards, including that temperature-measuring instruments be mounted 4 to 6 feet off the ground. In particular, according to our survey responses, only five active USHCN stations (less than 1 percent) were located on a rooftop. Figure 3: Active USHCN Stations in 2010 That Did Not Adhere to One or More NWS Siting Standards: [Refer to PDF for image: vertical bar graph] Standard not met: Distance to obstructions; Percentage of active USHCN stations: 33%. Standard not met: Distance to concrete/paved surfaces; Percentage of active USHCN stations: 20%. Standard not met: Height above surface; Percentage of active USHCN stations: 3%. Standard not met: Not over level terrain; Percentage of active USHCN stations: 2%. Standard not met: Other; Percentage of active USHCN stations: 1%. Source: GAO analysis of survey results. Notes: The percentages sum to more than the total of 42 percent of stations that did not meet siting standards because some stations did not meet more than one standard. Other reasons for not adhering to siting standards include location on a rooftop and temporary relocation because of construction. [End of figure] We also visited a nonprobability sample of 8 weather forecast offices and 19 stations in the USHCN that are managed by these offices. During these visits, we observed stations that were located closer to obstructions or to extensive concrete or paved surfaces than specified in the siting standards, although the degree to which the stations did not adhere to the standards varied. For example, figure 4 shows 2 stations that did not meet the siting standards. One station was located too close to a building and trees at a wildlife preserve in an otherwise relatively undeveloped area, but the other station was located in a relatively urban area and surrounded by a parking lot, building, and street. Figure 4: Comparison of Conditions at Two USHCN Stations That Do Not Meet NWS Siting Standards: [Refer to PDF for image: 2 photographs] Photographs depict the following: 1) Too close to a building; 2) Surrounded by a parking lot, a building and a street. Source: GAO. Note: The temperature-measuring instruments in each photograph are the taller objects with slatted devices on top of thin poles. The shorter objects are rain gauges. [End of figure] Weather Forecast Offices Cited a Variety of Factors That Contribute to Stations Not Adhering to Siting Standards: The two factors most commonly cited by NWS weather forecast offices responding to our survey as contributing to USHCN stations not adhering to one or more of the siting standards were (1) NWS's preference for locating stations at sites that provide a high degree of station stability and data continuity, even if these sites do not adhere to standards, and (2) the use of temperature-measuring equipment that limits NWS's ability to locate stations so that they adhere to the standards (see figure 5). Figure 5: Primary Factors Contributing to USHCN Stations Not Adhering to NWS Siting Standards: [Refer to PDF for image: horizontal bar graph] Factor: A preference for station stability and data continuity; Percentage of active USHCN stations: 24%. Factor: The use of temperature equipment attached to a cable; Percentage of active USHCN stations: 20%. Factor: Other reasons; Percentage of active USHCN stations: 15%. Factor: Inability to relocate instruments without causing a discontinuity in temperature records; Percentage of active USHCN stations: 8%. Factor: Preference of observers for where temperature equipment is located on their properties; Percentage of active USHCN stations: 6%. Factor: Limited funding to relocate stations; Percentage of active USHCN stations: 4%. Source: GAO analysis of survey results. Note: Other factors contributing to USHCN stations not adhering to siting standards include the difficulty of recruiting volunteer observers at sites that meet the standards and observers' properties' being too small to locate temperature-measuring instruments as far from obstructions as specified in the standards. [End of figure] * Preference for station stability and data continuity. In our survey of weather forecast offices, the most commonly cited factor contributing to USHCN stations not meeting the siting standards was a preference for locating stations at sites that provide stability and continuity of data. For example, officials in the Tampa weather forecast office told us that one USHCN station that was located in a downtown area and did not meet siting standards has a temperature record that begins before 1895, the first year of data used in the USHCN. They said they could either keep this station open or close it, since there were no other options that met the standards either on the current observer's property or in the surrounding area. They chose to keep the station open because of its long temperature record. * Limitations due to temperature-measuring equipment. The use of temperature-measuring equipment that is connected by a cable to an indoor readout device can require installing equipment closer to buildings than specified in the standards, according to our survey. Weather forecast office staff must dig trenches for the cables, and paved surfaces such as sidewalks and driveways, as well as the cost of cable for trenching, can limit the length of trenches and consequently the ability to locate stations so that they adhere to the siting standards. According to data from NCDC, about three-quarters of stations in the USHCN use such equipment. NWS headquarters officials told us they hope to replace cabled temperature-measuring equipment with new wireless equipment that can more easily be located in accordance with siting standards. Specifically, the NWS headquarters office with overall responsibility for the Cooperative Observer Program has developed a draft plan for the program that envisions replacing current equipment with wireless equipment. The draft plan does not specify the number of stations where equipment will be replaced but rather calls for evaluating weather-monitoring stations to determine if they meet the siting standards, identifying candidate stations for installing wireless equipment or relocating them to meet siting standards, and identifying stations that are candidates for being closed. NWS officials said, however, that the agency has not yet approved the plan for implementation. We did not specifically ask about wireless equipment in our survey, but 35 weather forecast offices entered comments expressing support for replacing the temperature-measuring equipment currently used at weather-monitoring stations with wireless equipment. Twenty of the offices specifically cited the ability to improve station siting as the reason for making this change. Comments entered by weather forecast offices on our survey, as well as the draft plan, also cited greater ease of installation and maintenance as additional benefits of wireless equipment. For example, installing wireless equipment would not require digging a trench for a cable. Even if NWS approves the draft plan, the use of wireless equipment may not address all siting issues. First, according to NWS officials, commercially available wireless equipment has not yet been developed that meets NWS standards for temperature observations at a cost that is feasible for use at weather-monitoring stations nationwide. The NWS official in charge of monitoring the development of wireless equipment said, however, that such equipment would most likely be available within 5 years. Second, the use of wireless equipment, when available, would not allow NWS to improve siting at all stations that do not currently meet the standards. For example, the observers' properties at some of the stations we visited were too small to allow any temperature-measuring equipment to be placed far enough from buildings or other obstructions to meet the siting standards, regardless of whether the equipment was wireless or cabled. NWS officials acknowledged that the use of wireless equipment would improve station siting but not eliminate all stations that currently do not meeting siting standards. NWS officials also described a wide range of other factors contributing to stations not adhering to siting standards. These include: * the difficulty of recruiting new volunteer observers at sites that meet standards, particularly as the nation's population has become more mobile and thus less apt to serve as long-term observers; * properties that may be too small or have trees or other features that make it difficult to locate instruments as far from obstructions as the standards specify; * the reluctance of observers to allow equipment sited in a location on their property that would meet the standards; * changes to the observer's property (e.g., growth of trees) or urbanization of the surrounding area that can cause the stations to not meet standards; and: * natural geographic features in certain areas, such as heavily forested or mountainous terrain, that can hamper the ability to meet the standards. NWS Weather Forecast Offices Have Generally but Not Always Met the Requirement to Annually Inspect Stations and Periodically Update USHCN Station Records: Our review of files for USHCN stations at 8 weather forecast offices, as well as our survey results, show that the offices have generally but not always met the requirement to annually inspect stations to maintain temperature-measuring equipment and determine if changes have occurred requiring station records to be updated, such as changes to siting conditions. Our file reviews also show that the offices generally but not always met the requirement to periodically update station records, even if no changes had taken place at a station. According to NCDC and NWS officials, it is important to annually visit stations and keep station records up to date so that users of the stations' temperature records, such as NCDC, know the conditions under which the observations were recorded. Any information NCDC has about these conditions, according to agency officials, can be used in conjunction with its statistical methods to identify significant shifts in a station's temperature data that are unrelated to actual warming or cooling trends and to adjust the data to remove such shifts. * Annual inspections. The results of our survey indicate that in 2010, 102 of 114 weather forecast offices met the annual inspection requirement for stations in the USHCN.[Footnote 17] According to our survey results, 12 offices did not meet the requirement at a total of 35 stations. In reviewing files at the 8 weather forecast offices we visited, we also found instances where the annual inspection requirement was not met in 2008 and 2009. Specifically, the results of our file reviews show that 3 of the 8 offices did not meet the annual inspection requirement for five stations in 2008, and 1 office did not meet the requirement for one station in 2009.[Footnote 18] In contrast, for the stations where the requirement had been met, the weather forecast offices had frequently conducted multiple inspections during a year. For example, office staff may have visited a station multiple times to repair equipment, to temporarily relocate temperature-measuring instruments to allow for construction at the observer's property, or to meet the requirement for semiannual inspections of stations that also record precipitation. * Station record updates. Until 2005, NWS required that station records be updated at least once every 10 years. At that time, NWS changed the requirement to once every 5 years. In reviewing files at the 8 weather forecast offices we visited, we found that two of the 8 offices had consistently met the requirement to update station records within 5 years.[Footnote 19] In contrast, at two of the other offices, the time between updates for four stations was over 10 years. At the remaining four offices, the time between updates for one or more stations was over 5 years but less than 10 years. For example, one office did not update a record dated February 2002 until January 2011-- almost 9 years after the previous update. When the weather forecast offices updated records, the types of changes they documented included those that can cause shifts in temperature data unrelated to any actual temperature change, including replacement or relocation of temperature-measuring equipment, changes in time of observation, and descriptions of obstructions. Through our survey and visits to 8 weather forecast offices, weather forecast office officials identified a number of challenges to their ability to ensure that station records are updated and to carrying out other responsibilities for managing stations in the Cooperative Observer Program, including those in the USHCN. In our survey, the most frequently cited challenge was that weather forecast offices rely on staff assigned to manage the stations to also assist with other office responsibilities. Competing mission requirements at the offices was a closely related and often-cited challenge. For example, weather forecast offices operate 24 hours a day, and office officials explained that staff assigned to manage the stations may also be expected to work shifts, which limits the time they can visit the stations. Some weather forecast offices we visited told us that turnover and reductions in the number of staff assigned to the Cooperative Observer Program results in the loss of institutional knowledge needed to manage weather-monitoring stations. Weather forecast offices, particularly those with large areas to cover, also identified long driving distances to stations as a challenge. For example, officials at one office we visited told us that completing the required annual station visits requires driving 17,000 miles per year; that the round-trip drive to some stations takes longer than 10 hours, leaving limited time to maintain equipment or install equipment at new stations; and that during the winter, some stations are inaccessible. NWS Does Not Centrally Track Whether USHCN Stations Meet Siting Standards or Have a Policy for Addressing Stations That Do Not Meet the Standards: NWS does not use its information systems to centrally track whether USHCN stations adhere to siting standards or if weather forecast offices are meeting the requirement to update station records at least once every 5 years. NWS also does not have an agencywide policy on what actions to take at stations that do not adhere to siting standards, which creates the potential for inconsistency in how weather forecast offices address such stations. The lack of centralized electronic tracking of performance information for the USHCN and the lack of an agencywide policy on the actions to take at stations that do not meet siting standards limit NWS's ability to manage the USHCN in accordance with performance management guidelines and federal internal control standards. NWS Does Not Have Centrally Available Information to Assess USHCN Performance: NWS's siting standards for weather-monitoring stations and the requirement that station records be updated at least once every 5 years in effect establish goals for each weather forecast office to meet. NWS does not, however, centrally capture data on the extent to which stations in the Cooperative Observer Program and the USHCN meet its siting standards and station record update requirement. NWS has an information system it uses to help manage weather-monitoring stations, but the system has several limitations. The information system allows the agency to record basic identifying information about the stations and the conditions under which volunteer observers record weather observations, including some information about siting conditions and the dates of station record updates. According to NWS officials, however, the agency did not design the system to centrally track adherence to siting standards or the requirement to update station records at least once every 5 years. As a result, NWS is limited in its ability to use its information system to track USHCN performance information. The information system NWS uses to track information related to Cooperative Observer Program stations, including those selected for the USHCN, has the following specific limitations: * Limited information on adherence to station siting standards. According to NWS headquarters officials, they cannot query their information system to identify the specific weather-monitoring stations that do not meet siting standards or the total number of stations that do not meet standards. For example, in 2009, NCDC requested that NWS verify siting conditions at USHCN stations for a study evaluating the effects of siting conditions on temperature trends.[Footnote 20] NWS headquarters did not have a way to easily gather this information and instead had to direct its regional offices to have weather forecast office staff verify siting conditions at the stations. According to NWS officials, weather forecast office staff did not make special visits to the stations to gather the information requested by NCDC, as these stations might not have been on their immediate visit schedule, but reviewed files instead. * Incomplete information on siting conditions. NWS guidance for using the information system directs weather forecast offices to enter descriptions of obstructions at weather-monitoring stations. We found that in some cases, however, such descriptions may not accurately reflect whether temperature-measuring equipment meets the siting standard related to obstructions. This discrepancy can arise because NWS guidance directs weather forecast offices to describe obstructions in relation to a station's rain gauge. We found that obstructions to temperature measuring-equipment can differ from those to the rain gauge, depending where the two instruments are located. For example, the record for one USHCN station we visited did not include any obstructions because the rain gauge was on a rooftop where there were no obstructions. The station's temperature-measuring equipment, however, was at ground level and surrounded by buildings on three sides and, as a result, did not meet NWS's standards for siting temperature sensors. (The station's temperature-measuring equipment can be seen in figure 6.) In addition, NWS guidance for using the information system does not direct weather forecast offices to enter descriptions of other conditions that might indicate whether specific siting standards are being met, such as proximity to extensive concrete or paved surfaces. Figure 6: A USHCN Station with Obstructions to Temperature-Measuring Equipment but No Obstructions Shown on the Station Record: [Refer to PDF for image: photograph with inset enlargement of measuring equipment] Source: GAO. [End of figure] * Limited ability to track whether the requirement to update station records is being met. The NWS information system is designed to allow weather forecast office staff to enter the date each station record is updated and to store previous versions of station records. NWS officials stated, however, that the information system is not set up to centrally track the performance of weather forecast offices in updating station records within the required 5-year time frame. In addition, the system is not set up to allow NWS to notify weather forecast offices when station records are nearing the 5-year mark and need to be updated. NWS regional offices and weather forecast offices are instead responsible for tracking the status of updates. For example, an official at one office we visited told us that he sometimes forgets to update station records within the 5-year time frame but that the NWS regional headquarters keeps track of the requirement. * Inconsistent identification of stations included in the USHCN. The NWS information system includes information about all stations in the Cooperative Observer Program, including those designated by NCDC as part of USHCN, but NWS does not consistently use the system to identify USHCN stations. Specifically, station records allow weather forecast offices to indicate whether stations are part of the USHCN, but offices had done so in only some of the station records we reviewed. As a result, NWS headquarters officials cannot use their information system to determine which stations NCDC has designated as part of the USHCN. Officials at some of the weather forecast offices we visited were also unsure which of the stations they manage had been designated as part of the USHCN. According to NCDC officials, it is important that weather forecast offices have the ability to determine which stations belong to the USHCN so that they can set appropriate priorities for the maintenance, repair, and replacement of temperature- measuring equipment at these stations. Our work related to the Government Performance and Results Act of 1993 [Footnote 21] and the experience of leading organizations have shown the importance of developing program performance goals that identify desired results of program activities and reliable information that can be used to assess results.[Footnote 22] NWS headquarters officials we spoke with acknowledged the need to centrally track performance information related to the management of Cooperative Observer Program stations, which includes those selected for the USHCN. The officials said they only recently began tracking the requirement that weather forecast offices inspect stations at least once annually. According to these officials, they selected annual station inspections as a performance indicator because the inspection requirement is easy to track using the current information system for managing the stations. NWS provided us with summary data on inspections for all stations in the Cooperative Observer Program, including USHCN stations. The data show that the percentage of stations for which weather forecast offices met the annual inspection requirement increased from 70 percent in 2005 to 80 percent in 2010.[Footnote 23] NWS headquarters officials also told us that the agency has begun upgrading its current information system and that they are considering options to notify weather forecast offices when updates of station records are overdue and to better track adherence to siting conditions. The officials said they hope to complete the upgrade by the end of fiscal year 2013, depending on the availability of funding. Options being considered for tracking siting conditions include photographs of stations and the use of a rating scale to summarize the extent to which stations adhere to siting standards, similar to the rating scale created for the newer networks developed specifically for climate monitoring (see appendix II for further details). According to our survey results, 63 percent of weather forecast offices believe that the use of photographs in the NWS information system would be either very helpful or extremely helpful in evaluating stations' adherence to siting standards. In addition, 52 percent of the offices responded that the option to check a box in a station's electronic record to indicate that it does not adhere to the standards would be either very helpful or extremely helpful. Some offices also suggested other tools they would consider helpful in evaluating siting conditions at stations, such as the use of commercially available satellite imagery and maps. NWS Does Not Have an Agencywide Policy to Address Stations That Do Not Adhere to Siting Standards: NWS does not have an agencywide policy for stations not adhering to siting standards that clarifies for staff in weather forecast offices whether the stations should be closed, relocated, or maintained in their present condition to preserve the continuity of their temperature records. Standards for internal control in the federal government call for federal agencies to document their policies and procedures to help managers achieve desired results.[Footnote 24] Without an agencywide policy, weather forecast offices do not have a basis for making consistent decisions about what actions to take at USHCN stations that do not adhere to siting standards. NWS headquarters officials we spoke with acknowledged that they had not developed an agencywide policy on the actions, if any, that weather forecast offices should take to address stations that do not adhere to siting standards. They said they recognized the need to develop an agencywide policy and that, in the absence of such a policy, decisions on how to address stations that do not meet siting standards are up to individual weather forecast offices. For example, they said weather forecast offices might consult with NCDC or state climatologists when deciding whether to close stations that do not meet siting standards, but that such outreach is not required. In the absence of an agencywide policy, the NWS western regional office directed weather forecast offices in the region to stop submitting data to NCDC from stations with "egregious" siting conditions in a format that would allow NCDC to use these data when analyzing long-term temperature trends. The meaning of "egregious" was not defined and was left to each weather forecast office to interpret. According to NWS officials, none of the other three regional offices in the contiguous United States have developed a similar policy. According to the western regional manager responsible for the Cooperative Observer Program, the region's policy affected about a dozen stations, which had not been designated as part of the USHCN. Through our visits to weather forecast offices, however, we found that, even in the western region, the offices did not consistently implement the region's policy. In particular, an official at one of the two offices we visited in the region told us he did not follow the policy because doing so would have affected the majority of stations in his state. NCDC officials told us that, as NWS develops a policy regarding how to address stations in the Cooperative Observer Program (including those designated as part of the USHCN) that do not meet siting standards, it should consider how NCDC uses the temperature data from the stations. Because the data from USHCN stations are used to identify long-term climate trends and the stations were thus selected in part on the basis of the stations' stability of measurement conditions and continuity of data, NCDC officials said they would caution NWS against relocating or closing stations that do not meet siting standards. NCDC officials said they would consider closing a station only in certain situations, such as an observer not following NWS guidelines when recording weather observations. Conclusions: Given the importance of data from the USHCN in monitoring and in formulating public policy related to climate change, it is important that the public and policymakers have confidence that the network is being managed effectively. NWS has developed station siting standards and management requirements for the USHCN, and performance management guidelines dictate that NWS should gather data on the extent to which these standards are being met. But NWS's information system does not centrally capture such information. As a result, the agency cannot easily measure the USHCN's performance against its siting standards and management requirements. Without more complete data on siting conditions, including when siting conditions change, it is difficult for agency management to assess the extent to which the stations meet its siting standards. Similarly, NWS does not have easily accessible data on when station records were last updated for monitoring whether the records are being updated at least once every 5 years as the agency requires. In addition, although federal internal control standards call for agencies to develop policies to maintain control over program activities, NWS has not established agencywide policy for what to do when, over years and decades, stations no longer adhere to its siting standards because conditions have changed. In the absence of such policy, it is not clear to weather forecast office officials whether stations that do not adhere to siting standards should remain open because data continuity is important for analyzing long-term climate trends, or whether the stations should be moved or closed. As a result, without a policy with actions for all offices to follow, weather forecast offices may be taking different approaches to address stations that do not meet siting standards. Recommendations for Executive Action: To improve NWS's ability to manage the USHCN in accordance with performance management guidelines and federal internal control standards, as well as to strengthen congressional and public confidence in the data the network provides, we recommend that the Secretary of Commerce direct the Administrator of NOAA to take the following two actions: * Enhance NWS's information system to centrally capture information that would be useful in managing stations in the USHCN, including (1) more complete data on siting conditions (including when siting conditions change), which would allow the agency to assess the extent to which the stations meet its siting standards, and (2) existing data on when station records were last updated to monitor whether the records are being updated at least once every 5 years as NWS requires. * Develop an NWS agencywide policy, in consultation with NCDC, on the actions weather forecast offices should take to address stations that do not meet siting standards. Agency Comments and Our Evaluation: We provided a copy of our draft report to the Department of Commerce for review and comment. In written comments from the department, NOAA agreed that it can improve its ability to manage the USHCN in accordance with performance management guidelines and federal internal control standards. NOAA also agreed with our two recommendations. Regarding our first recommendation, NOAA stated that NWS has begun the planning process to upgrade the existing information system that captures data for managing Cooperative Observer Program stations, including those that are a part of the USHCN. According to NOAA, the upgrade will include the ability to capture more complete data on siting conditions and to determine if a station's record has been updated in the last 5 years. Regarding the second recommendation, NOAA said that NWS will work with NCDC to develop a policy to assist weather forecast offices in taking action on stations that do not meet siting standards. NOAA also stated that it understood that, given the scope of our review, we did not assess the effect of stations not meeting siting standards on the reliability of the agency's analysis of temperature trends. Nevertheless, NOAA added that it was important for our findings to include a discussion of the published peer-reviewed studies that have explicitly examined the USHCN's data quality and its effects on the reliability of NOAA's temperature trend data. We did not include such a discussion in our report because this issue was outside the scope of our work. We did, however, reproduce NOAA's list of relevant studies on this topic together with its comments. NOAA also provided technical comments, which we incorporated into the report as appropriate. NOAA's comments are reproduced in appendix III. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 15 days from the report date. At that time, we will send copies to the appropriate congressional committees, the Secretary of Commerce, and other interested parties. In addition, this report will be available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or mittala@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix IV. Sincerely yours, Signed by: Anu K. Mittal: Director, Natural Resources and Environment: [End of section] Appendix I: Scope and Methodology: To determine how the National Oceanic and Atmospheric Administration (NOAA) selected stations for the U.S. Historical Climatology Network (USHCN) from the larger set of existing stations in the Cooperative Observer Program network, we reviewed documents from NOAA's National Climatic Data Center (NCDC) describing the selection process, interviewed NCDC officials, and analyzed NCDC data on stations in the USHCN. Specifically, we reviewed documents written by NCDC officials to identify the factors the agency considered important for monitoring long-term temperature trends, and we interviewed NCDC officials regarding these factors, as well as how they applied them in the selection process. We also obtained data from NCDC on the geographic distribution of USHCN stations across the contiguous 48 states and the length and completeness of the stations' temperature records. The data we obtained and analyzed came from NCDC's version 2 of the USHCN. [Footnote 25] We assessed the reliability of the data by electronically testing them and comparing selected samples to data from other NOAA sources to check for obvious errors in accuracy and completeness. We also reviewed information about the data and the systems used by NCDC to produce the data, interviewed NCDC officials knowledgeable about the data, and worked with officials to clarify inconsistencies before using the data in our analyses. We determined that the data were sufficiently reliable for reporting on the extent to which USHCN stations met the factors NCDC considered important in selecting the stations. To examine the extent to which USHCN stations meet National Weather Service (NWS) siting standards and management requirements for weather- monitoring stations, we developed and administered a survey of meteorologists-in-charge at the 116 NWS weather forecast offices responsible for managing stations in the network.[Footnote 26] Our questionnaire included questions about adherence to siting standards, reasons for stations not adhering to the standards, and general management challenges. The survey was Web based and accessible through a secure server. On February 7, 2011, we sent an e-mail notification to the 116 meteorologists-in-charge describing the survey and notifying them that it would be activated on the Internet shortly. On February 9, 2011, we formally activated the survey and sent another e- mail containing a link to the survey along with each respondent's unique username and password. We sent follow-up e-mail messages on February 16, 2011, and February 24, 2011, to those who had not yet responded. Then, starting on March 11, 2011, we contacted the remaining nonrespondents by telephone or e-mail. The questionnaire was available online until March 23, 2011. By that date, the surveys were completed by all 116 weather forecast offices, for a response rate of 100 percent. Because our survey covered all weather forecast offices in the contiguous 48 states, not a sample of them, it was not subject to sampling error. Surveys are, however, subject to nonsampling errors. For example, how a particular question is interpreted, sources of information available to respondents, and how the data are entered in a database or are analyzed can introduce unwanted variability into survey results. We took steps in developing the survey, collecting the data, and analyzing them to minimize such nonsampling errors. For example, GAO survey specialists designed the questionnaire in collaboration with GAO staff who had subject-matter expertise. In addition, we conducted four pretests of the draft questionnaire to ensure that the questions were clear and unambiguous, terminology was used correctly, the questionnaire did not place an undue burden on agency officials, the information could feasibly be obtained, and the survey was comprehensive and unbiased. We conducted each of the four pretests over the telephone with one or more NWS officials from each of the four NWS regions in the contiguous United States. On the basis of the feedback we received, we made changes to the content and format of the survey after each of the four pretests. When we analyzed the data, an independent GAO analyst checked all computer programs. Since this was a Web-based survey, respondents entered their answers directly into the electronic survey, eliminating the need to key data into a database, thus minimizing data entry errors. Our survey results are also subject to errors made by weather forecast office staff regarding the number of stations they reported as not adhering to siting standards. For example, one weather forecast office indicated in its survey response that one USHCN station in its area was on a rooftop, but office staff later told us that the survey response was wrong and that none of its USHCN stations is located on a rooftop. In addition, the number of stations reported by weather forecast offices as not adhering to siting standards is subject to the staff's interpretation of NWS's siting standards. For example, it is a matter of interpretation and judgment by NWS staff whether objects surrounding a station, such as trees or structures, are considered to be obstructions and thus whether a station is considered to meet or not meet the siting standards. The response to our survey from one weather forecast office we visited indicated that the temperature- measuring equipment at only one of its USHCN stations was closer to an obstruction than specified in the siting standards. We observed obstructions at all three stations we visited, however, and the records for these stations also listed obstructions. In calculating the percentages of the total number of active stations that met the specific criteria we asked about in our survey questions, we did not use responses from weather forecast offices that provided incomplete or inconsistent information. Depending on the percentage being calculated, we did not use responses from at most 6 of the 116 offices. For example, we did not use responses from 3 offices when calculating the percentage of active USHCN stations that did not adhere to one or more of the siting standards, and we did not use responses from 5 offices when calculating the percentage of active Cooperative Observer Program stations not adhering to one or more of the standards. To examine in greater depth the extent to which USHCN stations meet siting standards and management requirements for weather-monitoring stations, we visited a nonprobability sample of 8 NWS weather forecast offices. To ensure geographic distribution in the weather forecast offices we visited, we selected two offices in each of the four NWS regions in the contiguous United States, and we selected the specific offices we visited to ensure a range of sizes in terms of the offices' forecast areas. We also selected offices with differing structures for supervising staff assigned to manage the Cooperative Observer Program and with a range of other programs the offices are responsible for, such as marine forecasts. To examine adherence to siting standards for weather-monitoring stations, we observed siting conditions at a nonprobability sample of 19 USHCN stations. We selected stations to visit to ensure variety in the type of temperature-measuring equipment used at the stations and other factors that could affect siting conditions. During the station visits, we developed and used a checklist that tracked how well the conditions of the site met what was recorded in the stations' records. In addition, we reviewed files on USHCN stations at each of the weather forecast offices we visited. We reviewed records and annual inspection reports from a total of 81 USHCN stations. We entered information from the records and inspection reports into a database to capture information on the extent to which weather forecast offices adhered to the requirements to update station records and conduct annual inspections, among other things. Because we used a nonprobability sample to select weather forecast offices and USHCN stations to visit, the information we obtained from these visits cannot be generalized to other weather forecast offices or USHCN stations. The visits instead provided us with information on the perspectives of various participants in the weather forecast offices about managing weather-monitoring stations and examples of station siting conditions. To gather additional information on the extent to which USHCN stations meet NWS siting standards for weather-monitoring stations, we also reviewed academic literature that addressed issues and concerns related to siting of weather-monitoring stations, including those in the USHCN. We also reviewed NWS's policy directives related to station siting and interviewed officials in NWS headquarters, regional offices, and weather forecast offices who were responsible for managing the Cooperative Observer Program. In addition, we interviewed the person at the NWS training center responsible for training NWS staff on management and operation of the Cooperative Observer Program and individuals who have raised concerns about the extent to which stations in the USHCN are meeting siting standards. To evaluate the extent to which NWS tracks USHCN stations' adherence to siting standards and management requirements and has established a policy for addressing stations that do not adhere to siting standards, we took several actions. In particular, we evaluated the types of data that are captured in NWS's information system for managing weather- monitoring stations--the Cooperative Station Service Accountability system. We also interviewed NWS officials responsible for managing the USHCN, as well as NCDC officials; reviewed NWS policy directives, briefings, and memorandums related to managing the network; and examined data on the extent to which NWS conducted the required annual inspections of weather-monitoring stations. To determine the extent to which NWS has established a policy to address stations that do not adhere to siting standards, we reviewed NWS documents, such as agency directives, memos, briefings on the future of the Cooperative Observer Program, and an executive summary for a draft strategic plan for the program. In addition, we interviewed NCDC officials and NWS officials from regional, headquarters, and weather forecast offices and from the Cooperative Observer Program training center. [End of section] Appendix II: Other National Climate Networks: Since 2001, NOAA has supported establishment of two new networks of climate monitoring stations. The first to be established, the U.S. Climate Reference Network, is intended to detect indications of climate change at a national scale. This network consists of 114 stations in the contiguous 48 states, and NOAA has plans to expand the network to include stations in Alaska. The purpose of the second new network, the U.S. Regional Climate Reference Network, is to detect indications of climate change at a regional rather than national scale. As of July 2011, this network consisted of 63 stations in the southwestern United States, and NOAA hopes to complete the installation of stations for the network across the contiguous 48 states by about 2020, depending on the availability of funding. According to NOAA, once both networks are fully established, about 538 locations in the contiguous United States will have either a U.S. Climate Reference Network station or a U.S. Regional Climate Reference Network station. The station-siting standards for the two new networks have similarities to the siting standards established for the Cooperative Observer Program and that also apply to the USHCN, such as installation over level terrain and not on rooftops. The new networks also have differences, including instances where the siting standards for the new networks are more stringent. For example, the standards for the U.S. Climate Reference Network call for stations to be located farther from concrete or paved surfaces than specified in the standards for stations in the USHCN. The differences reflect the fact that, whereas NCDC designated stations for the USHCN from an existing network of NWS weather-monitoring stations, NOAA has specifically located and designed stations in the newer networks for monitoring the nation's climate. For example, the use of automated equipment, as well as solar power at many stations, allows for greater flexibility in locating stations in comparison with NWS weather-monitoring stations that rely on volunteer observers and equipment connected by a cable to an indoor readout device. Similarly, to the extent possible, NOAA has placed a priority on locating stations for the new networks on public lands, such as national and state parks. According to NOAA, in comparison with the properties of volunteer observers, such locations have a higher probability of continuing in their present condition without major changes for long periods of time (50 to 100 years). Figure 7 depicts a station in the U.S. Climate Reference Network. Figure 7: U.S. Climate Reference Network Station: [Refer to PDF for image: photograph] Source: NCDC. [End of figure] NOAA has also established a process to evaluate and select potential sites for stations in the new networks. For example, selecting stations for the U.S. Regional Climate Reference Network includes field and desk surveys, which involve collecting information about the site's condition from photographs and other sources; evaluation of the surveys by a site selection panel; a vote among panel members to decide among candidate locations; and final approval or disapproval of panel recommendations by a site selection lead. See table 1 for differences in the siting standards applied to the USHCN and the new climate networks. Table 1: Differences in Selected Siting Standards of National Climate Networks: Siting standard: Site-ranking system; Cooperative Observer Program, including stations in the USHCN: No ranking system; New networks: Five classes, with 1 being the best sited and 5 the worst; the extent to which stations meet the standard for distance from artificial heating sources is one of the primary criteria for temperature measurement. NCDC aims to select class 1 station locations. Siting standard: Distance from artificial heating sources; Cooperative Observer Program, including stations in the USHCN: 100 feet from any extensive concrete or paved surfaces; New networks: For class 1, more than 300 meters (or almost 1,000 feet) from artificial heating or reflecting surfaces, such as buildings, concrete surfaces, and parking lots. Siting standard: Distance from obstructions; Cooperative Observer Program, including stations in the USHCN: No closer than 4 times the estimated height of any nearby building, tree, fence, or similar obstruction; New networks: For class 1, no closer than about 10 times the height of surrounding vegetation. Source: GAO analysis of NOAA's siting standards. [End of table] [End of section] Appendix III: Comments from the Department of Commerce: United States Department of Commerce: The Secretary of Commerce: Washington, D.C. 20230: August 22, 2011: Ms. Anu K, Mittal: Director: Natural Resources and Environment: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Ms. Mittal: Thank you for the opportunity to review and comment on the Government Accountability Office's draft report entitled "Climate Monitoring: NOAA Can Improve Management of the U.S. Historical Climatology Network" (GAO-11-800). On behalf of the Department of Commerce, I have enclosed the National Oceanic and Atmospheric Administration's programmatic comments to the draft report. Sincerely, Signed by: Rebecca M. Blank: Acting Secretary of Commerce: Enclosure: [End of letter] Department of Commerce: National Oceanic and Atmospheric Administration: Comments to the Draft GAO Report Entitled "Climate Monitoring: NOAA Can Improve Management of U.S. Historical Climatology Network" (GAO-11- 800, August 2011): General Comments: The Department of Commerce's National Oceanic and Atmospheric Administration (NOAA) appreciates the opportunity to review the Government Accountability Office's (GAO) draft report on the U.S. Historical Climatology Network (USHCN) and agrees that NOAA can improve its ability to manage the USHCN in accordance with performance management guidelines and federal internal control standards. NOAA notes the statement in this report that the GAO "did not include an assessment into the effect of stations' not meeting siting standards on the reliability of NOAA's analysis of temperature trends." NOAA understands that these scientific findings were not the subject of this GAO review. Nevertheless, it is important context for GAO's findings to include a discussion of the published peer-reviewed studies that have explicitly examined the network's data quality and its effects on the reliability of NOAA's temperature trend findings Such studies have confirmed that corrections applied to USHCN data based on peer-reviewed algorithms can largely account for the impact of instrument and siting changes. In fact, adjusted USHCN temperatures are well aligned with recent measurements from instruments whose exposure characteristics meet the highest standards for climate monitoring. For GAO's reference, a list of relevant studies is included as an appendix to this response. NOAA Response to GAO Recommendations: The draft GAO report states, "To improve NWS's ability to manage the USHCN in accordance with performance management guidelines and federal internal control standards, as well as strengthen congressional and public confidence in the data the network provides, we recommend that the Secretary of Commerce direct the Administrator of NOAA to take the following two actions:" Recommendation is "Enhance NWS's information system to centrally capture information that would be useful in managing stations in the USHCN, including: * more complete data on siting conditions, including when siting conditions change, which would allow the agency to assess the extent to which the stations meet its siting standards, and; * existing data on when station records were last updated to monitor whether the records are being updated at least once every 5 years as NWS requires." NOAA Response: NOAA concurs with this recommendation. The National Weather Service (NWS) is currently in the planning stages to upgrade the existing information system that is used for Cooperative Observer Program (COOP), and therefore USHCN, metadata. This upgraded system will include the ability to capture more complete data on siting conditions and will record whether a station has been updated in the last 5 years. This system is planned to be operational by the end of fiscal year 2013, depending on the availability of funding. In order to better identify a COOP station as belonging to the USHCN dataset, the National Climatic Data Center (NCDC) will coordinate this designation with the NWS. In turn, the NWS will mark this designation in the COOP network database. Recommendation 2: "Develop an NWS agencywide policy, in consultation with NCDC, on actions weather forecast offices should take to address stations that do not meet siting standards." NOAA Response: NOAA concurs with this recommendation. The NWS, working with NCDC, will develop a policy to assist Weather Forecast Offices to take action regarding COOP stations that do not meet siting standards. Without a strong partnership between the nowcasting; forecasting, and climate monitoring communities, where all parties agree to the policy, the long-term reliability of the USHCN data set and COOP networks for climate purposes may not be fully realized. Appendix A: Peer-reviewed Studies Regarding Artificial Changes in Temperature Associated with Changes in Station Environment and Observing Practices: Hubbard, K. G., and X. Lin, 2006: Reexamination of Instrument Change Effects in the U.S. Historical Climatology Network. Geophysical Research Letters, 33, L15710, doi:10.1029/2006GL027069. Karl, T.R., C.N. Williams, Jr., P.J. Young, and W.M. Wendland, 1986: A Model to Estimate the Time of Observation Bias Associated with Monthly Mean Maximum, Minimum and Mean Temperatures for the United States. Journal of Climate And Applied Meteorology, 25 (2), 145-160. Menne, M. J., C. N. Williams Jr., and M. A. Palecki (2010), On the Reliability of the U.S. Surface Temperature Record, Journal of Geophysical Research, 115,1311108, doi:10.1029/2009.1D013094. Menne, KJ., C.N. Williams, and R.S. Vose, 2009: The United States Historical Climatology Network Monthly Temperature Data - Version 2. Bulletin of the American Meteorological Society, 90(7), 993-1107. Peterson, Thomas C., 2006: Examination of Potential Biases in Air TeMperature Caused by Poor Station Locations. Bulletin of the American Meteorological Society, 87, 1073-1089. Pielke, R. A., Sr., and Coauthors, 2007a: Documentation of Uncertainties and Biases Associated with Surface Temperature Measurement Sites for Climate Chance Assessment. Bulletin of the American Meteorological Society, 88, 913-928. Vose, R.S., C.N. Williams, T.C. Peterson, T.R. Karl, and D.R. Easterling, 2003: An Evaluation of the Time of Observation Bias Adjustment in the US Historical Climatology Network. Geophysical Research Letters, 30 (20), 2046, clim3-l--3-4 doi:10.1029/2003GL018111, 2003 (15 October 2003). [End of section] Appendix IV: GAO Contact and Staff Acknowledgments: GAO Contact: Anu K. Mittal, (202) 512-3841 or mittala@gao.gov: Staff Acknowledgments: In addition to the contact named above, Stephen D. Secrist (Assistant Director), Richard Bulman, William Carrigg, Joanna Chan, Ellen W. Chu, Joseph Cook, Alysia Davis, N'Kenge Gibson, Stuart Kaufman, Cheryl Peterson, Anne Rhodes-Kline, and Jerome Sandau made key contributions to this report. [End of section] Footnotes: [1] NOAA has 122 weather forecast offices, including 116 offices in the contiguous 48 states. Because NOAA established the USHCN to analyze trends in the contiguous 48 states, no USHCN stations are located in the 6 weather forecast offices in the Alaska and Pacific regions and in San Juan, Puerto Rico, although there are weather- monitoring stations in these areas. [2] Throughout this report, we refer to stations in the Cooperative Observer Program as weather-monitoring stations. [3] Before 1890, weather-monitoring stations were managed by the Army Signal Corps. [4] NCDC, which is part of NOAA's National Environmental Satellite, Data, and Information Service, is the nation's archive of weather data and uses these data to assess and monitor climate variation and change. [5] C. Fenimore et al., "United States," Bulletin of the American Meteorological Society, vol. 92, S175-S179 (2011). [6] NOAA has also developed siting standards for its new networks specifically designed for climate monitoring--the U.S. Climate Reference Network and the U.S. Regional Climate Reference Network. The standards for the new networks have similarities to those for the Cooperative Observer Program but also differences, including instances where the standards for the new networks are more stringent. Appendix II summarizes these similarities and differences. [7] World Meteorological Organization, Guide to Meteorological Instruments and Methods of Observation, 7th ed., WMO-No. 8 (Geneva, Switzerland: 2008). [8] Called the Cooperative Station Service Accountability system, NWS's information system is a national database containing information on all Cooperative Observer Program and USHCN stations. [9] Each of the 48 contiguous states is divided into as many as 10 climate divisions (areas that have a similar climate). For example, Nevada is divided into four climate divisions, and New York has the maximum of 10 climate divisions. [10] Active stations exclude stations that have closed, often due to the lack of an observer to record weather observations, as well as inactive stations that weather forecast offices may expect to return to active status or ultimately decide to close if no observer is available. NCDC does not exclude data from inactive and closed USHCN stations when analyzing temperature trends; instead, it uses the data available from when the stations were active. [11] T. R. Karl et al., United States Historical Climatology Network (HCN) Serial Temperature and Precipitation Data, ORNL/CDIAC-30, NDP- 019/R1 (Oak Ridge, Tenn.: Carbon Dioxide Information Analysis Center, Oak Ridge National Laboratory, U.S. Department of Energy, 1990). [12] NWS guidelines state that weather forecast offices should seek out weather station observers who are likely to record data over a long period and at around the same time each day and that the offices should avoid moving instruments at stations with a long record of temperature measurements. The guidelines also establish procedures for managing changes in measurement conditions when such changes occur. [13] We did not evaluate the methods used by NCDC to identify and remove shifts in temperature data caused by changes in measurement conditions, as this was outside the scope of our review. [14] F. T. Quinlan et al., United States Historical Climatology Network (HCN) Serial Temperature and Precipitation Data, NDP-019 (Oak Ridge, Tenn.: Carbon Dioxide Information Analysis Center, Oak Ridge National Laboratory, U.S. Department of Energy, 1987). [15] This percentage is based on survey responses we received from 113 of the 116 weather forecast offices. We did not use responses from the 3 offices that provided incomplete or inconsistent information needed to calculate this percentage. See appendix I for further information on the steps we took to verify the accuracy of survey responses. [16] This percentage is based on survey responses from 111 of the 116 weather forecast offices and does not include stations that do not measure air temperature. We did not use responses from the 5 offices that provided incomplete or inconsistent information needed to calculate this percentage. [17] We did not include survey responses from two weather forecast offices that provided inconsistent information. [18] The results of our file review are based on a nonprobability sample of weather forecast offices and cannot be generalized to other offices. [19] The results of our file review are based on a nonprobability sample of weather forecast offices and cannot be generalized to other offices. [20] M. Menne et al., "On the Reliability of the U.S. Surface Temperature Record," Journal of Geophysical Research, vol. 115, D11108 (2010). [21] Pub. L No. 103-62, § 3, 107 Stat. 285 (1993), codified at 5 U.S.C. § 306. The act was recently amended by the GPRA Modernization Act of 2010, Pub. L. No. 111-352, 124 Stat. 3866 (2011). [22] For example, see GAO, The Results Act: An Evaluator's Guide to Assessing Agency Annual Performance Plans, [hyperlink, http://www.gao.gov/products/GAO/GGD-10.1.20] (Washington, D.C.: April 1998). [23] NWS officials cautioned that there is potential for differing interpretations among weather forecast offices regarding the requirement for annual inspections. In particular, the requirement can be strictly interpreted as 365 days from the date of the last visit or more generally interpreted as at least one time during the course of a year. [24] GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999). [25] NCDC released version 2 of USHCN in 2009 to account for station closures and update the list of stations selected for inclusion in the network. NCDC also revised its statistical methods for use in version 2. According to NCDC officials, version 2.5 of USHCN replaced version 2 as of June 2011. The officials told us that version 2.5 is the U.S. component of NCDC's global network of weather stations. [26] NWS has 122 weather forecast offices, including 116 offices in the contiguous 48 states. Because NCDC established the USHCN for analysis of trends in the contiguous 48 states, no USHCN stations are located in the 6 weather forecast offices in NWS's Alaska and Pacific regions and in San Juan, Puerto Rico, although there are Cooperative Observer Program stations in these areas. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: