This is the accessible text file for GAO report number GAO-10-947 
entitled 'Environmental Protection Agency: EPA Needs to Complete a 
Strategy for Its Library Network to Meet Users' Needs' which was 
released on November 1, 2010. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Requesters: 

United States Government Accountability Office:
GAO: 

September 2010: 

Environmental Protection Agency: 

EPA Needs to Complete a Strategy for Its Library Network to Meet 
Users' Needs: 

GAO-10-947: 

GAO Highlights: 

Highlights of GAO-10-947, a report to congressional requesters. 

Why GAO Did This Study: 

The Environmental Protection Agency’s (EPA) library network provides 
agency staff and the public with access to environmental information. 
A 2006 attempt by EPA to reorganize its network by consolidating 
libraries and making more materials and services available online 
caused concern among users, and in 2007, EPA put a moratorium on its 
reorganization plans. Congress requested that GAO report on the 
reorganization and has again requested a follow-up on these issues.
Accordingly, GAO reviewed (1) the status of EPA’s overall strategy for 
its library network, (2) the status of EPA’s plan to reopen the 
libraries it closed and other actions planned or taken, (3) EPA’s 
efforts to digitize printed documents to make them electronically 
available, and (4) EPA’s efforts to communicate with staff and other 
stakeholders about its library network. GAO reviewed regulations and 
agency funding and inventory documents and interviewed EPA staff and 
contractors, as well as independent library professionals. GAO also 
assessed the reliability of EPA’s data on library holdings and from 
EPA’s staff survey on library use and needs. 

What GAO Found: 

Although EPA has taken a number of steps to meet the needs of library 
users, it has not completed a plan identifying an overall strategy for 
its library network, with implementation goals and a timeline of what 
it intends to accomplish. Scheduled for completion in 2008, the 
strategic plan was to provide EPA staff and the public a detailed view 
of EPA’s library operations and future direction. The draft outline of 
the strategic plan, however, is largely a placeholder list of current 
and planned EPA activities. For example, while it emphasizes the 
central role to be played by electronic library resources, the draft 
outline does not contain goals or a timeline for completing an 
inventory of holdings or digitizing those holdings. The draft outline 
also does not set out details of how funding decisions are to be made. 
Given the current economic environment, without a completed strategic 
plan, including a detailed strategy for acquiring, deploying, and 
managing funding, EPA may find itself hard-pressed to ensure that the 
network can meet its users’ needs. 

The agency has reopened libraries closed during reorganization, 
although about half the network’s 10 regional libraries are operating 
with reduced hours. EPA has also developed standards for the regional 
and headquarters libraries’ use of space, on-site collections, 
staffing, and services. The agency has also hired a national library 
program manager to carry out day-to-day activities and bring focus and 
cohesion to the network. Working closely with EPA management and 
library staff, the national library program manager, who is 
responsible for library network strategic planning, has set in motion 
a number of actions meant to improve library network operation and 
communication, including working closely with internal and external 
advisory boards and creating a library policy and related procedures. 

EPA has resumed digitizing some of its libraries’ documents, although 
it has not inventoried the network’s holdings. The agency is 
digitizing documents in three phases. Phase 1 was completed in January 
2007, phase 2 is scheduled for completion in December 2010, and 
planning has begun for phase 3. Because EPA has not taken a complete 
inventory of its library holdings, however, it cannot determine which 
documents, or how many, will need to be digitized and, consequently, 
cannot accurately estimate the total cost of digitization or how long 
it will take. 

Since we reported on the library network reorganization in 2008, EPA 
has taken steps to communicate with staff and other stakeholders about 
its library network, including providing information about the 
libraries and soliciting information from library users. EPA has also 
made improvements to the main Internet gateway to the network, making 
more documents available electronically and providing better access to 
electronic documents and services. Nevertheless, because EPA’s 2009 
survey of the information needs and library use of its staff had 
methodological flaws—similar to those GAO identified in 2008—the 
agency is unlikely to obtain accurate information that would enable it 
to make appropriate decisions on the corrective actions that would 
best address library users’ needs. 

What GAO Recommends: 

GAO recommends, among other actions, that EPA complete its strategic 
plan for the library network and ensure that survey methods provide 
reliable data on which to base decisions. With clarifications, EPA 
concurred with our recommendations. 

View [hyperlink, http://www.gao.gov/products/GAO-10-947] or key 
components. For more information, contact John Stephenson at (202) 512-
3841 or stephensonj@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

EPA Has Not Completed a Strategic Plan for Its Library Network 
Identifying an Overall Strategy for the Network: 

Since 2008, EPA Has Reopened Closed Libraries and Taken Other Actions: 

EPA Has Resumed Digitizing Unique EPA Documents but Has Not 
Inventoried Its Holdings: 

EPA Has Taken Steps to Communicate with Staff and Other Stakeholders 
about Its Network, but Its Staff Survey Was Flawed: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: Comments from the Environmental Protection Agency: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: EPA Library Operations Before, During, and After 
Reorganization: 

Table 2: Library Network Procedures for Implementing the May 2009 
Policy: 

Figure: 

Figure 1: EPA's Library Network: 

Abbreviations: 

EPA: Environmental Protection Agency: 

NEPIS: National Environmental Publications Internet Site: 

NSCEP: National Service Center for Environmental Publications: 

[End of section] 

United States Government Accountability Office:
Washington, DC 20548: 

September 30, 2010: 

Congressional Requesters: 

In 1971, soon after its creation, the Environmental Protection Agency 
(EPA) established a library network to provide agency staff and the 
public with access to environmental information. Comprising libraries 
and repositories at EPA headquarters and in regional EPA offices, 
research centers, and laboratories across the nation, this network 
houses information on a broad range of subjects, including science, 
environmental protection and management, and environmental laws. In 
2006, partly in anticipation of fiscal year 2007 budget reductions, 
EPA began a reorganization effort meant to consolidate libraries in 
the network and make more materials and services available online. The 
reorganization plan proposed closing libraries; dispersing, disposing 
of, or digitizing some collections; cutting operating hours at certain 
libraries; and altering librarian services. In 2006, EPA closed 5 out 
of the 26 libraries it operated at the time. 

As EPA was implementing this reorganization, Congress, professional 
library associations, labor unions, and other library users raised 
several concerns. In response to congressional and other concerns, in 
January 2007 EPA placed a moratorium on its reorganization activities, 
directing its staff to make no further changes in library services 
until library policy and procedures were adopted. During the 
moratorium, 5 libraries remained closed; no additional closures were 
made; there were no additional reductions in hours of operation, 
services, or resources; and the disposal of library materials and 
document digitization stopped. 

At your request, we reported on the reorganization in 2008.[Footnote 
1] In our report, we found that EPA had not followed its own 
recommended steps to prepare for reorganizing the libraries. Neither 
did the agency develop procedures to fully inform stakeholders, such 
as library users and library professionals, of the final configuration 
of the library network; rather, communication with stakeholders varied 
from library to library. We also found that EPA lacked an effective 
strategy or central leadership to ensure the continuity of library 
services after reorganization, and each library decided on its own 
whether to close its doors and how to disperse or dispose of its 
materials. Moreover, EPA did not specifically allot funds to help 
closing libraries manage their collections, instead leaving each 
program or regional office to use its annual funding for closure costs. 

In light of continuing congressional and public concern over EPA's 
plans for its library network, you asked us to follow up on these 
issues. Accordingly, this report reviews (1) the status of EPA's 
overall strategy for its library network, (2) the status of EPA's plan 
to reopen the libraries it closed and other actions planned or taken, 
(3) EPA's efforts to digitize printed documents to make them 
electronically available, and (4) EPA's efforts to communicate with 
staff and other stakeholders about its library network. 

To address these objectives, we reviewed relevant EPA funding and 
inventory documents, policies, plans, and guidance, as well as related 
regulations and requirements pertinent to the library network and 
efforts to improve its operations. We focused our review on EPA's 
Office of Environmental Information's headquarters library in 
Washington, D.C.; the 10 regional EPA libraries; and the two Office of 
Administration and Resources Management libraries, in Cincinnati, 
Ohio, and Research Triangle Park, North Carolina. We compared library 
operations before, during, and after the reorganization; obtained and 
reviewed the library network's policy and procedures; reviewed the 
agency's draft outline of a strategic plan for the library network; 
obtained and reviewed documents on EPA's digitization process; and 
reviewed EPA's efforts to communicate with and solicit input from 
users. We interviewed EPA librarians and library managers in selected 
EPA libraries, as well as EPA officials knowledgeable about EPA's 
library network and budget. We also interviewed management officials 
from the federal employees' union representing EPA staff and spoke 
with representatives from EPA's regional science councils, which 
consist of EPA scientists and technical specialists. We further sought 
information from library professionals, including representatives from 
the American Library Association and from contractors involved in 
digitizing EPA documents. After limited testing and discussions with 
EPA officials, we determined that EPA's data on library funding and on 
the number of digitized documents and those scheduled to be digitized 
were not sufficiently reliable for our purposes. Because these data 
were the only data available, however, we used them to discuss in 
broad terms EPA's funding history and the status of its digitization 
efforts, noting the data's limitations as appropriate. Appendix I 
describes our scope and methodology in greater detail. 

We conducted this performance audit from October 2009 through 
September 2010, in accordance with generally accepted government 
auditing standards. Those standards require that we plan and perform 
the audit to obtain sufficient, appropriate evidence to provide a 
reasonable basis for our findings and conclusions based on our audit 
objectives. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions based on our audit 
objectives. 

Background: 

EPA was established in December 1970 to consolidate a variety of 
federal research, monitoring, standard-setting, and enforcement 
activities into one agency to ensure the protection, development, and 
enhancement of the total environment.[Footnote 2] To help accomplish 
its mission, EPA in 1971 established a library network that came to 
comprise 26 libraries located across the country. The library network 
functions as a collection of independent local libraries, catering 
primarily to the needs of local EPA staff and walk-in public patrons. 
The libraries are funded and managed by several different regional and 
program offices at EPA (see figure 1). EPA defines network libraries 
as those of its libraries with an official membership presence in the 
global Online Computer Library Center system. 

Figure 1: EPA's Library Network: 

[Refer to PDF for image: illustrated U.S.map] 

Region: 1; 
Regional library: Boston, Massachusetts; 
Research laboratory library: Narragansett, Rhode Island. 

Region: 2; 
Regional library: New York, New York; 
Regional library: Philadelphia, Pennsylvania. 

Region: 3; 
Library and repository: Washington, DC; 
Specialty library[A]: Washington, DC; 
Specialty library[A]: Fort Meade, Maryland. 

Region: 4; 
Regional library: Atlanta, Georgia; 
Library and repository: Research Triangle Park, North Carolina; 
Research laboratory library: Research Triangle Park, North Carolina; 
Research laboratory library: Athens, Georgia; 
Research laboratory library: Gulf Breeze, Florida. 

Region: 5; 
Regional library: Chicago, Illinois; 
Library and repository: Cincinnati, Ohio; 
Research laboratory library: Ann Arbor, Michigan; 
Research laboratory library: Duluth, Minnesota. 

Region: 6; 
Regional library: Dallas, Texas; 
Research laboratory library: Ada, Oklahoma. 

Region: 7; 
Regional library: Kansas City, Kansas. 

Region: 8; 
Regional library: Denver, Colorado; 
Specialty library[A]: Denver, Colorado. 

Region: 9; 
Regional library: San Francisco, California; 
Research laboratory library: Las Vegas, Nevada. 
Specialty library[A]: 

Region: 10; 
Regional library: Seattle, Washington; 
Research laboratory library: Corvallis, Oregon. 

Sources: EPA; Map Resources (map). 

[A] EPA identifies as specialty libraries its Legislative Reference 
Library, Office of General Counsel Law Library, and Headquarters and 
Chemical Library, all of which are in Washington, D.C. The other 
specialty libraries are its National Enforcement Investigations Center 
Environmental Forensics Library in Denver, Colorado, and the 
Environmental Science Center Library at Ft. Meade, Maryland. 

[End of figure] 

The combined EPA library network collection contains a wide range of 
general information on environmental protection and management and on 
basic and applied sciences, as well as extensive coverage of topics 
related to the statutory mandates that EPA must meet. Several of the 
libraries maintain collections focused on special topics to support 
specific regional or program office projects. The libraries thus 
differ in function, scope of collections, extent of services, and 
public access. In addition to its physical locations and holdings, the 
EPA network provides staff and public access to its collections 
through the following: 

* its online library system, a Web-based database of all of EPA 
library holdings, also known as EPA's online "card catalog"; 

* interlibrary loans to another library within the network or to other 
libraries; 

* a Web site combining two databases--EPA's National Service Center 
for Environmental Publications (NSCEP) and its National Environmental 
Publications Internet Site (NEPIS)--which provide an online gateway 
for access to available print and digital documents, respectively; and: 

* desktop access for staff to online journals, the Federal Register, 
news, databases of bibliographic information, and article citations. 

In addition, librarians are available in each library to catalog and 
maintain collections and to assist EPA staff and the public with 
research. 

In 2003, EPA began studying options for operating its library network 
in the future. In August 2006, the agency issued a reorganization 
plan, titled EPA FY 2007 Library Plan: National Framework for the 
Headquarters and Regional Libraries. The focus of this plan was a 
reorganization of the headquarters library and the 10 regional 
libraries, all of which received substantial funding from EPA's Office 
of Environmental Information. The 2007 library plan identified a new 
model for library services, which consisted of three components: a 
coordinated library network, instead of stand-alone operations; more 
electronic delivery of services; and maintenance of existing essential 
services. During implementation of this plan, EPA closed the Chicago, 
Dallas, and Kansas City regional libraries and closed its headquarters 
library to physical access, although the headquarters library remained 
as one of three repositories for storing print collections.[Footnote 
3] Another library located at EPA headquarters within the Office of 
Prevention, Pesticides, and Toxic Substances (referred to as the 
"chemical library") was not subject to budget reductions and was not 
discussed in the reorganization plan, but, like the headquarters 
library, was also closed to physical access. EPA also reduced or 
eliminated the library staff at the closed libraries. Several other 
libraries reduced their operating hours, and some libraries disposed 
of their materials or dispersed them to other EPA libraries or to non-
EPA libraries. EPA also began to digitize EPA documents not currently 
in NEPIS, beginning with documents in the libraries being closed. 
EPA's reorganization plan also discussed how the closed libraries were 
to handle their collections, directing them primarily to disperse the 
collections to other libraries. 

EPA's implementation of its reorganization plan caused widespread 
concern among its staff, the public, interested parties, and Congress. 
In response to these concerns, congressional committees directed $1 
million in funding to restore the libraries recently closed or 
consolidated, asked us to review EPA's reorganization plan and its 
implementation, and directed EPA to prepare a report regarding actions 
to restore publicly available libraries. In addition, EPA in January 
2007 imposed a moratorium on its reorganization efforts. 

Until April 2007, EPA's library network operations had been guided by 
EPA's Information Resources Management Policy Manual, which stated 
that the library network was to provide EPA staff with access to 
information to carry out the agency's mission and that the libraries 
were to provide state agencies and the public with access to the 
library collection. The Policy Manual also defined the role of a 
national library program manager, who was to have responsibility for 
coordinating major activities of the library network, although not 
budget authority for the libraries. EPA replaced this manual in April 
2007 with an interim library policy and, in May 2009, with its final 
library policy. The final May 2009 policy also defined key roles and 
responsibilities, including those of the national library program 
manager and those of "federal library managers," who were to have 
first-line responsibility for operating the physical libraries and 
providing services. 

EPA lifted the moratorium in June 2009, following implementation of 
its May 2009 policy and 6 of 12 proposed procedures for the library 
network. After we issued our report in February 2008,[Footnote 4] 
Congress held hearings on EPA's library network reorganization 
efforts, which were followed by the release of EPA's March 2008 report 
addressed to Congress, in which EPA stated that it would reopen the 
closed libraries by September 30, 2008.[Footnote 5] In our 2008 report 
on EPA's library network reorganization, we assessed the 
reorganization effort against our past work on key practices and 
implementation steps to assist mergers and organizational 
transformations.[Footnote 6] These key practices include ensuring that 
top leadership drives the transformation, establishing a coherent 
mission and integrated strategic goals to guide the transformation, 
and setting implementation goals and a timeline to show progress from 
day one. 

EPA Has Not Completed a Strategic Plan for Its Library Network 
Identifying an Overall Strategy for the Network: 

Although it has been preparing a strategic plan for its library 
network for 3 years, EPA has not completed a plan identifying its 
overall network strategy, with implementation goals and a timeline for 
what it seeks to accomplish. In our 2008 report, we stated that EPA 
was developing a library strategic plan for 2008 and beyond, which was 
to detail library services for staff and the public and lay out a 
vision for the library network's future. EPA has had a draft outline 
of this strategic plan since July 2007. We reported that in October 
2007, EPA's Office of Environmental Information asked local unions 
throughout the agency to comment on a draft of the plan. 

The draft outline of the strategic plan envisions the library network 
as "the premier environmental library network that provides timely 
access to information and library services to its employees and the 
public" and proposes to realize this vision by increasing emphasis on 
electronic resources and using new information technologies.[Footnote 
7] The draft outline of the strategic plan also lists several 
principles as the foundation for present and future directions of the 
library network: setting overall goals and objectives and a direction 
for implementation; periodic review of the plan to evaluate progress 
and update strategies to respond to new opportunities or challenges; 
soliciting input from internal and external stakeholders; and 
developing the plan in a transparent manner by reporting progress and 
soliciting input from interested parties. According to EPA officials, 
since 2007, EPA has been in the process of assessing library users' 
needs, which is to be completed before they believe they can finish 
the strategic plan. EPA officials have stated that a working group led 
by the national library program manager is to resume work on the plan 
later in 2010. 

The draft outline of the strategic plan is largely a list of current 
and planned EPA activities--primarily placeholders to be completed. 
For example, under the heading, "Digitization," the text states that 
the digitization procedures will outline the methods to be used by EPA 
libraries to prepare and send EPA documents for digitization and 
inclusion in NEPIS; no goals or timeline for implementing these 
activities--which we previously reported are among the key components 
of successful organizational transformations--are given.[Footnote 8] 
We have found that an organization's transformation is strengthened 
when it publicizes implementation goals and a timeline to build 
momentum and show progress. Despite an emphasis on the central role to 
be played by electronic library resources, the draft outline of the 
strategic plan briefly describes procedures for packing and shipping 
documents to be digitized, without describing actions to be taken to 
digitize holdings or target dates for accomplishing those actions. EPA 
holds an enormous amount of environmental information, including 
publications generated by its program offices, as well as research 
publications generated under contracts, grants, and cooperative 
agreements. A large portion of this information exists only in print 
form. As part of its vision for the library network, according to the 
draft outline, EPA is seeking to convert this information into a 
digital format to make it more widely available and readily accessible 
to users. Yet the draft outline of the strategic plan does not 
describe criteria for deciding what documents should be digitized, for 
deciding whether or how to digitize copyrighted documents of value, or 
for scheduling the funding needed and a time frame for completing the 
digitization. Without such criteria, EPA cannot ensure that it is 
digitizing the most valuable or important documents or providing users 
with information most relevant to them. 

Furthermore, although the draft outline of the strategic plan includes 
a placeholder for a section describing a funding model for the 
network, it contains no detail. Under the heading, "Funding Model," 
the text states that this section in the plan will address how EPA 
will ensure that the network libraries have "adequate funding" and 
will discuss how funding decisions are made, along with the Office of 
Environmental Information's role in the funding process. But the draft 
outline of the strategic plan does not define what constitutes 
adequate funding, although inadequate funding has been a concern for 
the library network since fiscal year 2007. Until then, library 
spending had remained relatively stable, ranging from a high of $9.2 
million in fiscal year 2002 to a low of $8.2 million in fiscal year 
2006.[Footnote 9] In fiscal year 2007, when EPA's budget was reduced, 
library spending was $6.3 million. 

The draft outline of the strategic plan also does not set out the 
details of how funding decisions are made. Setting out details for how 
such decisions are to be made, to ensure that they are informed and 
transparent, is especially important because of the decentralized 
nature of the library network. The library network's funding remains 
subject to uncertainty in the future because the several different 
program and regional offices responsible for EPA's libraries generally 
decide how much to spend on their libraries out of funding available 
in larger accounts that support multiple activities. EPA's Office of 
Environmental Information, the primary source of funding for the 
regional libraries,[Footnote 10] typically provides funding through 
each region's support budget and generally gives regional management 
officials discretion on how to distribute this funding among the 
library and other support services, such as information technology, 
utilities, and mail room support. EPA officials told us that, starting 
in fiscal year 2010, they are increasing the amount of funding 
allocated to the libraries in the regions. The regions also obtain a 
much smaller portion of their library funding from other program 
offices. For example, the Superfund program office funds the storage 
and maintenance of information on the National Priorities List, EPA's 
list of the most seriously contaminated sites in the United States. 
The extent to which other program offices provide funding to the 
regional libraries varies. Thus, without a detailed strategy for how 
decisions are made to acquire, deploy, and manage funding resources 
across the library network, EPA may find it difficult, particularly in 
an era of declining budgets and competing national priorities, to 
achieve its vision for the library network and to fully meet the needs 
of library users. 

Moreover, although the draft outline of the strategic plan contains a 
section on communication among network libraries, it makes no mention 
of a strategy or a formal outreach plan to ensure that EPA 
communicates with and obtains feedback from users about improvements 
to its library network in a way that would allow it to measure whether 
such improvements are meeting users' needs. The section lists 
communication methods EPA is using, such as monthly network 
teleconferences among staff and federal managers. In addition, the 
section identifies comment cards, questionnaires, and a Web presence 
for how it solicits users' feedback, but there is no mention of how 
EPA plans to assess feedback on what is important to users or what 
improvements are working well or poorly. EPA has another draft 
document, titled "EPA Library Network Communication Strategies," whose 
purpose is to establish procedures by which libraries in the network 
are to communicate both internally and externally. Most of this 
document focuses on communication within the library network itself, 
explaining how the library network is coordinated and detailing 
mechanisms for internal communication, including annual meetings for 
library network staff and federal library managers. One of the final 
sections in this procedures document lists several means of 
communicating externally, including Web sites and various local 
options for libraries to reach out to local patrons, such as tours, 
signs, comment cards, and online feedback mechanisms. Beyond listing 
such mechanisms, however, this document, like the draft outline of the 
strategic plan, does not lay out a systematic communication strategy 
with communication and feedback performance goals that can be measured 
to determine progress. Without such a strategy, communication with 
library users is likely to remain piecemeal and reactive. 

Since 2008, EPA Has Reopened Closed Libraries and Taken Other Actions: 

EPA has reopened all of the libraries it closed in 2007 and has taken 
other actions to improve library operations. In its 2008 report 
addressed to Congress,[Footnote 11] the agency stated its commitment 
to have libraries in each region and at headquarters open to the 
public. EPA also committed to reestablishing on-site libraries for its 
staff and the public in the three regions where the libraries had been 
closed and at the combined headquarters and chemical library. EPA 
reopened all five closed libraries by September 2008, although the 
agency had to find new space for two of the three closed regional 
libraries and their collections, and all three of these regional 
libraries are operating on reduced schedules. Each of the reopened 
libraries was staffed with a professional librarian and required to 
maintain a collection of core reference materials and additional 
library resources to meet local needs and to ensure that staff had 
access to core library services and the public had access to the 
library and its collections. With the reopening of the closed 
libraries, one other regional library that had not been closed also 
began operating on reduced schedules, as compared with its hours 
before reorganization (see table 1). As of September 2010, about half 
of the 10 regional libraries were operating 4 days a week, rather than 
5 days as they were before EPA's reorganization efforts--a reduction 
in hours due largely to funding constraints, according to library 
officials. All of the libraries are staffed with at least one full-
time or part-time librarian, with several libraries having more than 
one librarian or additional library staff. 

Table 1: EPA Library Operations Before, During, and After 
Reorganization: 

Program office: Office of Environmental Information; 
Library, location: Headquarters library, Washington, D.C.[A]; 
Operating status of library: 
Before reorganization: Open 5 days, 40 hours; 
During reorganization: Closed to physical access; 
After reorganization: Open 5 days, 40 hours. 

Program office: Office of Prevention, Pesticides, and Toxic Substances; 
Library, location: Chemical library, Washington, D.C.; 
Operating status of library: 
Before reorganization: Open 5 days, 35 hours; 
During reorganization: Closed to physical access; 
After reorganization: Combined with headquarters library. 

Program office: Regional office; 
Library, location: Region 1 library, Boston, Massachusetts; 
Operating status of library: 
Before reorganization: Open 5 days, 35 hours; 
During reorganization: Reduced hours of operation 3 days, 18 hours; 
After reorganization: Open 4 days, 24 hours. 

Program office: Regional office; 
Library, location: Region 2 library, New York, New York; 
Operating status of library: 
Before reorganization: Open 4 days, 28 hours; 
During reorganization: Reduced hours of operation 3 days, 12 hours; 
After reorganization: Open 4 days, 28 hours. 

Program office: Regional office; 
Library, location: Region 3 library, Philadelphia, Pennsylvania; 
Operating status of library: 
Before reorganization: Open 5 days, 45 hours; 
During reorganization: Open 5 days, 45 hours; 
After reorganization: Open 5 days, 45 hours. 

Program office: Regional office; 
Library, location: Region 4 library, Atlanta, Georgia; 
Operating status of library: 
Before reorganization: Open 5 days, 42.5 hours; 
During reorganization: Open Some services provided by the library in 
Cincinnati, Ohio 5 days, 42.5 hours; 
After reorganization: Open 5 days, 42.5 hours. 

Program office: Regional office; 
Library, location: Region 5 library, Chicago, Illinois; 
Operating status of library: 
Before reorganization: Open 5 days, 25 hours; 
During reorganization: Closed to physical access; 
After reorganization: Open 4 days, 24 hours. 

Program office: Regional office; 
Library, location: Region 6 library, Dallas, Texas; 
Operating status of library: 
Before reorganization: Open 5 days, 40 hours; 
During reorganization: Closed to physical access; 
After reorganization: Open 5 days, 30 hours. 

Program office: Regional office; 
Library, location: Region 7 library, Kansas City, Kansas; 
Operating status of library: 
Before reorganization: Open 5 days, 40 hours; 
During reorganization: Closed to physical access; 
After reorganization: Open 4 days, 24 hours. 

Program office: Regional office; 
Library, location: Region 8 library, Denver, Colorado; 
Operating status of library: 
Before reorganization: Open 5 days, 40 hours; 
During reorganization: Open 5 days, 40 hours; 
After reorganization: Open 5 days, 40 hours. 

Program office: Regional office; 
Library, location: Region 9 library, San Francisco, California; 
Operating status of library: 
Before reorganization: Open 5 days, 24 hours; 
During reorganization: Reduced hours of operation 4 days, 24 hours; 
After reorganization: Open 4 days, 24 hours. 

Program office: Regional office; 
Library, location: Library, location: Region 10 library, Seattle, 
Washington; 
Operating status of library: 
Before reorganization: Open 5 days, 40 hours; 
During reorganization: Reduced hours of operation 5 days, 25.5 hours; 
After reorganization: Open 5 days, 40 hours. 

Source: GAO analysis of EPA data. 

Note: This table shows only those libraries whose operations were 
affected or potentially affected by reorganization. Days and hours of 
operation shown in the table are for both the public and EPA staff; 
the regional libraries in Atlanta, Denver, and Seattle are open 4 to 
10 hours per week longer for EPA staff than for the public. While the 
libraries in Regions 5, 6, and 7 were closed, services for these 
libraries were provided by another network library. 

[A] Library also serves as a repository. 

[End of table] 

In addition to reopening the closed libraries, according to EPA 
officials we spoke with, EPA developed standards for the regional and 
headquarters libraries' use of space, on-site collections, staffing, 
and services. These standards specify, for example, that the libraries 
make adequate space available for in-person interactions between 
library staff and users, that on-site collections and materials should 
address local and regional needs, that staff and the public have 
certain minimum hours of access per week (at least 4 days per week on 
a walk-in or appointment basis in the regional libraries and at least 
5 days per week on a walk-in or appointment basis in the headquarters 
library), and that the libraries provide interlibrary loans and 
reference or research assistance. 

One of the key actions taken by EPA in May 2007 was to hire a national 
library program manager, a position that had been vacant since 2005. 
[Footnote 12] Housed in EPA's Office of Information Analysis and 
Access, within the Office of Environmental Information, the national 
library program manager is charged with carrying out day-to-day 
activities of the library network and with bringing focus and cohesion 
to the network. Part of this charge involves agencywide responsibility 
for public information access, including strategic planning for the 
library network, and participating in policy formulation regarding 
access to EPA's public information. To fulfill this leadership role, 
EPA officials said, the national library program manager is to work 
closely with the management of EPA's Office of Environmental 
Information to set in motion a number of actions meant to improve 
library network operation and communication. To communicate with and 
gather feedback internally, the national library program manager 
initiated monthly teleconferences and annual meetings for all library 
managers and staff. Seeking to get the most out of the experience and 
knowledge of these library managers, librarians, and staff, the 
national library program manager established internal working groups 
to research improvement activities, address concerns, and recommend 
improvements. For example, the national library program manager 
established working groups on digitization, staff information needs, 
and development of the final May 2009 library policy and related 
procedures (see table 2). In addition, the national library program 
manager serves as the EPA-appointed representative in working with 
outside library professionals, specifically an external board of 
advisors created by the Federal Library and Information Center 
Committee, which advises EPA on future directions for the library 
network.[Footnote 13] 

Table 2: Library Network Procedures for Implementing the May 2009 
Policy: 

Procedure: EPA library materials dispersal; 
Purpose: Establish agencywide procedures for libraries to retain, 
reduce, disperse, or dispose of their library contents when 
appropriate; 
Status, approval date: Approved May 15, 2009. 

Procedure: EPA library usage statistics; 
Purpose: Establish agencywide procedures by which the libraries 
collect statistics on the services they provide EPA staff and the 
public; 
Status, approval date: Approved May 15, 2009. 

Procedure: Digitization processes for EPA libraries; 
Purpose: Establish agencywide procedures to identify, prepare, and 
send EPA documents to NSCEP for digitization so they may be added to 
the NEPIS database (formalizes existing steps for digitizing 
documents); 
Status, approval date: Approved May 15, 2009. 

Procedure: EPA library reference and research services; 
Purpose: Establish agencywide procedures to provide reference and 
research services to EPA staff and the public; 
Status, approval date: Approved May 15, 2009. 

Procedure: EPA repository library management; 
Purpose: Establish agencywide procedures to ensure that official EPA 
documents and other materials that are hard to replace but 
infrequently used are available in perpetuity in their originally 
published format; 
Status, approval date: Approved May 15, 2009. 

Procedure: EPA library facility management; 
Purpose: Establish agencywide procedures to provide, make use of, and 
manage the facility allotted for library operations; 
Status, approval date: Approved May 15, 2009. 

Procedure: Interlibrary loan service; 
Purpose: Establish agencywide procedures to borrow or acquire 
materials not available in the local collections for EPA staff and the 
public; 
Status, approval date: In draft, due to be approved before the end of 
2010. 

Procedure: Cataloging; 
Purpose: Establish agencywide procedures for libraries to catalog 
materials in the collections and provide access to remote electronic 
documents; 
Status, approval date: In draft, due to be approved before the end of 
2010. 

Procedure: Public access; 
Purpose: Establish agencywide procedures to improve public access to 
EPA documents and environmental information; 
Status, approval date: In draft, due to be approved before the end of 
2010. 

Procedure: Collection development; 
Purpose: Establish agencywide standard methods for libraries to 
acquire, organize, and manage materials in their local collections; 
Status, approval date: In draft, due to be approved before the end of 
2010. 

Procedure: Disaster response and emergency; 
Purpose: Establish agencywide procedures by which the libraries 
prepare for and respond to disasters and provide continuing operations 
during and after a disaster; 
Status, approval date: In draft, due to be approved before the end of 
2010. 

Procedure: Library network communications strategies; 
Purpose: Establish agencywide procedures by which the libraries in the 
network communicate, using a range of established mechanisms, with 
other EPA libraries, organizations, and the public; 
Status, approval date: In draft, due to be approved before the end of 
2010. 

Source: EPA. 

[End of table] 

EPA Has Resumed Digitizing Unique EPA Documents but Has Not 
Inventoried Its Holdings: 

EPA has restarted its process of digitizing some of its libraries' 
holdings, but because the agency has not completed an inventory of its 
holdings, it does not know the total number of documents to be 
digitized. According to EPA data, which our limited testing found to 
be insufficiently reliable, EPA had digitized 16,175 documents from 
its libraries as of January 2010. Creating an electronic copy of a 
document by means of digitization is relatively simple--essentially 
the same scanning process for making photocopies--although it can be 
time-consuming and expensive if the document contains special features 
such as foldout pages, cannot be taken apart, or needs to be digitized 
at a high level of resolution or in color. After the resulting 
electronic files are uploaded to EPA's Web databases, the 
administrator of EPA's online library catalog is to ensure that links 
to the digitized documents are included in the bibliographic records 
for each document. According to EPA officials, the present 
digitization effort will expand NEPIS, EPA's sole electronic archive 
of published material, which, according to EPA officials, contains 
40,000 publicly accessible digital documents as of June 2010, up from 
4,000 in 1996. 

According to EPA documents, the digitization process is to take place 
in three phases: 

* Phase 1 covered unique EPA documents held by the libraries that were 
closed under the reorganization plan.[Footnote 14] EPA data show that 
15,260 documents were digitized during this phase, which was completed 
in January 2007. 

* Phase 2 is to cover all remaining unique EPA documents except those 
larger than 11 by 17 inches. According to EPA officials, this phase is 
scheduled for completion in December 2010 and should produce 10,102 
additional digitized documents, bringing the total number of digitized 
library documents available to the public to over 25,000. 

* Phase 3 is to include EPA documents of which more than one copy 
exists in the library network, plus unique EPA documents larger than 
11 by 17 inches. As of July 2010, EPA was beginning to plan this phase. 

As of September 2010, the total estimated cost for digitizing EPA's 
library holdings remained unclear, in large part because EPA has not 
completed an inventory of its holdings and has therefore not 
determined the total number of documents that need to be digitized. 
When it began digitizing documents from the closed libraries in 2006 
under phase 1, EPA estimated that the project would cost $80,000--
$78,000 for scanning and $2,000 for uploading the digital files to the 
Web databases--although, according to agency officials, the agency did 
not track the actual costs. For phase 2 digitization, which began in 
fiscal year 2009, EPA estimated the cost at about $327,000. EPA has 
not estimated the cost or a completion date for its final, phase 3 
digitization effort, in part because the agency is still cataloging 
all its library holdings in a single database so it can inventory all 
the documents that need to be digitized. One regional librarian we 
spoke with, for example, told us that about 2,000 documents in the 
regional library's catalog were not in EPA's online library system, 
and it is still unknown which or how many of these documents will need 
to be digitized. Without a complete catalog or inventory of its 
holdings, EPA cannot determine which documents, or how many, will need 
to be digitized and, consequently, cannot accurately estimate the 
total cost of digitization or how long it will take. According to the 
national library program manager, an EPA workgroup is currently 
drafting a new cataloging procedure for the libraries and expects the 
procedure to be approved and implemented before the end of 2010. This 
procedure requires each network library to inventory its collection on 
a regular basis, either the entire collection every 3 years or one-
third of the collection each year. 

In addition, EPA library officials observed that a significant number 
of the documents in EPA libraries are copyrighted, and to date EPA 
does not plan to digitize them. EPA, like other federal agencies, 
often contracts with entities in the private sector to do work. In 
addition, EPA provides financial assistance in the form of grants and 
cooperative agreements to various recipients, such as state, local, 
and tribal governments; educational institutions; hospitals; and 
nonprofit organizations. Such assistance is documented in an 
assistance agreement. Both contracts and assistance agreements may 
result in the production of copyrighted documents. In the case of 
contracts, federal regulations provide that when a contractor is 
permitted to assert a copyright in any document(s) produced, the 
government has a license to display the copyrighted work publicly, 
which includes posting it on a Web site accessible to the public. 
[Footnote 15] In the case of works produced under assistance 
agreements, on the other hand, the government has a right to 
reproduce, publish, or otherwise use a copyrighted work for federal 
purposes, but EPA's Office of General Counsel has determined that 
inclusion in EPA's online public library would not constitute a 
federal government purpose.[Footnote 16] 

On the advice of EPA's general counsel, EPA's digitization workgroup 
has recommended digitizing documents created under contract but not 
those created under EPA's assistance agreements.[Footnote 17] 
According to EPA's grant awards database, these agreements have 
resulted in more than 21,000 grants valued at over $40 billion in 
taxpayer dollars. Some of these grants led to publications, resulting 
in a substantial body of publicly funded written material. According 
to EPA's general counsel, EPA may digitize such documents so that 
staff and others may use them for federal government purposes but may 
not disseminate them for other purposes. EPA may also seek permission 
from copyright holders to digitize and disseminate online copyrighted 
documents produced under assistance agreements, although some costs 
may be associated with obtaining such permissions--tracking down 
copyright holders after years, or even decades, have passed, for 
example--further complicating any estimation of total digitization 
costs. An alternative practice has been in use by the Federal Library 
and Information Network,[Footnote 18] the business arm of the Federal 
Library and Information Center Committee: permission to use 
copyrighted material produced under assistance agreements is sought at 
the time an agreement is established. If the prospective copyright 
holder grants permission, then a statement to that effect is 
incorporated into the assistance agreement, incurring minimal, if any, 
additional costs. Without permission from copyright holders, however, 
documents prepared under EPA assistance agreements, using taxpayer 
dollars, will remain unavailable online to the public. 

EPA Has Taken Steps to Communicate with Staff and Other Stakeholders 
about Its Network, but Its Staff Survey Was Flawed: 

EPA has taken steps to communicate with staff and other stakeholders 
about its library network--including providing information about the 
libraries as well as soliciting information from library users--but a 
2009 survey about its staff's information needs was flawed. In 
general, EPA staff and external stakeholders told us the agency is 
doing a better job of communicating with them and soliciting input on 
the operations and future direction of the library network, 
particularly at the local level. Representatives from EPA's employees' 
unions and regional science councils stated that communication about 
the library and its services--such as new resources, training, and 
open houses--is primarily done at the regional level, either through e-
mail or the region's intranet page. Although staff have not been 
directly solicited for feedback, according to the representatives, no 
outstanding issues regarding the libraries have been raised, except 
that a few representatives said they would like to see the libraries 
open 5 days and 40 hours per week. 

To keep library managers and staff engaged in improving library 
operations, EPA has adopted a number of techniques to communicate with 
them and solicit their input. These techniques have allowed EPA to 
gather staff input for policies and procedures, operational issues, 
and Web page improvements. Examples include the following: 

* The national library program manager holds monthly network 
teleconferences with library managers and staff on matters of interest 
to the entire network or on operational topics, such as the library 
policy and procedures. 

* The national library program manager also holds ad hoc 
teleconferences with library managers elsewhere in the network to 
discuss their libraries' needs. 

* Managers and staff use mailing lists to communicate with one another 
about daily library operations or requests for assistance. 

* For the last 3 years, EPA has held an annual network meeting in 
different locations for library managers and staff to foster 
collaboration, provide training, and share information about the 
network. At the last meeting, in October 2009, participants discussed 
ways to address results of the 2009 staff survey, prepared for the 
next round of digitization, and discussed ways to improve library 
services. The next annual meeting is scheduled for March 2011. 

* EPA and the union representing EPA staff agreed to create a union- 
management advisory board with six members--three union 
representatives and three from EPA management.[Footnote 19] The board 
reviews and makes recommendations on library network policy and 
procedures and will review the library network strategic plan once it 
is completed. 

* In December 2009, EPA instituted a pilot program, an "ask a 
librarian" live chat. Ten libraries are participating in the program, 
which lets users contact a librarian through an electronic link to 
request services. As of July 2010, the "ask a librarian" pilot was 
available only to EPA staff. 

To begin to realize its vision of effectively implementing new 
information technologies and making documents readily available 
electronically, EPA in 2007 engaged a contractor to review the user- 
friendliness of the combined Web page, or gateway, to the NSCEP and 
NEPIS online databases.[Footnote 20] The review identified ways to 
improve the site's effectiveness and overall functionality for users. 
EPA implemented many of these improvements.[Footnote 21] For example, 
the gateway home page now describes the purpose of each database 
(NSCEP for print materials and NEPIS for electronic documents) and 
what types of publications they contain, noting that they contain only 
EPA publications. EPA made several changes to the document display 
page as well, such as placing a navigation bar at the top and bottom 
of the document with large icons and providing a button that allows 
users to obtain a copy in one of three formats. The display page 
allows users to put a copy of frequently used documents in a holding 
area for later retrieval. Work is also under way to integrate Google 
search capabilities into this gateway, as well as the ability to 
refine the precision of searches with user-friendly "clouds" of 
related keywords. In addition, EPA has added easy ways for users to 
offer feedback, which EPA may then incorporate to make improvements. 
For example, the navigation bar on the document display page now 
includes a "report an error" button, and every page has a "contact us" 
link. Furthermore, the left frame of the site contains a link to a 
customer satisfaction survey, and the site also has a separate page 
for user feedback. 

EPA has also made outreach efforts to library professionals outside 
EPA--primarily by presenting and exhibiting at professional library 
association trade shows and conferences, attending external training, 
visiting other federal national libraries, and interacting with its 
external board of advisors. Ties with the external board have been a 
particularly important part of EPA's response to concerns over the 
agency's library reorganization. From June 2007 through February 2010, 
the national library program manager met with the board approximately 
20 times, working with it on the full range of key issues, from policy 
development to funding models to communication with stakeholders. The 
external board also advised EPA on the development of a survey to 
assess the information needs of EPA staff. 

One of the principles in the draft outline of the strategic plan is 
soliciting feedback from internal and external stakeholders about 
their information needs. To solicit such feedback from staff, EPA in 
early 2008, under the direction of the national library program 
manager, engaged a contractor to conduct interviews, hold focus 
groups, and conduct a Web-based survey. The survey was made available 
for approximately 1 month in 2009 via a secure Web site only to EPA 
staff (about 17,000 individuals), not all of whom were library users. 
After the survey was completed, the contractor conducted a series of 
focus groups and one-on-one interviews with EPA managers, focusing on 
significant issues identified in the Web-based survey; according to 
EPA officials, these in-person discussions were to help ensure that a 
comprehensive perspective of user needs was captured. EPA officials 
stated that the Web-based survey results corroborated what the agency 
learned in an earlier survey, done in 2004 to 2005. On the basis of 
the survey results, focus group discussions, and management 
interviews, the contractor developed recommendations for EPA's 
consideration. EPA received the results of the survey and discussions, 
along with the contractor's recommendations, in August 2009 and has 
assigned working groups of library staff to review the findings and 
suggest how EPA could address them. 

We found, however, that this survey had flaws, similar to those we 
identified in the 2004 to 2005 survey and discussed in our 2008 
report, which greatly reduce its usefulness.[Footnote 22] First, in 
both the earlier and the 2009 surveys, the response rate was 14 
percent, far lower than the 80 percent response rate that Office of 
Management and Budget guidance recommends for a survey to increase the 
likelihood that it adequately represents a universe of respondents. 
[Footnote 23] Neither EPA nor the contractor for the 2009 survey 
analyzed the results for the nonresponse bias that may occur at 
response rates lower than 80 percent, particularly if the group of 
respondents differs significantly in relevant ways from 
nonrespondents.[Footnote 24] Thus, EPA cannot be assured that either 
survey accurately described staff needs for information or their uses 
of the library. Second, respondents to both the 2009 and the earlier 
survey were self-selected, a methodology that often leads to biased 
samples, since the traits that affect a person's decision to 
participate in the survey--such as strong opinions or substantial 
knowledge--may be correlated with traits that affect survey responses; 
the result is an unrepresentative sample of possible respondents. The 
risk of potential bias through self-selection is increased by the fact 
that neither EPA nor the contractor for the 2009 survey instituted any 
safeguards to prevent respondents from submitting more than one survey 
each. Thus, there is no assurance that the survey results are unbiased 
and reflect a broad range of EPA staff perspectives and experiences. 
Third, in neither survey did EPA gather views from or determine the 
needs of other significant users of EPA libraries, such as state and 
local environmental organizations or the public at large.[Footnote 25] 
Although EPA officials told us that EPA is planning to assess the 
needs of public patrons, an assessment that does not correct the 
methodological weaknesses we found in EPA's two previous surveys of 
its staff is unlikely to produce results that accurately reflect the 
needs of public patrons. 

Conclusions: 

In the 4 years since EPA issued a reorganization plan for its library 
network, the agency has taken a number of steps to better communicate 
with, and meet the needs of, library users. EPA's lack of a completed 
strategic plan identifying its overall strategy for the network, 
however, leaves unclear how the agency will translate into reality its 
vision of a "premier environmental library" with an "emphasis on 
electronic resources." Steps the agency has taken, including hiring a 
national library program manager and establishing a uniform policy and 
some procedures for the libraries, have led to some improvements in 
library services and will undoubtedly enhance network cohesion. But 
without a completed strategic plan that contains implementation goals 
and timelines, neither EPA nor users of its libraries can have a clear 
view of what EPA plans to do, when EPA plans to do it, and whether 
EPA's actions will ultimately meet users' needs. In particular, 
without a strategy for acquiring, deploying, and efficiently 
allocating library funding, the library network could have difficulty 
maintaining high-quality service in the digital age. Moreover, EPA's 
approach to digitizing copyrighted works in the future--as well as the 
fact that the agency has not yet inventoried all library holdings--
could, if not revisited, detract significantly from the utility of the 
library network. Specifically, unless EPA revisits its decision not to 
digitize documents prepared with taxpayer dollars under assistance 
agreements, it may be missing opportunities to make these documents 
more readily available to users, including other federal users, who 
need them to better carry out their work. Finally, improvements to the 
library network's Internet gateway offer new means of seeking feedback 
from library patrons about their use of and need for library services. 
Nevertheless, EPA does not have a valid method for assessing those 
library users' needs. If, in future assessments of users' needs, EPA 
fails to correct the flawed methods of its previous staff surveys, the 
agency is unlikely to obtain accurate information that would enable it 
to make appropriate decisions on the corrective actions that would 
best address those needs. 

Recommendations for Executive Action: 

To ensure that EPA's library network continues to meet its users' 
needs, we recommend that the Administrator of EPA take the following 
six actions: 

* Complete EPA's strategic plan for the library network, including 
implementation goals and timelines. In so doing, EPA should outline 
details for how funding decisions are to be made, to ensure that they 
are informed and transparent. 

* Complete an inventory of the library network's holdings to identify 
what items remain to be digitized. 

* For assistance agreements already in place, EPA should digitize 
documents produced under the agreements and make them available to 
federal employees and other authorized users for federal government 
purposes. 

* In future assistance agreements, make explicit that EPA can include 
in the agency's public online database, without obtaining prior 
permission from the copyright holder, any documents produced under the 
agreements. 

* For future assistance agreements where EPA cannot make such an 
arrangement, EPA should digitize documents produced under the 
agreements and make them available to federal employees and other 
authorized users for federal government purposes. 

* Ensure that the data analysis protocols used for conducting surveys 
of users' needs--including sampling procedures and response rates--are 
sufficiently sound methodologically to provide reliable information on 
which to base decisions and allocate resources efficiently. 

Agency Comments and Our Evaluation: 

We provided EPA with a draft of this report for review and comment. 
With clarifications, the agency concurred with our recommendations. 
EPA acknowledged that the planning document available on the agency's 
Web site--which our report refers to as the draft outline of the 
strategic plan--has provided more of a working agenda than a strategic 
plan to guide the rebuilding of the library program. In responding to 
our recommendations, EPA wrote that it believes it now has enough 
information to develop an effective strategic plan for the library 
network and that it is time to complete and publish a formal plan 
identifying an overall network strategy, with implementation goals and 
a timeline for future accomplishments. The agency stated that it is 
moving forward with the strategic plan, which it aims to complete in 
fiscal year 2011. In addition, EPA said it will address the inventory 
of library holdings, completing a schedule for cataloging the 
inventory by November 1, 2010, and striving to complete the cataloging 
by September 30, 2011. The agency further agreed to take the necessary 
steps to ensure that any future assessments of users' needs employ 
methodologically sound data protocols and provide reliable information. 

Regarding our recommendations on the digitization of copyrighted 
documents produced under assistance agreements, EPA said it would 
address the feasibility and legality of digitizing products resulting 
from such agreements. For future assistance agreements, the agency 
said it will develop options for gaining advance permission to 
digitize products from these agreements and take these options to 
senior agency managers by mid-2011 for consideration and action. For 
existing assistance agreements, however, EPA wrote that, because of 
legal and technical constraints, it does not plan to digitize products 
produced under existing agreements. In further clarifying the agency's 
written comments, EPA officials told us that because the documents 
produced under existing assistance agreements are copyrighted, the 
agency cannot include them in its public online database. In the 
agency's view, EPA would therefore need to develop a forum for 
disseminating the documents to EPA staff and determine whether other 
federal employees needed access to the documents for federal 
government purposes. EPA officials also said that digitizing these 
documents was constrained by several factors, including agency 
priorities for which documents are to take precedence and efforts to 
identify which of the many types of assistance programs are likely to 
produce documents of most value to EPA staff. We have clarified the 
wording of our recommendations to eliminate any implied reference that 
EPA should make the copyrighted documents available to the general 
public in its online database, and we maintain that making copyrighted 
documents resulting from assistance agreements available solely for 
federal government purposes is permissible, feasible, and desirable. 

EPA's written comments appear in appendix II. EPA also provided 
technical corrections, which we incorporated. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies of this report 
to the appropriate congressional committees, Administrator of EPA, and 
other interested parties. in addition, this report will be available 
at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. 

If you or your staff members have any questions on this report, please 
contact me at (202) 512-3841 or stephensonj@gao.gov. Contact points 
for our Offices of Congressional Relations and Public Affairs may be 
found on the last page of this report. GAO staff who made major 
contributions to this report are listed in appendix III. 

Signed by: 

John B. Stephenson: 
Director, Natural Resources and Environment: 

List of Requesters: 

The Honorable Barbara Boxer: 
Chairman: 
Committee on Environment and Public Works: 
United States Senate: 

The Honorable Bart Gordon: 
Chairman: 
Committee on Science and Technology: 
House of Representatives: 

The Honorable Henry A. Waxman: 
Chairman: 
The Honorable John Dingell: 
Chairman Emeritus: 
Committee on Energy and Commerce: 
House of Representatives: 

[End of section] 

Appendix I: Scope and Methodology: 

To complete our work, we reviewed relevant Environmental Protection 
Agency (EPA) funding and inventory documents, policies, plans, 
guidance, and procedures, as well as related regulations and 
requirements pertinent to the library network and efforts to improve 
its operations. We limited our review to the 26 libraries belonging to 
EPA's library network, that is, libraries that are members of the 
Online Computer Library Center system. We focused on EPA's 
headquarters library, the 10 regional libraries funded in part by 
EPA's Office of Environmental Information, and the Office of 
Administration and Resources Management libraries in Cincinnati, Ohio 
(which is responsible for EPA's digitization and Web site 
maintenance), and Research Triangle Park, North Carolina. We compared 
library operations before, during, and after attempted reorganization 
in fiscal year 2007; obtained and reviewed library network policy and 
procedures; reviewed the agency's draft outline of a strategic plan 
for the library network; obtained and reviewed documents on EPA's 
digitization process; and reviewed EPA's efforts to communicate with 
and solicit input from users. We interviewed EPA librarians and 
library managers in selected EPA libraries, as well as Office of 
Environmental Information officials knowledgeable about EPA's library 
network and budget; when possible, we corroborated information 
provided to us during interviews with relevant documentation. We also 
interviewed management officials from the federal employees' union 
representing EPA staff and spoke with representatives from EPA's 
regional science councils, which consist of EPA scientists and 
technical specialists. We further sought information from library 
professionals, including representatives from the Library of Congress; 
the National Agriculture Library; and, through visits and interviews, 
from Lockheed Martin and Integrated Solutions and Services, 
contractors involved in digitizing EPA documents. 

In addition, we obtained information on library funding from each of 
the 26 libraries in the network from fiscal year 2002 to fiscal year 
2010. Because EPA does not specifically track funding for the 
libraries, the information provided contained a mix of outlays for 
some fiscal years and budget authority for other fiscal years. In 
addition, the information provided by each of the libraries reflected 
only spending by the library and not funding sources. For example, a 
large portion of funding for regional office libraries comes from the 
Office of Environmental Information, but these libraries also receive 
funding from other EPA program offices, such as Superfund. Also, 
funding data from the libraries contained a mix of funding for 
contract support; library staff salaries; and acquisition costs for 
books, journals, and other materials. We interviewed EPA budget 
officials to assess data reliability and performed a limited test to 
verify the accuracy and completeness of the data provided by the 
libraries. On the basis of this test and discussions with EPA 
officials, we concluded that the data were not reliable enough to 
include in our report. 

We also obtained data on the number of EPA and other documents that 
have already been digitized and the number still to be digitized. 
After limited testing and discussions with EPA officials, we 
determined that EPA's data on library funding and on the number of 
digitized documents and those scheduled to be digitized were not 
sufficiently reliable for our purposes. Because these data were the 
only data available, however, we used them to some extent, noting 
their limitations in our report as appropriate. We also reviewed 
documents about EPA's digitization process, guidance on what documents 
should or should not be digitized, and digitization contracts, and we 
discussed the contents of these documents with EPA and digitization-
contractor officials. We also discussed EPA's future digitization 
plans with Office of Environmental Information officials. 

In addition, we assessed EPA's survey of library users, examining the 
adequacy of the survey methodology, including response rate, sampling 
methodology, security measures, survey questions, and processes. To 
determine the adequacy of the response rate to EPA's survey, we 
followed an 80 percent response rate as a criterion, as Office of 
Management and Budget guidance recommends and we apply in our own 
surveys to increase the likelihood of sufficiently representing a 
universe of respondents. For surveys with response rates lower than 80 
percent, we also perform an analysis to determine the existence of 
nonresponse bias. To generate its survey sample, however, EPA relied 
on self-selection, using a Web site to make the survey available to 
approximately 17,000 EPA staff; the response rate achieved was 14 
percent. We performed a limited nonresponse analysis of EPA's survey 
data and determined that some staffing categories were represented in 
proportions different from those found in the population of EPA staff. 
Given the 14 percent response rate to EPA's survey, the nonrandom 
methodology that generated the sample, and the results of our limited 
analysis for nonresponse bias, we found EPA's survey results to be 
inadequate for EPA's purpose of obtaining a representative view of EPA 
library users. We also interviewed local union representatives from 
headquarters and some of EPA's regional offices. Furthermore, we 
interviewed regional science council representatives from some of the 
regional offices. The science councils are located in each regional 
office and consist of EPA scientists and technical specialists. To 
determine the extent to which EPA communicated with and solicited 
views from outside stakeholders, we interviewed representatives from 
several professional library associations and other external 
stakeholder groups, such as the American Library Association, the 
Library of Congress, the Federal Library and Information Center 
Committee, and the Union of Concerned Scientists. We also reviewed 
information EPA provided to the public via the EPA Web site or, when 
applicable, Federal Register notices. 

We conducted this performance audit from October 2009 through 
September 2010, in accordance with generally accepted government 
auditing standards. Those standards require that we plan and perform 
the audit to obtain sufficient, appropriate evidence to provide a 
reasonable basis for our findings and conclusions based on our audit 
objectives. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions based on our audit 
objectives. 

[End of section] 

Appendix II: Comments from the Environmental Protection Agency: 

United States Environmental Protection Agency: 
Office Of Environmental Information: 
Washington, D.C. 20460
[hyperlink, http://www.epa.gov] 
	
September 10, 2010 

Mr. John B. Stephenson: 
Director: 
Natural Resources and Environment: 
United States Government Accountability Office: 
Washington, DC 20548: 

Re: FPA Comments on the Government Accountability Office's (GAO) draft 
report to Congress entitled EPA Needs to Complete a Strategy for Its 
Library Network to Meet Users' Needs (GAO-l0-947): 

Dear Mr. Stephenson: 

This letter provides the U.S. Environmental Protection Agency's (EPA) 
comments on GAO's draft report entitled EPA Needs to Complete a 
Strategy for Its Library Network to Meet Users Needs (GA0-10-947). EPA 
appreciates the opportunity to review and comment on this report to 
Congress. 

EPA's focus over the past two years has been to restore confidence in 
our library network to deliver high quality library services to our 
staff and the public and to fulfill the commitments made in the 
Agency's February 14, 2008 letter to you. EPA believes that we have 
made great strides in all of the areas identified by the GAO in 2008, 
and we appreciate that our progress has been noted by GAO and others, 
including highly regarded members of the library profession. 

In considering your comments related to our strategic plan, we 
acknowledge that the current planning document available on our web 
site has provided more of a working agenda to guide the rebuilding of 
our library program. The outline and draft documentation has served as 
a living document to guide our library staff toward solid service 
delivery and enable them to explore innovative ways to benefit science 
and policy making at EPA. We believe that we have sufficient input and 
direction from our library users' survey, the exceptional 
collaboration and cooperation of the library community in meetings and 
conferences, the library support contractor and the Library Advisory 
Board to move forward. 

We agree, however, that having reached a new level of maturity in this 
library network, it is time to complete and publish a formal plan that 
identifies an overall network strategy, with implementation goals and 
timeline for future accomplishments. The Agency agrees to move forward 
with the strategic plan and to include the comments noted in the 
enclosure, which will be the major focus of our library team in FY 
2011. 

In addition to the strategic plan effort, we will address the 
inventory of library holdings, the feasibility and legality of 
digitizing products received by the Agency as the result of assistance 
agreements, and the design of any future user surveys in our follow-up 
to this report. 

In closing, EPA staff have benefited from this review and the 
thoughtful dialogue around the development of this report. We have 
addressed the specific recommendations individually, as an enclosure 
with this letter, to be included in the final report. Thank you again 
for the opportunity to comment. If you would like to discuss these 
matters further, please contact me at 202-564-6665 or Mr. Robin 
Gonzalez, Director of the Office of Information Analysis and
Access at 202-566-0600. 

Sincerely, 

Signed by: 

Malcolm D. Jackson: 
Assistant Administrator and Chief Information Officer: 

Enclosure: 

cc: 
Craig E. Hooks, Assistant Administrator, Office of Administration and 
Resources Management: 
David McIntosh, Associate Administrator, Office of Congressional and 
Intergovernmental Relations: 
Barbara J. Bennett, Chief Financial Officer: 
Scott Fulton, General Counsel: 
Assistant Regional Administrators: 

[End of letter] 

Enclosure: 
September 10, 2010: 
Mr. John B. Stephenson: 

Response to GAO Recommendations: EPA National Library Network: 

GAO Recommendation: 

Complete EPA's strategic plan for the library network, including 
implementation goals and timelines. In doing so, EPA should outline 
details for how funding decisions are to be made, to ensure that they 
are informed and transparent. 

EPA Response: 

EPA has made progress towards the goal of developing a strategic plan 
for the library network. The library network has engaged in many 
activities, both within and outside the agency. which has provided 
valuable information to inform the strategic planning process. These 
activities include developing and promulgating a library network 
policy and related procedures to establish standard operations across 
the network; conducting extensive and regular outreach with internal 
and external stakeholders, professional associations and federal 
partners about user information needs and library best practices; 
engaging with members of the library network through teleconferences 
and in-person meetings to discuss the business and goals of the 
network and related library services; enhancing library collections in 
response to user needs; exhibiting and participating in national 
library conferences as well as internal EPA conferences where users 
informally express their needs; implementing new services in response 
to user feedback, including live chat reference and critical 
electronic resources; planning for and implementing Phase II of the 
digitization project; and gathering user feedback through a 
centralized customer satisfaction feedback form. Additionally, EPA 
conducted a formal user needs assessment in FY2009. The findings, 
which correlate strongly with our ongoing, informal information 
gathering and communications, provide a valuable information framework 
upon which to base EPA's library strategic plan. EPA will develop a 
strategic plan for the library network using the needs assessment 
findings alongside the additional data gathered from users and other 
activities. The network has established a core work group of federal 
librarians who have committed to participate in the strategic planning 
process. This group will work together with the National Program 
Manager and other stakeholders to complete the strategic plan in FY11. 

GAO Recommendation: 

Complete an inventory of the library network's holdings to identify 
what items remain to be digitized. 

EPA Response: 

EPA agrees with this recommendation and has undertaken steps necessary 
to inventory library holdings. The library network has identified 
cataloging of library materials as a priority for individual 
libraries. The AWBERC Library in Cincinnati is providing cataloging 
services for network libraries at no cost. To date, 18 network 
libraries have taken advantage of this service, resulting in the 
cataloging of more than 8,000 additional items. This process will 
enable EPA to more accurately estimate the remaining items to be 
digitized. EPA will complete a schedule for cataloging the inventory 
by November I, 2010 and will strive to complete the cataloging by 
September 30, 2011. In addition, the library network has developed 
library procedures that outline requirements for verifying the 
completeness of collection inventories on a regular basis and is 
implementing the procedures. 

GAO Recommendation: 

For assistance agreements already in place, EPA should digitize 
documents produced under the agreements to make them more readily 
available to EPA employees and other users for federal government 
purposes. 

EPA Response: 

EPA recognizes that digitizing documents produced under assistance 
agreements would provide a useful resource to the Agency at large; 
however, the "federal purpose" license does not allow EPA to 
unilaterally make a financial assistance recipient's copyrighted 
material available to the public. As we have developed our 
digitization plans, we have focused on the unique documents held in 
network libraries, after which the Agency will be in a position to 
consider documents produced under assistance agreements.
Due to legal and technical constraints, EPA does not plan to digitize 
products from ongoing assistance agreements. As noted below, we will 
develop options for gaining permission from future agreement 
recipients to digitize products from their agreements. 

GAO Recommendation: 

In future assistance agreements, make explicit that EPA can include in 
the agency's public online database, without obtaining prior 
permission from the copyright holder, any documents produced under the 
agreements. 

EPA Response: 

Over the next six months, the Office of Environmental Information 
(0EI), working with the Office of Administration and Resources 
Management (OARM) and the Office of General Counsel (OGC) will develop 
options for gaining advance permission to digitize products from 
future assistance agreements. These options will be taken to Senior 
Agency managers in mid-2011 for their consideration and action. 

GAO Recommendation: 

For future assistance agreements where EPA cannot make such an 
arrangement, EPA should digitize documents produced under the 
agreements to make them more readily available to EPA employees and 
other users for federal government purposes. 

EPA Response: 

As noted above, this issue will be among the options evaluated by OEI, 
OARM and OGC. 

GAO Recommendation: 

Ensure that the data analysis protocols used for conducting surveys of 
users' needs — including sampling procedures and response rates — are 
sufficiently sound methodologically to provide reliable information on 
which to base decisions and allocate resources efficiently. 

EPA Response: 

EPA recognizes the concerns of GAO related to the methodology used in 
the recent needs assessment. We believe that we have sufficient 
information from the survey and our continuing feedback from users, 
the Library Advisory Board, and the library network staff to enable us 
to develop an effective strategic plan for the library network. The 
needs assessment survey was supplemented by a series of six focus 
group sessions with key user group segments and seven one-on-one in-
depth interviews with management level stakeholders. These additional 
venues focused on the deeper issues and learnings from the 
quantitative survey and were done to help ensure a comprehensive 
perspective of user needs. Additionally, the findings of the needs 
assessment correlate with other data gathered on an ongoing basis from 
library users, further suggesting we have valid information on which 
to base such a plan. EPA will take the steps necessary to ensure that 
any future assessments are done with data analysis protocols that are 
methodologically sound and provide reliable information for future 
decision making. 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

John Stephenson, (202) 512-3841 or stephensonj@gao.gov: 

Staff Acknowledgments: 

In addition to the contact above, Ed Kratzer, Assistant Director; 
Ellen W. Chu; Pamela Davidson; Les Mahagan; John C. Martin; Ben 
Shouse; and Jeannette Soares made key contributions to this report. 

[End of section] 

Footnotes: 

[1] GAO, Environmental Protection: EPA Needs to Ensure That Best 
Practices and Procedures Are Followed When Making Further Changes to 
Its Library Network, [hyperlink, 
http://www.gao.gov/products/GAO-08-304] (Washington, D.C.: Feb. 29, 
2008). 

[2] Reorganization Plan No. 3 of 1970, 35 Fed. Reg. 15623 (Dec. 2, 
1970) (5 U.S.C. Appendix 1). 

[3] A repository library is a central place that collects and 
preserves EPA documents and other materials deemed of value to the 
library network. EPA has designated three libraries as repository 
libraries, each with a subject specialty: the Headquarters Library for 
waste, pesticides, regional or local environmental topics deemed 
important to staff, and other topics not included elsewhere; the 
Research Triangle Park Library in Research Triangle Park, North 
Carolina, for air and health effects; and the Andrew W. Breidenbach 
Environmental Research Center in Cincinnati, Ohio, for water and risk 
assessment materials. 

[4] [hyperlink, http://www.gao.gov/products/GAO-08-304]. 

[5] Environmental Protection Agency, EPA National Library Network 
Report to Congress (Washington, D.C., 2008). EPA prepared and issued 
this report in response to a directive in the explanatory statement 
accompanying the fiscal year 2008 Consolidated Appropriations Act. 

[6] GAO, Results-Oriented Cultures: Implementation Steps to Assist 
Mergers and Organizational Transformations, [hyperlink, 
http://www.gao.gov/products/GAO-03-669] (Washington, D.C.: July 2, 
2003). 

[7] Environmental Protection Agency, "Draft Annotated Outline for the 
EPA Library Strategic Plan," Washington, D.C., July 25, 2007. 

[8] [hyperlink, http://www.gao.gov/products/GAO-03-669]. 

[9] These spending estimates, expressed in constant 2010 dollars, came 
from individual libraries in EPA's network and were provided to us by 
EPA. Aside from limited testing of the data from one region, we did 
not independently verify the reliability of the estimates. 

[10] Over the past decade, the Office of Environmental Information has 
provided from 41 to 47 percent of the library network's funding, 
except for fiscal years 2007 and 2008, when, because of reductions in 
its budget, the office provided 34 percent. In 2008, library spending 
was $7.3 million, reflecting $1 million added by Congress, and in 
fiscal year 2009, EPA increased library funding to $8.3 million. 

[11] Environmental Protection Agency, EPA National Library Network 
Report. 

[12] In our 2008 report (GAO-03-804, 36), we observed that it was 
essential that top leadership drive transformation of the library 
network and that without a national manager for the library network, 
EPA had no official providing essential oversight and guidance. 

[13] This board of advisors was established in response to a request 
from EPA for assistance in improving current and future library 
operations. Created in 1965, the Federal Library and Information 
Center Committee comprises the directors of the four national 
libraries--Library of Congress, National Library of Medicine, National 
Library of Education, and National Agriculture Library--and 
representatives of cabinet-level executive departments, as well as 
legislative, judicial, and independent federal agencies with major 
library programs; it is chaired by the Librarian of Congress. The 
committee's mission is to foster excellence in federal library and 
information services through interagency cooperation and to provide 
guidance and direction for the Federal Library and Information Network. 

[14] EPA defines a unique document as a document published by EPA or 
on behalf of EPA that exists in only one copy within EPA's National 
Library Network and that is not already electronically available in 
NEPIS. 

[15] Copyright in works arising under contracts is governed by Subpart 
27.4 of the Federal Acquisition Regulation (FAR) (48 C.F.R. §§ 27.400- 
27.409). FAR Data Rights Clause No. 52.227-14, Rights in Data--General 
(48 C.F.R. § 52.227-14), is found in most EPA research and development 
contracts. 

[16] 40 C.F.R. § 31.34 (copyrights in works produced under grants and 
cooperative agreements to state and local governments); 40 C.F.R. § 
30.36 (copyrights in works produced under grants and cooperative 
agreements to institutions of higher education, hospitals, and other 
nonprofits). 

[17] Environmental Protection Agency, Digitization of EPA Library 
Materials (Washington, D.C., 2009). 

[18] The Federal Library and Information Network is the business 
subsidiary of the Federal Library and Information Center Committee and 
serves federal libraries and information centers as their purchasing, 
training, and resource-sharing consortium. 

[19] In July 2008, after a February 2008 arbitration ruling, EPA 
management and representatives of its employees' union entered into a 
memorandum of agreement to resolve concerns surfaced during the 
reorganization. 

[20] Environmental Protection Agency, Target User Accessibility 
Review: NSCEP/NEPIS: EPA's Gateway to Free Digital and Paper 
Publications (Washington, D.C., 2008). 

[21] These efforts were aided by discussions with staff from other 
federal libraries, including the National Agricultural Library and the 
Library of Congress. 

[22] [hyperlink, http://www.gao.gov/products/GAO-08-304]. 

[23] Office of Management and Budget, Standards and Guidelines for 
Statistical Surveys, Washington, D.C., 2005, and Office of Management 
and Budget, Guidance on Agency Survey and Statistical Information 
Collections, Washington, D.C., 2006. 

[24] According to the Office of Management and Budget's guidance, a 
low response rate may indicate an error known as nonresponse bias, 
which can significantly lower the accuracy of survey results. Such 
bias may occur if survey participants differ substantially and 
systematically from nonparticipants in ways that might influence their 
responses to survey questions. Similarly, surveys with few respondents 
may also be biased because they may fail to capture the true range of 
variability in the surveyed population. 

[25] According to EPA estimates, 20 to 40 percent of the reference 
requests received by regional libraries come from the public. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: