Office of Personnel Management:

Agency Needs to Improve Outcome Measures to Demonstrate the Value of Its Innovation Lab

GAO-14-306: Published: Mar 31, 2014. Publicly Released: Apr 30, 2014.

Additional Materials:

Contact:

Seto J. Bagdoyan
(202) 512-4749
bagdoyans@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

In March 2012, the Office of Personnel Management (OPM) opened its innovation lab, a distinct physical space with a set of policies for engaging people and using technology in problem solving. The goals of OPM's innovation lab are to provide federal workers with 21st century skills in design-led innovation, such as intelligent risk-taking to develop new services, products, and processes. OPM's lab was built at a reported cost of $1.1 million, including facility upgrades and construction, equipment and training, and other personnel costs. The lab employs approximately 6 full-time equivalents, including a director, and in fiscal year 2013, the lab's operating costs were approximately $476,000, including salaries.

View of OPM's Innovation Lab

View of OPM's Innovation Lab

OPM's innovation lab is similar in mission and design to other innovation labs GAO reviewed, and OPM has incorporated some of the prevalent practices that other labs use to sustain their operations. Specifically, OPM is using its lab for a variety of projects, including as a classroom for building the capacity to innovate in the federal government. Lab staff indicated that they plan to begin long-term immersion projects—complex projects with diverse users—within a few months. OPM plans to develop and implement evaluation plans specific to each immersion project that will help them track cost benefits or performance improvement benefits associated with the projects.

Starting in March 2013, OPM lab staff began work on a program evaluation framework to more systematically measure the lab's progress toward meeting its overarching goals. In addition, lab staff members are tracking lab activities, such as classes and workshops, and are surveying lab users about the quality of their experience in the lab. However, they have not developed performance targets or measures related to project outcomes, and without a rigorous evaluation framework that can help OPM track the lab's performance, it will be hard to demonstrate that the lab is operating as originally envisioned.

While labs provide a physical space where innovators can convene, federal agencies are not fully aware of their growing community. However, OPM is taking steps to ensure work done in the lab is shared across OPM and with other federal innovators—for example, by hosting weekly training sessions in the lab on best practices. Studies show that information sharing and interorganizational networks can be a powerful driver supporting innovation.

Why GAO Did This Study

Organizations from around the globe are emphasizing that strategies promoting innovation are vital to solving complex problems. To try to instill a culture of innovation in its agency, OPM followed the lead of a number of private sector companies, nonprofit organizations, and government bodies by creating an innovation lab. GAO was asked to examine the lab.

Specifically, GAO 1) described the lab's start-up costs, staffing and organization, activities, and policies governing the lab's use, and 2) assessed how OPM's innovation lab compares to other organizations' innovation labs, including how it uses benchmarks and metrics and how it addresses challenges to innovation. GAO reviewed cost, staffing, and performance information. GAO also reviewed relevant literature on innovation and interviewed officials from public, private, and nonprofit organizations with innovation facilities similar to OPM's lab.

What GAO Recommends

Among other things, GAO recommends that the Director of OPM should direct lab staff to 1) develop a mix of performance targets and measures to help them monitor and report on progress toward lab goals, and 2) build on existing efforts to share information with other agencies that have innovation labs. OPM generally concurred with GAO's recommendations; in addition, they described the steps being taken and planned to refine their ongoing evaluation efforts and to further leverage other federal innovation labs.

For more information, contact Seto J. Bagdoyan at (202) 512-4749 or bagdoyans@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To help substantiate the lab's original goals of enhancing skills in innovation and supporting project-based problem solving, the Director of OPM should direct lab staff to develop a mix of performance targets and measures to help them monitor and report on their progress toward lab goals. Output targets could include number and type of lab activities over the next year. Outcome targets and measures should correspond to the lab's overarching goals to build organizational capacity to innovate and achieve specific innovations in concrete operational challenges.

    Agency Affected: Office of Personnel Management

  2. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To help substantiate the lab's original goals of enhancing skills in innovation and supporting project-based problem solving, the Director of OPM should direct lab staff to review and refine the set of survey instruments to ensure that taken as a whole, they will yield data of sufficient credibility and relevance to indicate the nature and extent to which the lab is achieving what it intends to accomplish or is demonstrating its value to those who use the lab space. For example, lab staff should consider the following actions: (1) Developing a standard set of questions across all service offerings. (2) Revising the format and wording of existing questions related to skills development to diminish the likelihood of social desirability bias and use post-session questions that ask, in a straight-forward way, about whether, or the extent to which, new information was acquired. (3) Replacing words or phrases that are ambiguous or vague with defined or relevant terminology (e.g., terms actually used in the session) so that the respondent can easily recognize a link between what is being asked and the content of the session.

    Agency Affected: Office of Personnel Management

  3. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To help substantiate the lab's original goals of enhancing skills in innovation and supporting project-based problem solving, the Director of OPM should direct lab staff to build on existing efforts to share information and knowledge within the federal innovation community. For example, OPM lab staff could reach out to other agencies with labs such as Census, the Department of Housing and Urban Development, and the National Aeronautics and Space Administration's Kennedy Space Center to share best practices and develop a credible evaluation framework.

    Agency Affected: Office of Personnel Management

 

Explore the full database of GAO's Open Recommendations »

Oct 20, 2014

Oct 9, 2014

Oct 8, 2014

Oct 2, 2014

Sep 30, 2014

Sep 26, 2014

Sep 25, 2014

Sep 19, 2014

Looking for more? Browse all our products here