Designing Evaluations: 2012 Revision (Supersedes PEMD-10.1.4)

This publication supersedes PEMD-10.1.4, Designing Evaluations, May 1, 1991. GAO assists congressional decision makers in their deliberations by furnishing them with analytical information on issues and options. Many diverse methodologies are needed to develop sound and timely answers to the questions the Congress asks. To provide GAO evaluators with basic information about the more commonly used methodologies, GAO’s policy guidance includes documents such as methodology transfer papers and technical guides.

This methodology transfer paper addresses the logic of program evaluation designs. It introduces key issues in planning evaluation studies of federal programs to best meet decision makers’ needs while accounting for the constraints evaluators face. It describes different types of evaluations for answering varied questions about program performance, the process of designing evaluation studies, and key issues to consider toward ensuring overall study quality.

To improve federal program effectiveness, accountability and service delivery, the Congress enacted the Government Performance and Results Act of 1993 (GPRA), establishing a statutory framework for performance management and accountability, including the requirement that federal agencies set goals and report annually on progress towards those goals and program evaluation findings. In response to this and related management reforms, federal agencies have increased their attention to conducting program evaluations. The GPRA Modernization Act of 2010 raised the visibility of performance information by requiring quarterly reviews of progress towards agency and governmentwide priority goals. Designing Evaluations is a guide to successfully completing evaluation design tasks. It should help GAO evaluators—and others interested in assessing federal programs and policies—plan useful evaluations and become educated consumers of evaluations.

Designing Evaluations is one of a series of papers whose purpose is to provide guides to various aspects of audit and evaluation methodology and indicate where more detailed information is available. It is based on GAO studies and policy documents and program evaluation literature. To ensure the guide’s competence and usefulness, drafts were reviewed by selected GAO, federal and state agency evaluators, and evaluation authors and practitioners from professional consulting firms. This paper updates a 1991 version issued by GAO’s prior Program Evaluation and Methodology Division. It supersedes that earlier version and incorporates changes in federal program evaluation and performance measurement since GPRA was implemented.