Performance Partnerships: Agencies Need to Better Identify Resource Contributions to Sustain Disconnected Youth Pilot Programs and Data to Assess Pilot Results
Fast Facts
Performance partnership initiatives allow federal agencies to give their grant recipients flexibility (such as in how they use funding) in exchange for assessing if outcomes improve. There are currently two such initiatives—one involves EPA's environmental grant programs and the other helps youth who are disconnected from school and work.
We found that both initiatives reduce administrative burdens and allow for flexibility in using grant funding. However, we recommended that the disconnected youth initiative better identify (1) resources that agencies will need to contribute to sustain the effort, and (2) data to assess pilot results.
Photo of EPA headquarters building.
Highlights
What GAO Found
Congress has authorized 2 federal performance partnership initiatives. The Environmental Protection Agency's (EPA) Performance Partnership Grants (PPG) has been in place for 20 years and allows state agencies to consolidate funds from up to 19 environmental program grants into a single PPG. The other, Performance Partnership Pilots for Disconnected Youth (disconnected youth pilots), is a more recent initiative authorized in 2014 that allows funding from multiple programs across multiple agencies to be combined into pilot programs serving disconnected youth. GAO identified 4 key characteristics shared by the 2 federal performance partnership initiatives. Specifically:
1. Documented agreement . Federal and non-federal partners identify goals, roles, and responsibilities. EPA and state agencies accomplish this through a PPG work plan. For each disconnected youth pilot, multiple federal agencies and non-federal partners, such as local government agencies and community-based organizations, use a performance partnership agreement.
2. Flexibility in using funding. PPGs combine funding from 2 or more EPA program grants. The disconnected youth pilots can combine funding from multiple programs across the agencies involved in the initiative.
3. Additional flexibilities. PPGs reduce administrative burden for state agencies, for example, by requiring only a single application for all grants in them. Disconnected youth pilots also provide non-federal partners flexibility to serve disconnected youth, including the ability to better tailor service interventions to their target populations.
4. Accountability for results . In both initiatives, non-federal partners report to federal partners on progress towards mutually-established goals. Partners in the disconnected youth pilots are also assessing results through national and pilot-specific program evaluations.
GAO's in-depth review of the disconnected youth pilots found that agencies had taken actions consistent with leading practices for collaboration and pilot design, such as establishing a leadership model for collaboration. Although the Office of Management and Budget (OMB) is responsible for coordinating agencies' overall efforts to implement the pilots, GAO identified additional actions that OMB should take in coordination with the agencies to help ensure future success.
Resources. Agencies have not fully identified the funding and staff resources each will need to contribute to sustain their efforts over the lifetime of the pilots. This is because agencies primarily have been focused on meeting near-term needs to support design and implementation. By fully identifying specific future financial and staff resource needs, agencies can better plan for their individual contributions to ensure they are sufficient to support the pilots.
Scalability. Agencies have not developed criteria to inform determinations about whether, how, and when to implement the flexibilities tested by the pilots in a broader context (this is known as scalability). Although the agencies identified a variety of data to collect, they have not identified criteria for assessing scalability. Officials involved in the pilots told GAO it was too early in pilot implementation to determine such criteria. By not identifying these criteria during the design of the pilots, they risk not collecting needed data during their implementation.
Why GAO Did This Study
The GPRA Modernization Act of 2010 established a framework intended to increase federal agencies' use of performance information and evidence in decision making. In performance partnerships, agencies and grant recipients manage toward outcomes, which can help measure program performance and collect evidence about what works to achieve desired outcomes. OMB has encouraged the use of such partnerships by agencies that make federal grants.
GAO is required by the act to report on how its implementation is affecting federal agency performance management. This report identifies the key characteristics of 2 existing performance partnerships. It also provides an in-depth review of the design, implementation, and evaluation of 1 of the 2 initiatives—the disconnected youth pilots.
To address these objectives, GAO reviewed relevant laws, regulations, and documents and selected 8 illustrative examples from the 2 partnership initiatives (4 each), based on various criteria, such as the type and number of grants included and location. GAO also interviewed federal and non-federal officials involved in these partnerships.
Recommendations
GAO recommends that OMB coordinate with federal agencies implementing the disconnected youth pilots to identify (1) agency resource contributions needed for the lifetime of the pilots and (2) criteria and related data for assessing scalability. OMB neither agreed nor disagreed with these recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of Management and Budget | To help ensure that the pilot programs for disconnected youth can be effectively implemented over the lifetime of the initiative, the Director of OMB should coordinate with relevant federal agencies to identify and estimate expected annual financial and staff resource contributions from each agency, including during the implementation and evaluation phases of the pilots. |
As of May 2021, the Office of Management and Budget (OMB) and relevant federal agencies have taken actions to address the intent of this recommendation. Since we issued our report in April 2017, federal agencies discontinued contributing financial resources to support joint pilot program activities, such as start-up grants, general technical assistance, or evaluations. According to OMB staff and officials at several agencies in August 2018, that change in contributions began with the fiscal year 2018 pilots. At that time, they all told us that relevant agencies would continue to provide staff support to the pilots, as needed. In November 2020, officials from the Department of Education (Education)--the lead agency for overall performance partnership efforts--informed us that two pilots would remain active until September 2021, with Education and the Department of Labor (Labor) contributing limited staff resources. Officials from both departments provided us with information in October and November 2020 about their expected staff resource contributions. An Education staff member was expected to spend an estimated 10 percent of their time monitoring implementation of the remaining two pilots. Similarly, several Labor staff were expected to spend a portion of their time continuing to oversee the related pilot evaluations and leading biweekly coordinating calls. By fully identifying and planning for these resource contributions, the agencies have greater assurance that those contributions will sustain their collaborative efforts and the overall pilot initiative.
|
| Office of Management and Budget | To ensure that federal agencies involved in the disconnected youth pilots are able to evaluate pilot outcomes and ultimately communicate to Congress whether and to what extent the flexibilities tested by the pilots should be integrated into broader efforts, the Director of OMB should coordinate with relevant federal agencies to identify criteria or standards for assessing scalability, and collect data needed to address those criteria or standards. |
As of May 2021, the Office of Management and Budget (OMB) and relevant federal agencies have taken actions to address this recommendation. In August 2018, OMB staff told us that they were coordinating relevant work with the Department of Health and Human Services (HHS) and the Department of Labor (DOL), which is leading the national evaluation for the disconnected youth pilot programs. In October 2020, this group identified specific criteria for assessing the scalability of the disconnected youth pilot programs in a memorandum. In addition, in April 2020, DOL officials told us that the department was collecting data to examine issues related to scalability as part of the national evaluation. From August 2020 to April 2021, DOL published nine products evaluating the pilots and the overall performance partnership effort. Several of those products identified success factors for implementing a performance partnership model, and are summarized in "National Evaluation of the Performance Partnership Pilots for Disconnected Youth (P3): Summary of the Evaluation's Reports and Findings" (dated February 2021 and made available on DOL's website in April 2021). For example, those factors include dedicating planning time for identifying and addressing needed systems change with partners, and providing sufficient guidance and technical assistance related to allowed flexibilities. This provides information decision makers need when considering if, when, and how to continue and expand these performance partnership efforts.
|