Managing for Results: Greater Transparency Needed in Public Reporting on the Quality of Performance Information for Selected Agencies' Priority Goals
GAO-15-788
Published: Sep 10, 2015. Publicly Released: Sep 10, 2015.
Skip to Highlights
Highlights
What GAO Found
The six agencies GAO reviewed generally did not publicly report on how they ensured the accuracy and reliability of performance information used to measure progress on their highest priority performance goals, referred to as agency priority goals (APGs). The GPRA Modernization Act of 2010 (GPRAMA) requires agencies to identify the following when publicly reporting on their APGs: 1) how performance information was verified and validated; 2) data sources; 3) level of accuracy required for intended use; 4) any limitations at the required level of accuracy; and 5) how the agency will compensate for such limitations (if needed) to reach the required level of accuracy. GPRAMA requires agencies to provide this information to the Office of Management and Budget (OMB) for publication on Performance.gov. GPRAMA also directs agencies to provide this information for performance goals, which include APGs, in their annual performance plans and reports. While all six agencies described how they ensured the quality of their performance information overall, GAO found discussions about performance information quality addressing all five GPRAMA requirements in only the Department of Homeland Security's (DHS) performance plans and reports.
|
Agency Priority Goal (APG) Performance Information Quality Discussion in Selected Agencies' Performance Plans and Reports for FY13-FY16 |
|||
|
Agency |
Description of how agency ensured performance information quality overall |
Number of APGs for FY14 and FY15 |
Description of how each APG met GPRAMA performance information quality requirements |
|
Agriculture |
Yes |
3 |
0 |
|
Defense |
Yes |
4 |
0 |
|
Homeland Security |
Yes |
3 |
3 |
|
Interior |
Yes |
6 |
0 |
|
Labor |
Yes |
3 |
0 |
|
National Aeronautics and Space Administration (NASA) |
Yes |
4 |
0 |
|
Total |
6 yes |
23 |
3 |
Source: GAO analysis of selected agencies' performance plans and reports. |GAO-15-788
OMB and the Performance Improvement Council (PIC)–a cross-agency council of agency performance improvement officers–established the Data Quality Cross-Agency Working Group in February 2015. The group has identified several goals, such as improving the reliability and quality of performance information, and could serve as a vehicle for disseminating good practices in public reporting on data quality.
Why GAO Did This Study
Federal agencies have not always clearly and transparently explained to Congress and the public how they ensure the quality of their performance information. GPRAMA requires agencies to publicly explain how they ensure the accuracy and reliability of their performance information used to assess progress for their APGs.
This is one of a series of GAO reports examining the implementation of GPRAMA, as required by the act. This report assesses how well selected agencies publicly reported on the quality of performance information used to measure progress on APGs. GAO selected six agencies–the Departments of Agriculture, Defense, Interior, and Labor, and NASA and DHS– based on GAO's 2013 federal managers survey on their agency's use of performance information. GAO reviewed information concerning these agencies' APGs published on Performance.gov and in their annual performance plans and reports.
Recommendations
GAO recommends that all six of the agencies work with OMB to describe on Performance.gov how they are ensuring the quality of their APGs's performance information, and that all agencies, except for DHS, also describe this information in their annual performance plans and reports. GAO also recommends that OMB, working with the PIC, focus on ways the PIC's data quality working group can improve public reporting for APGs. OMB did not comment on the recommendations, but the six agencies generally concurred or identified actions they planned to take to implement them.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Homeland Security | To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs. |
In response to our recommendation, DHS added an overall description on Performance.gov of how the agency ensures reliable performance information is reported to external audiences. DHS's explanation refers to the data quality appendix in its performance plan and report covering fiscal years 2015-2017 and a hyperlink is provided from Performance.gov to the DHS website where this appendix can be found. Further, DHS also made it easier to locate information concerning its APGs in this data quality appendix by consolidating the relevant discussion in one section.
|
| Department of Agriculture | To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs. |
We regularly followed up with USDA officials, met with them in July 2022, and provided feedback on proposed updates to the data quality discussions on Performance.gov for USDA's three fiscal year 2022-2023 APGs. These three APGs are (1) monitoring antimicrobial resistance in animal health pathogens through the National Animal Health Laboratory Network; 2) enhancing customer experience and improve trust; and 3) ensuring all Americans have access to high-speed, affordable broadband. In September 2022, updated data quality information for USDA's three fiscal year 2022-2023 APGs was posted on Performance.gov. Consistent with our recommendation, these updates clearly demonstrated how USDA is addressing each of the five data quality requirements for its three APGs.
|
| Department of Defense | To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs. |
In response to our recommendation, a DOD official reported in April 2021 that they had provided the required data quality information to OMB to be posted on Performance.gov before the change in administrations. DOD's APG updates for the fourth quarter of fiscal year 2020 explained how DOD is addressing each of the five data quality requirements for its three APGs for fiscal years 2020 and 2021 . This information is posted within the archives of President Trump's Administration's performance and management agenda on Performance.gov .
|
| Department of the Interior | To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs. |
Interior concurred with our recommendation and said it would add information to Performance.gov. Interior officials met with us in February 2022 to discuss this recommendation, and in the following months we provided feedback to these officials on how Interior could fully address these GPRAMA requirements on Performance.gov. In June and July 2022 progress updates to Performance.gov, Interior provided updated data quality information for the agency's three APGs for fiscal years 2022 and 2023. These include priority goals to 1) improve tribal land into trust processing; 2) facilitate safe and environmentally responsible offshore wind energy development; and 3) enhance water conservation and supply. Interior's data quality information clearly demonstrated how the agency is addressing each of the five data quality requirements for these three APGs.
|
| Department of Labor | To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs. |
In response to our recommendation, the Department of Labor (Labor) added data quality descriptions to Performance.gov for four of its five agency priority goals (APGs) for fiscal years 2018 and 2019 that explain how Labor ensures the accuracy and reliability of the performance information used to measure progress toward each APG. For Labor's remaining APG to begin implementation of a shared services model for administrative functions, the agency identifies data sources on Performance.gov and its performance report for fiscal year 2018 (which can be accessed through Performance.gov ) provides additional data quality discussion addressing the remaining four requirements
|
| National Aeronautics and Space Administration | To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Homeland Security, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by working with OMB to describe on Performance.gov how they are ensuring the quality of performance information used to measure progress towards their APGs. |
In response to our recommendation, NASA added data quality descriptions to Performance.gov for each of its four APGs for fiscal years 2016 and 2017 that explain how NASA tracks progress toward each APG, consistent with GPRAMA requirements.
|
| Department of Agriculture | To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs. |
In 2015, we found that the U.S. Department of Agriculture (USDA) had not fully addressed requirements in the GPRA Modernization Act of 2010 to report on the following data quality information for its annual priority goals (referred to as APGs) in its performance plan and report: 1) the means used to verify and validate [performance data]; 2) the sources for the data; 3) the level of accuracy required for the intended use of the data; 4) any limitations to the data at the required level of accuracy; and 5) how the agency will compensate for such limitations (if needed) to reach the required level of accuracy. While USDA did not comment on this recommendation, a USDA official stated that the agency would work with the Office of Management and Budget to put the required information on Performance.gov with a reference to this website in its annual performance plan. We periodically followed up with USDA officials, and in June and July of 2022 met with officials to discuss how agencies could address these requirements. In September 2022, updated data quality information for USDA's three APGs for fiscal years 2022 and 2023 was posted on Performance.gov clearly addressing each of the five data quality requirements. In March 2023. USDA published its Fiscal Year (FY) 2024 Annual Performance Plan and FY 2022 Annual Performance Report and FY 2023 Performance Plan. USDA's annual performance plan and report direct the public to Performance.gov for the required data quality explanation for each APG.
|
| Department of Defense | To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs. |
In response to our recommendation, a DOD official reported in April 2021 that DOD had provided the required data quality information to OMB to be posted on Performance.gov before the change in administrations. DOD's APG updates for the fourth quarter of fiscal year 2020 were posted on Performance.gov and explained how DOD is addressing each of the five data quality requirements for its three APGs for fiscal years 2020 and 2021. This information is posted within the archives of President Trump's Administration's performance and management agenda on Performance.gov. In regards to the recommendation to provide this information in its annual performance plan and report, DOD's performance plan for fiscal year 2021 and annual performance report for fiscal year 2019, as well as its performance report for fiscal year 2020, describe DOD's APGs and refer readers to Performance.gov for updates, which include the required data quality information.
|
| Department of the Interior | To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs. |
We periodically followed up with Interior officials and in May 2022 provided suggestions on how officials could more fully address the requirements on Performance.gov and in their annual performance plans and reports. In June and July 2022, updated data quality information for Interior's three APGs for fiscal years 2022 and 2023 was posted on Performance.gov, clearly addressing each of the five data quality requirements. In May 2023, Interior posted on its website its performance plan for fiscal years 2022 and 2023, and performance report for fiscal year 2021, directing the public to Performance.gov for more specific information about the APGs, including the data quality explanation.
|
| Department of Labor | To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs. |
The Department of Labor's (Labor) performance report for fiscal year 2018 (which also serves as its performance plan for fiscal year 2020) refers readers interested in its agency priority goals (APGs) to Performance.gov. In response to our recommendation, Labor added on Performance.gov data quality descriptions for four of its five APGs for fiscal years 2018 and 2019. For Labor's remaining APG to begin implementation of a shared services model for administrative functions , the agency identifies data sources on Performance.gov and its performance report for fiscal year 2018 provides additional data quality discussion for specific measures related to this APG which address the remaining requirements
|
| National Aeronautics and Space Administration | To improve the public reporting about how agencies are ensuring the quality of performance information used to measure progress towards their priority goals, the Secretaries of Agriculture, Defense, Interior, and Labor, and the Administrator of NASA should more fully address GPRAMA requirements and OMB guidance by describing in their agencies' annual performance plans and reports how they are ensuring the quality of performance information used to measure progress towards their APGs. |
In response to our recommendation, NASA added a data quality explanation to the section of its FY2015 Annual Performance Report/FY2017 Annual Performance Plan that describes how NASA tracks progress toward each of its four APGs for fiscal years 2016 and 2017, consistent with GPRAMA requirements.
|
| Office of Management and Budget | To help participating agencies improve their public reporting, the Director of OMB, working with the PIC Executive Director, should identify additional changes that need to be made in OMB's guidance to agencies related to ensuring the quality of performance information for APGs on Performance.gov. |
In July 2018, OMB staff shared with us a reporting template for Performance.gov for the fiscal years 2018 and 2019 APGs. This template allows agencies to provide information on data accuracy and reliability for individual APGs and agencies may provide hyperlinks to more detailed data quality discussions published in their annual performance plans and reports.
|
| Office of Management and Budget | To help participating agencies improve their public reporting, the Director of OMB, working with the PIC Executive Director, should identify practices participating agencies can use to improve their public reporting in their performance plans and reports of how they are ensuring the quality of performance information used to measure progress towards APGs. |
In addition to highlighting the need for better public reporting, our review also focused more attention on agencies' internal processes for verifying and validating performance information. In July 2015, we briefed the Performance Improvement Council's (PIC) Data Quality Cross-Agency Working Group on leading practices identified in our prior work for ensuring agencies are accurately verifying, validating, and reporting performance information, and shared GAO's performance and accountability report which describes our own data quality processes. In February 2016, the PIC Executive Director directed Performance Improvement Officers (PIOs) and their deputies to complete a self-assessment of their agency's data quality policies and procedures to ensure these were consistent with OMB guidance. PIC staff summarized the agencies' self-assessments at the March 2016 meeting of the PIOs for the Chief Financial Officers Act agencies. We reviewed presentation notes which confirmed that PIC staff had addressed our recommendation by summarizing for the PIOs the results of these self-assessments, identifying aspects of data quality in which agencies had generally rated their performance highest, and other aspects of data quality in which agencies had rated their performance lowest.
|
Full Report
GAO Contacts
Topics
Data integrityFederal agenciesGovernment operationsPerformance measuresPrioritizingReporting requirementsPerformance goalsTransparencyPerformance plansPerformance measurement