This is the accessible text file for GAO report number GAO-14-526 
entitled 'Managing For Results: OMB Should Strengthen Reviews of Cross-
Agency Goals' which was released on June 10, 2014.  

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO:  

Report to Congressional Addressees:  

June 2014:  

Managing For Results:  

OMB Should Strengthen Reviews of Cross-Agency Goals:  

GAO-14-526:  

GAO Highlights:  

Highlights of GAO-14-526, a report to congressional addressees.  

Why GAO Did This Study:  

The federal government faces complex, high-risk challenges, such as 
protecting our nation's critical information systems. Effectively 
managing these challenges is essential for national and economic 
security and public health and safety. However, responsibility for 
addressing these challenges often rests with multiple agencies. To 
effectively address them, shared goals and cross-agency collaboration 
are fundamental.  

This report responds to GAO's mandate to evaluate the implementation 
of GPRAMA. It assesses (1) what is known about progress made towards 
the interim CAP goals; and (2) how, if at all, quarterly progress 
reviews reflected GPRAMA requirements and leading practices for 
reviews, as well as how reviews contributed to improved cross-agency 
performance and collaboration. To address these objectives, GAO 
analyzed CAP goal status updates and other documents from OMB and CAP 
goal progress-review meetings, and interviewed OMB staff and CAP goal 
representatives. GAO compared this information to GPRAMA requirements 
and to leading practices for performance reviews previously reported 
on by GAO.  

What GAO Found:  

CAP Goal Progress. The GPRA Modernization Act of 2010 (GPRAMA) 
requires the Office of Management and Budget (OMB) to coordinate with 
agencies to: (1) establish outcome-oriented, federal government 
priority goals (known as cross-agency priority, or CAP, goals) with 
annual and quarterly performance targets and milestones; and (2) 
report quarterly on a single website now known as Performance.gov the 
results achieved for each CAP goal compared to the targets. In 
February 2012, OMB identified 14 interim CAP goals and subsequently 
published five quarterly updates on the status of the interim CAP 
goals on Performance.gov. While updates for eight of the goals 
included data that indicated performance towards an overall planned 
level of performance, only three also contained annual or quarterly 
targets that allowed for an assessment of interim progress. Updates 
for the other six goals did not report on progress towards a planned 
level of performance because the goals lacked either a quantitative 
target or the data needed to track progress. The updates on 
Performance.gov also listed planned activities and milestones 
contributing to each goal, but some did not include relevant 
information, including time frames for the completion of specific 
actions and the status of ongoing efforts. The incomplete information 
in the updates provided a limited basis for ensuring accountability 
for the achievement of targets and milestones.  

OMB Quarterly Progress Reviews. GPRAMA also requires that OMB—-with 
the support of the Performance Improvement Council (PIC)-—review CAP 
goal progress quarterly with goal leaders. OMB instituted processes 
for reviewing progress on the goals each quarter, which involved the 
collection of data from goal leaders and the development of a 
memorandum for the OMB Director. However, the information included in 
these memorandums was not fully consistent with GPRAMA requirements. 
For example, GPRAMA requires OMB to identify strategies for improving 
the performance of goals at risk of not being met, but this was not 
consistently done. Without this information, OMB leadership and others 
may not be able to adequately track whether corrective actions are 
being taken, thereby limiting their ability to hold officials 
accountable for addressing identified risks and improving performance.  

Leading Practices for Reviews. At the CAP-goal level, goal leaders for 
two CAP goals and one sub-goal instituted in-person progress reviews 
with officials from contributing agencies that were broadly consistent 
with the full range of leading practices for reviews, such as 
leadership involvement in reviews of progress on identified goals and 
milestones, and rigorous follow-up on issues identified through these 
reviews. In these cases, goal managers reported there were positive 
effects on performance, accountability, and collaboration. In 
contrast, review processes used by other goal leaders did not 
consistently reflect the full range of leading practices. Effective 
review processes consistently engage leaders and agency officials in 
efforts to identify and address performance deficiencies, and to 
ensure accountability for commitments. Thus, not using them may result 
in missed opportunities to hold meaningful performance discussions, 
ensure accountability and oversight, and drive performance 
improvement.  

What GAO Recommends:  

GAO is making seven recommendations to OMB to improve the reporting of 
performance information for CAP goals and ensure that CAP goal 
progress reviews meet GPRAMA requirements and reflect leading 
practices. OMB staff generally agreed to consider GAO's 
recommendations.  

View [hyperlink, http://www.gao.gov/products/GAO-14-526]. For more 
information, contact J. Christopher Mihm at (202) 512-6806 or 
mihmj@gao.gov.  

[End of section]  

Contents:  

Letter:  

Background:  

CAP Goal Leaders Reported Performance on Goals, but Many Lacked Key 
Information to Demonstrate Progress:  

OMB and Goal Leaders Established Processes for Reviewing Cross-Agency 
Priority Goal Progress, but Not All Review Processes Were Consistent 
with Requirements and Leading Practices:  

Conclusions:  

Recommendations for Executive Action:  

Agency Comments:  

Appendix I: Objectives, Scope, and Methodology:  

Appendix II: Interim Cross-Agency Priority Goals and Goal Statements:  

Appendix III: Interagency Group Membership and Meeting Frequency and 
Purpose:  

Appendix IV: Description of Interim Cross-Agency Priority Goal Review 
Processes:  

Appendix V: Full Text for Interactive Figure 2 on Frequency of Data 
Reporting for Cross-Agency Priority Goals' Overall Planned Levels of 
Performance:  

Appendix VI: GAO Contact and Staff Acknowledgments:  

Tables:  

Table 1: Leadership Involvement in Review Processes Reflecting Leading 
Practices:  

Table 2: Collection of Performance Data and Meeting Preparation for 
Review Processes Reflecting Leading Practices:  

Table 3: Participation by Key Officials in Review Processes Reflecting 
Leading Practices:  

Table 4: Follow-Up on Issues Raised During Review Meetings:  

Figures:  

Figure 1: Interim Cross-Agency Priority Goals, Positions of Goal 
Leaders, and Interagency Groups:  

Figure 2: Frequency of Data Reporting for Cross-Agency Priority goals' 
Overall Planned Levels of Performance 12:  

Figure 3: Identification of Strategies and Examples of Contributors 
for the STEM Education Cross-Agency Priority Goal:  

Figure 4: Strategic Sourcing Cross-Agency Priority Goal Update 
Provides Planned Actions Without Time Frames for Completion:  

Abbreviations:  

APG: Agency Priority Goal:  

CAP: Cross-Agency Priority:  

CEQ: Council on Environmental Quality:  

CIO: Chief Information Officer:  

DHS: Department of Homeland Security:  

EOP: Executive Office of the President:  

FSSI: Federal Strategic Sourcing Initiative:  

GPRAMA: GPRA Modernization Act of 2010:  

GSA: General Services Administration:  

HHS: Department of Health and Human Services:  

NASA: National Aeronautics and Space Administration:  

OFPP: Office of Federal Procurement Policy:  

OMB: Office of Management and Budget:  

OPM: Office of Personnel Management:  

OSTP: Office of Science and Technology Policy:  

PIC: Performance Improvement Council:  

PIV: Personal Identity Verification:  

SBA: Small Business Administration:  

SBIR: Small Business Innovation Research:  

STEM: Science, Technology, Engineering, and Mathematics:  

STTR: Small Business Technology Transfer:  

SSLC: Strategic Sourcing Leadership Council:  

VA: Department of Veterans Affairs:  

[End of section]  

United States Government Accountability Office: 
GAO:
441 G St. N.W. 
Washington, DC 20548:  

June 10, 2014:  

Congressional Addressees:  

Collaboration across agencies is fundamental to effectively addressing 
many complex, high-risk challenges facing the federal government, 
including protecting the nation's critical information systems, 
reducing billions of dollars lost through improper payments, and 
better managing the risks of climate change. Effectively managing 
these challenges is essential to national and economic security, 
public health and safety, and the integrity of federal programs. 
However, because these challenges are complex, responsibility for 
addressing them often rests with multiple agencies. Both Congress and 
the executive branch have recognized the need for improved 
collaboration across the federal government, and the GPRA 
Modernization Act of 2010 (GPRAMA) established a new framework and 
processes aimed at encouraging a more crosscutting and integrated 
approach to focusing on results and improving government performance 
and management.[Footnote 1] 

Among other things, GPRAMA requires the Office of Management and 
Budget (OMB) to coordinate with agencies to develop federal government 
priority goals (now known as cross-agency priority, or CAP, goals), 
which include outcome-oriented goals covering a limited number of 
crosscutting policy areas, as well as goals to improve management 
across the federal government. These goals are intended to cover areas 
where increased cross-agency collaboration is needed to improve 
progress towards shared, complex policy or management objectives, such 
as improving our nation's cybersecurity posture and addressing 
critical skills gaps in the federal workforce. In addition, in an 
effort to drive improved performance, transparency, and accountability 
across the federal government, GPRAMA requires the use of specific 
processes to encourage the application of leading management practices. 

One GPRAMA requirement is that leaders regularly review progress 
towards established agency and cross-agency goals.[Footnote 2] 
Specifically, the Director of OMB, with the support of the Performance 
Improvement Council (PIC), must review progress towards the CAP goals 
with the appropriate lead government official at least quarterly to 
increase management's collection and use of performance information to 
achieve crosscutting goals.[Footnote 3] Under GPRAMA, most large 
federal agencies have been holding quarterly reviews to assess 
progress in achieving their individual agency priority goals.[Footnote 
4] The emphasis on the creation of shared performance goals, the 
collection and analysis of data, and the regular review of these data 
with involved officials is designed to foster collaboration among 
agencies, ensure that interagency efforts are informed by data on 
performance, identify successful practices, and reinforce agency and 
collective accountability for the achievement of shared outcomes. 

This report is our response to a mandate that we evaluate the 
implementation of GPRAMA.[Footnote 5] Due to the timing of our work, 
we focused on efforts surrounding the 14 interim goals established in 
February 2012.[Footnote 6] This report assesses (1) what is known 
about progress made towards the interim CAP goals; and (2) how, if at 
all, quarterly progress reviews reflected GPRAMA requirements and 
leading practices for reviews, as well as how they contributed to 
improved cross-agency performance and collaboration. The experience 
with the interim goals can provide lessons for the implementation and 
management of the new CAP goals released with the fiscal year 2015 
budget. 

To address these objectives, we interviewed representatives of 13 of 
the 14 interim CAP goals.[Footnote 7] For 8 of the 13 goals, we spoke 
directly with the goal leader or deputy goal leader, along with, in 
some cases, other staff from agencies and OMB involved in supporting 
efforts related to the goals. For the other five goals, we met with 
agency officials or OMB staff playing a key role in the management of 
interagency efforts related to the goal.[Footnote 8] We also 
interviewed the goal leaders for 11 agency priority goals aligned 
with, or contributing to, a CAP goal. 

To address the first objective, and assess what is known about 
progress made towards the interim CAP goals, we analyzed performance 
information included in the quarterly status updates for each CAP goal 
published on Performance.gov, as well as relevant information 
collected through our interviews with CAP goal representatives. We 
compared the data and information made available through these updates 
with the GPRAMA requirement that Performance.gov includes the results 
achieved during the most recent quarter and overall trend data. To 
assess the reliability of performance data and information available 
through Performance.gov, we collected information from OMB and PIC 
staff and CAP goal representatives about data quality control 
procedures. We determined that the data and information were 
sufficiently reliable for our analysis of what was reported on 
Performance.gov about progress towards identified goals and milestones. 

To address the second objective and assess how OMB CAP goal progress 
reviews were conducted, we reviewed quarterly review memorandums 
developed by OMB staff. We also conducted interviews with staff from 
OMB and the PIC. We compared the contents of these memorandums with 
information required to be reviewed by the Director of OMB on a 
quarterly basis. 

To further address the second objective, we reviewed documents created 
for interagency meetings (where available), such as meeting agendas, 
presentation materials, notes, and attendee lists. We also interviewed 
sub-goal leaders from the Closing Skills Gaps and Entrepreneurship and 
Small Business CAP goals to learn more about their involvement in 
quarterly review processes for their CAP goals, and to learn of any 
impact the CAP goal designation or review processes had at the sub-
goal level. We selected these goals to reflect one review process that 
involved regular meetings between the goal leader and contributors 
dedicated to reviewing progress (Closing Skills Gaps), and one that 
did not rely on regular meetings at the CAP-goal level to review 
progress (Entrepreneurship and Small Business). Our findings from 
these interviews cannot be generalized to the other CAP goals. We then 
compared what we learned about the processes instituted by each goal 
leader to review progress to leading practices for performance reviews 
previously identified by GAO.[Footnote 9] 

Because the scope of our review was to examine the implementation of 
quarterly reporting and reviews, we did not evaluate whether these 
goals were appropriate indicators of performance, were sufficiently 
ambitious, or met other dimensions of quality. 

We conducted our work from May 2013 to June 2014 in accordance with 
generally accepted government auditing standards. Those standards 
require that we plan and perform the audit to obtain sufficient, 
appropriate evidence to provide a reasonable basis for our findings 
and conclusions based on our audit objectives. We believe that the 
evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. More detailed information 
on our scope and methodology appears in appendix I. 

Background:  

GPRAMA requires OMB to coordinate with agencies to develop long-term, 
outcome-oriented federal government priority goals for a limited 
number of crosscutting policy areas and management improvement areas 
every 4 years.[Footnote 10] Furthermore, with the submission of the 
fiscal year 2013 budget, GPRAMA required OMB to identify a set of 
interim priority goals.[Footnote 11] The President's 2013 budget 
submission included a list of 14 interim CAP goals, 9 of which were 
related to crosscutting policy areas and 5 of which were management 
improvement goals. 

The CAP Goal Leader. As required by GPRAMA, each of the interim CAP 
goals had a goal leader responsible for coordinating efforts to 
achieve each goal.[Footnote 12] CAP goal leaders were given 
flexibility in how to manage these efforts, and were encouraged by OMB 
to engage officials from contributing agencies by leveraging existing 
inter-agency working groups, policy committees, and councils. For 
information on the position of the goal leader and the interagency 
groups used to engage officials from agencies contributing to each 
interim CAP goal, see figure 1. For more information on the 
interagency groups used to engage agency officials in efforts related 
to each goal, see appendix III.[Footnote 13] 

Figure 1: Interim Cross-Agency Priority Goals, Positions of Goal 
Leaders, and Interagency Groups:  

[Refer to PDF for image: illustrated table]  

Cross-Agency Priority Goal: Broadband; 
Goal Leader Position: Deputy Chief Technology Officer for 
Telecommunications, Office of Science and Technology Policy; 
Interagency Group(s): Broadband Deployment on Federal Property Working 
Group and Spectrum Policy and Plans Steering Group.  

Cross-Agency Priority Goal: Closing Skills Gaps; 
Goal Leader Position: Director, Office of Personnel Management; 
Interagency Group(s): Closing Skills Gaps CAP Goal Leadership Team.  

Cross-Agency Priority Goal: Cybersecurity; 
Goal Leader Position: Special Assistant to the President and 
Cybersecurity Coordinator; 
Interagency Group(s): Cybersecurity CAP Goal Governance Committee.  

Cross-Agency Priority Goal: Data Center Consolidation. 
Goal Leader Position: Federal Chief Information Officer, Office of 
Management and Budget; 
Interagency Group(s): Federal Data Center Consolidation Task Force.  

Cross-Agency Priority Goal: Energy Efficiency; 
Goal Leader Position: Deputy Assistant to the President for Energy and 
Climate Change, Domestic Policy Council; 
Interagency Group(s): No interagency groups were used to manage 
efforts related to this goal.  

Cross-Agency Priority Goal: Entrepreneurship and Small Business; 
Goal Leader Position: Deputy Director for Technology and Innovation, 
Office of Science and Technology Policy; 
Interagency Group(s): Interagency groups were used to manage efforts 
at the sub-goal level.  

Cross-Agency Priority Goal: Exports; 
Goal Leader Position: Assistant to the President and Deputy National 
Security Advisor for International Economic Affairs; 
Interagency Group(s): Trade Promotion Coordinating Committee/Export 
Promotion Cabinet.  

Cross-Agency Priority Goal: Improper Payments; 
Goal Leader Position: Controller, Office of Management and Budget; 
Interagency Group(s): Do Not Pay Implementation Group and Senior 
Accountable Officials Group.  

Cross-Agency Priority Goal: Job Training; 
Goal Leader Position: Special Assistant to the President for Labor and 
Workforce Policy, Domestic Policy Council; 
Interagency Group(s): Job Training Data Access Working Group.  

Cross-Agency Priority Goal: Real Property; 
Goal Leader Position: Controller, Office of Management and Budget; 
Interagency Group(s): Federal Real Property Council.  

Cross-Agency Priority Goal: Science, Technology, Engineering and 
Mathematics (STEM) Education; 
Goal Leader Position: Special Assistant to the President, Domestic 
Policy Council; 
Interagency Group(s): Federal Coordination in STEM Education Task 
Force.  

Cross-Agency Priority Goal: Strategic Sourcing; 
Goal Leader Position: Administrator for Federal Procurement Policy, 
Office of Management and Budget. 
Interagency Group(s): Strategic Sourcing Leadership Council.  

Cross-Agency Priority Goal: Sustainability; 
Goal Leader Position: Chair, Council on Environmental Quality; 
Interagency Group(s): Steering Committee on Federal Sustainability.  

Cross-Agency Priority Goal: Veteran Career Readiness; 
Goal Leader Position: Policy Director of Veteran and Wounded Warriors 
and Military Families, National Security Council and Domestic Policy 
Council; 
Interagency Group(s): Interagency Policy Council and Veterans 
Employment Initiative Task Force.  

Source: GAO analysis of information from Performance.gov and 
interviews with CAP goal managers. GAO-14-526.  

[End of figure]  

According to OMB and PIC staff, because CAP goal leaders were 
responsible for managing efforts related to the achievement of the 
goals as part of a larger portfolio of responsibilities, staff from 
the PIC, OMB, and--in some cases--from agencies with project 
management responsibilities, provided additional capacity for 
coordinating interagency efforts and overseeing the process of 
collecting, analyzing, and reporting data. Specifically, PIC staff 
provided logistical support, assisting with the regular collection of 
data, updates to Performance.gov, and the development of CAP goal 
governance structures and working groups. They also provided support 
in the area of performance measurement and analysis. For example, PIC 
staff supported the Exports goal leader by informing discussions of 
how to measure the success and impact of export promotion efforts, 
providing expertise in the development and selection of appropriate 
performance measures, and assisting in the collection and analysis of 
relevant data. 

Progress Reviews. GPRAMA also requires that the Director of OMB, with 
the support of the PIC, review progress towards each CAP goal with the 
appropriate lead government official at least quarterly. Specifically, 
the law requires that these should include a review of progress during 
the most recent quarter, overall trends, and the likelihood of meeting 
the planned level of performance.[Footnote 14] As part of these 
reviews, OMB is to assess whether relevant agencies, organizations, 
program activities, regulations, tax expenditures, policies, and other 
activities are contributing as planned to the goal. The law also 
requires that OMB categorize the goals by risk of not achieving the 
planned level of performance and, for those at greatest risk of not 
meeting the planned level of performance, identify strategies for 
performance improvement.[Footnote 15] 

In an earlier evaluation of the implementation of quarterly 
performance reviews at the agency level, we found that regular, in-
person review meetings provide a critical opportunity for leaders to 
use current data and information to analyze performance, provide 
feedback to managers and staff, follow up on previous decisions or 
commitments, learn from efforts to improve performance, and identify 
and solve performance problems.[Footnote 16] As part of this work we 
also identified nine leading practices that can be used to promote 
successful performance reviews at the federal level. To identify these 
practices, we conducted a review of relevant academic and policy 
literature, including our previous reports. We refined these practices 
with additional information obtained from practitioners at the local, 
state, and federal level who shared their experiences and lessons 
learned.  

Text box:  

Nine Leading Practices That Can Be Used to Promote Successful 
Performance Reviews:  

* Leaders use data-driven reviews as a leadership strategy to drive 
performance improvement. 

* Key players attend reviews to facilitate problem solving. 

* Reviews ensure alignment between goals, program activities, and 
resources. 

* Leaders hold managers accountable for diagnosing performance 
problems and identifying strategies for improvement. 

* There is capacity to collect accurate, useful, and timely 
performance data. 

* Staff have skills to analyze and clearly communicate complex data 
for decision making. 

* Rigorous preparations enable meaningful performance discussions. 

* Reviews are conducted on a frequent and regularly scheduled basis. 

* Participants engage in rigorous and sustained follow-up on issues 
identified during reviews. 

Source: GAO-13-228.  

[End of text box]  

Reporting Requirements. In addition to requiring quarterly reviews, 
GPRAMA requires that OMB make information available on "a single 
website" (now known as Performance.gov) for each CAP goal on the 
results achieved during the most recent quarter, and overall trend 
data compared to the planned level of performance.[Footnote 17] In 
addition, information on Performance.gov is to include an assessment 
of whether relevant federal organizations, programs, and activities 
are contributing as planned, and, for those CAP goals at risk of not 
achieving the planned level of performance, information on strategies 
for performance improvement.[Footnote 18] 

New CAP Goals. As required by GPRAMA, in March 2014, OMB announced the 
creation of a new set of CAP goals in the fiscal year 2015 budget. It 
then identified 15 CAP goals with 4-year time frames on 
Performance.gov--7 mission-oriented goals and 8 management-focused 
goals. Five goal areas--Cybersecurity; Open Data; Science, Technology, 
Engineering, and Mathematics (STEM) Education; Strategic Sourcing; and 
Sustainability (renamed Climate Change (Federal Actions))--were 
carried over from the set of interim CAP goals, while the other 10 are 
new goal areas.[Footnote 19] OMB stated on Performance.gov that more 
detailed action plans for each of the goals, including specific 
metrics and milestones that will be used to gauge progress, will 
subsequently be released. The new CAP goals will also have co-leaders; 
one from an office within the Executive Office of the President (EOP) 
and one or more from federal agencies. According to OMB staff, this 
change was made to ensure that CAP goal leaders can leverage the 
convening authority of officials from the EOP while also drawing upon 
expertise and resources from the agency level. 

CAP Goal Leaders Reported Performance on Goals, but Many Lacked Key 
Information to Demonstrate Progress:  

Text box:  

GPRAMA Requirements for Establishing Planned Performance for CAP 
Goals:  

GPRAMA requires the Director of OMB to establish, in the annual 
federal government performance plan, a planned level of performance 
for each CAP goal for the year in which the plan is submitted and the 
next fiscal year, as well as quarterly performance targets for the 
goals. 

Source: 31 U.S.C. §§ 1115(a)(1) and (4). 

GPRAMA Requirements for Reporting CAP Goal Performance Information:  

GPRAMA requires the Director of OMB to publish on Performance.gov 
information about the results achieved during the most recent quarter 
and trend data compared to the planned level of performance for each 
CAP goal. 

Source: 31 U.S.C. §1122(c)(5).  

[End of text box]  

OMB released the federal government performance plan on 
Performance.gov concurrently with the fiscal year 2013 budget 
submission that identified the 14 interim CAP goals. The information 
on Performance.gov included a goal statement for each of the interim 
goals that established an overall planned level of performance. During 
the two-year interim goal period, OMB addressed the requirement to 
report on results achieved during the most recent quarter for each of 
the CAP goals by publishing 5 sets of quarterly updates to the interim 
CAP goals on Performance.gov. The first set of updates, for the fourth 
quarter of fiscal year 2012, was published in December 2012 and the 
final set of updates, for the fourth quarter of fiscal year 2013, was 
published in February 2014.[Footnote 20] These documents described 
general accomplishments made to date, specific actions completed, or 
both. The updates to the Broadband CAP goal, for instance, included 
short descriptions of general progress made towards each of the five 
strategies identified for achieving the goal, as well as specific 
milestones accomplished. 

The quarterly updates did not, however, consistently identify required 
interim planned levels of performance and data necessary to indicate 
progress being made toward the CAP goals. Updates to eight of the 
goals included quarterly, biannual, or annual data that indicated 
performance achieved to date toward the target identified in the goal 
statement. Three of the eight goals (Cybersecurity, Energy Efficiency, 
and Strategic Sourcing) also contained the required annual or 
quarterly targets that defined planned levels of performance, which 
allowed for an assessment of interim progress. For example, the 
Cybersecurity goal's updates stated that the goal would not be met 
within its established time frame, and provided quarterly performance 
data compared to quarterly targets for the entirety of the goal period 
to support the statement. In contrast, the updates for the other five 
goals did not contain annual or quarterly targets, which made it 
difficult to determine whether interim progress towards the goals' 
overall planned levels of performance was being made. For example, 
updates to the Exports goal included data on the total amount of U.S. 
exports by quarter for calendar years 2012 and 2013 but did not 
include a target level of performance for those years or quarters. 
Therefore, it was unclear whether the goal's overall planned level of 
performance of doubling U.S. exports by the end of 2014 is on track to 
be met. Furthermore, updates to six interim CAP goals did not include 
trend data to indicate progress being made towards the goals' overall 
planned levels of performance. 

Figure 2 below identifies the frequency with which data on CAP goal 
performance were reported, as well as the overall performance CAP goal 
leaders reported making compared to the goal's planned level of 
performance through the fourth quarter of fiscal year 2013.  

Figure 2: Frequency of Data Reporting for Cross-Agency Priority Goals' 
Overall Planned Levels of Performance:  

Interactive graphic:  

To print text version of this graphic, go to appendix V.  

Print instructions: Roll over each Cross-Agency Priority Goal to see 
more information about progress reported for the goal.  

Source: GAO analysis of information from Performance.gov.  

[End of figure]  

Through our review of information on Performance.gov and interviews 
with managers of the six interim CAP goals that did not report any 
data on progress towards the stated goal, we identified reasons that 
included:  

* Lack of quantitative planned level of performance (targets). The 
Entrepreneurship and Small Business CAP goal lacked a quantitative 
performance target. The quarterly updates to the goal explained that 
efforts were focused on the goal's 10 sub-goals. Most of these sub-
goals, however, also lacked quantitative performance targets. The 
deputy goal leader told us that some of the sub-goals did not have 
quantitative targets by design, as goal managers thought it more 
appropriate to use qualitative milestones to track progress towards 
them. The quarterly updates to the "Streamline immigration pathways 
for immigrant entrepreneurs" sub-goal lacked a quantitative target but 
had a range of qualitative milestones. For example, the Department of 
Homeland Security and the Department of State established a milestone 
to identify reforms needed to ease the application and adjudication 
processes for visas available to certain immigrant entrepreneurs. 

* Unavailable data. Some CAP goal managers told us that the data 
needed to assess and report progress toward their goals' performance 
targets were unavailable or not yet being collected. For example, a 
manager of the Job Training CAP goal told us that staff had not 
established a baseline number of participants served by federal job 
training programs against which progress towards the goal could be 
tracked. In addition, managers of the Real Property CAP goal told us 
that they did not have data available for tracking progress toward the 
goal of holding the federal real property footprint at its fiscal year 
2012 baseline level. 

Where key data were not reported, some goal managers took actions to 
obtain previously unavailable data or developed an alternative 
approach for assessing progress. 

Job Training CAP Goal. The first quarterly update for the Job Training 
CAP goal, published on Performance.gov in December 2012, stated that 
federal agencies were surveyed to compile a list of all job training 
programs in the federal government, including the number of 
participants served by those programs, and that a working group was 
developing a baseline for measuring progress towards the goal of 
preparing 2 million workers with skills training by 2015. A goal 
manager told us that the deputy goal leader and staff from the PIC 
gathered baseline information for most of the programs within the 
scope of the CAP goal, but that they were unable to complete the 
efforts by the end of the goal period. 

Real Property CAP Goal. Managers of the Real Property CAP goal told us 
that they worked to establish a baseline and metrics for measuring 
future performance and would be able to report on progress after the 
goal period ended. 

Closing Skills Gaps CAP Goal. A manager of the Closing Skills Gaps 
goal told us that the goal's managers decided early on that it did not 
make sense for each of the goal's identified mission-critical 
occupations to have the same skills gaps reduction target. Instead, 
managers of the goal's sub-goals identified efforts to reduce skills 
gaps in their specific occupations. They identified an individual 
targeted level of performance for that effort and collected and 
reported data on progress made towards the target. For instance, 
managers of the Acquisitions sub-goal established a target for 
increasing the certification rate of GS-1102 contract specialists to 
80 percent. The final quarterly status update to the Closing Skills 
Gaps CAP goal reported that the target was met and the certification 
rate increased to 81 percent. 

Veteran Career Readiness CAP Goal. The leader of the Veteran Career 
Readiness CAP goal told us that efforts were made to collect data to 
assess the veteran employment situation. For instance, she said that 
an interagency data-gathering working group reviewed sources of 
available data, integrated those data - such as the unemployment rate 
for various sub-populations of veterans - into dashboards for senior 
leadership review, and made proposals to improve data availability. In 
addition, the Army led a working group to develop a more complete 
picture of veterans receiving unemployment compensation. She said that 
these and other efforts led to a concerted effort to improve the 
availability of data, and to develop and implement metrics measuring 
career readiness and attendance in a veteran career transition 
assistance program. However, no data to track progress towards the 
overall goal were reported during the interim goal period. 

As we have previously reported, no picture of what the federal 
government is accomplishing can be complete without adequate 
performance information. However, OMB and CAP goal leaders did not 
identify interim planned levels of performance or targets for most of 
the interim CAP goals. Furthermore, they established a number of CAP 
goals for which data necessary to indicate progress towards the goal 
could not be reported. In so doing, they limited their ability to 
demonstrate progress being made towards most of the CAP goals and 
ensure accountability for results from those who helped to manage the 
goals. 

CAP Goal Leaders Described What Contributed to Goal Achievement, but 
in Some Cases Information Was Incomplete:  

Text box:  

GPRAMA Requirement for Establishing Milestones:  

GPRAMA requires the Director of OMB to establish, in the federal 
government performance plan, clearly defined quarterly milestones for 
the CAP goals. 

Source: Source: 31 U.S.C. § 1115(a)(5).  

GPRAMA Requirement for Reporting on Contributions towards Cross-Agency 
Priority Goals:  

GPRAMA requires that OMB identify the agencies, organizations, program 
activities, regulations, tax expenditures, policies, and other 
activities that contribute to each CAP goal on Performance.gov. It 
also requires OMB to make available on the website an assessment of 
whether relevant agencies, organizations, program activities, 
regulations, tax expenditures, policies, and other activities are 
contributing as planned. 

Source: 31 U.S.C. §§ 1122(c)(6)-(7).  

[End of text box]  

In the status updates that were published on Performance.gov, managers 
of each of the CAP goals reported the general approaches, strategies, 
or specific initiatives being employed to make progress towards the 
achievement of the goal, as well as the departments, agencies, and 
programs that were expected to contribute to goal achievement. For 
example, the leader of the Science, Technology, Engineering, and 
Mathematics (STEM) Education CAP goal identified a number of general 
strategies for making progress towards the achievement of its goal of 
increasing the number of graduates in STEM subjects by 1 million over 
the next 10 years, such as "Address the mathematics preparation gap 
that students face when they arrive at college" and "Identifying and 
supporting the role of technology and innovation in higher education." 
In addition, the goal leader identified a number of programs and goals 
within four departments and agencies that were likely to contribute in 
part or in whole to the goal. Figure 3 below illustrates how this 
information was presented in the update to the STEM Education CAP goal 
for the fourth quarter of fiscal year 2013. 

Figure 3: Identification of Strategies and Examples of Contributors 
for the STEM Education Cross-Agency Priority Goal:  

[Refer to PDF for image: illustration]  

Strategies:  

Strategies: 4;  

Identifying and implementing evidence-based practices to improve STEM 
teaching, and attract students to STEM courses: 5;  

More opportunities for students to do hands-on, real-world STEM 
activities through research experiences, especially in their first two 
years of college: 10;  

Address the mathematics preparation gap that students face when they 
arrive at college: 12;  

focus on women and underrepresented minorities: 13;  

Identifying and supporting the role of technology and innovation in 
higher education: 15.  

Contributors:  

Department of Education:  

* Mathematics and Science Partnerships Program/Effective Teaching and 
Learning for a Complete Education; 
* Investing in Innovation Fund; 

NASA: 
* National Space Grant College and Fellowship Program; 
* Minority University Research Education Program;  

National Science Foundation:
* Priority Goal: Develop a diverse and highly qualified STEM workforce 
motivated to participate at the frontiers; 
* Proposed K-16 Mathematics Education Program (in collaboration with 
ED);  

Department of Health and Human Services - National Institute 
of Health: 
* NIH Undergraduate Research Experiences to Support Science Learning; 
* NIH Intramural Summer Internship Program;  

Source: Performance.gov.  

[End of figure] 

In a May 2012 report on our work related to the CAP goals, we noted 
that information on Performance.gov indicated additional programs with 
the potential to contribute to each of the CAP goals may be identified 
over time.[Footnote 21] We then recommended that OMB review and 
consider adding to the list of CAP goal contributors the additional 
departments, agencies, and programs that we identified, as 
appropriate. OMB agreed with the recommendation, and in the quarterly 
updates to the CAP goals published in December 2012 and March 2013, 
OMB added some of the departments, agencies, and programs we 
identified in our work to some CAP goals' lists of contributors. For 
example, we had noted that 12 member agencies of the Trade Promotion 
Coordinating Committee had not been identified as contributors to the 
Exports CAP goal. OMB added additional information about contributors 
to the Exports goal in the update published in December 2012. 

During our review, in some cases CAP goal managers told us about 
additional organizations and program types that contributed to their 
goals, but which were not identified on Performance.gov or in our 
previous report.[Footnote 22] For example, the leader of the STEM 
Education CAP goal told us that representatives from the Smithsonian 
Institution led an interagency working group that contributed to key 
efforts towards achieving the goal. Although the CAP goal updates 
indicate that the Smithsonian Institution is involved in federal STEM 
education efforts, it was not identified in a dedicated list of 
contributors to the goal. We have previously found that federal STEM 
education programs are fragmented across a number of agencies. We 
continue to believe that the federal government's efforts to ensure 
STEM education programs are effectively coordinated must include all 
relevant efforts.[Footnote 23] Furthermore, the leader of the 
Broadband CAP goal told us that he is aware that tax deductions 
available to businesses making capital investments contributed to the 
goal by incentivizing investments in broadband. We have long referred 
to such deductions, along with other reductions in a taxpayer's 
liability that result from special exemptions and exclusions from 
taxation, credits, deferrals of tax liability, or preferential tax 
rates, as tax expenditures. As we have previously reported, as with 
spending programs, tax expenditures represent a substantial federal 
commitment to a wide range of mission areas.[Footnote 24] We have 
recommended greater scrutiny of tax expenditures. Periodic reviews 
could help determine how well specific tax expenditures work to 
achieve their goals and how their benefits and costs compare to those 
of programs with similar goals. As previously mentioned, GPRAMA also 
requires OMB to identify tax expenditures that contribute to CAP 
goals. However, tax expenditures were not reported as contributors to 
the Broadband CAP goal in the quarterly status updates published on 
Performance.gov. 

CAP Goal Leaders Identified Milestones for Tracking Progress, but in 
Some Cases Milestones Were Missing Key Information:  

Leading practices state that a clear connection between goals and day-
to-day activities can help organizations better articulate how they 
plan to accomplish their goals. In addition, a clear connection 
between goals and the programs that contribute to them helps to 
reinforce accountability and ensure that managers keep in mind the 
results their organizations are striving to achieve. Milestones--
scheduled events signifying the completion of a major deliverable or a 
set of related deliverables or a phase of work--can help organizations 
demonstrate the connection between their goals and day-to-day 
activities and that they are tracking progress to accomplish their 
goals. Organizations, by describing the strategies to be used to 
achieve results, including clearly defined milestones, can provide 
information that would help key stakeholders better understand the 
relationship between resources and results.[Footnote 25] 

The updates published on Performance.gov for each of the CAP goals 
identified at least some planned activities that were expected to 
contribute to the achievement of the goal. However, in some cases the 
descriptions of these planned activities were missing key information 
that would be found in clearly defined milestones.[Footnote 26] 
Without clearly defined milestones, CAP goal leaders cannot 
demonstrate that they have effectively planned to support the 
strategies for achieving their goals and are tracking progress toward 
goal achievement. Specifically, some planned activities for supporting 
CAP goal achievement lacked:  

* Alignment with strategies for goal achievement: Updates to the Job 
Training CAP goal described the goal's three main objectives and the 
six specific strategies that would be used to advance them. The 
objectives included, for example, "Continued delivery of effective and 
efficient job training programs," while the strategies included, for 
example, "Invest in Community Colleges" and "Help workers build skills 
while on the job." The updates also identified lists of "Milestones 
for the Upcoming Quarter" and "Longer-term Milestones," such as 
"Develop a performance data 'checklist' that can be used by agencies 
in designing job training programs and evaluations prior to sending 
out grant solicitations" and "Identify job training programs that 
would benefit from adoption of additional or enhanced job training 
performance measures." These planned activities, however, were not 
clearly aligned with the objectives or strategies they supported. 

* Time frames for completion: The first four updates for the Strategic 
Sourcing CAP goal included planned "next steps," which described 
actions that were to contribute to the achievement of the goal. These 
actions, however, lacked clear time frames for completion. Figure 4 
below illustrates the "next steps" identified for the Strategic 
Sourcing CAP goal in the update for the third quarter of fiscal year 
2013. 

Figure 4: Strategic Sourcing Cross-Agency Priority Goal Update 
Provides Planned Actions Without Time Frames for Completion. 

[Refer to PDF for image: illustration]  

Planned Actions Without Time Frames:  

Next Steps: 
* SSLC will review agency participation in FSSI solutions (current and 
proposed) and recommend policy, process, or workforce initiatives to 
improve adoption and participation;  

* SSLC will make recommendations on additional opportunities for 
consideration.  

Source: Performance.gov.  

[End of figure] 

* Completion status: The Real Property CAP goal update for the second 
quarter of fiscal year 2013 identified two planned actions as "next 
steps." "After agencies submit their Revised Cost Savings and 
Innovation Plans to OMB, OMB will evaluate agency plans to maintain 
their square footage baselines, while balancing mission requirements," 
and "Updates on agency square footage baselines and projects are 
forthcoming and will be posted on Performance.gov." These two actions 
were again identified as "next steps" in the update for the third 
quarter of fiscal year 2013, but no update was provided on the status 
of the actions. 

By establishing planned activities that, in many of the CAP goal 
updates, did not have information about their alignment with the 
strategies they supported, their time frames for completion, or their 
completion status, CAP goal leaders did not fully demonstrate that 
they had effectively planned to support goal achievement or were 
tracking progress toward the goal or identified milestones. 

OMB did not issue formal guidance to CAP goal leaders on the types of 
information that were to be included in the CAP goal updates, 
including information about contributors and milestones. Standards for 
internal control in the federal government emphasize the importance of 
documenting policies and procedures to provide a reasonable assurance 
that activities comply with applicable laws and regulations, and that 
managers review performance and compare actual performance to planned 
or expected results and analyze significant differences.[Footnote 27] 
OMB staff told us they provided an implementation plan template to 
goal leaders, which outlined the data elements to be reported in the 
quarterly status updates. The template was also used to collect 
information for internal and public reporting. Some CAP goal managers 
told us that OMB or PIC staff, in their role supporting the 
collection, analysis, and presentation of data on CAP goal 
performance, occasionally provided feedback on the information that 
the individuals submitted in draft updates that OMB reviewed before 
they were published on Performance.gov. For example, one CAP goal 
manager told us that during a review of an update submission PIC staff 
told him that he should develop additional milestones to be completed 
during a specific future fiscal year quarter. 

This is in contrast to the detailed guidance that OMB issued on the 
types of information that agencies must provide for the updates for 
agency priority goals (APG), which are also published quarterly on 
Performance.gov. The APG guidance includes explicit instructions for 
agencies to identify, as appropriate, the organizations, regulations, 
tax expenditures, policies, and other activities within and external 
to the agency that contribute to each APG, as well as key milestones 
with planned completion dates for the remainder of the goal period. 
Because guidance for the types of information that should have been 
included in the CAP goal updates was never formally established, CAP 
goal leaders were at a heightened risk of failing to take into account 
important contributors to the goal and providing incomplete 
information about milestones that could help demonstrate progress 
being made. 

OMB and Goal Leaders Established Processes for Reviewing Cross-Agency 
Priority Goal Progress, but Not All Review Processes Were Consistent 
with Requirements and Leading Practices:  

OMB Established a Quarterly Process for Reviewing Progress on CAP 
Goals, but Did Not Consistently Outline Improvement Strategies Where 
Goal Achievement Was at Risk:  

Text box:  

GPRAMA Requirement for OMB Progress Reviews:  

GPRAMA requires that, not less than quarterly, the Director of OMB, 
with the support of the PIC, shall review progress on the CAP goals, 
including progress during the most recent quarter, overall trends, and 
the likelihood of meeting the planned level of performance. GPRAMA 
also requires that, as part of these reviews, OMB categorize goals by 
their risk of not achieving the planned level of performance and, for 
those goals most at risk of not meeting the planned level of 
performance, identify strategies for performance improvement. 

Source: 31 U.S.C. §§ 1121(a)(1), (a)(4)-(5).  

[End of text box]  

As required by GPRAMA, OMB reviewed progress on CAP goals each 
quarter, beginning with the quarter ending June 30, 2012. This review 
process consisted of the collection of updated information for each 
CAP goal by OMB or PIC staff, and the development of a memorandum for 
the Director of OMB with information on the status of the CAP goals. 
To develop these memorandums, OMB staff told us that approximately 6 
weeks after the end of each quarter, OMB and PIC staff worked with CAP 
goal leaders to collect updated data and information on goal metrics 
and milestones, and to update the narratives supporting the data. CAP 
goal leaders, or staff assisting leaders with the management of 
efforts related to the goal, would provide this information to OMB 
using a template for the status updates ultimately published on 
Performance.gov. In addition to the memorandums developed for the 
Director of OMB, OMB published more detailed information through the 
quarterly status updates available on Performance.gov. 

OMB and PIC staff told us that to support OMB's quarterly review 
efforts, PIC staff were to conduct assessments rating the overall 
health of implementation efforts and goal leader engagement. They were 
also to assess the execution status of each goal, including the 
quality and trend of performance indicators. One purpose of these 
assessments was to identify areas where risks, such as goal leader 
turnover, could affect the ability to achieve the planned level of 
performance. Consistent with this intent, several of the quarterly OMB 
review memorandums we examined highlighted turnover in goal leader or 
deputy goal leader positions as risks, and suggested the need to find 
or approve replacements. Although PIC staff have been tasked with 
assessing these elements of CAP goal implementation, and said that 
there was a shared understanding between involved staff as to how 
these assessments would be carried out, the PIC has not documented its 
procedures or criteria for conducting these assessments. Standards for 
internal control in the federal government emphasize the importance of 
documenting procedures, including those for assessing performance. 
[Footnote 28] Without clearly established criteria and procedures, PIC 
staff lack a means to: consistently assess implementation efforts and 
execution across all goals; bring any deficiencies, risks, and 
recommended improvements identified to the attention of leadership; 
and ensure consistent application of criteria over time. 

While these quarterly review memorandums identified one goal as being 
at risk of not achieving the planned level of performance, and 
identified other instances where progress on goals had been slower 
than planned, the memorandums did not consistently outline the 
strategies that were being used to improve performance or address 
identified risks. For example, the Cybersecurity CAP goal was the one 
goal specifically described as being at risk of not achieving the 
planned level of performance, both in these memorandums and in the 
status updates on Performance.gov. Specifically, the memorandum for 
the third quarter of fiscal year 2012 identified the risk of not 
achieving the planned level of performance, and outlined seven 
specific risks facing the goal and the steps being taken to mitigate 
them. Similarly, the memorandum for the second quarter of fiscal year 
2013 also acknowledged that some agencies were at risk of not meeting 
their Cybersecurity CAP goal targets. However, in contrast to the 
earlier memorandums, no information was included about the specific 
steps that were being taken to mitigate these risks, although 
information on planned and ongoing actions to improve government-wide 
implementation was included in the milestones section of the status 
update for that quarter on Performance.gov. 

The memorandum for the fourth quarter of fiscal year 2012 also 
acknowledged that the pace of progress on the STEM Education and 
Closing Skills Gaps goals had been slower than expected. While the 
memorandum stated that additional OMB attention was needed to support 
implementation and assure sufficient progress, no information on the 
specific strategies being employed to improve performance was 
mentioned. According to OMB staff, however, these memorandums were 
used to inform subsequent conversations with OMB leadership, which 
would build on the information presented in the memorandums. 

Furthermore, because the data necessary to track progress for some 
goals were unavailable, the Director of OMB would not have been able 
to consistently review progress for all CAP goals, or make a 
determination about whether some CAP goals were at risk of meeting 
their planned levels of performance. This fact was acknowledged in the 
quarterly review memorandums for quarters one and two of fiscal year 
2013, which acknowledged that progress on three goals 
(Entrepreneurship and Small Business, Job Training, and STEM 
Education) was difficult to track, and that additional work was needed 
on data collection. However, no information on the specific steps that 
were being taken to address these shortcomings was included. A lack of 
specific information about the steps being taken to mitigate 
identified risk areas and improve performance could hinder the ability 
of OMB leadership--and others--to adequately track the status of 
efforts to address identified deficiencies or risks and to hold 
officials accountable for taking necessary actions. 

CAP Goal Leaders Established Processes to Review Progress, but Their 
Consistency with Leading Practices and Their Effects on Performance 
and Collaboration Varied:  

Text box:  

GPRAMA Requirement for Goal Leader and Agency Involvement in Progress 
Reviews:  

As part of the quarterly review process, GPRAMA requires that the 
Director of OMB review each priority goal with the appropriate lead 
government official, and include in these reviews officials from the 
agencies, organizations, and program activities that contribute to the 
achievement of the goal. 

Source: 31 U.S.C. §1121(a)(2).  

[End of text box]  

According to OMB staff, to encourage goal leaders and contributing 
agencies to take ownership of efforts to achieve the goals, OMB gave 
goal leaders flexibility to use different approaches to engage agency 
officials and review progress at the CAP-goal level. While guidance 
released by OMB in August 2012 encouraged goal leaders to leverage 
existing interagency working groups, committees, and councils in the 
management of the goals as much as practicable, it did not include 
information on the purpose of reviews, expectations for how reviews 
should be conducted to maximize their effectiveness as a tool for 
performance management and accountability, or the roles that CAP goal 
leaders and agency officials should play in the review process. 
[Footnote 29] Again, standards for internal control in the federal 
government emphasize the importance of documenting procedures for 
reviewing performance against established goals and objectives. 
[Footnote 30] 

This is in contrast to the detailed guidance that OMB released for 
agency priority goal and agency strategic objective reviews, which 
outlined the specific purposes of the reviews, how frequently they 
should be conducted, the roles and responsibilities of agency leaders 
involved in the review process, and how the reviews should be 
conducted.[Footnote 31] We also found that this guidance for reviews 
at the agency level was broadly consistent with the leading practices 
for performance reviews that we previously identified.[Footnote 32] 
While no official guidance was published to guide how reviews 
involving goal leaders and staff from contributing agencies could be 
conducted for the CAP goals, OMB staff said the principles of the 
guidance released for agency reviews, which reflected many of the 
leading practices, was referenced in conversations with CAP goal 
leaders and teams. 

OMB has emphasized that flexibility is needed to ensure that goal 
leaders can use review processes that are appropriate given the scope 
of interagency efforts, the number of people involved, and the 
maturity of existing reporting and review processes. The guidance for 
agency reviews gave agencies flexibility to design their performance 
review processes in a way that would fit the agency's mission, 
leadership preferences, organizational structure, culture, and 
existing decision-making processes. In our previous work, we detailed 
how several federal agencies had implemented quarterly performance 
reviews in a manner consistent with leading practices, but which were 
also tailored to the structures, processes, and needs of each 
agency.[Footnote 33] In this way, flexible implementation of review 
processes is possible within a framework that encourages the 
application of leading practices. 

A lack of clear expectations for how progress should be reviewed at 
the CAP-goal level resulted in a number of different approaches being 
used by goal leaders to engage officials from contributing agencies to 
review progress on identified goals and milestones, ranging from 
regular in-person review meetings led by the CAP goal leader to the 
review of written updates provided to the goal leader by officials 
from contributing agencies. See appendix IV for more detailed 
information on the various processes used by goal leaders to collect 
data on, and review progress towards, identified goals. 

Some Goal Leaders Used Review Processes Broadly Consistent with 
Leading Practices, and Noted Their Positive Effects on Performance, 
Accountability, and Collaboration:  

Instituting review processes consistent with the leading practices we 
previously identified can help ensure that reviews include meaningful 
performance discussions, provide opportunities for oversight and 
accountability, and drive performance improvement. Taken together, 
these leading practices emphasize the importance of leadership 
involvement in the review process, data collection and review meeting 
preparation, participation by key officials, and rigorous follow-up. 

Through our evaluation of how goal leaders and contributing agency 
officials reviewed progress towards the interim goals, we identified 
two CAP goals--Cybersecurity and Closing Skills Gaps--and one sub-goal-
-the Entrepreneurship and Small Business sub-goal on improving access 
to government services and information (BusinessUSA sub-goal)--where 
goal managers instituted in-person review processes with officials 
from contributing agencies that were broadly consistent with the full 
range of leading practices for reviews, which we have summarized in 
four categories below. The processes used by other CAP goal leaders to 
engage agency officials in the review of progress did not reflect the 
full range of leading practices. 

Leadership Involvement. Leading practices indicate that leaders should 
use frequent and regular progress reviews as a leadership strategy to 
drive performance improvement and as an opportunity to hold people 
accountable for diagnosing performance problems and identifying 
strategies for improvement. The direct and visible engagement of 
leadership is vital to the success of such reviews. Leadership 
involvement helps ensure that participants take the review process 
seriously and that decisions and commitments can be made. The goal 
leaders managing the Cybersecurity and Closing Skills Gaps goals, as 
well as the BusinessUSA sub-goal, were directly involved in leading in-
person reviews for these goals, and in using them as opportunities to 
review progress, identify and address performance problems, and hold 
agency officials accountable for progress on identified goals and 
milestones, as detailed in table 1. 

Table 1: Leadership Involvement in Review Processes Reflecting Leading 
Practices:  

Cybersecurity: 
* Throughout the year the goal leader and deputy goal leader led 
reviews with agencies at risk of not achieving the planned level of 
performance or exhibiting other risk factors. They held reviews with 
eight agencies in 2013; 
* These reviews were supplemented by meetings of the goal's governance 
committee, which met twice each quarter to review and discuss agency 
progress; 
* In the reviews, agency officials discussed with the goal leaders 
their performance on key metrics, recommendations for improvement, 
other issues that could affect performance and the risks those 
presented.  

Closing Skills Gaps: 
* The goal leader led quarterly review meetings involving all leaders 
of six identified sub-goal areas; 
* Meetings were used to review progress on performance goals 
established for each sub-goal, as well as challenges encountered and 
future actions with each of the sub-goal leaders; 
* Also provided the goal leader with an opportunity to discuss 
specific actions that the goal leader or staff could take to support 
or facilitate progress on the sub-goals.  

BusinessUSA: 
* The BusinessUSA sub-goal leader, in his capacity as Executive 
Director of BusinessUSA, led monthly meetings of the BusinessUSA 
Steering Committee; 
* Among other things, meetings were used to review progress on the 
implementation and development of the BusinessUSA website, the 
performance of the website and customer contact center in the most 
recent quarter, planned actions, and the status of key deliverables.  

Source: GAO analysis of documentation and interviews with cross-agency 
priority goal representatives. GAO-14-526.  

[End of table]  

Data Collection and Review Meeting Preparation. Leading practices also 
indicate that those managing review processes should have the capacity 
to collect, analyze, and communicate accurate, useful, and timely 
performance data, and should rigorously prepare for reviews to enable 
meaningful performance discussions. The collection of current, 
reliable data on the status of activities and progress towards goals 
and milestones is critical so that those involved can determine 
whether performance is improving, identify performance problems, 
ensure accountability for fulfilling commitments, and learn from 
efforts to improve performance. The ability to assess data to identify 
key trends and areas of strong or weak performance, and to communicate 
this to managers and staff effectively through materials prepared for 
reviews, is also critical. As detailed in table 2, those supporting 
the Cybersecurity and Skills Gap goals, and the BusinessUSA sub-goal, 
instituted processes to regularly collect and analyze data on progress 
towards identified goals and milestones, and to ensure these data 
would be communicated through materials prepared for review meetings. 

Table 2: Collection of Performance Data and Meeting Preparation for 
Review Processes Reflecting Leading Practices:  

Cybersecurity: 
* Each quarter, Department of Homeland Security (DHS) officials 
collected and analyzed data on cybersecurity performance that agencies 
submitted through CyberScope, an online data collection portal; 
* Prior to each review, DHS staff would develop presentation slides 
and materials with key data from the agency participating in the 
review, including projected performance on key metrics for the next 8 
quarters; 
* Prior to each review, DHS staff would also meet with officials from 
the participating agency to prepare for the issues that would be 
discussed and to address any errors in the data or information 
provided.  

Closing Skills Gaps: 
* Each quarter, prior to the quarterly review meeting, Office of 
Personnel Management (OPM) staff would collect updated information 
from sub-goal leaders, including recent progress on identified sub-
goal performance metrics, achievements, and planned actions; 
* This information was compiled into a set of presentation slides for 
the review meeting, with information for each sub-goal presented in a 
consistent fashion; 
* OPM and OMB staff and sub-goal leaders would also hold preparatory 
phone calls in advance of the review meetings which, according to OPM 
staff, helped ensure that participants would focus their discussion on 
the most pressing issues. They noted that this was important given 
that the time available to discuss each sub-goal during the review 
meeting was limited.  

BusinessUSA: 
* Each quarter, the BusinessUSA project management office would 
collect data on key performance metrics, such as the number of unique 
visitors to the website and satisfaction with the customer contact 
center; 
* These data on website and contact center performance, as well as 
future targets, were included in the presentation slides developed for 
the monthly Steering Committee meetings; 
* These slides also listed specific deliverables and milestones 
related to actions being taken to improve the operational and 
technological performance of BusinessUSA, as well as the experience of 
customers; 
* The slides also listed specific risks in each of these three areas, 
along with, on a scale of 1 to 5, their likelihood and the severity of 
their potential consequences.  

Source: GAO analysis of documentation and interviews with cross-agency 
priority goal representatives. GAO-14-526.  

[End of table]  

Participation by Key Officials. Leading practices indicate that key 
players involved in efforts to achieve a goal should attend reviews to 
facilitate problem solving. This is critical as their participation 
enables those involved to break down information silos, and to use the 
forum provided by the review to communicate with each other, identify 
improvement strategies, and agree on specific next steps. Reviews for 
both the Cybersecurity and Closing Skills Gaps CAP goals, and the 
BusinessUSA sub-goal, were structured so that relevant agency 
officials playing a key role in efforts to carry out the goal were 
included, as detailed in table 3. 

Table 3: Participation by Key Officials in Review Processes Reflecting 
Leading Practices:  

Cybersecurity: 
* Review meetings were attended by key agency officials responsible 
for the implementation of cybersecurity requirements in their agencies; 
* This included the agency Chief Information Officer (CIO), Deputy 
CIO, Chief Information Security Officer, and Performance Improvement 
Officer; 
* Staff from DHS, National Security Staff, and OMB involved in the 
administration of cybersecurity requirements also participated in the 
reviews.  

Closing Skills Gaps: 
* Quarterly review meetings were attended by agency and administration 
officials serving as the leaders for each of the six sub-goal areas, 
and who were responsible for efforts to drive progress toward each 
specific sub-goal; 
* The meetings were also attended by staff from OPM that provided 
support to the sub-goal leaders, and staff from the OMB Office of 
Performance and Personnel Management.  

BusinessUSA: 
* Monthly BusinessUSA Steering Committee meetings were attended by mid-
to senior-level program, technology and customer service managers from 
each of the departments and agencies participating in the initiative.  

Source: GAO analysis of documentation and interviews with cross-agency 
priority goal representatives. GAO-14-526.  

[End of table]  

Review Follow-Up. Leading practices indicate that participants should 
engage in sustained follow-up on issues identified during reviews, 
which is critical to ensure the success of the reviews as a 
performance improvement tool. Important follow-up activities include 
identifying and documenting specific follow-up actions stemming from 
reviews, those responsible for each action item, as well as who will 
be responsible for monitoring and follow-up. Follow-up actions should 
also be included as agenda items for subsequent reviews to hold 
responsible officials accountable for addressing issues raised and 
communicating what was done. Goal managers for the Cybersecurity and 
Closing Skills Gap CAP goals, as well as the BusinessUSA sub-goal, 
took steps to follow up on action items identified in these meetings, 
and to ensure that steps were taken towards their completion, as 
detailed in table 4. 

Table 4: Follow-Up on Issues Raised During Review Meetings:  

Cybersecurity: 
* Following reviews, agency officials would develop an after-action 
plan outlining specific actions that agency officials would be 
responsible for completing, such as the development of a plan for the 
improved implementation of a specific requirement or holding 
discussions on how to address a specific issue with knowledgeable 
officials from another agency; 
* DHS officials would periodically follow up with agencies on their 
efforts to address these action items, asking them to document 
specific steps taken.  

Closing Skills Gaps: 
* Following each quarterly review meeting, OPM staff supporting each 
sub-goal would follow up with sub-goal leaders on items identified for 
follow-up during the meetings; 
* Sub-goal leaders were then responsible for taking action to address 
these items; 
* To help manage efforts at the sub-goal level, sub-goal leaders 
developed action plans outlining specific deliverables, those 
responsible for implementation, time frames for completion, and 
potential barriers to their successful completion.  

BusinessUSA: 
* At BusinessUSA Steering Committee meetings, the sub-goal leader 
would review action items identified and discussed during the previous 
month's meeting; 
* The sub-goal leader would also discuss progress made over the 
previous month and specific steps that would be taken to further 
progress; 
* Following each meeting, staff would distribute summaries of the 
meeting to participants, which included specific actions to be 
completed by the BusinessUSA program office or agency representatives.  

Source: GAO analysis of documentation and interviews with cross-agency 
priority goal representatives. GAO-14-526.  

[End of table]  

Review Effects. Goal leaders and managers we interviewed said that 
these review processes were valuable in driving improved performance, 
establishing a greater sense of accountability for progress on the 
part of contributors, and in providing a forum for interagency 
communication and collaboration. For example, according to DHS staff 
involved in the management of the Cybersecurity CAP goal, 
implementation of Personal Identity Verification (PIV) requirements 
across the federal government had been stagnant for several years 
prior to the introduction of cybersecurity as a CAP goal. The review 
process was used to hold agencies accountable for improved PIV 
implementation, which helped bring an increased focus on the issue and 
drive recent progress. Since the reviews were instituted in 2012, DHS 
has reported improved PIV adoption in civilian agencies, which has 
increased from 1.24 percent in fiscal year 2010, to 7.45 percent in 
fiscal year 2012, to 19.67 percent in the fourth quarter of fiscal 
year 2013. According to data from DHS, while still falling short of 
the target, this has contributed to the overall increase in PIV 
adoption across the federal government--including both civilian 
agencies and the Department of Defense--from 57.26 percent in fiscal 
year 2012 to 66.61 percent in fourth quarter of fiscal year 2013. 
[Footnote 34] DHS staff also added that agencies generally had not 
previously collaborated on cybersecurity issues or worked to identify 
best practices. According to DHS staff, the reviews have created an 
important point of collaboration between DHS, OMB, National Security 
Staff, and agencies, and provided an opportunity to inform agencies of 
best practices and connect them with other agencies that are meeting 
their targets to learn from them. 

Similarly, OPM officials and sub-goal leaders involved in the 
management of the Closing Skills Gap CAP goal said that the quarterly 
review meetings were a critical means to ensure sub-goal leaders and 
staff were demonstrating progress. Having sub-goal leaders report out 
on progress, and hear about the progress made in other sub-goal areas, 
provided additional pressure for continuous improvement and the need 
to remain focused on driving progress towards their goals. Having the 
goal leader lead the review was also a way to demonstrate leadership 
commitment to the achievement of each sub-goal. According to OPM 
officials and sub-goal leaders, the review meetings also served as an 
important forum for discussing innovative approaches being taken to 
address skills gaps in different areas, opportunities for 
collaboration to address challenges shared by different sub-goals, and 
how leaders could leverage the efforts of other sub-goals to drive 
progress on their own. 

The BusinessUSA sub-goal leader said that having it as the basis for a 
CAP sub-goal elevated the cross cutting nature of the initiative. In 
addition to reviewing performance information and the status of 
deliverables, discussions at inter-agency Steering Committee meetings 
were used to discuss how contributors could work together to meet the 
initiative's performance goals. This communication and coordination 
led to connections between agencies and to discussions about how 
programs could be working in a more integrated way. For example, these 
discussions were used to identify ways that programs could more 
effectively integrate program information on the BusinessUSA website 
to increase customer satisfaction. 

Other Goal Leaders Did Not Use Review Processes Consistent with the 
Full Range of Leading Practices for Reviews:  

We found that the processes used by other CAP goal leaders to engage 
agency officials in the review of progress, which are summarized in 
appendix IV, did not reflect the full range of leading practices. For 
example, the process for reviewing progress on the Job Training CAP 
goal involved staff from the PIC collecting updates on recent 
milestones from agencies, which were then compiled in the quarterly 
status update and reviewed by the goal leader. This approach was used 
by the goal leaders for the Broadband and STEM Education CAP goals to 
review progress as well. While goal leaders and managers for these 
goals indicated that they used the collection and review of 
information as an opportunity to communicate with officials from 
contributing agencies, this approach contrasts with OMB guidance for 
reviews of agency priority goals, which states explicitly that 
performance reviews should not be conducted solely through the sharing 
of written communications. As OMB noted in its guidance, in-person 
engagement of leaders in performance reviews greatly accelerates 
learning and performance improvement, and personal engagement can 
demonstrate commitment to improvement, ensure coordination across 
agency silos, and enable rapid decision making. 

While not employing the full range, goal leaders for a number of goals 
did use processes that reflected one or more leading practices. For 
example, many CAP goal leaders led or participated in interagency 
meetings with representatives of contributing agencies. While these 
were used to facilitate interagency communication and collaboration on 
the development of plans and policies, it was unclear whether many of 
these meetings were consistently used to review progress on identified 
CAP goals and milestones. The goal leader for the Strategic Sourcing 
CAP goal used processes that reflected leadership involvement, 
participation by key officials, and the collection and analysis of 
relevant data. Specifically, according to goal managers, the goal 
leader led regular meetings of the Strategic Sourcing Leadership 
Council (SSLC), which were attended by senior procurement officials 
from eight agencies that combine to make up almost all of the federal 
government's total procurement spending. 

To prepare for each SSLC meeting, staff from OMB's Office of Federal 
Procurement Policy (OFPP) held a meeting for supporting staff from 
each agency, who would then prepare the SSLC member from their agency 
for the issues to be discussed in the SSLC meeting. OFPP also 
established a regular data collection process where each agency would 
report on its adoption and spending rates for two strategic sourcing 
options, which would then be used for the purposes of reporting on the 
CAP goal. However, it was unclear how regularly, if at all, SSLC 
meetings were used to engage agency officials in the review of data on 
agency adoption of, and spending on, strategic sourcing options, or 
how regularly meetings were used to review progress that was being 
made towards the CAP goal. It was also unclear what mechanisms, if 
any, were used to ensure rigorous follow-up on issues raised in these 
meetings, a key leading practice, as there were no official meeting 
minutes maintained. The lack of an official record could hinder follow-
up and accountability for any identified actions that need to be taken. 

Some Goal Leaders Reported Minimal Effects on Performance and 
Collaboration:  

Representatives of some goals stated that it was difficult to isolate 
the impact of the CAP goal designation, and its associated reporting 
and review requirements, on performance and collaboration. According 
to some goal managers, because their interim goals were based on 
initiatives that had been previously established in executive orders 
or Presidential memorandums, much of the interagency activity 
supporting their efforts would have happened without the CAP goal 
designation and its reporting and review requirements. For example, a 
manager for the Data Center Consolidation CAP goal told us that the 
previously established Federal Data Center Consolidation Initiative 
was used to drive progress and that the CAP goal designation and 
quarterly reporting and review requirements had little impact. 
Similarly, Job Training CAP goal managers said that interagency 
collaboration on job training issues had been established prior to the 
creation of the CAP goal, that the goal's reporting and review 
requirements were incidental to the contributors' ongoing work, and 
that it did not add an additional level of accountability for the 
completion of job training initiatives. However, this is a goal where 
no data were reported to demonstrate its impact on federal job 
training programs, and which was identified in multiple OMB reviews as 
having slower than anticipated progress due, in part, to extended 
periods of time in which there was no deputy CAP goal leader to 
provide support necessary to improve coordination and collaboration. 

While many CAP goal leaders and staff we interviewed noted the 
progress they had made with their existing interagency meetings and 
approaches, a lack of clear expectations or guidance for how review 
processes at the CAP goal level should be carried out can lead to a 
situation where reviews are implemented in a manner that is not 
informed by, or fully consistent with, leading practices. This could 
result in missed opportunities to realize the positive effects on 
performance and accountability that can stem from the implementation 
of review processes that regularly and consistently involve leaders 
and agency officials in the analysis of performance data to identify 
and address performance deficiencies, and use rigorous follow-up to 
ensure accountability for commitments. 

Conclusions:  

Many of the meaningful results that the federal government seeks to 
achieve require the coordinated efforts of more than one federal 
agency. GPRAMA's requirement that OMB establish CAP goals offers a 
unique opportunity to coordinate cross-agency efforts to drive 
progress in priority areas. That opportunity will not be realized, 
however, if the CAP goal reporting and review requirements and leading 
review practices are not followed. The reporting and review 
requirements for the CAP goals, and leading practices for the reviews, 
are designed to ensure that relevant performance information is used 
to improve performance and results, and that OMB and goal leaders 
actively lead efforts to engage all relevant participants in 
collaborative performance improvement initiatives and hold them 
accountable for progress on identified goals and milestones. 

OMB reported performance information in the quarterly CAP goal status 
updates it published on Performance.gov. While updates for most goals 
reported data on performance towards the identified planned level of 
performance, the information in the updates did not always present a 
complete picture of progress towards identified goals and milestones. 
For example, while updates for 8 of the 14 goals included data that 
indicated performance towards the identified overall planned level of 
performance, only 3 also contained annual or quarterly targets that 
allowed for an assessment of interim progress. Updates for the other 6 
of the 14 goals did not report on performance towards the goal's 
primary performance target because the goal was established without a 
quantitative target or because goal managers were unable to collect 
the data needed to track performance. In other cases, planned 
activities that were identified as contributing to the goal were 
sometimes missing important elements, including alignment with the 
strategies for goal achievement they supported, a time frame for 
completion, or information on their implementation status. The 
incomplete picture of progress that many of the updates gave limited 
the ability of goal leaders and others to ensure accountability for 
the achievement of targets and milestones. 

Holding regular progress reviews that are consistent with GPRAMA 
requirements and the full range of leading practices can produce 
positive effects on performance and collaboration. Engaging 
contributors in regular reviews of data on performance can help ensure 
interagency efforts are informed by information on progress towards 
identified goals and milestones, which can be used to identify and 
address areas where goal or milestone achievement is at risk. Reviews 
can also be used to reinforce agency and collective accountability for 
the achievement of individual and shared outcomes, helping to ensure 
that efforts to improve performance or address identified risks are 
implemented. Lastly, reviews can be used to foster greater 
collaboration, ensuring opportunities for communication and 
coordination between officials involved in efforts to achieve shared 
outcomes. While OMB and CAP goal leaders instituted processes for 
reviewing progress on the interim CAP goals, if GPRAMA requirements 
and leading practices for reviews are not consistently followed, it 
may result in missed opportunities to improve performance, hold 
officials accountable for achieving identified goals and milestones, 
and ensure agency officials are coordinating their activities in a way 
that is directed towards the achievement of shared goals and 
milestones. 

Recommendations for Executive Action:  

We recommend that the Director of OMB take the following three 
actions:  

* Include the following in the quarterly reviews of CAP goal progress, 
as required by GPRAMA: a consistent set of information on progress 
made during the most recent quarter, overall trends, and the 
likelihood of meeting the planned level of performance; goals at risk 
of not achieving the planned level of performance; and the strategies 
being employed to improve performance. 

* Work with the PIC to establish and document procedures and criteria 
to assess CAP goal implementation efforts and the status of goal 
execution, to ensure that the PIC can conduct these assessments 
consistently across all goals and over time. 

* Develop guidance similar to what exists for agency priority goal and 
strategic objective reviews, outlining the purposes of CAP goal 
progress reviews, expectations for how the reviews should be carried 
out, and the roles and responsibilities of CAP goal leaders, agency 
officials, and OMB and PIC staff in the review process. 

To ensure that OMB and CAP goal leaders include all key contributors 
and can track and report fully on progress being made towards CAP 
goals overall and each quarter, we recommend that the Director of OMB 
direct CAP goal leaders to take the following four actions:  

* Identify all key contributors to the achievement of their goals; 

* Identify annual planned levels of performance and quarterly targets 
for each CAP goal; 

* Develop plans to identify, collect, and report data necessary to 
demonstrate progress being made towards each CAP goal or develop an 
alternative approach for tracking and reporting on progress quarterly; 
and:  

* Report the time frames for the completion of milestones; the status 
of milestones; and how milestones are aligned with strategies or 
initiatives that support the achievement of the goal. 

Agency Comments:  

We provided a draft of this report for review and comment to the 
Director of OMB, the Secretaries of Commerce and Homeland Security, 
the Director of the Office of Personnel Management, the Administrator 
of the Small Business Administration, as well as the officials we 
interviewed to collect information on the interim CAP goals from the 
Council on Environmental Quality, Department of Education, Department 
of Labor, Department of Veterans Affairs, National Science Foundation, 
and the Office of Science and Technology Policy. OMB and PIC staff 
provided oral comments on the draft, and we made technical changes as 
appropriate. OMB staff generally agreed to consider our 
recommendations. For example, while they said that OMB and PIC staff 
will continue to work directly with CAP goal leaders to convey 
suggested practices for reviewing performance, they will consider 
referencing principles and practices for data-driven performance 
reviews in future Circular A-11 guidance related to the management of 
CAP goals. Furthermore, while they noted that quantitative performance 
data for some key measures may not available on a quarterly basis, 
they said that they will continue to work to develop more robust 
quarterly targets. Officials or staff from the Departments of Commerce 
and Veterans Affairs, and the Office of Science and Technology Policy 
provided technical comments, which we incorporated as appropriate. 

We are sending copies of this report to the Director of OMB as well as 
appropriate congressional committees and other interested parties. The 
report is also available at no charge on the GAO website at 
[hyperlink, http://www.gao.gov]. 

If you or your staff have any questions concerning this report, please 
contact me at (202) 512-6806 or mihmj@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. Key contributors to this report are 
listed in appendix VI.  

Signed by:  

J. Christopher Mihm: 
Managing Director, Strategic Issues:  

List of Congressional Addressees:  

The Honorable Thomas R. Carper: 
Chairman: 
The Honorable Tom Coburn, M.D.
Ranking Member: 
Committee on Homeland Security and Governmental Affairs: 
United States Senate:  

The Honorable Mark R. Warner: 
Chairman: 
Task Force on Government Performance: 
Committee on the Budget: 
United States Senate:  

The Honorable Elijah E. Cummings: 
Ranking Member: 
Committee on Oversight and Government Reform: 
House of Representatives:  

[End of section]  

Appendix I: Objectives, Scope, and Methodology:  

This report is part of our response to a mandate that we evaluate the 
implementation of the federal government priority goals under the GPRA 
Modernization Act of 2010 (GPRAMA).[Footnote 35] Due to the timing of 
our work, we focused on the implementation of the reporting and review 
requirements for the 14 interim cross-agency priority (CAP) goals 
established in February 2012.[Footnote 36] Specifically, this report 
assesses (1) what is known about progress made towards the interim CAP 
goals; and (2) how, if at all, quarterly progress reflected GPRAMA 
requirements and leading practices for data-driven reviews, as well as 
how they contributed to improved cross-agency performance and 
collaboration. 

To address these objectives, we interviewed representatives of 13 of 
the 14 interim goals.[Footnote 37] For 8 of the 13 goals we spoke 
directly with the goal leader or deputy goal leader, along with, in 
some cases, staff from Office of Management and Budget (OMB) and 
agencies involved in supporting efforts related to the goals. For the 
other five goals (Closing Skills Gaps, Cybersecurity, Data Center 
Consolidation, Exports, and Job Training) we met with agency officials 
or OMB staff playing a key role in the management of interagency 
efforts related to the CAP goal. During these interviews, we asked 
officials questions concerning how the goal leader and officials from 
contributing agencies reviewed progress on the goal; the interagency 
groups used to engage agency officials and manage efforts related to 
the goal; the role that staff from OMB and the Performance Improvement 
Council (PIC) played in the review process; and any impact the CAP 
goal designation and review processes had on performance, 
collaboration, and accountability. We also participated in interviews 
with the goal leaders of 11 agency priority goals that were aligned 
with, or identified as a contributor to, a CAP goal.[Footnote 38] 

To further address the first objective, and assess what is known about 
progress made toward the interim CAP goals, we analyzed information on 
identified performance metrics and milestones included in the 
quarterly status updates for each CAP goal published on 
Performance.gov. We also analyzed relevant information collected 
through our interviews with CAP goal leaders, deputies, and supporting 
staff. We compared the data and information made available through the 
quarterly status updates with requirements in GPRAMA that 
Performance.gov include information for each goal on results achieved 
during the most recent quarter and overall trend data. To assess the 
reliability of performance data and information available through 
Performance.gov we collected information from OMB and PIC staff, and 
CAP goal representatives, about data quality control procedures. We 
determined that the data and information were sufficiently reliable 
for our analysis of what was reported on Performance.gov about 
progress towards identified goals and milestones. 

To address the second objective, we reviewed quarterly review 
memorandums developed for OMB leadership for five quarters, from the 
third quarter of fiscal year 2012 to the third quarter of fiscal year 
2013.[Footnote 39] We compared the contents of these review 
memorandums with requirements for the OMB quarterly reviews 
established in GPRAMA. We also interviewed staff from OMB and the PIC 
to discuss the various approaches being used to review progress at the 
CAP-goal level, the data collection and review process, and the role 
of the PIC in supporting the quarterly review process. 

To further address the second objective we reviewed (where available) 
documents created for interagency meetings, such as meeting agendas, 
presentation materials, meeting notes, and attendee lists. We also 
observed one quarterly review meeting held for the Closing Skills Gap 
goals, and conducted interviews with sub-goal leaders from the Closing 
Skills Gaps and Entrepreneurship and Small Business CAP goals. These 
interviews were used to learn more about the involvement of officials 
from contributing agencies in the quarterly review process for each 
CAP goal, the processes that had been established to review progress 
at the sub-goal level, and to gain a more complete picture of 
participating agency officials' perceptions of the impact of the CAP 
goals and review processes. 

We selected these sub-goals through a two-part process. Of the eight 
CAP goals for which we had completed interviews through the end of 
2013, the team selected one goal for which the goal leader held 
quarterly meetings dedicated to reviewing progress toward the CAP goal 
with the goal's contributors (Closing Skills Gaps). The team also 
selected a second goal for which the goal leader used a review process 
that did not rely on quarterly meetings between the goal leader and 
contributing agencies (Entrepreneurship and Small Business). To ensure 
that the team would have at least one goal representing each type of 
goal, the team also ensured that one goal would be an outcome-oriented 
policy goal and one goal would be a management goal. For both the 
Closing Skills Gaps and Entrepreneurship and Small Business CAP goals 
the team then selected four sub-goals for interviews. For the Closing 
Skills Gaps CAP goal the team interviewed the sub-goal leaders for the 
Economist; Information Technology/Cybersecurity; Science, Technology, 
Engineering, and Mathematics (STEM) Education; and Human Resources sub-
goals. For the Entrepreneurship and Small Business CAP goal the team 
held interviews with the sub-goal leaders for the sub-goals to 
"Accelerate commercialization of Federal research grants," "Advance 
federal small business procurement goals," "Improve access to 
government services and information," and "Streamline immigration 
pathways for immigrant entrepreneurs." These were selected to ensure 
that the team would capture sub-goals in which a range of approaches 
for measuring and reviewing progress were being used. Specifically, 
sub-goals were selected to ensure the team would have some that did, 
and did not, hold regular meetings, and some that did, and did not, 
track quantitative measures of performance or milestones with time 
frames. Our selection of these sub-goals was nonstatistical and 
therefore our findings from these interviews are not generalizable to 
the other CAP goals. 

We compared what we learned about review processes at the CAP goal and 
sub-goal levels, through interviews and the collection of 
documentation, used by leaders from each goal against leading 
practices for performance reviews previously identified by GAO. 

Because the scope of our review was to examine the implementation of 
quarterly progress reviews, we did not evaluate whether these goals 
were appropriate indicators of performance, sufficiently ambitious, or 
met other dimensions of quality. 

We conducted our work from May 2013 to June 2014 in accordance with 
generally accepted government auditing standards. Those standards 
require that we plan and perform the audit to obtain sufficient, 
appropriate evidence to provide a reasonable basis for our findings 
and conclusions based on our audit objectives. We believe that the 
evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives.  

[End of section]  

Appendix II: Interim Cross-Agency Priority Goals and Goal Statements:  

Goal: Broadband; 
Goal Statement: As part of expanding all broadband capabilities, 
ensure 4G wireless broadband coverage for 98 percent of Americans by 
2016.  

Goal: Closing Skills Gaps; 
Goal Statement: Close critical skills gaps in the federal workforce to 
improve mission performance. By September 30, 2013, close the skills 
gaps by 50 percent for three to five critical federal government 
occupations or competencies, and close additional agency-specific high 
risk occupation and competency gaps.  

Goal: Cybersecurity; 
Goal Statement: Executive branch departments and agencies will achieve 
95 percent implementation of the administration's priority 
cybersecurity capabilities by the end of FY 2014. These capabilities 
include strong authentication, trusted Internet connections, and 
continuous monitoring.  

Goal: Data Center Consolidation; 
Goal Statement:Improve information technology service delivery, reduce 
waste, and save $3 billion in taxpayer dollars by closing at least 
2,500 data centers by fiscal year 2015.  

Goal: Energy Efficiency; 
Goal Statement: Increase energy productivity (amount of real gross 
domestic product in dollars/energy demand) 50 percent by 2030.  

Goal: Entrepreneurship and Small Business; 
Goal Statement: Increase federal services to entrepreneurs and small 
businesses with an emphasis on 1) startups and growing firms and 2) 
underserved markets.  

Goal: Exports; 
Goal Statement: Double U.S. exports by the end of 2014.  

Goal: Improper Payments; 
Goal Statement: The federal government will achieve a payment accuracy 
rate of 97 percent by the end of 2016.  

Goal: Job Training; 
Goal Statement: Ensure our country has one of the most skilled 
workforces in the world by preparing 2 million workers with skills 
training by 2015 and improving the coordination and delivery of job 
training services.  

Goal: Real Property; 
Goal Statement: The federal government will maintain the fiscal year 
2012 square footage baseline of its office and warehouse inventory.  

Goal: Science, Technology, Engineering, and Mathematics (STEM) 
education; 
Goal Statement: In support of the president's goal that the U.S. have 
the highest proportion of college graduates in the world by 2020, the 
federal government will work with education partners to improve the 
quality of STEM education at all levels to help increase the number of 
well-prepared graduates with STEM degrees by one-third over the next 
10 years, resulting in an additional 1 million graduates with degrees 
in STEM subjects.  

Goal: Strategic sourcing; 
Goal Statement: Reduce the costs of acquiring common products and 
services by agencies' strategic sourcing of at least two new 
commodities or services in both 2013 and 2014, that yield at least a 
10 percent savings. In addition, agencies must increase their use of 
Federal Strategic Sourcing Initiative vehicles by at least 10 percent 
in both fiscal years 2013 and 2014.  

Goal: Sustainability; 
Goal Statement: By 2020, the federal government will reduce its direct 
greenhouse gas emissions by 28 percent and will reduce its indirect 
greenhouse gas emissions by 13 percent by 2020 (from 2008 baseline).  

Goal: Veteran Career Readiness; 
Goal Statement: By September 30, 2013, increase the percent of 
eligible service members who will be served by career readiness and 
preparedness programs from 50 percent to 90 percent in order to 
improve their competitiveness in the job market.  

Source: Performance.gov. GAO-14-526.  

[End of table]  

[End of section]  

Appendix III: Interagency Group Membership and Meeting Frequency and 
Purpose:  

Goal leaders for 13 of 14 cross-agency priority (CAP) goals leveraged 
interagency groups for the purposes of coordinating efforts designed 
to contribute to progress on the cross-agency priority goal. This 
appendix includes information on the membership of these interagency 
groups, the frequency with which they met, and the purposes of those 
meetings. 

Goal or sub-goal: Broadband; 
Interagency group: Broadband Deployment on Federal Property Working 
Group; 
Membership: Fourteen agencies with federal property management or 
transportation funding responsibilities, and broadband or other 
related expertise; 
Meeting frequency: Quarterly; 
Goal or sub-goal: To discuss best practices on broadband-related land 
management issues, and actions to implement an executive order on 
accelerating broadband infrastructure deployment.  

Goal or sub-goal: Broadband; 
Interagency group: Spectrum Policy and Plans Steering Group; 
Membership: Senior officials from agencies considered major spectrum 
stakeholders and users of spectrum, including the Departments of 
Defense, Justice, Homeland Security (DHS), Commerce, and the National 
Aeronautics and Space Administration (NASA); 
Meeting frequency: Quarterly; 
Goal or sub-goal:To provide advice on spectrum policy and strategic 
plans, discuss commercial transfer of federal agency spectrum, and 
resolve issues affecting federal/non-federal users.  

Goal or sub-goal: Closing Skills Gaps; 
Interagency group: Closing Skills Gaps CAP Goal Leadership Team; 
Membership: Goal leader and sub-goal leaders for six sub-goal areas; 
Meeting frequency: Quarterly; 
Goal or sub-goal: To review progress on performance metrics and 
actions taken to close skills gaps in each of the six sub-goal areas.  

Goal or sub-goal: Cybersecurity: 
Interagency group: Cybersecurity CAP Goal Governance Committee; 
Membership: Officials from National Institute of Standards and 
Technology, General Services Administration (GSA), DHS, National 
Security Staff, Office of Management and Budget (OMB) and Performance 
Improvement Council; 
Meeting frequency: Twice each quarter; 
Goal or sub-goal: Beginning in 2013, a meeting was held each quarter 
prior to the collection of data on agency progress on cybersecurity 
metrics. Another was held after data had been collected and analyzed 
to review and discuss agency progress.  

Goal or sub-goal: Data Center Consolidation; 
Interagency group: Federal Data Center Consolidation Task Force; 
Membership: Data center consolidation program managers from 24 federal 
agencies; 
Meeting frequency: Quarterly; 
Goal or sub-goal: To identify and disseminate key information about 
solutions and processes to help agencies make progress towards data 
center consolidation goals.  

Goal or sub-goal: Energy Efficiency: 
No interagency groups were used to manage efforts related to this 
goal.  

Goal or sub-goal: Entrepreneurship and Small Business; 
Interagency groups were used to manage efforts at the sub-goal level.  

Goal or sub-goal: Improve Access to Government Information and 
Services Sub-Goal: 
Interagency group: BusinessUSA Executive Committee; 
Membership: Senior-level representatives from 24 participating 
agencies; 
Meeting frequency: Quarterly; 
Goal or sub-goal: To oversee strategy, resources and timetables for 
the development of the BusinessUSA website, resolve interagency issues 
and ensure department/agency viewpoints are represented.  

Goal or sub-goal: Improve Access to Government Information and 
Services Sub-Goal: 
Interagency group: BusinessUSA Steering Committee; 
Membership: Mid-to-senior level program, technology and customer 
service managers from 24 participating agencies; 
Meeting frequency: Monthly; 
Goal or sub-goal: To assist the BusinessUSA program management office 
coordinate the design, development, and operation of the BusinessUSA 
website, and to track and monitor performance metrics on customer 
service and outcomes.  

Goal or sub-goal: Commercialization of Federal Research Grants Sub-
Goal; 
Interagency group: Small Business Innovation Research (SBIR) and Small 
Business Technology Transfer (STTR) Program Manager Group; 
Membership: SBIR/STTR program managers from11 agencies, and 
coordinating officials from Small Business Administration (SBA) and 
Office of Science and Technology Policy (OSTP); 
Meeting frequency: Monthly; 
Goal or sub-goal: To discuss the development of SBIR/STTR program 
policy directives, the implementation of requirements, outreach and 
access to the programs, and program best practices.  

Goal or sub-goal: Streamlining Pathways for Immigrant Entrepreneurs 
Sub-Goal; 
Interagency group: Interagency immigrant entrepreneurship group; 
Membership: Officials from DHS, Departments of State and Commerce, 
OSTP, SBA, and Domestic Policy Council; 
Meeting frequency: As necessary; 
Goal or sub-goal: To provide updates on relevant agency activities and 
identify opportunities for interagency collaboration.  

Goal or sub-goal: Small Business Procurement Sub-Goal; 
Interagency group: Small Business Procurement Advisory Council; 
Membership: Office of Small and Disadvantaged Business representative 
from federal agencies; 
Meeting frequency: Monthly; 
Goal or sub-goal: To share best practices for expanding contracting to 
small and disadvantaged businesses, and reviewing progress on agency 
simplified-acquisition threshold goals.  

Goal or sub-goal: Small Business Procurement Sub-Goal; 
Interagency group: White House Small Business Procurement Working 
Group; 
Membership: Agency principals and staff from 27 federal agencies; 
Meeting frequency: Annually; 
Goal or sub-goal: To provide officials from the White House, SBA, 
Commerce, and OMB with an opportunity to meet with senior agency 
leaders and discuss the steps agencies are taking to increase small 
business contracting.  

Goal or sub-goal: Exports; 
Interagency group: Trade Promotion Coordinating Committee/Export 
Promotion Cabinet; 
Membership: Principals (cabinet secretaries and deputies) and staff 
from 20 agencies involved in export policy, service, finance, and 
oversight; 
Meeting frequency: 2-3 times a year; 
Goal or sub-goal: To review progress on deliverables supporting the 
National Export Strategy, communications, and the status of individual 
export promotion initiatives.  

Goal or sub-goal: Improper Payments; 
Interagency group: Do Not Pay Implementation Group; 
Membership: Officials from 24 federal agencies; 
Meeting frequency: Bi-weekly to monthly; 
Goal or sub-goal: To review the status of agency implementation of Do 
Not Pay requirements and milestones, and guidance for implementation.  

Goal or sub-goal: Improper Payments; 
Interagency group: Senior Accountable Officials Group; 
Membership: Officials from agencies with "high-priority" programs, as 
designated by OMB; 
Meeting frequency: As necessary; 
Goal or sub-goal: To discuss the government-wide improper payment 
initiative and overall strategy.  

Goal or sub-goal: Job Training; 
Interagency group: Job Training Data Access Working Group; 
Membership: Officials from Departments of Education, Health and Human 
Services (HHS), Labor, and OMB; 
Meeting frequency: As Necessary; 
Goal or sub-goal: To discuss expanding access to job training 
performance data, and opportunities to promote its use at the local, 
state, and federal levels.  

Goal or sub-goal: Real Property; 
Interagency group: Federal Real Property Council; 
Membership: Senior Real Property Officers from 24 federal agencies; 
Meeting frequency: Quarterly; 
Goal or sub-goal: Among other policy discussions, to discuss the 
development of agency "Freeze the Footprint" plans.  

Goal or sub-goal: Sustainability; 
Interagency group: Steering Committee on Federal Sustainability; 
Membership: Officials from Council on Environmental Quality, OMB, and 
Senior Sustainability Officers from federal agencies; 
Meeting frequency: Biannually; 
Goal or sub-goal: To discuss policy to guide the federal government on 
sustainability issues, and to discuss sustainability goals.  

Goal or sub-goal: Science, Technology, Engineering, and Mathematics 
(STEM) Education; 
Interagency group: Federal Coordination in STEM Education Task Force; 
Membership: Representatives from 12 federal agencies - including 
Department of Education, National Institutes of Health, NASA, and 
National Science Foundation - and the Smithsonian Institution; 
Meeting frequency: Every 4-6 weeks, during the development of the 5-
year strategic plan; 
Goal or sub-goal: To develop a 5-year strategic plan for federal 
support for STEM education.  

Goal or sub-goal: Strategic Sourcing; 
Interagency group: Representatives from Departments of Defense, 
Energy, and Veterans Affairs (VA), DHS, HHS, GSA, NASA, and SBA; 
Meeting frequency: Monthly; 
Goal or sub-goal: To discuss the development and adoption of strategic 
sourcing options.  

Goal or sub-goal: Veteran Career Readiness; 
Interagency group: Interagency Policy Council; 
Membership: Representatives from Departments of Defense, Education, 
and Labor, HHS, VA, Office of Personnel Management (OPM), OMB, SBA; 
Meeting frequency: Every 2 months; 
Goal or sub-goal: To review ongoing policy initiatives and 
opportunities for collaboration between agencies.  

Goal or sub-goal: Veteran Career Readiness; 
Interagency group: Veterans Employment Initiative Task Force; 
Membership: Representatives from Departments of Defense, Education, 
and Labor, VA, OPM, and SBA; 
Meeting frequency: Monthly; 
Goal or sub-goal: To develop and implement a redesigned veterans 
transition program.  

Source: GAO analysis of information and documentation collected from 
CAP goal representatives. GAO-14-526.  

[End of table]  

[End of section]  

Appendix IV: Description of Interim Cross-Agency Priority Goal Review 
Processes:  

Goal: Broadband; 
Description of review process: 
* The goal leader tasked staff from the Department of Commerce 
(Commerce) with collecting updated information from contributing 
agencies for quarterly updates; 
* This information was then reviewed by the goal leader, who would 
follow-up directly with staff from contributing agencies, as 
necessary.  

Goal: Closing Skills Gaps; 
Description of review process: 
* Each quarter the goal leader hosted a review meeting with the 
leaders of six sub-goal areas to review progress on performance goals 
established for each sub-goal, as well as achievements, challenges and 
future plans.  

Goal: Cybersecurity; 
Description of review process: 
* Beginning in 2013, a Cybersecurity Cross-Agency Priority Goal 
Governance Committee, made up of officials from the National Institute 
of Standards and Technology, General Services Administration, Homeland 
Security (DHS), National Security Staff, Office of Management and 
Budget (OMB), and the Performance Improvement Council (PIC) would meet 
twice each quarter to review agency data on progress towards 
cybersecurity goals; 
* Throughout the year, the goal leader and deputy goal leader would 
also host review meetings with officials from agencies at risk of not 
achieving cybersecurity performance targets, or that exhibited other 
risk factors.  

Goal: Data Center Consolidation; 
Description of review process: 
* Each quarter, staff from OMB would collect data on agency data 
center closures, and then review these data to evaluate each agency's 
progress; 
* Once a year, the Federal Chief Information Officer also held 
PortfolioStat review meetings with officials from each agency. 
According to OMB staff, during these reviews participants reviewed 
metrics from across the agency's information technology portfolio, 
which included, in some cases, those related to data center 
consolidation.  

Goal: Energy Efficiency; 
Description of review process: 
* Each quarter staff supporting the goal leader would collect updated 
information on contributing agency priority goals for the purposes of 
updating the quarterly status update.  

Goal: Entrepreneurship and Small Business; 
Description of review process: 
* Each quarter the deputy goal leader would collect updated 
information on goals and milestones from the leaders of each of 10 sub-
goals for the purposes of developing the quarterly status update; 
* The deputy goal leader would follow-up with sub-goal leaders or 
agency officials, as necessary, to address issues or questions about 
the status of efforts. The goal leader would then review and approve 
the quarterly status update; 
* Some sub-goal leaders would hold in-person meetings with officials 
from contributing agencies to, among other things, review progress on 
identified goals and milestones. See appendix III for information on 
interagency groups that were used to manage efforts for four of the 
sub-goals.  

Goal: Exports; 
Description of review process: 
* Each quarter the goal leader, with the assistance of staff from 
Commerce and the PIC, would collect updated information on relevant 
agency metrics and activities for the purposes of updating the 
quarterly status update; 
* Periodic meetings of the Export Promotion Cabinet/Trade Promotion 
Coordinating Committee, and its Small Business and other working 
groups, were also used to discuss the status of export promotion 
efforts and progress on specific deliverables.  

Goal: Improper Payments; 
Description of review process: 
* Each year OMB would collect and report data on agency improper 
payment rates; 
* Staff from the OMB Office of Federal Financial Management led 
monthly meetings with agency representatives to discuss the 
implementation of the Do Not Pay initiative, which was designed to 
contribute to the reduction of improper payments. The Department of 
Treasury, as the agency leading implementation of the Do Not Pay 
initiative, would track agency progress on implementation milestones.  

Goal: Job Training; 
Description of review process: 
* Each quarter staff from the PIC would collect updated information on 
progress towards agency milestones, and work with the goal leader on 
the development of the quarterly status update.  

Goal: Real Property; 
Description of review process: 
* After this goal was revised in the second quarter of 2013, a new 
review process to track agency adherence to the goal was under 
development by OMB.  

Goal: Sustainability; 
Description of review process: 
* Twice a year the Council on Environmental Quality (CEQ) would 
collect and review quantitative and qualitative data on agency 
progress towards established sustainability goals, including the 
reduction of agency greenhouse gas emissions; 
* Following the collection of these data, the goal leader hosted 
meetings of the Steering Committee on Federal Sustainability, which 
were used to discuss federal sustainability policy and progress on 
sustainability goals; 
* According to CEQ staff, the goal leader and CEQ staff would meet 
with representatives from agencies about sustainability issues on an 
ad hoc basis. In instances where there was a gap between an agency's 
actual performance and the target established in that area, the goal 
leader, or other staff from CEQ, would meet with officials from that 
agency to discuss ways to address the performance gap.  

Goal: Science, Technology, Engineering, and Mathematics (STEM) 
Education; 
Description of review process: 
* Each quarter the goal leader would collect updated information on 
agency milestones for inclusion in the quarterly status updates; 
* Progress on some identified strategies to achieve the goal, such as 
the National Science Foundation's efforts to improve undergraduate 
STEM education, were reviewed at the agency level. After progress was 
reviewed at the agency level the information was passed onto the goal 
leader and reported publicly in the quarterly status update.  

Goal: Strategic Sourcing; 
Description of review process: 
* Each quarter the General Services Administration would collect data 
on agency adoption and spending rates for the Federal Strategic 
Sourcing Initiative (FSSI) solutions for domestic delivery and office 
supplies; 
* The Strategic Sourcing Leadership Council met bi-monthly to guide 
the creation and adoption of new FSSI options, and, as part of that 
effort, might review quarterly data on agency adoption and spending 
rates.  

Goal: Veteran Career Readiness; 
Description of review process: 
* According to the goal leader, each month staff from the Departments 
of Defense and Veterans Affairs, and the PIC, would provide data for 
"one-pagers" and other status update documents with key pieces of 
relevant information, such as the veterans' unemployment rate and the 
number of active employers on the Veteran's Job Bank. These one-pagers 
would be used to inform regular Interagency Policy Council (IPC) 
discussions, along with more specific briefing memorandums, which were 
used to cover the latest issues, keep stakeholders focused on overall 
outcomes, and to inform discussion around specific outliers. Some of 
the data in these one-pagers would also be incorporated into the 
quarterly status updates; 
* More frequently, issue papers and data analysis were provided to 
Veterans Employment Initiative (VEI) Task Force and IPC members as 
needed to address topical issues; 
* Ongoing milestone reviews held by the VEI Task Force and its 
associated working groups on Education, Employment, Transition, and 
Entrepreneurship, provided an opportunity to discuss strategies being 
employed to improve performance.  

Source: GAO analysis of information and documentation collected from 
CAP goal representatives. GAO-14-526.  

[End of table]  

[End of section]  

Appendix V: Full Text for Interactive Figure 2 on Frequency of Data 
Reporting for Cross-Agency Priority Goals' Overall Planned Levels of 
Performance:  

This appendix includes the print version of the text and rollover 
graphics contained in interactive figure 2. 

Cross-Agency Priority Goal: Cybersecurity; 
Overall Planned Level of Performance: ... achieve 95 percent 
implementation of the Administration's priority cybersecurity 
capabilities by the end of fiscal year 2014; 
Frequency of Data Reporting for Overall Goal: Quarterly. 

Data reported for primary performance goal:  

[Refer to PDF for image: table]  

Continuous Monitoring: 
USG-Wide Quarterly Results: 
FY2013 Q3: 82.07%; 
FY2013 Q4: 80.94%; 
FY14 Targets: 
Target FY 2013: 95%.  

Strong Authentication: 
USG-Wide Quarterly Results: 
FY2013 Q3: 66.70%; 
FY2013 Q4: 66.61%; 
FY14 Targets: 
Target FY 2013: 75%.  

TIC Consolidation: 
USG-Wide Quarterly Results: 
FY2013 Q3: 84.17%; 
FY2013 Q4: 86.43%; 
FY14 Targets: 
Target FY 2013: 95%.  

TIC Capabilities: 
USG-Wide Quarterly Results: 
FY2013 Q3: 83.78%; 
FY2013 Q4: 87.48%; 
FY14 Targets: 
Target FY 2013: 100%.  

Weighted Average: 
USG-Wide Quarterly Results: 
FY2013 Q3: 80.14%; 
FY2013 Q4: 80.56%; 
FY14 Targets: 
Target FY 2013: 93%.  

[End of figure]  

Cross-Agency Priority Goal: Data Center Consolidation; 
Overall Planned Level of Performance: ... save $3 billion in taxpayer 
dollars by closing at least 2500 data centers by fiscal year 2015; 
Frequency of Data Reporting for Overall Goal: Quarterly. 

Cross-Agency Priority Goal: Data reported for primary performance goal; 
Overall Planned Level of Performance: "Agencies have already closed 
640 data centers..."  

Cross-Agency Priority Goal: Energy Efficiency; 
Overall Planned Level of Performance: Increase Energy Productivity…50 
percent by 2030; 
Frequency of Data Reporting for Overall Goal: Quarterly. 

Data reported for primary performance goal:  

Figure: U.S. Energy Productivity: Four-Quarter Moving Average:  

[Refer to PDF for image: line graph]  

Real GDP per Million BTU Total Energy Consumption:  

2010, Q4: 
Energy Productivity: $150.7.  

2011, Q1: 
Energy Productivity: $151.2.  

2011, Q2: 
Energy Productivity: $152.0.  

2011, Q3: 
Energy Productivity: $152.8.  

2011, Q4: 
Energy Productivity: $154.5.  

2012, Q1: 
Energy Productivity: $158.1.  

2012, Q2: 
Energy Productivity: $159.8.  

2012, Q3: 
Energy Productivity: $161.7.  

2012, Q4: 
Energy Productivity: $162.8.  

2013, Q1: 
Energy Productivity: $161.8.  

2013, Q2: 
Energy Productivity: $162.2.  

2013, Q3: 
Energy Productivity: $162.9.  

[End of figure]  

Cross-Agency Priority Goal: Exports; 
Overall Planned Level of Performance: Double U.S. exports by the end 
of 2014; 
Frequency of Data Reporting for Overall Goal: Quarterly. 

Data reported for primary performance goal:  

[Refer to PDF for image: table]  

Time frame: CY 2009, total; 
Total U.S. Exports: $1.58 trillion.  

Time frame: CY 2010, total; 
Total U.S. Exports: $1.8 trillion.  

Time frame: CY 2011, total; 
Total U.S. Exports: $2.1 trillion.  

Time frame: CY 2012, total; 
Total U.S. Exports: $2.2 trillion.  

Time frame: CY 2013, Q1; 
Total U.S. Exports: $558 billion.  

Time frame: CY 2013, Q2; 
Total U.S. Exports: $564.7 billion.  

Time frame: CY 2013 Q3; 
Total U.S. Exports: $568.5 billion.  

Time frame: CY 2014, target; 
Total U.S. Exports: $3.16 trillion.  

[End of figure]  

Cross-Agency Priority Goal: Strategic Sourcing; 
Overall Planned Level of Performance: ... agencies' strategic sourcing 
of at least two new commodities or services in both 2013 and 2014, 
that yield at least a 10 percent savings... In addition, agencies must 
increase their use of Federal Strategic Sourcing Initiative vehicles 
by at least 10 percent in both fiscal years 2013 and 2014; 
Frequency of Data Reporting for Overall Goal: Quarterly. 

Data reported for primary performance goal:  

[Refer to PDF for image: table]  

Time frame: FY 2011, total; 
Actual: 309.  

Time frame: FY 2011, target; 
Actual: N/A.  

Time frame: FY 2012, Q1; 
Actual: 76.  

Time frame: FY 2012, Q2; 
Actual: 85.  

Time frame: FY 2012, Q3; 
Actual: 96.  

Time frame: FY 2012, Q4; 
Actual: 131.  

Time frame: FY 2012, total; 
Actual: 388.  

Time frame: FY 2012, target; 
Actual: 340.  

Time frame: FY 2013, Q1; 
Actual: 81.  

Time frame: FY 2013, Q2; 
Actual: 90.  

Time frame: FY 2013, Q3; 
Actual: 91;  

Time frame: FY 2013, Q4; 
Actual: 128.  

Time frame: FY 2013, total; 
Actual: 390.  

Time frame: FY 2013, target; 
Actual: 427.  

[End of figure]  

Cross-Agency Priority Goal: Broadband; 
Overall Planned Level of Performance: ... ensure 4G wireless broadband 
coverage for 98 percent of Americans by 2016; 
Frequency of Data Reporting for Overall Goal: Biannually. 

Data reported for primary performance goal:  

[Refer to PDF for image: table]  

Time frame: CY 2010, as of 12/31/10; 
Access via Wireless Technology: 62%; 
Access via Any Technology: 93%.  

Time frame: CY 2011, as of 12/31/11; 
Access via Wireless Technology: 79%; 
Access via Any Technology: 95%.  

Time frame: CY 2012, as of 6/30/12; 
Access via Wireless Technology: 81%; 
Access via Any Technology: 96%.  

Time frame: CY 2012, as of 12/31/12; 
Access via Wireless Technology: 90%; 
Access via Any Technology: 98%;  

Time frame: CY 2013, as of 6/3013; 
Access via Wireless Technology: [Empty]; 
Access via Any Technology: [Empty].  

Time frame: CY 2013, as of 12/31/13; 
Access via Wireless Technology: [Empty]; 
Access via Any Technology: [Empty].  

[End of figure]  

Cross-Agency Priority Goal: Improper Payments; 
Overall Planned Level of Performance: ... achieve a payment accuracy 
rate of 97 percent by the end of 2016; 
Frequency of Data Reporting for Overall Goal: Annually. 

Data reported for primary performance goal:  

[Refer to PDF for image: table]  

Government-wide Improper Payment Rate (in percent): 
FY 2009: 5.42%; 
FY 2010: 5.29%; 
FY 2011: 4.69%; 
FY 2011: 4.35%.  

[End of figure]  

Cross-Agency Priority Goal: Sustainability; 
Overall Planned Level of Performance: By 2020, the Federal Government 
will reduce its direct greenhouse gas emissions by 28 percent and will 
reduce its indirect greenhouse gas emissions by 13 percent by 2020...; 
Frequency of Data Reporting for Overall Goal: Annually. 

Data reported for primary performance goal:  

"As a whole, the Federal government is on track to meet the 2020 
greenhouse gas (GHG) emissions reduction goal of a 28 percent 
reduction for Scope 1 and 2 emissions. By the end of fiscal year 2012, 
the Federal government reduced these emissions by 15 percent from the 
2008 fiscal year baseline."  

Cross-Agency Priority Goal: Closing Skills Gaps; 
Overall Planned Level of Performance: ... By September 30, 2013, close 
the skills gaps by 50 percent for 3 to 5 critical Federal Government 
occupations or competencies...; 
Frequency of Data Reporting for Overall Goal: Data Not Reported. 

Data reported for primary performance goal: 
"Data Not Reported". 

Cross-Agency Priority Goal: Entrepreneurship and Small Businesses; 
Overall Planned Level of Performance: Increase federal services to 
entrepreneurs and small businesses with an emphasis on 1) startups and 
growing firms and 2) underserved markets; 
Frequency of Data Reporting for Overall Goal: Data Not Reported. 

Data reported for primary performance goal: 
"Data Not Reported". 

Cross-Agency Priority Goal: Job Training; 
Overall Planned Level of Performance: ...[prepare] 2 million workers 
with skills training by 2015...; 
Frequency of Data Reporting for Overall Goal: Data Not Reported. 

Data reported for primary performance goal: 
"Data Not Reported". 

Cross-Agency Priority Goal: Real Property; 
Overall Planned Level of Performance: The Federal Government will 
maintain the fiscal year 2012 square footage baseline of its office 
and warehouse inventory; 
Frequency of Data Reporting for Overall Goal: Data Not Reported. 

Data reported for primary performance goal: 
"Data Not Reported". 

Cross-Agency Priority Goal: Science, Technology, Engineering, and Math 
(STEM) Education; 
Overall Planned Level of Performance: ...increase the number of well-
prepared graduates with STEM degrees by one-third over the next 10 
years, resulting in an additional 1 million graduates with degrees in 
STEM subjects; 
Frequency of Data Reporting for Overall Goal: Data Not Reported. 

Data reported for primary performance goal: 
"Data Not Reported". 

Cross-Agency Priority Goal: Veteran Career Readiness; 
Overall Planned Level of Performance: By September 30, 2013, increase 
the percent of eligible service members who will be served by career 
readiness and preparedness programs from 50 percent to 90 percent...; 
Frequency of Data Reporting for Overall Goal: Data Not Reported. 

Data reported for primary performance goal: 
"Data Not Reported". 

Source: GAO analysis of information from Performance.gov. GAO-14-526.  

[End of table]  

[End of section]  

Appendix VI: GAO Contact and Staff Acknowledgments:  

GAO Contact:  

J. Christopher Mihm, (202) 512-6806 or mihmj@gao.gov:  

Staff Acknowledgments:  

In addition to the contact named above, Elizabeth Curda (Assistant 
Director) and Adam Miles supervised the development of this report. 
Virginia Chanley, Jehan Chase, Steven Putansu, Stacy Ann Spence, and 
Dan Webb made significant contributions to this report. Deirdre Duffy 
and Robert Robinson also made key contributions. 

[End of section]  

Footnotes:  

[1] Pub. L. No. 111-352, 124 Stat. 3866 (Jan. 4, 2011). GPRAMA 
significantly enhanced the Government Performance and Results Act of 
1993, Pub. L. No. 103-62, 107 Stat. 285 (Aug. 3, 1993). 

[2] As our previous work has shown, holding regular meetings between 
leadership and responsible program managers to review progress is an 
effective leadership strategy that has been widely adopted by 
governments at the state and local level to make better-informed 
decisions and drive progress towards established goals. 

[3] 31 U.S.C. § 1121(a). Executive Order No. 13450, Improving 
Government Program Performance, 72 Fed. Reg. 64519 (Nov. 15, 2007) 
established the PIC, whose establishment was later codified under 
GPRAMA. The PIC is chaired by OMB's Deputy Director for Management and 
composed of Performance Improvement Officers from various federal 
agencies. The PIC is to assist the Director of OMB in achieving the 
CAP goals and in implementing the planning, reporting and use of 
performance information requirements related to the CAP goals. 31 
U.S.C. § 1124(b). 

[4] GAO, Managing for Results: Data-Driven Performance Reviews Show 
Promise But Agencies Should Explore How to Involve Other Relevant 
Agencies, [hyperlink, http://www.gao.gov/products/GAO-13-228] 
(Washington, D.C.: Feb. 27, 2013). 

[5] We have previously released a series of reports under our mandate 
in GPRAMA to periodically examine the implementation of the law. For 
example, see GAO, Managing for Results: GAO's Work Related to the 
Interim Crosscutting Priority Goals under the GPRA Modernization Act, 
[hyperlink, http://www.gao.gov/products/GAO-12-620R] (Washington, 
D.C.: May 31, 2012) and Managing for Results: Executive Branch Should 
More Fully Implement the GPRA Modernization Act to Address Pressing 
Governance Challenges, [hyperlink, 
http://www.gao.gov/products/GAO-13-518] (Washington, D.C.: June 26, 
2013). 

[6] A 15TH interim CAP goal--Open Government Data--was added when the 
status of the CAP goals was updated on Performance.gov in September 
2013. Since this goal was not included in the initial list of goals 
identified in February 2012, it was excluded from the scope of our 
review.  

[7] We requested an interview with representatives of the Energy 
Efficiency CAP goal, including the goal leader. However, no 
representatives of this CAP goal were made available for an interview. 
We were, however, informed that this goal was a compilation of related 
agency priority goals. We interviewed the goal leaders for two agency 
priority goals, from the Department of Energy and the Environmental 
Protection Agency, identified as contributing to the Energy Efficiency 
CAP goal.  

[8] Throughout this report, we refer to agency and OMB staff that 
supported efforts related to the CAP goals as "goal managers."  

[9] [hyperlink, http://www.gao.gov/products/GAO-13-228]. 

[10] 31 U.S.C. § 1120(a). GPRAMA specifies that the management 
improvement goals must cover financial management, human capital 
management, information technology management, procurement and 
acquisition management, and real property management.  

[11] Pub. L. No. 111-352, § 14(a)(1), 31 U.S.C. § 1115 note. 

[12] The goal leaders were often supported by deputy goal leaders. 

[13] For information on successful implementation approaches used by 
interagency groups, see GAO, Managing for Results: Implementation 
Approaches Used to Enhance Collaboration in Interagency Groups, 
[hyperlink, http://www.gao.gov/products/GAO-14-220] (Washington, D.C.: 
Feb. 14, 2014). 

[14] 31 U.S.C. § 1121(a)(1). 

[15] 31 U.S.C. §§ 1121(a)(4)-(5). 

[16] [hyperlink, http://www.gao.gov/products/GAO-13-228].  

[17] Throughout the report we refer to the single website required by 
GPRAMA as Performance.gov.  

[18] 31 U.S.C. § 1122(c). 

[19] The other 10 CAP goals are Insider Threat and Security Clearance; 
Job-Creating Investment; Infrastructure Permitting Modernization; 
Service Members and Veterans Mental Health; Customer Service; Smarter 
IT Delivery; Shared Services; Benchmark and Improve Mission-Support 
Operations; Lab-to-Market; and People and Culture.  

[20] The set of updates for the second quarter of fiscal year 2013 did 
not contain updates to the Energy Efficiency and Veteran Career 
Readiness CAP goals. According to OMB, the Energy Efficiency goal was 
not updated during the quarter due to revisions based on recent policy 
developments. The Veteran Career Readiness goal was not updated due to 
ongoing policy discussions relating to the implementation of the 
Department of Defense Transition Assistance Program. 

[21] [hyperlink, http://www.gao.gov/products/GAO-12-620R].  

[22] [hyperlink, http://www.gao.gov/products/GAO-12-620R]. While 
GPRAMA and OMB's guidance define contributing programs in fairly 
general terms, our previous work has identified more specific examples 
of different program types the federal government uses to achieve many 
of its goals: contracts, direct services, grants, regulations, 
research and development, and tax expenditures. See GAO, Managing for 
Results: Executive Branch Should More Fully Implement the GPRA 
Modernization Act to Address Pressing Governance Challenges, 
[hyperlink, http://www.gao.gov/products/GAO-13-518] (Washington, D.C.: 
June 26, 2013). 

[23] GAO, 2012 Annual Report: Opportunities to Reduce Duplication, 
Overlap and Fragmentation, Achieve Savings, and Enhance Revenue, 
[hyperlink, http://www.gao.gov/products/GAO-12-342SP] (Washington, 
D.C.: Feb. 28, 2012). 

[24] GAO, Tax Expenditures: Background and Evaluation Criteria and 
Questions, [hyperlink, http://www.gao.gov/products/GAO-13-167SP] 
(Washington, D.C.: Nov. 29, 2012).  

[25] GAO, Agencies' Annual Performance Plans Under the Results Act: An 
Assessment Guide to Facilitate Congressional Decisionmaking, 
[hyperlink, http://www.gao.gov/products/GAO/GGD/AIMD-10.1.18] 
(Washington, D.C.: Feb. 1, 1998) and Managing for Results: Agencies 
Should More Fully Develop Priority Goals under the GPRA Modernization 
Act, [hyperlink, http://www.gao.gov/products/GAO-13-174] (Washington, 
D.C.: Apr. 19, 2013). 

[26] [hyperlink, http://www.gao.gov/products/GAO-13-174]; [hyperlink, 
http://www.gao.gov/products/GAO-13-228]; and GAO, Managing for 
Results: Critical Issues for Improving Federal Agencies' Strategic 
Plans, [hyperlink, http://www.gao.gov/products/GAO/GGD-97-180] 
(Washington, D.C.: Sept. 16, 1997). 

[27] GAO, Internal Control: Standards for Internal Control in the 
Federal Government, [hyperlink, 
http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: 
Nov. 1, 1999). 

[28] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1].  

[29] OMB Circular No. A-11, Preparation, Submission, and Execution of 
the Budget, pt. 6 (August 2012). 

[30] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. 

[31] OMB Circular No. A-11, Preparation, Submission, and Execution of 
the Budget, pt. 6 (August 2012). 

[32] [hyperlink, http://www.gao.gov/products/GAO-13-228]. 

[33] [hyperlink, http://www.gao.gov/products/GAO-13-228]. 

[34] A target of 75.31 percent for PIV adoption was established for 
the fourth quarter of 2013. 

[35] Pub. L. No. 111-352, § 15(b)(1) , 124 Stat. 3866, 3883 (2011). 

[36] A 15th CAP goal--Open Government Data--was added when the status 
of the CAP goals was updated on Performance.gov in September 2013. 
Since this goal was not included in the initial list of goals 
identified in February 2012 it was excluded from the scope of our 
review.  

[37] We requested interviews with representatives of the Energy 
Efficiency CAP goal, including the goal leader. However, no 
representatives of this CAP goal were made available for an interview. 
We were, however, informed that this goal was a compilation of related 
agency priority goals. We interviewed the goal leaders for two agency 
priority goals, from the Department of Energy and the Environmental 
Protection Agency, identified as contributors to the Energy Efficiency 
CAP goal.  

[38] GPRAMA requires certain agencies, as determined by OMB, to 
develop a limited number of agency priority goals every 2 years. These 
goals are to reflect the highest priorities of each selected agency, 
as identified by the head of the agency, 31 U.S.C. § 1120.  

[39] At the time OMB provided these memorandums for our review, the 
memorandum for the fourth quarter of fiscal year 2013 was not yet 
available. 

[End of section]  

GAO's Mission:  

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO's commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability.  

Obtaining Copies of GAO Reports and Testimony:  

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
"E-mail Updates."  

Order by Phone:  

The price of each GAO publication reflects GAO's actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO's 
website, [hyperlink, http://www.gao.gov/ordering.htm].  

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537.  

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information.  

Connect with GAO:  

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov].  

To Report Fraud, Waste, and Abuse in Federal Programs:  

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470.  

Congressional Relations:  

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548.  

Public Affairs:  

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548.  

[End of document]