Human Capital: Agencies Should More Fully Evaluate the Costs and Benefits of Executive Training
Highlights
What GAO Found
Chief Human Capital Officers (CHCO) from 26 agencies reported that, from fiscal years 2008 through 2012, they spent almost $57 million (in constant 2012 dollars) on executive training provided by external providers. CHCOs reported using the Office of Personnel Management (OPM) Federal Executive Institute and the Harvard Kennedy School of Government most often. Agencies are required to maintain records of training plans, expenditures, activities, and (since 2006) to report training data to OPM. However, half of the CHCOs reported data to GAO that they deemed incomplete, or with limitations. For example, two agencies did not include travel related costs; one did not include costs for course materials; another did not include costs from all components. OPM officials agree that training cost data reported by agencies continues to be unreliable, and is probably lower than actual agency expenditures. OPM officials said they are meeting with agencies to address data deficiencies. However, OPM has not set interim milestones for meeting with agencies or established a timeframe to improve reporting. One leading practice is to establish such interim milestones and timeframes, in order to demonstrate progress towards achieving goals. By not establishing interim milestones and timeframes for improving the reliability of executive training cost data, OPM may be missing an opportunity to better position itself to hold agencies accountable for improving their data.
Most CHCOs reported evaluating participant reaction and changing their training programs based on participant input, but fewer reported assessing training impact on agency mission. Agencies are required by statute and OPM implementing regulations to evaluate how well training programs contribute to mission accomplishment and performance goals. OPM is not sharing lessons learned from agencies that have experience assessing executive training impact on agency mission. However, OPM acknowledged that some agencies struggle with these evaluations; in response, OPM has issued guidance on the subject. CHCOs cited time, costs, and difficulty as reasons for not conducting the required evaluations and reported the need for additional OPM assistance. Eight agencies reported conducting these evaluations and may have lessons learned from which other agencies could benefit. For example, VA has a process for assessing the impact of executive training on its mission that it has used to make better training investment decisions.
CHCOs identified opportunities for agencies and OPM to achieve efficiencies in executive training. CHCOs said agencies could (1) increase interagency cooperation by sharing training facilities and expanding eligibility to Senior Executive Service (SES) candidates from other agencies, and (2) implement or expand computer-based training. CHCOs also said OPM could, among other things, (1) help centralize training offerings by creating a centrally funded SES candidate development program, and (2) assist agencies in identifying programs open to other agencies and departments. By not assessing lessons learned and other efficiencies identified by agencies, OPM may be missing an opportunity to better position agencies to achieve these efficiencies in executive training.
Career SES and SES candidates generally said external executive training is useful and valuable, but suggested that it would be more cost-effective for the government to negotiate prices as a large-scale buyer, versus individual agencies purchasing training.
Why GAO Did This Study
The federal government annually spends millions of dollars on executive training programs administered by external training providers. GAO was asked to review the costs and value of this training. This review (1) describes what is known about how much CHCO Council agencies have spent on executive training offered by external providers and assesses the reliability of available cost data; (2) assesses how CHCO Council agencies evaluate the effectiveness of the training; (3) compares executive training efficiency opportunities identified by agency CHCOs to leading practices; and (4) provides views of career SES and SES candidates from selected agencies on the value of the training they received. GAO obtained information from CHCO Council agencies through a questionnaire, and selected three of 26 CHCO Council agencies--the Departments of Energy, Treasury, and Veterans Affairs--to provide illustrative examples. GAO based its selection, in part, on workforce size, number of career SES, and total training costs.
Recommendations
GAO recommends that OPM (1) establish interim milestones for meeting with agencies to address training data deficiencies and establish time frames for improving the reliability of agency data, (2) share information and examples of how agencies have evaluated the impact of executive training on agency mission and goals, and (3) assess potential efficiencies identified by agencies for possible government-wide implementation. OPM concurred with the recommendations and has taken useful steps. GAO still believes that more effective activities can be taken.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of Personnel Management |
Priority Rec.
To help ensure that agencies track and report comparable and reliable cost data and perform evaluations that assess the impact of executive training on agency performance or missions, the Director of OPM, in coordination with the CHCO Council, should establish interim milestones for meeting with agencies in order to address training data deficiencies and to establish well-defined timeframes for improving the reliability of the data in its Enterprise Human Resources Integration database. |
In May 2016, OPM reported taking the following actions in response to our recommendation. OPM established timeframes for improving the reliability of agency training data and is meeting with agencies to address training data deficiencies. In 2013, OPM established the Federal Training Investments Work Group, a group of Chief Learning Officers, to help OPM define data elements that would be captured within agency automated learning management systems. Subsequently, in 2015, OPM made a presentation to the Chief Learning Officer's Council on EHRI training data and related data elements agencies would report to OPM. OPM reported that the agency is regularly providing summaries of fiscal year training data in EHRI to agencies and meeting with agencies to address training data deficiencies. During January 2015 and 2016, OPM provided agencies with summary training data from the previous fiscal year to validate the accuracy of their training reports. OPM reported providing technical assistance to five CHCO agencies that did not submit training data for FY 2015. Hence, by establishing interim milestones to address training data deficiencies and timeframes for improving training data, OPM has positioned itself to hold agencies accountable for improving their executive training cost data.
|
| Office of Personnel Management | To help ensure that agencies track and report comparable and reliable cost data and perform evaluations that assess the impact of executive training on agency performance or missions, the Director of OPM, in coordination with the CHCO Council, should improve assistance to agencies regarding evaluating the impact of executive training on mission and goals, for example by sharing information and examples of how agencies could better conduct such evaluations. |
In June 2016, Office of Personnel Management (OPM) reported taking the following actions in response to our recommendations. In 2014, OPM conducted a survey of agencies to assess the impact of Senior Executive Service Onboarding activities and presented results on measuring the impact of onboarding and training activities at a roundtable of training program managers. OPM developed job aids, templates and related materials for evaluating leadership development programs and shared evaluation materials via its Training and Development Policy Wiki and training listserv. OPM conducted several training workshops in 2014 and 2015 on evaluating the impact of executive training that included strategies on making the case for the training's impact on the agency mission. Also, in September 2015, OPM hosted a Chief Human Capital Officers Council Training Academy session on training program evaluations that included strategies and tools to evaluate the impact of leadership development programs on agency mission and organizational performance goals. Hence, by providing training and additional resources to agencies, OPM is assisting agencies in better evaluating the impact executive training on the agency's performance.
|
| Office of Personnel Management | To enhance the efficiency of executive training, the Director of OPM, in coordination with the CHCO Council, should assess potential efficiencies identified by agencies for possible government-wide implementation, and then take the steps necessary to implement these, such as updating the guidance governing executive training programs. |
In 2016, the Office of Personnel Management (OPM) reported taking the following actions in response to our recommendations. OPM worked with the Federal Chief Learning Officers Council in 2015 to issue new guidance to help federal agencies develop current and aspiring federal supervisors and managers. The guidance provided agencies with direction on developing individuals in executive positions and included recommended training on leadership competencies important for managerial success. OPM developed and distributed job aids and guidance materials that included recommended executive training for the onboarding process. OPM issued a report on effective ways to develop leadership competencies, based on a review of research by federal agencies known for innovative and effective practices in developing executive competencies. Additionally, OPM collaborated with the Chief Human Capital Officers (CHCO) Council to share information with federal agencies on low-cost and free leadership development courses listed on OPM's Managers Corner in the HR University website. Hence, by working with CHCOs to identify efficiencies for leadership development training OPM has better positioned itself to help agencies achieve efficiencies in executive training.
|