Department of State: Additional Steps Are Needed to Improve Strategic Planning and Evaluation of Training for State Personnel
Highlights
Because the U.S. Department of State (State) is the lead U.S. foreign affairs agency, its personnel require certain knowledge, skills, and abilities to address the global challenges and security threats facing the United States. State devoted about $255 million to personnel training in fiscal year 2010; the department's Foreign Service Institute (FSI) is the primary training provider for State's more than 66,000 Foreign Service, civil service, and locally employed staff (LE staff) worldwide. GAO was asked to examine (1) State's purpose and structure for training personnel and (2) the extent to which State's training incorporates elements for effective training programs. GAO reviewed and analyzed data and documentation related to the agency's training efforts; completed a training assessment using a tool developed based on prior GAO guidance; and interviewed officials in Washington, D.C., and at 12 overseas posts.
State's purpose for training personnel is to develop the men and women the United States requires to fulfill its leadership role in world affairs and to advance and defend U.S. interests. State guidance outlines key training roles, including FSI's primary role in developing training policies and facilitating necessary training, and the Bureau of Human Resources' role in assigning employees to training and working with FSI to help ensure it meets their needs. Other bureaus, offices, and posts also share responsibilities for training. FSI currently offers more than 700 classroom courses, and has recently increased its focus on distance learning. Overall, about 40 percent of personnel training over the last 5 fiscal years, on average, was in foreign language skills. Other training for personnel generally focused on developing leadership, management, and other professional and technical skills and knowledge. State has taken many steps to incorporate the interrelated elements of an effective training program--planning, design, implementation, and evaluation--into its extensive training for personnel; however, the department's strategic approach to workforce training has several key weaknesses. The department demonstrated a variety of ways in which it has endeavored to develop an effective training program, such as by compiling an annual training plan, and implementing a range of training evaluation mechanisms and a learning management system that can be used to track training delivery. However, GAO's analysis found several gaps in the department's efforts to strategically plan and prioritize training, ensure efficient and effective training design and delivery, and determine whether or how training and development efforts contribute to improved performance and desired results. For example: (1) State lacks a systematic, comprehensive training needs assessment process incorporating all bureaus and overseas posts. (2) State developed training continuums to provide information for employees about training opportunities, career paths, and how training can help employees attain career goals, but the continuums do not provide complete and accurate information, and other guidance does not cover all personnel. (3) State lacks formal guidance for curriculum design and for data collection and analysis, and thus cannot be assured that proper practices and procedures are systematically and comprehensively applied. (4) State could not sufficiently demonstrate consistent and appropriate support for training, because the department does not track detailed information on training cost and delivery that would allow for an analysis and comparison of employees in different groups, bureaus, regions, or posts. (5) State's performance measures for training generally do not fully address training goals, and are generally output- rather than outcome-oriented. GAO is making several recommendations for State to improve strategic planning and evaluation of the department's efforts to train personnel, including for improvements to State's efforts to assess training needs and efforts to ensure training achieves desired results. State reviewed a draft of this report and generally agreed with our recommendations.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of State | To help ensure that State's personnel training is connected to improving individual and agency performance and that department resources are directed to actual training needs and priorities, the Secretary of State should direct FSI and the Bureau of Human Resources, in collaboration with other bureaus and offices, as appropriate, to develop and implement a plan for a systematic, comprehensive training needs assessment process, incorporating all bureaus and posts. |
State agreed that assessing training needs was a critical human resources function with this recommendation and in 2011, State implemented its first annual post and bureau training needs assessment. The survey solicits the top five training priorities from each bureau and post. The Office of the Executive Director at FSI reviews the survey results and provides them to the various departments, who in turn develop an action plan in response to the top five training needs identified. The finalized action plan is shared with the Director's office.
|
| Department of State | To enhance State's efforts to provide transparent, complete, and accurate information to help employees plan training and development throughout their careers, the Secretary of State should direct FSI and other bureaus and offices, as appropriate, to collaborate in developing and updating information for employees on training to ensure that employees have complete and accurate guidance, including information on any mandatory, required, and recommended training for specific employee groups. |
As of August 2014, State has developed a number of "toolkits" for specific groups of Foreign Service and civil service employees, to assist the employees and their supervisors in determining appropriate training at different points in their careers. In addition, in February 2012 State developed a one-page Leadership Training Roadmap as a resource for Foreign Service and Civil Service employees that outlines mandatory leadership and supervisory training and examples of elective and external training for all career levels. The training guidance is updated periodically and available to employees on FSI's website. In January 2014, the agency fully implemented this recommendation by issuing a training toolkit for all locally employed staff. The toolkit provides an overview of learning opportunities for entry-, mid-, and senior-level LE staff. The toolkit lists courses by discipline and indicates whether they are mandatory, recommended, or elective, as well as provide information on additional training resources, including training websites, a recommended reading list, and information about various training providers.
|
| Department of State | To help ensure that State's performance measures for training provide meaningful data and more fully address the department's training goals, the Secretary of State should direct FSI and the Bureau of Human Resources to review the performance measures and revise or enhance the measures, as appropriate. |
In December 2012, State's Foreign Service Institute (FSI) updated and revised the objectives, descriptions and measures for the department's personnel training goals in FSI's fiscal year 2015-2017 bureau strategy. For example, for the strategy's goal: "workforce meets priority diplomatic and operational requirements as a result of FSI training," FSI added a new objective: "FSI curricula reflect the requirements of administration priorities such as [the Quadrennial Diplomacy and Development Review (QDDR)], and employees are prepared to anticipate and handle emerging political, social, economic trends in an increasing changing world." The strategy includes two new measures of success for this objective: the creation of new/updated courses that reflect QDDR priorities; and the assessment of the impact of training through training evaluations. FSI plans to use evaluations to assess the impact of its training products at various points and judge if and to what degree skills and knowledge are being transferred to employees.
|
| Department of State | To enhance State's capacity to evaluate workforce training, the Secretary of State should direct FSI and the Bureau of Human Resources to develop a data collection and analysis plan for training, including guidance for determining the methods, timing, and responsibilities for training data collection, as well as how results will be used. |
State agreed with the findings in the GAO report and has implemented this recommendation. Specifically, in May 2014 FSI created a Training Advisory Group with representatives from each of FSI's schools and centers, which developed and finalized policies and standards for the collection and analysis of data to evaluate personnel training as of March 2015. The purpose of the policies and standards is to help ensure that FSI structures and evaluates training programs to ensure quality, relevance, effectiveness, and impact so that the foreign affairs community has the knowledge and skills needed to achieve U.S. foreign policy objectives. For example, related to training evaluation, the policies and standards state that each FSI school/center will assess the relevance, impact, and value of training through conducting evaluations at multiple levels, from participant reaction to results. The policies and standards also state that each FSI school/center will systematically collect, analyze, and report evaluation data in order to support evidence-based decision making. In addition, among other things, the standards state that training evaluation reports will contain sufficient data and analysis of key metrics and indicators and how they relate to performance goals and learning objectives to substantiate conclusions and recommendations; that each school/center will incorporate core questions/protocols into evaluation processes to facilitate analysis of results of training programs; and that FSI will create and maintain a central repository for evaluation survey data for the purposes of analysis for institute-wide decision making.
|
| Department of State | To improve State's ability to determine whether it is providing consistent and appropriate support and funding for employee training and development across employee groups and locations, the Secretary of State should direct FSI and the Bureau of Human Resources to identify ways to improve the collection and analysis of training data and results, such as by enhancing the level of detailed information gathered to determine whether employees across groups and locations are getting needed training, and enhancing efforts to determine the impact of training. These efforts should also include steps to further incorporate locally employed (LE) staff into State's training evaluation mechanisms. |
State is taking steps to make staff aware of data already gathered. For example, the Human Resource Bureau has developed "dashboards" to share information with supervisors on who has completed mandatory leadership training. Foreign Service Institute (FSI) also sends training reports to embassies twice a year, but these reports do not cover all forms of training. However, it is not clear that this reflects improvements in the collection and analysis of training data. As of June 2015, State had not taken further action to address this recommendation.
|