This is the accessible text file for GAO report number GAO-11-438T entitled 'Department Of State: Additional Steps Are Needed to Improve Strategic Planning and Evaluation of Training for State Personnel' which was released on March 8, 2011. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Testimony: Before the Subcommittee on Oversight of Government Management, the Federal Workforce, and the District of Columbia, Committee on Homeland Security and Governmental Affairs, U.S. Senate: For Release on Delivery: Expected at 10:00 a.m. EST: Tuesday, March 8, 2011: Department Of State: Additional Steps Are Needed to Improve Strategic Planning and Evaluation of Training for State Personnel: Statement of Jess T. Ford, Director: International Affairs and Trade: GAO-11-438T: Mr. Chairman and Members of the Subcommittee: I am pleased to be here today to discuss the U.S. Department of State's (State) efforts to train its personnel. My testimony is based on our report, which is being released today.[Footnote 1] Because State is the lead U.S. foreign affairs agency, its personnel require certain knowledge, skills, and abilities to equip them to address the global security threats and challenges facing the United States-- including the threat of Al-Qaeda and other terrorist organizations, HIV/AIDS and other pandemics, environmental degradation, nuclear proliferation, and failed states. In fiscal years 2006 through 2010, State's funding for training personnel grew by about 62 percent, and the department requested more than $266 million in fiscal year 2011 for programs providing training in professional skills such as foreign language proficiency, area studies, information technology, consular duties, and others needed for the conduct of foreign relations. [Footnote 2] State's Foreign Service Institute (FSI) is the primary training provider for the department's more than 66,000 Foreign Service, civil service, and locally employed staff worldwide.[Footnote 3] Our prior work has identified staffing and foreign language shortfalls at State, including challenges the department has faced in filling positions at the mid-level in particular, and in attracting qualified personnel for some hardship posts.[Footnote 4] The department is currently in the midst of what it has called the most challenging military-to-civilian transition in U.S. history in Iraq, one of the posts of greatest hardship where State personnel serve. Recent departmental initiatives--in particular, "Diplomacy 3.0," a multiyear effort launched in March 2009 with a primary aim of increasing the size of State's Foreign Service by 25 percent and the civil service by 13 percent--have underscored the importance of training to equip personnel to fulfill State's leadership role in world affairs and to advance and defend U.S. interests abroad. Today I will discuss State's purpose and structure for training personnel, including leadership, management, professional, and area studies training, contributing to diplomatic readiness of State's Foreign Service and civil service personnel and locally employed staff overseas. I will also discuss the extent to which State's personnel training incorporates elements of effective federal training programs. [Footnote 5] Over the course of our work on this issue, we reviewed and analyzed data and documentation related to State's training efforts, such as strategic and workforce planning documents, funding data, and data on personnel participation in training, as well as legislative, regulatory, and State policy and procedural criteria relevant to training. In addition, we reviewed training evaluation mechanisms used by FSI. We interviewed key officials from 26 State bureaus and offices in Washington, D.C., including FSI, the Bureau of Human Resources, and the six geographic bureaus. We also conducted semistructured telephone interviews with State officials with training-related responsibilities at 12 overseas missions, and interviewed officials from State's regional training centers located in Bangkok, Thailand; Ft. Lauderdale, Florida; and Frankfurt, Germany. With input from State, we completed a training assessment to determine the extent to which the department's personnel training incorporates elements of effective training programs. We used the results of this assessment to identify any gaps in State's training based on criteria identified in GAO, the Office of Personnel Management (OPM), State, and other legislative and regulatory guidance and policy. In addition, in light of work that we recently published on shortfalls in State personnel's foreign language skills[Footnote 6], we did not focus on language training. We also did not include within our scope an assessment of "hard skills" (e.g., security and law enforcement) training provided by State's Bureau of Diplomatic Security. We conducted this performance audit from July 2009 to January 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. More information on our scope and methodology and detailed findings is available in the full report.[Footnote 7] In brief, Mr. Chairman, we found that State has taken many steps to incorporate the interrelated elements of an effective training program--planning, design, implementation, and evaluation--into its training for personnel,[Footnote 8] but the department's strategic approach to workforce training could be improved in several key areas. Specifically, we identified five areas where State can improve its training. First, State lacks a comprehensive training needs assessment process incorporating all bureaus and posts. Second, State developed guidance for employees about training opportunities, career paths, and how training can help employees attain career goals, but the guidance does not provide complete and accurate information. Third, State lacks a data collection and analysis plan for evaluating training, and thus cannot be assured that proper practices and procedures are systematically and comprehensively applied. Fourth, State could not sufficiently demonstrate consistent and appropriate support for training, because the department does not track detailed information on training cost and delivery that would allow for an analysis and comparison of employees in different groups, bureaus, regions, or posts. Lastly, State performance measures for training generally do not fully address training goals, and are generally output-rather than outcome-oriented. State's Annual Training Plan states that "the purpose of the department's training program is to develop the men and women our nation requires to fulfill our leadership role in world affairs and to advance and defend U.S. interests." FSI is State's primary training provider, offering entry-, mid-, and senior-level training for employees as they progress through their careers. State guidance outlines key training roles, including FSI's primary role in developing training policies and facilitating necessary training, and the Bureau of Human Resource's role in assigning employees to training and working with FSI to help ensure it meets their needs. Other bureaus, offices, and posts also share responsibilities for training. FSI offers over 700 classroom courses, and has recently increased its focus on distance learning. We found that overall, about 40 percent of personnel training over the last 5 fiscal years, on average, was in foreign language skills. Other training for personnel generally focused on developing leadership, management, and other professional and technical skills and knowledge. State's personnel training reflects numerous aspects of effective training programs, based on our assessment using the criteria GAO previously identified. For example, State maintains a workforce training plan, as required by federal regulations.[Footnote 9] FSI leads efforts to prepare the plan annually; the plan is linked to State's overall strategic plan, and presents a business case for proposed training investments. FSI also publishes an annual schedule of courses, which provides information for employees on FSI classroom and distance learning course offerings. The schedule of courses generally includes information for each course such as a brief description, any prerequisites, course objectives, and relevant competencies. As another example of a positive practice, State has made an effort to use advances in technologies to enhance training efforts. The number of distance learning offerings, as well as employee participation in distance learning, has increased in recent years. For example, State's latest annual training plan reported that FSI developed 20 new custom distance learning courses during the prior year, and data showed time spent by personnel completing distance learning courses more than doubled from fiscal years 2006 through 2009--from about 113,000 hours in fiscal year 2006 to about 254,000 in fiscal year 2009. In addition, we found that State has a range of training evaluation mechanisms in place, including mid-and post- training course evaluations. Since 2006, FSI has conducted an annual training survey. FSI reported most respondents to the 2010 survey were, in general, satisfied or very satisfied with training.[Footnote 10] However, although State has developed an extensive program for training personnel, our analysis found several gaps in the department's efforts to strategically plan and prioritize training, ensure efficient and effective training design and delivery, and determine whether or how training and development efforts contribute to improved performance and desired results. Each of the issue areas we identified broadly relates to multiple elements, attributes, and indicators throughout the interrelated training and development process. While an agency's training program is not necessarily expected to address every indicator identified in the GAO guidance, based on our assessment, we identified strategic weaknesses related to these areas as particularly important to ensuring effective planning, design, implementation, and evaluation of personnel training. For example, we found that: * State lacks a systematic, comprehensive training needs assessment process incorporating all bureaus and overseas posts. Since 2007, State human resource reports noted that bureaus have not formally conducted annual training needs assessments, and identified this as an issue that should be addressed to help provide a realistic basis for planning, budgeting, and directing training. According to the reports, the Bureau of Human Resources intended to form an interoffice working group to develop a comprehensive plan and implementation guidance to support a department-wide effort for assessing training needs. However, State had not yet formed an interoffice working group as of November 2010. * State developed guidance--known as training continuums--to provide information for employees about training opportunities, career ladders and paths, and how training can help employees attain career goals, but the guidance documents do not provide complete and accurate information for employees. While the documents state that they were designed to provide a broad overview of appropriate training that should be considered as employees plan their careers in the department, including information on mandatory, recommended, and suggested courses, we found issues that raised questions about their usefulness and reliability as employee resources. For example, we found that specific training requirements designated by bureaus and posts for certain groups of employees are not always identified in the training guidance. A key official from FSI's executive office acknowledged that the guidance documents do not include complete and accurate information for employees on training, and noted that the documents have not been reviewed to ensure they uniformly reflect departmental policies or standards. * State has not developed a data collection and analysis plan for evaluating training, which could help ensure that appropriate procedures and criteria for evaluating training are systematically applied across the board. As a result, it is not clear whether or how State systematically makes decisions regarding how training programs will be evaluated using different methods or tools, or how results will be used. Our prior work highlights the importance of planning and conducting evaluations of the effectiveness of training and development efforts and notes that a data collection and analysis plan can set priorities for evaluations and systematically cover the methods, timing, and responsibilities for an agency's data collection. [Footnote 11] While State has implemented mechanisms to evaluate training, including course evaluations and an annual training survey, these mechanisms do not fully incorporate locally employed staff, and primarily focus on participant satisfaction or reaction to training, rather than desired results, such as improved quality or efficiency of work. * State could not sufficiently demonstrate consistent and appropriate support for training, because it does not track detailed data and information on training cost and delivery that would allow for an analysis and comparison of employees in different employee groups, bureaus, regions, and posts. For example, State could not provide data on the percentage of foreign affairs or political officers that had completed required, recommended, or suggested training for their areas of work. Although State tracks some data related to training funding and delivery, the department does not have sufficient information that could be used to ensure consistent and appropriate support for training, or to help determine whether managers and employees have needed training tools and resources. This is especially important given evidence of variances in training across the department. For example, while training officials we interviewed at some bureaus and posts indicated they had sufficient funding and support for training, others noted they faced significant resource challenges that impacted the ability of employees to get necessary training. * State has developed several training-related goals and measures, but the measures do not fully address the goals, and are generally output rather than outcome oriented. As a result, they do not provide a clear means of determining whether State's training efforts achieve desired results. For example, one training goal listed in FSI's fiscal year 2012 strategic resource plan, "workforce meets priority diplomatic and operational requirements as a result of FSI training," includes priorities and objectives to expand and enhance language training, support training in stability operations, support new hire training, and enhance public diplomacy training. However, the goal's two measures, "language training success rate at FSI," and "development of training continuum to support State's Office of Reconstruction and Stabilization," are both output measures and do not fully address the priority areas for the goal, such as support for new hire training or public diplomacy training. State's budget and focus on training have increased in recent years, but the department has also faced, and will likely continue to face, fluctuating and constrained resources and competing priorities when determining what training is critical to its mission. Without concerted efforts to further incorporate effective practices, State cannot ensure training resources are targeted strategically, are not wasted, and achieve cost-effective and timely results desired, and thus cannot be assured that its employees are trained and equipped to meet the challenges of their mission. Our report being issued today includes several recommendations for the Secretary of State to improve strategic planning and evaluation of the department's efforts to train personnel, including for improvements to State's efforts to assess training needs and efforts to ensure training achieves desired results. State generally agreed with our findings and recommendations. Mr. Chairman, this concludes my prepared statement. I would be pleased to respond to any questions you or other Members of the Subcommittee may have at this time. GAO Contact and Staff Acknowledgments: For questions regarding this testimony, please contact Jess T. Ford, (202)512-4268 or fordj@gao.gov. In addition to the contact named above, Anthony Moran, Assistant Director; Lisa Helmer; Shirley Min; Joe Carney; Virginia Chanley; Kieran Cusack; David Dayton; Patrick Lockett; Reid Lowe; and Mary Moutsos provided significant contributions to the work. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this testimony. [End of section] Footnotes: [1] GAO, State Department: Additional Steps Are Needed to Improve Strategic Planning and Evaluation of Training for State Personnel, [hyperlink, http://www.gao.gov/products/GAO-11-241] (Washington, D.C.: January 2011). [2] According to State, the total number of Foreign Service, civil service, and locally employed personnel increased from about 57,000 in September 2006 to more than 66,000 as of September 2010, an increase of about 17 percent. [3] State's locally employed staff include foreign nationals and U.S. citizen residents employed via direct-hire appointments, personal services agreements, or personal services contracts. [4] See GAO, State Department: Persistent Staffing and Foreign Language Gaps Compromise Diplomatic Readiness, [hyperlink, http://www.gao.gov/products/GAO-09-1046T] (Washington, D.C.: September 24, 2009); and State Department: Staffing and Foreign Language Shortfalls Persist Despite Initiatives to Address Gaps, [hyperlink, http://www.gao.gov/products/GAO-07-1154T] (Washington, D.C.: August 1, 2007). [5] We previously developed guidance for assessing federal strategic training and development efforts, including identifying four essential and interrelated elements of the training and development process: (1) planning, (2) design, (3) implementation, and (4) evaluation. The guidance includes key attributes of effective federal training programs to consider when assessing each of the four elements, along with indicators related to each attribute. This guidance can be used to identify potential gaps or areas where improvements may be made to help ensure that training and development investments are targeted strategically and not wasted on efforts that are irrelevant, duplicative, or ineffective. GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO-04-546G] (Washington, D.C.: Mar. 1, 2004). [6] GAO, Department of State: Comprehensive Plan Needed to Address Persistent Foreign Language Shortfalls, [hyperlink, http://www.gao.gov/products/GAO-09-955] (Washington, D.C.: Sept. 17, 2009). Our latest report notes that according to State, the department has taken several steps to address prior GAO recommendations related to language training needs and challenges, such as developing an analytical model to better assess resource needs, including training, to meet language requirements, and implementing mechanisms to ensure a strategic approach to addressing foreign language needs. [7] [hyperlink, http://www.gao.gov/products/GAO-11-241]. [8] [hyperlink, http://www.gao.gov/products/GAO-04-546G]. [9] 5 C.F.R. § 410.201. [10] According to State, the 2010 annual training survey was sent to a random sample of 5,105 Foreign Service and civil service employees, as well as eligible family members. Among other things, the survey asked respondents to rate FSI's training delivery methods, training programs, and customer service. We determined that the results of this survey were sufficiently reliable to provide a general indication of employee satisfaction with training. [11] Such a plan could also include guidelines to help ensure the agency makes an ongoing effort to improve the quality and breadth of data gathered. Our prior work also noted that developing and using such a plan can guide an agency in a systematic approach to assessing the effectiveness and efficiency of both specific training and development programs and more comprehensively assessing its entire training and development effort. [hyperlink, http://www.gao.gov/products/GAO-04-546G]. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: