Defense Acquisition Workforce:
DOD's Training Program Demonstrates Many Attributes of Effectiveness, but Improvement Is Needed
GAO-11-22: Published: Oct 28, 2010. Publicly Released: Oct 28, 2010.
The President has announced his intention to improve the acquisition process, particularly given the half a trillion dollars the federal government spent in fiscal year 2009 on acquiring goods and services. The Department of Defense (DOD) spent $384 billion in fiscal year 2009 on goods and services--double what it spent in 2001. A high-quality workforce with the right competencies and skill sets will be critical to improving DOD acquisitions. GAO was mandated to determine the efficacy of DOD's certification training for its acquisition workforce. GAO assessed (1) DOD's capability to provide certification training, (2) the extent that such training reaches members of the workforce, and (3) the extent that previous training recommendations have been implemented. To conduct this work, GAO compared DOD's certification training to GAO guidance for effective training programs and analyzed policies, data, and previous reports on acquisition training.
DOD's certification training program--provided by the Defense Acquisition University (DAU)--generally demonstrates the capability to provide effective training, though some attributes of an effective training program are lacking. DAU ensures that strategic and tactical changes are promptly incorporated into training; uses centralized and decentralized training approaches in design and development; collects data during implementation to ensure feedback on its training programs; and analyzes its training during evaluation. However, DOD lacks complete information on the skill sets of the current acquisition workforce and does not have outcome-based metrics to assess results achieved in enhancing workforce proficiency and capability through training efforts. In 2009, GAO recommended that the Secretary of Defense identify and update on an ongoing basis the number and skill sets of the total acquisition workforce--including civilian, military, and contractor personnel--that the department needs to fulfill its mission. DOD agreed and to date has completed about one-fifth of its workforce competency assessments. At the end of fiscal year 2009, 90 percent of DOD's acquisition workforce personnel had completed required certification training or were within required time frames to do so, according to DAU data. However, DAU reports that it cannot provide for all training requested for the entire acquisition workforce. DAU has offered more courses in recent years, and high-priority personnel--those needing to complete classes for certification in their current position--constitute the majority in DAU classes. DAU plans the number and location of its classes based on data that DOD officials noted are generally incomplete when submitted, and DAU must adapt during the year to support new requirements as they are identified. DAU has identified the need for a new, integrated student information system that will provide better insight into the workforce it supports and is in the early stages of its procurement. DOD reports that most of the training-related recommendations from previous reviews--the Gansler Commission, the Panel on Contracting Integrity, and a prior GAO report--have been fully implemented and some actions are still under way. DOD has either fully or partially implemented 15 of the 19 recommendations GAO reviewed. Both the Army and the Office of the Secretary of Defense have taken steps to respond to the Gansler Commission recommendations. Most of the recommendations made by the Panel on Contracting Integrity have been implemented, with the exception of two recommendations related to assessing guidance and reviewing a specific training topic. GAO made four recommendations pertaining to the Defense Contract Audit Agency's government auditing standards training and expertise, of which one has been partially implemented and three have not been implemented, but some actions have been taken. GAO recommends DOD establish milestones for developing metrics to measure how certification training improves acquisition workforce capability and a time frame for acquiring and implementing an integrated information system. DOD concurred with the second but not the first recommendation. GAO continues to believe DOD needs to develop additional metrics.
Recommendations for Executive Action
Status: Closed - Implemented
Comments: Although DOD did not concur with this recommendation, DOD officials agree that metrics are needed and believe the Department continues to focus its training to help improve organizational performance. According to a DOD official, in 2012 the Defense Acquisition University (DAU) completed its first efforts in conducting Kirkpatrick Level 4 evaluations of training, intended to determine the impact of training on organizational performance. Additional analysis was completed in FY13 which collected and analyzed survey results from more than 326,000 course evaluations to better understand how course evaluation data can be used to study the relationship between DAU training and business results. The FY13-15 DAU strategic plan contains milestones for the expanded use of Kirkpatrick Level 4 evaluations and the implementation of a learning analytics program to document the impact of DAU learning assets on workforce and acquisition outcomes.
Recommendation: To improve the development, implementation, and evaluation of acquisition workforce training, and in order to demonstrate and track how training efforts contribute to improved acquisition workforce performance, the Secretary of Defense should direct the Office of the Under Secretary of Defense for Acquisition (USD(AT&L)) to establish milestones for the development of metrics to measure how acquisition certification training improves the proficiency and capability of the acquisition workforce.
Agency Affected: Department of Defense
Status: Closed - Implemented
Comments: DOD concurred with this recommendation and has implemented it. In January 2011, GSA awarded a contract to the Campus Management Corporation for delivery of an enterprise-wide, commercial off-the-shelf student information system for the Defense Acquisition University (DAU). DAU has developed a project life-cycle plan with design, configuration, user testing, and training occurring during 2011 and 2012. DAU plans for the system to go live in July 2012. Defense Acquisition Workforce Development Fund tax money will be used to support contract costs associated with development and implementation of the student information system.
Recommendation: To improve the development, implementation, and evaluation of acquisition workforce training, and in order to improve DOD's ability to identify specific acquisition training needs for planning and front-end analysis, the Secretary of Defense should direct the Office of the Under Secretary of Defense for Acquisition (USD(AT&L)) to establish a time frame for completion and ensure resources are available for implementing an enterprisewide, integrated student information system.
Agency Affected: Department of Defense