This is the accessible text file for GAO report number GAO-13-524 entitled 'Information Technology: Additional Executive Review Sessions Needed to Address Troubled Projects' which was released on June 13, 2013. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Requesters: June 2013: Information Technology: Additional Executive Review Sessions Needed to Address Troubled Projects: GAO-13-524: GAO Highlights: Highlights of GAO-13-524, a report to congressional requesters. Why GAO Did This Study: While IT investments have the potential to make organizations more efficient, many federal IT projects experience cost overruns, schedule delays, and performance shortfalls. To help address these shortfalls, OMB established TechStats—face-to-face meetings to terminate or turnaround IT investments that are failing or are not producing results. GAO was asked to evaluate the implementation of TechStats. GAO’s objectives were to (1) identify key characteristics of TechStats held to date; (2) evaluate whether selected agencies are conducting TechStats in accordance with OMB guidance, and (3) analyze the extent to which reported TechStat results are tracked and validated. To do so, GAO selected four agencies-—Agriculture, Commerce, HHS, and DHS—- because these were the agencies with the highest number of at-risk investments. GAO analyzed OMB and agency documentation, compared agency processes to TechStat guidance, compared efforts to validate reported outcomes to leading practices, and interviewed OMB and agency officials. What GAO Found: Since January 2010, the Office of Management and Budget (OMB) and selected agencies have held multiple TechStat Accountability Sessions (TechStats) on information technology (IT) investments that varied in terms of function, significance, and risk. As of April 2013, OMB reported conducting 79 TechStats, which focused on 55 investments at 23 federal agencies. The four agencies conducted 37 TechStats covering 28 investments. About 70 percent of the OMB- and 76 percent of agency- led TechStats on major investments were considered medium- to high- risk at the time of the TechStat. However, the number of at-risk TechStats held to date is relatively small compared to the current number of medium- and high-risk IT investments. Until OMB and agencies develop plans to address these investments, the investments will likely remain at risk. Table: Number of TechStats as a Percentage of Medium- and High-Risk Investments, as of May 2013: OMB-led: Governmentwide: Number of at-risk IT investments that have undergone a TechStat: 30; Number of at-risk IT investments: 162; 18.5%. OMB- or Agency-led: Department of Agriculture (Agriculture): Number of at-risk IT investments that have undergone a TechStat: 6; Number of at-risk IT investments: 15; Percentage: 40.0%. Department of Commerce (Commerce): Number of at-risk IT investments that have undergone a TechStat: 7; Number of at-risk IT investments: 12; Percentage: 58.3%. Department of Homeland Security (DHS): Number of at-risk IT investments that have undergone a TechStat: 8; Number of at-risk IT investments: 26; Percentage: 30.8%. Department of Health and Human Services (HHS): Number of at-risk IT investments that have undergone a TechStat: 2; Number of at-risk IT investments: 16; Percentage: 12.5%. Agency average: 33.3%. Source: GAO analysis of IT Dashboard data as of August 2012. [End of table] The selected agencies are generally conducting TechStats in accordance with OMB guidance. OMB’s TechStat guidance includes 15 key requirements, such as when TechStats should be implemented, what participants should be included, and how outcomes should be tracked and reported. DHS implemented all of the TechStat requirements. Commerce, HHS, and Agriculture implemented a majority of the requirements, but each had shortcomings. For example, these agencies did not consistently create memorandums with responsible parties and due dates for action items. Fully implementing OMB’s guidance could better position agencies to effectively manage and resolve problems on IT investments. OMB and selected agencies have tracked and reported positive results from TechStats, with most resulting in improved governance. OMB also reported in 2011 that federal agencies achieved almost $4 billion in life-cycle cost savings as a result of TechStat sessions. However, GAO was unable to validate OMB’s reported results because OMB did not provide artifacts showing that it ensured the results were valid. From GAO’s selected agencies, three investments had cost implications. Agencies provided supporting documentation for about $22.2 million in cost savings and avoidances. Until OMB obtains and shares information on the methods used to validate reported results, it will be difficult for the results to be independently validated and for OMB to provide assurance to Congress and the public that TechStats are achieving their intended impact. What GAO Recommends: GAO is making recommendations to OMB to require agencies to address high-risk investments and to report on how they validated the outcomes. GAO is also making recommendations to selected agencies to address weaknesses in following OMB’s TechStat guidance. OMB and Commerce officials generally agreed with GAO’s recommendations. Agriculture partially agreed with GAO’s assessment; neither it nor HHS commented on the recommendations. View GAO-13-524. For more information, contact David A. Powner at (202) 512-9286 or pownerd@gao.gov. [End of section] Contents: Letter: Background: OMB and Selected Agencies Have Held Multiple TechStats on a Diverse Set of IT Investments, but Other At-Risk Investments Have Not Yet Been Assessed: Selected Agencies are Generally Conducting TechStats in Accordance with OMB Guidance, but Areas for Improvement Exist: OMB and Agencies Reported Positive Results and Cost Savings from Holding TechStats, but Are Not Ensuring Results Are Valid: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Objectives, Scope, and Methodology: Appendix II: Comments from the Department of Agriculture: Appendix III: Comments from the Department of Commerce: Appendix IV: GAO Contact and Staff Acknowledgments: Tables: Table 1: IT Dashboard CIO Rating Colors, Based on a Five-Point Scale for CIO Ratings: Table 2: Investments on which OMB Reported Holding TechStats: Table 3: Reported Spending on Major Investments on Which OMB Reported Holding a TechStat, through September 2012: Table 4: Dashboard Ratings for the Major IT Investments that were Subject to a TechStat: Table 5: Investments Selected for Agency-led TechStats from Four Selected Agencies: Table 6: Reported Spending on Major Investments on which Agencies Performed a TechStat, through September 2012: Table 7: CIO Ratings for Major Investments on Which Agencies Held a TechStat: Table 8: Number of TechStats as a Percentage of Medium-or High-Risk Investments, as of May 2013: Table 9: High Risk Investments that Have Not Undergone an OMB-led or Agency-led TechStat, as of May 2013: Table 10: Key Requirements for TechStats: Table 11: Agencies' Implementation of OMB TechStat Guidance: Table 12: Outcomes of Agency-Led TechStat Reviews for Four Selected Agencies (as of March 2013): Figures: Figure 1: Number of OMB-led TechStats, by Agency, as of April 2013: Figure 2: Primary Function of Investments Subject to a TechStat: Figure 3: Number of Major and Non-Major Investments that Underwent a TechStat: Figure 4: Level of TechStat at Four Selected Agencies: Figure 5: Primary Function of Investments that Underwent a TechStat at Four Selected Agencies: Figure 6: Number of Major and Non-Major Investments that Underwent a TechStat at Four Selected Agencies: Figure 7: OMB's Reported Cost Savings and Avoidances, as of November 2011: Figure 8: Cost Savings and Avoidances for Agency-Led TechStat Sessions, as of November 2011: Abbreviations: Agriculture: Department of Agriculture: CIO: Chief Information Officer: Commerce: Department of Commerce: DHS: Department of Homeland Security: DOD: Department of Defense: DOT: Department of Transportation: Education: Department of Education: Energy: Department of Energy: EPA: Environmental Protection Agency: GSA: General Services Administration: HHS: Department of Health and Human Services: HUD: Department of Housing and Urban Development: Interior: Department of the Interior: IT: information technology: Justice: Department of Justice: NARA: National Archives and Records Administration: OMB: Office of Management and Budget: OPM: Office of Personnel Management: SBA: Small Business Administration: SSA: Social Security Administration: State: Department of State: Treasury: Department of the Treasury: USAID: U.S. Agency for International Development: VA: Department of Veterans Affairs: [End of section] United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: June 13, 2013: The Honorable Thomas R. Carper: Chairman: The Honorable Tom Coburn, M.D. Ranking Member: Committee on Homeland Security and Governmental Affairs: United States Senate: The Honorable Claire McCaskill: Chairman: Subcommittee on Financial and Contracting Oversight: Committee on Homeland Security and Governmental Affairs: United States Senate: The Honorable Susan Collins: United States Senate: Many information technology (IT) projects undertaken by the federal government have cost hundreds of millions of dollars more than anticipated and taken years longer to deploy than was originally expected. To help address these issues, the Office of Management and Budget (OMB) launched TechStat Accountability Sessions (TechStats) in January 2010. A TechStat is a face-to-face, evidence-based accountability review of an IT investment that enables the federal government to intervene to turnaround, halt, or terminate projects that are failing or are not producing results. OMB began leading TechStat sessions on agency IT projects in 2010, and subsequently required federal agencies to start holding them too. You asked us to evaluate how OMB and selected federal agencies are implementing TechStats. Our objectives were to (1) identify key characteristics of TechStats conducted by OMB and selected agencies, (2) evaluate whether selected agencies are conducting TechStats in accordance with OMB guidance, and (3) analyze the extent to which reported results from TechStat review sessions are documented, tracked, and validated. To do so, we selected four agencies--the Department of Agriculture (Agriculture), the Department of Commerce (Commerce), the Department of Health and Human Services (HHS), and the Department of Homeland Security (DHS)--based on two factors: the number of at-risk IT investments shown on OMB's IT Dashboard as of August 2012 and a determination on whether the agency had led TechStat sessions during 2011. We analyzed OMB's list of the TechStat sessions it had led through February 2013 as well as the latest budget data for each of these investments to determine key characteristics of the investments that underwent OMB-led TechStats. We also analyzed the four selected agencies' lists of the agency-led TechStat reviews held as of March 2013 as well as budgetary and other supporting documentation for each investment and review to determine key characteristics of the investments that underwent agency-led TechStats. We compared the OMB and agency TechStat documentation against OMB guidance to identify strengths or weaknesses in the agencies' approaches. We also compared OMB's and the agencies' reported TechStat results to OMB's guidance on reporting agency-led outcomes and other relevant best practices. [Footnote 1] In addition, we interviewed officials at OMB and our four selected agencies to better understand the TechStat process and associated results from these reviews. While we were able to corroborate information about the IT investments, we were unable to determine the reliability of the reported outcomes and cost savings for the OMB-led TechStats because OMB did not provide evidence that would allow us to verify its reported data. We conducted this performance audit from September 2012 to June 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix I provides additional details on our objectives, scope, and methodology. Background: If done correctly, investments in IT have the potential to make organizations more efficient in fulfilling their missions. For example, Department of Defense (DOD) officials recently reported that an IT system supporting military logistics has improved the organization's performance by providing real-time information about road conditions, construction, incidents, and weather to facilitate rapid deployment of military assets.[Footnote 2] Also, Federal Aviation Administration officials reported that an IT system supporting weather data processing has improved aviation operations by integrating terminal and aircraft sensor data with forecast data from the National Weather Service, and providing it to air traffic controllers. These officials estimated that the system allows them to increase airspace capacity by 25 percent in certain weather conditions.[Footnote 3] However, as we have previously reported, federal IT projects too frequently incur cost overruns and schedule slippages while contributing little to mission-related outcomes. For example, in May 2010, we reported that after spending $127 million over 9 years on an outpatient scheduling system, the Department of Veterans Affairs has not implemented any of the system's capabilities and was essentially starting over.[Footnote 4] Further, in May 2012, we reported that while IT should enable government to better serve the American people, the federal government had not achieved expected productivity improvements--despite spending more than $600 billion on IT over the past decade.[Footnote 5] OMB plays a key role in overseeing how federal agencies manage their IT investments by working with them to better plan, justify, and determine how to manage them. Each year, OMB and federal agencies work together to determine how much the government plans to spend on IT projects and how these funds are to be allocated. OMB also guides agencies in developing sound business cases for IT investments and establishing management processes for overseeing these investments throughout their life cycles. The scope of this undertaking is quite large: in planning for fiscal year 2014, 27 federal agencies reported plans to spend about $76.5 billion on 8,142 IT investments.[Footnote 6] Roles and Responsibilities for Overseeing Federal IT Investments: Over the last three decades, Congress has enacted several laws to assist agencies and the federal government in managing IT investments. For example, the Paperwork Reduction Act of 1995[Footnote 7] specifies OMB and agency responsibilities for managing IT. Among its provisions, this law establishes agency responsibility for maximizing the value and assessing and managing the risks of major information systems initiatives.[Footnote 8] It also requires that OMB develop and oversee policies, principles, standards, and guidelines for federal agency IT functions, including periodic evaluations of major information systems. In addition, to assist agencies in managing their investments, Congress enacted the Clinger-Cohen Act of 1996.[Footnote 9] This law requires OMB to establish processes to analyze, track, and evaluate the risks and results of major capital investments in information systems made by federal agencies. It also requires that OMB report to Congress on the net program performance benefits achieved as a result of these investments. As set out in these laws, OMB is to play a key role in helping federal agencies manage their investments by working with them to better plan, justify, and determine how much they need to spend on projects and how to manage approved projects. Within OMB, the Office of E-government and Information Technology, headed by the Federal Chief Information Officer (CIO), directs the policy and strategic planning of federal IT investments and is responsible for oversight of federal technology spending. Agency CIOs are also expected to have a key role in IT management. Federal law, specifically the Clinger-Cohen Act, has defined the role of the CIO as the focal point for IT management, requiring agency heads to designate CIOs to lead reforms that would help control system development risks; better manage technology spending; and achieve real, measurable improvements in agency performance. In September 2011, we reported that federal CIOs are not consistently responsible for all of the areas assigned by law or identified as critical to effective IT management.[Footnote 10] For example, although most of the CIOs were responsible for capital planning and investment management, we found that CIOs are less frequently responsible for information management duties such as records management and privacy requirements. In an August 2011 memo, OMB reiterated the primary areas of responsibility for agency CIOs.[Footnote 11] This memo detailed four areas in which the CIO should have a lead role: IT governance, program management, commodity services, and information security. It emphasized the role of the CIO in driving the investment review process, including TechStats, and the CIO's responsibility over the entire IT portfolio for an agency. OMB Established the IT Dashboard to Better Oversee and Manage Federal IT Investments: To help carry out its oversight role and further improve the transparency into and oversight of agencies' IT investments, in June 2009 OMB publicly deployed a website, known as the IT Dashboard.[Footnote 12] The Dashboard displays federal agencies' cost, schedule, and performance data for over 700 major federal IT investments at 27 federal agencies that comprise about $40 billion of the federal budget. According to OMB, these data are intended to provide both a historical and a near-real-time perspective on the performance of these investments. OMB analysts are expected to use the Dashboard to identify IT investments that are experiencing performance problems. Using data drawn from federal agency budget submissions,[Footnote 13] the IT Dashboard provides information on an IT investment's primary function, as defined by the Federal Enterprise Architecture.[Footnote 14] For fiscal year 2012 submissions, agencies were required to select a primary function from categories within the Federal Enterprise Architecture business reference models. The primary functions available for IT investments in fiscal year 2012 submissions were: * administrative management; * community and social services; * controls and oversight; * correctional activities; * defense and national security; * disaster management; * economic development; * education; * energy; * environmental management; * financial management; * general government; * general science and innovation; * health; * homeland security; * human resource management; * income security; * information and technology management; * intelligence operations; * internal risk management and mitigation; * international affairs and commerce; * law enforcement; * legislative relations; * litigation and judicial activities; * natural resources; * planning and budgeting; * public affairs; * regulatory development; * revenue collection; * supply chain management; * transportation; * workforce management. The Dashboard visually presents performance ratings for individual investments using metrics that OMB has defined--cost, schedule, and the CIO's evaluation of risk. To develop the CIO's risk evaluation, OMB instructed agency CIOs to assess their IT investments against a set of six evaluation factors, including risk management, requirements management, contractor oversight, historical performance, and human capital. The CIO assigns a rating of 1 to 5 based on his or her best judgment of the level of risk facing the investment. OMB then translates the agency CIO's numerical assignment for an investment into a color for depiction on the Dashboard, with green signifying low or moderately low risk, yellow signifying medium risk, and red signifying moderately high or high risk (see table 1). Table 1: IT Dashboard CIO Rating Colors, Based on a Five-Point Scale for CIO Ratings: Rating (by agency CIO): 5 - Low risk; Color code: Green. Rating (by agency CIO): 4 - Moderately low risk; Color code: Green. Rating (by agency CIO): 3 - Medium risk; Color code: Yellow. Rating (by agency CIO): 2 - Moderately high risk; Color code: Red. Rating (by agency CIO): 1 - High risk; Color code: Red. Source: OMB's IT Dashboard. [End of table] OMB Initiated TechStat Accountability Sessions to Turn Around or Terminate Failing IT Projects: In January 2010, OMB began conducting TechStats to enable the federal government to intervene to turnaround, halt, or terminate IT projects that are failing or are not producing results. TechStats are face-to- face, evidence-based reviews of an at-risk IT investment. OMB used CIO ratings from the IT Dashboard, among other sources, to select at-risk investments for the TechStats it conducted from 2010 through 2011. Subsequently, as part of the Federal CIO's 25-point IT Reform Plan, [Footnote 15] OMB empowered agency CIOs to hold their own TechStat sessions within their respective agencies, and required federal agencies to hold at least one TechStat by March 2011. The IT Reform Plan also required agencies to roll the TechStat model out to its component-level agencies and bureaus (bureaus) by June 2012. To do this, agencies were required to make agency CIOs responsible for deploying the necessary tools and training on how to conduct TechStat reviews and have at least one bureau conduct TechStat reviews by June 2012. In August 2011, OMB required agency CIOs to continue holding TechStat sessions.[Footnote 16] In establishing and rolling out the TechStat sessions, OMB stated that it expects that the sessions will help strengthen IT governance, improve line-of-sight between project teams and senior executives, increase the precision of ongoing measurement of IT program health, and boost the quality and timing of interventions to keep projects on track. We have found that the TechStat model is consistent with government and industry best practices for overseeing IT investments, including our own guidance on IT investment management processes. [Footnote 17] TechStat sessions hold value by focusing management attention on troubled projects and establishing clear action items to turn the projects around or terminate them. GAO Has Previously Reported on Federal IT Investment Oversight and Reform: Over the past several years, we issued numerous reports and testimonies on OMB's initiatives to highlight troubled IT projects. [Footnote 18] We made multiple recommendations to OMB and federal agencies to enhance the oversight and transparency of federal IT projects. For example, in 2005 we recommended that OMB develop a central list of projects and their deficiencies, and analyze that list to develop governmentwide and agency assessments of the progress and risks of the investments, identifying opportunities for continued improvement.[Footnote 19] In 2006, we recommended that OMB develop a single aggregate list of high-risk projects and their deficiencies and use that list to report to Congress on progress made in correcting high-risk problems.[Footnote 20] As a result, OMB started publicly releasing aggregate data on its internal list of mission-critical projects that needed to improve (called its Management Watch List) and disclosing the projects' deficiencies. The agency also established a High-Risk List, which consisted of projects identified as requiring special attention from oversight authorities and the highest levels of agency management. In June 2009, OMB replaced the Management Watch List and the High-Risk List when it deployed a public website--the IT Dashboard--to further improve the transparency and oversight of agencies' IT investments. In 2010 and 2011, we reported that while the Dashboard was an important tool for monitoring major IT projects, the cost and schedule ratings were not always accurate for selected agencies.[Footnote 21] We made recommendations to improve the accuracy of the data and, more recently, found that the accuracy had improved. In April 2012, we reported on the progress of OMB and selected federal agencies on action items in OMB's IT Reform Plan.[Footnote 22] We found that, of 10 selected action items, agencies had completed 3 and made progress on the other 7. However, we found that agencies lacked time frames for completing 5 of those 7 in-progress action items, and only had performance measures for 4 of the 10 selected action items. Thus, we recommended, among other things, that OMB ensure that action items are completed prior to the IT Reform Plan's June 2012 deadline and establish time frames and performance measures for the action items. OMB agreed with our recommendations to complete action items and provide deadlines, but disagreed with our recommendation to establish performance measures. OMB has made progress with completing incomplete action items, but more remains to be done. For example, one action item involved issuing contracting guidance to support modular development and OMB completed this action in June 2012. However, another action item involved ensuring agency data center consolidation plans were in place so agencies could close 800 data centers by 2015, and we recently reported that all but 1 of the 24 agencies' plans were incomplete.[Footnote 23] Most recently, in October 2012, we reported specifically on the CIO rating portion of the IT Dashboard at six selected agencies.[Footnote 24] We found that CIOs at six federal agencies rated the majority of their IT investments as low risk, and that many ratings remained constant over time. For ratings that did change, we found two agencies reported more investments with reduced risk compared to earlier risk ratings; the other four agencies reported more investments with increased risk. In addition, we reported instances where the CIO ratings did not appropriately reflect significant cost, schedule, and performance issues reported by GAO and others. We recommended that the Federal CIO analyze agency trends reflected in Dashboard CIO ratings, and report the results of this analysis in future budget submissions. OMB agreed with the recommendation and recently released some information on trends as part of the 2014 budget. We have work under way to evaluate this information. OMB and Selected Agencies Have Held Multiple TechStats on a Diverse Set of IT Investments, but Other At-Risk Investments Have Not Yet Been Assessed: OMB and the four selected agencies have held multiple TechStats on IT investments that varied in terms of function, significance, amount spent to date, and risk level. Specifically, from January 2010 through April 2013, OMB reported leading 79 TechStat sessions, which focused on 55 IT investments at 23 federal agencies. These investments covered 21 functional areas (such as information and technology management, law enforcement, and health), and consist of 45 major, 8 non-major, and 2 unrated investments.[Footnote 25] For the 45 major investments that OMB reviewed, almost 70 percent had a CIO rating of medium to high risk at the time of the TechStat review. The four agencies selected for our review--Agriculture, Commerce, DHS, and HHS--held a total of 37 TechStat sessions covering 28 investments from January 2011 through March 2013. Most of the investments underwent a TechStat session at the agency level, but 8 underwent a TechStat review at the bureau level, and 2 underwent TechStat reviews at both. The agency-led TechStats covered 14 functional areas, and consist of 21 major and 7 non-major investments. For the 21 major investments, about 76 percent of the agency-led TechStats were on investments that had a CIO rating of medium-to high-risk. While both OMB and agencies have made progress in holding TechStat sessions, there is more that could be done. Specifically, the number of TechStats held to date is relatively small compared to the total number of medium-and high-risk IT investments. Further, there are multiple high-risk IT investments spending millions of dollars that have not yet been assessed. Two of the selected agencies--Agriculture and Commerce--had reviewed all of their high-risk investments, and DHS has plans to review its remaining high-risk investments. However, HHS has not yet established plans to review all of its high-risk investments. Also, OMB does not have plans or schedules for assessing the other high-risk IT investments. Until OMB and agencies intervene to turn around these at-risk projects, the government will continue to spend limited IT investment dollars on underperforming projects. OMB Reported Holding 79 TechStats for a Variety of IT Investments: As of April 2013, OMB reported conducting 79 TechStat reviews, with 59 reviews occurring in 2010, 8 in 2011, 11 in 2012, and 1 so far in 2013. OMB conducted fewer TechStats in recent years because it expected the agencies to increase the number of agency-led TechStats. Most of the OMB-led TechStats were at DHS and DOD. OMB has conducted at least one TechStat at 23 agencies (including the Office of the Director of National Intelligence). It has not held a TechStat at the Department of Labor, the National Aeronautics and Space Administration, the National Science Foundation, the Nuclear Regulatory Commission, or the Smithsonian Institution. Figure 1 shows the total number of reported OMB-led TechStats at each agency as of April 2013. Figure 1: Number of OMB-led TechStats, by Agency, as of April 2013: [Refer to PDF for image: vertical bar graph] Agency: Agriculture; Number of TechStats: 4. Agency: Army Corps; Number of TechStats: 1. Agency: Commerce; Number of TechStats: 4. Agency: DHS; Number of TechStats: 11. Agency: DOD; Number of TechStats: 10. Agency: DOE; Number of TechStats: 1. Agency: DOI; Number of TechStats: 7. Agency: DOJ; Number of TechStats: 4. Agency: DOT; Number of TechStats: 5. Agency: Education; Number of TechStats: 1. Agency: EPA; Number of TechStats: 1. Agency: GSA; Number of TechStats: 3. Agency: HHS; Number of TechStats: 2. Agency: HUD; Number of TechStats: 4. Agency: NARA; Number of TechStats: 2. Agency: ODNI; Number of TechStats: 2. Agency: OPM; Number of TechStats: 2. Agency: SBA; Number of TechStats: 2. Agency: SSA; Number of TechStats: 1. Agency: State; Number of TechStats: 1. Agency: Treasury; Number of TechStats: 4. Agency: USAID; Number of TechStats: 1. Agency: VA; Number of TechStats: 6. Source: GAO analysis of OMB and agency data. Note: Key to agency names: Agriculture = U.S. Department of Agriculture; Army Corps = Army Corps of Engineers; Commerce = Department of Commerce; DHS = Department of Homeland Security; DOD = Department of Defense; DOE = Department of Energy; DOI = Department of the Interior; DOJ = Department of Justice; DOT = Department of Transportation; Education = Department of Education; EPA = Environmental Protection Agency; GSA = General Services Administration; HHS = Department of Health and Human Services; HUD = Department of Housing and Urban Development; NARA = National Archives and Records Administration; ODNI = Office of the Director of National Intelligence; OPM = Office of Personnel Management; SBA = Small Business Administration; SSA = Social Security Administration; State = Department of State; Treasury = Department of the Treasury; USAID = U.S. Agency for International Development; and; VA = Department of Veterans Affairs. [End of figure] These 79 TechStat reviews included 55 IT investments from 23 federal agencies. OMB also reported leading follow-up sessions for many of these investments. For example, OMB led 4 TechStats on a DHS investment called the Federal Emergency Management Agency--National Flood Insurance Program Information Technology Systems and Services. Table 2 identifies the investments for which OMB led TechStats, the date of the first session, and the total number of sessions that OMB has held on that investment. Table 2: Investments on which OMB Reported Holding TechStats: Agency: Agriculture; Investment name: Conservation Delivery Streamline Initiative; Date of first TechStat held: 11/15/2011; Number of TechStats: 1. Agency: Agriculture; Investment name: Web-based Supply Chain Management; Date of first TechStat held: 3/2/2010; Number of TechStats: 3. Agency: Army Corps of Engineers; Investment name: Consolidated IT Infrastructure/Office Automation/Telecommunications; Date of first TechStat held: 1/29/2010; Number of TechStats: 1. Agency: Commerce; Investment name: Bureau of Industry and Security Export Control Automated Support System 2000+; Date of first TechStat held: 4/20/2010; Number of TechStats: 2. Agency: Commerce; Investment name: National Oceanic and Atmospheric Administration National Air Quality Forecast Capability; Date of first TechStat held: 2/24/2010; Number of TechStats: 1. Agency: Commerce; Investment name: Patent File Wrapper Program; Date of first TechStat held: 9/20/2010; Number of TechStats: 1. Agency: Department of Defense; Investment name: Air Force-Integrated Personnel and Pay System; Date of first TechStat held: 10/27/2010; Number of TechStats: 2. Agency: Department of Defense; Investment name: Expeditionary Combat Support System; Date of first TechStat held: 5/7/2010; Number of TechStats: 3. Agency: Department of Defense; Investment name: Future Personnel and Pay Solution; Date of first TechStat held: 10/27/2010; Number of TechStats: 2. Agency: Department of Defense; Investment name: Integrated Personnel and Pay System - Army; Date of first TechStat held: 10/27/2010; Number of TechStats: 3. Agency: Department of Education; Investment name: Grants Management; Date of first TechStat held: 5/21/2010; Number of TechStats: 1. Agency: Department of Energy; Investment name: Performance and Accountability for Grants in Energy; Date of first TechStat held: 6/4/2010; Number of TechStats: 1. Agency: Department of Housing and Urban Development; Investment name: Human Resources Automated Services; Date of first TechStat held: 6/5/2012; Number of TechStats: 1. Agency: Department of Housing and Urban Development; Investment name: Office of the CIO Transformation Initiative; Date of first TechStat held: 9/27/2010; Number of TechStats: 2. Agency: Department of Housing and Urban Development; Investment name: Voucher Management; Date of first TechStat held: 5/31/2012; Number of TechStats: 1. Agency: Department of Justice; Investment name: Federal Bureau of Investigation Next Generation Identification program; Date of first TechStat held: 4/21/2010; Number of TechStats: 2. Agency: Department of Justice; Investment name: Federal Bureau of Investigation Sentinel; Date of first TechStat held: 10/18/2010; Number of TechStats: 1. Agency: Department of Justice; Investment name: Litigation Case Management System; Date of first TechStat held: 9/9/2010; Number of TechStats: 1. Agency: Department of State; Investment name: Foreign Assistance Coordination and Tracking System; Date of first TechStat held: 2/24/2010; Number of TechStats: 1. Agency: Department of the Interior; Investment name: Bureau of Land Management Collections and Billings System; Date of first TechStat held: 9/17/2012; Number of TechStats: 1. Agency: Department of the Interior; Investment name: Bureau of Ocean Energy Management, Regulation and Enforcement Outer Continental Shelf Connect; Date of first TechStat held: 3/17/2010; Number of TechStats: 1. Agency: Department of the Interior; Investment name: Capstone Facility Management System; Date of first TechStat held: 3/30/2010; Number of TechStats: 1. Agency: Department of the Interior; Investment name: Consolidated Infrastructure, Automation, Telecomm; Date of first TechStat held: 10/27/2010; Number of TechStats: 1. Agency: Department of the Interior; Investment name: Incident Management, Analysis, and Reporting System; Date of first TechStat held: 4/29/2010; Number of TechStats: 2. Agency: Department of the Interior; Investment name: Reclamation Mission Support Services; Date of first TechStat held: 6/29/2012; Number of TechStats: 1. Agency: Department of the Treasury; Investment name: Bank Secrecy Act IT Modernization; Date of first TechStat held: 4/15/2010; Number of TechStats: 1. Agency: Department of the Treasury; Investment name: Integrated Financial System; Date of first TechStat held: 4/7/2010; Number of TechStats: 1. Agency: Department of the Treasury; Investment name: IT Infrastructure Telecommunications Systems and Support; Date of first TechStat held: 9/29/2010; Number of TechStats: 1. Agency: Department of the Treasury; Investment name: Treasury Enterprise Identity, Credential and Access Management; Date of first TechStat held: 10/25/2010; Number of TechStats: 1. Agency: Department of Transportation; Investment name: En Route Automation Modernization; Date of first TechStat held: 11/9/2010; Number of TechStats: 1. Agency: Department of Transportation; Investment name: Federal Motor Carrier Safety Administration Application and Information Services Modernization; Date of first TechStat held: 8/30/2012; Number of TechStats: 1. Agency: Department of Transportation; Investment name: Instrument Flight Procedure Automation; Date of first TechStat held: 4/14/2010; Number of TechStats: 1. Agency: Department of Transportation; Investment name: Logical Access and Authorization Control Service; Date of first TechStat held: 3/11/2010; Number of TechStats: 2. Agency: Department of Veterans Affairs; Investment name: Benefits 21st Century Paperless Delivery of Veterans Benefits; Date of first TechStat held: 10/7/2010; Number of TechStats: 1. Agency: Department of Veterans Affairs; Investment name: Enterprise IT Support; Date of first TechStat held: 4/30/2012; Number of TechStats: 1. Agency: Department of Veterans Affairs; Investment name: InterAgency 21st Century Personal Identity Verification; Date of first TechStat held: 1/21/2010; Number of TechStats: 3. Agency: Department of Veterans Affairs; Investment name: Medical 21st Century Pharmacy; Date of first TechStat held: 10/7/2010; Number of TechStats: 1. Agency: DHS; Investment name: Automated Commercial Environment/International Trade Data System; Date of first TechStat held: 3/31/2010; Number of TechStats: 2. Agency: DHS; Investment name: Customs and Border Protection - Secure Border Initiative; Date of first TechStat held: 3/10/2010; Number of TechStats: 1. Agency: DHS; Investment name: Homeland Security Information Network; Date of first TechStat held: 5/11/2010; Number of TechStats: 2. Agency: DHS; Investment name: National Flood Insurance Program Information Technology Systems and Services; Date of first TechStat held: 8/4/2010; Number of TechStats: 4. Agency: DHS; Investment name: U.S. Citizenship and Immigration Services Transformation; Date of first TechStat held: 3/28/2012; Number of TechStats: 2. Agency: Environmental Protection Agency; Investment name: Financial Replacement System; Date of first TechStat held: 1/13/2010; Number of TechStats: 1. Agency: General Services Administration; Investment name: Federal Supply Service 19; Date of first TechStat held: 10/6/2010; Number of TechStats: 1. Agency: General Services Administration; Investment name: Integrated Acquisition Environment; Date of first TechStat held: 9/26/2012; Number of TechStats: 2. Agency: HHS; Investment name: Food and Drug Administration MedWatch Plus; Date of first TechStat held: 10/8/2010; Number of TechStats: 1. Agency: HHS; Investment name: Secure One Health and Human Services; Date of first TechStat held: 2/4/2010; Number of TechStats: 1. Agency: National Archives and Records Administration; Investment name: Electronic Records Archives Program; Date of first TechStat held: 6/17/2010; Number of TechStats: 2. Agency: Office of Personnel Management; Investment name: Enterprise Infrastructure Operations; Date of first TechStat held: 1/29/2010; Number of TechStats: 1. Agency: Office of Personnel Management; Investment name: Retirement Systems Modernization; Date of first TechStat held: 10/19/2010; Number of TechStats: 1. Agency: Office of the Director of National Intelligence; Investment name: Business Transformation Office; Date of first TechStat held: 6/3/2010; Number of TechStats: 2. Agency: Small Business Administration; Investment name: Identity and Access Management and Homeland Security Presidential Directive-12; Date of first TechStat held: 3/10/2010; Number of TechStats: 1. Agency: Small Business Administration; Investment name: Loan Management and Accounting System; Date of first TechStat held: 4/8/2010; Number of TechStats: 1. Agency: Social Security Administration; Investment name: Citizen Access Routing Enterprise through 2020; Date of first TechStat held: 11/1/2010; Number of TechStats: 1. Agency: U.S. Agency for International Development; Investment name: OpsMaster; Date of first TechStat held: 11/1/2011; Number of TechStats: 1. Source: GAO analysis of OMB and agency data. [End of table] These 55 investments span 21 primary functional areas.[Footnote 26] The most common functional area was information and technology management, followed by homeland security and human resource management. Figure 2 provides a numerical depiction of the functions of those investments that were subject to a TechStat. Figure 2: Primary Function of Investments Subject to a TechStat: [Refer to PDF for image: vertical bar graph] Primary function: Administrative Management; Number of investments: 2. Primary function: Controls and Oversight; Number of investments: 1. Primary function: Disaster Management; Number of investments: 1. Primary function: Economic Development; Number of investments: 3. Primary function: Education; Number of investments: 1. Primary function: Energy; Number of investments: 2. Primary function: Environmental Management; Number of investments: 1. Primary function: Financial Management; Number of investments: 2. Primary function: General Government; Number of investments: 2. Primary function: Health; Number of investments: 1. Primary function: Homeland Security; Number of investments: 5. Primary function: Human Resource Management; Number of investments: 5. Primary function: Income Security; Number of investments: 1. Primary function: Information and Technology Management; Number of investments: 11. Primary function: International Affairs and Commerce; Number of investments: 1. Primary function: Law Enforcement; Number of investments: 3. Primary function: Litigation and Judicial Activities; Number of investments: 1. Primary function: Natural Resources; Number of investments: 2. Primary function: Planning and Budgeting; Number of investments: 1. Primary function: Supply Chain Management; Number of investments: 4. Primary function: Transportation; Number of investments: 3. Primary function: Unknown; Number of investments: 2. Source: GAO analysis of OMB and agency Exhibit 53 data. [End of figure] OMB held most of its TechStats on major IT investments, with 45 of its 55 investments rated as major investments. OMB also held 8 TechStats on non-major investments and 2 on investments lacking a major/non- major designation. Figure 3 shows the breakdown of major and non-major investments. Figure 3: Number of Major and Non-Major Investments that Underwent a TechStat: [Refer to PDF for image: pie-chart] Major: 45; Non-major: 8; Not designated: 2. Source: GAO analysis of OMB and agency Exhibit 53 data. [End of figure] Thirty-eight of the 45 major investments that underwent an OMB-led TechStat have reportedly cost the federal government about $16 billion through fiscal year 2012.[Footnote 27] DHS's investments that were the subject of a TechStat session accounted for most of the cost, with a reported $4.9 billion spent. DOT follows with investments that have reportedly cost $2.8 billion through September 2012. The reported cost of individual investments ranged from $4.18 million to $3.2 billion. Table 3 details how much each agency has spent on its major investment. Table 3: Reported Spending on Major Investments on Which OMB Reported Holding a TechStat, through September 2012: Home agency/investment: Agriculture; Reported spending: $194.68 million. Home agency/investment: Agriculture; Conservation Delivery Streamline Initiative; Reported spending: $26.63 million. Home agency/investment: Agriculture; Web-based Supply Chain Management; Reported spending: $168.05 million. Home agency/investment: Department of Defense; [Empty]; Reported spending: $1.698 billion. Home agency/investment: Department of Defense; Air Force Integrated Personnel and Pay System; Reported spending: $169.42 million. Home agency/investment: Department of Defense; Expeditionary Combat Support System; Reported spending: $1.348 billion. Home agency/investment: Department of Defense; Integrated Personnel and Pay System - Army; Reported spending: $180.01 million. Home agency/investment: Department of Education; [Empty]; Reported spending: $30.98 million. Home agency/investment: Department of Education; Grants Management; Reported spending: $30.98 million. Home agency/investment: Department of Energy; [Empty]; Reported spending: $28.35 million. Home agency/investment: Department of Energy; Performance and Accountability for Grants in Energy; Reported spending: $28.35 million. Home agency/investment: Department of Housing and Urban Development; Reported spending: $102.02 million. Home agency/investment: Department of Housing and Urban Development; Human Resources Automated Services; Reported spending: $30.55 million. Home agency/investment: Department of Housing and Urban Development; Voucher Management; Reported spending: $71.47 million. Home agency/investment: Department of Justice; Reported spending: $1.290 billion. Home agency/investment: Department of Justice; Federal Bureau of Investigation Next Generation Identification program; [Empty]; Reported spending: $780.45 million. Home agency/investment: Department of Justice; Federal Bureau of Investigation Sentinel; Reported spending: $509.74 million. Home agency/investment: Department of State; Reported spending: $23.20 million. Home agency/investment: Department of State; Foreign Assistance Coordination and Tracking System; Reported spending: $23.20 million. Home agency/investment: Department of the Interior; Reported spending: $443.22 million. Home agency/investment: Department of the Interior; Bureau of Land Management - Collections and Billings System; Reported spending: $40.52 million. Home agency/investment: Department of the Interior; Incident Management, Analysis, and Reporting System; Reported spending: $64.75 million. Home agency/investment: Department of the Interior; Reclamation Mission Support Services; Reported spending: $337.95 million. Home agency/investment: Department of the Treasury; Reported spending: $502.82 million. Home agency/investment: Department of the Treasury; Bank Secrecy Act IT Modernization; Reported spending: $31.49 million. Home agency/investment: Department of the Treasury; Integrated Financial System; Reported spending: $353.31 million. Home agency/investment: Department of the Treasury; Treasury Enterprise Identity, Credential and Access Management; Reported spending: $118.03 million. Home agency/investment: Department of Transportation; Reported spending: $2.778 billion[A]. Home agency/investment: Department of Transportation; En Route Automation Modernization; Reported spending: $2.677 billion. Home agency/investment: Department of Transportation; Federal Motor Carrier Safety Administration Application and Information Services Modernization; Reported spending: $12.30 million. Home agency/investment: Department of Transportation; Instrument Flight Procedure Automation; Reported spending: $87.99 million. Home agency/investment: Department of Veterans Affairs; Reported spending: $1.788 billion. Home agency/investment: Department of Veterans Affairs; Benefits 21st Century Paperless Delivery of Veterans Benefits; Reported spending: $388.81 million. Home agency/investment: Department of Veterans Affairs; Enterprise IT Support; Reported spending: $1.306 billion. Home agency/investment: Department of Veterans Affairs; InterAgency 21st Century Personal Identity Verification; Reported spending: $57.55 million. Home agency/investment: Department of Veterans Affairs; Medical 21st Century Pharmacy; Reported spending: $35.49 million. Home agency/investment: DHS; Reported spending: $4.974 million[A]. Home agency/investment: DHS; Automated Commercial Environment /International Trade Data System; Reported spending: $3.176 billion. Home agency/investment: DHS; Homeland Security Information Network; Reported spending: $237.59 million. Home agency/investment: DHS; National Flood Insurance Program Information Technology Systems and Services; Reported spending: $104.7 million2. Home agency/investment: DHS; U.S. Citizenship and Immigration Services Transformation; Reported spending: $1.456 billion. Home agency/investment: Environmental Protection Agency; Reported spending: $200.34 million. Home agency/investment: Environmental Protection Agency; Financial Replacement System; Reported spending: $200.34 million. Home agency/investment: General Services Administration; Reported spending: $726.73 million[A]. Home agency/investment: General Services Administration; Federal Supply Service 19; Reported spending: $228.49 million. Home agency/investment: General Services Administration; Integrated Acquisition Environment; Reported spending: $498.25 million. Home agency/investment: HHS; Reported spending: $263.64 million. Home agency/investment: HHS; Food and Drug Administration MedWatch Plus; Reported spending: $71.85 million. Home agency/investment: HHS; Secure One Health and Human Services; Reported spending: $191.79 million. Home agency/investment: National Archives and Records Administration; Reported spending: $495.99 million. Home agency/investment: National Archives and Records Administration; Electronic Records Archive Program; Reported spending: $495.99 million. Home agency/investment: Office of Personnel Management; Reported spending: $339.78 million. Home agency/investment: Office of Personnel Management; Enterprise Infrastructure Operations; Reported spending: $339.78 million. Home agency/investment: Small Business Administration; Reported spending: $4.18 million. Home agency/investment: Small Business Administration; Identity and Access Management and Homeland Security Presidential Directive-12; Reported spending: $4.18 million. Home agency/investment: Social Security Administration; Reported spending: $130.94 million. Home agency/investment: Social Security Administration; Citizen Access Routing Enterprise through 2020; Reported spending: $130.94 million. Home agency/investment: Total; Reported spending: $16.014 billion. Source: GAO analysis of OMB and agency Exhibit 300 data. [A] Due to rounding, subtotals may not add up to agency totals. [End of table] Of the 45 major IT investments that were subject to a TechStat, 67 percent were considered medium-to high-risk investments at the time they were chosen for a TechStat. Specifically, 9 had high-or moderately high-risk (red) CIO ratings on the IT Dashboard, 21 had medium-risk (yellow) ratings, and 9 had low-or moderately low-risk (green) ratings. In addition, 6 investments' risk levels were not identified. Table 4 shows the Dashboard ratings for the major IT investments that underwent a TechStat. Table 4: Dashboard Ratings for the Major IT Investments that were Subject to a TechStat: Agency: Agriculture; Investment name: Conservation Delivery Streamline Initiative; Date of first TechStat: 11/15/2011; Dashboard rating at time of first TechStat: Medium risk. Agency: Agriculture; Investment name: Web-based Supply Chain Management; Date of first TechStat: 3/2/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: Commerce; Investment name: Patent File Wrapper Program; Date of first TechStat: 9/20/2010; Dashboard rating at time of first TechStat: Moderately high risk. Agency: Department of Defense; Investment name: Air Force-Integrated Personnel and Pay System; Date of first TechStat: 10/27/2010; Dashboard rating at time of first TechStat: No rating[A]. Agency: Department of Defense; Investment name: Expeditionary Combat Support System; Date of first TechStat: 5/7/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: Department of Defense; Investment name: Integrated Personnel and Pay System - Army; Date of first TechStat: 10/27/2010; Dashboard rating at time of first TechStat: No rating[A]. Agency: Department of Education; Investment name: Grants Management; Date of first TechStat: 5/21/2010; Dashboard rating at time of first TechStat: Moderately low risk. Agency: Department of Energy; Investment name: Performance and Accountability for Grants in Energy; Date of first TechStat: 6/4/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: Department of Housing and Urban Development; Investment name: Human Resources Automated Services; Date of first TechStat: 6/5/2012; Dashboard rating at time of first TechStat: Moderately low risk. Agency: Department of Housing and Urban Development; Investment name: Office of CIO Transformation Initiative; Date of first TechStat: 9/27/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: Department of Housing and Urban Development; Investment name: Voucher Management; Date of first TechStat: 5/31/2012; Dashboard rating at time of first TechStat: Medium risk. Agency: Department of Justice; Investment name: Federal Bureau of Investigation Next Generation Identification program; Date of first TechStat: 4/21/2010; Dashboard rating at time of first TechStat: No rating[A]. Agency: Department of Justice; Investment name: Federal Bureau of Investigation Sentinel; Date of first TechStat: 10/18/2010; Dashboard rating at time of first TechStat: Moderately high risk. Agency: Department of State; Investment name: Foreign Assistance Coordination and Tracking System; Date of first TechStat: 2/24/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: Department of the Interior; Investment name: Bureau of Land Management Collections and Billings System; Date of first TechStat: 9/17/2012; Dashboard rating at time of first TechStat: Moderately low risk. Agency: Department of the Interior; Investment name: Bureau of Ocean Energy Management, Regulation and Enforcement Outer Continental Shelf Connect; Date of first TechStat: 3/17/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: Department of the Interior; Investment name: Capstone Facility Management System; Date of first TechStat: 3/30/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: Department of the Interior; Investment name: Consolidated Infrastructure, Automation, Telecomm; Date of first TechStat: 10/27/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: Department of the Interior; Investment name: Incident Management, Analysis, and Reporting System; Date of first TechStat: 4/29/2010; Dashboard rating at time of first TechStat: Moderately low risk. Agency: Department of the Interior; Investment name: Reclamation Mission Support Services; Date of first TechStat: 6/29/2012; Dashboard rating at time of first TechStat: Low risk. Agency: Department of the Treasury; Investment name: Bank Secrecy Act IT Modernization; Date of first TechStat: 4/15/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: Department of the Treasury; Investment name: Integrated Financial System; Date of first TechStat: 4/7/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: Department of the Treasury; Investment name: IT Infrastructure Telecommunications Systems and Support; Date of first TechStat: 9/29/2010; Dashboard rating at time of first TechStat: No rating[A]. Agency: Department of the Treasury; Investment name: Treasury Enterprise Identity, Credential and Access Management; Date of first TechStat: 10/25/2010; Dashboard rating at time of first TechStat: Moderately high risk. Agency: Department of Transportation; Investment name: En Route Automation Modernization; Date of first TechStat: 11/9/2010; Dashboard rating at time of first TechStat: High risk. Agency: Department of Transportation; Investment name: Federal Motor Carrier Safety Administration Application and Information Services Modernization; Date of first TechStat: 8/30/2012; Dashboard rating at time of first TechStat: Medium risk. Agency: Department of Transportation; Investment name: Instrument Flight Procedure Automation; Date of first TechStat: 4/14/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: Department of Veterans Affairs; Investment name: Benefits 21st Century Paperless Delivery of Veterans Benefits; Date of first TechStat: 10/7/2010; Dashboard rating at time of first TechStat: Low risk. Agency: Department of Veterans Affairs; Investment name: Enterprise IT Support; Date of first TechStat: 4/30/2012; Dashboard rating at time of first TechStat: Low risk. Agency: Department of Veterans Affairs; Investment name: InterAgency 21st Century Personal Identity Verification; Date of first TechStat: 1/21/2010; Dashboard rating at time of first TechStat: No rating[A]. Agency: Department of Veterans Affairs; Investment name: Medical 21st Century Pharmacy; Date of first TechStat: 10/7/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: DHS; Investment name: Automated Commercial Environment/International Trade Data System; Date of first TechStat: 3/31/2010; Dashboard rating at time of first TechStat: Moderately high risk. Agency: DHS; Investment name: Homeland Security Information Network; Date of first TechStat: 5/11/2010; Dashboard rating at time of first TechStat: Moderately high risk. Agency: DHS; Investment name: National Flood Insurance Program Information Technology Systems and Services; Date of first TechStat: 8/4/2010; Dashboard rating at time of first TechStat: Moderately high risk. Agency: DHS; Investment name: U.S. Citizenship and Immigration Services Transformation; Date of first TechStat: 3/28/2012; Dashboard rating at time of first TechStat: Moderately high risk. Agency: Environmental Protection Agency; Investment name: Financial Replacement System; Date of first TechStat: 1/13/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: General Services Administration; Investment name: Federal Supply Service 19; Date of first TechStat: 10/6/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: General Services Administration; Investment name: Integrated Acquisition Environment; Date of first TechStat: 9/26/2012; Dashboard rating at time of first TechStat: Medium risk. Agency: HHS; Investment name: Food and Drug Administration MedWatch Plus; Date of first TechStat: 10/8/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: HHS; Investment name: Secure One Health and Human Services; Date of first TechStat: 2/4/2010; Dashboard rating at time of first TechStat: No rating[A]. Agency: National Archives and Records Administration; Investment name: Electronic Records Archives Program; Date of first TechStat: 6/17/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: Office of Personnel Management; Investment name: Enterprise Infrastructure Operations; Date of first TechStat: 1/29/2010; Dashboard rating at time of first TechStat: Moderately high risk. Agency: Office of Personnel Management; Investment name: Retirement Systems Modernization; Date of first TechStat: 10/19/2010; Dashboard rating at time of first TechStat: Medium risk. Agency: Small Business Administration; Investment name: Identity and Access Management and Homeland Security Presidential Directive-12; Date of first TechStat: 3/10/2010; Dashboard rating at time of first TechStat: Moderately low risk. Agency: Social Security Administration; Investment name: Citizen Access Routing Enterprise through 2020; Date of first TechStat: 11/1/2010; Dashboard rating at time of first TechStat: Moderately low risk. Source: GAO analysis of OMB and agency IT Dashboard data. [A] This investment did not have a CIO rating on the IT Dashboard at the time of the investment's first TechStat. [End of table] Selected Agencies Held 37 TechStats on a Diverse Set of IT Investments: From January 2011 through March 2013, the four selected agencies-- Agriculture, Commerce, DHS, and HHS--held 37 TechStats that covered 28 different investments. Commerce held the most, with 15 TechStat sessions, followed by DHS, HHS, and Agriculture, with 8, 7, and 7 sessions, respectively. Table 5 lists the investments, the date of the first TechStat session, and the total number of TechStats by agency. Table 5: Investments Selected for Agency-led TechStats from Four Selected Agencies: Agency: Agriculture; Investment name: Automated Timber Sales Accounting; Date of first TechStat: 2/26/2013; Number of TechStats: 1. Agency: Agriculture; Investment name: Comprehensive Loan Program; Date of first TechStat: 3/24/2011; Number of TechStats: 2. Agency: Agriculture; Investment name: Emerging Information Technology Architecture; Date of first TechStat: 10/31/2012; Number of TechStats: 1. Agency: Agriculture; Investment name: Farm Program Modernization; Date of first TechStat: 11/28/2012; Number of TechStats: 1. Agency: Agriculture; Investment name: Food Safety and Inspection Service Public Health Information Consolidation Projects (now called Public Health Information System); Date of first TechStat: 7/28/2011; Number of TechStats: 1. Agency: Agriculture; Investment name: Web-based Supply Chain Management; Date of first TechStat: 1/14/2013; Number of TechStats: 1. Agency: Commerce; Investment name: Bureau of Industry and Security Commerce Exporter Support System; Date of first TechStat: 8/8/2012; Number of TechStats: 1. Agency: Commerce; Investment name: Business Continuity/Disaster Recovery; Date of first TechStat: 9/20/2012; Number of TechStats: 1. Agency: Commerce; Investment name: Census - Data Access and Dissemination System; Date of first TechStat: 8/3/2011; Number of TechStats: 1. Agency: Commerce; Investment name: Census IT Infrastructure; Date of first TechStat: 6/28/2012; Number of TechStats: 1. Agency: Commerce; Investment name: Export Control and Automated Support System 2000+; Date of first TechStat: 6/20/2011; Number of TechStats: 1. Agency: Commerce; Investment name: National Marine Fisheries Service Office of Law Enforcement Wide Area Network; Date of first TechStat: 12/16/2011; Number of TechStats: 1. Agency: Commerce; Investment name: National Oceanic and Atmospheric Administration/ National Weather Service/Automated Surface Observing System; Date of first TechStat: 11/6/2012; Number of TechStats: 1. Agency: Commerce; Investment name: National Telecommunications and Information Administration--Radio Spectrum Management--Federal Spectrum Management System; Date of first TechStat: 10/9/2012; Number of TechStats: 1. Agency: Commerce; Investment name: National Weather Service Telecommunication Gateway System (Legacy, Replacement, and Critical Infrastructure Protection); Date of first TechStat: 7/8/2011; Number of TechStats: 1. Agency: Commerce; Investment name: Next Generation Weather Radar System Product Improvement; Date of first TechStat: 3/25/2011; Number of TechStats: 2. Agency: Commerce; Investment name: U.S. Patent and Trademark Office Financial Management System; Date of first TechStat: 1/20/2011; Number of TechStats: 1. Agency: Commerce; Investment name: Weather Radio Improvement Project; Date of first TechStat: 6/8/2011; Number of TechStats: 3. Agency: DHS; Investment name: Customs and Border Protection Infrastructure; Date of first TechStat: 3/8/2012; Number of TechStats: 1. Agency: DHS; Investment name: Domestic Nuclear Detection Office--Joint Analysis Center Collaborative Information System; Date of first TechStat: 2/22/2012; Number of TechStats: 1. Agency: DHS; Investment name: Enterprise Performance Analysis; Date of first TechStat: 12/12/2012; Number of TechStats: 1. Agency: DHS; Investment name: Immigration and Customs Enforcement--Electronic Health Record System; Date of first TechStat: 11/9/2011; Number of TechStats: 1. Agency: DHS; Investment name: Immigration and Customs Enforcement--Student and Exchange Visitor Information System; Date of first TechStat: 3/30/2011; Number of TechStats: 2. Agency: DHS; Investment name: DHS Infrastructure Transformation Program; Date of first TechStat: 8/12/2011; Number of TechStats: 1. Agency: DHS; Investment name: National Protection and Programs Directorate--Federal Protective Service Risk Assessment and Management Program; Date of first TechStat: 6/21/2012; Number of TechStats: 1. Agency: HHS; Investment name: Enterprise Human Resources and Personnel; Date of first TechStat: 4/27/2011; Number of TechStats: 4. Agency: HHS; Investment name: Food and Drug Administration Office of Regulatory Affairs Mission Accomplishments and Regulatory Compliance Services; Date of first TechStat: 12/8/2011; Number of TechStats: 1. Agency: HHS; Investment name: One Stop Service Solution; Date of first TechStat:12/28/2011; Number of TechStats: 2. Source: GAO analysis of agency data. [End of table] Although most of the agency-led TechStat reviews were held at the agency level, three of the agencies in our review have also been holding them at the bureau or component agency level. Specifically, of the agencies in our review, 18 of the 28 investments underwent at least one TechStat review at the agency level, 8 underwent a TechStat review at the bureau level, and 2 underwent TechStat reviews at both. Of the selected agencies, Agriculture had not held a bureau-level TechStat (see figure 4). Figure 4: Level of TechStat at Four Selected Agencies: [Refer to PDF for image: stacked vertical bar graph] Agency: Agriculture; Agency: 6; Bureau: 0; Bureau and agency: 0. Agency: Commerce; Agency: 7; Bureau: 4; Bureau and agency: 1. Agency: DHS; Agency: 3; Bureau: 3; Bureau and agency: 1. Agency: HHS; Agency: 2; Bureau: 1; Bureau and agency: 0. Source: GAO analysis of agency data. [End of figure] Like the OMB-led reviews, the investments selected by agencies for TechStat reviews had diverse primary functional areas. Specifically, 25 of the investments that underwent agency-led TechStats at the four selected agencies cover 14 different functional areas.[Footnote 28] The most common area was information and technology management (6 investments), followed by environmental management (4 investments) (see figure 5). Figure 5: Primary Function of Investments that Underwent a TechStat at Four Selected Agencies: [Refer to PDF for image: vertical bar graph] Primary function: Community and Social Services; Number of investments: 1. Primary function: Disaster Management; Number of investments: 1. Primary function: Economic Development; Number of investments: 1. Primary function: Environmental Management; Number of investments: 4. Primary function: Financial Management; Number of investments: 1. Primary function: General government; Number of investments: 2. Primary function: Health; Number of investments: 2. Primary function: Homeland Security; Number of investments: 1. Primary function: Human Resource Management; Number of investments: 1. Primary function: Information and Technology Management; Number of investments: 6. Primary function: International Affairs and Commerce; Number of investments: 2. Primary function: Law Enforcement; Number of investments: 1. Primary function: Public Affairs; Number of investments: 1. Primary function: Supply Chain Management; Number of investments: 1. Source: GAO analysis of agency Exhibit 53 data. [End of figure] Most of the investments that underwent a TechStat review at the four agencies were major investments, with a quarter of the sessions focusing on non-major investments (see figure 6). Figure 6: Number of Major and Non-Major Investments that Underwent a TechStat at Four Selected Agencies: [Refer to PDF for image: pie-chart] Major: 21; Non-major: 7. Source: GAO analysis of agency Exhibit 53 data. [End of figure] The four agencies reported spending $7.8 billion through the end of fiscal year 2012 on 20 of the 21 major investments that underwent an agency-led review.[Footnote 29] The cost of individual investments ranged from $6.22 million for a new Commerce investment to $3.7 billion for a DHS investment. Table 6 provides a summary of the amount spent on major investments through September 2012. Table 6: Reported Spending on Major Investments on which Agencies Performed a TechStat, through September 2012: Agency: Agriculture; Reported cost: $698.01 million. Agency: Agriculture; Investment name: Automated Timber Sales Accounting; Reported cost: $10.93 million. Agency: Agriculture; Investment name: Comprehensive Loan Program; Reported cost: $111.58 million. Agency: Agriculture; Investment name: Emerging Information Technology Architecture; Reported cost: $63.02 million. Agency: Agriculture; Investment name: Farm Program Modernization; Reported cost: $233.69 million. Agency: Agriculture; Investment name: Public Health Information Consolidation Projects (now called Public Health Information System); Reported cost: $110.74 million. Agency: Agriculture; Investment name: Web-based Supply Chain Management; Reported cost: $168.05 million. Agency: Commerce; Reported cost: $1.883 billion. Agency: Commerce; Investment name: Bureau of Industry and Security Commerce Exporter Support System; Reported cost: $6.22 million. Agency: Commerce; Investment name: Census - Data Access and Dissemination System; Reported cost: $375.51 million. Agency: Commerce; Investment name: Census IT Infrastructure; Reported cost: $1.074 billion. Agency: Commerce; Investment name: National Telecommunications and Information Administration - Radio Spectrum Management - Federal Spectrum Management System; Reported cost: $37.75 million. Agency: Commerce; Investment name: National Oceanic and Atmospheric Administration/ National Weather Service Telecommunication Gateway System (Legacy, Replacement, and Critical Infrastructure Protection); Reported cost: $218.27 million. Agency: Commerce; Investment name: Next Generation Weather Radar System Product Improvement; Reported cost: $148.31 million. Agency: Commerce; Investment name: Weather Radio Improvement Project; Reported cost: $23.40 million. Agency: DHS; Reported cost: $4.868 billion. Agency: DHS; Investment name: Customs and Border Protection Infrastructure; Reported cost: $3.712 billion. Agency: DHS; Investment name: Immigration and Customs Enforcement--Electronic Health Record System; Reported cost: $21.43 million. Agency: DHS; Investment name: Immigration and Customs Enforcement--Student and Exchange Visitor Information System; Reported cost: $255.30 million. Agency: DHS; Investment name: DHS Infrastructure Transformation Program; Reported cost: $824.67 million. Agency: DHS; Investment name: National Protection and Programs Directorate - Federal Protective Service Risk Assessment and Management Program; Reported cost: $54.51 million. Agency: HHS; Reported cost: $334.82 million. Agency: HHS; Investment name: Enterprise Human Resources and Personnel; Reported cost: $148.43 million. Agency: HHS; Investment name: Food and Drug Administration Office of Regulatory Affairs Mission Accomplishments and Regulatory Compliance Services; Reported cost: $186.39 million. Investment Total: Reported cost: $7.784 billion[A]. Source: GAO analysis of agency Exhibit 300 data. Note: [A] Due to rounding, subtotals may not add up to totals. [End of table] Similar to the OMB-led TechStats, the majority of the agency-led TechStats on major investments were on investments that were considered to be a medium-to high-risk investment. Specifically, 16 of the 21 major investments that agencies chose for a TechStat had either a medium- or high-risk rating on the Dashboard. Five investments that were chosen received a moderately low-risk rating on the Dashboard. Table 7 provides the details. Table 7: CIO Ratings for Major Investments on Which Agencies Held a TechStat: Agency: Agriculture; Investment name: Automated Timber Sales Accounting; Date of first TechStat: 3/26/2013; CIO rating at time of first TechStat: Moderately high risk. Agency: Agriculture; Investment name: Comprehensive Loan Program; Date of first TechStat: 3/24/2011; CIO rating at time of first TechStat: Moderately low risk. Agency: Agriculture; Investment name: Emerging Information Technology Architecture; Date of first TechStat: 10/31/2012; CIO rating at time of first TechStat: Medium risk. Agency: Agriculture; Investment name: Farm Program Modernization; Date of first TechStat: 11/28/2012; CIO rating at time of first TechStat: Medium risk. Agency: Agriculture; Investment name: Food Safety and Inspection Service Public Health Information Consolidation Projects (now called Public Health Information System); Date of first TechStat: 7/28/2011; CIO rating at time of first TechStat: Medium risk. Agency: Agriculture; Investment name: Web-based Supply Chain Management; Date of first TechStat: 1/14/2013; CIO rating at time of first TechStat: Medium risk. Agency: Commerce; Investment name: Bureau of Industry and Security Commerce Exporter Support System; Date of first TechStat: 8/8/2012; CIO rating at time of first TechStat: Moderately high risk. Agency: Commerce; Investment name: Census - Data Access and Dissemination System; Date of first TechStat: 8/3/2011; CIO rating at time of first TechStat: Medium risk. Agency: Commerce; Investment name: Census IT Infrastructure; Date of first TechStat: 6/28/2012; CIO rating at time of first TechStat: Medium risk. Agency: Commerce; Investment name: Export Control and Automated Support System 2000+; Date of first TechStat: 6/20/2011; CIO rating at time of first TechStat: Moderately low risk. Agency: Commerce; Investment name: National Weather Service Telecommunication Gateway System (Legacy, Replacement, and Critical Infrastructure Protection); Date of first TechStat: 7/8/2011; CIO rating at time of first TechStat: Moderately high risk. Agency: Commerce; Investment name: Next Generation Weather Radar System Product Improvement; Date of first TechStat: 3/25/2011; CIO rating at time of first TechStat: Medium risk. Agency: Commerce; Investment name: Radio Spectrum Management--Federal Spectrum Management System; Date of first TechStat: 10/9/2012; CIO rating at time of first TechStat: Moderately low risk. Agency: Commerce; Investment name: Weather Radio Improvement Project; Date of first TechStat: 6/8/2011; CIO rating at time of first TechStat: Moderately high risk. Agency: DHS; Investment name: Customs and Border Protection Infrastructure; Date of first TechStat: 3/8/2012; CIO rating at time of first TechStat: Medium risk. Agency: DHS; Investment name: Immigration and Customs Enforcement--Electronic Health Record System; Date of first TechStat: 11/9/2011; CIO rating at time of first TechStat: Moderately low risk. Agency: DHS; Investment name: Immigration and Customs Enforcement--Student and Exchange Visitor Information System; Date of first TechStat: 3/30/2011; CIO rating at time of first TechStat: Medium risk. Agency: DHS; Investment name: DHS Infrastructure Transformation Program; Date of first TechStat: 8/12/2011; CIO rating at time of first TechStat: Medium risk. Agency: DHS; Investment name: National Protection and Programs Directorate--Federal Protective Service Risk Assessment and Management Program; Date of first TechStat: 6/21/2012; CIO rating at time of first TechStat: Moderately high risk. Agency: HHS; Investment name: Enterprise Human Resources and Personnel; Date of first TechStat: 4/27/2011; CIO rating at time of first TechStat: Medium risk. Agency: HHS; Investment name: Food and Drug Administration Office of Regulatory Affairs Mission Accomplishments and Regulatory Compliance Services; Date of first TechStat: 12/8/2011; CIO rating at time of first TechStat: Moderately low risk. Source: GAO analysis of IT Dashboard and agency data. [End of table] OMB and Agencies Have Assessed Relatively Few At-Risk IT Investments: While the TechStat process was developed to address at-risk investments, the number of TechStats that OMB and selected agencies performed to date on medium-or high-risk investments represents a small percentage of the current number of IT investments with a medium- or high-risk CIO rating. Specifically, the OMB-led TechStats represent roughly 18.5 percent of the investments across the government that have a medium-or high-risk CIO rating. For the four selected agencies, the number of TechStats represents about 33 percent of the investments that have a medium-or high-risk CIO rating. Results within agencies vary from 12.5 to 58.3 percent. Table 8 provides the details on the four agencies' at-risk investments and the number that have undergone OMB-led or agency-led TechStats. Table 8: Number of TechStats as a Percentage of Medium-or High-Risk Investments, as of May 2013: OMB-led: Governmentwide; Number of at-risk IT investments that have undergone a TechStat: 30; Number of at-risk IT investments: 162; Percentage: 18.5%. Agency-led or OMB-led: Agriculture; Number of at-risk IT investments that have undergone a TechStat: 6; Number of at-risk IT investments: 15; Percentage: 40.0%. Commerce; Number of at-risk IT investments that have undergone a TechStat: 7; Number of at-risk IT investments: 12; Percentage: 58.3%. DHS; Number of at-risk IT investments that have undergone a TechStat: 8; Number of at-risk IT investments: 26; Percentage: 30.8%. HHS; Number of at-risk IT investments that have undergone a TechStat: 2; Number of at-risk IT investments: 16; Percentage: 12.5%. Agency average: Percentage: 33.3%. Source: GAO analysis of IT Dashboard data as of August 2012. [End of table] Focusing specifically on the riskiest investments (those with a moderately high-or high-risk rating), millions of dollars are being spent on investments that have not yet undergone a TechStat. As of May 2013, the IT Dashboard listed 28 major investments as having a high-or moderately high-risk CIO rating. Thirteen of these investments have not undergone a TechStat.[Footnote 30] Since these investments total over $172 million in fiscal year 2013 spending, there is potential for savings if these high-risk investments are turned around. Table 9 provides details on the high-risk investments that have not been the subject of a TechStat. Table 9: High-Risk Investments that Have Not Undergone an OMB-led or Agency-led TechStat, as of May 2013: Agency: Department of Education; Investment: Cyber Security Operations; Total FY13 spending: $7.75 million. Agency: Department of Education; Investment: Enterprise Business Collaboration; Total FY13 spending: $2.25 million. Agency: Department of Education; Investment: Integrated Data Collection and Performance Reporting; Total FY13 spending: $4.49 million. Agency: Department of Education; Investment: Security Authorization; Total FY13 spending: $3.80 million. Agency: Department of Education; Investment: System Security Authorizations; Total FY13 spending: $4.07 million. Agency: Department of Health and Human Services; Investment: Centers for Medicare and Medicaid Services Center for Consumer Information and Insurance Oversight IT Infrastructure; Total FY13 spending: $32.45 million. Agency: Department of Health and Human Services; Investment: Unified Financial Management System: Modernization Program; Total FY13 spending: $37.59 million. Agency: Department of the Interior; Investment: National Park Service - National Fee Collection Point of Sale System; Total FY13 spending: $0.90 million. Agency: Department of Veterans Affairs; Investment: Benefits Legacy; Total FY13 spending: $36.36 million. Agency: Department of Veterans Affairs; Investment: Enterprise Telephony Strategy; Total FY13 spending: $0.0. Agency: DHS; Investment: Federal Emergency Management Agency Infrastructure; Total FY13 spending: $36.75 million. Agency: DHS; Investment: Federal Protective Service Tac Com Equipment and Support; Total FY13 spending: $3.91 million. Agency: National Archives and Records Administration; Investment: Description and Authority Services; Total FY13 spending: $1.91 million. Agency: Total; Total FY13 spending: $172.23 million. Source: GAO analysis of IT Dashboard data. [End of table] DHS officials noted that they plan to hold TechStat sessions on both of their high-risk investments in June 2013. HHS officials stated that the department plans to hold TechStats in the future, but has not established schedules for their high-risk investments. In total, there are 11 high-risk investments that do not have a scheduled TechStat or other management review. OMB and agency officials stated that they are continuing to monitor selected investments to determine whether to hold a TechStat. Also, two agencies' officials explained that just because an IT investment has a high-risk CIO rating does not necessarily mean that the investment is troubled. For example, Commerce officials stated that they list some investments as high risk while they wait for program documentation. However, in its guidance on the IT Dashboard, OMB notes that it expects that CIO ratings should be maintained as an accurate representation of the state of an IT investment. Agency CIOs are able to immediately adjust risk levels if they determined that the investment no longer merits a high-risk rating. Moreover, missing documentation could be a minor issue as Commerce suggests, or in the case of requirements documents or test plans, it could be a significant indicator of a troubled program. It is the CIO's responsibility to determine which is the case and to adjust their ratings accordingly. Until agencies and OMB develop plans and schedules to address their at-risk projects, the investments will likely remain at risk. Selected Agencies are Generally Conducting TechStats in Accordance with OMB Guidance, but Areas for Improvement Exist: The four selected agencies are generally conducting TechStats in accordance with OMB guidance. In 2011, when OMB decided to move beyond conducting its own TechStats and to have agencies conduct them too, OMB provided agencies guidance through the IT Reform Plan, a memorandum, and the TechStat Toolkit, which is available on the CIO Council's website.[Footnote 31] These documents include 15 key requirements, including when TechStats should be implemented by the agencies, what participants should be included, how at-risk investments should be chosen, and how outcomes should be tracked and reported. The requirements can be grouped into four broad categories: scope, governance, process, and outcomes. Table 10 shows key requirements for TechStat reviews. Table 10: Key Requirements for TechStats: Category: Scope; Agency TechStat requirements: * Conduct TechStats at the department level by March 2011; * Conduct TechStats at the bureau level by June 2012; * Conduct TechStats at a more granular level than major IT investments, including on subsets of major investments (called projects) and on non-major investments. Category: Governance; Agency TechStat requirements: * Ensure that sessions are led by CIO; * Embrace the TechStat model within existing capital planning and investment control procedures and embed TechStat accountability within its existing governance structure; * Ensure that the standard invitation to a TechStat review includes the TechStat team, Investment Manager, Business Owner, and Investment Review Board members; * Ensure that the majority of the preparation for a TechStat review is performed by the TechStat team and CIO with support from Investment Managers and Business Owners. Category: Process; Agency TechStat requirements: * Institute training programs on the TechStat process; * Identify at-risk programs for TechStat sessions using sources including (but not limited to) agency performance management system, federal IT Dashboard (CIO rating, cost and schedule rating), exhibit 300 analysis (past and present), rebaseline reviews and other project documentation, GAO and Inspector General reports, congressional testimony, news media, and human intelligence; * Document action steps, deadlines, and responsibilities identified during the TechStat in a memorandum to the investment; * Ensure that the TechStat action items and recovery plans are actively managed and monitored by the TechStat team; * Track all action items in a consolidated repository. Category: Outcomes; Agency TechStat requirements: * Track each investment reviewed in a TechStat into one of the following outcomes: proceed with minimal intervention, accelerated delivery, budget reduction, or termination; * Share outcomes with OMB as part of lessons learned; * Publish TechStat results in an internal collaborative tool for primary stakeholders, including the status of action items, investment disposition decisions, and best practices and success stories identified. Source: GAO analysis of OMB data. [End of table] The four selected agencies implemented most of OMB's 15 key requirements. Specifically, DHS implemented all 15 requirements, Commerce fully implemented 14 requirements and partially implemented 1 requirement, HHS fully implemented 11 of the requirements and partially implemented 4 requirements, and Agriculture fully implemented 10 of the requirements, partially implemented 3 requirements, and did not implement 2 requirements. All four agencies fully implemented eight of OMB's requirements: (1) the appropriate people are invited to the TechStat meetings, (2) sessions are led by the CIO, (3) the preparation for the TechStat is performed by the TechStat team and the CIO, (4) investments selected by TechStats are done so by the prescribed criteria, (5) all action items are tracked in a consolidated repository, (6) each investment is tracked into an outcome, (7) outcomes are shared with OMB, and (8) results are published in an collaborative tool. The requirement with the least implementation involves documenting action items, deadlines, and responsible parties in a memorandum following each TechStat. Table 11 provides details regarding the agencies' implementation of OMB guidance on TechStats. Table 11: Agencies' Implementation of OMB TechStat Guidance: Scope: Agency TechStat requirements: Conduct TechStats at the department level by March 2011; Agriculture: Agency has fully implemented the requirement; Commerce: Agency has fully implemented the requirement; HHS: Agency has partially implemented the requirement; DHS: Agency has fully implemented the requirement; Description: Agriculture, Commerce, and DHS successfully conducted TechStats at the department level by March 2011. HHS held its first TechStat after the OMB-required deadline. Agency TechStat requirements: Conduct TechStats at the bureau level by June 2012; Agriculture: Agency has partially implemented the requirement; Commerce: Agency has fully implemented the requirement; HHS: Agency has partially implemented the requirement; DHS: Agency has fully implemented the requirement; Description: Commerce and DHS bureaus conducted TechStats by June 2012. HHS bureaus conducted TechStats after the OMB deadline. Agriculture provided selected bureaus training on how to conduct TechStats in October 2012; however, it has not yet instructed its bureaus to hold TechStats. Agency TechStat requirements: Governance: Conduct TechStats at a more granular level than major IT investments, including on subsets of major investments (called projects) and on non-major investments; Agriculture: Governance: Agency has not implemented the requirement; Commerce: Governance: Agency has fully implemented the requirement; HHS: Governance: Agency has fully implemented the requirement; DHS: Governance: Agency has fully implemented the requirement; Description: Governance: While Commerce, HHS, and DHS have conducted TechStats on non-major IT investments, Agriculture has not. The agency plans to instruct its components to hold TechStats on non-major investments following the review of all of the major investments. Governance: Agency TechStat requirements: Ensure that sessions are led by CIO; Agriculture: Agency has fully implemented the requirement; Commerce: Agency has fully implemented the requirement; HHS: Agency has fully implemented the requirement; DHS: Agency has fully implemented the requirement; Description: All four agencies ensured that TechStat sessions are led by the CIO, or the bureau CIO, as appropriate. Agency TechStat requirements: Embrace TechStat model within existing capital planning and investment control procedures and embed TechStat accountability within its existing governance structure; Agriculture: Agency has not implemented the requirement; Commerce: Agency has fully implemented the requirement; HHS: Agency has fully implemented the requirement; DHS: Agency has fully implemented the requirement; Description: Commerce, DHS, and HHS have embraced the TechStat model within the existing capital planning and investment control procedures and embedded accountability within its existing governance structure. Agriculture's integrated governance structure does not include TechStats. Agency officials stated that these documents are in the process of being reviewed and approved. Agency TechStat requirements: Ensure that the standard invitation to a TechStat review includes the TechStat team, Investment Manager, Business Owner, and Investment Review Board members; Agriculture: Agency has fully implemented the requirement; Commerce: Agency has fully implemented the requirement; HHS: Agency has fully implemented the requirement; DHS: Agency has fully implemented the requirement; Description: All four agencies ensured that the invitation to a TechStat includes the appropriate people, as prescribed by the guidance. Agency TechStat requirements: Process: Ensure that the majority of the preparation for a TechStat review is performed by the TechStat team and CIO with support from Investment Managers and Business Owners; Agriculture: Process: Agency has fully implemented the requirement; Commerce: Process: Agency has fully implemented the requirement; HHS: Process: Agency has fully implemented the requirement; DHS: Process: Agency has fully implemented the requirement; Description: Process: All four agencies ensured that the majority of the preparation for a TechStat is performed by the TechStat team and CIO (or bureau CIO, as appropriate) with support from Investment Managers and Business Owners. Process: Agency TechStat requirements: Institute training programs on the TechStat process; Agriculture: Agency has partially implemented the requirement; Commerce: Agency has fully implemented the requirement; HHS: Agency has fully implemented the requirement; DHS: Agency has fully implemented the requirement; Description: Commerce, HHS, and DHS have instituted training programs on the TechStat process. Agriculture provided training in October 2012, but several bureaus did not take the training. The agency plans to conduct the training again in April 2013. Agency TechStat requirements: Identify programs for TechStat sessions using sources including (but not limited to) agency performance management system, federal IT Dashboard (CIO rating, cost and schedule rating), exhibit 300 analysis (past and present), rebaseline reviews and other project documentation, GAO and inspector general reports, congressional testimony, news media, and human intelligence; Agriculture: Agency has fully implemented the requirement; Commerce: Agency has fully implemented the requirement; HHS: Agency has fully implemented the requirement; DHS: Agency has fully implemented the requirement; Description: All four agencies used sources that include those suggested by OMB to identify programs for TechStat sessions. Agency TechStat requirements: Document action steps, deadlines, and responsibilities in a memorandum following each TechStat; Agriculture: Agency has partially implemented the requirement; Commerce: Agency has partially implemented the requirement; HHS: Agency has partially implemented the requirement; DHS: Agency has fully implemented the requirement; Description: DHS consistently created memorandums that include action steps, deadlines, and responsibilities. Agriculture and Commerce consistently created memorandums, but did not consistently include all three required elements. HHS did not consistently create memorandums to the investment. For example, there was no memorandum created for the bureau-led TechStat on the Food and Drug Administration Office of Regulatory Affairs Mission Accomplishments and Regulatory Compliance Services. Further, in another case, action item lists did not list responsible parties. Agency TechStat requirements: Ensure that the TechStat action items and recovery plans are actively managed and monitored by the TechStat team; Agriculture: Agency has fully implemented the requirement; Commerce: Agency has fully implemented the requirement; HHS: Agency has partially implemented the requirement; DHS: Agency has fully implemented the requirement; Description: Agriculture, Commerce, and DHS consistently ensured that TechStat action items are actively managed and monitored. HHS partially fulfills this requirement. Specifically, HHS monitored the status of six action items, but did not track two action items from 2011 to closure. Agency TechStat requirements: Outcomes: Track all action items in a consolidated repository; Agriculture: Outcomes: Agency has fully implemented the requirement; Commerce: Outcomes: Agency has fully implemented the requirement; HHS: Outcomes: Agency has fully implemented the requirement; DHS: Outcomes: Agency has fully implemented the requirement; Description: Outcomes: All four agencies tracked action items in a repository. Outcomes: Agency TechStat requirements: Track each investment reviewed in a TechStat into one of the following outcomes: proceed with minimal intervention, accelerated delivery, budget reduction, or termination; Agriculture: Agency has fully implemented the requirement; Commerce: Agency has fully implemented the requirement; HHS: Agency has fully implemented the requirement; DHS: Agency has fully implemented the requirement; Description: All four agencies tracked investments into the prescribed categories. Agency TechStat requirements: Share outcomes with OMB as part of lessons learned; Agriculture: Agency has fully implemented the requirement; Commerce: Agency has fully implemented the requirement; HHS: Agency has fully implemented the requirement; DHS: Agency has fully implemented the requirement; Description: All four agencies consistently shared outcomes with OMB. Agency TechStat requirements: Agency TechStat requirements: Publish TechStat results in an internal collaborative tool for primary stakeholders, including the status of action items, investment disposition decisions, and best practices and success stories identified; Agriculture: Agriculture: Agency has fully implemented the requirement; Commerce: Commerce: Agency has fully implemented the requirement; HHS: HHS: Agency has fully implemented the requirement; DHS: DHS: Agency has fully implemented the requirement; Description: Description: All four agencies published TechStat results in an internal collaborative tool. Source: GAO analysis of OMB and agency data. [End of table] While the four selected agencies have largely implemented OMB's guidance on conducting TechStats, three agencies have selected areas where they can improve. For example, Agriculture created memorandums following a TechStat, but did not consistently include responsible parties; Commerce created memorandums, but did not include deadlines; and HHS did not always create memorandums or monitor all of its action items to closure. Agency officials noted several reasons for not fully implementing OMB's guidance. Specifically, Agriculture officials noted that they are updating their capital planning guidance to include TechStat reviews, but that this guidance is in the process of being reviewed and approved. Also, Commerce and HHS officials noted that OMB gave agencies flexibility in exactly how to implement their guidance. While OMB did provide agencies flexibility in selecting investments for TechStat reviews and conducting those reviews, the requirements are clearly delineated in OMB instructions and training. Fully implementing OMB's TechStat guidance could better position the agencies to realize the benefits of the TechStat initiative--including strengthening overall IT governance and oversight, and proactively identifying and resolving problems before investments experience delays or cost overruns. OMB and Agencies Reported Positive Results and Cost Savings from Holding TechStats, but Are Not Ensuring Results Are Valid: The IT Reform Plan and related OMB guidance instructed agencies to track and report on the outcomes of their TechStat sessions (including improved governance, accelerated deliveries, and terminated projects), and on any associated cost implications (such as cost savings or cost avoidances). OMB and the four agencies we reviewed have tracked and reported positive results from TechStats, with most resulting in improved governance or accelerated deliveries. OMB also reported that federal agencies achieved over $3 billion in cost savings or avoidances as a result of the OMB-led TechStats in 2010 and $900 million from agency- led TechStats in 2011. Using a different calculation formula, OMB also reported that TechStats resulted in $63.5 million in cost implications in 2012.[Footnote 32] However, we were unable to validate the reported outcomes and associated savings because OMB did not provide supporting artifacts or demonstrate the steps that OMB analysts took to verify the agencies' data. Without documentation or an explanation of its method in validating agencies' reported results and cost savings, it will be difficult for OMB to provide a sufficient level of confidence to Congress and the public that the information it has presented is credible. OMB and Agencies Reported Positive Results from TechStats: Both OMB and agencies reported achieving positive results from their respective TechStat sessions. Specifically, in 2011 OMB staff reported achieving a variety of positive outcomes from the OMB-led TechStat sessions, including 11 investments being reduced in scope, four investments that were canceled, and multiple investments with accelerated program delivery. We also identified four investments that were the focus of an OMB-led TechStat that were subsequently terminated and two other investments that were split into multiple smaller investments to improve governance and accelerate the delivery of discrete capabilities.[Footnote 33] Further, we previously found that the June 2010 TechStat on the National Archives and Records Administration's Electronic Records Archives investment resulted in six corrective actions, including halting fiscal year 2012 development funding pending the completion of a strategic plan.[Footnote 34] We estimated that the program would likely experience cost overruns of between $205 and $405 million if the agency completed the program as originally designed. Seeking to improve the reporting of outcomes, OMB instructed federal agencies to track the results of each agency-led TechStat session into one of six outcomes: accelerated delivery, improved governance, reduced scope, eliminated duplication, halted, or terminated. In December 2011, OMB reported a summary of the outcomes of 294 agency- led TechStats across the federal government.[Footnote 35] Specifically, OMB reported the following outcomes: * 49 percent resulted in accelerated delivery, * 42 percent resulted in improved governance, * 3 percent did not report results, * 2 percent resulted in terminations, * 1 percent resulted in reducing the scope, * 1 percent eliminated duplication, and: * 1 percent halted the investment. In addition, the four agencies in our review reported on the results of their agency-led TechStats, with the majority resulting in improved governance. Specifically, out of 36 TechStat reviews, 28 resulted in improved governance, 4 in accelerated delivery, 2 in terminations, and 2 in a reduced scope. Table 12 provides a summary of reported results, by agency. Table 12: Outcomes of Agency-Led TechStat Reviews for Four Selected Agencies (as of March 2013): Accelerated delivery: Commerce: 0; DHS: 4; HHS: 0; Agriculture: 0; Total: 4 (11%). Improved governance: Commerce: 14; DHS: 3; HHS: 5; Agriculture: 6; Total: 28 (78%). Reduced scope: Commerce: 0; DHS: 0; HHS: 2; Agriculture: 0; Total: 2 (6%). Terminated: Commerce: 1; DHS: 1; HHS: 0; Agriculture: 0; Total: 2 (6%). Total: Commerce: 15; DHS: 8; HHS: 7; Agriculture: 6; Total: 36. Source: GAO analysis of agency data. [End of table] OMB Reported Significant Cost Benefits from Holding TechStats: In conjunction with the reported outcomes of agency-led TechStats, OMB instructed agencies to provide information on the cost implications of the outcomes. These implications could include cost savings (a reduction in actual expenditures) or cost avoidances (an action taken immediately that will reduce costs in the future). In addition, OMB's guidance on performance reporting notes that performance data should be appropriately accurate and reliable for their intended use. [Footnote 36] This guidance further describes verification and validation techniques that OMB encourages agencies to use in internal assessments, including ensuring that supporting documentation is maintained and readily available, data are verified as appropriate to the needed level of accuracy, and data limitations are explained and documented. OMB has reported cost savings and avoidances from OMB-and agency-led TechStat reviews. Specifically, OMB has reported[Footnote 37] that federal agencies achieved about $3 billion in cost savings or avoidances as a result of the OMB-led TechStats held in 2010 and 2011, and $900 million from agency-led TechStats in 2011. See figure 7 for the reported cost implications for both OMB-led and agency-led TechStats as of November 2011. Figure 7: OMB's Reported Cost Savings and Avoidances, as of November 2011: [Refer to PDF for image: stacked vertical bar graph] On track: OMB-led: $18 million; Agency-led: $0. Halted: OMB-led: $0; Agency-led: $20.8 million. Improved governance: OMB-led: $153.9 million; Agency-led: $153.9 million. Accelerated delivery: OMB-led: $983.3 million; Agency-led: $30 million. Terminated, eliminated, or duplication: OMB-led: $759.8 million; Agency-led: $575.5 million. Reduced scope or budget reduction: OMB-led: $1.330 billion; Agency-led: $151.5 million. Source: GAO analysis of OMB data. [End of figure] In addition, OMB reported that the cost savings and avoidances for the agency-led TechStat reviews, as of November 2011, came from 10 of the 27 agencies under their purview. Department of Transportation had the most cost implications, with $510 million. See figure 8 for a depiction of the cost implications by agency. Figure 8: Cost Savings and Avoidances for Agency-Led TechStat Sessions, as of November 2011: [Refer to PDF for image: vertical bar graph] Agency: DOD; Cost Implications: $150.7 million. Agency: DOT; Cost Implications: $510 million. Agency: Energy; Cost Implications: $5 million. Agency: Interior; Cost Implications: $50.5 million. Agency: NRC; Cost Implications: $0.8 million. Agency: NSF; Cost Implications: $0.2 million. Agency: OPM; Cost Implications: $153.3 million. Agency: SSA; Cost Implications: $19.2 million. Agency: Treasury; Cost Implications: $0.1 million. Agency: VA; Cost Implications: $11.9 million. Source: GAO analysis of CIO Council data. Note: Key to agency names: DOD = Department of Defense; DOT = Department of Transportation; Energy = Department of Energy; Interior = Department of the Interior; NRC = Nuclear Regulatory Commission; NSF = National Science Foundation; OPM = Office of Personnel Management; SSA = Social Security Administration; Treasury = Department of the Treasury; and; VA = Department of Veterans Affairs. [End of figure] Two of the agencies in our review--DHS and HHS--reported $23 million in cost implications from three agency-led TechStats: * DHS reported a $14 million cost avoidance after deciding to decommission one part of the Federal Protective Service's Risk Assessment and Management Program; * HHS reported saving $8.2 million by reengineering redundant business processes supporting the Food and Drug Administration's Mission Accomplishment and Regulatory Compliance Service project; and: * HHS reported $800,000 in cost implications associated with improving governance and reducing the scope of the One Stop Service Solution project (now called GovZone). More recently, OMB has been reporting cost savings from agency-led TechStat reviews in quarterly reports to Congress.[Footnote 38] From December 2011 through December 2012, OMB identified a total of $63.5 million in cost implications from agency-led TechStats. OMB staff stated that they are calculating the cost savings in these quarterly reports differently than their prior reports on cost savings, and thus the costs should not be compared.[Footnote 39] OMB and Agencies Are Not Ensuring Reported Outcomes and Cost Implications Are Valid: When collecting data, it is important to have assurance that the data are accurate. Best practices in implementing the Government Performance and Results Act of 1993 emphasize the need for agencies, when providing information, to explain the procedures used to verify or validate their data.[Footnote 40] Specifically, agencies should ensure that reported data are sufficiently complete, accurate, and consistent, and also identify any significant data limitations. Explaining the limitations of the information can provide a context for understanding and assessing the challenges agencies face in gathering, processing, and analyzing needed data. Such a presentation of data limitation can also help identify the actions needed to improve the agency's ability to measure its performance. More recently, we have reiterated the importance of providing OMB with complete and accurate data.[Footnote 41] DHS and HHS, the two agencies in our review that reported cost savings and avoidances, generally perform analyses to establish these estimates. DHS provided supporting documentation for how the cost implications for its investment were calculated and HHS provided supporting documentation for its larger investment. We were unable to validate the cost savings for HHS's smaller investment because the agency was unable to provide supporting documentation. From a governmentwide perspective, OMB staff explained that they review agencies' cost implication data for completeness and quality. However, we were unable to validate OMB's reported outcomes and almost $4 billion in cost implications because OMB did not provide documentation on any steps that it or the agencies took to ensure the validity of the agencies' outcome and cost data. For example, OMB staff did not provide memorandums documenting action items from the OMB-led TechStats, the outcomes identified for OMB-and agency-led TechStats, documentation identifying which investments resulted in cost savings, documentation demonstrating the methodology used to calculate the cost savings, or a summary of steps the agencies took to validate reported cost savings. Moreover, OMB does not require agencies to report on what steps they took to verify their reported outcomes and cost savings. By not requiring agencies to report on their efforts to validate reported outcomes and cost savings, it is not evident that OMB is following its own guidance for ensuring that performance data are reliable and accurate. Moreover, OMB is not providing reasonable assurance to Congress and the public that the information it has presented is credible. One entity that was formed to assist OMB in its oversight of the TechStat results was the CIO Council's subcommittee on IT Governance and TechStats, but according to OMB and agency officials this subcommittee has already been dissolved. The CIO Council's Management Best Practices Committee formed the subcommittee to report on the outcomes and lessons learned from implementing the TechStat process. This subcommittee was to play a key role in continuing to mature the TechStat process. However, in 2012, the CIO Council dissolved the subcommittee because the committee chairs determined that it was no longer needed. Conclusions: Consistent with government and industry best practices for overseeing IT investments, TechStat sessions hold value by focusing management attention on troubled projects and establishing clear action items to turn the projects around or terminate them. While OMB and agencies are reporting positive results from holding TechStat sessions, neither are doing enough to ensure that at-risk investments are undergoing review, sound processes are in place, and reported results are valid. Specifically, agencies are reviewing only about a third of their at- risk IT investments. Until OMB and agencies develop plans and schedules for addressing these at-risk investments, the investments will likely remain at risk. The four agencies we reviewed had implemented most of the OMB-required TechStat processes, but three had shortfalls in selected processes. For example, Agriculture had not yet incorporated TechStats in its investment management processes, Commerce had not consistently included deadlines for action items in its TechStat memoranda, and HHS had not consistently created action item memoranda following TechStats or tracked its action items to completion. Addressing these shortfalls could better position these agencies to realize the full benefits that TechStats offer. OMB regularly reports on cost savings associated with TechStats, but it has not taken basic steps to provide reasonable assurance to Congress and the public that these data are valid. Until OMB requires agencies to report on what they did to validate cost savings data and shares this information, neither Congress nor the public can be assured that TechStats are as effective as reported. While the CIO Council recently dissolved its subcommittee responsible for reviewing TechStats, the council is one entity that is uniquely positioned to assist OMB in its oversight of the TechStat results. Recommendations for Executive Action: To ensure that TechStat sessions are having the appropriate impact in the oversight of underperforming projects, we are making three recommendations to OMB. Specifically, we recommend that the Director of the Office of Management and Budget direct the Federal Chief Information Officer to: * require agencies to conduct TechStats for each IT investment rated with a moderately high-or high-risk CIO rating on the IT Dashboard, unless there is a clear reason for not doing so; * require agencies to report to OMB on efforts to validate the outcomes, cost savings, and cost avoidances resulting from TechStat: * sessions; this information should be summarized when OMB reports on governmentwide outcomes; and: * direct the Federal CIO Council to track the outcome of TechStat sessions and to support OMB's efforts to validate the resulting cost savings it reports to Congress. In addition, we are making a recommendation to the Secretaries of Agriculture and Commerce to: * address the weaknesses in agency-and bureau-led TechStat processes and management outlined in this report. We are also making two recommendations to the Secretary of Health and Human Services to: * establish a plan and schedule for addressing each IT investment rated with a moderately high-or high-risk CIO rating on the IT Dashboard; such a plan could include conducting a TechStat session, and: * address the weaknesses in agency-and bureau-led TechStat processes and management outlined in this report. Agency Comments and Our Evaluation: We requested comments from OMB, Agriculture, Commerce, DHS, and HHS on a draft of our report. In that draft report, we had made recommendations to the Secretaries of Agriculture, Commerce, and DHS to establish a plan and schedule for addressing their high-risk investments that had not yet undergone TechStat reviews. However, these agencies completed these steps prior to the issuance of this report. Therefore, we removed the applicable recommendations. OMB and three agencies provided comments on our draft report. OMB generally agreed with our recommendations; Commerce agreed with our recommendation; and Agriculture and HHS did not agree or disagree with our recommendations. DHS declined to provide comments. Each agency's comments are discussed in more detail below: * In comments provided via e-mail, staff from OMB's Office of E- Government and Information Technology generally concurred with our findings and recommendations and stated that OMB and the agencies are currently taking appropriate steps to meet the recommendations. OMB provided additional details as follows: - In commenting on our finding that it did not ensure the validity of the outcomes and savings it reported, OMB noted that it is confident in the agency validation of cost-savings and avoidances through the TechStat sessions, and that it was speculative to conclude that the reported cost savings data were not valid because we could not actually assess these figures. OMB did not provide supporting documentation for the cost savings calculations and explained that the process used to validate the cost figures was deliberative. However, part of a sound process for ensuring validity involves documenting the procedures used to verify or validate the data; as such, we were unable to validate OMB's reported outcomes and almost $4 billion in cost implications from 2010 and 2011[Footnote 42] because OMB did not provide documentation on any steps that it or the agencies took to ensure the validity of the agencies' outcome and cost data. For example, OMB staff did not provide memorandums documenting action items from the OMB-led TechStats, the outcomes identified for OMB-and agency-led TechStats, documentation identifying which investments resulted in cost savings, documentation demonstrating the methodology used to calculate the cost savings, or a summary of steps the agencies took to validate reported cost savings. Moreover, OMB did not require agencies to report on what steps they took to verify their reported outcomes and cost savings. Without disclosing the steps it took to validate the outcomes and cost savings, OMB has not provided reasonable assurance to Congress and the public that the information it has presented is credible. - OMB also noted that since June 2012, OMB and agencies have validated and transmitted cost-savings and avoidance data associated with TechStat sessions to Congress on a quarterly basis through the Integrated, Efficient, and Effective Use of IT Report. These reports summarize the validation methods OMB and agencies undertake. We acknowledge that these reports summarize methods through which the costs savings and avoidances were documented; however, due to a lack of supporting documentation, we could only validate $22.2 million of the $63.5 million cost savings and avoidances OMB recently reported. * In written comments, the Department of Agriculture's Chief Information Officer partially agreed with our assessment of the agency's TechStat process but did not specify the part of the assessment with which she disagreed. The department stated that it would continue to conduct periodic reviews of its IT investments to ensure a review of all major IT investments. In addition, the department stated that IT Dashboard ratings alone are not true indicators that an investment is poorly performing or underperforming and that it takes into consideration other factors, such as earned value management data, when determining an investment's performance. We agree that the CIO rating should reflect the CIO's assessment of the risk and should use many factors in determining the rating. The department's written comments are provided in appendix II. * In written comments, the Department of Commerce's Acting Secretary concurred with our recommendation and stated that the department is in full compliance with the recommendation. The department plans to ensure that all TechStat memos have a specific point of contact and a specific due date. The department's written comments are provided in appendix III. * In comments provided via e-mail, a management analyst within the Department of Health and Human Services' Office of the Assistant Secretary for Legislation stated that the department had no general comments. The department provided technical comments, which we have incorporated as appropriate. In its technical comments, HHS commented on our finding that the agency did not satisfy OMB's requirement to hold its first TechStat session by March 2011. The HHS officials stated that the agency had appropriately initiated the process for holding a TechStat prior to OMB's March 2011 deadline. While we acknowledge that the agency initiated the process prior to the deadline, the actual session was held after the deadline. If you or your staff have any questions on the matters discussed in this report, please contact me at (202) 512-9286 or at pownerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV. Signed by: David A. Powner Director, Information Technology Management Issues: [End of section] Appendix I: Objectives, Scope, and Methodology: Our objectives were to (1) identify key characteristics of TechStat Accountability Sessions (TechStats) conducted by the Office of Management and Budget (OMB) and selected agencies, (2) evaluate whether selected agencies are conducting TechStats in accordance with OMB guidance, and (3) analyze the extent to which reported results from TechStat review sessions are documented, tracked, and validated. In conducting our review, we selected four agencies based on two factors: the number of investments with medium-, moderately high-, and high-risk Chief Information Officer (CIO) ratings on the information technology (IT) Dashboard as of August 2012, and a determination on whether the agency had led its own TechStats in 2011. The four agencies we selected--the Departments of Agriculture (Agriculture), Commerce (Commerce), Health and Human Services (HHS), and Homeland Security (DHS)--had the highest number of medium-, moderately high-, and high-risk investments. In addition, in December 2011, the CIO Council reported that all four agencies had held their own TechStat sessions in 2011.[Footnote 43] We assessed the reliability of the IT Dashboard data and the data on agency-led TechStats by comparing them to agency documents, and found that the data were sufficient for our purpose of selecting agencies for further review. To address the first objective, we obtained OMB's list of the TechStat sessions it had led through April 2013, and downloaded the latest budget data for each of these investments.[Footnote 44] We analyzed, summarized, and--where appropriate--graphically depicted the data. We also analyzed documentation from our four selected agencies about the agency-led TechStat reviews, including (but not limited to) briefing slides, meeting minutes, corrective action plans, and meeting invitations. We downloaded budget data as of September 2012 for each of the investments subject to an agency-led TechStat review at these four agencies, and summarized the data. We also downloaded updated budget data in May 2013 to determine the current number of investments that have yellow and red CIO ratings and have not undergone a TechStat, and summarized budget data for each of these investments. We also interviewed OMB, Agriculture, Commerce, DHS, and HHS officials regarding the TechStat sessions. We assessed the reliability of OMB's and the four agencies' lists of reported TechStat sessions by seeking corroboration for the sessions held at our selected agencies. We determined that OMB's and the four agencies' lists were sufficient for our purposes. To address the second objective, we analyzed OMB's guidance to agencies on conducting TechStat reviews and identified15 key requirements. We then compared these requirements to the selected agencies' documentation, including capital planning and investment control plans and guidance, agency-specific TechStat guidance and training materials, meeting minutes from investment review board meetings, and electronic submissions from the agency to OMB regarding the TechStat sessions. To identify whether agencies were documenting, tracking, and monitoring action items--one of the 15 key requirements-- we conducted a random sample of 10 percent of each agency's action items and sought documentation from each of the agencies to support the status of the selected action items. This sample is not generalizable to the entire population of action items from each agency, but we determined it was sufficiently reliable for our purposes of determining whether the agencies were tracking and monitoring action items. We also interviewed officials from OMB and our four selected agencies about their respective TechStat processes. To address the third objective, we identified OMB's guidance to agencies on reporting the results of agency-led TechStat sessions. We analyzed the selected agencies' and OMB's reported results in each of OMB's outcome categories and attempted to validate OMB's and the agencies' reported cost implications. We also reviewed quarterly reports from OMB to Congress to identify additional cost implication information.[Footnote 45] As noted in the body of this report, we were unable to determine the reliability of the reported cost implications because OMB did not provide artifacts demonstrating how it validated the data or evidence that it obtained agency validation of the data. We conducted this performance audit from September 2012 to June 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: Comments from the Department of Agriculture: USDA: United States Department of Agriculture: Departmental Management: Office of the Chief Information Officer: 1400 Independence Avenue S.W. Washington, DC 20250: May 14, 2013: David A. Powner: Director: Information Technology Management Issues: Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Powner: In response to GAO Draft Report on Information Technology: Additional Executive Review Sessions Needed to Address Troubled Projects, GAO-13- 524. USDA partially agrees with the GAO assessment of the USDA TechStat process. USDA has developed a Department level TechStat schedule to conduct one (1) TechStat session per month to ensure a review of all major IT investments. Since January of this year, USDA has conducted three TechStats and will conduct TechStats through the remainder of 2013. USDA has conducted a total of nine (9) agency led TechStats on IT investments rated red or yellow on the IT Dashboard. USDA conducted two (2) Agency led TechStats in 2011 and three (3) Agency led TechStats in 2012. USDA will continue to conduct periodic reviews of its IT investments to ensure proper oversight and management of taxpayer dollars. The IT Dashboard ratings alone are not true indicators that an investment is poorly performing or underperforming. USDA takes into consideration the IT Dashboard ratings, but also looks at the investment's Earned Value Management data and programmatic milestone data, to determine whether the IT investments are meeting cost, schedule, and performance objectives. Again, thank you for the opportunity to provide these comments. If you have any questions regarding this letter, please contact me at (202) 720-8833. Sincerely, Signed by: Cheryl Cook: Chief Information Officer: [End of section] Appendix III: Comments from the Department of Commerce: United States Department of Commerce: The Secretary of Commerce: Washington, D.C. 20230: May 10, 2013: Mr. David A. Powner: Director, Information Technology Management Issues: United States Government Accountability Office: Washington, DC 20548: Dear Mr. Powner: Thank you for the opportunity to comment on the draft report from the U.S. Government Accountability Office (GAO) entitled Information Technology: Additional Executive Review Sessions Needed to Address Troubled Projects (GA0-13-524). The Department of Commerce takes very seriously the performance of its information technology (IT) portfolio and the proper stewardship of the taxpayer dollars which fund that portfolio. We concur with the report's recommendation that the Department (a) establish a plan and schedule for addressing each IT investment with a moderately high or high risk rating by the Chief Information Officer on the IT Dashboard, and (b) address the weaknesses in agency- and bureau-led TechStat processes and management as outlined in the report. The Department is pleased to announce that we are in full compliance with both recommendations, as outlined in the attached Statement of Actions. The Department of Commerce strives to continuously improve the performance of its IT portfolio and to that end has institutionalized best practices throughout the Department and its bureaus to ensure a healthy, goal-focused IT investment portfolio. If you have any questions, please contact Terri Ware, Director of Programs and Project Management for the Chief Information Officer, at 202-482-3138. Sincerely, Signed by: Rebecca M. Blank: Acting Secretary of Commerce: Enclosure: Department of Commerce's Statement of Actions: GAO Draft Report "Information Technology: Additional Executive Review Sessions Needed to Address Troubled Projects" (GAO-13-524, June 2013): The U.S. General Accountability Office made two recommendations for the Department of Commerce. Recommendation: Establish a plan and schedule for addressing each IT investment rated with a moderately high or high risk CIO rating on the IT Dashboard; such a plan could include conducting a Techstat accountability session. Commerce Response: The Department is pleased to report that we fully comply with this GAO recommendation. Currently, in support of the Department's Balanced Scorecard program, we monitor, track, and report IT investments "rated with a moderately high or high risk CIO rating," that is, those rated "red" by the Chief Information Officer on the Office of Management and Budget's IT Dashboard. Investments rated "red" for 3 or more consecutive months within a 12-month period are scheduled for a TechStat review. Recommendation: Address the weaknesses in agency- and bureau-led TechStat processes and management outlined in this [GAO-13-524] report. Commerce Response: The Department is pleased to report that we fully comply with this GAO recommendation. For all future TechStat memos we will make sure that all memos have actions that have a specific point of contact named and a specific due date for completion. [End of section] Appendix IV: GAO Contact and Staff Acknowledgments: GAO Contact: David A. Powner, (202) 512-9286 or pownerd@gao.gov: Staff Acknowledgments: In addition to the contact named above, individuals making contributions to this report included Colleen Phillips (Assistant Director), Rebecca Eyler, Kate Feild, and Jessica Waselkow. [End of section] Footnotes: [1] GAO, Agencies' Annual Performance Plans under the Results Act: An Assessment Guide to Facilitate Congressional Decisionmaking, [hyperlink, http://www.gao.gov/products/GAO/GGD/AIMD-10.1.18] (Washington, D.C.: February 1998). [2] This system is the Global Combat Support System-Joint Increment 7. [3] This system is the Integrated Terminal Weather System. [4] GAO, Information Technology: Management Improvements Are Essential to VA's Second Effort to Replace Its Outpatient Scheduling System, [hyperlink, http://www.gao.gov/products/GAO-10-579] (Washington, D.C.: May 27, 2010). [5] GAO, Information Technology Reform: Progress Made; More Needs to Be Done to Complete Actions and Measure Results, [hyperlink, http://www.gao.gov/products/GAO-12-745T] (Washington, D.C.: May 24, 2012). [6] The 27 federal agencies are the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, Housing and Urban Development, Justice, Labor, State, the Interior, the Treasury, Transportation, Veterans Affairs; the Environmental Protection Agency; the General Services Administration; the National Aeronautics and Space Administration; National Archives and Records Administration; National Science Foundation; Nuclear Regulatory Commission; Office of Personnel Management; Small Business Administration; Smithsonian Institution; Social Security Administration; U.S. Agency for International Development; and the U.S. Army Corps of Engineers. [7] 44 U.S.C. § 3501 et seq. [8] According to OMB guidance, a major investment is a system or acquisition requiring special management attention because of its importance to the mission or function of the agency, a component of the agency, or another organization; has significant program or policy implications; has high executive visibility; has high development, operating, or maintenance costs; is funded through other than direct appropriations; or is defined as major by the agency's capital planning and investment control process. [9] 40 U.S.C. § 11101 et seq. [10] GAO, Federal Chief Information Officers: Opportunities Exist to Improve Role in Information Technology Management, [hyperlink, http://www.gao.gov/products/GAO-11-634] (Washington, D.C.: Sept. 15, 2011). [11] OMB, Memorandum for Heads of Executive Departments and Agencies, M-11-29 (Washington, D.C.: Aug. 8, 2011). [12] [hyperlink, http://www.itdashboard.gov]. [13] Two different budget submissions, called exhibit 53s and exhibit 300s, provide the data accessible through the IT Dashboard. Exhibit 53s list all of the IT investments and their associated costs within a federal organization. An Exhibit 300, also called the Capital Asset Plan and Business Case, is used to justify resource requests for major IT investments and is intended to enable an agency to demonstrate, to its own management and to OMB, that a major investment is well planned. [14] In February 2002, OMB established the Federal Enterprise Architecture program. According to OMB, it is intended to facilitate governmentwide improvement through cross-agency analysis and identification of duplicative investments, gaps, and opportunities for collaboration, interoperability, and integration within and across agency programs. [15] OMB, 25 Point Implementation Plan to Reform Federal Information Technology Management (Washington, D.C.: Dec. 9, 2010). [16] OMB, M-11-29. [17] GAO, Information Technology Investment Management: A Framework for Assessing and Improving Process Maturity, [hyperlink, http://www.gao.gov/products/GAO-04-394G] (Washington, D.C.: Mar. 2004). [18] For example: GAO, Information Technology Reform: Progress Made; More Needs to Be Done to Complete Actions and Measure Results, [hyperlink, http://www.gao.gov/products/GAO-12-461] (Washington, D.C.: Apr. 26, 2012); Information Technology: OMB Needs to Improve Its Guidance on IT Investments, [hyperlink, http://www.gao.gov/products/GAO-11-826] (Washington, D.C.: Sept. 29, 2011); and Information Technology: Management and Oversight of Projects Totaling Billions of Dollars Need Attention, [hyperlink, http://www.gao.gov/products/GAO-09-624T] (Washington, D.C.: Apr. 28, 2009). [19] GAO, Information Technology: OMB Can Make More Effective Use of Its Investment Reviews, [hyperlink, http://www.gao.gov/products/GAO-05-276] (Washington, D.C.: Apr. 15, 2005). [20] GAO, Information Technology: Agencies and OMB Should Strengthen Processes for Identifying and Overseeing High Risk Projects, [hyperlink, http://www.gao.gov/products/GAO-06-647] (Washington, D.C.: June 15, 2006). [21] GAO, IT Dashboard: Accuracy Has Improved, and Additional Efforts Are Under Way to Better Inform Decision Making, [hyperlink, http://www.gao.gov/products/GAO-12-210] (Washington, D.C.: Nov. 7, 2011); Information Technology: OMB Has Made Improvements to Its Dashboard, but Further Work Is Needed by Agencies and OMB to Ensure Data Accuracy, [hyperlink, http://www.gao.gov/products/GAO-11-262] (Washington, D.C.: Mar. 15, 2011); and Information Technology: OMB's Dashboard Has Increased Transparency and Oversight, but Improvements Needed, [hyperlink, http://www.gao.gov/products/GAO-10-701] (Washington, D.C.: July 16, 2010). [22] [hyperlink, http://www.gao.gov/products/GAO-12-461]. [23] GAO, Data Center Consolidation: Agencies Making Progress on Efforts, but Inventories and Plans Need to Be Completed, [hyperlink, http://www.gao.gov/products/GAO-12-742] (Washington, D.C.: Jul. 19, 2012). [24] GAO, Information Technology Dashboard: Opportunities Exist to Improve Transparency and Oversight of Investment Risk at Select Agencies, [hyperlink, http://www.gao.gov/products/GAO-13-98] (Washington, D.C.: Oct. 16, 2012). [25] Two OMB-led TechStats were on IT investments that have not been identified as major or non-major. [26] Two investments were not categorized into functional areas. [27] Expenditure data were not available on seven major IT investments that were not included in the agencies' recent budget submissions. [28] The budget submissions for three investments did not identify their functional areas. [29] Expenditure data are not available for one investment that was recently eliminated. [30] Ten have had an OMB-led TechStat and five have had an agency-led TechStat. [31] GAO has also identified effective practices for using "Stat" meetings, or data-driven performance reviews, as a performance improvement tool. See GAO, Managing for Results: Data-Driven Performance Reviews Show Promise But Agencies Should Explore How to Involve Other Relevant Agencies, [hyperlink, http://www.gao.gov/products/GAO-13-228] (Washington, D.C.: Feb. 27, 2013). [32] According to OMB staff, the original report identified cost implications based on life-cycle costs of the investment, whereas the quarterly reports identify only near-term cost implications. [33] DHS's Customs and Border Protection - Secure Border Initiative; Interior's Capstone Facility Management System; Interior's Bureau of Ocean Energy Management, Regulation and Enforcement Outer Continental Shelf Connect; and Office of Personnel Management's Retirement Systems Modernization were terminated. Housing and Urban Development's Transformation Initiative and Department of the Interior's Consolidated Infrastructure Automation Telecomm investments were split into smaller investments. [34] GAO, Information Technology: Continued Improvements in Investment Oversight and Management Can Yield Billions in Savings, [hyperlink, http://www.gao.gov/products/GAO-11-511T] (Washington, D.C.: Apr. 12, 2011). [35] CIO Council, Management Best Practices Committee, A Year in Review: Outcomes and Lessons Learned from Implementing Agency-Led TechStat Reviews Across the Federal Government (Washington, D.C.: Dec. 8, 2011). OMB staff told us that these cost implications were calculated over the entire life cycle of the investments. [36] OMB, Circular A-11, Section 260: Annual Performance Reporting (Washington, D.C.: August 2012). [37] For example, the Federal CIO, OMB, and the CIO Council reported these cost savings and avoidances in 25 Point Implementation Plan to Reform Federal Information Technology Management (Washington, D.C.: Dec. 9, 2010); Year in Review: Outcomes and Lessons Learned from Implementing Agency-Led TechStat Reviews Across the Federal Government, (Dec. 8, 2011); Federal Information Technology FY2013 Budget Priorities: "Doing More with Less;" Quarterly Report to Congress: Integrated, Efficient, and Effective Uses of Information Technology (Washington, D.C.: June 25, 2012). [38] Under the Consolidated Appropriations Act, 2012, (P.L. 112-74), the Director of OMB was required to submit quarterly reports identifying the savings achieved by OMB's governmentwide information technology reform efforts with the savings identified by fiscal year, agency, and appropriation. Pub. L. No. 112-74, 125 Stat. 896 (2011). See OMB, Quarterly Report to Congress: Integrated, Efficient, and Effective Uses of Information Technology (Washington, D.C.: June 25, 2012); Quarterly Report to Congress: Integrated, Efficient, and Effective Uses of Information Technology (Washington, D.C.: July 30, 2012); and Quarterly Report to Congress: Integrated, Efficient, and Effective Uses of Information Technology (Washington, D.C.: Jan. 31, 2013). [39] According to OMB staff, the original report identified cost implications based on life-cycle costs of the investment, whereas the quarterly reports identify only near-term cost implications. [40] [hyperlink, http://www.gao.gov/products/GAO/GGD/AIMD-10.1.18]. [41] [hyperlink, http://www.gao.gov/products/GAO-11-262]. [42] For example, the Federal CIO, OMB, and the CIO Council reported these cost savings and avoidances in 25 Point Implementation Plan to Reform Federal Information Technology Management (Washington, D.C.: Dec. 9, 2010); Year in Review: Outcomes and Lessons Learned from Implementing Agency-Led TechStat Reviews Across the Federal Government, (Dec. 8, 2011); Federal Information Technology FY2013 Budget Priorities: "Doing More with Less;" Quarterly Report to Congress: Integrated, Efficient, and Effective Uses of Information Technology (Washington, D.C.: June 25, 2012). [43] CIO Council, Management Best Practices Committee, A Year in Review: Outcomes and Lessons Learned from Implementing Agency-Led TechStat Reviews Across the Federal Government (Washington, D.C.: Dec. 8, 2011). [44] We reviewed exhibit 300 and 53 data linked on the IT Dashboard as of the last update in August 2012. [45] Under the Consolidated Appropriations Act, 2012, OMB is required to report to congressional appropriations committees on cost savings associated with its IT reform initiatives. [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E-mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]