Information Technology:

Agencies Need to Improve the Implementation and Use of Earned Value Techniques to Help Manage Major System Acquisitions

GAO-10-2: Published: Oct 8, 2009. Publicly Released: Nov 6, 2009.

Additional Materials:

Contact:

David A. Powner
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

In fiscal year 2009, the federal government planned to spend about $71 billion on information technology (IT) investments. To more effectively manage such investments, in 2005 the Office of Management and Budget (OMB) directed agencies to implement earned value management (EVM). EVM is a project management approach that, if implemented appropriately, provides objective reports of project status, produces early warning signs of impending schedule delays and cost overruns, and provides unbiased estimates of anticipated costs at completion. GAO was asked to assess selected agencies' EVM policies, determine whether they are adequately using earned value techniques to manage key system acquisitions, and eval- uate selected investments' earned value data to determine their cost and schedule performances. To do so, GAO compared agency policies with best practices, performed case studies, and reviewed documenta- tion from eight agencies and 16 major investments with the highest levels of IT development-related spending in fiscal year 2009.

While all eight agencies have established policies requiring the use of EVM on major IT investments, these policies are not fully consistent with best practices. In particular, most lack training requirements for all relevant personnel responsible for investment oversight. Most policies also do not have adequately defined criteria for revising program cost and schedule baselines. Until agencies expand and enforce their EVM policies, it will be difficult for them to gain the full benefits of EVM. GAO's analysis of 16 investments shows that agencies are using EVM to manage their system acquisitions; however, the extent of implementation varies. Specifically, for 13 of the 16 investments, key practices necessary for sound EVM execution had not been implemented. For example, the project schedules for these investments contained issues--such as the improper sequencing of key activities--that undermine the quality of their performance baselines. This inconsistent application of EVM exists in part because of the weaknesses contained in agencies' policies, combined with a lack of enforcement of policies already in place. Until key EVM practices are fully implemented, these investments face an increased risk that managers cannot effectively optimize EVM as a management tool. Furthermore, earned value data trends of these investments indicate that most are currently experiencing shortfalls against cost and schedule targets. The total life-cycle costs of these programs have increased by about $2 billion. Based on GAO's analysis of current performance trends, 11 programs will likely incur cost overruns that will total about $1 billion at contract completion--in particular, 2 of these programs account for about 80 percent of this projection. As such, GAO estimates the total cost overrun to be about $3 billion at program completion (see figure). However, with timely and effective management action, it is possible to reverse negative trends so that the projected cost overruns may be reduced.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should modify policies governing EVM to ensure that they address the weaknesses that we identified, taking into consideration the criteria used in this report.

    Agency Affected: Department of Commerce

    Status: Open

    Comments: As of April 2013, the Department of Commence had not updated its policies governing earned value management to include a standard structure for defining work products. We will continue to monitor the department's progress in implementing this recommendation.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to implement the EVM practices that address the detailed weaknesses that we identified in appendix II, taking into consideration the criteria used in this report.

    Agency Affected: Department of Agriculture

    Status: Open

    Comments: The Modernize and Innovate the Delivery of Agriculture Services (MIDAS) program conducted an integrated baseline review in October 2010, but as of July 2013 has not provided evidence that its integrated master schedule has been updated. We will continue to monitor the department's progress in implementing this recommendation.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to implement the EVM practices that address the detailed weaknesses that we identified in appendix II, taking into consideration the criteria used in this report.

    Agency Affected: Department of Transportation

    Status: Closed - Implemented

    Comments: The Department of Transportation's En Route Automation Modernization program took several actions to implement key earned value management (EVM) practices. Specifically, in August 2012, the program conducted an integrated baseline review--a key practice needed to effectively determine the objective measure of earned value and develop and validate the performance measurement baseline. In addition, the program took actions to improve the reliability of the program's schedule, including conducting schedule reviews on a weekly basis and using performance metrics to track the success of dependent activities, thus positioning the program to better execute planned work activities.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to implement the EVM practices that address the detailed weaknesses that we identified in appendix II, taking into consideration the criteria used in this report.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: The Department of Homeland Security's Automated Commercial Environment (ACE) and Integrated Deepwater System--Common Operational Picture (COP) programs took actions to implement key earned value management (EVM) practices. Specifically, in June 2011, the ACE program conducted an integrated baseline review to validate its cost and schedule performance measurement baseline. During this review, the program assessed the quality of the integrated master schedule, including that all tasks were included and that resources had been assigned to these tasks, thus positioning the program to better execute planned work activities. In addition, the Deepwater COP program took actions to improve its ability to schedule and execute work, analyze EVM performance data, and execute management actions to mitigate risks. Specifically, the COP program increased its oversight of the contractor's EVM system though monthly reviews of earned value data in order to more closely monitor the program's schedule, analyze earned value performance data, and coordinate corrective actions--areas of which we also reported as having weaknesses.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to implement the EVM practices that address the detailed weaknesses that we identified in appendix II, taking into consideration the criteria used in this report.

    Agency Affected: Department of Veterans Affairs

    Status: Closed - Implemented

    Comments: The Department of Veterans Affairs formulated policies as part of its Program Management and Accountability System that incorporate critical practices of earned value management. Based on these new policies, the department found the VistA-FM program to be poorly planned and managed, areas of which we also reported as having weaknesses. As a result of this evaluation and our recommendation, the department eliminated the program.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to implement the EVM practices that address the detailed weaknesses that we identified in appendix II, taking into consideration the criteria used in this report.

    Agency Affected: Department of Commerce

    Status: Closed - Implemented

    Comments: The Department of Commerce's Field Data Collection Automation (FDCA) program improved its ability to establish a comprehensive earned value management (EVM) system by implementing metrics to analyze the program schedule and determine if actions are needed to address scheduling issues, such as the improper sequencing of tasks. Further, the program improved its ability to ensure reliable data by integrating EVM data analysis into tools used for monitoring contractor performance. Lastly, the FDCA program has taken steps to improve its use of earned value data to manage the acquisition. For example, the program took action to mitigate risks by integrating risk review and analysis into the Decennial Census performance dashboard.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should modify policies governing EVM to ensure that they address the weaknesses that we identified, taking into consideration the criteria used in this report.

    Agency Affected: Department of Agriculture

    Status: Open

    Comments: As of July 2013, the Department of Agriculture updated its policy governing earned value management (EVM), including establishing criteria for program rebaselining; however, the updated policy does not include a standard structure for defining work products. In addition, the department developed an EVM training course for executives; however, this training is not required in policy, nor has it been offered. We will continue to monitor the department's progress in implementing this recommendation.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should modify policies governing EVM to ensure that they address the weaknesses that we identified, taking into consideration the criteria used in this report.

    Agency Affected: Department of Homeland Security

    Status: Open

    Comments: The Department of Homeland Security stated that an earned value management (EVM) Working Group was formed in January 2011 to update the existing EVM policy, dated 2007. However, as of April 2013, an updated EVM policy had not been completed. We will continue to monitor the department's progress in implementing this recommendation.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should modify policies governing EVM to ensure that they address the weaknesses that we identified, taking into consideration the criteria used in this report.

    Agency Affected: Department of Veterans Affairs

    Status: Closed - Implemented

    Comments: In March 2010, the Department of Veterans Affairs addressed the weaknesses we identified by establishing its Program Management and Accountability System which included clear criteria for implementing the process on IT projects, established a standard structure for defining work products, and identified rebaselining criteria.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to take action to reverse current negative performance trends, as shown in the earned value data, to mitigate the potential cost and schedule overruns.

    Agency Affected: Department of Commerce

    Status: Closed - Implemented

    Comments: The Department of Commerce's Field Data Collection Automation (FDCA) program improved its ability to assess and to make timely decisions about contractor performance by integrating earned value data analysis into key program management tools. As a result, by July 2010, the FDCA program had reversed negative performance trends and had outperformed it planned cost target by about $5 million, thus mitigating the potential cost overruns expected on this key system acquisition program.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to implement the EVM practices that address the detailed weaknesses that we identified in appendix II, taking into consideration the criteria used in this report.

    Agency Affected: National Aeronautics and Space Administration

    Status: Closed - Implemented

    Comments: The National Aeronautics and Space Administration's James Webb Space Telescope (JWST) program and Mars Science Laboratory (MSL) program took several actions to implement key earned value management practices. Specifically, the JWST program established a work breakdown structure dictionary, instituted schedule health checks, and increased the frequency of variance analysis. Additionally, the MSL program also instituted schedule health checks and began analyzing root causes and corrective actions for variances.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to take action to reverse current negative performance trends, as shown in the earned value data, to mitigate the potential cost and schedule overruns.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: The Department of Homeland Security's Automated Commercial Environment (ACE) program switched from using a cost-plus-fixed-fee contract structure, where costs can be adjusted as a result of changes in the work to be performed, to using a firm-fixed-price structure with its major contractor. Consistent with this approach, in June 2010, the ACE program agreed to a firm-fixed-price task order with its major contractor that included the remaining implementation work through program completion. Because a firm-fixed-price contract places maximum risk on the contractor to provide the deliverable at the time, place, and price negotiated by the contractor and the government, the ACE program has reduced overall program risk and mitigated the potential for cost overruns (including those associated with schedule delays) on this key system acquisition program.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should modify policies governing EVM to ensure that they address the weaknesses that we identified, taking into consideration the criteria used in this report.

    Agency Affected: Department of Justice

    Status: Closed - Implemented

    Comments: In November 2009, the Department of Justice updated its earned value management (EVM) policy to require training for all personnel with investment oversight and program management responsibilities. In addition, the updated policy requires the use of a standard structure for defining work products. As a result, the Department of Justice has improved its ability to implement EVM across the agency.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to implement the EVM practices that address the detailed weaknesses that we identified in appendix II, taking into consideration the criteria used in this report.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Department of Defense's Warfighter Information Network-Tactical (WIN-T) and the Joint Tactical Radio System-Handheld, Manpack, Small Form Fit (JTRS-HMS) programs took actions to implement key earned value management (EVM) practices. For example, in May 2010, the WIN-T program conducted an integrated baseline review and identified a list of action items intended to improve, among other things, the measurement of earned value and the quality of the program's cost and schedule baselines. The program also improved its ability to use EVM data for decision-making purposes by updating the performance baseline as program changes occurred. Specifically, the program management team documented baseline changes requests to identify work activities that had increased or decreased in value and the rationale for the adjustments, which were subsequently approved by the WIN-T Program Manager. In addition, the JTRS-HMS program sought assistance from the Defense Contract Management Agency to evaluate the program's master schedule on a monthly-basis to identify weaknesses for corrective action.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to take action to reverse current negative performance trends, as shown in the earned value data, to mitigate the potential cost and schedule overruns.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Department of Defense's Warfighter Information Network-Tactical (WIN-T) program and the Joint Tactical Radio System-Handheld, Manpack, Small Form Fit (JTRS-HMS) programs took actions to improve their negative performance trends. For example, in May 2010, the WIN-T program conducted a review of the performance baseline and identified a list of action items intended to improve, among other things, the measurement of earned value and the quality of the program's schedule baseline. The WIN-T Program Office and prime contractor also conducted comprehensive "deep dive" reviews at a major subcontractor where schedule performance data for planned work were discussed. By January 2011, the WIN-T program had reduced its schedule variance by approximately $4.5 million. In addition, the JTRS-HMS program restructured the contractor management team overseeing system development efforts by replacing key personnel, such as the JTRS Program Director, and reorganizing staff to better align with specific work products. Further, the program identified significant schedule drivers and took action to address them by, for example, revising the program schedule to focus on the delivery of smaller incremental capabilities. By April 2011, the JTRS-HMS program had reduced its schedule variance by approximately $6.3 million. As a result, the Department of Defense has improved schedule performance and reduced the potential for cost overruns associated with schedule delays that likely would have been incurred on these key system acquisition programs.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should modify policies governing EVM to ensure that they address the weaknesses that we identified, taking into consideration the criteria used in this report.

    Agency Affected: Department of Defense

    Status: Open

    Comments: The Department of Defense stated that its Program Assessment and Root Cause Analysis (PARCA) office completed a review of the Defense Acquisition University's (DAU) Earned Value Management (EVM) curricula in April 2012 and subsequently briefed its recommendations to DAU representatives. Further, the Department of Defense stated that PARCA is planning to work with DAU to implement its findings and recommendations, including developing a new EVM training course. As of June 2013, this course had not yet been completed. We will continue to monitor the department's progress in implementing this recommendation.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to take action to reverse current negative performance trends, as shown in the earned value data, to mitigate the potential cost and schedule overruns.

    Agency Affected: National Aeronautics and Space Administration

    Status: Closed - Implemented

    Comments: The National Aeronautics and Space Administration's (NASA) James Webb Space Telescope (JWST) program and Mars Science Laboratory (MSL) program took several actions to mitigate potential cost and schedule overruns. For the JWST program, this included rebaselining its prime contract in May 2011 and improving its earned value management (EVM) practices, while the MSL program also took steps to improve its EVM practices, particularly by analyzing the root causes and corrective actions for negative variances shown in the EVM data. These actions helped NASA to mitigate the potential for cost and schedule overruns on these key system acquisition programs. Specifically, as of August 2012, the JWST program had achieved cost and schedule performance in-line with its May 2011 baseline, while the MSL program launched on schedule in November 2011, when measured against its fiscal year 2010 baseline.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to take action to reverse current negative performance trends, as shown in the earned value data, to mitigate the potential cost and schedule overruns.

    Agency Affected: Department of Transportation

    Status: Closed - Implemented

    Comments: The Surveillance and Broadcast System program reported on earned value data trends to an executive-level council on a monthly basis and, by July 2010, had reduced its negative schedule variance. These actions have reduced the potential schedule delays expected on this key system acquisition program.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to take action to reverse current negative performance trends, as shown in the earned value data, to mitigate the potential cost and schedule overruns.

    Agency Affected: Department of Veterans Affairs

    Status: Closed - Implemented

    Comments: The Department of Veterans Affairs formulated policies as part of its Program Management and Accountability System that incorporated critical practices of earned value management. Based on these new policies, the department found the VistA-FM program to be poorly planned and managed, areas of which we also reported as having weaknesses. As a result of this evaluation and our recommendation, the department eliminated the program.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should modify policies governing EVM to ensure that they address the weaknesses that we identified, taking into consideration the criteria used in this report.

    Agency Affected: National Aeronautics and Space Administration

    Status: Closed - Implemented

    Comments: In December 2011, the National Aeronautics and Space Administration updated its project management guidance for IT projects to define and require the use of a standard work breakdown structure. In addition, the updated guidance includes criteria for rebaselining and a description of the required steps to be taken in the event of a rebaseline.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to take action to reverse current negative performance trends, as shown in the earned value data, to mitigate the potential cost and schedule overruns.

    Agency Affected: Department of Justice

    Status: Closed - Implemented

    Comments: In August 2010, the Department of Justice's Next Generation Identification (NGI) program established an Executive Change Control Board to better prevent contract scope increases. This board is responsible for, among other things, reviewing and approving any change requests that affect the scope, milestones, and/or cost of the NGI development contract. In addition, the department established an independent verification and validation contractor with responsibility for independently reporting on a monthly basis the status of the NGI program, including earned value data. As a result, by February 2011, NGI had reduced the program's cost variance to approximately 1.6 percent of the cumulative cost of work performed, thus improving cost performance and reducing the potential cost overruns expected on this key system acquisition program.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should direct key system acquisition programs to take action to reverse current negative performance trends, as shown in the earned value data, to mitigate the potential cost and schedule overruns.

    Agency Affected: Department of Agriculture

    Status: Open

    Comments: As of July 2013, the Modernize and Innovate the Delivery of Agriculture Services (MIDAS) program had not yet taken action to reverse negative performance trends to mitigate potential cost and schedule overruns. When we confirm actions taken to mitigate and reverse these trends, we will provide updated information.

    Recommendation: In order to address the weaknesses identified in agencies' policies and practices in using EVM, the Secretaries of the Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Transportation, and Veterans Affairs and the Administrator of the National Aeronautics and Space Administration should modify policies governing EVM to ensure that they address the weaknesses that we identified, taking into consideration the criteria used in this report.

    Agency Affected: Department of Transportation

    Status: Closed - Implemented

    Comments: In July 2010, the Department of Transportation updated its earned value management (EVM) policy to include consistent requirements for complying with industry EVM standards. In addition, the updated policy requires EVM training for executive-level staff and the use of a standard structure for defining work products. As a result, the Department of Transportation has improved its ability to implement EVM across the agency.

    Jul 22, 2014

    Jun 17, 2014

    Jun 11, 2014

    Jun 10, 2014

    May 28, 2014

    May 21, 2014

    May 12, 2014

    May 7, 2014

    Apr 30, 2014

    Looking for more? Browse all our products here