Air Traffic Control:

FAA Uses Earned Value Techniques to Help Manage Information Technology Acquisitions, but Needs to Clarify Policy and Strengthen Oversight

GAO-08-756: Published: Jul 18, 2008. Publicly Released: Jul 18, 2008.

Additional Materials:

Contact:

David A. Powner
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

In fiscal year 2008, the Federal Aviation Administration (FAA) plans to spend over $2 billion on information technology (IT) investments--many of which support FAA's air traffic control modernization. To more effectively manage such investments, in 2005 the Office of Management and Budget required agencies to use earned value management (EVM). EVM is a project management approach that, if implemented appropriately, provides objective reports of project status, produces early warning signs of impending schedule delays and cost overruns, and provides unbiased estimates of a program's total costs. Among other objectives, GAO was asked to assess FAA's policies for implementing EVM on its IT investments, evaluate whether the agency is adequately using these techniques to manage key IT acquisitions, and assess the agency's efforts to oversee EVM compliance. To do so, GAO compared agency policies with best practices, performed four case studies, and interviewed key FAA officials.

FAA has established a policy requiring the use of EVM on its major IT acquisition programs, but key components of this policy are not fully consistent with best practices of leading organizations. Specifically, FAA fully met four and partially met three components of an effective EVM policy. For example, FAA requires its program managers to obtain EVM training, but it does not enforce completion of this training or require other relevant personnel to obtain this training. Until FAA expands and enforces its policy, it will be difficult for the agency to gain the full benefits of EVM. FAA is using EVM to manage IT acquisition programs, but not all programs are ensuring that their earned value data are reliable. Case studies of four programs demonstrated that all are using or planning to use EVM systems. However, of the three programs currently collecting EVM data, only one program is adequately ensuring that its earned value data are reliable. Another program is limited in its ability to ensure data reliability because it was initiated before earned value was required. The third program did not adequately validate contractor performance data. For example, GAO found anomalies in which the contractor reported spending funds without accomplishing work and others in which the contractor reported accomplishing work while crediting funds to the government. Until programs undertake a rigorous validation of their EVM data, FAA faces an increased risk that managers may not be getting the information they need to effectively manage the programs. FAA has taken important steps to oversee program compliance with EVM policies, but its oversight process lacks sufficient rigor. Through its recurring assessments, FAA has reported that most programs have improved their earned value capabilities over time, and that 74 percent of the programs were fully compliant with national standards. However, FAA's assessments are not thorough enough to identify anomalies in contractor data, and its progress reports do not distinguish between systems that collect comprehensive data and those that do not. As a result, FAA executives do not always receive an accurate view of the quality of a program's EVM data when making investment decisions on that program.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In response to our recommendation, in January 2010, FAA modified its earned value management oversight processes by including an evaluation of contractor performance data to identify issues that may undermine the validity of these data. As a result, FAA managers will be better positioned to identify and correct data quality issues on their programs in a timely manner, and also make decisions based on reliable data.

    Recommendation: To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to direct the Value Management Office to improve its oversight processes by including an evaluation of contractors' performance data as part of its program assessment criteria, when FAA has the authority to do so.

    Agency Affected: Department of Transportation

  2. Status: Closed - Implemented

    Comments: In response to our recommendation, in July 2008, the ERAM program revised the format of its monthly contractor performance reports to include a section dedicated to earned value reporting anomalies, whereby the contractor must explain and account for significant data anomalies. Further, where applicable, the contractor performance reports include corrective action plans to resolve the anomalies identified. As a result, the ERAM program will be better positioned to identify and correct data quality issues in a timely manner, and also make decisions based on reliable data.

    Recommendation: To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to direct the ERAM program office to work with FAA's Value Management Office to develop a corrective action plan to resolve these problems.

    Agency Affected: Department of Transportation

  3. Status: Closed - Implemented

    Comments: In response to our recommendation, in July 2008, the ERAM program revised the format of its monthly contractor performance reports to include a section dedicated to earned value reporting anomalies, whereby the contractor must explain and account for significant data anomalies. Further, where applicable, the contractor performance reports include corrective action plans to resolve the anomalies identified. As a result, the ERAM program will be better positioned to identify and correct data quality issues in a timely manner, and also make decisions based on reliable data.

    Recommendation: To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to direct the En Route Automation Modernization (ERAM) program office to work with FAA's Value Management Office to determine the root causes for the anomalies found in the contractor's EVM reports.

    Agency Affected: Department of Transportation

  4. Status: Closed - Implemented

    Comments: In response to our recommendation, in April 2011, FAA modified its acquisition policies governing EVM to include reasons for rebaselining. Specifically, the revised policy requires program offices to perform a root cause analysis to determine why significant cost and schedule variances occurred and establish mitigation plans. As a result, FAA executive managers will be positioned to more effectively oversee programs and reverse negative cost and schedule trends.

    Recommendation: To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to modify acquisition policies governing EVM to define acceptable reasons for rebaselining and require programs seeking to rebaseline to (1) perform a root cause analysis to determine why significant cost and schedule variances occurred and (2) establish mitigation plans to address the root cause.

    Agency Affected: Department of Transportation

  5. Status: Closed - Implemented

    Comments: In response to our recommendation, in March 2010, FAA updated its acquisition management policy to establish EVM training requirements for staff responsible for the management and oversight of FAA major and non-major acquisition programs or projects. In addition, FAA updated its policy to establish a certification process to better ensure that its acquisition workforce meets applicable training requirements, including those related to EVM. Among other things, the revised policy establishes requirements for acquiring and maintaining certification, as well as assigns responsibility for monitoring and reporting compliance with certification requirements. As a result, FAA has improved its ability to ensure that acquisition staff have the appropriate skills to validate and interpret EVM data, and that its executives fully understand the data they are given in order to make informed decisions.

    Recommendation: To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to modify acquisition policies governing EVM to enforce existing EVM training requirements and expand these requirements to include senior executives responsible for investment oversight and program staff responsible for program oversight.

    Agency Affected: Department of Transportation

  6. Status: Closed - Implemented

    Comments: In response to our recommendation, in November 2010, FAA updated its standard work breakdown structure to delineate work based on specific product deliverables. Among other things, the product-oriented structure is to be used to develop key program tracking documents, such as program schedules and contractor performance reports. As a result, FAA programs will be better positioned to identify and address specific deliverables that are causing cost or schedule overruns.

    Recommendation: To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to modify acquisition policies governing EVM to require the use of a product-oriented standard work breakdown structure.

    Agency Affected: Department of Transportation

  7. Status: Closed - Implemented

    Comments: In response to our recommendation, FAA improved its oversight processes through its program EVM surveillance assessments. In particular, in its September 2011 EVM assessment summary report, FAA found that, of the 19 major FAA programs whose EVM systems were fully assessed, all were generally in compliance with the agency's EVM standards. These standards require that EVM be performed on the total program effort (including government and contractor work). As a result, FAA can better ensure that the assessment of EVM data is consistent across the agency and that senior executives can have increased confidence in the quality of major acquisition programs' EVM data when making investment decisions.

    Recommendation: To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to direct the Value Management Office to improve its oversight processes by distinguishing between programs that collect earned value data on fully integrated programs and those that do not in its agencywide progress reports to provide transparency to decision makers.

    Agency Affected: Department of Transportation

 

Explore the full database of GAO's Open Recommendations »

Nov 18, 2014

Oct 9, 2014

Sep 26, 2014

Sep 25, 2014

Sep 23, 2014

Sep 12, 2014

Jul 31, 2014

Jul 23, 2014

Jun 25, 2014

Looking for more? Browse all our products here