Air Traffic Control: FAA Uses Earned Value Techniques to Help Manage Information Technology Acquisitions, but Needs to Clarify Policy and Strengthen Oversight
GAO-08-756
Published: Jul 18, 2008. Publicly Released: Jul 18, 2008.
Skip to Highlights
Highlights
In fiscal year 2008, the Federal Aviation Administration (FAA) plans to spend over $2 billion on information technology (IT) investments--many of which support FAA's air traffic control modernization. To more effectively manage such investments, in 2005 the Office of Management and Budget required agencies to use earned value management (EVM). EVM is a project management approach that, if implemented appropriately, provides objective reports of project status, produces early warning signs of impending schedule delays and cost overruns, and provides unbiased estimates of a program's total costs. Among other objectives, GAO was asked to assess FAA's policies for implementing EVM on its IT investments, evaluate whether the agency is adequately using these techniques to manage key IT acquisitions, and assess the agency's efforts to oversee EVM compliance. To do so, GAO compared agency policies with best practices, performed four case studies, and interviewed key FAA officials.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Transportation | To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to modify acquisition policies governing EVM to require the use of a product-oriented standard work breakdown structure. |
In response to our recommendation, in November 2010, FAA updated its standard work breakdown structure to delineate work based on specific product deliverables. Among other things, the product-oriented structure is to be used to develop key program tracking documents, such as program schedules and contractor performance reports. As a result, FAA programs will be better positioned to identify and address specific deliverables that are causing cost or schedule overruns.
|
| Department of Transportation | To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to modify acquisition policies governing EVM to enforce existing EVM training requirements and expand these requirements to include senior executives responsible for investment oversight and program staff responsible for program oversight. |
In response to our recommendation, in March 2010, FAA updated its acquisition management policy to establish EVM training requirements for staff responsible for the management and oversight of FAA major and non-major acquisition programs or projects. In addition, FAA updated its policy to establish a certification process to better ensure that its acquisition workforce meets applicable training requirements, including those related to EVM. Among other things, the revised policy establishes requirements for acquiring and maintaining certification, as well as assigns responsibility for monitoring and reporting compliance with certification requirements. As a result, FAA has improved its ability to ensure that acquisition staff have the appropriate skills to validate and interpret EVM data, and that its executives fully understand the data they are given in order to make informed decisions.
|
| Department of Transportation | To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to modify acquisition policies governing EVM to define acceptable reasons for rebaselining and require programs seeking to rebaseline to (1) perform a root cause analysis to determine why significant cost and schedule variances occurred and (2) establish mitigation plans to address the root cause. |
In response to our recommendation, in April 2011, FAA modified its acquisition policies governing EVM to include reasons for rebaselining. Specifically, the revised policy requires program offices to perform a root cause analysis to determine why significant cost and schedule variances occurred and establish mitigation plans. As a result, FAA executive managers will be positioned to more effectively oversee programs and reverse negative cost and schedule trends.
|
| Department of Transportation | To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to direct the En Route Automation Modernization (ERAM) program office to work with FAA's Value Management Office to determine the root causes for the anomalies found in the contractor's EVM reports. |
In response to our recommendation, in July 2008, the ERAM program revised the format of its monthly contractor performance reports to include a section dedicated to earned value reporting anomalies, whereby the contractor must explain and account for significant data anomalies. Further, where applicable, the contractor performance reports include corrective action plans to resolve the anomalies identified. As a result, the ERAM program will be better positioned to identify and correct data quality issues in a timely manner, and also make decisions based on reliable data.
|
| Department of Transportation | To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to direct the ERAM program office to work with FAA's Value Management Office to develop a corrective action plan to resolve these problems. |
In response to our recommendation, in July 2008, the ERAM program revised the format of its monthly contractor performance reports to include a section dedicated to earned value reporting anomalies, whereby the contractor must explain and account for significant data anomalies. Further, where applicable, the contractor performance reports include corrective action plans to resolve the anomalies identified. As a result, the ERAM program will be better positioned to identify and correct data quality issues in a timely manner, and also make decisions based on reliable data.
|
| Department of Transportation | To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to direct the Value Management Office to improve its oversight processes by including an evaluation of contractors' performance data as part of its program assessment criteria, when FAA has the authority to do so. |
In response to our recommendation, in January 2010, FAA modified its earned value management oversight processes by including an evaluation of contractor performance data to identify issues that may undermine the validity of these data. As a result, FAA managers will be better positioned to identify and correct data quality issues on their programs in a timely manner, and also make decisions based on reliable data.
|
| Department of Transportation | To improve FAA's ability to effectively implement EVM on its IT acquisition programs, the Secretary of Transportation should direct the Acting FAA Administrator to direct the Value Management Office to improve its oversight processes by distinguishing between programs that collect earned value data on fully integrated programs and those that do not in its agencywide progress reports to provide transparency to decision makers. |
In response to our recommendation, FAA improved its oversight processes through its program EVM surveillance assessments. In particular, in its September 2011 EVM assessment summary report, FAA found that, of the 19 major FAA programs whose EVM systems were fully assessed, all were generally in compliance with the agency's EVM standards. These standards require that EVM be performed on the total program effort (including government and contractor work). As a result, FAA can better ensure that the assessment of EVM data is consistent across the agency and that senior executives can have increased confidence in the quality of major acquisition programs' EVM data when making investment decisions.
|
Full Report
Topics
Air traffic control systemsAir transportationContract oversightCost analysisCost overrunsData collectionData integrityEarned value management systemsEmployee trainingFunds managementInformation technologyInternal controlsIT acquisitionsPerformance measuresPolicy evaluationProcurement policyProcurement practicesProgram evaluationProgram managementSchedule slippagesStandardsTransportation planningTransportation policiesCost estimatesPolicies and proceduresProgram costsProgram goals or objectivesProgram implementation