Electronic Records Archive:

National Archives Needs to Strengthen Its Capacity to Use Earned Value Techniques to Manage and Oversee Development

GAO-11-86: Published: Jan 13, 2011. Publicly Released: Feb 4, 2011.

Additional Materials:

Contact:

David A. Powner
(202) 512-9286
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Since 2001, the National Archives and Records Administration (NARA) has been working to develop an Electronic Records Archive (ERA) to preserve and provide access to massive volumes and all types of electronic records. However, in acquiring this system, NARA has repeatedly revised the program schedule and increased the estimated costs for completion from $317 million to $567 million. NARA is to manage this acquisition using, among other things, earned value management (EVM). EVM is a project management approach that, if implemented appropriately, provides objective reports of project status and unbiased estimates of anticipated costs at completion. GAO was asked to (1) assess whether NARA is adequately using EVM techniques to manage the acquisition and (2) evaluate the earned value data to determine ERA's cost and schedule performance. To do so, GAO compared agency and contractor documentation with best practices, evaluated earned value data to determine performance trends, and interviewed cognizant officials.

NARA has, to varying degrees, established selected best practices needed to manage the ERA acquisition through EVM, but weaknesses exist in most areas. For example, the scope of effort in ERA's work breakdown structure is not adequately defined, thus impeding the ability to measure progress made on contractor deliverables. These weaknesses exist in part because NARA lacks a comprehensive EVM policy, training, and specialized resources and also frequently replans the program. As a result, NARA has not been positioned to identify potential cost and schedule problems early and thus has not been able to take timely actions to correct problems and avoid program schedule delays and cost increases. ERA's earned value data trends do not accurately portray program status due to the program's weaknesses in implementing EVM; however, historical program trends indicate that future cost overruns will likely be between $195 million and $433 million to fully develop ERA as planned and between $205 and $405 million at program end. In contrast, the contractor's estimated cost overrun is $2.7 million. Without more useful earned value data, NARA will remain unprepared to effectively oversee contractor performance and make realistic projections of program costs. GAO recommends, among other things, that NARA establish a comprehensive plan for all remaining work; improve the accuracy of earned value performance reports; and engage executive leadership in correcting negative trends. NARA generally concurred with GAO's recommendations.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To improve NARA's ability to effectively implement EVM on its ERA system acquisition program, and while the current system development contract is active, the Archivist of the United States should direct the NARA CIO to direct the ERA program to establish a comprehensive baseline (through an integrated master schedule) for all remaining work on contract.

    Agency Affected: National Archives and Records Administration

    Status: Closed - Implemented

    Comments: NARA agreed with our recommendation. In February 2011, NARA conducted a review of the ERA program's cost and schedule baseline. Shortly thereafter, NARA finalized its baseline integrated master schedule, which reflects the agency's plan to complete remaining ERA development work on contract by the end of fiscal year 2011.

    Recommendation: To improve NARA's ability to effectively implement EVM on its ERA system acquisition program, and while the current system development contract is active, the Archivist of the United States should, direct the NARA CIO to ensure that the ERA program obtains reliable EVM performance reports, taking into consideration the data anomalies and weaknesses identified in this report.

    Agency Affected: National Archives and Records Administration

    Status: Closed - Implemented

    Comments: NARA agreed with our recommendation. The agency began conducting more detailed reviews of contractor-provided performance reports. These reviews resulted in NARA identifying a number of data anomalies, including instances where more earned value had been achieved than what had been spent on certain items. In addition, NARA increased outside surveillance support by the Defense Contract Management Agency in reviewing contractor-provided data.

    Recommendation: To improve NARA's ability to effectively implement EVM on its ERA system acquisition program, and while the current system development contract is active, the Archivist of the United States should direct the NARA CIO to engage senior NARA and contractor leadership/oversight officials to direct attention to reversing current negative performance trends, as shown in the earned value data, and take action to mitigate the potential cost and schedule overruns.

    Agency Affected: National Archives and Records Administration

    Status: Closed - Implemented

    Comments: In response to our recommendation, NARA's developed a corrective action plan which outlined its intention to expand presentations and reports to senior NARA executives to include information on cost and schedule. NARA subsequently provided additional information on cost, schedule, and performance in briefings to senior agency officials and conducted monthly presentations to senior NARA and contractor leadership and oversight officials. As a result, NARA management had a more reliable factual basis for subsequent decisions about the ERA program's direction.

    Recommendation: To improve NARA's ability to effectively implement EVM on its ERA system acquisition program, and while the current system development contract is active, the Archivist of the United States should direct the NARA CIO to include as part of its acquisition policy governing EVM requirements for (1) EVM training for senior executives and program staff responsible for ERA investment oversight and (2) ongoing surveillance of the ERA program's EVM system to ensure its compliance with industry standards.

    Agency Affected: National Archives and Records Administration

    Status: Open

    Comments: As of September 2102, NARA had not updated their acquisition policies to include EVM requirements for (1) EVM training for senior executives and program staff responsible for ERA investment oversight and (2) ongoing surveillance of the ERA program's EVM system to ensure its compliance with industry standards. We will continue to monitor the agency's progress in implementing this recommendation.

    Recommendation: To improve NARA's ability to effectively implement EVM on its ERA system acquisition program, and while the current system development contract is active, the Archivist of the United States should direct the NARA CIO to ensure that the ERA program has the appropriate level of specialized staff in place to perform EVM analysis and oversight activities.

    Agency Affected: National Archives and Records Administration

    Status: Open

    Comments: As of September 2012, NARA stated that the ERA program has the appropriate level of specialized staff in place to perform EVM analysis and oversight activities. When we confirm further actions taken by the agency in response to this recommendation, we will provide updated information.

    Recommendation: Taking into consideration the new ERA program direction, the Archivist of the United States should direct the CIO), using a gap analysis of the work completed through fiscal year 2011, to determine and clearly define the remaining work that will be pursued in the future ERA system development phase (Phase 2 and the original ERA requirements set.

    Agency Affected: National Archives and Records Administration

    Status: Open

    Comments: As of September 2012, NARA stated they will revisit this recommendation when or if a Phase 2 ERA development is initiated. We will continue to monitor the agency's progress in implementing this recommendation.

    Recommendation: Taking into consideration the new ERA program direction, the Archivist of the United States should direct the CIO to direct the ERA program to develop new cost and schedule estimates for a comprehensive Phase 2 baseline, as well as for the total program life cycle. In combination with the above action, this should provide the program with enough information to disclose to the Congress the exact work that will be accomplished and the cost of that work.

    Agency Affected: National Archives and Records Administration

    Status: Open

    Comments: As of September 2012, NARA stated they will revisit this recommendation when or if a Phase 2 ERA development is initiated. We will continue to monitor the agency's progress in implementing this recommendation.

    Recommendation: Taking into consideration the new ERA program direction, the Archivist of the United States should direct the CIO to direct the ERA program to implement the EVM practices that address the detailed weaknesses that we identified in this report, taking into consideration the criteria used, including (1) establishing a comprehensive Phase 2 baseline (through an integrated master schedule) that has been validated through an integrated baseline review and limits the use of nonobjective metrics; (2) ensuring that reliable reports of EVM performance are being produced, including records of work completed, forecasts of estimates at completion, and explanations/corrective actions for variances and data anomalies; and (3) engaging senior NARA leadership/oversight officials to ensure that earned value data are being used for decision-making purposes, including holding and documenting executive meetings to ensure that cost and schedule risks/issues have been tracked to closure, negative performance trends are mitigated, and major updates made to the baseline have been validated through an integrated baseline review.

    Agency Affected: National Archives and Records Administration

    Status: Open

    Comments: As of September 2012, NARA stated they will revisit this recommendation when or if a Phase 2 ERA development is initiated. We will continue to monitor the agency's progress in implementing this recommendation.

    Jul 23, 2014

    Jul 8, 2014

    Dec 5, 2013

    Sep 10, 2013

    Aug 22, 2013

    Jun 20, 2013

    May 9, 2013

    Apr 4, 2013

    Mar 13, 2013

    Looking for more? Browse all our products here