NASA: Earned Value Management Implementation across Major Spaceflight Projects Is Uneven
Highlights
What GAO Found
The National Aeronautics and Space Administration's (NASA) 10 major spaceflight projects discussed in this report have not yet fully implemented earned value management (EVM). As a result, NASA is not taking full advantage of opportunities to use an important tool that could help reduce acquisition risk. GAO assessed the 10 projects against three fundamental EVM practices that, according to GAO's best practices cost guide, are necessary for maintaining a reliable EVM system. GAO found shortfalls in two of three fundamental practices. Specifically, we found that
- More than half of the projects did not use an EVM system that was fully certified as compliant with the industry EVM standard.
- Only 4 of the 10 projects established formal surveillance reviews, which ensure that key data produced by the system was reliable. The remaining 6 projects provided evidence of monthly EVM data reviews; however, the rigor of both the formal and informal surveillance reviews is questionable given the numerous data anomalies GAO found.
GAO also found that 3 projects had reliable EVM data while 7 had only partially reliable data. For the EVM data to be considered reliable per best practices it must be complete and accurate with all data anomalies explained.
NASA EVM focal points, headquarters officials, project representatives, and program executives cited cultural and other challenges as impediments to the effective use of EVM at the agency. Traditionally, NASA's culture has focused on managing science and engineering challenges and not on monitoring cost and schedule data, like an effective EVM system produces. As a result, several representatives said this information traditionally has not been valued across the agency. This sentiment was also echoed in a NASA study of EVM implementation. Also cited as a challenge to the effective use of EVM was NASA's insufficient number of staff with the skills to analyze EVM data. Without a sufficient number of staff with such skills, NASA's ability to conduct a sound analysis of the EVM data is limited. However, NASA has not conducted an EVM skills gap analysis to determine the extent of its workforce needs.
NASA has undertaken several initiatives aimed at improving the agency's use of EVM. For example, NASA strengthened its spaceflight management policy to reflect the industry EVM standard and has developed the processes and tools for projects to meet these standards through its new EVM system. While these are positive steps, the revised policy contains only the minimum requirements for earned value management. For example, it lacks a requirement for rigorous surveillance of how projects are implementing EVM and also does not require use of the agency's newly developed EVM system to help meet the new requirements. NASA has attempted to address EVM shortcomings through policy changes over the years, but these efforts have failed to adequately address the cultural resistance to implementing EVM.
Why GAO Did This Study
NASA historically has experienced cost growth and schedule slippage in its portfolio of major projects and has taken actions to improve in this area, including adopting the use of EVM. EVM is a tool developed to help project managers monitor risks. GAO was asked to examine (1) the extent to which NASA is using EVM to manage its major space flight acquisitions, (2) the challenges that NASA has faced in implementing an effective EVM system, and (3) NASA's efforts to improve its use of EVM. To address these questions, GAO obtained contractor and project EVM data and used established formulas and tools to analyze the data and assess NASA's implementation of EVM on 10 major spaceflight projects; interviewed relevant NASA headquarters, center and mission directorate officials on their views on EVM; and reviewed prior reports on EVM and organizational transformations. GAO compared NASA policies and guidance on EVM to best practices contained in GAO's cost estimating best practices guide.
Recommendations
GAO recommends that NASA establish a time frame for requiring new spaceflight projects to implement its new EVM system; conduct an EVM skills gap assessment; develop a change management plan for EVM; and strengthen its EVM requirements by requiring projects to implement formal EVM surveillance. NASA concurred with two recommendations and partially concurred with two others citing resource constraints. Despite NASA's plans to address some issues GAO identified, not addressing all key issues lessens the usefulness of EVM at NASA.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| National Aeronautics and Space Administration |
Priority Rec.
To improve NASA management and oversight of its spaceflight projects, the NASA Administrator should direct the appropriate offices to establish a time frame by which all new spaceflight projects will be required to implement NASA's newly developed EVM system, unless the project is proposing to use a certified system, to ensure that in-house efforts are compliant with ANSI/EIA-748. The time frame selected should take in to account the need to increase NASA's institutional capability for conducting EVM and analyzing and reporting the data. |
NASA said that they are committed to using one project at each Center to implement the EVM Capability process. As this rollout occurs the Centers have the responsibility to develop an institutional capability that is capable of implementing the agency EVMS process on subsequent projects. Currently, Goddard Space Flight Center, Marshall Space Flight Center and Kennedy Space Center have implemented the process and have the capability to implement it on future projects. Johnson Space Center is rolling out the process on the Multi-purpose Crew Vehicle (Orion) project in fiscal year 2015 and is expected to have the Center capability at the end of fiscal year 2015. As future projects implement EVM at these Centers, NASA expects that they will follow the agency EVM process. NASA said that the Johnson Space Center, Kennedy Space Center, Goddard Space Flight Center, and Marshall Space Flight Center are NASA's four largest Centers from a total dollars standpoint, and comprise 98 percent of the NASA's space flight project EVM requirements. In the summer of 2015, the NASA EVM Steering Committee will meet to discuss options for candidate projects at Glenn Research Center and Langley Research Center. According to NASA, the remaining Centers do not have any projects or contracts that require EVM.
|
| National Aeronautics and Space Administration | To improve NASA management and oversight of its spaceflight projects, the NASA Administrator should direct the appropriate offices to conduct an EVM skills gap analysis to identify areas requiring augmented capability across the agency. Based on the results of the assessment, develop a workforce training plan to address any deficiencies. |
NASA completed the EVM skill gap survey. In addition, NASA developed an EVM training plan and briefed to the EVM Working Group which is comprised of organizational EVM Focal Points. They have responsibility for supporting the projects within their organizations with EVM support and training. Some of the new courses, i.e., EVM tools, within the training plan have already been taught at the Marshall Space Flight Center, Kennedy Space Center, Johnson Space Center and Goddard Space Flight Center. One of the outcomes of the skill gap survey was that it identified that 98 percent of NASA's EVM requirements are at these four Centers. Therefore, the focus in the short-term will be to provide EVM training to those four Centers. Other training courses will be taught at the other Centers over the next few months/years. In addition, some of the existing courses were updated to ensure that the agency's training packages are current. Also, the agency also is working closely with the Project Planning and Control working group to include the EVM training as part of the agency's overall Project Planning and Control training curriculum.
|
| National Aeronautics and Space Administration | To improve NASA management and oversight of its spaceflight projects, the NASA Administrator should direct the appropriate offices to develop an EVM change management plan to assist managers and employees throughout the agency with accepting and embracing earned value techniques while reducing the operational impact on the agency. The plan should include a strategy for having the agency's senior leadership communicate their commitment to implementation of EVM. |
NASA developed a change management plan and it was approved by the EVM Steering Committee. The EVM Steering Committee is comprised of senior level NASA officials and provides guidance to the agency on the implementation of EVM. The plan has been briefed to the EVM Focal Points and is expected to be implemented by the EVM Focal Points within their perspective organizations. The plan focuses on the rollout of the EVM Capability, EVM surveillance, and training.
|
| National Aeronautics and Space Administration |
Priority Rec.
To improve NASA management and oversight of its spaceflight projects, and to improve the reliability of project EVM data, the NASA Administrator should direct the appropriate offices to modify the NASA Procedural Requirements (NPR) 7120.5 to require projects to implement a formal surveillance program that: (1) Ensures anomalies in contractor-delivered and in-house monthly earned value management reports are identified and explained, and report periodically to the center and mission directorate's leadership on relevant trends in the number of unexplained anomalies. (2) Ensures consistent use of work breakdown structures (WBS) for both the EVM report and the schedule. (3) Ensures that lower level EVM data reconcile to project level EVM data using the same WBS structure. (4) Improves underlying schedules so that they are properly sequenced using predecessor and successor dependencies and are free of constraints to the extent practicable so that the EVM baseline is reliable. |
In December 2018, the Agency Program Management Council approved a Corrective Action Plan to Enhance EVM Implementation, which included steps to put into operation formal EVMS surveillance. In February 2019, the EVM Steering Committee approved additional funding to execute the Corrective Action Plan initiative and begin implementation of formal EVM surveillance. In August 2021, NASA included the Earned Value Management System Surveillance requirement in NASA Procedural Requirement 7120.5F - NASA Space Flight Program and Project Management Requirements that establishes the requirements by which NASA formulates and implements space flight programs and projects.
|