NASA:

Implementing a Knowledge-Based Acquisition Framework Could Lead to Better Investment Decisions and Project Outcomes

GAO-06-218: Published: Dec 21, 2005. Publicly Released: Jan 23, 2006.

Additional Materials:

Contact:

Cristina T. Chaplain
(202) 512-3600
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The National Aeronautics and Space Administration (NASA) plans to spend over $100 billion on capabilities and technologies to achieve the initial goals of the President's 2004 Vision for Space Exploration. In the past, NASA has had difficulty meeting cost, schedule, and performance objectives for some of its projects because it failed to adequately define project requirements and quantify resources. NASA will be further challenged by a constrained federal budget and a shrinking experienced NASA workforce. To help face these challenges and manage projects with greater efficiency and accountability, NASA recently updated its program and project management policy and is developing an agencywide systems engineering policy. GAO has issued a series of reports on the importance of obtaining critical information and knowledge at key junctures in major system acquisitions to help meet cost and schedule objectives. This report (1) evaluates whether NASA's policy supports a knowledge-based acquisition approach and (2) describes how NASA centers are implementing the agency's acquisition policies and guidance.

While NASA's revised policy for developing flight systems and ground support projects incorporates some of the best practices used by successful developers, it lacks certain key criteria and major decision reviews that support a knowledge-based acquisition framework. For example, NASA's policy requires projects to conduct a major decision review before moving from formulation to implementation. Further, before moving from formulation to implementation, projects must validate requirements and develop realistic cost and schedule estimates, human capital plans, a preliminary design, and a technology plan--key elements for matching needs to resources. However, NASA's policies do not require projects to demonstrate technologies at high levels of maturity before program start. By not establishing a minimum threshold for technology maturity, NASA increases the risk that design changes will be required later in development, when such changes are typically more costly to make. In addition, although NASA's policy does require project managers to establish a continuum of technical and management reviews, it does not specify what these reviews should be, nor does it require major decision reviews at other key points in a product's development. Acquiring knowledge at key junctures will become increasingly important as NASA proceeds to implement elements of the Vision. Without a major decision review at key milestones to ensure that the appropriate level of knowledge has been achieved to proceed to the next phase, the risk of cost and schedule overruns, as well as performance shortfalls, increases. NASA centers have varying approaches for implementing the agency's policies and guidance. Some centers have established product development criteria that are similar to the criteria used in a knowledge-based acquisition, while other centers have not. As a result, each center reports a different level and type of knowledge about a project at key decision points. Centers also rely on project managers and systems engineers to employ good project management and systems engineering practices. However, given the loss of experienced project managers and the decline of in-house systems engineering and technical capabilities, that reliance could be problematic. These situations make it difficult for decision makers to evaluate projects on the same basis and make sound investment decisions and tradeoffs based on those evaluations. A standardized, knowledge-based approach would prepare NASA to face competing budgetary priorities and better position the agency to make difficult decisions regarding the investment in and termination of projects.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: In order to close the gaps between NASA's current acquisition environment and best practices on knowledge-based acquisition, NASA should take steps to ensure NASA projects follow a knowledge-based approach for product development. Specifically, the NASA Administrator should direct the Office of the Chief Engineer, in drafting its systems engineering policy, to incorporate requirements in the policy for flight systems and ground support projects to capture specific product knowledge by key junctures in project development. The demonstration of this knowledge should be used as exit criteria for decision making (1) before projects are approved to transition from formulation to implementation, the policy should require that projects demonstrate that key technologies have reached a high maturity level; (2) before projects are approved to transition from final design to fabrication, assembly, and test, the policy should require that projects demonstrate that the design is stable; and (3) before projects are approved to transition into production, the policy should require projects to demonstrate that the design can be manufactured within cost, schedule, and quality targets.

    Agency Affected: National Aeronautics and Space Administration

    Status: Closed - Implemented

    Comments: NASA issued their system engineering handbook on March 13, 2006, without incorporating best practices for entrance and exit criteria as requirements for its technical reviews. As an alternative, NASA has temporarily placed entrance and exit criteria for their technical reviews consistent with best practices in the appendix of the revised policy. According to a NASA representative, the reason why the agency did not incorporate such changes was due to resistance from several of its centers. The representative stated that the plan is to transition these criteria from the appendix to the requirements document in the next revision. In the area of technology maturity, a second NASA representative indicated that NASA plans to incorporate language in 7120.5D, the revision to NPR 7120.5C, to require high levels of technology maturity before moving into the implementation phase of NASA's lifecycle. In the area of production, NASA did not include production readiness reviews as part of their minimum requirements in its systems engineering policy NPR 7123. Once NPR 7120.5D has been issued and the key decision points proposed in that document are formalized, NASA plans to issue an interim directive for NPR 7123 that requires a production review for projects that enter a production phase, such as the Crew Exploration Vehicle. For projects that do not enter a production phase, production success criteria is included in criteria for the critical design review (CDR) milestone.

    Recommendation: In order to close the gaps between NASA's current acquisition environment and best practices on knowledge-based acquisition, NASA should take steps to ensure NASA projects follow a knowledge-based approach for product development. Specifically, the NASA Administrator should direct the Office of the Chief Engineer to revise NASA Procedural Requirement 7120.5C to institute additional major decision reviews following the Non-Advocate Review for flight systems and ground support projects, which result in recommendations to the appropriate decision authority. These reviews should be tied to the key junctures during project development mentioned above in order to increase the likelihood that cost, schedule, and performance requirements of the project will be met.

    Agency Affected: National Aeronautics and Space Administration

    Status: Closed - Implemented

    Comments: In response to our recommendation, NASA issued a revision to the policy, NPR 7120.5D, in March 2007. The revised policy institutes several key decision points (KDP) in development lifecycle for flight systems and ground support projects. At each KDP, a decision authority is responsible for authorizing the transition to the next lifecycle phase for the project. The changes apparent in NPR 7120.5D meet the intent of our recommendation and bring the policy in line with best practices for product development.

    Jul 23, 2014

    Jul 16, 2014

    Jun 20, 2014

    May 15, 2014

    May 8, 2014

    Apr 15, 2014

    Mar 12, 2014

    Feb 4, 2014

    Jan 13, 2014

    Jan 8, 2014

    Looking for more? Browse all our products here