National Nuclear Security Administration: Actions Needed to Clarify Use of Contractor Assurance Systems for Oversight and Performance Evaluation
Highlights
What GAO Found
The National Nuclear Security Administration (NNSA) has not fully established policies or guidance for using information from contractor assurance systems (CAS) to conduct oversight of management and operating (M&O) contractors. As a result, NNSA does not have standards for ensuring that contractors are overseen consistently. For example, at the headquarters level, NNSA has not provided guidance beyond its general framework for assessing the maturity of contractors' CAS to determine whether information from CAS is sufficiently reliable for federal oversight purposes. In the absence of headquarters level policy, NNSA field offices—federal offices located at contractor operated sites and responsible for day-to-day oversight of M&O contractors—have established their own procedures for using information from CAS to conduct oversight, but these procedures also are not always complete and differ among field offices. For example, five of NNSA's seven field offices reported having complete procedures for assessing CAS maturity, but these procedures describe different processes and rating scales for conducting such assessments. The other two field offices reported not having such procedures. NNSA had designed a process for validating field offices' oversight approaches, including the extent to which these approaches use information from CAS, but NNSA discontinued this process after determining that it had not been effective. Discontinuing this process without replacing it with another form of validation eliminated the internal control activity NNSA designed to assure the effectiveness and consistency of oversight approaches across the nuclear security enterprise, including the appropriate use of information from CAS.
NNSA also has not established policies or guidance specific to using information from CAS to evaluate M&O contractor performance. Neither NNSA policy nor NNSA's Handbook published in 2013 to guide the performance evaluation process includes information on how or to what extent NNSA officials should use information from CAS in evaluating M&O contractors' performance. Some field office officials told GAO they developed their own procedures on performance evaluation. GAO reviewed these procedures and found they were not sufficiently detailed for using information from CAS to evaluate contractors' performance.
NNSA has not determined whether it has sufficient, qualified personnel to implement its framework for using information from CAS for oversight or for performance evaluation. NNSA officials GAO interviewed were unable to identify any studies that had been completed that assessed this question. Field office officials have raised concerns that staffing levels and the mix of staff skills may not be adequate to conduct appropriate oversight in the future and that this may result in overreliance on information from CAS without the ability to ensure that this information is sufficiently mature. In 2013, concerned about their capacity to fully support all oversight requirements, field offices called on NNSA headquarters to initiate a review of field office staffing resources needed to implement the oversight and performance evaluation framework and whether a staffing model including shared technical staff among field offices could address these concerns. NNSA began this review, but NNSA headquarters officials said it was not completed, the data reviewed are now too old to be useful, and that the agency has no plans to complete it. These officials said that they plan on discussing staffing issues with senior leadership in 2015.
Why GAO Did This Study
NNSA is responsible for overseeing the work of seven M&O contractors that execute its programs across eight nuclear security enterprise sites and evaluating their performance. NNSA directed its M&O contractors in 2011 to implement CAS—systems designed and used by contractors to assure their own performance—that NNSA could also leverage for oversight purposes and thereby improve efficiency. To determine the extent to which to use contractor-generated information made available through CAS, NNSA is to apply a framework for evaluating the risk of contractors' activities, their past performance, and their CAS maturity.
GAO was asked to review NNSA's implementation of its framework for using CAS. This report examines the extent to which NNSA has fully established policies and guidance for using information from CAS to (1) oversee M&O contractors and (2) evaluate M&O contractors' performance; and whether NNSA has determined it has sufficient, qualified personnel to implement its framework for using information from CAS for these two purposes. To conduct its work, GAO surveyed all NNSA field offices and analyzed key policies and guidance on NNSA's use of information from CAS.
Recommendations
GAO is recommending, among other things, that NNSA develop guidance on using information from CAS to oversee and evaluate M&O contractors, reinstitute a process for evaluating oversight effectiveness, and study staffing needs. NNSA agreed with these recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| National Nuclear Security Administration | To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should establish comprehensive NNSA policies and guidance, beyond a general framework as included in NAP-21, for using information from CAS to conduct oversight of M&O contractors, clarifying whether CAS is to cover mission-related activities, and describing how to conduct assessments of risk, CAS maturity, and the level of the contractor's past performance. |
We recommended that NNSA establish comprehensive policies and guidance, beyond a general framework, for using information from contractor assurance systems (CAS) to conduct oversight of management and operating (M&O) contractors, clarifying whether CAS is to cover mission-related activities and describing how to conduct assessments of risk, CAS maturity, and the level of the contractor's past performance. NNSA agreed with the recommendation and has taken an important step to revise its policy. However, NNSA needs to take additional action. Specifically, NNSA approved a revised corporate site governance policy in August 2016, and NNSA further revised its policy in October 2019. The revised policy improves on the agency's prior policy by clarifying one element in our recommendation that CAS is to cover mission-related activities. NNSA field offices have generally adopted updated procedures outlining their governance systems. These are a step in the right direction, but our review of site level guidance does not show how NNSA could conduct assessments of risk, CAS maturity, and level of contractor's past performance. As a result, NNSA's policy and procedures are still a general framework. Specifically, NNSA needs to develop guidance for NNSA headquarters' and field offices' procedures to use information from CAS and appropriately balance use of information from CAS with other more direct activities to oversee M&O contractors. As of July 2024, NNSA officials said the agency plans to update its current corporate site governance policy in late 2024 to address GAO's remaining criteria for closure. GAO will continue to monitor the agency's progress.
|
| National Nuclear Security Administration | To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should work with field office managers to establish field office procedures consistent with headquarters policy and guidance to support assessment practices for determining appropriate oversight approaches. |
NNSA concurred with the recommendation. As of October 2019, NNSA field offices have generally adopted updated procedures for assessment practices, and these updated procedures are generally consistent with the framework described in NNSA's August 2016 revised corporate site governance policy.
|
| National Nuclear Security Administration | To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should reestablish a process for reviewing the effectiveness of field offices' oversight approaches, including their use of information from CAS. |
NNSA concurred with this recommendation and issued Supplemental Directive (SD) 226.1B, which provides an approach for reviewing the effectiveness of field offices' oversight approaches. The SD requirements include a peer review process and provide attributes that, if followed, may result in improved field oversight activities. According to NNSA, the agency has completed three peer reviews--pilot peer reviews at the Nevada National Security Site in July 2017 and the Lawrence Livermore National Laboratory in November 2017, and another peer review at the National Nuclear Security Administration Production Office (Y-12 National Security Complex/Pantex Plant) in April 2018--with additional peer reviews scheduled for later in 2018 (Sandia) and 2019 (Kansas City Plant, Los Alamos, Savannah River).
|
| National Nuclear Security Administration | To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should revise NNSA policy, guidance, and procedures on performance evaluation to fully address how and under what circumstances those responsible for evaluating M&O contractors' performance should use information from CAS for this purpose. |
NNSA concurred with the recommendation. NNSA's revised Supplemental Directive addresses the use of CAS information in evaluating M&O contractor performance, NNSA issued a revised performance management policy (NAP-4C), and NNSA field offices have generally adopted updated procedures outlining their governance systems. Furthermore, NNSA's 2020 Corporate Performance Evaluation Process Annual Implementation Guidance says "Federal oversight activities must evaluate the effectiveness of CAS, use CAS information including safety performance measures, objectives, and commitments to assess effectiveness of contractor programs and processes, and perform risk-based oversight."
|
| National Nuclear Security Administration | To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should assess NNSA's staffing needs to determine whether it has sufficient, qualified personnel to conduct oversight activities consistent with comprehensive policies and guidance, including the use of information from CAS. |
NNSA concurred with this recommendation and has assessed its staffing needs through a study by the U.S. Office of Personnel Management (OPM) and a study by NNSA's Office of Cost Estimating and Program Evaluation (CEPE). According to NNSA, it is actively pursuing a staffing strategy for defensible and sustainable oversight based on the results of these studies. Specifically, the studies informed staffing plans to identify positions needed for mission success that NNSA directed each program, functional, and field office to develop in October 2018. The staffing plans are currently being reviewed by the Staffing Plan Review Group, which is charged to examine position title, function, occupational series, grade, supervisory status, geographic locality and mission requirement and identify if additional staff are needed above the levels in the OPM and CEPE studies. The Staffing Plan Review Group will present its findings to the Administrator near the end of 2019.
|