Department of Homeland Security:

Better Planning and Assessment Needed to Improve Outcomes for Complex Service Acquisitions

GAO-08-263: Published: Apr 22, 2008. Publicly Released: May 8, 2008.

Additional Materials:

Contact:

John P. Hutton
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Department of Homeland Security (DHS) has relied on service acquisitions to meet its expansive mission. In fiscal year 2006, DHS spent $12.7 billion to procure services. To improve service acquisition outcomes, federal procurement law establishes a preference for a performance-based approach, which focuses on developing measurable outcomes rather than prescribing how contractors should perform services. GAO was asked to (1) evaluate the implementation of a performance-based approach in the context of service acquisitions for major, complex investments, and (2) identify management challenges that may affect DHS's successful acquisitions for major investments, including those using a performance-based approach. GAO reviewed judgmentally selected contracts for eight major investments at three DHS components totaling $1.53 billion in fiscal years 2005 and 2006; prior GAO and DHS Inspector General reviews; management documents and plans; and related data, including 138 additional contracts for basic services.

All service contracts for the eight major, complex investments GAO reviewed had outcome-oriented requirements; however, four of these contracts did not have well-defined requirements, a complete set of measurable performance standards, or both. These service contracts experienced cost overruns, schedule delays, or did not otherwise meet performance expectations. In contrast, service contracts for the other four investments GAO reviewed had well-defined requirements linked to measurable performance standards. Contractors had begun work on three of these four contracts and performed within budget meeting the standards. This finding is consistent with prior GAO work on service acquisitions, which has highlighted the criticality of sound acquisition planning to develop well-defined requirements and measurable performance standards to achieving desired outcomes. In the four cases that had negative outcomes, program officials identified the contractor performance weaknesses through quality assurance surveillance and took corrective actions. Prior GAO work has found that if acquisitions, including those that are performance-based, are not appropriately planned, structured, and monitored, there is an increased risk that the government may receive products or services that are over budget, delivered late, and of unacceptable quality. In managing its service acquisitions, including those that are performance based, DHS has faced workforce and oversight challenges. Prior GAO work has highlighted the importance of having the right people with the right skills to achieve successful acquisition outcomes. Contracts for two major investments with negative cost and schedule outcomes did not have the staff needed to adequately plan and execute the contracts. Further, while representatives for several of the contracts GAO reviewed indicated that contracting and program staff worked well together, some senior acquisition representatives at the component level indicated that a lack of collaboration between these key stakeholders has been a challenge when developing and managing complex service acquisitions. In terms of oversight, component contracting and program officials said they used a performance-based approach to the maximum extent practicable; however, DHS does not have reliable data to facilitate required reporting, informed decisions, and analyzing acquisition outcomes. GAO's review also found that about half of an additional 138 contracts for basic services identified as performance-based did not have any of the elements intended to foster good outcomes: a performance work statement, measurable performance standards, and a quality assurance surveillance plan. DHS's Chief Procurement Officer (CPO)--who is responsible for departmentwide oversight of acquisitions--has several efforts under way to address some of these workforce and oversight issues. One initiative is an acquisition oversight program that is intended to assess (1) compliance with federal acquisition guidance, (2) contract administration, and (3) business judgment. However, this oversight program has not yet included an evaluation of the outcomes of contracting methods such as performance-based service acquisition.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To increase DHS's ability to achieve improved outcomes for its service acquisitions, including those that are performance-based, the Secretary of Homeland Security should routinely assess requirements for major, complex investments to ensure they are well-defined and develop consistently measurable standards linked to those requirements.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: DHS concurred with this recommendation and stated that it would be implemented through revisions to the DHS investment review process. DHS subsequently issued a revised management directive which provides requirements definition guidance, reestablishes the acquisition review process, and calls for department leadership to routinely assess requirements for major investments. DHS also established an Acquisition Program Management Division (now the Office of Program Accountability and Risk Management) to support these assessments, and updated the Homeland Security Acquisition Manual to clarify acquisition planning requirements. Based on these actions we are closing this recommendation as implemented. However, in June 2010 and September 2012, we reported that more work is needed to fully implement DHS's acquisition guidance, and that not all major programs had been reviewed. Further implementation of acquisition oversight initiatives should provide more insight as to whether requirements are well-defined with consistently measurable performance standards, which is reflected in more recent recommendations (see GAO-12-833).

    Recommendation: To increase DHS's ability to achieve improved outcomes for its service acquisitions, including those that are performance-based, the Secretary of Homeland Security should, at a departmentwide level, systematically evaluate the outcomes of major investments and relevant contracting methods.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: DHS concurred with this recommendation and stated it would be implemented through revisions to the DHS investment review process. DHS subsequently issued a revised management directive that requires department leadership to review all major acquisitions and their acquisition plans before they are initiated. This guidance applies to performance-based major acquisitions. Based on this action we are closing this recommendation as implemented. However, we reported in June 2010 and September 2012 that more work is needed to fully implement the guidance, which is reflected in more recent recommendations (see GAO-12-833).

    Recommendation: To increase DHS's ability to achieve improved outcomes for its service acquisitions, including those that are performance-based, the Secretary of Homeland Security should continuously improve the quality of Federal Procurement Data System-Next Generation data to facilitate the ability to accurately identify and assess the use and outcomes of various contracting methods.

    Agency Affected: Department of Homeland Security

    Status: Closed - Implemented

    Comments: DHS concurred with this recommendation, and stated that as part of the Office of the Chief Procurement Officer (OCPO) oversight reviews, OCPO validates the accuracy of the Federal Procurement Data System-Next Generation (FPDS-NG) data samples, including whether contracts have been properly coded as performance-based. DHS participated in developing a May 2008 memorandum issued by the Office of Federal Procurement Policy (OFPP) on improving FPDS data quality. The memorandum builds on and supersedes fiscal year 2007 guidance for verifying, validating, and certifying FPDS data. DHS now submits an annual certification based on statistical sampling to OFPP. Further, the DHS Data Quality Plan provides data accuracy rates for performance-based acquisitions, and since August 2008, FPDS-NG validation rules require users to capture data on performance-based acquisitions. The Data Quality Plan states that the OCPO will institutionalize frequent, systematic analysis of targeted data elements, including those for performance-based acquisitions, and will provide feedback to DHS's Heads of Contracting Activities (HCA) for follow-up corrective actions. In January 2010, DHS issued a policy memorandum to the component HCAs that required FPDS data accuracy goals to be included in contracting officer performance plans, and DHS is in the process of implementing this requirement. In January 2011, DHS certified in a memo to OFPP that the accuracy rate for FPDS data increased by 3 percent from fiscal year 2009 to fiscal year 2010. In January 2012, DHS certified in a memo to OFPP that the accuracy rate for FPDS data increased by another percent from fiscal year 2010 to fiscal year 2011, achieving an accuracy rate of 92.9 percent.

    Jul 29, 2014

    Jul 24, 2014

    Jul 16, 2014

    Jun 27, 2014

    Jun 24, 2014

    Jun 23, 2014

    Jun 18, 2014

    Looking for more? Browse all our products here