This is the accessible text file for GAO report number GAO-14-694 entitled 'Healthcare.Gov: Ineffective Planning and Oversight Practices Underscore the Need for Improved Contract Management' which was released on July 30, 2014. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Requesters: July 2014: Healthcare.Gov: Ineffective Planning and Oversight Practices Underscore the Need for Improved Contract Management: GAO-14-694: GAO Highlights: Highlights of GAO-14-694, a report to congressional requesters. Why GAO Did This Study: In March 2010, the Patient Protection and Affordable Care Act required the establishment of health insurance marketplaces by January 1, 2014. Marketplaces permit individuals to compare and select insurance plans offered by private insurers. For states that elected not to establish a marketplace, CMS was responsible for developing a federal marketplace. In September 2011, CMS contracted for the development of the FFM, which is accessed through Healthcare.gov. When initial enrollment began on October 1, 2013, many users encountered challenges accessing and using the website. GAO was asked to examine various issues surrounding the launch of the Healthcare.gov website. Several GAO reviews are ongoing. This report assesses, for selected contracts, (1) CMS acquisition planning activities; (2) CMS oversight of cost, schedule, and system capability changes; and (3) CMS actions to address contractor performance. GAO selected two task orders and one contract that accounted for 40 percent of CMS spending and were central to the website. For each, GAO reviewed contract documents and interviewed CMS program and contract officials as well as contractors. What GAO Found: The Centers for Medicare & Medicaid Services (CMS) undertook the development of Healthcare.gov and its related systems without effective planning or oversight practices, despite facing a number of challenges that increased both the level of risk and the need for effective oversight. CMS officials explained that the task of developing a first-of-its-kind federal marketplace was a complex effort with compressed time frames. To be expedient, CMS issued task orders to develop the federally facilitated marketplace (FFM) and federal data services hub (data hub) systems when key technical requirements were unknown, including the number and composition of states to be supported and, importantly, the number of potential enrollees. CMS used cost-reimbursement contracts, which created additional risk because CMS is required to pay the contractor's allowable costs regardless of whether the system is completed. CMS program staff also adopted an incremental information technology development approach that was new to CMS. Further, CMS did not develop a required acquisition strategy to identify risks and document mitigation strategies and did not use available information, such as quality assurance plans, to monitor performance and inform oversight. CMS incurred significant cost increases, schedule slips, and delayed system functionality for the FFM and data hub systems due primarily to changing requirements that were exacerbated by oversight gaps. From September 2011 to February 2014, FFM obligations increased from $56 million to more than $209 million. Similarly, data hub obligations increased from $30 million to nearly $85 million. Because of unclear guidance and inconsistent oversight, there was confusion about who had the authority to approve contractor requests to expend funds for additional work. New requirements and changing CMS decisions also led to delays and wasted contractor efforts. Moreover, CMS delayed key governance reviews, moving an assessment of FFM readiness from March to September 2013-—just weeks before the launch—and did not receive required approvals. As a result, CMS launched Healthcare.gov without verification that it met performance requirements. Late in the development process, CMS identified major performance issues with the FFM contractor but took only limited steps to hold the contractor accountable. In April and November 2013, CMS provided written concerns to the contractor about product quality and responsiveness to CMS direction. In September 2013, CMS program officials became so concerned about the contractor's performance that they moved operations to the FFM contractor's offices to provide on-site direction. At the time, CMS chose to forego actions, such as withholding the payment of fee, in order to focus on meeting the website launch date. Ultimately, CMS declined to pay about $267,000 in requested fee. This represents about 2 percent of the $12.5 million in fees paid to the FFM contractor. CMS awarded a new contract to another firm for $91 million in January 2014 to continue FFM development. As of June 2014, costs on the contract had increased to over $175 million due to changes such as new requirements and other enhancements, while key FFM capabilities remained unavailable. CMS needs a mitigation plan to address these issues. Unless CMS improves contract management and adheres to a structured governance process, significant risks remain that upcoming open enrollment periods could encounter challenges. What GAO Recommends: GAO recommends that CMS take immediate actions to assess increasing contract costs and ensure that acquisition strategies are completed and oversight tools are used as required, among other actions. CMS concurred with four recommendations and partially concurred with one. View [hyperlink, http://www.gao.gov/products/GAO-14-694]. For more information, contact William T. Woods at (202) 512-4841 or woodsw@gao.gov. [End of section] Contents: Letter: Background: Oversight Weaknesses and Lack of Adherence to Planning Requirements Compounded Acquisition Planning Challenges: Changing Requirements and Oversight Gaps Contributed to Significant Cost Growth, Schedule Delays, and Reduced Capabilities during FFM and Data Hub Development: CMS Identified Significant Contractor Performance Issues for the FFM Task Order but Took Limited Action: Conclusions: Recommendations for Executive Action: Agency Comments, Third-Party Views, and Our Evaluation: Appendix I: Objectives, Scope, and Methodology: Appendix II: Cumulative Cost Increases for the Task Orders for Developing the Federally Facilitated Marketplace System and Federal Data Services Hub Task Orders: Appendix III: Comments from the Department of Health and Human Services: Appendix IV: GAO Contact and Staff Acknowledgments: Figures: Figure 1: Timeline of Key Healthcare.gov Events: Figure 2: Overview of Healthcare.gov and Selected Supporting Systems: Figure 3: Key Contract Phases and Selected Activities: Figure 4: Cumulative Obligation Increases for the Task Orders for Developing the Federally Facilitated Marketplace System and Federal Data Services Hub: Figure 5: Planned Schedule of Development Milestone Reviews in the Federally Facilitated Marketplace System and Federal Data Services Hub Task Orders: Figure 6: Completion Status of Federally Facilitated Marketplace System Modules at the End of the Task Order, February 2014: Figure 7: Federally Facilitated Marketplace System Contractor Performance during Development: Figure 8: Cumulative Obligations for Accenture Federal Services Contract to Continue FFM Development as of June 5, 2014: Abbreviations: CCIIO: Center for Consumer Information and Insurance Oversight: CHIP: State Children's Health Insurance Program: CGI Federal: CGI Federal Inc. CMS: Centers for Medicare & Medicaid Services: COR: contracting officer's representative: data hub: federal data services hub: DOD: Department of Defense: FAR: Federal Acquisition Regulation: FFM: federally facilitated marketplace: GTL: government task leader: HHS: Department of Health and Human Services: HHSAR: Department of Health and Human Services Acquisition Regulation: IT: Information technology: IRS: Internal Revenue Service: OAGM: Office of Acquisition and Grants Management: OCIIO: Office of Consumer Information and Insurance Oversight: OIS: Office of Information Services: OPM: Office of Personnel Management: PPACA: Patient Protection and Affordable Care Act: QSSI: QSSI, Inc. VA: Department of Veterans Affairs: [End of section] United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: July 30, 2014: Congressional Requesters: The Patient Protection and Affordable Care Act (PPACA), enacted in March 2010, made fundamental changes to the availability and affordability of health insurance coverage.[Footnote 1] A central provision of the law required the establishment of state health insurance exchanges, now commonly referred to as marketplaces, by January 1, 2014. Marketplaces permit individuals to compare and select private health insurance plans. For states that elected not to establish a marketplace, PPACA required the federal government to establish and operate a federal marketplace.[Footnote 2] The Centers for Medicare & Medicaid Services (CMS) within the Department of Health and Human Services (HHS) was responsible for designing, developing, and implementing the information technology (IT) systems needed to support the federal marketplace which users access via the Healthcare.gov website. CMS largely relied on contractors to develop, build, and operate the necessary information technology systems. When initial enrollment began on October 1, 2013, many users were unable to successfully access and use the Healthcare.gov website to obtain health insurance information due to problems such as website failures, errors, and slow response times. Given the high degree of congressional interest in examining the development, launch, and other issues associated with accessing the federal marketplace through the Healthcare.gov website, GAO is conducting a body of work in order to assist Congress with its oversight responsibilities. This report examines selected contracts and task orders central to the development and launch of the Healthcare.gov website by assessing (1) CMS acquisition planning activities; (2) CMS oversight of cost, schedule, and system capability changes; and (3) actions taken by CMS to identify and address contractor performance issues. To address these objectives, we reviewed the Federal Procurement Data System-Next Generation, which is the government's procurement database, to identify CMS contracts and task orders related to the IT systems supporting the Healthcare.gov website and amounts obligated from fiscal year 2010 through March 2014. We performed data reliability assessments and confirmed that the data were sufficiently reliable for our purposes. Based on this information as well as interviews with CMS contracting and program officials, we selected one contract and two task orders issued under an existing 2007 contract for our review.[Footnote 3] The contract and task orders combined accounted for more than 40 percent of the total CMS reported obligations related to the development of Healthcare.gov and its supporting systems as of March 2014. Specifically, we selected the task orders issued to CGI Federal Inc. (CGI Federal) for the development of the federally facilitated marketplace (FFM) system and to QSSI, Inc. (QSSI) for the development of the federal data services hub (data hub) in September 2011--and the contract awarded to Accenture Federal Services in January 2014 to complete FFM development and enhance existing functionality. To assess CMS acquisition planning activities, we reviewed the Federal Acquisition Regulation (FAR) and relevant HHS/CMS policies and guidance and evaluated contract file documents. To assess CMS oversight of cost, schedule, and system capability changes, we reviewed contract modifications, contract deliverables, contractor monthly status reports, and other documents. To assess actions taken by CMS to identify and address contractor performance issues, we identified monitoring requirements and analyzed contract file documentation. To support work on all three objectives, we interviewed contracting officials in CMS's Office of Acquisition and Grants Management and program officials in CMS's Office of Information Services. In addition, we interviewed the contractors to obtain their perspective on CMS's oversight of cost, schedule, and system capabilities. Appendix I provides additional details about our scope and methodology. We conducted this performance audit from January 2014 to July 2014, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: Each marketplace created under PPACA is intended to provide a seamless, single point of access for individuals to enroll in qualified health plans,[Footnote 4] apply for income-based financial subsidies established under the law and, as applicable, obtain an eligibility determination for other health coverage programs, such as Medicaid or the State Children's Health Insurance Program (CHIP).[Footnote 5] To obtain health insurance offered through the marketplace, individuals must complete an application and meet certain eligibility requirements defined by PPACA, such as being a U.S. citizen or legal immigrant. For those consumers determined eligible, the marketplaces permit users to compare health plans and enroll in the plan of their choice. States had various options for marketplace participation, including (1) establishing their own state-based marketplace, (2) deferring to CMS to operate the federal marketplace in the state, or (3) participating in an arrangement called a partnership marketplace in which the state assists with some federal marketplace operations.[Footnote 6] In our June 2013 report on CMS efforts to establish the federal marketplace, we concluded that certain factors--such as the evolving scope of marketplace activities required in each state--suggested the potential for implementation challenges going forward.[Footnote 7] In comments on a draft of that report, HHS emphasized the progress it had made since PPACA became law and expressed its confidence that marketplaces would be open and functioning in every state on October 1, 2013. Timeline of Key Events: PPACA required the establishment of marketplaces in each state by January 2014. Based on the expectation that individuals and families would need time to explore their coverage options and plan issuers would need time to process plan selections, HHS established October 1, 2013, as the beginning of the enrollment period for all marketplaces, including the federal marketplace.[Footnote 8] Figure 1 shows a timeline of major contracting, legal or regulatory, and organizational events during that development period, as well as future milestones through the beginning of open enrollment for 2015. Figure 1: Timeline of Key Healthcare.gov Events: [Refer to PDF for image: timeline] 2010: March: [Legislation/regulation] The Patient Protection and Affordable Care Act (PPACA) signed into law requiring establishment of operational marketplaces by January 1, 2014. April: [Organization] Department of Health and Human Services (HHS) established the Office of Consumer Information and Insurance Oversight (OCIIO) to implement certain PPACA requirements including marketplace provisions. September: [Contracting] HHS awarded a contract on behalf of OCIIO to develop the business architecture for the federally facilitated marketplace (FFM) system and federal data services hub (data hub). 2011: January: [Organization] OCIIO moved to the Centers for Medicare & Medicaid Services (CMS) and becomes the Center for Consumer Information and Insurance Oversight. July: [Contracting] CMS issued the final Request for Proposal for FFM and data hub development. September: [Contracting] Task orders issued for FFM and data hub development. 2012: January: [Contracting] Task order for data hub development finalized after bid protest. March: [Legislation/regulation] HHS issued a final rule establishing October 1, 2013 as the beginning of the initial enrollment period and providing details regarding exchange establishment and operation. December: [Legislation/regulation] Deadline for states establishing a state-based marketplace to submit a declaration of intent and blueprint to CMS. 2013: February: [Legislation/regulation] Deadline for states seeking to participate in federally-facilitated partnership marketplaces to submit a declaration of intent and blueprint to CMS. July: [Legislation/regulation] Consumers began to set up initial Healthcare.gov accounts. October: [Legislation/regulation] Initial open enrollment began. 2014: Events after the October 1 launch of Healthcare.gov: January: [Contracting] CMS awarded a letter contract to further develop FFM functionality.[A] March: [Legislation/regulation] Deadline for 2014 enrollment. In some cases, this deadline was extended. April: [Contracting] CMS definitized the January 2014 letter contract.[B] November: [Legislation/regulation] Open enrollment for 2015 coverage scheduled to begin. Source: GAO analysis of the Patient Protection and Affordable Care Act, federal regulations, and Centers for Medicare & Medicaid Services data. GAO-14-694. Notes: [A] A letter contract is a written preliminary contractual instrument that authorizes the contractor to begin work immediately. FAR § 16.603. [B] A contract is considered definitized when final agreement on contract terms and conditions is reached. [End of figure] Healthcare.gov and Supporting Systems: The Healthcare.gov website is supported by several systems, including the FFM and the federal data services hub. Additional components include the Enterprise Identity Management System that confirms the consumer's identity when entering the system.[Footnote 9] Healthcare.gov Website: Healthcare.gov is the Internet address of a federal government- operated website that serves as the online user interface for the federal marketplace. The website allows the consumer to create an account, input required information, view health care plan options and make a plan selection. FFM System: The FFM accepts and processes data entered through the website and was intended to provide three main functions: * Eligibility and enrollment. This module guides applicants through a step-by-step process to determine their eligibility for coverage and financial assistance, after which they are shown applicable coverage options and have the opportunity to enroll. * Plan management. This module interacts primarily with state agencies and health plan issuers. The module is intended to provide a suite of services for activities such as submitting, monitoring, and renewing qualified health plans. * Financial management. This module facilitates payments to issuers, including premiums and cost-sharing reductions, and collects data from state-based marketplaces. Other FFM functions include services related to system oversight, communication and outreach strategies, and customer service. Federal Data Services Hub: The data hub routes and verifies information among the FFM and external data sources, including other federal and state sources of information and issuers.[Footnote 10] For example, the data hub confirms an applicant's Social Security number with the Social Security Administration and connects to the Department of Homeland Security to assess the applicant's citizenship or immigration status. The data hub's connection with other federal and state databases enables exchanges to determine whether an applicant is eligible for or enrolled in some other type of health coverage, such as the Department of Defense's (DOD) TRICARE program or Medicaid--and therefore ineligible for subsidies to offset the cost of marketplace plans. [Footnote 11] The data hub also communicates with issuers by providing enrollment information and receiving enrollment confirmation in return. See figure 2 for an overview of Healthcare.gov and selected supporting systems. Figure 2: Overview of Healthcare.gov and Selected Supporting Systems: [Refer to PDF for image: illustration] Healthcare.gov website: Enterprise Identity Management System: Federally Facilitated Marketplace System: * Eligibility & enrollment: Provides services to make eligibility determinations, plan selections and complete enrollment; * Plan management: Business processes for managing qualified health plans and issuers; * Financial management: Facilitates payments to issuers, including premiums and cost-sharing reductions, and collects data from state- based marketplaces; * Other functions. Federal Data Services Hub: Federal agencies: * Department of Defense; * Department of Homeland Security; * Internal Revenue Service; * Office of Personnel Management; * Peace Corps; * Social Security Administration; * Veterans Affairs; States: * State based marketplaces; * State Medicaid and Children's Health Insurance Program offices. Source: GAO analysis of Centers for Medicare & Medicaid Services data. GAO-14-694. [End of figure] Federal Implementation Costs: While CMS was tasked with oversight of marketplace establishment, several other federal agencies also have implementation responsibilities. Three agencies--CMS, the Internal Revenue Service (IRS), and the Department of Veterans Affairs (VA)--reported almost all of the IT-related obligations supporting the implementation of the Healthcare.gov and its supporting systems.[Footnote 12] IT-related obligations include funds committed for the development or purchase of hardware, software, and system integration services, among other activities. These obligations totaled approximately $946 million from fiscal year 2010 through March 2014, with CMS obligating the majority of this total. CMS Contracts and Task Orders for Healthcare.gov and Its Supporting Systems: As of March 2014, CMS reported obligating $840 million for the development of Healthcare.gov and its supporting systems, over 88 percent of the federal total. According to agency data, these obligations were spread across 62 contracts and task orders. We focused our review on two CMS task orders issued under an existing 2007 contract. The task orders were for the development of two core Healthcare.gov systems--the FFM and the data hub. We also reviewed a letter contract awarded by CMS in January 2014 to continue FFM development. The two task orders and the additional contract account for $369 million, or more than 40 percent, of the total CMS reported obligations as of March 2014. Acquisition Process: The contract and task orders we examined are subject to the Federal Acquisition Regulation System, which provides uniform policies and procedures for acquisition by all executive agencies. The system includes the HHS acquisition regulation, which implements or supplements the FAR. HHS's supplement to the FAR, which contain additional HHS policies and procedures, is referred to as the Department of Health and Human Services Acquisition Regulation (HHSAR). The FAR and HHSAR address issues pertaining to the contracting process and include activities related to three phases: pre-award, competition and award, and post-award. See figure 3 for an overview of these phases and selected activities related to each. Figure 3: Key Contract Phases and Selected Activities: [Refer to PDF for image: illustration] Pre-award: * Acquisition planning; * Requirements analysis; * Solicitation preparation. Competition and award: * Evaluation of offers; * Negotiation and discussion; * Selection of awardees. Post-award: * Contract administration; * Performance and monitoring; * Termination and closeout. Source: GAO analysis of Federal Acquisition Regulation. GAO-14-694. [End of figure] To implement and oversee PPACA's marketplace and private health insurance requirements, HHS established the Office of Consumer Information and Insurance Oversight (OCIIO) in April 2010 as part of the HHS Office of the Secretary. In January 2011, the OCIIO moved to CMS and became the Center for Consumer Information and Insurance Oversight (CCIIO). Within CMS, establishment of the federal marketplace was managed by CCIIO, with responsibilities shared with the Office of Information Services (OIS), and the Office of Acquisition and Grants Management (OAGM). HHS's acquisition process for the data hub and FFM task orders involved multiple participants, including: * The contracting officer. The contracting officer has the authority to enter into, administer, and/or terminate contracts and make related determinations. The contracting officer is responsible for ensuring performance of all necessary actions for effective contracting, ensuring compliance with the terms of the contract, and safeguarding the interests of the United States in its contractual relationships. * The contracting officer's representative (COR). The COR--also referred to as the contracting officer's technical representative--is designated in writing by the contracting officer to perform specific technical or administrative functions. Unlike the contracting officer, a COR has no authority to make any commitments or changes that affect price, quality, quantity, delivery, or other terms and conditions of the contract and cannot direct the contractor or its subcontractors to operate in conflict with the contract terms and conditions. * The government task leader (GTL). The GTL is a representative of the program office who assists the COR and is responsible for day-to-day technical interaction with the contractor. The GTL is also responsible for monitoring technical progress, including the surveillance and assessment of performance, and performing technical evaluations as required, among other responsibilities. Oversight Weaknesses and Lack of Adherence to Planning Requirements Compounded Acquisition Planning Challenges: CMS undertook the development of Healthcare.gov and its related systems without effective planning or oversight practices, despite facing a number of challenges that increased both the level of risk and the need for oversight. According to CMS program and contracting officials, the task of developing a first-of-its-kind federal marketplace was a complex effort that was exacerbated by compressed time frames and changing requirements. CMS contracting officials explained that meeting project deadlines was a driving factor in a number of acquisition planning activities, such as the selection of a cost-reimbursement contract, the decision to proceed with the contract award process before requirements were stable, and the use of a new IT development approach. These actions increased contract risks, including the potential for cost increases and schedule delays, and required enhanced oversight. However, CMS did not use information available to provide oversight, such as quality assurance surveillance plans. CMS also missed opportunities to consider the full range of risks to the acquisition by not developing a written acquisition strategy, even though the agency was required to do so. As a result, key systems began development with risks that were not fully identified and assessed. Acquisition Planning Activities Carried High Levels of Risk for the Government: Meeting project deadlines was a driving factor in a number of acquisition planning activities. HHS had 15 months between enactment of PPACA and the agency's request for proposal to develop requirements for the FFM and data hub. In a prior report on acquisition planning at several agencies, including HHS, we found that the time needed to complete some pre-solicitation planning activities--such as establishing the need for a contract, developing key acquisition documents such as the requirements document, the cost estimate, and, if required, the acquisition plan; and obtaining the necessary review and approvals--could be more than 2 years. The time needed depended on factors that were present for this acquisition including complexity of the requirements, political sensitivity, and funding.[Footnote 13] CMS program officials noted challenges developing requirements for a complex, first-of-its-kind system in these compressed time frames and indicated that more time was needed. The FFM and data hub task orders were issued under an existing 2007 contract for enterprise system development. This approach was reasonable in these circumstances because, according to contracting officials, the task orders could be issued more quickly than using a full and open competitive approach. The 2007 contract had been awarded to 16 vendors who were then eligible to compete for individual task orders. The 2007 contract was specifically established to improve efficiency when new IT requirement arose--such as the federal marketplace development. The 16 eligible contractors had experience with CMS's IT architecture and could come up to speed quickly. The solicitation for the 2007 contract sought contractors with experience in software design, development, testing and maintenance in complex systems environments to provide a broad range of IT services including planning, design, development, and technical support, among others. Of the 16 eligible contractors, four contractors responded with proposals for each system. CMS used a source selection process that considered both cost and non- cost factors. This type of source selection process is appropriate when it may be in the best interest of the agency to consider award to other than the lowest priced offer or the highest technically rated offer.[Footnote 14] In this case, the request for proposals indicated that cost and non-cost factors were weighted equally. The non-cost factors for technical evaluation included logical and physical design, project plan, and staffing plan, among others. In addition, CMS considered contractor past performance, but did not include that factor in the technical evaluation. CMS determined that the selected contractors for both task orders offered the most advantageous combination of technical performance and cost. Requirements for Developing the FFM System Were Not Well Defined When the Task Order Was Issued: The FAR requires that agencies ensure that requirements for services are clearly defined.[Footnote 15] In addition, in our August 2011 review of opportunities to build strong foundations for better services contracts, we found that well-defined requirements are critical to ensuring the government gets what it needs from service contractors. We also found that program and contracting officials at the four agencies we reviewed--which included HHS--noted that defining requirements can be a challenging part of acquisition planning and is a shared responsibility between program and contracting officials. [Footnote 16] Further, our March 2004 report on software-intensive defense acquisitions found that while requirements for a project can change at any point, officials must aggressively manage requirements changes to avoid a negative effect on project results, such as cost increases and schedule delays.[Footnote 17] In order to begin work quickly, CMS proceeded with the award process before FFM contract requirements, which included general technical requirements for system development, were finalized. For example, at the time the task order was issued, CMS did not yet know how many states would opt to develop their own marketplaces and how many would participate in the federally facilitated marketplace, or the size of their uninsured populations.[Footnote 18] CMS also had not completed rulemaking necessary to establish key marketplace requirements. The statement of work for the FFM acknowledged a number of these unknown requirements, for example, stating that requirements for state support were not fully known and the FFM system "must be sufficiently robust to provide support of state exchange requirements at any point in the life cycle." In addition, the FFM statement of work noted that the requirements related to a number of FFM services would be finalized after contract award, including services related to all three main functional areas--eligibility and enrollment, financial management, and plan management--as well as system oversight, communication, and customer service. The technical requirements for both the FFM and data hub were developed by CMS staff with contractor support[Footnote 19] and documented in a statement of work for each task order.[Footnote 20] Both statements called for the contractor to design a "solution that is flexible, adaptable, and modular to accommodate the implementation of additional functional requirements and services." However, according to CMS program officials, requirements for data hub development were more clearly defined at the time that task order was issued than FFM requirements. These officials also stated that, prior to issuing the task order, CMS was able to develop a prototype for the data hub and a very clear technical framework to guide the contractor, but due to still-changing requirements, CMS could not provide the same guidance for FFM development. We have previously found that unstable requirements can contribute to negative contract outcomes, including cost overruns and schedule delays.[Footnote 21] CMS Used a Contract Type That Carried Risk for the Government and Required Additional Oversight: In response to unsettled requirements, CMS contracting officials selected a type of cost reimbursement contract known as a cost-plus- fixed-fee contract for both the FFM and data hub task orders. According to the FAR, these contracts are suitable when uncertainties in requirements or contract performance do not permit the use of other contract types.[Footnote 22] Under a cost reimbursement contract, the government pays all of the contractor's allowable incurred costs to the extent prescribed in the contract. These contracts are considered high risk for the government because of the potential for cost escalation and because the government pays a contractor's allowable cost of performance regardless of whether the work is completed. In recent years, the federal government has taken steps to minimize the use of cost reimbursement contracts.[Footnote 23] While CMS's use of the cost-plus-fixed-fee contract type may have been a reasonable choice under the circumstances, the related risks increased the need for oversight. In our November 2007 report on internal control deficiencies at CMS, we found that certain contracting practices, such as the frequent use of cost reimbursement contracts, increased cost risks to CMS because CMS did not implement sufficient oversight for cost reimbursement contracts at that time.[Footnote 24] However, in planning documents for the two task orders, CMS acknowledged the increased responsibilities and risks associated with managing a cost reimbursement contract and included a number of oversight elements in the task orders to support contract oversight and manage risks. These elements included contract deliverables such as earned value management reports,[Footnote 25] monthly financial and project status reports, and a quality assurance surveillance plan.[Footnote 26] Both task orders required that a quality assurance surveillance plan be provided within 45 days after award. This plan is intended to ensure that systematic quality assurance methods are used in administration of the contract and provides for government oversight of the quality, quantity, and timeliness of contractor performance. The FAR requires that contract quality assurance be performed as may be necessary to determine that the supplies or services conform to contract requirements.[Footnote 27] However, we found that the quality assurance surveillance plans were not used to inform oversight. For example, contracting and program officials, including the COR and contracting officer, were not sure if the quality assurance surveillance plan had been provided as required by the FFM and data hub task orders. Although a copy was found by CMS staff in June 2014, officials said they were not aware that the document had been used to review the quality of the contractor's work. Instead, CMS program officials said they relied on their personal judgment and experience to determine quality. CMS Selected a New IT Development Approach to Save Time, but Increased Risks: To help manage compressed time frames for FFM and data hub development, CMS program officials adopted an iterative IT development approach called Agile that was new to CMS. Agile development is a modular and iterative approach that calls for producing usable software in small increments, sometimes referred to as sprints, rather than producing a complete product in longer sequential phases. [Footnote 28] The Office of Management and Budget issued guidance in 2010 that advocated the use of shorter delivery time frames for federal IT projects, an approach consistent with Agile.[Footnote 29] However, CMS program officials acknowledged that when FFM and data hub development began in September 2011, they had limited experience applying an Agile approach to CMS IT projects. In 2011, CMS developed updated guidance to incorporate the Agile IT development approach with its IT governance model, but that model still included sequential reviews and approvals and required deliverables at pre-determined points in the project. In our July 2012 report, we found a number of challenges associated with introducing Agile in the federal environment.[Footnote 30] Specifically, we found that it was difficult to ensure that iterative projects could follow a standard, sequential approach and that deviating from traditional procedural guidance to follow Agile methods was a challenge. We also reported that new tools and training may be required, as well as updates to procurement strategies. Therefore, the new approach that CMS selected in order to speed work also carried its own implementation risks. CMS Did Not Fully Adhere to HHS Acquisition Planning Requirements and Missed Opportunities to Capture and Consider Risks Important to the Program's Success: While a number of CMS's acquisition planning actions were taken in an effort to manage acquisition challenges, CMS missed opportunities to fully identify and mitigate the risks facing the program. HHS acquisition policy requires the development of a written acquisition strategy for major IT investments, such as the FFM system.[Footnote 31] According to HHS policy, an acquisition strategy documents the factors, approach, and assumptions that guide the acquisition with the goal of identifying and mitigating risks.[Footnote 32] HHS provides a specific acquisition strategy template that requires detailed discussion and documentation of multiple strategy elements, including market factors and organizational factors, among others. According to program officials, the acquisition planning process for the FFM and data hub task orders began in 2010, prior to HHS's decision to move its Office of Consumer Information and Insurance Oversight (OCIIO) to CMS, and continued into early 2011. Program officials stated that the planning process included discussions of an acquisition strategy. However, CMS program and contracting staff did not complete the required acquisition strategy for FFM and data hub development. According to contracting and program officials, CMS has not been preparing acquisition strategies for any of its major IT acquisitions, not just those related to systems supporting Healthcare.gov. This is a longstanding issue. In November 2009 we found deficiencies in CMS contract management internal controls practices such as the failure to follow existing policies and the failure to maintain adequate documentation in contract files.[Footnote 33] According to CMS contracting officials, CMS is planning steps to strengthen the agency's program and project management, including training related to the acquisition strategy requirement. Contracting officials from OAGM explained that at CMS the majority of acquisition planning is done by the program office and OAGM began discussions of the upcoming task orders related to Healthcare.gov and its supporting systems with program officials in February 2011. In June 2011, OAGM accepted a Request for Contract package--a set of documents used to request and approve a contract action--from the program office. The package documents some elements of an acquisition strategy. Specifically, it indicated the type of contract to be used and the selected contract approach; however, the documents do not include the rationale for all decisions and did not address a number of planning elements required in HHS acquisition strategy, such as organizational factors, technological factors, and logistics. In the absence of an acquisition strategy, key risks and plans to manage them were not captured and considered as required. The acquisition strategy provides an opportunity to highlight potential risk areas and identify ways to mitigate those risks. For example, the strategy guidance requires the consideration of organizational factors that include management and their capabilities, available staff and their skills, and risks associated with the organizational structure. Organizational factors were a potential risk area for these projects because the CMS organizations responsible for the FFM and data hub experienced significant changes just prior to and during the planning period. Specifically, OCIIO was established in 2010 and integrated into CMS in January 2011, just prior to the beginning of planning discussions with OAGM. According to CMS contracting and program officials, some of the 246 OCIIO staff transitioned to the new CCIIO and others joined CMS's Office of Information Services (OIS) and OAGM. In the context of these organizational changes and the other considerable project risks, the acquisition strategy could have been a powerful tool for risk identification and mitigation. By failing to adhere to this requirement, CMS missed opportunities to explain the rationales for acquisition planning activities and to fully capture and consider risks important to the success of the program. Changing Requirements and Oversight Gaps Contributed to Significant Cost Growth, Schedule Delays, and Reduced Capabilities during FFM and Data Hub Development: CMS incurred significant cost increases, schedule slips, and reduced system functionality in the development of the FFM and data hub systems--primarily attributable to new and changing requirements exacerbated by inconsistent contract oversight. From September 2011 to February 2014, estimated costs for developing the FFM increased from an initial obligation of $56 million to more than $209 million; similarly, data hub costs increased from an obligation of $30 million to almost $85 million. New and changing requirements drove cost increases during the first year of development, while the complexity of the system and rework resulting from changing CMS decisions added to FFM costs in the second year. In addition, required design and readiness governance reviews were either delayed or held without complete information and CMS did not receive required approvals. Furthermore, inconsistent contractor oversight within the program office and unclear roles and responsibilities led CMS program staff to inappropriately authorize contractors to expend funds. FFM and Data Hub Task Orders Experienced Significant Increases: Obligations for both the FFM and data hub rose significantly during the two-and-a-half-year development period, with the FFM task order increasing almost four-fold, from $55.7 million obligated when issued in late 2011 to more than $209 million obligated by February 2014. Similarly, the data hub task order almost tripled, increasing from $29.9 million to $84.5 million during the same period.[Footnote 34] Figure 4 shows FFM and data hub obligation growth during this time. Figure 4: Cumulative Obligation Increases for the Task Orders for Developing the Federally Facilitated Marketplace System and Federal Data Services Hub: [Refer to PDF for image: interactive graphic] Rollover green and light blue circles for more information. Please see appendix II for the print version. Source: GAO analysis of Centers for Medicare & Medicaid Services data. GAO-14-694. [End of figure] New and Changing Requirements Drove Cost Increases throughout System Development: Development cost increases for the FFM and data hub were due to a combination of factors, including costs associated with adding or changing requirements. For example, CMS was aware that a number of key business requirements for the FFM and data hub would not be known until after the task orders were issued in September 2011, and it acknowledged some of these uncertainties in the statements of work, such as noting that the actual number of states participating in the federal marketplace and the level of support each state required was not expected to be known until January 2013. We previously found in March 2004 that programs with complex software development experienced cost increases and schedule delays when they lacked controls over their requirements, noting that leading software companies found changing requirements tend to be a major cause of poor software development outcomes.[Footnote 35] Subsequent modifications to the FFM and data hub task orders show the costs associated with adding requirements beyond those initial uncertainties. For example, CMS obligated an additional $36 million to the FFM and $23 million to the data hub in 2012, in large part to address requirements that were added during the first year of development, such as increasing infrastructure to support testing and production and adding a transactional database. Some of these new requirements resulted from regulations and policies that were established during this period. For example, in March 2012, federal rulemaking was finalized for key marketplace functions, resulting in the need to add services to support the certification of qualified health plans for partnership marketplace states. Other requirements emerged from stakeholder input, such as a new requirement to design and implement a separate server to process insurance issuers' claims and enrollment data outside of the FFM. CMS program officials said that this resulted from health plan issuers' concerns about storing proprietary data in the FFM. The FFM and data hub task orders were both updated to include this requirement in 2012, which was initially expected to cost at least $2.5 million. System Complexities and Rework Further Added to FFM Costs in the Second Year: During the second year of development, from September 2012 to September 2013, the number of task order modifications and dollars obligated for the development of the FFM and data hub continued to increase. New requirements still accounted for a portion of the costs, but the second-year increases also reflected the previously unknown complexities of the original requirements and associated rework, particularly for the FFM. For example, according to the FFM contractor, one of the largest unanticipated costs came from CMS' directions to purchase approximately $60 million in software and hardware that was originally expected to be provided by another Healthcare.gov contractor. Most of these costs were added through task order modifications in 2013. In April 2013, CMS added almost $28 million to the FFM task order to cover work that was needed because of the increasingly complex requirements, such as additional requirements to verify income for eligibility determination purposes. The FFM contractor said some of these costs resulted from CMS's decisions to start product development before regulations and requirements were finalized, and then to change the FFM design as the project was ongoing, which delayed and disrupted the contractor's work and required them to perform rework. In addition, CMS decisions that appeared to be final were reopened, requiring work that had been completed by the contractor to be modified to account for the new direction. This included changes to various templates used in the plan management module and the application used by insurance issuers, as well as on-going changes to the user interface in the eligibility and enrollment module. According to the FFM contractor, CMS changed the design of the user interface to match another part of the system after months of work had been completed, resulting in additional costs and delays. In November 2012, the contractor estimated that the additional work in the plan management module alone could cost at least $4.9 million. By contrast, CMS program officials explained that the data hub generally had more stable requirements than the FFM, in part due to its functions being less technically challenging and because CMS had had more time to develop the requirements. While the obligations for the data hub also increased at the same rate as the FFM in the first year of development, they did so to a lesser degree during the second year. According to the data hub contractor, these increases were due to CMS-requested changes in how the work was performed, which required additional services, as well as hardware and software purchases. CMS Experienced Schedule Delays, Conducted Incomplete Governance Oversight Reviews, and Delayed Some Capabilities for the FFM and Data Hub: In addition to increased costs, the FFM and data hub experienced schedule delays, which contributed to CMS holding incomplete governance oversight reviews and eventually reduced the capabilities it expected the FFM contractor to produce by the October 1, 2013, deadline. CMS Delayed Scheduled Governance Reviews, Reducing Time Available for FFM and Data Hub Testing and Implementation Reviews: CMS initially established a tight schedule for reviewing the FFM and data hub development in order to meet the October 1, 2013, deadline for establishing enrollment through the website. Each task order lists the key governance reviews that the systems were required to meet as they progressed through development. The FFM and data hub task orders initially required the contractors to be prepared to participate in most of the CMS governance reviews-- including a project baseline and final detailed design reviews--within the first 9 months of the awards. This would allow CMS to hold the final review needed to implement the systems--operational readiness-- at least 6 months before the Healthcare.gov launch planned for October 1, 2013. In April 2013, CMS extended the requirements analysis and design phase. According the CMS program officials, requirements were still changing and more time was needed to finalize the FFM design. As a result, CMS compressed time frames for conducting reviews for the testing and implementation phases. Under the revised schedule, the contractor had until the end of September 2013--immediately prior to the date of the planned launch--to complete the operational readiness review, leaving little time for any unexpected problems to be addressed despite the significant challenges the project faced. Figure 5 shows the schedule of planned and revised development milestone reviews in the FFM and data hub task orders. Figure 5: Planned Schedule of Development Milestone Reviews in the Federally Facilitated Marketplace System and Federal Data Services Hub Task Orders: [Refer to PDF for image: illustration] Federal data services hub: Initiation, concept and planning: * Architecture review, 03/2012; * Project baseline review, 03/2012. Requirements, analysis and design: * Final detailed design review, 06/2012. Development and test: * Operational readiness review, 09/2012. Implementation. Open enrollment begins, 10/2013. Federally Facilitated Marketplace System – Initial plan (September 2011): Initiation, concept and planning: * Architecture review, 10/2011; * Project baseline review, 03/2012. Requirements, analysis and design: * Final detailed design review, 06/2012. Development and test: * Operational readiness review, 03/2013. Implementation. Open enrollment begins, 10/2013. Federally Facilitated Marketplace System – Revised plan (April 2013): Initiation, concept and planning: * Architecture review, 12/2011; [Revised date] * Project baseline review, 03/2012. Requirements, analysis and design: * Final detailed design review, 03/2013. [Revised date] Development and test: * Operational readiness review, 09/2013. [Revised date] Implementation. Open enrollment begins, 10/2013. 1. Architecture review: Ensures that business needs are sound and conform to the Centers for Medicare & Medicaid Services (CMS) information technology architecture before design begins. This review also examines the proposed projects' relationship with existing systems and determines if they can be leveraged. 2. Project baseline review: Obtain management approval that the project's scope, cost, and schedule have been established and an appropriate management strategy is in place. This review also includes an assessment of risks to each baseline. 3. Final detailed design review: Ensures that the proposed design meets the stated business needs and is complete according to CMS standards prior to development. This review also raises and resolves any issues that might pose risks during subsequent phases, such as testing and implementation, and which may be more costly or require additional time to correct in the later stages. 4. Operational readiness review: Ensures that a system has completed its implementation process according to plan and is ready to enter the operations and maintenance phase. Security is a significant component of this review. Source: GAO analysis of Centers for Medicare & Medicaid Services data. GAO-14-694. [End of figure] The four reviews shown in figure 5--architecture, project baseline, final detailed design, and operational readiness--are among those required under the exchange life cycle framework, the governance model CMS specifically designed to meet the need to quickly develop the FFM and data hub using the Agile development approach.[Footnote 36] The life cycle framework requires technical reviews at key junctures in the development process, such as a final detailed design review to ensure that the design meets requirements before it is developed and tested. To accommodate different development approaches, the life cycle framework allows program offices leeway regarding how some reviews are scheduled and conducted, permitting more informal technical consultations when holding a formal review would cause delays. However, the framework requires that the four governance or milestone reviews be approved by a CMS governance board. Some Governance Reviews Were Not Fully Conducted or Approved: Despite the revised FFM schedule, it is not clear that CMS held all of the governance reviews for the FFM and data hub or received the approvals required by the life cycle framework. The framework was developed to accommodate multiple development approaches, including Agile. A senior CMS program official said that although the framework was used as a foundation for their work, it was not always followed throughout the development process because it did not align with the modified Agile approach CMS had adopted. CMS program officials explained that they held multiple reviews within individual development sprints--the short increments in which requirements are developed and software is designed, developed, and tested to produce a building block for the final system. However, CMS program officials indicated that they were focused on responding to continually changing requirements which led to them participating in some governance reviews without key information being available or steps completed. Significantly, CMS held a partial operational readiness review for the FFM in September 2013, but development and testing were not fully completed and continued past this date. As a result, CMS launched the FFM system without the required verification that it met performance requirements. Furthermore, the life cycle framework states that CMS must obtain governance-board approval before the systems proceed to the next phase of development, but we did not see evidence that any approvals were provided. CMS records show that CMS held some governance reviews, such as design readiness reviews. However, the governance board's findings identified outstanding issues that needed to be addressed in subsequent reviews and they were not approved to move to the next stage of development. CMS Postponed Some FFM Capabilities to Meet Deadlines: By March 2013, CMS recognized the need to extend the task orders' periods of performance in order to allow more time for development. CMS contract documents from that time estimated that only 65 percent of the FFM and 75 percent of the data hub would be ready by September 2013, when development was scheduled to be completed. Recognizing that neither the FFM nor the data hub would function as originally intended by the beginning of the initial enrollment period, CMS made trade-offs in an attempt to provide necessary system functions by the October 1, 2013, deadline. Specifically, CMS prioritized the elements of the system needed for the launch, such as the FFM eligibility and enrollment module, and postponed the financial module, which would not be needed until post-enrollment. CMS also delayed elements such as the Small Business Health Options Program marketplace, initially until November 2013, and then until 2015. See figure 6 for the modules' completion status as of the end of the task order in February 2014. Figure 6: Completion Status of Federally Facilitated Marketplace System Modules at the End of the Task Order, February 2014: [Refer to PDF for image: illustration] Federally Facilitated Marketplace System (FFM): Eligibility & enrollment: Provide services to make eligibility determinations, plan selections and complete enrollment: * Partially complete: Some elements were developed and working, but functions such as enrollment reprocessing will be completed under a new contract. Plan management: Business processes for managing qualified health plans and issuers: * Complete: This module was mostly developed and can be used in the FFM. Financial management: Facilitates payments to issuers, including premiums and cost-sharing reductions, and collects data from state-based marketplaces: * Not complete: The least developed module. Key functions, including risk adjustment, are being developed under a new contract. Source: GAO analysis of Centers for Medicare & Medicaid Services data. GAO-14-694. [End of figure] In September 2013, CMS extended the amount of time allotted for development under the FFM and data hub task orders, which accounted for the largest modifications. The additional obligations--$58 million for the FFM and $31 million for the data hub--included some new elements, such as costs associated with increasing FFM capacity needed to support anticipated internet traffic, but our review of the revised statements of work show that the additional funding was primarily for the time needed to complete development work rather than new requirements. After the FFM was launched on October 1, 2013, CMS took a number of steps to respond to system performance issues through modifications to the FFM task order. These efforts included adding more than $708,000 to the FFM task order to hire industry experts to assess the existing system and address system performance issues. CMS also greatly expanded the capacity needed to support internet users, obligating $1.5 million to increase capacity from 50 terabytes to 400 terabytes for the remainder of the development period. While CMS program officials said that the website's performance improved, only one of the three key components specified in the FFM task order was completed by the end of the task order's development period. (See figure 6.) According to program officials, the plan management module was complete, but only some of the elements of the eligibility and enrollment module were provided and the financial management remained unfinished. Unclear Contract Oversight Responsibilities Exacerbated FFM and Data Hub Cost Growth: We identified approximately 40 instances during FFM development in which CMS program staff inappropriately authorized contractors to expend funds totaling over $30 million because those staff did not adhere to established contract oversight roles and responsibilities. Moreover, CMS contract and program staff inconsistently used and reviewed contract deliverables on performance to inform oversight. CMS Staff Inappropriately Authorized Contractors to Expend Funds: The FFM task order was modified in April 2013 to add almost $28 million to cover cost increases that had been inappropriately authorized by CMS program officials in 2012.[Footnote 37] This issue also affected the data hub task order, which had an estimated $2.4 million cost increase over the same period. In November 2012, the FFM contractor informed CMS of a potential funding shortfall due to work and hardware that CMS program officials had directed the contractor to provide. The FAR provides that the contracting officer is the only person authorized to change the terms and conditions of the contract. Further, other government personnel shall not direct the contractor to perform work that should be the subject of a contract modification. [Footnote 38] The federal standards for internal control also state that transactions and significant events need to be authorized and executed by people acting within the scope of their authority, to ensure that only valid transactions to commit resources are initiated. [Footnote 39] CMS documents show that the cost growth was the result of at least 40 instances in which work was authorized by various CMS program officials, including the government task leader (GTL)--who is responsible for day-to-day technical interaction with the contractor-- and other staff with project oversight responsibilities, who did not have the authority to approve the work. This was done without the knowledge of the contracting officer or the contracting officer's representative. This inappropriately authorized work included adding features to the FFM and data hub, changing designs in the eligibility and enrollment module, and approving the purchase of a software license. CMS later determined that the work was both necessary and within the general scope of the task order but the cost of the activities went beyond the estimated cost amount established in the order and thus required a modification. Inappropriate Authorizations Due to Unclear Oversight Responsibilities: A senior CMS program official described a three-pronged approach to contract oversight that involved various CMS offices, including the COR and GTL in the program offices, and the contracting officer in OAGM. The COR and GTL were assigned overlapping responsibilities for monitoring the contractor's technical performance, but CMS's guidance to clarify their roles did not fully address the need to ensure that directions given to contractors were appropriate. CMS program officials said the guidance was issued in 2006, several years before the FFM and data hub task orders were issued. The guidance generally noted that CORs are responsible for financial and contractual issues while GTLs have day-to-day technical interactions with the contractors. However, the guidance did not clarify the limitations on COR's and GTL's authorities, such as not providing contractors with technical direction to perform work outside the scope of the contract. CMS program officials also described difficulties clarifying oversight responsibilities in organizations that were new to CMS, which contributed to the inappropriately authorized work. Program responsibilities were shared between CCIIO, which was primarily responsible for developing business requirements, and the information technology staff in OIS, where the GTL and COR were located. CCIIO was relatively new to CMS, having been incorporated shortly before the FFM and data hub task orders were issued. OIS program officials explained that CCIIO was not as experienced with CMS's organization and did not strictly follow their processes, including for oversight. CMS documents show that there were concerns about inappropriate authorizations prior to the cost growth identified in late 2012, as officials in the OIS acquisition group had repeatedly cautioned other OIS and CCIIO staff about inappropriately directing contractors. Furthermore, CMS program officials said that CCIIO staff did not always understand the cost and schedule ramifications associated with the changes they requested. As the FFM in particular was in the phase of development in which complexities were emerging and multiple changes were needed, there were a series of individual directions that, in sum, exceeded the expected cost of the contract. As a result of the unauthorized directions to contractors, the CMS contracting officer had to react to ad hoc decisions made by multiple program staff that affected contract requirements and costs rather than directing such changes by executing a contract modification as required by the FAR. In April 2013, shortly after the inappropriate authorizations and related cost increases for the FFM and data hub task orders were identified, a senior contracting official at CMS sent instructions on providing technical directions to contractors to the program offices that had been involved in the authorizations and to CMS directors in general. Specifically, the program offices were reminded to avoid technical direction to contractors--particularly when there is an immediate need for critical functions--which might constitute unauthorized commitments by the government. This instruction has not been incorporated into existing guidance on the roles and responsibilities of the CORs and GTLs. CMS contracting and program officials also reported additional steps to bolster contract oversight such as reminding the FFM contractor not to undertake actions that result in additional costs outside of the statement of work without specific direction from the contracting officer. CMS Provided Inconsistent Oversight of Contract Performance: It was not always clear which CMS officials were responsible for reviewing and accepting contractor deliverables, including items such as the required monthly status and financial reports and the quality assurance surveillance plan that aid the government in assessing the costs and quality of the contractor's work. According to contracting officials, reviewing such deliverables helped to provide the additional oversight that cost-reimbursable task orders require per the FAR to reduce risks of cost growth. However, particularly in the first year of FFM development, contract documentation shows repeated questions about who was responsible for reviewing the deliverables and difficulties finding the documents. Both task orders were ultimately modified to require that deliverables be provided to the contracting officer, who had previously just been copied on transmittal letters, in addition to the program office. In September 2012, the COR oversight function transferred to the acquisition group within CMS's OIS and a new COR was assigned to manage both the FFM and data hub task orders. A CMS program official explained that the acquisition group typically fulfills the COR role for CMS contracts and that it had been unusual for those functions to be provided by another office. Upon assuming oversight responsibilities, the new COR could not locate a complete set of FFM and data hub deliverables and the original COR was unable to provide them. Instead, the new COR had to request all monthly status and financial reports directly from the contractors. When the new COR began reviewing the reports in the fall of 2012, he said he noticed that the FFM contractor had not been projecting the burn rate, a key measure that shows how quickly money is being spent. The COR asked the contractor to provide the figures in November 2012, at which point the cost growth was identified, even though the contract had been modified in August 2012 to add almost $36 million to the task order. We found that the burn rate was not included in earlier reports, but its absence had gone unnoticed due to ineffective contract oversight. In November 2007, we had found internal control deficiencies at CMS related to the inadequate review of contractor costs.[Footnote 40] CMS Identified Significant Contractor Performance Issues for the FFM Task Order but Took Limited Action: CMS took limited action to address significant FFM contractor performance issues as the October 1, 2013, deadline for establishing enrollment through the website neared, and ultimately hired a new contractor to continue FFM development. Late in the development process, CMS became increasingly concerned with CGI Federal's performance. In April and November 2013, CMS provided written concerns to CGI Federal regarding its responsiveness to CMS's direction and FFM product quality issues. In addition, in August 2013, CMS was prepared to take action to address the contractor's performance issues that could have resulted in withholding of fee; however, CMS ultimately decided to work with CGI Federal to meet the deadline. CMS contracting and program officials stated that the contract limited them to only withholding fee as a result of rework. Ultimately, CMS declined to pay only about $267,000 of requested fee. This represented about 2 percent of the $12.5 million in fee paid to CGI Federal. Rather than pursue the correction of performance issues with CGI Federal, in January 2014 CMS awarded a new one-year contract to Accenture Federal Services for $91 million to continue FFM development. This work also has experienced cost increases due to new requirements and other enhancements, with costs increasing to over $175 million as of June 2014. CMS Deemed Early Contractor Performance Satisfactory and Took Limited Action to Address Significant Contractor Performance Issues as the Deadline Neared: CMS generally found CGI Federal and QSSI's performance to be satisfactory in September 2012, at the end of the first year of development. CMS noted some concerns related to FFM contractor performance, such as issues completing development and testing on time; however, CMS attributed these issues to the complexity of the FFM and CMS's changing requirements and policies.[Footnote 41] Further, according to program officials, during the first year of FFM development, few defined products were to be delivered as requirements and the system's design were being finalized. For example, as previously identified in this report, under the revised FFM development schedule the final detailed design review for the FFM--a key development milestone review to ensure that the design meets requirements before it is developed and tested, was delayed from June 2012 to March 2013. Therefore, CMS had limited insight into the quality of CGI Federal's deliverables during the first year as development and testing of certain FFM functionality had not yet been completed. CMS found QSSI's performance satisfactory in September 2012. CMS program officials told us that they did not identify significant contractor performance issues during data hub development, and that the data hub generally worked as intended when Healthcare.gov was launched on October 1, 2013. CMS Identified Significant FFM Contractor Performance Issues as the Deadline Approached, but CMS Opted Against Taking Remedial Contractual Actions at That Time: During the second year of development, which began in September 2012, CMS identified significant FFM contractor performance issues as the October 1 deadline approached (see figure 7). In April 2013, CMS identified concerns with CGI Federal's performance, including not following CMS's production deployment processes and failing to meet established deadlines, as well as continued communication and responsiveness issues. To address these issues, the contracting officer's representative (COR) sent an email to CGI Federal outlining CMS's concerns and requesting that CGI Federal provide a plan for correcting the issues moving forward. CMS accepted CGI Federal's mitigation plan. The plan included changes, according to CGI Federal officials, to accommodate CMS' communication practices, which CGI Federal believed to be the root cause of some of the CMS-identified issues. CMS contracting officials said that they were satisfied with CGI Federal's overall mitigation approach, which seemed to address the performance issues that CMS had identified at that time. Figure 7: Federally Facilitated Marketplace System Contractor Performance during Development: [Refer to PDF for image: illustration] 2011: Dollars obligated: $55.7 million. 2012: September: Centers for Medicare & Medicaid Services (CMS) rated CGI Federal Inc.'s (CGI Federal) performance satisfactory from 9/2011 - 9/2012. 2013: April: CMS requested that CGI Federal provide a plan to manage communication issues. August: CMS sent a letter to CGI Federal requesting a plan to address a number of technical issues. CMS management pulled the letter back the next day. September: CMS identified deep-rooted problems with critical software defects during testing and demonstration. CMS temporarily relocated several staff members to CGI Federal's location in Herndon, Virginia to provide on-site direction leading up to the website launch. November: CMS sent a letter to CGI Federal requesting a plan to address multiple technical issues. December/January: CMS did not pay a portion of CGI Federal's fee for rework. 2014: January: CMS replaced CGI Federal with Accenture Federal Services, awarding a letter contract to further develop and enhance existing federally facilitated marketplace system functionality. 02/2014[A]: Dollars obligated: $209.6 million. Source: GAO analysis of Centers for Medicare & Medicaid Services data. GAO-14-694. Notes: [A] The development period of performance ended in February 2014, and CMS chose not to exercise option years provided for in the task order. [End of figure] According to CMS program officials, they grew increasingly concerned with CGI Federal's performance late in the development process in June and July 2013 as the scheduled launch date approached. Specifically, CMS program officials identified concerns with FFM technical and code quality during early testing of the enrollment process. The initial task order schedule had called for the FFM's development and test phase to be complete by this point, but these efforts were delayed in the revised schedule. CMS program officials explained that they identified issues such as inconsistent error handling, timeouts, and pages going blank. Overall, more than 100 defects were identified, which resulted in delays while CGI Federal worked to correct them. According to CGI Federal officials, the code reflected the instability of requirements at that time. However, once requirements were more stable, after October 2013, the contractor was able to quickly make improvements to the FFM's performance. In August 2013, CMS contracting and program officials decided to take formal action to address their concerns with CGI Federal's performance by drafting a letter to the contractor. Specifically, CMS identified concerns with the contractor's code quality, testing, failure to provide a key deliverable, and scheduled releases not including all agreed upon functionality. The letter further stated that CMS would take aggressive action, such as withholding fee in accordance with the FAR, if CGI Federal did not improve or if additional concerns arose. However, the contracting officer withdrew the letter one day after it was sent to CGI Federal, after being informed that the CMS Chief Operating Officer preferred a different approach. CMS contracting and program officials told us that, rather than pursue the correction of performance issues, the agency determined that it would be better to collaborate with CGI Federal in completing the work needed to meet the October 1, 2013, launch. CMS contracting officials told us that the agency did not subsequently take any remedial actions to address the issues outlined in the August 2013 letter. By early September 2013, CMS program officials told us that they became so concerned about the contractor's performance that CMS program staff moved their operations to CGI Federal's location in Herndon, Virginia to provide on-site direction leading up to the FFM launch. CMS had identified issues such as deep-rooted problems with critical software defects during testing and demonstration of the product and CGI Federal's inability to perform quality assurance adequately including full testing of software. According to CMS program officials, CMS staff members worked on-site with CGI Federal for several weeks to get as much functionality available by October 1, 2013, as possible, deploying fixes and new software builds daily. CMS Took Some Actions to Hold the FFM Contractor Accountable after the Healthcare.gov Launch: After the Healthcare.gov launch on October 1, 2013, CMS contracting officials began preparing a new letter detailing their concerns regarding contractor performance which was sent to CGI Federal in November 2013. In its letter, CMS stated that CGI Federal had not met certain requirements of the task order statement of work, such as FFM infrastructure requirements including capacity and infrastructure environments, integration, change management, and communication issues--some of which had been previously expressed in writing to CGI Federal. In addition, CMS stated that some of these issues contributed to problems that Healthcare.gov experienced after the October 1, 2013 launch. CMS's letter also requested that CGI Federal provide a plan to address these issues. CGI Federal responded in writing, stating that it disagreed with CMS's assertion that CGI Federal had not met the requirements in the FFM statement of work. In its letter, CGI Federal stated that delays in CMS's establishment and finalization of requirements influenced the time available for development and testing of the FFM. CGI Federal further stated that disruptions to its performance as a result of delays in finalizing requirements were compounded by the scheduled launch date, which resulted in CMS reprioritizing tasks and compressing time frames to complete those tasks. CGI Federal officials said they did not provide a formal plan for addressing CMS's concerns because they regarded them as unfounded, but agreed to work with CMS to avoid future issues and optimize the FFM's performance.[Footnote 42] In addition, after the October 1, 2013, launch, CMS contracting officials told us that they provided additional FFM oversight by participating in daily calls with CGI Federal on the stability of the FFM and the status of CGI Federal's work activities. Contracting officials told us that the increased oversight of FFM development helped to fix things more quickly. Further, the COR increasingly issued technical direction letters to clarify tasks included in the FFM statement of work and focus CGI Federal's development efforts. [Footnote 43] For example, CMS issued several technical direction letters to CGI Federal in October 2013, directing CGI Federal to follow the critical path for overall performance improvement of the FFM, purchase software licenses, and collaborate with other stakeholders, among other things. According to program officials, written technical direction letters issued by the COR had more authority than technical direction provided by the GTL. CMS Declined to Pay FFM Contractor Fee for Rework: CMS contracting and program officials explained that they found it difficult to withhold the contractor's fee under FAR requirements. As discussed earlier in this report, the development work for the FFM was conducted through a cost-plus-fixed-fee task order, through which the government pays the contractor's allowable costs, plus an additional fee that was negotiated at the time of award. This means that despite issues with CGI Federal's performance, including CGI Federal's inability to deliver all functionality included in the FFM statement of work, CMS was required to pay CGI Federal for allowable costs under the FFM task order. CGI Federal's task order provides that, if the services performed do not conform with contract requirements, the government may require the contractor to perform the services again for no additional fee.[Footnote 44] If the work cannot be corrected by re-performance, the government may, by contract or otherwise, perform the services and reduce any contractor's fee by an amount that is equitable under the circumstances, or the government may terminate the contract for default.[Footnote 45] Even though CMS was obligated to pay CGI Federal's costs for the work it had performed for the FFM, CMS contracting and program officials said they could withhold only the portion of the contractor's fee that it calculated was associated with rework to resolve FFM defects. Ultimately, CMS declined to pay about $267,000 of the fixed fee requested by CGI Federal. This is approximately 2 percent of the $12.5 million in fixed fee that CMS paid to CGI Federal. Officials from CGI Federal said that they disagreed with the action and that the CMS decisions were not final and they could reclaim the fee by supplying additional information. CMS contracting and program officials told us that it was difficult to distinguish rework from other work. For example, program officials explained that it was difficult to isolate work that was a result of defects versus other work that CGI Federal was performing, and then calculate the corresponding portion of fee to withhold based on hours spent correcting defects. Contractor's Total Fee Increased during Development: Through each contract modification, as CMS increased the cost of development, it also negotiated additional fixed fee for the FFM and data hub contractors. Under the original award of $55.7 million, CGI Federal would have received over $3.4 million in fee for work performed during the development period. As of February 2014, when CMS had obligated over $209 million dollars for the FFM effort, CMS negotiated and CGI Federal was eligible to receive more than $13.2 million in fee.[Footnote 46] As of May 2014, CMS had paid CGI Federal $12.5 million in fee. Likewise, CMS negotiated additional fixed fee for the data hub task order, QSSI's eligible fee rose from over $716,000 under the original $29.9 million award to more than $1.3 million for work performed through February 2014. Costs Continue to Increase with New FFM Contractor: Rather than pursue the correction of performance issues and continuing FFM development with CGI Federal, CMS determined that its best chance of delivering the system and protecting the government's financial interests would be to award a new contract to another vendor. In January 2014, CMS awarded a one-year sole source contract (cost-plus- award-fee) with an estimated value of $91 million to Accenture Federal Services to transition support of the FFM and continue the FFM development that CGI Federal was unable to deliver.[Footnote 47] CMS's justification and approval document for the new award states that the one-year contract action is an interim, transitory solution to meet CMS's immediate and urgent need for specific FFM functions and modules- -including the financial management module.[Footnote 48] This work has also experienced cost increases. Figure 8 shows increases in obligations for the Accenture Federal Services contract since award in January 2014. Figure 8: Cumulative Obligations for Accenture Federal Services Contract to Continue FFM Development as of June 5, 2014: [Refer to PDF for image: illustration] Dollars obligated: 01/2014: $45 million[A]; 02/2014: $60 million; 04/2014: $105 million; 05/2014: $144 million[B]; 06/2014: $175.4 million. Source: GAO analysis of Centers for Medicare & Medicaid Services data. GAO-14-694. Notes: [A] The total contract value was initially estimated to be $91 million, but CMS obligated $45 million at the time of award. [B] CMS modified the Accenture Federal Services contract three times in May 2014. [End of figure] The financial management module of the FFM includes the services necessary to spread risk among issuers and to accomplish financial interactions with issuers. Specifically, this module tracks eligibility and enrollment transactions and subsidy payments to insurance plans, integrates with CMS's existing financial management system, provides financial accounting and outlook for the entire program, and supports the reconciliation calculation and validation with IRS. According to the CMS justification and approval document, CMS estimated that it would cost $91 million over a one-year period for Accenture Federal Services to complete the financial management module and other FFM enhancements. As of June 5, 2014, the one-year contract had been modified six times since contract award and CMS had obligated more than $175 million as a result of new requirements, changes to existing requirements, and new enhancements. For example, CMS modified the contract to incorporate additional work requirements and functionality related to the Small Business Health Options Program marketplace, state-based marketplace transitions, and hardware acquisition. CMS had yet to fully define requirements for certain FFM functionality, including the financial management module, when the new contract to continue FFM development was awarded in January 2014. Accenture Federal Services representatives told us that while they had a general understanding of requirements at the time of award, their initial focus during the period January through April 2014 was on transitioning work from the incumbent contractor and clarifying CMS' requirements. Accenture Federal Services representatives attributed contract increases during this period to their increased understanding of requirements, as well as clarifying additional activities requested under the original contract. Further, although the justification and approval document stressed that delivery of the financial management module was needed by mid-March 2014, contracting and program officials explained that time frames for developing the module were extended post-award, and as of June 2014, the financial management module was still under development. Financial management module functionality is currently scheduled to be implemented in increments from June through December 2014. Conclusions: CMS program and contracting staff made a series of planning decisions and trade-offs that were aimed at saving time, but which carried significant risks. While optimum use of acquisition planning and oversight was needed to define requirements, develop solutions, and test them before launching Healthcare.gov and its supporting systems, the efforts by CMS were plagued by undefined requirements, the absence a required acquisition strategy, confusion in contract administration responsibilities, and ineffective use of oversight tools. In addition, while potentially expedient, CMS did not adhere to the governance model designed for the FFM and data hub task orders, resulting in an ineffectual governance process in which scheduled design and readiness reviews were either diminished in importance, delayed, or skipped entirely. By combining that governance model with a new IT development approach the agency had not tried before, CMS added even more uncertainty and potential risk to their process. The result was that problems were not discovered until late, and only after costs had grown significantly. As FFM contractor performance issues were discovered late in development, CMS increasingly faced a choice of whether to stop progress and pursue holding the contractor accountable for poor performance or devote all its efforts to making the October deadline. CMS chose to proceed with pursuing the deadline. After October 1, 2013, CMS decided to replace the contractor, but in doing so had to expend additional funds to complete essential FFM functions. Ultimately, more money was spent to get less capability. Meanwhile, CMS faces continued challenges to define requirements and control costs to complete development of the financial management module in the FFM. Unless CMS takes action to improve acquisition oversight, adhere to a structured governance process, and enhance other aspects of contract management, significant risks remain that upcoming open enrollment periods could encounter challenges going forward. Recommendations for Executive Action: In order to improve the management of ongoing efforts to develop the federal marketplace, we recommend that the Secretary for Health and Human Services direct the Administrator of the Centers for Medicare & Medicaid Services to take the following five actions: * Take immediate steps to assess the causes of continued FFM cost growth and delayed system functionality and develop a mitigation plan designed to ensure timely and successful system performance. * Ensure that quality assurance surveillance plans and other oversight documents are collected and used to monitor contractor performance. * Formalize existing guidance on the roles and responsibilities of contracting officer representatives and other personnel assigned contract oversight duties, such as government task leaders, and specifically indicate the limits of those responsibilities in terms of providing direction to contractors. * Provide direction to program and contracting staff about the requirement to create acquisition strategies and develop a process to ensure that acquisition strategies are completed when required and address factors such as requirements, contract type, and acquisition risks. * Ensure that information technology projects adhere to requirements for governance board approvals before proceeding with development. Agency Comments, Third-Party Views, and Our Evaluation: We provided a draft of this product to the Department of Health and Human Services and the Centers for Medicare & Medicaid Services for review and comment. In its written comments, which are reprinted in appendix III, HHS concurred with four of our five recommendations and described the actions CMS is taking to improve its contracting and oversight practices. HHS partially concurred with our recommendation that CMS assess the causes of continued FFM cost growth. The agency says that CMS already has assessed the reasons for cost growth under the CGI Federal task order and that any increase in costs since the contract with Accenture Federal Services for continued development of the FFM was finalized is attributable to additional requirements, not cost overruns. We recognize that much of the increase in costs under the Accenture Federal Services contract is due to new requirements or enhancements. Nevertheless, based on our review of the contract modifications, not all the increase in costs from $91 million to more than $175 million, when measured from the initial projection, is attributable to new requirements. For example, as CMS stated in its comments, after additional analysis CMS determined a $30 million cost increase was needed to complete the contract's original scope of work. We continue to believe that a further assessment is needed to ensure that costs as well as requirements are under control and that the development of the FFM is on track to support the scheduled 2015 enrollment process. All three contractors, as well as HHS, provided additional technical comments, which we incorporated in the report where appropriate. We are sending copies of this report to the Secretary of Health and Human Services and the Administrator of the Centers for Medicare & Medicaid Services. In addition, the report is available at no charge on the GAO website at [hyperlink, http://www.gao.gov/]. If you or your staff have any questions about this report, please contact William T. Woods at (202) 512-4841 or woodsw@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV. Signed by: William T. Woods: Director, Acquisition and Sourcing Management: Signed by: Valerie C. Melvin: Director, Information Management and Technology Resources Issues: List of Congressional Requesters: The Honorable Ron Wyden: Chairman: The Honorable Orrin Hatch: Ranking Member: Committee on Finance: United States Senate: The Honorable Thomas R. Carper: Chairman: The Honorable Tom Coburn, M.D. Ranking Member: Committee on Homeland Security and Governmental Affairs: United States Senate: The Honorable Lamar Alexander: Ranking Member: Committee on Health, Education, Labor and Pensions: United States Senate: The Honorable Charles E. Grassley: Ranking Member: Committee on the Judiciary: United States Senate: The Honorable Jon Tester: Chairman: Subcommittee for Efficiency and Effectiveness of Federal Programs and the Federal Workforce: Committee on Homeland Security and Governmental Affairs: United States Senate: The Honorable Claire McCaskill: Chairman: Subcommittee on Financial and Contracting Oversight: Committee on Homeland Security and Governmental Affairs: United States Senate: The Honorable Fred Upton: Chairman: Committee on Energy and Commerce: House of Representatives: The Honorable Darrell Issa: Chairman: The Honorable Elijah E. Cummings: Ranking Member: Committee on Oversight and Government Reform: House of Representatives: The Honorable Dave Camp: Chairman: The Honorable Sander M. Levin: Ranking Member: Committee on Ways and Means: House of Representatives: The Honorable Greg Walden: Chairman: Subcommittee on Communications and Technology: Committee on Energy and Commerce: House of Representatives: The Honorable Joseph R. Pitts: Chairman: Subcommittee on Health: Committee on Energy and Commerce: House of Representatives: The Honorable Tim Murphy: Chairman: Subcommittee on Oversight and Investigations: Committee on Energy and Commerce: House of Representatives: The Honorable Mike Coffman: Chairman: Subcommittee on Oversight and Investigations: Committee on Veterans' Affairs: House of Representatives: The Honorable Charles Boustany, Jr. Chairman: The Honorable John Lewis: Ranking Member: Subcommittee on Oversight: Committee on Ways and Means: House of Representatives: The Honorable Mark Begich: United States Senate: The Honorable Michael Bennet: United States Senate: The Honorable Richard Blumenthal: United States Senate: The Honorable Robert P. Casey, Jr. United States Senate: The Honorable Al Franken: United States Senate: The Honorable Kay R. Hagan: United States Senate: The Honorable Tim Kaine: United States Senate: The Honorable Amy Klobuchar: United States Senate: The Honorable Mary Landrieu: United States Senate: The Honorable Joe Manchin, III: United States Senate: The Honorable Jeffrey A. Merkley: United States Senate: The Honorable Bill Nelson: United States Senate: The Honorable Mark Pryor: United States Senate: The Honorable Jeanne Shaheen: United States Senate: The Honorable Mark Udall: United States Senate: The Honorable Mark R. Warner: United States Senate: The Honorable Ron Barber: House of Representatives: The Honorable John Barrow: House of Representatives: The Honorable Tulsi Gabbard: House of Representatives: The Honorable Pete P. Gallego: House of Representatives: The Honorable Duncan Hunter: House of Representatives: The Honorable Mike Kelly: House of Representatives: The Honorable Ann McLane Kuster: House of Representatives: The Honorable Daniel W. Lipinski: House of Representatives: The Honorable Patrick E. Murphy: House of Representatives: The Honorable Scott Peters: House of Representatives: The Honorable Kyrsten Sinema: House of Representatives: The Honorable Filemon Vela: House of Representatives: [End of section] Appendix I: Objectives, Scope, and Methodology: This report examines selected contracts and task orders central to the development and launch of the Healthcare.gov website by assessing (1) Centers for Medicare & Medicaid Services (CMS) acquisition planning activities; (2) CMS oversight of cost, schedule, and system capability changes; and (3) actions taken by CMS to identify and address contractor performance issues. To address these objectives, we used various information sources to identify CMS contracts and task orders related to the information technology (IT) systems supporting the Healthcare.gov website. Specifically, we reviewed data in the Federal Procurement Data System- Next Generation, which is the government's procurement database, to identify CMS contracts and task orders related to the IT systems supporting the Healthcare.gov website and amounts obligated for fiscal years 2010 through March 2014. In addition, we reviewed CMS provided data on the 62 contracts and task orders related to the IT systems supporting the Healthcare.gov website and amounts obligated as of March 2014. To select contracts and task orders to include in our review, we analyzed Federal Procurement Data System-Next Generation and CMS data to identify contracts and task orders that represent large portions of spending for Healthcare.gov and its supporting systems. We then selected one contract and two task orders issued under an existing 2007 contract and interviewed contracting officials in CMS's Office of Acquisition and Grants Management and program officials in CMS's Office of Information Services to confirm that these contracts are central to development of Healthcare.gov and its supporting systems.[Footnote 49] The contract and task orders combined accounted for more than 40 percent of the total CMS reported obligations related to the development of Healthcare.gov and its supporting systems as of March 2014. Specifically, we selected the task orders issued to CGI Federal Inc. (CGI Federal) for the development of the federally facilitated marketplace (FFM) system and to QSSI, Inc. QSSI for the development of the federal data services hub (data hub) in September 2011--and the contract awarded to Accenture Federal Services in January 2014 to continue FFM development and enhance existing functionality. To describe federal implementation costs for Healthcare.gov and its supporting systems, we interviewed program officials and obtained relevant documentation to identify eight agencies that reported IT- related obligations or used existing contracts and task orders or operating budgets to support the development and launch of the Healthcare.gov website. These eight agencies include the Centers for Medicare & Medicaid Services (CMS), Internal Revenue Service (IRS), Social Security Administration, Veterans Administration (VA), Peace Corps, Office of Personnel Management, Department of Defense (DOD), and Department of Homeland Security. We then obtained and analyzed various types of agency-provided data to identify overall IT-related costs for Healthcare.gov and its supporting systems. Three agencies, including CMS, IRS, and VA reported almost all of the IT-related obligations supporting the implementation of Healthcare.gov and its supporting systems as of March 2014. We performed data reliability checks on contract obligation data provided by these three agencies, such as checking the data for obvious errors and comparing the total amount of funding obligated for each contract and task order as reported by each agency to data on contract obligations in Federal Procurement Data System-Next Generation or USASpending.gov.[Footnote 50] We found that these data were sufficiently reliable for the purpose of this report. To assess CMS acquisition planning activities, we reviewed Federal Acquisition Regulation (FAR) and relevant Department of Health and Human Services (HHS)/CMS policies and guidance. We also evaluated contract file documents for three selected contracts and task orders, including acquisition planning documentation, request for proposal, statements of work, cost estimates, and technical evaluation reports to determine the extent to which CMS's acquisition planning efforts met FAR and HHS/CMS requirements. In assessing CMS's acquisition planning efforts, we looked for instances where CMS took steps to mitigate acquisition program risks during the acquisition planning phase, including choice of contract type and source selection methodology. In addition, we interviewed CMS contracting and program officials to gain a better understanding of the acquisition planning process for select contracts and task orders including the rationale for choosing the selected contract type and the analysis conducted to support the source selection process. We also reviewed prior GAO reports on CMS contract management to assess the extent to which CMS's acquisition planning activities addressed issues previously identified by GAO. To assess CMS oversight of cost, schedule, and system capability changes, we analyzed contract file documents for one selected contracts and two task orders. As part of our assessment of the selected contracts and task orders, we reviewed contract modifications, contractor monthly status and financial reports, statements of work, contractor deliverables, schedule documentation, and contracting officer's representative files, and meeting minutes to determine if there were any changes and whether system development proceeded as scheduled. We performed a data reliability check of cost data for selected contracts and task orders by comparing contract modification documentation to contract obligation data in Federal Procurement Data System-Next Generation. To evaluate the extent to which CMS adhered to its governance process, we compared the governance model the agency intended would guide the design, development, and implementation of Healthcare.gov and its supporting systems, to the development process the agency actually used for the FFM and data hub. We also obtained and analyzed documentation from governance reviews to identify the date and content of the reviews to determine if key milestone reviews were held in accordance to the development schedule. In addition, we reviewed FAR and federal standards for internal control for contract oversight to evaluate the extent to which CMS's approach to contract oversight for the selected contracts and task orders met FAR and federal internal control standards. We interviewed CMS contracting and program officials to gain a better understanding of FFM and data hub cost, schedule, and system capabilities, and to obtain information on the organization and staffing of offices and personnel responsible for performance monitoring for selected contracts and task orders. We also interviewed contractors to obtain their perspective on CMS's oversight of cost, schedule, and system capabilities. Further, as part of our assessment of CMS's development approach for the FFM and data hub, we reviewed prior GAO work regarding information technology and development. To assess actions taken by CMS to identify and address contractor performance issues, we reviewed relevant FAR and HHS guidance for contract monitoring and inspection of services to identify steps required for selected contracts and task orders and recourse options for unsatisfactory performance. In addition, we obtained and analyzed contract file documentation including contracting officer's representative files, contractor deliverables, contractor monthly status and financial reports, contractor performance evaluations, and meeting minutes to determine the extent to which performance was reported and what steps, if any, were taken to address any issues. To determine contractor fee not paid during development, we obtained and analyzed CMS contractor invoice logs and contract payment notifications. We also interviewed CMS contracting and program officials to obtain additional information regarding contractor performance and actions taken by CMS, if any, to address contractor performance issues. We conducted this performance audit from January 2014 to July 2014, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: Cumulative Cost Increases for the Task Orders for Developing the Federally Facilitated Marketplace System and Federal Data Services Hub Task Orders: Federally Facilitated Marketplace System (FFM): Task order issued/modified: Issuance; Date: 9/30/2011; Obligation: $55,744,082; Total obligated to date: $55,744,082; Description: FFM task order issued to CGI Federal. Task order issued/modified: Modification 1; Date: 8/26/2012; Obligation: $35,771,690; Total obligated to date: $91,515,772; Description: Obligates an additional $35.8 million, primarily to provide for new and increased system requirements resulting from program office decisions and finalized regulations. Task order issued/modified: Modification 2; Date: 11/16/2012; Obligation: 0; Total obligated to date: $91,515,772; Description: No cost modification for administrative purposes, including identifying a new contracting officer's representative. Task order issued/modified: Modification 3; Date: 4/30/2013; Obligation: $27,688,008; Total obligated to date: $119,203,779; Description: Obligates an additional $27.7 million needed to avert a potential cost overrun. The funding supports an increased level of effort to add system functionality not included in the statement of work and increased infrastructure needs. Task order issued/modified: Modification 4; Date: 5/10/2013; Obligation: $474,058; Total obligated to date: $119,677,837; Description: Obligates approximately $474,000 for additional infrastructure requirements, specifically requirements for the content delivery network that delivers web services. Task order issued/modified: Modification 5; Date: 9/1/2013; Obligation: $58,143,472; Total obligated to date: $177,821,309; Description: Modified to extend the period of performance for FFM development until February 28, 2014, and obligate an additional $58.1 million, primarily to support the extension. Task order issued/modified: Modification 6; Date: 9/19/2013; Obligation: $18,215,807; Total obligated to date: $196,037,116; Description: Obligates an additional $18.2 million to purchase a software license. Task order issued/modified: Modification 7; Date: 10/4/2013; Obligation: 0; Total obligated to date: 196,037,116; Description: Modified to issue a change order directing the contractor to develop and implement an identity management software solution. Task order issued/modified: Modification 8; Date: 10/21/2013; Obligation: $1,479,309; Total obligated to date: $197,516,425; Description: Obligates $1.5 million to increase capacity of the content delivery network from 50 terabytes to 400 terabytes. Task order issued/modified: Modification 9; Date: 12/24/2013; Obligation: $6,981,666; Total obligated to date: $204,498,091; Description: Obligates $7.0 million to definitize the change order issued under Modification 7. It also funds software licenses and the industry experts hired to improve system performance. Task order issued/modified: Modification 10; Date: 1/10/2014; Obligation: 0; Total obligated to date: $204,498,091; Description: Modified to issue a change order directing the contractor to begin transitioning services to a new contractor. Task order issued/modified: Modification 11; Date: 2/21/2014; Obligation: $5,133,242; Total obligated to date: $209,631,333; Description: Obligates $4.8 million to definitize the change order issued under Modification 10 and fund post-transition consulting services through April 30, 2014. Data Hub: Task order issued/modified: Issuance; Date: 9/30/2011; Obligation: $29,881,693; Total obligated to date: $29,881,693; Description: Data hub task order issued to QSSI. Task order issued/modified: Modification 1; Date: 1/18/2012; Obligation: ($4,180,786); Total obligated to date: $25,700,907; Description: Modified to cancel a stop work order that was issued due to a GAO bid protest and direct the contractor to continue performance of the task order. Obligations are reduced by $4.2 million in accordance with the contractor's revised task order proposal (submitted as part of the bid protest process). Task order issued/modified: Modification 2; Date: 9/4/2012; Obligation: $23,017,077; Total obligated to date: $48,717,984; Description: Obligates an additional $23.0 million, primarily to provide for new and increased system requirements resulting from program office decisions and finalized regulations. Task order issued/modified: Modification 3; Date: 11/16/2012; Obligation: 0; Total obligated to date: $48,717,984; Description: No cost modification for administrative purposes, including identifying a new contracting officer's representative. Task order issued/modified: Modification 4; Date: 6/1/2013; Obligation: $4,991,614; Total obligated to date: $53,709,598; Description: Obligates $5.0 million to fund an electronic data interchange tool and related labor to support enrollment services. Task order issued/modified: Modification 5; Date: 9/1/2013; Obligation: $30,817,530; Total obligated to date: $84,527,128; Description: Modified to extend the period of performance for data hub development until February 28, 2014, and obligate an additional $30.8 million, primarily to support the extension. Task order issued/modified: Modification 6; Date: 11/15/2013; Obligation: 0; Total obligated to date: $84,527,128; Description: No cost modification to transfer funds among contract line items and revise personnel. Task order issued/modified: Modification 7; Date: 2/25/2014; Obligation: $15,130,711; Total obligated to date: $99,657,839; Description: Modified to exercise option year 1: Operations and Maintenance. Source: GAO analysis of Centers for Medicaid & Medicare Services data. GAO-14-694: [End of table] [End of section] Appendix III: Comments from the Department of Health and Human Services: Department of Health & Human Services: Office of The Secretary: Assistant Secretary for Legislation: Washington, DC 20201: July 25, 2014: William T. Woods: Director, Acquisition and Sourcing Management: U.S. Government Accountability Office: 441 G Street NW: Washington. DC 20548: Dear Mr. Woods: Attached are comments on the U.S. Government Accountability Office's (GAO) report entitled, "Healthcare.gov: Ineffective Planning and Oversight Practices Underscore the Need for Improved Contract Management" (GAO-14-694). The Department appreciates the opp0l1unity to review this report prior to publication. Sincerely. Signed by: Jim R. Esquea: Assistant Secretary for Legislation: Attachment: General Comments Of The Department Of Health And Human Services (HHS) On The Government Accountability Office's (GAO) Draft Report Entitled, "Healthcare.Gov: Ineffective Planning And Oversight Practices Underscore The Need For Improved Contracts" (GAO-14-694): The Department of Health and Human Services (HHS) and the Centers for Medicare & Medicaid Services (CMS) appreciates the opportunity to review and comment on the GAO draft report. The draft report contains five recommendations for the Secretary of HHS to direct to the Administrator of CMS. HHS and CMS respond to the recommendations below. HHS and CMS are committed to expanding affordable, comprehensive health coverage to more Americans through the establishment and operation of the Federally-facilitated Marketplace (FFM or Marketplace). The success of the Marketplace depends on effective planning and contract management, and CMS is focused on improving Marketplace operations and contractor oversight through strong governance, defined authorities, and clear requirements. CMS appreciates the GAO recommendations, which will further assist CMS in implementing innovative, consumer-facing IT projects that serve millions of Americans. After the enactment of the Affordable Care Act in March 2010, HHS and CMS faced a unique and difficult challenge - to establish a first-of- its-kind online Marketplace to determine consumers' eligibility for coverage and insurance affordability programs, and enroll them in coverage beginning January 1,2014. With that broad goal, and facing limited time and resources, as well as changing requirements in response to input from, states, issuers, and consumers, as well as rulings from the U.S. Supreme Court, HHS and CMS launched the FFM and the Data Services Hub on October 1, 2013. While the initial launch and the user experience were unacceptable, the functionality of the Marketplace steadily improved through strong management oversight and additional technical expertise. By the end of the open enrollment period, the FFM had helped over 5.4 million consumers select private health insurance coverage and assisted millions more in getting coverage through Medicaid. CMS is improving the management of the Marketplace and is confident that its contractors will deliver the needed capabilities for the 2015 open enrollment period in a timely and cost-efficient manner. CMS has already assessed the causes of cost grovv1h and schedule delays with its CGI contract. In response to that assessment, CMS ended its cost plus fixed fee contract with CGI and awarded a new cost plus award fee contract with Accenture. In addition, CMS modified its definitive one- year agreement with Accenture to incorporate additional work requirements and functionality related to items such as the Small Business Health Options Program (SHOP), State Based Marketplace (SBM) transitions, and hardware acquisition. These represent new requirements and additional functionality rather than cost overruns. CMS will use required contract deliverables to track continued performance and mitigate the need for additional funding. This continuous oversight will help limit any unanticipated costs that may arise as we continue to develop the system for the next open enrollment. CMS is committed to improving the management of the Marketplace to ensure that this investment will serve consumers for years to come. CMS is building on the lessons learned during the launch of the FFM and the first open enrollment period to ensure effective management of the Marketplace that is focused on clear lines of authority, prioritization of requirements and deliverables, and metric-driven quality reviews for its Healthcare.gov contracts and for contracts across the agency. This improvement is organized around three core supports - a strong management structure within CMS, the improved structure of Marketplace contracts, and a strengthened acquisition workforce supported by clear strategy, policy, and training. The strong management structure, which will focus priorities and provide clear direction, includes: * A new operations-focused CMS Principal Deputy Administrator for agency-wide policy and operational program coordination. * A new, permanent Marketplace CEO with responsibility and accountability for leading the FFM, managing relationships with SBMs, and running the Center for Consumer Information and Insurance Oversight (CCIIO), which is the program office mainly responsible for the implementation of the FFM. * A new, permanent Marketplace Chief Technology Officer who will report to the new Marketplace CEO and work closely with the Deputy Chief Operating Officer and the Office of Information Services within CMS in order to ensure proper alignment of project milestones and deliverables. * A program manager for the FFM that is responsible for overseeing contractor performance and governance reviews. The improved structure of Marketplace contracts includes: * The end of CMS's contract with CGI and a new cost plus award fee contract with Accenture to continue building and operating the FFM through the 2015 enrollment period. This contract is defined by clear deliverables and deadlines, as well as improved communication structures. The contract is structured to incentivize exceptional performance and control costs by basing the award fee upon Accenture's performance. * A systems integrator contract with QSSI which provides program expertise and coordinates the work with CMS and its contractors to ensure clear accountability, efficient use of resources, and prioritization of deliverables. * Continued support for QSSI as it operates the Data Services Hub, which as the GAO noted in the draft report, worked as intended when launched on October 1,2013. The strengthened acquisition workforce is being improved through collaboration with HHS and cross-department efforts, and it includes: * Additional training for CMS acquisition personnel including program and project managers, contracting officers, and contracting officer's representatives (CORs). * Dissemination of best practices for contract/program management across the Agency. * Establishment of a CMS enterprise-wide approach to program and project management through the initiatives that improve the coordination of program managers and other members of the acquisition team. GAO Recommendation: The Administrator of CMS take immediate steps to assess the causes of continued FFM cost growth and schedule delays and develop a mitigation plan designed to ensure timely and successful system performance. HHS Response: HHS partially concurs with the OAO recommendation. CMS has already assessed the causes of cost growth and schedule delays with its CGI contract, and in response to that assessment, CMS ended its cost plus fixed contract with COL, and awarded a new cost plus award fee contract with Accenture. CMS awarded this type of contract because it better controls costs and rewards performance. Additionally, CMS and Accenture, through a series of multi-day meetings in early 2014, finalized a definitive one-year agreement with well-defined requirements and ensured that both eMS and Accenture staff understood these requirements and their scope. This clear understanding of requirements limits the possibility of inappropriate authorizations that could lead to out-of-scope work, reduces risk, and allows CMS to conduct more stringent oversight. CMS is using the required contract deliverables to routinely track Accenture's performance and to identify performance issues quickly and take effective remedial action, if necessary. CMS disagrees with GAO's assertion in the draft report that there has been "continued cost growth" since the Accenture agreement was finalized. The increases since the initial estimate of $91 million reflect a thorough analyses and finalization of the requirements as the letter contract was negotiated, along with contract changes that have added additional requirements and functionality. As noted in the draft report, in the CMS justification and approval document, CMS estimated that it would cost $91 million over a one-year period for Accenture to complete the FFM and support the 2015 enrollment period. After thorough analysis to assess needs, CMS determined that $121 million was a more appropriate amount to complete this work. In addition, CMS has modified that definitive one-year agreement to incorporate additional work requirements and functionality related to the SHOP, SBM transitions, and hardware acquisition. These represent new requirements and additional functionality rather than cost overruns. GAO Recommendation: The Administrator of CMS ensure the quality assurance surveillance plans and other oversight documents are collected and used to monitor contractor performance. HHS Response: HHS concurs with this recommendation. CMS has policies and procedures in place to monitor contractor performance and has taken steps to better enforce those policies. CMS has a quality assurance surveillance plan in place for the Marketplace contract with Accenture and is currently using it to monitor performance. CMS has also implemented several other strategies to monitor Marketplace contractors' performance including requiring greater collaboration and coordination between CMS and its contractors, increasing the number and frequency of contract deliverables, and instituting value measures to more effectively monitor cost control within the contract. GAO Recommendation: The Administrator of CMS formalize existing guidance on the roles and responsibilities of a Contracting Officer Representatives (CORs) and other personnel assigned contract oversight duties, such as Government Task Leaders, and specifically indicate the limits of those responsibilities in terms of providing direction to contractors. HHS Response: HHS concurs with this recommendation and is currently working with its acquisition personnel to ensure there is a clear understanding of roles and responsibilities. In April 2013, CMS issued an internal memorandum that reminded all staff of the roles and responsibilities of acquisition personnel and provided guidance on what constitutes proper technical direction and ways to avoid unauthorized commitments. This informal guidance stopped inappropriate authorizations by individuals who did not have specific delegated contracting authority. CMS is currently formalizing this guidance to remind personnel of appropriate roles and responsibilities. HHS and CMS are also implementing initiatives to improve training for contract and program personnel. HHS established acquisition Learning Communities to provide integrated training for members of the acquisition community. This training is designed to assist participants in understanding the acquisition lifecycle for various goods and services. These sessions outline the roles and responsibilities of each member of the acquisition workforce through the entire acquisition lifecycle, and provide hands-on experience through HHS-specific training scenarios. Along with the HHS training, CMS offers extensive training in for its contracting officers, CORs, and program and project managers. This additional training includes classes focused on strategic planning and implementation and risk management. Specifically, this includes approximately 140 classes that provide staff the opportunity to complete the training requirements for certification in contract and program management. CMS is also providing training for program managers who oversee CMS' major IT investments that will detail certification requirements, the roles and responsibilities of the program manager, including the preparation of an acquisition strategy. Furthermore, CMS is currently developing initiatives where program managers and other members of the acquisition team can share best practices and ideas with each other, coordinate the program management efforts between the key program offices within CMS, and establish an enterprisewide approach to program and project management. GAO Recommendation: The Administrator of CMS provide direction to program and contracting staff about the requirement to create acquisition strategies and develop a process to ensure that acquisition strategies are completed when required and address factors such as requirements, contract type, and acquisition risks. HHS Response: HHS concurs with this recommendation and is currently taking steps to insure that Program Managers fully understand their roles and responsibilities, including the requirement to prepare an Acquisition Strategy. The Federal Acquisition Regulation (FAR) requires acquisition strategies and plans for all programs or projects that are augmented by acquiring contractor support or services. HHS is currently updating guidance on the use of acquisition strategies and expects to have that guidance issued in late summer 2014. HHS is also providing a series of training opportunities to the HHS operating and staff divisions to ensure that the roles and responsibilities of all involved in the acquisition lifecycle are aware of requirements to develop and execute quality acquisition strategies for approved projects. Additionally, CMS has reassessed its program managers' assignments for each of its major IT investments. In late August 2014, the Office of Acquisition and Grants Management will be conducting a Learning Community meeting for assigned program managers to insure that they understand their roles and responsibilities, requirements for certification, and the importance of preparing an acquisition strategy. GAO Recommendation: The Administrator of CMS ensure that information technology projects adhere to requirements for governance board approvals before proceeding with development. HHS Response: CMS concurs with this recommendation, and adopted and enforced a strict governance structure to manage the scope and quality of Marketplace deliverables. CMS oversees Marketplace development through weekly senior leadership meetings as well as weekly management meetings. In addition, changes to priorities or requirements must be approved by a specific change control board that ensures requirements are carefully aligned and prioritized. Functional and technical teams collaborate and coordinate on planning and execution through daily meetings staffed with development, operations, and maintenance contractors, lead federal policy, operations and technical staff, and representatives from the Systems Integrator. In order to ensure there is integration at the staff level, CMS has also increased coordination and collaboration across functional, technical, and program areas through designated primary and secondary staff members who are held responsible and accountable from each of the business and technical teams. The Systems Integrator works with these teams to monitor, assess, and identify potential technical and operations issues. They work with CMS staff to develop solutions and ensure that effective and timely decisions are made to meet Marketplace deadlines. Through constant process improvement, the current management oversight and decision- making governance structure represents the application of key lessons learned and best practices for policy, requirements management, operations, technology implementation, contracts, and schedule. [End of section] Appendix IV: GAO Contact and Staff Acknowledgments: GAO Contact: William T. Woods, (202) 512-4841 or woodsw@gao.gov: Staff Acknowledgments: In addition to the contact named above, W. William Russell, Assistant Director; Jennifer Dougherty; Elizabeth Gregory-Hosler; Andrea Yohe; Susan Ditto; Julia Kennon; John Krump; Ken Patton; Roxanna Sun; and Kevin Walsh made key contributions to this report . [End of section] Footnotes: [1] Pub. L. No. 111-148, 124 Stat. 119 (2010). [2] PPACA also requires the creation of Small Business Health Options Program exchanges, where small businesses can shop for and purchase health coverage for their employees. [3] The existing contract is a multiple-award, indefinite-delivery, indefinite-quantity contract (hereinafter referred to as the 2007 contract). This contract type provides for an indefinite quantity, within stated limits, of supplies or services during a fixed period. The Government places orders for individual requirements. Quantity limits may be stated as number of units or as dollar values. FAR § 16.504. [4] A qualified health plan is an insurance plan that is certified by a marketplace to offer coverage through that marketplace. [5] Medicaid is a joint federal-state program that finances health care coverage for certain low-income individuals. CHIP is a federal- state program that provides health care coverage to children 18 years of age and younger living in low-income families whose incomes exceed the eligibility requirements for Medicaid. [6] States seeking to operate a state-based marketplace were required to submit an application to CMS in December 2012. States electing not to establish a state-based marketplace, but seeking to participate in a partnership marketplace were required to complete an abbreviated version of that application by February 2013. States electing not to establish a state-based exchange or participate in a partnership exchange were not required to submit an application to CMS. [7] GAO, Patient Protection and Affordable Care Act: Status of CMS Efforts to Establish Federally Facilitated Health Insurance Exchanges, [hyperlink, http://www.gao.gov/products/GAO-13-601] (Washington, D.C.: June 19, 2013). [8] HHS proposed October 1, 2013, as the start of the initial open enrollment period in a July 2011 proposed rule and included this date in the statement of work for both the FFM and data hub task orders. 76 Fed. Reg. 41866 (July 15, 2011). CMS issued a final rule adopting this date in March 2012. 77 Fed. Reg. 18310 (Mar. 27, 2012) (codified at 45 C.F.R. § 155.410(b)). [9] GAO, Patient Protection and Affordable Care Act: Preliminary Results of Undercover Testing of Enrollment Controls for Health Care Coverage and Consumer Subsidies Provided Under the Act, [hyperlink, http://www.gao.gov/products/GAO-14-705T] (Washington, D.C.: July 23, 2014). GAO is also conducting additional work that will provide information on Healthcare.gov and its supporting systems. [10] The federal sources of information include data sources at the Social Security Administration, the Internal Revenue Service, the Department of Homeland Security, the Department of Veterans Affairs (VA), the Department of Defense, the Peace Corps, and the Office of Personnel Management. [11] These subsidies include premium tax credits to offset qualified health plan premium costs and cost-sharing reductions to reduce policyholders' out-of-pocket payments, including deductibles and co- payments, for covered services. [12] An obligation is a definite commitment that creates a legal liability of the government for the payment of goods and services ordered or received, or a legal duty on the part of the United States that could mature into a legal liability by virtue of actions of another party. [13] In an August 2011 report, GAO recommended that HHS collect information about the time frames needed for pre-solicitation acquisition planning activities to establish time frames for when program officials should begin acquisition planning. This recommendation has not yet been implemented. A second recommendation from this report--that HHS ensure that agency and component guidance clearly define the role of cost estimating and incorporating lessons learned in acquisition planning, as well as specific requirements for what should be included in documenting these elements in the contract file--has been implemented. See GAO, Acquisition Planning: Opportunities to Build Strong Foundation for Better Services Contracts, [hyperlink, http://www.gao.gov/products/GAO-11-672] (Washington, D.C.: Aug. 9, 2011). [14] FAR § 15.101-1(a). [15] FAR § 37.503(a). [16] [hyperlink, http://www.gao.gov/products/GAO-11-672]. [17] GAO, Defense Acquisitions: Stronger Management Practices Are Needed to Improve DOD's Software-Intensive Weapon Acquisitions. [hyperlink, http://www.gao.gov/products/GAO-04-393] (Washington, D.C.: Mar. 1, 2004). [18] Under PPACA, states had to obtain CMS approval to establish and operate their own marketplaces for 2014 by January 1, 2013. 42 U.S.C. § 18041(c)(1)(B). [19] The Program Support Center in the Office of the Secretary awarded a contract in September 2010 on behalf of OCIIO to develop the business architecture for the FFM and data hub. This contract was transferred to CMS when OCIIO became CCIIO within CMS. [20] According to CMS contracting and program officials, requirements development was done simultaneously for the two task orders, with the potential for both task orders to be awarded to the same contractor. [21] See, for example, [hyperlink, http://www.gao.gov/products/GAO-11-672] and GAO, Department of Homeland Security: Better Planning and Assessment Needed to Improve Outcomes for Complex Service Acquisitions, [hyperlink, http://www.gao.gov/products/GAO-08-263] (Washington, D.C.: Apr. 22, 2008). In this report GAO made three recommendations to the Secretary of Homeland Security to achieve improved outcomes for its service acquisitions. [22] FAR §16.301-2(a)(1) & (2). [23] In 2009, the President released a Memorandum (M-09-25) calling for a reduction in the use of high-risk contracts. In 2012, DOD, GSA, and NASA adopted as final rule amending the FAR to implement a section of the Duncan Hunter National Defense Authorization Act for Fiscal Year 2009 that addresses the use and management of cost-reimbursement contracts. 77 Fed. Reg. 12925 (Mar. 2, 2012). [24] See GAO, Centers for Medicare and Medicaid Services: Internal Control Deficiencies Resulted in Millions of Dollars of Questionable Contract Payments, [hyperlink, http://www.gao.gov/products/GAO-08-54] (Washington, D.C.: Nov. 15, 2007). We made nine recommendations to the Administrator of CMS to improve internal control and accountability in the contracting process and related payments to contractors. All nine recommendations have been implemented. [25] Earned value management is a project management tool that integrates project scope with cost, schedule and performance elements for purposes of project planning and control. FAR § 2.101. [26] The task orders also required additional oversight mechanisms, such as CMS governance milestone reviews. These included a Project Baseline Review intended to assess the project plan's scope, schedule and risk, and an Operational Readiness Review to determine if the product was ready to support business operations. [27] FAR § 46.401. [28] In 2012, GAO reported on the use of Agile methods in the Federal government. See GAO, Software Development: Effective Practices and Federal Challenges in Applying Agile Methods, [hyperlink, http://www.gao.gov/products/GAO-12-681] (Washington, D.C.: July 27, 2012). In this report we made one recommendation to the Federal CIO Council to encourage the sharing of these practices. [29] OMB, 25 Point Implementation Plan to Reform Federal Information Technology Management (Washington, D.C.: Dec. 9, 2010) and Immediate Review of Financial Systems IT Projects, M-10-26 (Washington, D.C.: June 28, 2010). [30] [hyperlink, http://www.gao.gov/products/GAO-12-681]. [31] HHS defines a major IT investment as an IT investment that involves one or more of the following: (1) has total planned outlays of $10 million or more in the budget year; (2) is for financial management and obligates more than $500,000 annually; (3) is otherwise designated by the HHS CIO as critical to the HHS mission or to the administration of HHS programs, finances, property or other resources; (4) has life-cycle costs exceeding $50 million. [32] HHS Acquisition Policy Memorandum 2009-05, Attachment A. [33] GAO, Centers for Medicare and Medicaid Services: Deficiencies in Contract Management Internal Control Are Pervasive, [hyperlink, http://www.gao.gov/products/GAO-10-60] (Washington, D.C.: Oct. 23, 2009) and [hyperlink, http://www.gao.gov/products/GAO-08-54]. In [hyperlink, http://www.gao.gov/products/GAO-10-60] we made 10 recommendations to the Administrator of CMS, OAGM management, and the Secretary of HHS to ensure adherence to FAR requirements and other control objectives. Nine of the 10 recommendations have been implemented. [34] As of April 2014, CMS had obligated more than $103 million for the data hub, which includes post-development operational and maintenance functions. [35] [hyperlink, http://www.gao.gov/products/GAO-04-393]. [36] The Exchange Life Cycle framework was also designed to support other IT efforts for the marketplaces, such as state-based exchanges. This framework was derived from CMS's Integrated IT Investment & System Life Cycle Framework and HHS's Enterprise Performance Life Cycle. During the course of the contracts, the Exchange Life Cycle Framework was replaced with CMS's Expedited Life Cycle process. [37] The cost increase was originally estimated to be $32 million in December 2012, but was negotiated to the lesser figure in the subsequent contract modification. [38] FAR § 43.102(a). [39] GAO, Standards for Internal Control in the Federal Government. [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999). [40] [hyperlink, http://www.gao.gov/products/GAO-08-54]. [41] CMS reported this information in the Contractor Performance Assessment Reporting System --the government-wide evaluation reporting tool for all past performance reports on contracts and orders. This report card assesses a contractor's performance and provides a record, both positive and negative, on a given contractor during a specific period of time. Each assessment is based on objective facts and supported by program and contract management data, such as cost performance reports, customer comments, quality reviews, technical interchange meetings, financial solvency assessments, construction/ production management reviews, contractor operations reviews, functional performance evaluations, and earned contract incentives. [42] CMS and CGI Federal exchanged a series of letters regarding CGI Federal's performance under the FFM task order in November 2013. In its initial response to CMS's November 2013 letter, CGI Federal addressed each issue identified by CMS and provided additional context on a variety of factors that CGI Federal believed influenced the FFM's development. [43] Technical direction letters provide supplementary guidance to contractors regarding tasks contained in their statements of work or change requests. [44] FAR Clause 52.246-5(d). In addition, CGI Federal's task order also provides that failure of the contractor to submit required reports when due or failure to perform or deliver required work, supplies, or services, may result in the withholding of payments under the contract unless such failure arises out of causes beyond the control, and without the fault or negligence of the contractor. HHSAR Clause 352.242-73. [45] FAR Clause 52.246-5(e). [46] The over $13.2 million in fee CGI Federal was eligible to receive includes fee for work performed during development and for post- transition support and consulting services from March to April 2014. [47] Under a cost-plus-award-fee contract, an award fee is intended to provide an incentive for excellence in such areas as cost, schedule, and technical performance; award of the fee is a unilateral decision made solely by the government. FAR § § 16.401(e)(2) and 16.405-2. [48] Contracts awarded on other than a full and open competitive basis must be justified and approved. FAR § 6.303. [49] The existing contract is a multiple-award, indefinite-delivery, indefinite-quantity contract (hereinafter referred to as the 2007 contract). This contract type provides for an indefinite quantity, within stated limits, of supplies or services during a fixed period. The Government places orders for individual requirements. Quantity limits may be stated as number of units or as dollar values. FAR § 16.504. [50] USAspending.gov is a free, publicly accessible website established by the Office of Management and Budget containing data on federal awards (e.g., contracts, loans, and grants) across the government. The Federal Procurement Data System-Next Generation, the primary government-wide contracting database, is one of the main data sources for this website. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]