Healthcare.gov: CMS Has Taken Steps to Address Problems, but Needs to Further Implement Systems Development Best Practices
Highlights
What GAO Found
Several problems with the initial development and deployment of Healthcare.gov and its supporting systems led to consumers encountering widespread performance issues when trying to create accounts and enroll in health plans:
Inadequate capacity planning: The Centers for Medicare & Medicaid Services (CMS) did not plan for adequate capacity to support Healthcare.gov and its supporting systems.
Software coding errors: CMS and its contractors identified errors in the software code for Healthcare.gov and its supporting systems, but did not adequately correct them prior to launch.
Lack of functionality: CMS had not implemented all planned functionality prior to the initial launch of Healthcare.gov and its supporting systems.
Since the initial launch, CMS has taken steps to address these problems, including increasing capacity, requiring additional software quality reviews, and awarding a new contract to complete development and improve the functionality of key systems. After it took these actions, performance issues affecting Healthcare.gov and its supporting systems were significantly reduced.
In addition, CMS did not consistently apply recognized best practices for system development, which contributed to the problems with the initial launch of Healthcare.gov and its supporting systems.
Requirements were not effectively managed: Requirements management helps ensure that a project's plans and work products are aligned with the needs of users. However, CMS did not always ensure that requirements were approved and were linked to source and lower-level requirements. As a result, CMS was hindered in ensuring that expected functionality for the system was delivered.
System testing was inconsistent. Testing is essential for ensuring that a system operates as intended. However, Healthcare.gov and its supporting systems were not fully tested prior to launch, and test documentation was missing key elements such as criteria for determining whether a system passed a test. Thus, CMS's assurance that these systems would perform as intended was limited.
Project oversight was not effective. Oversight includes monitoring a project's progress and taking corrective actions when its performance deviates from what is planned. However, CMS's oversight was limited by an unreliable schedule, lack of estimates of work needed to complete the project, unorganized and outdated project documentation, and inconsistent reviews of project progress.
As it has undertaken further development, CMS has made improvements in some of these areas, by, for example, establishing new requirements management processes and improving test documentation. However, weaknesses remain in its application of requirements, testing, and oversight practices. In addition, the Department of Health and Human Services (HHS) has not provided adequate oversight of the Healthcare.gov initiative through its Office of the Chief Information Officer. The Office of Management and Budget's (OMB) oversight role was limited, and GAO has previously recommended that it improve oversight of IT projects' performance.
Why GAO Did This Study
The Patient Protection and Affordable Care Act required the establishment of health insurance marketplaces to assist individuals in obtaining health insurance coverage. CMS, a component of HHS, was responsible for establishing a federally facilitated marketplace for states that elected not to establish their own. This marketplace is supported by an array of IT systems, which are to facilitate enrollment in qualifying health plans. These include Healthcare.gov, the website that serves as the consumer portal to the marketplace, as well as systems for establishing user accounts, verifying eligibility, and facilitating enrollment.
GAO was asked to review CMS's management of the development of IT systems supporting the federal marketplace. Its objectives were to (1) describe problems encountered in developing and deploying systems supporting Healthcare.gov and determine the status of efforts to address deficiencies and (2) determine the extent to which CMS applied disciplined practices for managing and overseeing the development effort, and the extent to which HHS and OMB provided oversight. To do this, GAO reviewed program documentation and interviewed relevant CMS and other officials.
Recommendations
GAO is recommending that CMS take seven actions to implement improvements in its requirements management, system testing, and project oversight, and that HHS improve its oversight of the Healthcare.gov effort. HHS concurred with all of the recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Health and Human Services | To improve requirements management for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to document the approval process for functional and technical design requirements documentation. |
In December 2016, we verified that, in response to our recommendation, CMS had documented an approval process for functional and technical design requirements. As a result of these actions, the agency is better positioned to ensure that requirements are understood and approved by system stakeholders.
|
| Department of Health and Human Services | To improve requirements management for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to implement the CMS procedure to obtain signatures from the three key stakeholders--the CMS business owner, the CMS approval authority, and the contractor organization approving authority--to ensure that stakeholders have a shared understanding of all business, functional, and technical requirements for systems supporting Healthcare.gov prior to developing them. |
In January 2017, we verified that, in response to our recommendation, CMS had implemented an approval process and obtained the signatures of key stakeholders when developing business, functional, and technical requirements. As a result of these actions, CMS is better positioned to ensure that requirements for Healthcare.gov systems are clearly defined, agreed upon, and approved before development begins.
|
| Department of Health and Human Services | To improve systems testing processes for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to document and approve systems testing policy and procedures, including (1) the use of the system testing tool designed to integrate systems development and systems testing and (2) requirements for stakeholder review of systems test documentation that is intended to ensure proper test coverage and to validate the results. |
In May 2017, we verified that CMS met the intent of our recommendation by requiring the use of certain tools for systems testing and requiring stakeholder review of systems test documentation. As a result of these actions, the agency is better positioned to ensure that testing was properly carried out and the system will function as intended.
|
| Department of Health and Human Services | To improve systems testing processes for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to require key information in system test plans, as recommended by best practices, including the means by which the quality of testing processes will be assured, and the identification of responsibilities for individuals or groups carrying out testing. |
In May 2017, we verified that CMS met the intent of our recommendation by requiring review and approval of testing documentation and defining roles and responsibilities of those carrying out the testing. As a result of these actions, the agency is better positioned to ensure that testing will be consistently executed and of sufficient quality to validate the system will address requirements.
|
| Department of Health and Human Services | To improve systems testing processes for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to require and ensure key information is included in test cases, as recommended by best practices, such as all outputs and exact values; test case dependencies; inputs required to execute each test case; and information about whether each test item has passed or failed testing. |
In December 2016, we verified that, in response to our recommendation, CMS had incorporated key elements defined in best practices, such as expected testing outputs, inputs required to execute the test cases, and information about whether each test item has passed or failed elements defined in best practices into their test cases we reviewed. As a result, the agency is better positioned to ensure the systems supporting Healthcare.gov are performing as intended.
|
| Department of Health and Human Services | To improve oversight processes for systems development activities related to systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to ensure schedules for the Healthcare.gov effort are well constructed by, among other things, (1) logically sequencing activities, (2) confirming the critical paths are valid, and (3) identifying reasonable total float. |
In October 2015, we verified that, in response to our recommendation, CMS had improved the development of schedules by, in general, logically sequencing all work activities, ensuring a valid critical path, and reflecting a reasonable total float. As a result, the agency's project schedules should now be a more effective tool for gauging project progress and estimates of timelines for completing project activities.
|
| Department of Health and Human Services | To improve oversight processes for systems development activities related to systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to develop and implement policy and procedures for estimating level of effort to ensure effort is estimated at the appropriate level (requirements or program area), and define how levels of effort will be used to monitor system development progress. |
In September 2015, we verified that CMS, in response to our recommendation, had developed and documented procedures for estimating levels of effort and had defined how levels of effort are to be used to monitor system development progress. By taking these actions, the agency should be better positioned to ensure that level-of-effort estimations are applied in a consistent manner and provide for more accurate monitoring of progress.
|
| Department of Health and Human Services | To improve oversight for Healthcare.gov and its supporting systems, the Secretary of Health and Human Services should direct the Department of Health and Human Services' Chief Information Officer to carry out authorized oversight responsibilities. Specifically, the Chief Information Officer should ensure the department-wide investment review board is active and carrying out responsibilities for overseeing the performance of high-risk IT investments such as those related to Healthcare.gov. |
In May 2017, we verified that the HHS IT Steering Committee, which serves as the department's IT investment review board for projects such as Healthcare.gov, was actively reviewing projects. As a result of these actions, the agency is better able to exercise their oversight role and may be better able to take corrective actions on poorly performing projects.
|