Information Technology: Opportunities Exist to Improve Management of DOD's Electronic Health Record Initiative
Highlights
The Department of Defense (DOD) provides medical care to 9.6 million active duty service members, their families, and other eligible beneficiaries worldwide. DOD's Military Health System has long been engaged in efforts to acquire and deploy an electronic health record system. The latest version of this initiative--the Armed Forces Health Longitudinal Technology Application (AHLTA)--was expected to give health care providers real-time access to individual and military population health information and facilitate clinical support. However, the system's early performance was problematic, and DOD recently stated that it intended to acquire a new electronic health record system. GAO was asked to (1) determine the status of AHLTA, (2) determine DOD's plans for acquiring its new system, and (3) evaluate DOD's acquisition management of the initiative. To do this, GAO reviewed program plans, reports, and other documentation and interviewed DOD officials.
After obligating approximately $2 billion over the 13-year life of its initiative to acquire an electronic health record system, as of September 2010, DOD had delivered various capabilities for outpatient care and dental care documentation. DOD had scaled back other capabilities it had originally planned to deliver, such as replacement of legacy systems and inpatient care management. In addition, users continued to experience significant problems with the performance (speed, usability, and availability) of the portions of the system that have been deployed. DOD has initiated efforts to improve system performance and enhance functionality and plans to continue its efforts to stabilize the AHLTA system through 2015, as a "bridge" to the new electronic health record system it intends to acquire. According to DOD, the planned new electronic health record system--known as the EHR Way Ahead--is to be a comprehensive, real-time health record for service members and their families and beneficiaries. The system is expected to address performance problems, provide unaddressed capabilities such as comprehensive medical documentation, capture and share medical data electronically within DOD, and improve existing information sharing with the Department of Veterans Affairs. As of September 2010, the department had established a planning office, and this office had begun an analysis of alternatives for meeting the new system requirements. Completion of this analysis is currently scheduled for December 2010. Following its completion, DOD expects to select a technical solution for the system and release a delivery schedule. DOD's fiscal year 2011 budget request included $302 million for the EHR Way Ahead initiative. Weaknesses in key acquisition management and planning processes contributed to AHLTA having fewer capabilities than originally expected, experiencing persistent performance problems, and not fully meeting the needs of users. (1) A comprehensive project management plan was not established to guide the department's execution of the system acquisition. (2) A tailored systems engineering plan did not exist to guide the technical development of the system, an effort that was characterized by significant complexity. (3) Requirements were incomplete and did not sufficiently reflect user and operational needs. (4) An effective plan was not used to improve users' satisfaction with the system. DOD has initiated efforts to bring its processes into alignment with industry best practices. However, it has not carried out a planned independent evaluation to ensure it has made these improvements. Until it ensures that these weaknesses are addressed, DOD risks undermining the success of further efforts to acquire electronic health record system capabilities. GAO is recommending that DOD take six actions to help ensure that it has disciplined and effective processes in place to manage the acquisition of further electronic health record system capabilities. In written comments on a draft of this report, DOD concurred with GAO's recommendations and described actions planned to address them.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense | To help guide and ensure the successful completion of the AHLTA stabilization effort, the Secretary of Defense, through the Assistant Secretary of Defense for Health Affairs, should direct the Military Health System's (MHS) Chief Information Officer (CIO) to develop and maintain a comprehensive project plan that includes key elements, such as the project's scope, cost, schedule, and risks and update the plan to provide key information for stakeholders on the project's plans and status; further, to help ensure that the Electronic Health Record (EHR) Way Ahead does not have shortfalls similar to those experienced with AHLTA, this management practice should be implemented as part of the planning for this important initiative. |
DOD concurred with this recommendation and developed a comprehensive project management plan, dated December 31, 2011, for the Armed Forces Health Longitudinal Technology Application Composite Health Care System (AHLTA/CHCS). The project plan reflected the approach that the program office used in managing enhancements to the system and was in accordance with DoD acquisition program guidelines in support of the AHLTA/CHCS initiative. The plan contained an integrated master schedule that provided key elements, including project scope, status, resources, detailed tasks and milestones, as well as a risk management plan, encompassing the identification, mitigation, and reporting of risks.
|
| Department of Defense | To help guide and ensure the successful completion of the AHLTA stabilization effort, the Secretary of Defense, through the Assistant Secretary of Defense for Health Affairs, should direct the MHS CIO to develop a systems engineering plan in accordance with DOD guidance to address the technical complexities of delivering a worldwide electronic health record system; further, to help ensure that the EHR Way Ahead does not have shortfalls similar to those experienced with AHLTA, this management practice should be implemented as part of the planning for this important initiative. |
DOD concurred with the recommendation to develop a systems engineering plan in accordance with DOD guidance to address the technical complexities of delivering a worldwide electronic health record system. The department subsequently provided its iEHR Capstone Systems Engineering Plan that was approved in August 2012. The plan reflected the succession of DOD projects from AHLTA to EHR Way Ahead to iEHR. It also discussed the technical strategies and complexities of the iEHR program and mapped key acquisition milestone events and deliverables to specific engineering activities.
|
| Department of Defense | To help guide and ensure the successful completion of the AHLTA stabilization effort, the Secretary of Defense, through the Assistant Secretary of Defense for Health Affairs, should direct the MHS CIO to ensure that its requirements development process involves system users throughout the development process, to obtain an understanding of what will satisfy their needs; further, to help ensure that the EHR Way Ahead does not have shortfalls similar to those experienced with AHLTA, this management practice should be implemented as part of the planning for this important initiative. |
DOD concurred with the recommendation to ensure that its requirements development process involved system users throughout the development process, in order to obtain an understanding of what will satisfy their needs. The department noted that it would continue to engage system users and manage the requirements development process in accordance with this recommendation. DOD subsequently provided us the Office of the Chief Information Officer Integrated Requirements Design Process Concept of Operations, dated January 23, 2009, that described the requirements development and management processes, which rely on its user community's full participation throughout the system lifecycle. Additionally, the department provided its Systems Engineering Plan, dated August 8, 2012, that identified a gated agile approach to development that required the functional community and the system's program office to collaboratively reprioritize and assess capability requirements as the development progressed.
|
| Department of Defense | To help guide and ensure the successful completion of the AHLTA stabilization effort, the Secretary of Defense, through the Assistant Secretary of Defense for Health Affairs, should direct the MHS CIO to ensure the establishment of bidirectional traceability for all system requirements; further, to help ensure that the EHR Way Ahead does not have shortfalls similar to those experienced with AHLTA, this management practice should be implemented as part of the planning for this important initiative. |
DOD concurred with the recommendation to establish bidirectional traceability for all system requirements. To address the need for bidirectional traceability across the departments, DoD leveraged its Program Management Accountability Tool and acquired "Software as a Service" (SaaS), which enabled requirements to be tracked against planned and actual execution of dollars; and included associated risks, schedule, and lessons learned at the product level. Additionally, DOD implemented the Serena Dimensions RM repository tool to facilitate bidirectional traceability between high-level business needs and the functional and technical requirements driving the acquisition of IT systems to fulfill those needs.
|
| Department of Defense | To help guide and ensure the successful completion of the AHLTA stabilization effort, the Secretary of Defense, through the Assistant Secretary of Defense for Health Affairs, should direct the MHS CIO to develop and document a plan for improving user satisfaction that prioritizes improvement projects; identifies needed resources; includes schedules for improvement efforts, including future user feedback surveys; and links efforts to measurable outcomes and specific user needs; further, to help ensure that the EHR Way Ahead does not have shortfalls similar to those experienced with AHLTA, this management practice should be implemented as part of the planning for this important initiative. |
DOD concurred with the recommendation to develop and document a plan for improving user satisfaction and, in August 2012, developed a comprehensive user satisfaction survey tool to gauge AHLTA's frequencies and scope of use, as well as users' satisfaction related to the system's ease of use, speed, and availability. The survey also identified outcome measures, including the timeliness of completing patient encounters and the prioritization of needed improvements. The department used the survey to evaluate the worldwide deployment of AHLTA Release 3.3, which was completed in December 2012.
|
| Department of Defense | To help guide and ensure the successful completion of the AHLTA stabilization effort, the Secretary of Defense, through the Assistant Secretary of Defense for Health Affairs, should direct the MHS CIO to establish acquisition management processes in accordance with industry best practices, including identifying milestones and a completion date for the external evaluation that MHS's processes are at maturity level 2 of the Capability Maturity Model Integration for Acquisition; further, to help ensure that the EHR Way Ahead does not have shortfalls similar to those experienced with AHLTA, this management practice should be implemented as part of the planning for this important initiative. |
DOD concurred with the recommendation to follow DOD acquisition program guidelines and establish a milestone for an external review in accordance with the Capability Management Maturity Model guidelines. However, DOD subsequently provided a memorandum, dated November 14, 2011, that postponed formal external assessments of all Military Health System projects conducted by the Software Engineering Institute, which included the assessment for maturity levels 2 and 3 of the Capability Maturity Model Integration for Acquisition.
|