VA/DOD Federal Health Care Center: Costly Information Technology Delays Continue and Evaluation Plan Lacking
Highlights
What GAO Found
Officials at the Department of Veterans Affairs (VA) and Department of Defense (DOD) Captain James A. Lovell Federal Health Care Center (FHCC) have continued to make progress implementing provisions of the Executive Agreement’s 12 integration areas, but delays in the information technology (IT) area have proven costly. Specifically, for 6 integration areas, all provisions have been implemented. Some of these areas were implemented at the time of GAO’s 2011 report, including establishing the facility’s governance structure and patient priority system, while 2 areas—quality assurance and contingency planning—were more recently implemented. In addition, 5 integration areas, such as property and fiscal authority, remain in progress. However, as previously reported by GAO, there have been delays implementing 1 of the integration areas—IT—which have resulted in additional costs for the FHCC, although the FHCC has been unable to quantify the total costs resulting from these delays. Despite an investment of more than $122 million for IT capabilities at the FHCC, VA and DOD have not completed work on all components required by the Executive Agreement, which were to have been in place in time for the FHCC’s opening in October 2010. These delays have resulted in additional costs and administrative burden for the FHCC because of the need for workarounds to address them. There also are other IT capabilities required by the Executive Agreement that are ill-defined and for which plans have not been established.
Although they are required by the National Defense Authorization Act (NDAA) for Fiscal Year 2010 to assess the FHCC at the end of the 5-year demonstration, VA and DOD officials said the departments have not yet established an evaluation plan. Officials told GAO that in addition to the performance data already being collected from 15 integration benchmarks established by the Executive Agreement, the departments also expect to consider other factors; however, these factors, which may include performance measures, have not yet been established. VA and DOD officials also have not yet established the standards, such as target scores for the benchmarks, the departments will use to evaluate FHCC performance. GAO has previously found that well-defined measures and standards are essential to a sound evaluation plan. Furthermore, without VA and DOD agreement on the measures and standards, FHCC leadership is unable to track progress and make any midcourse adjustments to improve performance in areas VA and DOD have determined are necessary for the FHCC’s success. Although including measures of FHCC costs in the evaluation would be consistent with the FHCC’s purpose, VA and DOD departmental priorities, and federal financial accounting standards, no such cost measures have been established for evaluating the FHCC.
Why GAO Did This Study
The NDAA for Fiscal Year 2010 authorized VA and DOD to establish a 5-year demonstration to integrate VA and DOD medical care into a first-of-its-kind FHCC in North Chicago, Illinois. Expectations for the FHCC are outlined in the Executive Agreement signed by VA and DOD in April 2010.
The NDAA for Fiscal Year 2010, as amended by the NDAA for Fiscal Year 2012, directed GAO to report on the FHCC demonstration in 2011, 2012, and 2015. This is the second of the three reports and examines (1) to what extent VA and DOD have continued to implement the Executive Agreement to establish and operate the FHCC and (2) what plan, if any, VA and DOD have to assess the provision of care and operations of the FHCC.
To conduct its work, GAO reviewed FHCC documents; interviewed VA, DOD, and FHCC officials; and reviewed related GAO work.
Recommendations
GAO recommends that VA and DOD (1) determine the costs associated with the workarounds required because of delays in implementing IT capabilities laid out in the FHCC Executive Agreement; (2) develop plans with clear definitions, specifications, deliverables, and time frames for IT capabilities required by the Executive Agreement but not yet defined; (3) develop and agree to an evaluation plan, to include all performance measures and standards to be used in evaluating the FHCC demonstration; and (4) establish measures related to the cost-effectiveness of the FHCC as part of their evaluation. VA and DOD generally concurred and noted steps to address GAOs recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Veterans Affairs | To clarify IT requirements within the Executive Agreement, to enable VA and DOD to make an informed recommendation about whether the FHCC should continue after the end of the demonstration, and to provide useful information for other integrations that may be considered in the future, the Secretaries of Veterans Affairs and Defense should determine the costs associated with the workarounds required because of delayed IT capabilities at the FHCC for each year of the demonstration, including the costs of hiring additional staff and of managing the administrative burden caused by the workarounds. |
In response to GAO's 2012 report, and as part of work completed for VA and DOD's evaluation of the FHCC, during March-July 2015 VA and DOD officials compiled information on costs associated with the FHCC's IT capabilities during the demonstration. This information included costs of workarounds that had resulted from delayed or unimplemented IT capabilities, such as for pharmacy staff hired to manually check DOD patient records for allergy information (due to the FHCC's lack of orders portability for pharmacy), and for additional laboratory staff (to address errors and latency with laboratory orders portability capabilities). However, VA and DOD were not able to determine the costs associated with all IT workarounds due in part to data limitations.
|
| Department of Defense | To clarify IT requirements within the Executive Agreement, to enable VA and DOD to make an informed recommendation about whether the FHCC should continue after the end of the demonstration, and to provide useful information for other integrations that may be considered in the future, the Secretaries of Veterans Affairs and Defense should determine the costs associated with the workarounds required because of delayed IT capabilities at the FHCC for each year of the demonstration, including the costs of hiring additional staff and of managing the administrative burden caused by the workarounds. |
In response to GAO's 2012 report, and as part of work completed for VA and DOD's evaluation of the FHCC, during March-July 2015 VA and DOD officials compiled information on costs associated with the FHCC's IT capabilities during the demonstration. This information included costs of workarounds that had resulted from delayed or unimplemented IT capabilities, such as for pharmacy staff hired to manually check DOD patient records for allergy information (due to the FHCC's lack of orders portability for pharmacy), and for additional laboratory staff (to address errors and latency with laboratory orders portability capabilities). However, VA and DOD were not able to determine the costs associated with all IT workarounds due in part to data limitations.
|
| Department of Veterans Affairs | To clarify IT requirements within the Executive Agreement, to enable VA and DOD to make an informed recommendation about whether the FHCC should continue after the end of the demonstration, and to provide useful information for other integrations that may be considered in the future, the Secretaries of Veterans Affairs and Defense should develop plans with clear definitions and specific deliverables, including time frames for two IT capabilities-documentation of patient care to support medical and dental operational readiness and outpatient appointment enhancements-and formalize these plans, for example, by incorporating them into the Executive Agreement. |
The Department of Defense (DOD) and the Department of Veterans Affairs (VA) are both working with the same contractor to deploy a single, interoperable electronic health record. Both VA and DOD opted to deploy a commercial system in order to create a seamless flow of digital records between the military branches as veterans transition to VA for care. While DOD and VA use two versions of the commercial platform, the electronic health record operates as one system and each patient has one record, no matter which facility they visit. DOD and VA have started to deploy the new electronic health record at a few health care facilities. The Federal Health Care Center (FHCC) is expected to deploy the new electronic health record early in the deployment process as a way to test the interoperability of the electronic health record. Because of DOD's and VA's efforts to deploy a new interoperable electronic health record, the need for the FHCC to develop its own plans and deliverables, including time frames, for the two IT capabilities is not necessary.
|
| Department of Defense | To clarify IT requirements within the Executive Agreement, to enable VA and DOD to make an informed recommendation about whether the FHCC should continue after the end of the demonstration, and to provide useful information for other integrations that may be considered in the future, the Secretaries of Veterans Affairs and Defense should develop plans with clear definitions and specific deliverables, including time frames for two IT capabilities-documentation of patient care to support medical and dental operational readiness and outpatient appointment enhancements-and formalize these plans, for example, by incorporating them into the Executive Agreement. |
The Department of Defense (DOD) and the Department of Veterans Affairs (VA) are both working with the same contractor to deploy a single, interoperable electronic health record. Both VA and DOD opted to deploy a commercial system in order to create a seamless flow of digital records between the military branches as veterans transition to VA for care. While DOD and VA use two versions of the commercial platform, the electronic health record operates as one system and each patient has one record, no matter which facility they visit. DOD and VA have started to deploy the new electronic health record at a few health care facilities. The Federal Health Care Center (FHCC) is expected to deploy the new electronic health record early in the deployment process as a way to test the interoperability of the electronic health record. Because of DOD's and VA's efforts to deploy a new interoperable electronic health record, the need for the FHCC to develop its own plans and deliverables, including time frames, for the two IT capabilities is not necessary.
|
| Department of Veterans Affairs | To clarify IT requirements within the Executive Agreement, to enable VA and DOD to make an informed recommendation about whether the FHCC should continue after the end of the demonstration, and to provide useful information for other integrations that may be considered in the future, the Secretaries of Veterans Affairs and Defense should expeditiously develop and agree to an evaluation plan, including the performance measures and standards, such as target scores, to be used to evaluate the FHCC demonstration, and formalize the plan, for example, by incorporating it into the Executive Agreement. |
In November 2013, VA and DOD contracted with an independent entity, Knowesis, Inc., to conduct a comprehensive evaluation and prepare a documented assessment of the FHCC demonstration. As part of this comprehensive evaluation, Knowesis, Inc. was required to develop an evaluation plan as well as "appropriate performance measures and standards based on industry standards for measuring success of similar undertakings," and was to use those standards to gauge the overall effectiveness and success of the FHCC. The evaluation plan was provided to VA and DOD by Knowesis, Inc. in January 2014 as a formalized deliverable per the contract.
|
| Department of Defense | To clarify IT requirements within the Executive Agreement, to enable VA and DOD to make an informed recommendation about whether the FHCC should continue after the end of the demonstration, and to provide useful information for other integrations that may be considered in the future, the Secretaries of Veterans Affairs and Defense should expeditiously develop and agree to an evaluation plan, including the performance measures and standards, such as target scores, to be used to evaluate the FHCC demonstration, and formalize the plan, for example, by incorporating it into the Executive Agreement |
In November 2013, VA and DOD contracted with an independent entity, Knowesis, Inc., to conduct a comprehensive evaluation and prepare a documented assessment of the FHCC demonstration. As part of this comprehensive evaluation, Knowesis, Inc. was required to develop an evaluation plan as well as "appropriate performance measures and standards based on industry standards for measuring success of similar undertakings," and was to use those standards to gauge the overall effectiveness and success of the FHCC. The evaluation plan was provided to VA and DOD by Knowesis, Inc. in January 2014 as a formalized deliverable per the contract.
|
| Department of Veterans Affairs | To clarify IT requirements within the Executive Agreement, to enable VA and DOD to make an informed recommendation about whether the FHCC should continue after the end of the demonstration, and to provide useful information for other integrations that may be considered in the future, the Secretaries of Veterans Affairs and Defense should establish measures related to the cost-effectiveness of the FHCC's care and operations to be included as a part of the evaluation plan. |
The FHCC evaluation plan developed by Knowesis, Inc. and provided to VA and DOD in January 2014 outlined six categories of demonstrated positive mutual benefit on which Knowesis would evaluate the FHCC. These categories included cost effectiveness as well as readiness, access to care, quality care, patient satisfaction, and staff satisfaction.
|
| Department of Defense | To clarify IT requirements within the Executive Agreement, to enable VA and DOD to make an informed recommendation about whether the FHCC should continue after the end of the demonstration, and to provide useful information for other integrations that may be considered in the future, the Secretaries of Veterans Affairs and Defense should establish measures related to the cost-effectiveness of the FHCC's care and operations to be included as a part of the evaluation plan. |
The FHCC evaluation plan developed by Knowesis, Inc. and provided to VA and DOD in January 2014 outlined six categories of demonstrated positive mutual benefit on which Knowesis would evaluate the FHCC. These categories included cost effectiveness as well as readiness, access to care, quality care, patient satisfaction, and staff satisfaction.
|