This is the accessible text file for GAO report number GAO-15-55 entitled 'VA Health Care: Improvements Needed in Monitoring Antidepressant Use for Major Depressive Disorder and in Increasing Accuracy of Suicide Data' which was released on December 12, 2014. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to the Chairman, Subcommittee on Oversight and Investigations, Committee on Veterans' Affairs, House of Representatives: November 2014: VA Health Care: Improvements Needed in Monitoring Antidepressant Use for Major Depressive Disorder and in Increasing Accuracy of Suicide Data: GAO-15-55: GAO Highlights: Highlights of GAO-15-55, a report to the Chairman, Subcommittee on Oversight and Investigations, Committee on Veterans' Affairs, House of Representatives. Why GAO Did This Study: In 2013, VA estimated that about 1.5 million veterans required mental health care, including services for MDD. MDD is a debilitating mental illness related to reduced quality of life and productivity, and increased risk for suicide. VA also plays a role in suicide prevention. GAO was asked to review how VA tracks veterans prescribed antidepressants and what suicide data VA uses in its prevention efforts. This report examines (1) VA's data on veterans with MDD, including those prescribed an antidepressant; (2) the extent that veterans with MDD who are prescribed antidepressants receive recommended care and the extent to which VA monitors such care; and (3) the quality of data VA requires VAMCs to collect on veteran suicides. GAO analyzed VA data, interviewed VA officials, and conducted site visits to six VAMCs selected based on geography and population served. From each of these six VAMCs, GAO also reviewed five randomly selected medical records for veterans diagnosed with MDD and prescribed an antidepressant in 2012, as well as all completed BHAP templates. The results cannot be generalized across VA but provide insights. What GAO Found: GAO's analysis of Department of Veterans Affairs (VA) data for fiscal years 2009 through 2013 shows that about 10 percent of veterans who received VA health care services were diagnosed with major depressive disorder (MDD). MDD is characterized by depressed mood or loss of interest along with other symptoms for 2 weeks or more that represent a change in the way individuals function from their previous behaviors. Because GAO found diagnostic coding discrepancies in 11 of the 30 veterans' medical records it reviewed from six VA medical centers (VAMC), VA's data may understate the prevalence of MDD among veterans being treated through VA, to the extent that such discrepancies may permeate VA's data. One treatment for MDD is the use of medications such as antidepressants. According to GAO's analysis, 94 percent of veterans diagnosed with MDD were prescribed at least one antidepressant. VA policy states that antidepressant treatment must be consistent with VA's current clinical practice guideline (CPG); however, GAO's review of 30 veterans' medical records identified deviations from selected MDD CPG recommendations for most veterans reviewed. For example, 26 of the 30 veterans were not assessed using a standardized assessment tool at 4 to 6 weeks after initiation of treatment, as recommended in the CPG. Additionally, 10 veterans did not receive follow up within the time frame recommended in the CPG. GAO found that VA does not have a system- wide process in place to identify and fully assess whether the care provided is consistent with the CPG. As a result, VA does not know the extent to which veterans with MDD who have been prescribed antidepressants are receiving care as recommended in the CPG and whether appropriate actions are taken by VAMCs to mitigate potentially significant risks to veterans. The demographic and clinical data that VA collects on veteran suicides were not always complete, accurate, or consistent. VA's Behavioral Health Autopsy Program (BHAP) is a quality initiative to improve VA's suicide prevention efforts by identifying information that VA can use to develop policy and procedures to help prevent future suicides. The BHAP templates are a mechanism by which VA collects suicide data from VAMC's review of veteran medical records. GAO's review of 63 BHAP templates at five VAMCs found that 40 of the templates that VAMCs submitted to VA Central Office had incomplete data. Also, GAO found that the BHAP templates VAMCs submitted contained inaccurate data. For example, 6 BHAP templates included a date of death that was incorrect based on information in the veteran's medical record, and 9 BHAP templates included an incorrect number of outpatient VA mental health visits in the last 30 days. Moreover, GAO found that VAMCs submitted inconsistent information because they interpreted VA's guidance on completing the BHAP templates differently. This situation was further exacerbated because BHAP templates prepared by VAMCs are generally not being reviewed at any level within the Department for completeness, accuracy, and consistency. Lack of complete, accurate, and consistent data and poor oversight can inhibit VA's ability to identify, evaluate and improve ways to better inform its suicide prevention efforts. What GAO Recommends: GAO recommends that VA identify and address MDD coding discrepancies; implement processes to review data and assess deviations from recommended care; and implement processes to improve completeness, accuracy, and consistency of veteran suicide data. VA concurred with GAO's recommendations and described its plans to implement them. View [hyperlink, http://www.gao.gov/products/GAO-15-55]. For more information, contact Randall B. Williamson at (202) 512-7114 or williamsonr@gao.gov. [End of section] Contents: Letter: Background: VA Data Show That 10 Percent of Veterans Had MDD and Most Were Prescribed at Least One Antidepressant, but VA Data May Underestimate MDD Prevalence: Not All Veterans in Our Review Received CPG-Recommended Care and VA Lacks Mechanisms to Determine Whether This Care Is Provided: Demographic and Clinical Data VA Collects on Veteran Suicides Were Not Always Complete or Accurate, and VAMCs Applied Instructions for Gathering Suicide Data Differently: Conclusions: Recommendations for Executive Action: Agency Comments: Appendix I: Scope and Methodology: Appendix II: Department of Veterans Affairs Medical Centers' Tracking of Veterans at High Risk for Suicide: Appendix III: Department of Veterans Affairs' Use of Data Related to Suicides and Suicide Behavior: Appendix IV: Comments from the Department of Veterans Affairs: Appendix V: GAO Contact and Staff Acknowledgments: Tables: Table 1: Characteristics of Veterans Who Received Health Care through the Department of Veterans Affairs and had a Diagnosis of Major Depressive Disorder, Fiscal Years 2009-2013: Table 2: Number of Veterans in GAO's Sample Not Receiving Care As Recommended in the Clinical Practice Guideline (CPG) for Major Depressive Disorder (MDD): Table 3: Description of Department of Veterans Affairs and Department of Defense Data Files Used to Examine Characteristics of Veterans with Major Depressive Disorder for Fiscal Years 2009 through 2013: Table 4: Characteristics of Veterans Affairs Medical Centers (VAMC) and Community-Based Outpatient Clinics Selected for GAO's Review: Figures: Figure 1: Number of Behavioral Health Autopsy Program (BHAP) Post- Mortem Chart Analysis Templates with Incomplete or Inaccurate Data: Figure 2: Excerpts of the Department of Veterans Affairs' (VA) Behavioral Health Autopsy Program (BHAP) Template in Which Selected VA Medical Centers Submitted Incomplete Data: Figure 3: Excerpts of the Department of Veterans Affairs' (VA) Behavioral Health Autopsy Program (BHAP) Template in Which Selected VA Medical Centers Submitted Inaccurate Data: Figure 4: Excerpts of the Department of Veterans Affairs' (VA) Behavioral Health Autopsy Program (BHAP) Template in Which Selected VA Medical Centers Submitted Inconsistent Data: Abbreviations: BHAP: Behavioral Health Autopsy Program: BHL: Behavioral Health Laboratory: CPG: clinical practice guideline: DOD: Department of Defense: Handbook: Uniform Mental Health Services in VA Medical Centers and Clinics handbook: MDD: major depressive disorder: OIG: Office of Inspector General: OMHO: Office of Mental Health Operations: OMHS: Office of Mental Health Services: PHQ: Patient Health Questionnaire: SPAN: Suicide Prevention Application Network: TIDES: Translating Initiatives for Depression into Effective Solutions: VA: Department of Veterans Affairs: VAMC: Veterans Affairs medical center: VISN: Veterans Integrated Service Network: [End of section] United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: November 12, 2014: The Honorable Mike Coffman: Chairman: Subcommittee on Oversight and Investigations: Committee on Veterans' Affairs: House of Representatives: Dear Mr. Chairman: In 2013, the Department of Veterans Affairs (VA) estimated that about 1.5 million veterans required mental health services, which VA provides in a variety of settings, including VA medical centers (VAMC), community-based outpatient clinics, and residential treatment programs.[Footnote 1] Mental health treatment includes services for depression--a mood disorder that causes a persistent feeling of sadness and loss of interest. One type of depression, major depressive disorder (MDD), is a particularly debilitating mental illness and is associated with reduced quality of life, reduced productivity, and increased risk for suicide. These negative effects underscore the importance of timely, evidence-based assessment for and treatment of MDD, which may include medications, such as antidepressants, psychotherapy, or a combination of both. Treatment of veterans with MDD can improve their occupational and social functioning and their overall well-being. In addition to providing ongoing care to veterans with MDD, VA plays a role in suicide risk assessment and prevention among veterans. According to VA, about one-quarter of the 18 to 22 veterans who die by suicide each day were receiving care through VA.[Footnote 2] Research has identified numerous risk factors for suicide among veterans, which include substance use disorder, physical impairments, previous suicide attempts, and depression. Additionally, life stressors, such as marital or financial problems, contribute to a veteran's risk of suicide. Given the debilitating effects of depression, VA's monitoring of veterans with MDD is critical to ensuring positive health outcomes. Additionally, the relatively high veteran suicide rate makes it important that VA effectively employs data related to suicides in its prevention efforts. You asked us to review how VA tracks and manages veterans prescribed antidepressants and what suicide data VA uses in its suicide prevention efforts. In this report, we examine: 1. the data VA has on veterans with MDD, including the extent to which they were prescribed antidepressants; 2. the extent to which veterans with MDD who are prescribed an antidepressant receive recommended care and the extent to which VA monitors such care; and: 3. the information VA requires VAMCs to collect on veteran suicides. To examine the data VA has on veterans with MDD, including the extent to which veterans were prescribed antidepressants, we analyzed national VA data from fiscal years 2009 through 2013 and interviewed VA officials.[Footnote 3] The data we analyzed included information on veterans' demographic characteristics, health care services, and medications provided through VA.[Footnote 4] We considered a veteran to have MDD if, in at least one fiscal year included in our review, the veteran had two or more outpatient encounters or at least one inpatient hospital stay associated with a diagnosis of MDD.[Footnote 5] To ensure the reliability of the data we analyzed, we interviewed VA Central Office officials, reviewed relevant documentation and veterans' medical records, and conducted electronic testing to identify missing data and obvious errors.[Footnote 6] On the basis of these activities, we determined that the data we analyzed were sufficiently reliable for our purposes. We also conducted site visits to 6 of the 150 VAMCs selected for variation in complexity of health care services offered, geographic location, and number of veterans using mental health services. These six VAMCs were located in Canandaigua, New York; Gainesville, Florida; Iowa City, Iowa; Philadelphia, Pennsylvania; Phoenix, Arizona; and Reno, Nevada. [Footnote 7] At each site, we interviewed VAMC staff about the data maintained on veterans diagnosed with MDD. In addition, we reviewed a random, nongeneralizeable sample of medical records for five veterans treated at each of the six VAMCs for a total of 30 veterans. We selected veterans for review that were diagnosed with MDD and had a new treatment episode of an antidepressant in calendar year 2012. Starting with the initial encounter in 2012 that was associated with the new antidepressant prescription, we reviewed the medical records to determine if the diagnostic code entered for subsequent encounters was consistent with a diagnosis of MDD. To analyze the extent to which VA monitors whether veterans with MDD who are prescribed an antidepressant receive care as recommended by VA's clinical practice guideline (CPG) for MDD, we reviewed VA policy documents and interviewed VA Central Office officials responsible for developing and implementing VA mental health policy.[Footnote 8] We identified the oversight processes VA has in place to monitor the recommendations in the CPG and assessed whether this oversight provides VA with adequate information to identify nonconformance with recommended practices, assess the risk, and address nonconformance, as appropriate. We selected three CPG recommendations for inclusion in our review that had among the highest strength of research evidence, were sufficiently specific to enable us to determine the extent to which VA providers were following the recommendation, and would not require clinical judgment to determine the extent to which VA providers were following the recommendation.[Footnote 9] To examine how the recommendations we selected were implemented, we reviewed the 30 medical records from our selected six VAMCs to assess the extent to which the antidepressant treatment-related care VAMCs provided was consistent with the CPG recommendations. We also interviewed VAMC officials regarding the care provided to veterans with MDD prescribed an antidepressant. Results from our 30 medical record reviews cannot be generalized to the VAMC visited or other VAMCs. To analyze the information VA requires VAMCs to collect on veteran suicides, we reviewed VA policies, guidance, and documents related to VA's suicide prevention efforts to identify the data collected by VA staff on veteran suicides. We also interviewed VA Central Office and other officials responsible for VA's suicide prevention program. Additionally, through the site visits to six VAMCs, we obtained documents and interviewed officials regarding the collection of veteran suicide data and suicide prevention initiatives. We compared 63 data templates related to veteran suicides completed by VAMC staff as of the date of our site visits--or the date of our request for virtual site visits--to information included in the veterans' medical records and to templates we received from VA Central Office.[Footnote 10] We identified fields in the documents to review based on whether the field related to aspects of VA treatment--including treatment for mental health conditions--and the date of the veteran's death. We identified these fields because they did not require clinical judgment to assess. Results from our review of veteran suicide data can be generalized to the VAMCs we visited, but cannot be generalized to other VAMCs. We also interviewed relevant staff of the six Veterans Integrated Service Networks (VISN), or regional networks of care, for the sites we visited to obtain information on suicide prevention efforts in that VISN.[Footnote 11] For more information on our scope and methodology, see appendix I. We conducted this performance audit from November 2013 to November 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: VA provides care to veterans with mental health needs through its 150 VAMCs, which may include both specialty mental health care settings-- including mental health clinics--and other settings that may provide mental health services but focus primarily on other types of care, such as primary care. VA has implemented a program to co-locate mental health care providers within primary care settings in an effort to promote effective treatment of common mental health conditions in the primary care environment while allowing mental health specialists to focus on veterans with more severe mental illnesses. Care for Veterans with MDD: According to VA, the prevalence of MDD in primary care settings among veterans being treated through VA is higher than that among the general population. MDD is characterized by the presence of depressed mood or loss of interest or pleasure along with other symptoms for a period of at least 2 weeks that represent a change in previous functioning.[Footnote 12] VA has policies and guidance in place related to treating veterans with MDD. For example, the Uniform Mental Health Services in VA Medical Centers and Clinics handbook (Handbook), which defines VA's minimum clinical requirements for mental health services, requires that VA facilities provide evidence-based treatment through the administration of medication, when indicated, consistent with the MDD CPG.[Footnote 13] The CPG is guidance intended by VA to reduce current practice variation between clinicians and provide facilities with a structured framework to help improve patient outcomes.[Footnote 14] The MDD CPG provides evidence-based recommendations as guidance for clinicians who provide care for veterans with MDD.[Footnote 15] The MDD CPG includes approximately 200 recommendations to provide information and assist in decision making for clinicians who provide care for adults with MDD.[Footnote 16] For example, the CPG recommends that standardized assessments of depressive symptoms, such as the nine item Patient Health Questionnaire (PHQ-9), should be used at the initial assessment of MDD symptoms, to monitor treatment response at 4- 6 weeks after initiation of treatment, after each change in treatment, and periodically thereafter until full remission is achieved.[Footnote 17] Evidence shows that follow-up assessment is effective 4-6 weeks after initiation of treatment, making timely follow-up visits an important part of clinicians' ability to assess whether the current treatment plan is effective or should be modified. According to the MDD CPG, veterans with MDD treated with antidepressants should be closely observed, particularly at the beginning of treatment and following dosage changes, to maximize veterans' recovery and to mitigate any negative treatment effects, including worsening of depressive symptoms. The CPG should not take the place of the clinician's clinical judgment. VA Suicide Prevention Efforts: Beginning in June 2006, VA implemented several initiatives aimed at suicide prevention, including appointing a National Suicide Prevention Coordinator, developing data systems to increase understanding of suicide among veterans and inform VA suicide prevention programs, and instituting suicide prevention programs in VAMCs throughout the country. Additionally, VA Central Office established the Center of Excellence for Suicide Prevention and the Veterans Crisis Line in 2007. The Center of Excellence collects VA suicide prevention program data, which provides information on veteran suicide completions and suicide attempts for veterans receiving VA care, as well as those veterans not receiving VA care. VA's Veterans Crisis Line provides toll-free, confidential support 24 hours per day for veterans, their families, and their friends through phone, online chat, or text message.[Footnote 18] In fiscal year 2013, the Veterans Crisis Line fielded approximately 287,000 calls, 54,800 online chats, and 11,300 text messages. As part of VAMCs' suicide prevention programs, the Handbook requires each VAMC to have a suicide prevention coordinator whose responsibilities include: * establishing and maintaining a list of veterans assessed to be at high risk for suicide; * monitoring these high-risk veterans; * responding to referrals from staff and the Veterans Crisis Line; * collaborating with community organizations and partners; * training staff members who have contact with veterans at the VAMC, community organizations, and partners; and: * collecting and reporting information on veterans who die by suicide and who attempt suicide. See appendix II for more information on VAMCs' tracking of veterans at high risk for suicide. VA Suicide Data: VA Central Office uses several mechanisms to collect data on veteran suicides to help improve its suicide prevention efforts. One such mechanism includes data submitted by suicide prevention coordinators at VAMCs on known veterans who die by suicide.[Footnote 19] Beginning in December 2012, VA Central Office began a national initiative to collect demographic, clinical, and other related information on veteran suicides as a quality improvement initiative to improve its suicide prevention efforts by identifying information that can be used by VA Central Office to develop policy and procedures to help prevent future veteran deaths.[Footnote 20] This initiative, the Behavioral Health Autopsy Program (BHAP), replaced previous VA Central Office requirements to collect data on completed suicides.[Footnote 21] VA Central Office officials explained that they transitioned to the BHAP initiative to collect more systematic and comprehensive information about suicides, to incorporate interviews of family members of those who die by suicide, and to collect more contextual information. According to VA, the BHAP quality initiative has been adapted from a traditional psychological autopsy research framework that emphasizes the importance of information from outside sources as well as from those within the health care setting. The BHAP initiative is being implemented by VA in four phases: * Phase 1--Standardized chart reviews: VAMCs' suicide prevention coordinators are required to complete standardized chart reviews for all veterans' suicides known to VAMC staff and reported on or after October 1, 2012.[Footnote 22] These reviews include specific information on a veteran's utilization of VA health care services, as well as a veteran's mental health diagnoses and risk factors for suicide. VA Central Office has instructed suicide prevention coordinators to use all available information, including VA medical records and information from a veteran's family members to complete the chart review. These reviews are submitted to VA Central Office through completion of a BHAP Post-Mortem Chart Analysis Template (BHAP template) and VA Central Office has provided suicide prevention coordinators with a BHAP Guide on how to complete the fields in the BHAP template. VA Central Office requires VAMCs to submit the BHAP template within 30 days of VAMC staff becoming aware of a veteran's death by suicide. * Phase 2--Interviews with family members: In fiscal year 2013, VA Central Office began conducting interviews with family members of veterans who have died by suicide to obtain information on suicide risks, barriers to care, and suggestions for new programs to prevent suicide.[Footnote 23] * Phase 3--Clinician questionnaire: This phase, which has not yet been implemented, will include an interview with the last provider that saw the veteran prior to his or her death. VA officials stated that there are no plans to begin this phase within calendar year 2014, and they have not established a future time table for implementing this phase. * Phase 4--Public record review: This phase, which has also not been implemented, will be used to locate public records to identify stressors in the veteran's life, such as a bankruptcy or divorce. Officials stated that there are no plans to begin this phase within calendar year 2014, and they have not established a future time table for implementing this phase. Since beginning the BHAP initiative, VA Central Office has internally issued two interim reports on data and trends from the submitted BHAP templates as part of Phase 1. The reports include information for veterans who died by suicide, both with and without a history of VA health care service utilization. Analyses of data on demographic characteristics, case information, period of service, and risk and protective factors were included for all veterans. Data on clinical characteristics and indicators of increased risk at the time of the veteran's last contact with a VA provider were limited to veterans that utilized VA health care services. In addition to the BHAP initiative, VA also requires VAMCs to collect and submit data on suicide attempts and completions through the following mechanisms.[Footnote 24] * Suicide Prevention Application Network (SPAN): Through SPAN, VAMCs submit information to VA Central Office on the number of veterans that completed suicide, the number of suicide attempts, and indicators of suicide prevention efforts, such as outreach events conducted each month by suicide prevention coordinators. * Suicide behavior reports: VAMC clinicians must complete a suicide behavior report when they learn that a veteran attempted or completed suicide and add that report to the respective veteran's medical record. This report includes the date and time of the event, and other observations related to the suicide attempt or completed suicide. According to VA policy, information from suicide behavior reports is used for National Patient Safety reporting requirements and to populate SPAN. * Root cause analyses: Patient safety managers at VAMCs complete root cause analyses for suicide attempts and completed suicides under certain circumstances, such as when the attempt occurs at the VAMC during an inpatient stay or within 72 hours of being discharged from inpatient care. Root cause analyses are used to identify the factors that contributed to adverse events or close calls and any steps VAMCs could implement to prevent similar events in the future. See appendix III for how VAMC and VISN officials we interviewed told us they have utilized data related to suicides and suicide behavior. VA Data Show That 10 Percent of Veterans Had MDD and Most Were Prescribed at Least One Antidepressant, but VA Data May Underestimate MDD Prevalence: Data for fiscal years 2009 through 2013 show that about 10 percent of veterans who received health care services through VA were diagnosed with MDD, and of those, 94 percent were prescribed an antidepressant. However, due to diagnostic coding discrepancies we found in a sample of veterans' medical records, VA's data may not accurately reflect the prevalence of MDD among veterans. VA Data Show About 10 Percent of Veterans Had a Diagnosis of MDD, and Almost All Were Prescribed at Least One Antidepressant: Based on our analysis of VA data from veterans' medical records and administrative sources, 532,222 veterans--about 10 percent of veterans who received health care services through VA[Footnote 25]--had a diagnosis of MDD from fiscal years 2009 through 2013. Among those veterans, the majority (60 percent) were 35 to 64 years of age. Most (86 percent) were not veterans of the recent conflicts in Iraq and Afghanistan.[Footnote 26] In addition, most of these veterans were male (87 percent) and the highest proportion was white (68 percent) and non-Hispanic (87 percent). See table 1 for a summary of characteristics of veterans who had a diagnosis of MDD from fiscal years 2009 through 2013. Table 1: Characteristics of Veterans Who Received Health Care through the Department of Veterans Affairs and had a Diagnosis of Major Depressive Disorder, Fiscal Years 2009-2013: Veteran characteristics: Age: 18-24; Number of veterans with major depressive disorder: 3,347; Percentage of veterans with major depressive disorder: 0.6%. Veteran characteristics: Age: 25-34; Number of veterans with major depressive disorder: 54,054; Percentage of veterans with major depressive disorder: 10.2%. Veteran characteristics: Age: 35-44; Number of veterans with major depressive disorder: 57,458; Percentage of veterans with major depressive disorder: 10.8%. Veteran characteristics: Age: 45-54; Number of veterans with major depressive disorder: 96,321; Percentage of veterans with major depressive disorder: 18.1%. Veteran characteristics: Age: 55-64; Number of veterans with major depressive disorder: 164,640; Percentage of veterans with major depressive disorder: 30.9%. Veteran characteristics: Age: 65-74; Number of veterans with major depressive disorder: 115,358; Percentage of veterans with major depressive disorder: 21.7%. Veteran characteristics: Age: 75+; Number of veterans with major depressive disorder: 41,044; Percentage of veterans with major depressive disorder: 7.7. Veteran characteristics: Era of service: Recent conflicts in Iraq & Afghanistan[A]; Number of veterans with major depressive disorder: 75,934; Percentage of veterans with major depressive disorder: 14.3%. Veteran characteristics: Era of service: All other eras; Number of veterans with major depressive disorder: 456,288; Percentage of veterans with major depressive disorder: 85.7%. Veteran characteristics: Sex: Male; Number of veterans with major depressive disorder: 460,660; Percentage of veterans with major depressive disorder: 86.6%. Veteran characteristics: Sex: Female; Number of veterans with major depressive disorder: 71,562; Percentage of veterans with major depressive disorder: 13.5%. Veteran characteristics: Race: White; Number of veterans with major depressive disorder: 361,921; Percentage of veterans with major depressive disorder: 68.0%. Veteran characteristics: Race: Black; Number of veterans with major depressive disorder: 96,511; Percentage of veterans with major depressive disorder: 18.1%. Veteran characteristics: Race: Asian & Pacific Islander; Number of veterans with major depressive disorder: 7,098; Percentage of veterans with major depressive disorder: 1.3%. Veteran characteristics: Race: Native American; Number of veterans with major depressive disorder: 3,484; Percentage of veterans with major depressive disorder: 0.7%. Veteran characteristics: Race: Other[B]; Number of veterans with major depressive disorder: 23,375; Percentage of veterans with major depressive disorder: 4.4%. Veteran characteristics: Race: Not Identified; Number of veterans with major depressive disorder: 14,727; Percentage of veterans with major depressive disorder: 2.8%. Veteran characteristics: Race: Missing Information; Number of veterans with major depressive disorder: 25,106; Percentage of veterans with major depressive disorder: 4.7%. Veteran characteristics: Ethnicity: Non-Hispanic; Number of veterans with major depressive disorder: 465,448; Percentage of veterans with major depressive disorder: 87.5%. Veteran characteristics: Ethnicity: Hispanic; Number of veterans with major depressive disorder: 38,577; Percentage of veterans with major depressive disorder: 7.3%. Veteran characteristics: Ethnicity: Not Identified; Number of veterans with major depressive disorder: 14,438; Percentage of veterans with major depressive disorder: 2.7%. Veteran characteristics: Ethnicity: Missing Information; Number of veterans with major depressive disorder: 13,759; Percentage of veterans with major depressive disorder: 2.6%. Source: GAO analysis of Veterans Affairs and Department of Defense data. GAO-15-55. Notes: N=532,222 veterans. Percentages may not equal 100 due to rounding. [A] These conflicts include Operation Iraqi Freedom, Operation Enduring Freedom, and Operation New Dawn. [B] This category includes veterans who identified themselves as Hispanic or multiple race. [End of table] We also found that about 499,000 of the 532,222 (94 percent) veterans who had a diagnosis of MDD from fiscal years 2009 through 2013 were prescribed at least one antidepressant. Of those veterans, the majority (about 73 percent) were dispensed a 12-week supply of an antidepressant at the start of an MDD episode. Fewer veterans (about 58 percent) were dispensed a 6-month supply of an antidepressant over the course of their treatment. Receiving a 12-week supply of an antidepressant can be important for addressing depressive symptoms initially, while continued treatment after remission of depressive symptoms, such as receiving a 6-month supply of an antidepressant, is associated with a decreased risk of relapse, according to the CPG. VA's Data May Not Fully Reflect the Extent to Which Veterans Have MDD Due to a Lack of Diagnostic Coding Precision by Clinicians: Based on our review of the documentation in 30 veterans' medical records from VA's medical record system, we found that over one-third (11) had diagnostic coding discrepancies. Specifically, these 11 veterans had at least one encounter where the clinician documented a diagnosis of MDD in the veteran's medical record, but the clinician did not code the encounter accordingly. Instead, the clinician coded the encounter as "depression not otherwise specified," a less specific code.[Footnote 27] According to the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders, depression not otherwise specified is to be used to code disorders with depressive features that do not meet criteria for MDD and other depressive disorders, [Footnote 28] or to indicate depressive symptoms about which there is inadequate or contradictory information.[Footnote 29] VA's data on the number of veterans with MDD are based on the diagnostic codes associated with patient encounters, so the discrepancies we found indicate that the number of veterans with MDD is most likely not fully reflected in these data. Accurately identifying the veteran population with MDD is critical to assessing Department performance in treating veterans in accordance with the MDD CPG and measuring health outcomes for these veterans. VA Central Office reviewed the 11 medical records where we found coding discrepancies and agreed that the encounters were not coded accurately. According to a VA Central Office official, the encounters we identified were corrected in the veterans' medical record. According to VHA Handbook 1907.03 - Health Information Management Clinical Coding Program Procedures, VAMCs are required to monitor the accuracy of coding and provide training as necessary in order to help ensure accurate coding. VAMC officials from all six sites in our review said that monthly or quarterly coding audits are conducted at their facilities and the findings of those audits are reviewed and action is taken to correct issues with the accuracy and reliability of coding. However, at five of the six VAMCs in our review, those audits focus on billable encounters--that is, encounters that are billed to a third party, such as private health insurance plans--in part because of the potential opportunity for facilities to collect third-party revenue from these encounters.[Footnote 30] Among the 11 veterans' medical records where we identified coding discrepancies, all of the discrepancies were associated with outpatient, nonbillable encounters, the coding of which, according to a VA Central Office official, is not typically conducted by VAMC medical coders--staff who are trained specifically in medical coding terminology and standards and are responsible for coding inpatient admissions and discharges--or subject to coding audits. Diagnostic coding in VA's medical record system for outpatient encounters is typically performed by clinicians. VISN officials and VA medical center clinicians we interviewed said that clinicians do not place a lot of importance on selecting a more precise diagnostic code because it does not significantly change the patient care that is provided or the type of treatment prescribed. In addition, in the interest of expediency, clinicians may select a previously used or frequently used diagnostic code for depression rather than take the time to search for a more precise code. For example, within the medical record, clinicians may access a list of previous or current diagnoses applicable to the veteran (commonly referred to as the "problem list") or a list of frequently used diagnostic codes in the facility. According to VISN and VAMC officials, the problem list is not typically kept up to date by clinicians and as a result, MDD may not be listed and readily available for clinicians to select. As a result of our review, VA Central Office officials reported that they had discovered a software mapping error in VA's medical record system where the selection of MDD as a diagnosis when using a keyword search function may result in the selection of the depression not otherwise specified diagnostic code by mistake.[Footnote 31] Officials stated that they anticipate that the software error--which applies to all VAMCs--would be fixed by November 2014. Officials also stated that the solution would apply only to those encounters coded from that point going forward and would not retroactively correct any coding discrepancies that may have occurred before the error was addressed. VA Central Office officials could not tell us if any of the 11 coding discrepancies that we identified were a result of this software error. Officials at most of the six VISNs we spoke with do not conduct reviews of medical coding done by clinicians. However, as a result of our inquiry, one VISN we interviewed reported in the late spring of 2014 that it had extracted data on MDD-related encounters and noticed the high use of depression not otherwise specified coding for the facilities within its VISN, as well as all VAMCs nationwide. Officials from this VISN said the lack of coding specificity has implications for being able to accurately examine health outcomes related to the treatment of depression and that they are planning to further analyze encounter data within their VISN to determine the appropriateness of diagnostic coding based upon medical record documentation. As of September 2014, the VISN had not reported any additional steps to address this issue. Not All Veterans in Our Review Received CPG-Recommended Care and VA Lacks Mechanisms to Determine Whether This Care Is Provided: Based on the three CPG recommendations we selected, veterans in our review with MDD who have been prescribed antidepressants did not always receive care as recommended in the MDD CPG. Additionally, VA does not know the extent to which veterans with MDD who have been prescribed antidepressants are receiving care as recommended in the CPG, and VA Central Office has not developed mechanisms to determine the extent to which mental health care delivery conforms to the recommendations in the MDD CPG. Veterans We Reviewed Did Not Always Receive Care As Recommended: We found that almost all of the 30 veterans with MDD who have been prescribed antidepressants included in our review did not receive care in accordance with the three MDD CPG recommendations we reviewed. VA policy states that antidepressant treatment must be consistent with VA's current, evidence-based CPG.[Footnote 32] However, VA Central Office mental health officials were unable to tell us what it means to provide care that is consistent with the CPG, because, while a veteran's treatment should be informed by the CPG recommendations, determining the extent to which the treatment is consistent with CPG recommendations would need to be done on a veteran-by-veteran basis. The CPG is intended to reduce practice variation and help improve patient outcomes, but without an understanding of the extent to which veterans are receiving care that is consistent with the CPG, VA may be unable to ensure that it meets the intent of the CPG and improves veteran health outcomes. Through our review of 30 medical records from the six VAMCs we selected, we found examples of deviations from the CPG recommendations for almost all veterans in our review.[Footnote 33] Table 2 below depicts the specific recommendations we reviewed and the number of veterans that did not receive care consistent with the corresponding CPG recommendation. Table 2: Number of Veterans in GAO's Sample Not Receiving Care As Recommended in the Clinical Practice Guideline (CPG) for Major Depressive Disorder (MDD): CPG recommendation: To enhance antidepressant treatment, veterans should be educated on when to take the medication, possible side effects, risks, and the expected duration of treatment, among other things; Number of veterans not receiving care as recommended in the CPG for MDD: 6 of 30 veterans lacked documentation of patient education when the medication was prescribed. CPG recommendation: Standardized assessments of depressive symptoms, such as the Patient Health Questionnaire-9, should be used to monitor treatment at 4-6 weeks after initiation of treatment and after each change in treatment; Number of veterans not receiving care as recommended in the CPG for MDD: 26 of 30 veterans were not assessed using a standardized assessment tool at 4-6 weeks after initiation of treatment; 18 of 30 veterans were not assessed using a standardized assessment tool at any encounter[B]; 10 of 30 veterans did not have a follow-up encounter that occurred 4-6 weeks after initiation of treatment[C]. CPG recommendation: A plan should be developed that addresses the duration of antidepressant treatment, among other things; Number of veterans not receiving care as recommended in the CPG for MDD: 1 veteran of 30 did not have a planned date for follow up and plan for future care documented in the veteran's medical record at the initial encounter. Source: GAO analysis of Veterans Affairs (VA) data. GAO-15-55. Note: We included 30 veterans in our review. Our review began with the encounter during which a VA clinician ordered an antidepressant to treat depressive symptoms (initial encounter) and five follow-up encounters with a VA clinician, or sooner if the veteran did not have five follow-up encounters. Our review was limited to encounters during which the antidepressant treatment was reviewed, including encounters during which side effects and treatment effect were assessed, but no change was made to medication orders. [A] Of the 30 veterans included in our review, only 6 were assessed using a standardized assessment at the initial encounter where antidepressant medication was prescribed. VA Central Office officials explained that they would expect a standardized assessment to be conducted at the start of an antidepressant to establish a baseline score. [B] Three veterans did not receive a follow-up appointment at all. Two veterans did not show for scheduled appointments that were within the CPG recommended time frame. Five veterans did not have a follow-up encounter until after 6 weeks. [End of table] For example, the CPG recommends that veterans' depressive symptoms be assessed at 4-6 weeks after initiation of antidepressant treatment to determine the efficacy of the treatment. However, medical records of 10 of the 30 veterans we reviewed indicated that they did not have a follow-up encounter within the CPG's recommended 4-6 week time frame. Of those 10 veterans, * 3 veterans did not have any follow-up encounters related to mental health; * 2 veterans did not show for scheduled appointments that were within the CPG recommended time frame; and: * 5 veterans did not have a follow-up encounter until after 6 weeks. This follow up ranged from 7 weeks to 50 weeks after the initial encounter.[Footnote 34] Some clinicians at VAMCs we visited described instances in which they generally do not follow the CPG recommendations.[Footnote 35] For example, not all clinicians use a standardized assessment to evaluate a veteran's symptoms. Officials from two VAMCs we visited explained that they do not routinely use the PHQ-9. According to one of these VAMCs, the standard of care is a clinical interview and observation. However, the CPG recommendation states that the PHQ-9 combined with a clinical interview should be used to obtain the necessary information about symptoms and symptom severity. It also states that the PHQ-9 improves diagnostic accuracy and aids treatment decisions by quantifying symptom severity. Furthermore, clinicians may also prefer different time frames for follow up, which are not always consistent with the CPG recommendation. For example, one clinician told us that he tries to see the veteran 3 weeks after the initial appointment, and clinicians at another VAMC like to see the veteran within 6 weeks, but told us that this is sometimes difficult due to scheduling constraints.[Footnote 36] VA Lacks Mechanisms to Determine the Extent to Which Veterans Are Receiving Care Consistent with CPG Recommendations: VA does not know the extent to which veterans are receiving care consistent with the MDD CPG. While deviations from recommended practice may be appropriate in many cases due to clinician discretion, VA has not fully assessed whether these examples are acceptable deviations from the CPG. According to the federal internal control standard for risk assessment, agencies should comprehensively identify risks, assess the possible effects, if any, and determine what actions should be taken to mitigate any significant risks. VA Central Office has not developed a mechanism to fully identify deviations that could impede veterans' recovery that may result when VAMCs do not provide care consistent with the MDD CPG. VA Central Office officials explained that the CPG recommendations are guidelines that clinicians can use to inform and guide clinical decision making. VA officials told us that VA cannot require the use of all recommendations in all cases; rather, CPG recommendations should be applied on a case-by-case basis based on the needs of the veteran and with clinician judgment. One official also said it would be difficult to check every CPG recommendation to ensure that clinicians are providing care consistent with the CPG, but stated that VA could identify for review those recommendations that may put veterans' health at risk if not followed.[Footnote 37] However, with no mechanism to assess whether the care provided is consistent with the CPG, VA is unable to ensure that deviations from recommended care are identified. While monitoring full compliance with CPG recommendations may be difficult, there are nevertheless ways to address the issue. In fact, VA Central Office and some VAMCs have implemented mechanisms to determine the extent to which veterans are receiving care that is consistent with some of the CPG recommendations; however, these mechanisms do not fully assess all deviations that could impede a veteran's recovery, as illustrated by the following. * In fiscal year 2014, VA began to review mental health care delivery at VAMCs through a psychopharmacology quality improvement initiative consisting of a series of prescribing practices metrics, although this initiative does not fully assess the extent to which care is consistent with the MDD CPG.[Footnote 38] One metric--the proportion of veterans with depression prescribed three or more concurrent antidepressant medications for 60 or more continuous days--relates to 2 of the over 200 MDD CPG recommendations.[Footnote 39] According to one VA Central Office official, the initial phase of the initiative is focused on decreasing variability in prescribing practices among VAMCs. To start the initiative, VA compiled metric profiles for each VAMC, which included a description of how each VAMC performed for all metrics as well as the performance of each VAMC relative to the VISN and national averages. VA requires that all VAMCs address metrics where performance was lower than the national average. Currently, VA is assisting VAMCs with developing action plans to address these metrics. Because the initiative is in the early stages, it is too soon to determine how VA Central Office and VAMCs will review and remedy deviations from the two CPG recommendations addressed by this initiative. According to VA Central Office officials, the psychopharmacology quality improvement initiative is a mechanism VA currently has to look at CPG recommendations, and VA could add additional metrics addressing other CPG recommendations in the future. However, VA Central Office officials stated that other CPG recommendations would require a more in depth analysis, such as medical record reviews, and it would be difficult to develop metrics for some of the recommendations.[Footnote 40] * While VA Central Office does not have a mechanism to determine the extent to which VAMCs are providing care consistent with a majority of the CPG recommendations, some VAMCs have implemented a software system to help ensure that veterans with MDD who are prescribed antidepressants receive care consistent with the CPG when the veteran is treated in a primary care clinic.[Footnote 41] At the time of our site visits, three of the VAMCs we visited were using such a system-- the Behavioral Health Laboratory (BHL).[Footnote 42] The BHL is designed to manage the behavioral health needs of veterans through telephone or in-person visits. As part of the system, clinicians can use a structured interview--including a PHQ-9--that assesses veterans' mental health symptoms in a way that is consistent with the CPG recommendation for follow-up assessment. Although the BHL can be used to help ensure care is provided consistent with a few of the recommendations in the CPG, the BHL is not used to monitor all veterans prescribed antidepressants. Generally, VAMCs use the BHL to monitor veterans being treated for mental health conditions, such as MDD, in primary care clinics, and to participate, veterans can be referred by their primary care clinician or request to participate. [Footnote 43] Demographic and Clinical Data VA Collects on Veteran Suicides Were Not Always Complete or Accurate, and VAMCs Applied Instructions for Gathering Suicide Data Differently: We found that demographic, clinical, and other data submitted to VA Central Office on veteran suicides were not always completely or correctly entered into the BHAP Post-Mortem Chart Analysis Templates-- a mechanism by which VA Central Office collects veteran suicide data from VAMCs' review of veterans' medical records. (Figure 1 shows the number of BHAP templates we found with incomplete or inaccurate data.) Moreover, VAMCs interpreted and applied instructions for completing the BHAP templates differently. We also found that most VAMCs and VISNs we reviewed and VA Central Office did not review suicide data for accuracy. Figure 1: Number of Behavioral Health Autopsy Program (BHAP) Post- Mortem Chart Analysis Templates with Incomplete or Inaccurate Data: [Refer to PDF for image: horizontal bar graph] Incomplete and inaccurate data from BHAP templates: Veteran was receiving VA services but BHAP template did not indicate such: Number of BHAP templates: 23. Data fields left blank: Number of BHAP templates: 40. Date of death incorrect: Number of BHAP templates: 6. Number of outpatient mental health visits incorrect: Number of BHAP templates: 9. Source: GAO analysis of VA data. GAO-15-55. [End of figure] Veteran Suicide Data Are Incomplete: We found that over half of the 63 BHAP templates we examined had incomplete information.[Footnote 44] The data either lacked veteran enrollment information, or other specific fields were omitted. Moreover, the data were lacking entirely for certain known veteran suicides. Incomplete data limits VA Central Office's ability to identify information that can be used to help VA Central Office develop policy and procedures to prevent veteran deaths. Lack of veteran enrollment information. Approximately one-third (23) of the BHAP templates we reviewed did not indicate whether the veteran was enrolled in VA health care services, even though the veteran had a VA medical record.[Footnote 45] * Eight did not indicate that the veteran had received VA services when the templates were submitted by three of the VAMCs in our review, even though these VAMCs provided care to these veterans. * Fifteen BHAP templates submitted by two VAMCs in our review originally indicated that the veteran was receiving VA care; however, when we reviewed the submitted BHAP templates we received from VA Central Office for the same 15 veterans, the BHAP templates did not indicate that the veteran was being seen in the VA.[Footnote 46] VA Central Office used enrollment information when compiling the most recent BHAP interim report, which is part of VA Central Office's quality improvement efforts for its suicide prevention program. [Footnote 47] Specifically, VA Central Office included clinical data in the BHAP interim report only for veterans utilizing VA services. Therefore, clinical data for the 23 veterans we identified would not be included in the interim report. Missing one-third of the data from its analysis, as was the case in our sample, could have a detrimental effect on the trends VA Central Office reports and uses to improve its suicide prevention efforts. Requested data was omitted. Forty of the 63 BHAP templates we reviewed included various data fields where no response was provided, resulting in incomplete data. For example, for 19 templates, VAMC staff did not enter requested data as to whether the veteran had all or some of 15 active psychiatric symptoms within 12 months prior to the veteran's date of death.[Footnote 48] Also, 9 templates did not include an answer for the number of previous suicide attempts by the veteran. Officials from one VAMC told us that they left this field blank if the veteran did not have any previous suicide attempts, rather than entering a "0," even though the BHAP Guide states that officials should enter the appropriate number of previous suicide attempts. Officials at one VAMC told us that fields are sometimes left blank if the standardized answers available on the BHAP template are not adequate; that is, the answer for that veteran does not fit into one of the answers provided on the BHAP template.[Footnote 49] Officials at two VAMCs stated that it is sometimes easy to overlook fields in the BHAP template, resulting in unanswered questions. Filling in all fields in the BHAP template, rather than leaving the field blank, is important because some blank fields are counted as "missing" or "no" in the analysis conducted by VA Central Office for the BHAP interim reports. This, in turn, could affect the suicide trends reported. For example, for the number of previous suicide attempts, blank fields are counted as "missing" in the BHAP interim report, rather than "0" previous suicide attempts as officials from one VAMC intended. In other cases, such as for psychiatric symptoms, missing fields are counted as "no," meaning that the veteran did not have these symptoms. In at least one BHAP template, the answer for the psychiatric symptom of isolation was left blank, and would therefore be counted as negative in the interim report despite the fact that officials from the one VAMC told us that the veteran did have this symptom. See figure 2, which provides an excerpt of the fields from the BHAP template in which VAMCs provided incomplete data. Figure 2: Excerpts of the Department of Veterans Affairs' (VA) Behavioral Health Autopsy Program (BHAP) Template in Which Selected VA Medical Centers Submitted Incomplete Data: [Refer to PDF for image: template excerpt] Is the Case concerning a Veteran who WAS enrolled in VA Health Care Services? This box should be checked to indicate whether the veteran was enrolled in VA health care services. Prominent Psychiatric Symptoms: The corresponding bubble should be filled in to indicate active symptoms the veteran may have experienced anytime within 12 months prior to the date of death.[A] Number of Previous Suicide Attempts: This box should contain the number of previous suicide attempts, as indicated in the veteran's electronic medical record. Source: GAO analysis of VA documents. GAO-15-55. [A] The BHAP template has changed over time and "unknown" has not always been an option in this section of the BHAP template. [End of figure] Data were lacking entirely for certain known veteran suicides. We found that VAMCs did not always submit BHAP templates for all veteran suicides known to the facility, as required by the BHAP Guide. VA Central Office does not have a process in place to determine whether it is receiving the BHAP templates for all known veteran suicides.[Footnote 50] For example, one VAMC had completed 13 BHAP templates at the time of our site visit but had not submitted them; however, neither the VAMC nor VA Central Office were aware that these templates had not been submitted until after we requested them from VA Central Office. The suicide prevention coordinator at this VAMC told us that the BHAP templates were forwarded to another official at the VAMC, rather than being submitted through VA Central Office's process, and that the BHAP templates were never submitted. As a result of our inquiry, the VAMC submitted these templates to VA Central Office. In another example, officials at a different VAMC told us that, at the time of our site visit, they had recently begun completing and submitting BHAP templates, beginning with veteran suicides occurring in fiscal year 2014. VA Central Office officials told us that VAMCs can start submitting BHAP templates at any point, and officials are not requiring the VAMCs to go back and submit information on all suicides since October 1, 2012. However, this practice is contrary to VA policy, which states that VAMCs should submit BHAP templates for all suicides known to the facility and reported on or after October 1, 2012. Veteran Suicide Data Are Inaccurate: Of the 63 BHAP templates we reviewed, we found numerous instances of inaccurate data submitted on BHAP templates, as illustrated by the following examples. * Incorrect date of death: Six BHAP templates included a date of death that was incorrect based on information in the veteran's medical record.[Footnote 51] The difference in the dates of death in the veterans' medical records and the dates of death in the BHAP templates ranged from 1 day to 1 year. For example, one BHAP template indicated that the veteran died in the year after the veteran's actual date of death. Another BHAP template appeared to use the date the suicide behavior report was completed, rather than the veteran's actual date of death. The suicide behavior report was completed 69 days after the veteran's date of death.[Footnote 52] The accuracy of the date of death recorded in the BHAP template is important because it is used as a point of reference to calculate other fields, such as the number of mental health visits in the last 30 days. * Incorrect number of mental health visits: Nine BHAP templates included the incorrect number of outpatient VA mental health visits in the last 30 days.[Footnote 53] For example, one BHAP template indicated that the veteran had five outpatient mental health visits, including three non-mental health visits that should not have been included in the total number of mental health visits for this veteran. Another BHAP template indicated the veteran had been seen once by a mental health provider in the last 30 days; however, we found in reviewing the medical records that this veteran had not been seen by a mental health provider during this time period. This veteran would be included in the BHAP interim report as having a mental health visit, and, as a result, VA's data would include an inaccurate count of the number of veterans with mental health visits in the last 30 days. [Footnote 54] Without accurate information, VA cannot use this information to determine whether policies or procedures need to be changed to ensure that veterans at high risk for suicide are being seen more frequently by a mental health provider to help prevent suicides in the future. See figure 3, which provides an excerpt of the fields from the BHAP template in which VAMCs provided inaccurate data. Figure 3: Excerpts of the Department of Veterans Affairs' (VA) Behavioral Health Autopsy Program (BHAP) Template in Which Selected VA Medical Centers Submitted Inaccurate Data: [Refer to PDF for image: template excerpt] Date of Death: This field should include the month, date, and year of the veteran's death. Number of Outpatient VA MH visits in the last 30 days? This field indicates the total number of outpatient mental health visits in the last 30 days. Source: GAO analysis of VA documents. GAO-15-55. [End of figure] VAMCs Have Interpreted and Applied Instructions for Completing the BHAP Templates Differently: We found several situations where VAMCs interpreted and applied instructions for completing the BHAP templates differently, as illustrated in the following examples.[Footnote 55] * We found inconsistencies in how different VAMCs arrived at answers provided in the BHAP templates. For example, one VAMC included a visit to an immunization clinic as the veteran's final visit, while another VAMC did not include this type of visit, even though this was the last time the veteran was seen in person. The BHAP Guide indicates that the final visit should be the last time the veteran had in-person contact with any VAMC staff, but the BHAP Guide does not identify the different types of visits that should be counted.[Footnote 56] VA Central Office officials stated that a visit to an immunization clinic should be included as the final visit with the veteran. When VAMCs do not provide consistent data, VA Central Office will receive and use inconsistent data in preparing its trend reports, such as BHAP interim reports, which are intended to be used to improve suicide prevention efforts. * We also found instances in which BHAP templates included information that did not conform to the instructions in the BHAP Guide on how to complete the BHAP medical record reviews. - Last contact did not always represent the last time a VAMC official spoke with the veteran: The BHAP Guide instructions specify that the last contact recorded in the BHAP template should be the last recorded interaction with the veteran, which could be in person, through a phone call, or through email. Five of the 63 BHAP templates we reviewed did not indicate the last time an official spoke directly to the veteran. One BHAP template counted a phone call with a veteran's spouse after the veteran's death as the last contact with the veteran. The BHAP template also counted this phone conversation as an "in- person" interaction. The remaining four BHAP templates included a date for the last contact that was prior to the date for the veteran's final in-person visit at the VAMC. In these instances, the veterans' in-person visit should have been counted as the last contact. From this flawed information, VA would not be able to determine reliable trends for the amount of time between the last contact with the veteran and the veteran's date of death for reports that it prepares, such as the BHAP interim report. - Suicide prevention coordinator contact and referral not within BHAP time period: The BHAP Guide specifies that VAMCs should indicate in the BHAP template whether there was a suicide prevention coordinator contact or referral made within 3 months prior to the veteran's date of death. In 3 of the 63 cases we reviewed, we found that the suicide prevention coordinators checked the box indicating that they saw the veteran or had a referral within 3 months of the veteran's death. However, in each of these cases we found that the contact was made more than 3 months prior to the veteran's death, so it should not have been counted. A suicide prevention coordinator from one VAMC said she was unaware of the time period requirement and a suicide prevention coordinator at another VAMC stated that time frames should be added to the BHAP template, rather than just included in the BHAP Guide. The BHAP interim reports include the number of veterans that had a suicide prevention coordinator contact or referral, and by including information on contacts or referrals that are outside the BHAP Guide time frame, these reports may be at risk of misreporting trends in this area. See figure 4, which provides an excerpt of the fields from the BHAP template in which VAMCs provided inconsistent data. Figure 4: Excerpts of the Department of Veterans Affairs' (VA) Behavioral Health Autopsy Program (BHAP) Template in Which Selected VA Medical Centers Submitted Inconsistent Data: [Refer to PDF for image: template excerpt] Date of Final Visit: The final visit should include the last time the veteran had in-person contact with VA staff. Date of Last Contact: The date of last contact should be the last recorded personal interaction with the veteran, including in-person visits, phone calls, or contact via email. SPC Contact/Referral? The BHAP Guide indicates that this field should include referrals within 3 months of the date of death; however, this time frame is not specified in the BHAP template. Source: GAO analysis of VA documents. GAO-15-55. [End of figure] VA policy and guidance states that the BHAP template should be completed for all suicides known to the facility, but at the five VAMCs we visited, these data were not always being reported.[Footnote 57] However, the policy and instructions do not explicitly state that veterans not being seen by VA should be included, and in the absence of this declaration, some VAMCs interpreted the instructions to mean that only veterans being seen by VA should be included in the data submitted. Therefore, two VAMCs have submitted data only for veterans being treated by VA, while the others include data on all known veteran suicides--whether they have been treated by VA or not. This further adds to the inconsistencies in the information that VAMCs submit on the BHAP templates. VA Central Office officials told us that BHAP templates should be completed for both veterans utilizing VA health care services, as well as those veterans not being seen in the VA, and that this requirement has been discussed at training sessions and during conference calls with suicide prevention coordinators. For example, during a suicide prevention conference in November 2013, a VA Central Office official informed participants that the BHAP template should be completed for all suicides reported through SPAN, which VA Central Office officials previously told us includes veterans that were not receiving VA care. The inconsistency in VAMC officials' understanding of which veterans should have a completed BHAP template results in inconsistent data being reported to VA Central Office. While VA was in the process of updating its suicide prevention coordinator manual, we brought this issue to VA's attention. In August 2014, VA made modifications to the manual that indicated that VA is changing its policy--now requiring that the BHAP template should be completed only for veterans receiving VA services. However, the guidance continues to be unclear on whether suicide prevention coordinators should complete BHAP templates for veterans not receiving VA care.[Footnote 58] VAMCs, VISNs, and VA Central Office Do Not Review Suicide Data: We found that BHAP templates are not being reviewed by VA officials at any level for accuracy, completeness, and consistency. Therefore, our findings at five VAMCs could be symptomatic nationwide and other VAMCs may also be submitting incomplete, inaccurate, and inconsistent suicide-related information and VA may not be getting the data it needs across the Department to make appropriate resource decisions and develop new policy. VA policy states that it is the VISN's and VAMC's decision whether to conduct reviews of BHAP data prior to submission to VA Central Office. With few exceptions, VAMCs and VISNs we visited generally do not conduct data checks on the information submitted in the BHAP templates. Additionally, VA Central Office does not review the information for accuracy and completeness in the BHAP templates it receives. This approach is inconsistent with internal control standards for the federal government, which state that agencies should have controls over information processes, including procedures and standards to ensure the completeness and accuracy of processed data. [Footnote 59] Officials at one VAMC told us that VAMC staff compare the BHAP data and the veteran's medical record prior to submitting the BHAP template to VA Central Office to ensure accuracy. In response to our review, another VAMC implemented a procedure to check the accuracy and completeness of their BHAP templates prior to submission. The procedure at this VAMC requires the suicide prevention coordinator and case manager to independently complete the BHAP template and compare their responses. The BHAP templates are then reviewed by the Assistant Mental Health Clinic Director prior to submission. We also found that VA lacks sufficient controls to ensure the quality of the existing BHAP data. For example, VA Central Office officials said there are no automated data checks to ensure the accuracy of data it uses for the BHAP interim report, such as checking to ensure that the date of last contact with the veteran that is recorded in the BHAP template is not after the veteran's date of death. Although officials removed apparent duplicates in submitted BHAP templates by matching the veteran's name and social security number while compiling the data for the most recent BHAP interim report, they do not conduct data checks to help identify some of the incomplete or inaccurate data we found in our review. Conclusions: Given the negative effects of MDD, it is important to provide timely, evidence-based treatment for veterans with MDD, and VA's ability to monitor these veterans is critical to ensuring positive outcomes. However, our findings demonstrate that VA may not be fully aware of the population of veterans with MDD due to a lack of coding precision by clinicians. This can limit VA's ability to assess the Department's performance in treating veterans as recommended in the MDD CPG and in measuring health outcomes for veterans. Additionally, VA does not have mechanisms in place to ensure that the Department is able to identify deviations from CPG-recommended care and remedy those that could impede veterans' recovery. Even if VA did have mechanisms in place, the coding discrepancies we identified would limit VA's ability to extract accurate data on all veterans diagnosed with MDD, therefore hindering VA's ability to determine the extent to which veterans are receiving care consistent with the CPG recommendations for MDD. The CPG recommendations are meant to improve veteran outcomes by providing maximum relief from the debilitating symptoms of MDD, and VA cannot ensure that the care veterans receive is consistent with those recommendations. The existence of incomplete, inaccurate, and inconsistent information submitted through VA's BHAP templates limits the Department's ability to accurately evaluate its suicide prevention efforts and identify trends in veteran suicides through the BHAP initiative. Specifically, data drawn from incomplete, inaccurate, and inconsistent BHAP templates limit the Department's opportunities to learn from past veteran suicides and ultimately diminish efforts to improve its suicide prevention activities. VAMCs, VISNs, and VA Central Office generally lack a process to ensure that the data that are submitted and used by VA Central Office to identify trends in veteran suicides are complete, accurate, and consistent. Checking and verifying the data submitted to VA Central Office would help ensure that changes made to suicide prevention efforts by VAMCs, VISNs, and VA are based on actual trends in veteran suicides. Without clear VA Central Office instructions to guide how VAMCs and VISNs should complete BHAP templates and report suicide data, the validity of suicide data and the effectiveness of VA's actions will be hampered. Recommendations for Executive Action: We recommend that the Secretary of Veterans Affairs direct the Under Secretary for Health to take the following six actions: To more accurately estimate the prevalence of MDD and identify enrolled veterans with MDD, VA should: * identify the extent to which there is imprecise diagnostic coding of MDD by further examining encounters with a diagnostic code of depression not otherwise specified, which could be incorporated into VAMCs' ongoing review of diagnostic coding accuracy, and: * determine and address the factor(s) contributing to the imprecise coding based on the results of those examinations. For example, feedback and additional training could be provided to clinicians regarding the importance of diagnostic code accuracy, or VA's medical record could be enhanced to facilitate the selection of a more accurate diagnostic code. To ensure that veterans are receiving care in accordance with the MDD CPG, VA should: * implement processes to review data on veterans with MDD prescribed antidepressants to evaluate the level of risk of any deviations from recommended care and remedy those that could impede veterans' recovery. To improve VA's efforts to inform its suicide prevention activities, VA should: * ensure that VAMCs have a process in place to review data on veteran suicides for completeness, accuracy, and consistency before the data are submitted to VA Central Office, * clarify guidance on how to complete BHAP templates to ensure that VAMCs are submitting consistent data on veteran suicides, and: * implement processes to review data on veteran suicides submitted by VAMCs for accuracy and completeness. Agency Comments: We provided a draft of this report to VA for comment. In its written comments, reproduced in appendix IV, VA generally agreed with our conclusions and concurred with our recommendations. In addition, VA provided information on its plans for implementing each recommendation, with estimated completion dates in calendar year 2015. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report's date. At that time, we will send copies of this report to the appropriate congressional committees; the Secretary of Veterans Affairs; the VA Under Secretary for Health; and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. As agreed with your office, unless you announce the contents of this report earlier, we plan no further distribution of it until 30 days from the report date. If you or your staff have any questions, please contact me at (202) 512-7114 or williamsonr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Major contributors to this report are listed in appendix V. Sincerely yours, Signed by: Randall B. Williamson: Director, Health Care: [End of section] Appendix I: Scope and Methodology: Characteristics of Veterans Diagnosed with Major Depressive Disorder (MDD): To describe the characteristics of veterans diagnosed with MDD from fiscal years 2009 through 2013, we analyzed Department of Veterans Affairs (VA) and Department of Defense (DOD) data. (See table 3.) These data included information on veterans' demographic characteristics as well as clinical information on health care services and medications provided through VA.[Footnote 60] Veterans were classified as having a diagnosis of MDD if, in at least one fiscal year included in our review, they had two or more outpatient encounters or at least one inpatient hospital stay with a diagnosis of MDD.[Footnote 61] Table 3: Description of Department of Veterans Affairs and Department of Defense Data Files Used to Examine Characteristics of Veterans with Major Depressive Disorder for Fiscal Years 2009 through 2013: Data files: Demographic files; Description: Includes information extracted from the Department of Veterans Affairs (VA) Corporate Data Warehouse on veterans' age; date of death, if applicable; sex; race; and ethnicity. The Corporate Data Warehouse is a national data repository comprising data from several VA clinical and administrative systems. Information on whether veterans served during recent conflicts in Iraq and Afghanistan was obtained from a Department of Defense roster. Data files: Medical SAS Inpatient Datasets, Acute Care Dataset; Description: Includes information on acute care inpatient stays at VA medical centers (VAMC) that last at least 24 hours, including admission and discharge date as well as diagnosis codes. This dataset comprises information on inpatient care that is entered into veterans' electronic medical records by staff at VAMCs. Data files: Fee basis inpatient files[A]; Description: Includes information on certain acute care inpatient stays at non-VAMCs, including admission and discharge dates as well as diagnostic codes. VA pays for these stays at non-VAMCs when VA cannot offer needed care, when a non-VA provider would be economical, or on an emergency basis when travel to a VA facility is medically infeasible. The fee basis inpatient files for each fiscal year are limited to information on inpatient stays at non-VAMCs that were paid for by VA during the corresponding fiscal year. Data files: Outpatient encounter files; Description: Includes information on outpatient services rendered by VA providers, including the date of the services and diagnoses treated. This file comprises information on individual outpatient encounters entered into veterans' electronic medical records by staff at VA medical facilities. Encounters are defined as a professional contact between a patient and a VA provider with responsibility for diagnosing, evaluating, and treating veterans' conditions. Data files: Fee basis outpatient services files[A]; Description: Includes information on certain outpatient care rendered by non-VA providers, including the date of the visit and the reason for the visit. VA pays for these services from non-VA providers when VA cannot offer needed care, when a non-VA provider would be economical, or on an emergency basis when travel to a VA facility is medically infeasible. The fee basis outpatient services files for each fiscal year are limited to information on outpatient encounters that were paid for by VA during the corresponding fiscal year. Data files: Pharmacy Benefits Management prescription database; Description: Includes information on medications ordered by VA providers, including medication name and dosage instructions. The pharmacy database comprises information on medication orders, which are entered into local pharmacy databases by staff at VA medical facilities. Source: GAO. GAO-15-55. [A] The two data sets have retained the name fee basis though VA now refers to this type of care as non-VA medical care. [End of table] Specifically, we examined the following: - Number of veterans diagnosed with MDD. We used a demographic file provided by VA to determine the number of veterans diagnosed with MDD. - Characteristics of veterans diagnosed with MDD. We used demographic files provided by VA and DOD to describe characteristics of veterans diagnosed with MDD. In particular, the veteran characteristics we examined included the following: - Age. We created seven categories for veterans' ages as of September 30, 2013--the end of fiscal year 2013, which corresponds to the last date of data we included in our analysis. These categories are as follows: (a) 18-24, (b) 25-34, (c) 35-44, (d) 45-54, (e) 55-64, (f) 65- 74, and (g) 75 and older. - Era of service. We categorized veterans as either being veterans of the recent conflicts in Iraq and Afghanistan--Operation Iraqi Freedom, Operation Enduring Freedom, and Operation New Dawn--or of other eras of service.[Footnote 62] - Sex. We categorized veterans as either being female or male. - Race and ethnicity. We created categories to describe veterans' race and ethnicity (Hispanic and non-Hispanic). These categories are consistent with the Office of Management and Budget's 1997 Standards for Maintaining, Collecting, and Presenting Federal Data on Race and Ethnicity.[Footnote 63] * Extent to which veterans diagnosed with MDD were prescribed at least one antidepressant. Using data from the Pharmacy Benefits Management database, we examined the extent to which VA providers prescribed at least one antidepressant for veterans diagnosed with MDD from fiscal years 2009 through 2013. This includes antidepressants prescribed to treat depression as well as those prescribed to treat other conditions.[Footnote 64] * The percentage of veterans with MDD dispensed a 12-week and a 6- month supply of an antidepressant. Using VA data we obtained from the Medical SAS Inpatient Datasets, Acute Care Dataset; Outpatient Encounter Files; Fee Basis Outpatient and Inpatient Services Files; and Pharmacy Benefits Management Database, we calculated the percentage of veterans with MDD dispensed a 12-week and a 6-month supply of an antidepressant according to statistical programming logic provided by VA.[Footnote 65] These measures are intended to assess the effectiveness of antidepressant medication management and are based on performance measures developed by the National Committee for Quality Assurance.[Footnote 66] In addition, these measures are consistent with the VA/DOD Clinical Practice Guideline for Management of Major Depressive Disorder, which indicates that continued antidepressant treatment, after acute depressive symptoms have resolved, decreases the incidence of relapse of MDD.[Footnote 67] We selected six VA medical centers (VAMC) at the following locations to visit: Canandaigua, New York; Gainesville, Florida; Iowa City, Iowa; Philadelphia, Pennsylvania; Phoenix, Arizona; and Reno, Nevada.[Footnote 68] These VAMCs represent different facility complexity groups, serve populations of veterans that differ in terms of the extent of use of mental health services, and are located in different Veterans Integrated Service Networks (VISN), or regional networks of care.[Footnote 69] To gather additional perspectives, for each VAMC we visited, we selected one associated community-based outpatient clinic to visit. In particular, we visited community-based outpatient clinics in the following locations: Cedar Rapids, Iowa; Globe, Arizona; Gloucester, New Jersey; Lecanto, Florida; Fallon, Nevada; and Rochester, New York. (See table 4.) As part of our site visits, we reviewed a nongeneralizeable sample of five medical records for each of these six VAMCs for a total of 30 veterans.[Footnote 70] We reviewed these medical records to determine if the diagnostic code entered for all encounters--starting with the initial encounter in 2012 when the veteran was diagnosed with MDD and prescribed an antidepressant--was consistent with a diagnosis of MDD. Table 4: Characteristics of Veterans Affairs Medical Centers (VAMC) and Community-Based Outpatient Clinics Selected for GAO's Review: VAMC location: Canandaigua, New York; Facility complexity group, 2011: 3; Percentage of veterans that used mental health services, 2012 (above or below average): 24.6% (above); Veterans Integrated Service Network (VISN): 2; Community-based outpatient clinic location: Rochester, New York. VAMC location: Gainesville, Florida[A]; Facility complexity group, 2011: 1a; Percentage of veterans that used mental health services, 2012 (above or below average): 22.2% (below); Veterans Integrated Service Network (VISN): 8; Community-based outpatient clinic location: Lecanto, Florida[A]. VAMC location: Iowa City, Iowa; Facility complexity group, 2011: 1c; Percentage of veterans that used mental health services, 2012 (above or below average): 16.7% (below); Veterans Integrated Service Network (VISN): 23; Community-based outpatient clinic location: Cedar Rapids, Iowa. VAMC location: Philadelphia, Pennsylvania; Facility complexity group, 2011: 1b; Percentage of veterans that used mental health services, 2012 (above or below average): 28.3% (above); Veterans Integrated Service Network (VISN): 4; Community-based outpatient clinic location: Gloucester, New Jersey. VAMC location: Phoenix, Arizona; Facility complexity group, 2011: 1c; Percentage of veterans that used mental health services, 2012 (above or below average): 24.3% (above); Veterans Integrated Service Network (VISN): 18; Community-based outpatient clinic location: Globe, Arizona. VAMC location: Reno, Nevada[A]; Facility complexity group, 2011: 2; Percentage of veterans that used mental health services, 2012 (above or below average): 22.4% (below); Veterans Integrated Service Network (VISN): 21; Community-based outpatient clinic location: Fallon, Nevada[A]. Source: GAO analysis of VA data. GAO-15-55. Note: The Department of Veterans Affairs (VA) assigns each VAMC a complexity score between 1 and 3, with level 1 being the most complex, using a facility complexity model. Level 1 is broken down further into 1a, 1b, and 1c. That model uses multiple variables to measure facility complexity arrayed along four categories, namely patient population served, clinical services offered, education and research complexity, and administrative complexity. Each VAMC is assigned to a single regional network of care, called a VISN. [A] In contrast to the other site visits, which were conducted in- person, we conducted a virtual site visit via telephone interviews to these locations. [End of table] To select medical records for review, we completed the following steps: * Randomly generated a list of individuals with a new prescription for an antidepressant in calendar year 2012. * Selected the first five individuals in the list that met the following inclusion criteria: - Veteran status. - Had a diagnosis of MDD in calendar year 2012. For the purposes of medical record reviews, we classified a veteran as having a diagnosis of MDD if, based on how the veteran's patient care encounters were coded or on the narrative contained in clinical notes in the veteran's medical record, the veteran had (a) at least two outpatient encounters with a diagnosis of MDD, or (b) at least one inpatient stay with a diagnosis of MDD.[Footnote 71] - Had a new treatment episode of antidepressants in calendar year 2012. New treatment episodes were defined as an initiation of antidepressant treatment following a period during which the veteran was either (1) not prescribed an antidepressant or (2) noncompliant with and had not picked up prescriptions for a previously prescribed antidepressant. To ensure the reliability of the data we analyzed, we interviewed VA Central Office officials, reviewed relevant documentation and veterans' medical records, and conducted electronic testing to identify missing data and obvious errors. On the basis of these activities, we determined that the data we analyzed were sufficiently reliable for our purposes. However, as discussed in the report, we described limitations of the data due to the coding discrepancies we found. VA's Oversight of the Extent to Which Veterans with MDD Prescribed Antidepressants Are Receiving Care As Recommended in the CPG: To examine the extent to which VAMCs are providing care to veterans with MDD who are prescribed antidepressants as recommended in the CPG, we reviewed relevant VA policy documents. On the basis of that review, we found that VA policy requires all care sites, VAMCs, and community- based outpatient clinics to provide evidence-based antidepressant treatment when indicated for depression and that such care must be consistent with current VA clinical practice guidelines.[Footnote 72] The relevant VA clinical practice guideline, the VA/DOD Clinical Practice Guideline for Management of Major Depressive Disorder, provides evidence-based recommendations for providers on how to monitor veterans prescribed antidepressants; these recommendations are based on a review of depression research outcomes. These recommendations are based on available research at the time of publication of the guideline and are intended to provide information to assist providers in treatment decision-making.[Footnote 73] From the guideline's recommendations related to monitoring veterans prescribed antidepressants, we judgmentally selected three recommendations for inclusion in our review. In particular, we selected recommendations that (1) had among the highest strength of research evidence, (2) were sufficiently specific to enable us to determine the extent to which VA providers were following the recommendation, and (3) would not require clinical judgment to determine the extent to which VA providers were following the recommendation. The following recommendations were included in our review: * To enhance antidepressant treatment, veterans should be educated on when to take the medication, possible side effects, risks, and the expected duration of treatment, among other things; * Standardized assessments of depressive symptoms, such as the Patient Health Questionnaire-9, should be used to monitor treatment response at 4-6 weeks after initiation of treatment, after each change in treatment, and periodically thereafter until full remission is achieved; and: * A plan should be developed that addresses the duration of antidepressant treatment, among other things. After selecting these recommendations for our review, we examined the extent to which veterans were receiving care consistent with these CPG recommendations at the six VAMCs we visited.[Footnote 74] To do this, we interviewed VAMC clinicians to determine whether and how they were following these recommendations. At each VAMC, officials interviewed included members of the executive leadership team, primary care and mental health providers, and pharmacists.[Footnote 75] Additionally, as part of our examination of the extent to which VAMCs are providing care consistent with the selected guideline recommendations, we reviewed the sample of five veterans' medical records per VAMC used as part of our review of MDD coding. For each medical record, we reviewed documentation contained in the selected veterans' medical records to assess the extent to which the antidepressant treatment-related care VA providers rendered was consistent with the selected CPG recommendations included in our review. Our review commenced with the encounter during which a VA clinician ordered an antidepressant to treat depressive symptoms. Our review ended after five follow-up encounters with a VA clinician, or sooner if the veteran did not have five follow-up encounters. Our review was limited to encounters during which the antidepressant treatment was reviewed, including encounters during which side effects and treatment effect were assessed, but no change was made to medication orders. We did not include, for example, an encounter with an orthopedist during which the fact that the veteran had been prescribed an antidepressant was simply noted. We provided the VAMCs with the instances where we found the medical record documentation was not consistent with the selected CPG recommendations. The VAMCs confirmed our answers or provided additional support if they believed the care was consistent with the CPG. To examine VA's oversight of the care VAMCs provide to veterans with MDD who are prescribed an antidepressant, we reviewed VA's oversight of the Uniform Mental Health Services in VA Medical Centers and Clinics handbook and CPG requirements and evaluated whether this oversight provides VA with adequate information to identify nonconformance with recommended practices, assess the risk of any nonconformance, and address nonconformance, as appropriate.[Footnote 76] As part of this review, we reviewed VA's oversight in the context of federal standards for internal control for risk assessment. [Footnote 77] The internal control for risk assessment refers to an agency's ability to comprehensively identify risks, assess the possible effect, if any, and determine what actions should be taken to mitigate significant risks. We then interviewed officials from VA Central Office, including officials from the Office of Mental Health Services (OMHS), the Office of Mental Health Operations, and the Office of Analytics and Business Intelligence; and the six VISNs that oversee the VAMCs we visited who are responsible for overseeing compliance with VA's requirements, including VA's requirement that all VA facilities provide evidence- based antidepressant treatment when indicated for depression and that such care be consistent with current VA clinical practice guidelines. Through our interviews, we obtained information on the oversight activities conducted by VA Central Office and the extent to which VA Central Office followed up with VAMCs to ensure that they corrected problems identified through these oversight activities. In addition, we obtained and reviewed relevant documents regarding VA oversight, including internal reports and VAMCs' plans to correct problems identified through oversight activities. Information VA Requires VAMCs to Collect on Veteran Suicides: To analyze the information VA requires VAMCs to collect on veteran suicides, we first reviewed VA policies, guidance, and documents related to VA's suicide prevention efforts to identify the mechanisms by which VA collects veteran suicide data from VAMCs. We also interviewed VA Central Office and other officials responsible for VA's suicide prevention program, including officials from OMHS and the Center of Excellence for Suicide Prevention. We also interviewed VAMC officials and relevant staff of the six VISNs for the sites we visited to obtain information on suicide prevention initiatives. Next, through the site visits to six VAMCs, we obtained documents and interviewed officials regarding the collection of veteran suicide data. We obtained all completed templates from the Behavioral Health Autopsy Program (BHAP) related to VA's collection of data on veterans that died by suicide as of the time of our site visit or at the time we requested the documents for virtual site visits.[Footnote 78] One VAMC had not completed any of these BHAP documents because they had not had a veteran die by suicide since the beginning of the program. Therefore, our analysis includes a review of documents from five of the six VAMCs we visited. Through review of the documents, we noted any fields missing data, such as a field that requires a yes or no answer but neither answer is provided. Additionally, using professional judgment, we identified fields in the documents to review based on whether the field related to aspects of VA treatment-- including treatment for mental health conditions--and the date of the veteran's death.[Footnote 79] We identified these fields because they did not require clinical judgment to assess. Using the parameters in the corresponding guide for filling out these documents, including time frames, we compared these fields to information included in the veteran's medical record and noted differences between our answers and the answers provided by the VAMCs in the documents. To ensure that we received the final, submitted versions, we also requested these documents from VA Central Office for each of the five VAMCs. We compared these documents to the documents we received from the VAMCs. We used the documents from the VAMCs as the starting point; therefore, we only analyzed the templates for veterans identified by the VAMCs.[Footnote 80] During the course of our review we learned that the template for these documents had changed over time. If additional fields were included in the templates obtained from VA Central Office, but were not originally included in the templates obtained from the VAMCs, we did not review these fields. We generally used the answer from the document obtained from VA Central Office, which is the final submitted version, unless a field originally had an answer in the template from the VAMC, but was blank or not answered in the template from VA Central Office. In those cases, we used the answer from the VAMC document. We provided the VAMCs with the fields where the answers in the VAMC's documents did not match our answers based on our review of the medical record. The VAMCs confirmed our answers or provided additional support for their original answer. Results from our review of veteran suicide data can be generalized to the VAMCs we visited, but cannot be generalized to other VAMCs. We conducted this performance audit from November 2013 to November 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: Department of Veterans Affairs Medical Centers' Tracking of Veterans at High Risk for Suicide: The Department of Veterans Affairs' (VA) Uniform Mental Health Services in VA Medical Centers and Clinics handbook (Handbook) requires VA medical centers (VAMC) to have a suicide prevention coordinator whose responsibilities include establishing and maintaining a list of veterans assessed to be at high risk for suicide and monitoring these veterans.[Footnote 81] The Handbook also requires suicide prevention coordinators to ensure that providers follow up on missed appointments for high-risk veterans to ensure patient safety and in order to initiate problem-solving about any tensions or difficulties in the veteran's ongoing care. Whether a veteran is determined to be at high risk for suicide is based on clinical judgment made after an evaluation of risk factors-- such as history of past suicide attempts or recent discharge from an inpatient mental health unit--protective factors--such as positive social support, positive coping skills, and positive therapeutic relationships--and the presence or absence of warning signs. Indicators that a veteran is at high risk for suicide include a current verified report or witnessed suicide attempt; identification of current serious suicidal ideation that requires an immediate change in the treatment plan, such as hospitalization; and the presence of any of the following warning signs: threatening to hurt or plan to kill oneself; looking for specific ways to kill oneself and seeking access to such means, such as pills or weapons; and talking or writing about death, dying, or suicide when these actions are out of the ordinary for the person. The Handbook requires each VAMC to have a process for establishing a patient record flag to help ensure that veterans determined to be at high risk for suicide are provided with follow up for all missed mental health and substance abuse appointments. The primary purpose of the patient record flag is to communicate to staff that a veteran is at high risk for suicide and VA policy states that the presence of a flag should be considered when making treatment decisions. Suicide prevention coordinators are responsible for assessing, in conjunction with the treating clinician, the risk of suicide in individual veterans, ensuring these veterans have a "High Risk for Suicide" Patient Record Flag on their medical record, and reviewing the list of high-risk veterans at least every 90 days. We interviewed suicide prevention coordinators as part of our site visits with six VAMCs to obtain information on how they track veterans determined to be at high risk for suicide. At four of the VAMCs we visited, suicide prevention coordinators used an electronic spreadsheet to track information on these veterans. For example, the spreadsheets include information such as whether the veteran has a patient record flag on their medical record and when the flag needs to be reviewed, the date for the veteran's next scheduled follow-up appointment, whether the veteran has a safety plan, and the veteran's assigned psychiatrist.[Footnote 82] Officials from one VAMC told us that they maintain the list daily, adding and removing veterans as necessary. Officials stated that the circumstances under which a veteran would be removed from the spreadsheet varied, but veterans are generally removed because their patient record flag has been removed and the officials no longer consider the veteran to be at high risk for suicide. The two remaining VAMCs use other mechanisms to track veterans at high risk for suicide. Officials from one VAMC told us that they use the Suicide Prevention Application Network (SPAN) to query high-risk patients at the VAMC. The SPAN database contains veteran information, demographic characteristics, and information on suicide attempts and completed suicides, among other things. According to officials, the information for each veteran in SPAN includes the date the veteran was assessed as being at high risk, as well as the date that the veteran needs to be seen for follow up, if applicable. After our site visit, officials told us they plan to periodically pull a list of all veterans with an active high risk flag in VA's medical record for the VAMC and cross-reference that list to veterans being tracked for high suicide risk by the suicide prevention coordinator in SPAN to ensure all high-risk veterans are tracked. Officials from the other VAMC told us that their case managers each have their own list of veterans that they track and the suicide prevention coordinator we spoke with stated that he does not keep a master list of all veterans that are at high risk for suicide.[Footnote 83] [End of section] Appendix III: Department of Veterans Affairs' Use of Data Related to Suicides and Suicide Behavior: Veterans Affairs medical centers (VAMC) collect and submit data on veteran suicides to the Department of Veterans Affairs (VA) Central Office through the Suicide Prevention Application Network (SPAN) and suicide behavior reports.[Footnote 84] VAMCs also collect and report data through root cause analyses.[Footnote 85] Additionally, VA Central Office uses the data from SPAN to prepare reports that are sent to the VAMCs and Veterans Integrated Service Networks (VISN).[Footnote 86] VA Central Office officials stated that they expect VAMCs and VISNs to use these reports and collected data to improve suicide prevention efforts and program evaluation. Through site visits at six VAMCs that we conducted as part of our review and through interviews with corresponding VISN officials, we identified examples of how some VAMCs and VISNs are utilizing veteran suicide data to improve their suicide prevention efforts. * VAMC and VISN officials have used SPAN to create initiatives based on trends in the data. For example, - Officials at one VAMC stated that they use the information collected in SPAN to provide data for performing statistical analyses on the outreach conducted, to study suicide attempts and completions across the VAMC catchment area, to understand the means by which veterans are dying by suicide, and to study the use of high-risk flags. - Officials at a VISN explained that through a review of the SPAN data about a year ago, officials learned that 60 percent of suicides in the VISN were completed using a gun. After conducting research on the subject, the VISN began a firearm safety initiative, which includes notifying veterans by mail that they can receive four gun locks each upon request, with no questions asked. * VAMC officials have made programmatic changes to their suicide prevention efforts based on trends in the suicide data they are collecting and reviewing. For example, - At one VAMC, officials told us that they reviewed suicide behavior reports and, as a result of trends identified in these reports, drafted a policy for medication restriction for veterans at risk of overdosing. Specifically, over a 3-year period, five or six veterans receiving VA care repeatedly attempted suicide by overdose, typically when they were intoxicated. VAMC officials created a work group to draft policy that mitigates risk for medication overdose among high- risk veterans. At the time of our site visit, the group was exploring creating a patient record flag that would be included in the veteran's medical record for overdose risk indicating that medication supplies should be restricted for these veterans and the possibility of using automated pill dispensers to dispense medications to these veterans. - Through their work reviewing suicide-related information, the suicide prevention team at another VAMC identified a trend in its suicide data. In particular, they noted that some veterans were given a 90-day supply of the same medications that the veteran recently tried to use to overdose. The suicide prevention team mentioned this to a clinical pharmacist who had also noticed this issue. The VAMC is now trying to restrict days of supply for these types of veterans, but there is no formal policy about this and no plans to craft such a policy. Additionally, officials from this VAMC stated that they have added items to the standardized suicide behavior report template to help them to collect additional useful information, such as active medications and pain score at the time of the last visit. - Officials from one VAMC stated that through a review of medical records and autopsy reports for veterans who died by suicide, they found that a vast majority of veterans who died by suicide were not being seen by a mental health provider. In response, officials provided education to primary care providers. VAMC officials also noticed that veterans receiving care for pain were dying by suicide at a high rate. As a result, the VAMC has started an initiative with the pain clinic, and, as part of this initiative, the chief of the pain management clinic consults with psychiatry on veterans at risk for suicide. * Officials at a VISN described changes made in response to the suicide data in fiscal year 2012, which showed that a percentage of veterans who completed suicide had no ongoing mental health care. These veterans mainly received care from VA primary care providers. To address this, the VISN partnered with the Center of Excellence for Suicide Prevention and local university psychologists to help VA primary care providers at community-based outpatient clinics formulate mental health plans. [End of section] Appendix IV: Comments from the Department of Veterans Affairs: Department of Veterans Affairs: Washington, DC 20420: October 21, 2014: Mr. Randall B. Williamson: Director, Health Care: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Williamson: The Department of Veterans Affairs (VA) has reviewed the Government Accountability Office's (GAO) draft report, "VA HEALTH CARE: Improvements Needed in Monitoring Antidepressant Use for Major Depressive Disorder and in Increasing Accuracy of Suicide Data" (GAO- 15-55). VA generally agrees with GAO's conclusions and concurs with GAO's recommendations to the Department. The enclosure specifically addresses GAO's recommendations and provides an action plan for each. VA appreciates the opportunity to comment on your draft report. Sincerely, Signed by: Jose D. Riojas: Chief of Staff: Enclosure: Department of Veterans Affairs (VA) Response to Government Accountability Office (GAO) Draft Report "VA Health Care: Improvements Needed in Monitoring Antidepressant Use for Major Depressive Disorder and in Accuracy of Suicide Data" (GAO-15-55): GAO Recommendation: The Secretary of Veterans Affairs direct the Under Secretary of Health to take the following six actions: To more accurately estimate the prevalence of MDD and identify enrolled veterans with MDD, VA should: Recommendation 1: identify the extent to which there is imprecise diagnostic coding of MDD by further examining encounters with a diagnostic code of depression not otherwise specified, which could be incorporated into VAMCs' ongoing review of diagnostic coding accuracy. VA Comment: Concur. The Department of Veterans Affairs (VA) agrees that precise and reliable diagnoses are an important guide to treatment planning. The Veterans Health Administration (VHA) will examine patterns of diagnostic coding among VHA patients with new episodes of depression treatment by evaluating diagnosis patterns and treatment settings. This will include examination of the available data regarding structured assessments for depression Population Health Questionnaire (PHQ) scores in association with diagnostic specificity and treatment settings. For example, VA may expect that initial assessments performed in primary care settings are coded with less specificity, relative to follow-up assessments conducted in mental health clinic settings. Target Completion Date: January 2015. Recommendation 2: determine and address the factor(s) contributing to the imprecise coding based on the results of those examinations. For example, feedback and additional training could be provided to clinicians regarding the importance of diagnostic code accuracy, or VA's medical record could be enhanced to facilitate the selection of a more accurate diagnostic code. VA Comment: Concur. In addition to the analyses described above in Recommendation 1, VA will conduct focused chart reviews to evaluate whether further information regarding the care processes and clinical communications may help to explain changes in diagnostic specificity among individuals who at some point receive Major Depressive Disorder (MDD) diagnoses. VHA will consult with a sample of treatment providers in primary care and in mental health clinic settings to analyze VHA's understanding of diagnostic code selection and to develop steps to enhance diagnostic coding practices. In part, future actions are dependent on the findings associated with Recommendation 1. However, identification and correction of factors contributing to imprecise coding was initiated based upon the findings of the GAO review of 30 records. Informatics staff initiated a national review of the encounter forms for Mental Health appointments and identified the software mapping error in the diagnosis search feature that was referenced in the report. Steps have been taken to correct this mapping error, and VHA expects that this fix will be deployed to the field in November 2014. Additionally, in September 2014 both Mental Health and Health Information Management Service staff in the field were notified via email distribution and conference calls regarding this software error. If additional coding reviews reveal further coding inaccuracies, a plan will be developed to determine and address the factors contributing to coding variance from the diagnosis supported in the documentation of the progress note. Target Completion Date: March 2015. To ensure that veterans are receiving care in accordance with the MOD CPG, VA should: Recommendation 3: implement processes to review data on veterans with MDD prescribed antidepressants to evaluate the level of risk of any deviations from recommended care and remedy those that could impede veterans recovery. VA Comment: Concur. Using the available measures of antidepressant treatment practices (e.g., the Healthcare Effectiveness Data and Information Set) measure of adequate antidepressant continuation following a new antidepressant start, VHA will examine associations between treatment practices and indicators of Veteran recovery and/or adverse outcomes. This will include assessing changes in PHQ scores, as available, as well as associations with potential adverse outcomes such as inpatient psychiatric admissions. VA notes that such analyses are complicated by potential confounds (e.g., greater treatment receipt among patients with greater severity, with severity also associated with greater likelihood of negative outcomes). Future actions will be informed by the aforementioned analysis. Target Completion Date: March 2015. To improve VA's efforts to inform its suicide prevention activities, VA should: Recommendation 4: ensure that VAMCs have a process in place to review data on veteran suicides for completeness, accuracy, and consistency before the data are submitted to VA Central Office. VA Comment: Concur. This is data that frequently changes as more data become available. The information is all submitted to the Center of Excellence for Suicide Prevention (CoE) as it becomes available and there is the ability to change the data once it is submitted. Delaying the submission until it is complete would result in several non- entries. Therefore, sites are encouraged to submit what they have when it is available. In the future, the CoE will ask for verification of review by facility leadership of all submissions during the previous 6 months prior to analyzing or rolling up the data. The CoE will create an initial box on the chart review form that will acknowledge this has been completed at the facility level. Target Completion Date: June 2015. Recommendation 5: clarify guidance on how to complete BHAP templates to ensure that VAMCs are submitting consistent data on veteran suicides. VA Comment: Concur. New guidance has been incorporated into the Suicide Prevention Coordinator (SPC) Guidebook and it was redistributed to the field. In addition, a Frequently Asked Question sheet was developed and sent to the field in September 2014 (Attachment A), and CoE staff will be present on monthly SPC calls to respond to any questions and needs. Recommendation 6: implement processes to review data on veteran suicides submitted by VAMCs for accuracy and completeness. VA Comment: Concur. A software program is being written to compare Chart Reviews from the Behavioral Health Autopsy Program to the Suicide Prevention Access Network data on a monthly basis. If there are differences in who submitted them, the information will be pulled and returned to the SPC for clarification. In addition, all submissions will be reviewed for completeness 3 months after submission and returned for clarification, if needed, prior to any analysis or roll up of the data. This will be completed prior to facility leadership review. Target Completion Date: December 2015. [End of section] Appendix V: GAO Contact and Staff Acknowledgments: GAO Contact: Randall B. Williamson, (202) 512-7114, williamsonr@gao.gov. Staff Acknowledgments: In addition to the contact named above, Marcia A. Mann, Assistant Director; Emily Binek; Muriel Brown; Stella Chiang; Cathleen Hamann; Melanie Krause; Daniel Lee; Lisa Opdycke; Sarah Resavy; and Jennifer Whitworth made key contributions to this report. [End of section] Footnotes: [1] For purposes of this report, references to VAMCs include the associated community-based outpatient clinics. [2] VA/Department of Defense (DOD) Assessment and Management of Risk for Suicide Working Group, VA/DOD Clinical Practice Guideline for Assessment and Management of Patients at Risk for Suicide (June 2013). [3] Data on whether veterans served during recent conflicts in Iraq and Afghanistan came from DOD. [4] The data include health care services and medications provided by non-VA providers but paid for by VA through its Non-VA Medical Care Program, formerly known as the Fee Basis Program. Non-VA medical care is the practice of paying for veterans' health care services outside of VA when VA medical facilities are not feasibly available. [5] The definition of MDD as defined in the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders was effective for a majority of the time period that our study covers. According to the American Psychiatric Association, the core symptoms and duration of these symptoms to meet the definition of MDD have not changed from the fourth to fifth edition of the manual. American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders, 4TH ed., Text Revision (DSM-IV-TR) (Washington, D.C.: American Psychiatric Association, 2000). [6] Throughout the report, we use the phrase medical records to refer to electronic medical records. [7] In contrast to the other site visits, which were completed in person, we completed the site visits to the VAMCs located in Gainesville, Florida, and Reno, Nevada, virtually through telephone interviews. [8] The MDD CPG is formally known as the VA/DOD Clinical Practice Guideline for Management of Major Depressive Disorder (May, 2009). The MDD CPG was issued by the joint VA/DOD Evidence-Based Practice Work Group in 2009. Formed in 1999 and composed of VA and DOD officials, the VA/DOD Evidence-Based Practice Work Group makes decisions about which CPGs for specific conditions will be developed and oversees their development. VA's Office of Mental Health Services is responsible for developing mental health policy and the Office of Mental Health Operations (OMHO) is responsible for implementing these policies. [9] The three recommendations related to monitoring veterans prescribed antidepressants from the CPG that we judgmentally selected were (1) to enhance antidepressant treatment, veterans should be educated on when to take the medication, possible side effects, risks, and the expected duration of treatment; (2) standardized assessments of depressive symptoms should be used to monitor treatment response at 4-6 weeks after initiation of treatment, after each change in treatment, and periodically thereafter until full remission is achieved; and (3) a plan should be developed that addresses the duration of antidepressant treatment. [10] The Behavioral Health Autopsy Program (BHAP) Post-Mortem Chart Analysis Template (BHAP template) is a mechanism by which VA Central Office collects veteran suicide data from VAMCs' review of veterans' medical records. One BHAP template had been completed as of the time of our site visit, but had not been received by VA Central Office; therefore, we did not include this template in our review. [11] Each VAMC is assigned to a single VISN. [12] These symptoms include significant weight loss; insomnia or excessive sleeping; psychomotor agitation or retardation; fatigue or loss of energy; feelings of worthlessness or excessive or inappropriate guilt; diminished ability to think or concentrate, or indecisiveness; and recurrent thoughts of death, recurrent suicidal ideation without a specific plan, or a suicide attempt or a specific plan for committing suicide. American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders, 5TH ed. (Arlington, VA: American Psychiatric Association, 2013). [13] Veterans Health Administration Handbook 1160.01, Uniform Mental Health Services in VA Medical Centers and Clinics (Sept. 11, 2008). The Handbook also requires that VAMCs provide evidence-based psychotherapy, such as Cognitive Behavioral Therapy, to veterans when appropriate. However, these services fall outside the scope of our review. [14] The CPG also states that it is intended to "identify outcome measures to support the development of practice-based evidence that can ultimately be used to improve the clinical guidelines." [15] The CPG is based on a review of research outcomes available at the time of publication. Evidence-based care refers to approaches that have consistently been shown in controlled research to be effective for a particular condition or conditions. [16] The CPG for MDD is organized into 22 main topics, such as "initial treatment" and "treatment response." Each main topic contains a number of action statements that relate to approximately 200 recommendations for the management of veterans with MDD. [17] The PHQ-9 is a diagnostic tool, which uses the nine MDD diagnosis symptoms as criteria to help clinicians make a criteria-based diagnosis of depressive disorders and measure depression severity to aid treatment decisions. [18] The Veterans Crisis line can be reached by calling 1-800-273-8255 and pressing 1, online at [hyperlink, http://www.VeteransCrisisLine.net], or sending a text message to 838255. [19] Veteran suicide data is submitted by VAMCs to VA's Center of Excellence for Suicide Prevention located in VISN 2. The Center of Excellence was created by VA Central Office, and for the purposes of our report, we refer to the Center of Excellence as part of VA Central Office. [20] As part of this initiative, VA collects veterans' demographic data, such as date of birth, date of death, race, ethnicity, and gender. VA also collects clinical data, such as the date of the veteran's final VA visit, whether the final visit was outpatient or inpatient, and the results of mental health screens, including depression screens. [21] Prior to BHAPs, VA Central Office required suicide prevention coordinators to complete a review for suicide completions and attempts that identified, for example, the contributing factors for the event. Suicide prevention coordinators would then complete an aggregate analysis of all reported veteran suicides. [22] VAMCs can also submit reviews for veterans who did not utilize VA health care services. Veterans not being seen within the VA will not have clinical information available in VA's medical records, but VAMCs can report information that is known through other mechanisms, such as a coroner's report or the veteran's family members, if available. According to VA, a comprehensive suicide prevention program requires timely and accurate information beyond that acquired from veterans being seen in VA. Data on these veterans are needed to, among other things, improve understanding of suicide among all veterans. [23] Suicide prevention coordinators are responsible for asking family members if they are interested in participating in an interview. According to VA Central Office officials, a social worker at VA's Center of Excellence for Suicide Prevention conducts the family interview and multiple participants may be included in the interview. [24] VAMC officials may send additional information to VISN and VA officials through issue briefs, which document specific factual information regarding unusual incidents, such as deaths, disasters, or anything else that happens at a VAMC, such as a veteran suicide, that might generate media interest or affect care. VAMCs are also required to complete peer reviews for suicides that occur within 30 days of any clinical encounter with a VA clinician. Peer review for quality management is used when there is a need to determine whether a provider's actions associated with an adverse event were clinically appropriate--that is, whether another provider with similar expertise would have taken similar action. [25] This estimate is based on published Congressional Research Service data on the number of veterans who received health care services through VA from fiscal years 2009 through 2013 (roughly 5.5 million). [26] These conflicts include Operation Iraqi Freedom, Operation Enduring Freedom, and Operation New Dawn. [27] An encounter is a professional contact between a patient and a clinician in an outpatient or inpatient setting; a patient visit can consist of multiple encounters. [28] Other depressive disorders include dysthymic disorder, adjustment disorder with depressed mood, and adjustment disorder with mixed anxiety and depressed mood. [29] American Psychiatric Association: Diagnostic and Statistical Manual. Depression not otherwise specified is referred to in this edition as depressive disorder not otherwise specified. [30] Officials at one VAMC in our review said that their facility conducts annual audits of nonbillable encounters, in part, to track the productivity of their providers and to maximize the amount of funding allocated to their facility by VA Central Office based on workload measures. [31] For example, according to a VA Central Office official, if the clinician used the keyword search function and searched under the word "major depression", "major depressive disorder" would be returned as a possible diagnosis choice for selection. Currently, as a result of the software mapping error, if this selection is made, the encounter would be coded as depression not otherwise specified. [32] Uniform Mental Health Services in VA Medical Centers and Clinics handbook. [33] We reviewed five medical records from each of the six VAMCs we selected to determine whether veterans at these VAMCs were receiving care consistent with select recommendations from the CPG. [34] Of these 5 veterans, 2 had treatment plans in the medical record that did not specify a time frame for follow up. One veteran returned in 7 weeks, as prescribed in the veteran's treatment plan. However, the remaining 2 veterans had a time frame in their treatment plan, but no explanation for why the follow-up encounter occurred outside this time frame. The first veteran's treatment plan instructed the veteran to return in 6 weeks; however, the veteran's follow-up encounter occurred approximately 12 weeks after the initial encounter. The second veteran's treatment plan instructed the veteran to return in 6- 8 weeks, but the veteran was not seen until approximately 50 weeks after the initial encounter. [35] As a result of our review, officials at one VAMC told us that they are developing plans to work with clinicians at the VAMC to implement measurement tools, including the PHQ-9. [36] VA's Office of Inspector General (OIG) documented concerns about the timeliness of veterans' access to follow-up mental health care, which could affect the ability of veterans to receive follow-up with a clinician after beginning or changing antidepressant treatment. VA OIG, Veterans Health Administration: Review of Veterans' Access to Mental Health Care (Washington, D.C.: Apr. 23, 2012). [37] In a June 2014 report, the Institute of Medicine also raised concerns with VA clinicians' adherence to CPG recommendations. This report reviewed the VA/DOD Clinical Practice Guideline for Management of Post-Traumatic Stress and stated that VA does not have a mechanism for the systematic collection and analysis of data for assessing the quality of care for post-traumatic stress disorder and that, as a result, VA often does not know whether veterans have received evidence- based treatments or whether these treatments are producing positive outcomes. The Institute of Medicine recommended that VA use evidence- based treatments as the treatment of choice for post-traumatic stress disorder, and these treatments should be delivered with fidelity to their established protocols. Institute of Medicine, Treatment for Posttraumatic Stress Disorder in Military and Veteran Populations Report Brief (Washington, D.C.: June 2014). [38] VA's OMHO developed the psychopharmacology initiative in fiscal year 2014. This initiative is intended to address four areas: (1) possible overprescribing, (2) possible problems in clinical management, (3) misalignment between prescribing and diagnosis, and (4) missed opportunities. The metrics generally assess all veterans with any depression diagnoses, such as depression not otherwise specified; only one metric is specific to MDD. VA Central Office officials explained that data for this initiative are pulled from clinical and administrative records, but that extracting data can be challenging because, for example, clinicians may not be recording data in an extractable field in the medical record. VA Central Office officials also told us that they collect data on the number of veterans who receive 12-week and 6-month supplies of antidepressant medication, which relates to a CPG recommendation stating that, for veterans who achieve remission with antidepressants, treatment should be continued at the same dose for an additional 6-12 months to decrease the risk of relapse. VA/DOD Clinical Practice Guideline for Management of Major Depressive Disorder (May, 2009) pg. 73. [39] Specifically, these CPG recommendations state that veterans who are diagnosed with MDD should receive an initial trial of a single antidepressant, and that a second antidepressant medication may be considered for veterans who have had a partial response to treatment with one antidepressant. VA/DOD Clinical Practice Guideline for Management of Major Depressive Disorder (May, 2009). Other antidepressant metrics include: the proportion of veterans with depression receiving medication from three or more of four psychotropic classes (including antidepressants) for 60 or more continuous days; the proportion of inpatient or outpatient initial visits for new MDD diagnosis with antidepressant receipt where there is no coverage for 84 days out of the first 114 days following the initial antidepressant fill; and the proportion of antidepressant prescriptions given with no medical or psychiatric indication. However, these metrics do not relate to specific recommendations in the CPG. [40] VA also provides VAMCs data on the number of veterans screened annually using a standardized screening tool, a two item Patient Health Questionnaire (PHQ-2) and the percentage of veterans who screen positive for depression and have a timely suicide behavior evaluation completed. These measures meet CPG recommendations for annual screening for MDD and evaluation of suicide risk with a positive depression screen. Another mechanism VA uses to assess mental health care at VAMCs is site visits conducted by VA's OMHO. During these site visits, OMHO assesses, for example, how the number of veterans screened annually for depression compares to the national average as well as any trends within the VAMC. According to OMHO, the purpose of these site visits is to assess mental health care at individual VAMCs, specifically implementation and adherence to VA standards for mental health services. Once OMHO has identified both exemplary areas and areas for growth, OMHO then works with the mental health lead at the corresponding VISN to develop a strategic action plan for the VAMC, which will be monitored for implementation progress on a quarterly basis. However, these site visits do not look at specific CPG recommendations. [41] Officials at one VAMC we visited told us that they use a clinical tool to track veterans being treated for mental health conditions. The mental health tool includes 67,349 unique patients, and an official explained that they can run queries of the clinical tool--for example, for veterans participating in substance abuse treatment who did not return for a drug screen--by pulling both process and outcome variables including diagnostic codes, lab results, and medication lists. [42] VA policy states that VAMCs must have integrated mental health services that include co-located collaborative care and care management. Care management can be based on the BHL, Translating Initiatives for Depression into Effective Solutions (TIDES), or other evidence-based strategies approved by the Office of Mental Health Services. According to VA, TIDES facilitates collaboration between primary care clinicians and mental health specialists with support from a depression care manager, who, under the supervision of a mental health specialist, assists the primary care clinician in the assessment and ongoing management of depressed patients. [43] According to officials at one VAMC we visited, clinicians previously used pharmacy data on a weekly basis to determine whether there were any veterans with new antidepressant orders from primary care who were not referred to the BHL. Officials told us that they stopped running the weekly query, since they consistently found that all eligible veterans had been referred to the BHL. However, in advance of our site visit, officials ran the query again and noted that some veterans were not being referred to the BHL. As a result, they plan to resume completing the weekly queries. [44] We analyzed BHAP templates from five of the VAMCs at the time of our site visit or at the time we requested the BHAP templates. One VAMC had not completed any BHAP templates at the time of our site visit because they did not have a veteran die by suicide since the beginning of the BHAP initiative. Therefore, our analysis does not include BHAP templates from this VAMC. [45] This field in the BHAP template states: "Is this case concerning a Veteran who was enrolled in VA Health Care Services?" For those veterans who were being seen in the VA, the box associated with this field should be checked. [46] VA Central Office officials stated that this field could have become unchecked prior to submission of the template to Central Office, but that it was not the result of a programming error. For example, the box may have become unchecked after the VAMC official saved the template to a local location, but prior to the VAMC submitting the template to VA Central Office. [47] According to one VA Central Office official, for the first interim report, officials matched veterans from the BHAP templates to medical record information to determine whether the veteran was receiving VA health care services. [48] The 15 psychiatric symptoms in the BHAP template are isolation, anxiety, depressed mood, ruminations, suicidal ideation, sleep problems, intrusive memories, drug seeking behaviors, flashbacks, guilt/remorse, hallucinations, command suicide hallucination, alcohol withdrawal, agitation, and impulsivity. [49] For example, the template answers for psychiatric symptoms originally included "yes" and "no." The template has been updated to also include "unknown," but this answer was not an option for all templates in our review. However, officials at two VAMCs provided us with the answers for fields that were left blank, including the fields for psychiatric symptoms. [50] One VA Central Office official told us that for the first BHAP interim report, officials matched the number of suicide behavior reports to the number of BHAP templates using the veterans' social security numbers to determine that a BHAP template was submitted for each completed suicide behavior report. According to VA Central Office officials, this process was not conducted for the most recent BHAP interim report. [51] We could not confirm date of death for two BHAP templates because the veterans' medical records did not contain a date of death. When confirming the lack of date of death in the medical records with the VAMC, officials stated that they took the date of death for one of the templates from the suicide behavior report. However, we did not find a suicide behavior report in the veteran's medical record. [52] Two BHAP templates had dates of death that were 1 day off, one BHAP template was off by 3 days, and one BHAP template had a date of death off by 5 days. [53] Although the BHAP template asks for the number of mental health visits, VA Central Office officials told us that they expect VAMCs to report mental health encounters, rather than visits. An encounter is a professional contact between a patient and a clinician in an outpatient or inpatient setting; a patient visit can consist of multiple encounters. [54] According to VA Central Office officials, missing information in this field is treated as "0" in the interim report. [55] The BHAP Guide can be used as a reference when completing the BHAP template because the BHAP Guide contains specific time frames for some of the fields, several of which are not outlined in the BHAP template. [56] The BHAP Guide states that the final visit could include a mental health or a dental visit, but does not specify if a visit to an immunization clinic should be counted. The BHAP template includes a field for type of visit. One option as an answer for this field is "other," and the type of visit can be included in a text box in the template. [57] VA Central Office has provided policy on the BHAP template through a memorandum and has also provided suicide prevention coordinators with a BHAP Guide, which explains how to fill out the BHAP template. [58] In its comments on a draft of this report, VA stated that in September 2014 it provided guidance to VAMCs to complete BHAP templates for veterans not receiving VA services. [59] See GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999). [60] In general, veterans must enroll in VA health care to receive VA's medical benefits package, a set of services that includes hospital and outpatient services and prescription drugs. Veterans who served in the active military and who were discharged or released under conditions other than dishonorable may be eligible for VA health care. Some non-veterans may also be eligible for VA health care, such as spouses and children of veterans who have been rated permanently and totally disabled for a service-connected disability. We excluded non-veterans from our analysis. [61] We identified this approach through discussions with VA clinical and data experts. This approach is consistent with previous GAO work. See, for example, GAO, VA Mental Health: Number of Veterans Receiving Care, Barriers Faced, and Efforts to Increase Access, [hyperlink, http://www.gao.gov/products/GAO-12-12] (Washington, D.C.: Oct. 14, 2011). [62] We were not able to further describe veterans by other eras of service, such as Vietnam, because according to a VA official, VA data on era of service are unreliable. DOD roster data was used to determine service in Operation Enduring Freedom, Operation Iraqi Freedom, and Operation New Dawn. [63] Office of Management and Budget, Provisional Guidance on the Implementation of the 1997 Standards for Federal Data on Race and Ethnicity (Washington, D.C.: Dec. 15, 2000). [64] Antidepressants, like other types of medications, may be used for off-label uses--that is, for a condition or patient population for which the drug has not been approved or in a manner that is inconsistent with information found on the approved drug label. For example, certain antidepressants may be used to treat anxiety and fibromyalgia. See K.J. Stone, A.J. Viera, and C.L. Parman, "Off-Label Applications for SSRIs" American Family Physician, vol. 68, no. 3 (2003). [65] VA's approach to measuring its performance on these measures uses data regarding care rendered at VA facilities only; however, in order to account for care rendered by non-VA providers but paid for by VA, we incorporated fee basis data. [66] The National Committee for Quality Assurance's 2013 antidepressant medication management quality measures are known as Effective Acute Phase Treatment and Effective Continuation Phase Treatment. [67] Evidence-Based Practice Work Group. VA/DOD Clinical Practice Guideline for Management of Major Depressive Disorder (Washington, D.C.: May 2009). [68] In contrast to the other site visits, which were completed in person, we completed the site visits to the VAMCs located in Gainesville, Florida, and Reno, Nevada, virtually through telephone interviews. [69] VA assigns each VAMC a complexity score between 1 and 3, with level 1 being the most complex, using a facility complexity model. Level 1 is broken down further into 1a, 1b, and 1c. That model uses multiple variables to measure facility complexity arrayed along four categories, namely patient population served, clinical services offered, education and research complexity, and administrative complexity. Each VAMC is assigned to a single VISN. [70] We use the phrase medical records to refer to electronic medical records. [71] This approach for classifying veterans as having MDD is consistent with the approach used for the purposes of our data analysis, which we identified in discussions with VA. [72] Veterans Health Administration Handbook 1160.01, Uniform Mental Health Services in VA Medical Centers and Clinics (Sept. 11, 2008). [73] Although the guideline is not intended to define a standard of care, through policy, VA has set an expectation that providers follow the guideline's recommendations. [74] For the purposes of this report, we refer to VAMCs to include community-based outpatient clinics. [75] We also interviewed primary care and mental health providers at each community-based outpatient clinic. [76] Veterans Health Administration Handbook 1160.01, Uniform Mental Health Services in VA Medical Centers and Clinics (Sept. 11, 2008). [77] See GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999) and GAO, Internal Control Management and Evaluation Tool, [hyperlink, http://www.gao.gov/pro]ducts/GAO-01-1008G (Washington, D.C.: August 2001). [78] One template had been completed as of the time of our site visit, but had not been received by VA Central Office; therefore, we did not include this template in our review. [79] Aspects of treatment paid for by VA, but rendered by non-VA providers (non-VA care) were not included in the scope of this review. [80] We received additional templates from VA Central Office, but these were not analyzed because the VAMC had not provided us with templates for these veterans. [81] Veterans Health Administration Handbook 1160.01, Uniform Mental Health Services in VA Medical Centers and Clinics (Sept. 11, 2008). [82] A safety plan is a prioritized written list of coping strategies and sources of support, such as individuals or agencies the veteran can contact, for the veteran to use during or preceding suicidal crisis in order to help lower the veteran's imminent risk of suicidal behavior. According to VA Central Office officials, the plan is created with clinician, veteran, and family interaction. [83] Officials at this VAMC told us that their case managers are responsible for monitoring veterans flagged as being at high risk for suicide. [84] SPAN includes information on veterans, such as the number of suicide completions, the number of non-fatal suicide attempts, and other indicators of suicide prevention efforts. VAMC clinicians must complete a suicide behavior report when they learn that a veteran exhibited self-harming behavior and include that report in the respective veteran's medical record. [85] Root cause analyses are used to identify the factors that contributed to adverse events or close calls and to identify any steps VAMCs could implement to prevent similar events in the future. Root cause analyses are completed under certain circumstances, such as when the act occurs during a VAMC inpatient stay or within 72 hours of being discharged from inpatient care. [86] VISNs are regional networks of care. Each VAMC is assigned to a single VISN. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]