Skip to main content

Veterans Benefits Management System: Ongoing Development and Implementation Can Be Improved; Goals Are Needed to Promote Increased User Satisfaction

GAO-15-582 Published: Sep 01, 2015. Publicly Released: Sep 15, 2015.
Jump To:
Skip to Highlights

Highlights

What GAO Found

The Veterans Benefits Administration (VBA) within the Department of Veterans Affairs (VA) has made progress in developing and implementing the Veterans Benefits Management System (VBMS), with deployment of the system to all of its regional offices as of June 2013. While 95 percent of records related to veterans' disability claims are electronic and reside in the system, additional capabilities have not yet been completed, such as automation of the steps associated with a veteran's request for an increase in benefits. Further, VBA has not yet developed and implemented pension processing capabilities in VBMS, nor has it articulated when the system will support appeals processing. The VBMS program reported receiving funding of about $1 billion from fiscal years 2009 to 2015, at which time system completion was originally planned. Although development of the system is expected to continue beyond 2015, the incremental approach VA is using to develop and implement VBMS has not yet produced a plan that identifies when the system will be completed and can be expected to fully support disability and pension claims processing and appeals. Thus, it will be difficult for VA to hold its managers accountable for meeting its time frame and for demonstrating progress.

As VA continues its efforts to complete development and implementation of the system, three areas could benefit from increased management attention.

Cost estimating: The program office does not have a reliable estimate of the cost for completing the system. Without such an estimate, VA management and the department's stakeholders have a limited view of the system's future resource needs, and the program risks not having sufficient funding to complete development and implementation of the system.

System availability: Although VBA has improved its performance for ensuring the system is available to users, it has not established system response time goals. Without such goals, users do not have an expectation of the system response times they can anticipate and management does not have an indication of how well the system is performing relative to performance goals.

System defects: While the program has actively managed system defects, a recent system release included unresolved defects that impacted system performance and users' experiences. Continuing to deploy releases with large numbers of defects that reduce system functionality could adversely affect users' ability to process disability claims in an efficient manner.

While VBA has employed various methods to obtain VBMS users' feedback, it has neither established goals to define user satisfaction, nor conducted a survey of claims processing employees to obtain a more comprehensive picture of overall customer satisfaction. GAO's survey of VBMS users estimated that a majority report satisfaction with the system, but that one group of users who are responsible for examining claims decisions was considerably less satisfied. Although the results of GAO's survey provide VBA with useful data about users' satisfaction with the system, the absence of user satisfaction goals limits the utility of survey results. Specifically, without having established goals to define user satisfaction, VBA does not have a basis for gauging the success of its efforts to promote satisfaction with the system.

Why GAO Did This Study

VBA pays disability benefits for disabling conditions incurred or aggravated while in military service, while pension benefits are for low-income veterans who are either elderly or have disabilities unrelated to military service. In fiscal year 2014, the department paid about $58 billion in disability compensation and about $5 billion in pension claims. The disability claims process has been the subject of attention by Congress and others, due in part, to long waits for processing claims and a large backlog of claims. To process disability and pension claims more efficiently, VA began implementation of an electronic, paperless system in 2009. GAO was asked to study VBMS. Specifically, GAO (1) assessed VA's progress toward completing the development and implementation of VBMS and (2) determined to what extent users report satisfaction with the system. To do so, GAO reviewed relevant program documentation, administered a survey to a stratified random sample of about 3,500 users, and interviewed appropriate VA officials.

Recommendations

GAO recommends that VA develop a plan for completing VBMS, establish goals for system response time, minimize the incidence of high and medium priority system defects for future VBMS releases, assess user satisfaction, and establish satisfaction goals to promote improvement. VA concurred with the recommendations and described actions it is planning to take in response, except for the first recommendation. GAO continues to believe development of a plan for completing the system is important.

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Veterans Affairs
Priority Rec.
To improve VA's efforts to effectively complete the development and implementation of VBMS, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits and the Chief Information Officer to develop an updated plan for VBMS that includes (1) a schedule for when VBA intends to complete development and implementation of the system, including capabilities that fully support disability claims, pension claims, and appeals processing and (2) the estimated cost to complete development and implementation of the system.
Closed – Implemented
The Department of Veterans Affairs (VA) concurred with our recommendation and as of July 2020, was continuing to develop requirements for VBMS in order to implement functionality to replace legacy information systems. In addition, the department is processing disability, pension, and appeals claims through VBMS. The department provided an estimate of the cost to complete remaining development and implementation of VBMS, which should allow VA to hold its managers accountable for demonstrating progress needed to replace legacy information systems.
Department of Veterans Affairs To improve VA's efforts to effectively complete the development and implementation of VBMS, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits and the Chief Information Officer to establish goals for system response time and use the goals as a basis for periodically reporting actual system performance.
Closed – Implemented
The Department of Veterans Affairs (VA) concurred with this recommendation and reported that the Veterans Benefits Management System (VBMS) program office has developed draft metrics for performance of the system. Specifically, VA stated that the office has established key performance indicators as a basis for monitoring the response times of the most commonly executed user transactions (or work events) within VBMS. According to the department, these indicators have been incorporated into the application's continuous monitoring tools for all service level agreements and these agreements are enforced by the VA Service Level Management Board. In April 2018, the department provided the goals for system response time and the reported VBMS response times that are tracked on a monthly basis. As a result, VA is better equipped to track the performance of VBMS against established goals for the system's performance.
Department of Veterans Affairs To improve VA's efforts to effectively complete the development and implementation of VBMS, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits and the Chief Information Officer to reduce the incidence of high- and medium-priority level defects that are present at the time of future VBMS releases.
Closed – Implemented
The Department of Veterans Affairs (VA) concurred with this recommendation and reiterated its plans and procedures for decreasing the incidence of defects in each Veterans Benefits Management System (VBMS) release. Weekly defect reports from May 2020 to October 2020 indicated a reduction in the total number of defects present at the time of system releases, including high- and medium-priority level defects. As a result, the department can be better assured that the quality of system releases will not impact users' ability to process claims efficiently.
Department of Veterans Affairs To improve VA's efforts to effectively complete the development and implementation of VBMS, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits and the Chief Information Officer to develop and administer a statistically valid survey of VBMS users to determine the effectiveness of steps taken to make improvements in users' satisfaction.
Closed – Not Implemented
The Department of Veterans Affairs (VA) concurred with this recommendation and in January 2017, conducted a survey of VBMS users that was sent to over 16,000 claims processors at each of its 56 regional offices. Although 52 percent of respondents indicated that they were very satisfied or satisfied with VBMS, the department received only about 2500 responses to the survey for a 15 percent response rate. This low response rate raises concern about whether the survey results were statistically valid. We asked VA about actions the department has taken to ensure the statistical validity of its survey results and, per its response in July 2019, VA did not take any actions to ensure the statistical validity of the survey and does not have any plans to take any further action.
Department of Veterans Affairs To improve VA's efforts to effectively complete the development and implementation of VBMS, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits and the Chief Information Officer to establish goals that define customer satisfaction with VBMS and report on actual performance toward achieving the goals based on the results of GAO's survey of VBMS users and any future surveys VA conducts.
Closed – Not Implemented
The Department of Veterans Affairs (VA) concurred with this recommendation and conducted a survey of VBMS users in January 2017. However, per its response in July 2019, the department had yet to develop customer satisfaction goals for VBMS that would provide users with an expectation of the system response times they should anticipate, and management with an indication of how well the system is performing relative to performance goals. Additionally, the department indicated that it does not plan to conduct another survey or develop customer satisfaction goals for VBMS.

Full Report

GAO Contacts

Valerie C. Melvin
Managing Director
Information Technology and Cybersecurity

Media Inquiries

Sarah Kaczmarek
Managing Director
Office of Public Affairs

Public Inquiries

Topics

AppealsClaimsClaims processingDisabilitiesDisability benefitsInformation resources managementInformation technologyPerformance measuresProgram managementStrategic planningSystems evaluationVeterans benefitsVeterans disability compensationVeterans pensions