Veterans' Benefits:

Increased Focus on Evaluation and Accountability Would Enhance Training and Performance Management for Claims Processors

GAO-08-561: Published: May 27, 2008. Publicly Released: May 27, 2008.

Additional Materials:

Contact:

Daniel Bertoni
(202) 512-5988
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Faced with an increase in disability claims, the Veterans Benefits Administration (VBA) is hiring a large number of new claims processing staff. We were asked to determine: (1) What training is provided to new and experienced claims processors and how uniform is this training? (2) To what extent has VBA planned this training strategically, and how well is the training designed, implemented, and evaluated? and (3) To what extent is the performance management system for claims processors consistent with generally accepted practices? To answer the questions, GAO reviewed documents including VBA policies and training curricula; interviewed VBA central office officials; visited 4 of VBA's 57 regional offices, which were selected to achieve diversity in geographic location, number of staff, and officewide accuracy in claims processing; and compared VBA's training and performance management to generally accepted practices identified by GAO.

VBA has a standardized training curriculum for new staff and a training requirement for all staff, but does not hold staff accountable for meeting this requirement. The curriculum for new staff includes what is referred to as centralized training and training at their home offices. All claims processors must complete 80 hours of training annually, which may cover a mix of topics identified centrally and by regional offices. Individual staff members face no consequences for failing to meet the training requirement, however, and VBA has not tracked training completion by individuals. It is implementing a new system that should provide this capacity. Although VBA has taken steps to plan its training strategically, the agency does not adequately evaluate training and may be falling short in training design and implementation. VBA has a training board that assesses its overall training needs. However, the agency does not consistently collect feedback on regional office training, and both new and experienced staff GAO interviewed raised issues with their training. Some new staff raised concerns about the consistency of training provided by different instructors and about the usefulness of an on-line learning tool. Some experienced staff believe that 80 hours of training annually is not necessary, some training was not relevant for them, and workload pressures impede training. The performance management system for claims processors generally conforms to GAO-identified key practices, but the formula for assigning overall ratings may prevent managers from fully acknowledging and rewarding staff for higher levels of performance. The system aligns individual and organizational performance measures and requires that staff be given feedback throughout the year. However, VBA officials raised concerns about the formula used to assign overall ratings. Almost all staff in the offices GAO visited were placed in only two of five overall rating categories, although managers said greater differentiation would more accurately reflect actual performance differences. The Department of Veterans Affairs (VA) has not examined the ratings distribution, but acknowledges a potential issue with its formula and is considering changes.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: The Department of Veterans Affairs (VA) has now implemented its VA Learning Management System (VALMS) in all Veterans Benefits Administration(VBA) regional offices. Every VBA employee has an account in VALMS and must use the system to record completion of required training. Supervisors in the VBA regional offices use the VALMS to monitor the training completion of their employees. They may log into VALMS and see the training assigned to an individual employee and the employee's progress on completing this training. Also, the VALMS automatically notifies supervisors when employees fail to complete a scheduled training. Using the information from the VALMS, supervisors hold individual employees accountable for meeting their training requirement, and may decide what consequences should be imposed on an employee or what other steps should be taken to ensure an employee completes training.

    Recommendation: The Secretary of Veterans Affairs should direct VBA to use information from its new learning management system to hold individual VSRs and RVSRs accountable for completing whatever annual training requirement it determines is appropriate.

    Agency Affected: Department of Veterans Affairs

  2. Status: Closed - Implemented

    Comments: Since we issued our report, the Veterans Benefits Administration (VBA) has taken steps to implement this recommendation. VBA surveyed 193 claims processors across 23 regional offices, who completed new staff training during 2009, about the Training and Performance Support System (TPSS). VBA asked, for example, about the overall quality of TPSS, whether TPSS modules were an efficient use of employees' time, and whether employees received sufficient support from regional office instructors when using the TPSS.

    Recommendation: The Secretary of Veterans Affairs should direct VBA to collect and review feedback from staff on the training conducted at the regional offices to determine whether regional offices find the Training and Performance Support System (TPSS) a useful learning tool and, if not, what adjustments are needed to make it more useful.

    Agency Affected: Department of Veterans Affairs

  3. Status: Closed - Implemented

    Comments: Since we issued our report, the Veterans Benefits Administration (VBA) has made an effort to more consistently collect feedback on the relevance of regional office training. In July 2009, VBA rolled out a new, on-line training evaluation system. This system provides regional office training managers with a standardized set of questions for collecting feedback from staff on training provided at the regional office. When staff complete the on-line questions, the results are fed into a database and can be viewed by training managers or others in the agency. The questions include, for example, whether the training will help staff do their jobs more effectively.

    Recommendation: The Secretary of Veterans Affairs should direct VBA to collect and review feedback from staff on the training conducted at the regional offices to determine the extent to which regional offices provide training that is relevant to VSRs' and RVSRs' work, given varying levels of staff experience.

    Agency Affected: Department of Veterans Affairs

  4. Status: Closed - Implemented

    Comments: Since the issuance of our report, the Veterans Benefits Administration (VBA) has taken steps to assess the relevance of training provided to claims processors to satisfy their annual training requirement, and has made policy changes intended to ensure the relevance of this training. For example, in 2009 VBA collected the opinions of regional office training managers on the 80-hour training requirement, through an annual meeting and monthly calls. Also, starting in March 2010, VBA has been collecting feedback from claims processors on the usefulness of their training through mandatory Core Technical Training Requirements evaluations. Through November 2010, claims processors had submitted over 25,000 survey responses. As a result of the feedback received from various regional office staff, VBA has determined that 80 hours of training is appropriate but has made some policy changes to enhance the relevance of the training provided to fulfill this requirement. VBA now allows experienced staff to count 20 hours of time spent as an instructor towards their own training requirement, it has developed separate training curricula for intermediate and journey-level claims processors, and it now requires that the majority of training topics each year must be focused on addressing national and local error trends.

    Recommendation: The Secretary of Veterans Affairs should direct VBA to collect and review feedback from staff on the training conducted at the regional offices to determine if the 80-hour annual training requirement is appropriate for all Veterans Service Representatives (VSRs) and Rating Veterans Service Representatives (RVSRs).

    Agency Affected: Department of Veterans Affairs

  5. Status: Closed - Implemented

    Comments: The Veterans Benefits Administration (VBA) has assessed its performance appraisal system for claims processors, determined that adjustments are needed, and is now implementing changes intended to ensure a more meaningful distribution of ratings. On April 1, 2011, VBA implemented a Performance Appraisal Pilot at the Denver and Little Rock Regional Offices. During the pilot, managers and supervisors are required to apply a newly developed mock performance appraisal process, which will be applied concurrently with the existing appraisal system. This will apply to the journeyman-level VSR positions at these two regional offices. The revised performance appraisal system makes several changes aimed at achieving a greater distribution of ratings: (1) includes 5 levels of achievement for each element rather than 3; (2) adds a numerical factor to each level of achievement; (3) requires the assignment of percent weights to each element in increments of 5; (4) requires that no one element be assigned a weight greater than 60 percent; and (5) requires non-critical elements be weighted less than any critical element; and provides for a weighted score between 0 and 400. A performance evaluation will be conducted after September 30, 2011, for each employee using the pilot and current performance appraisal systems to assess whether the new approach results in an improved distribution of ratings (i.e., more ratings of excellent).

    Recommendation: The Secretary of Veterans Affairs should examine the distribution of claims processing staff across overall performance categories to determine if its performance appraisal system clearly differentiates between overall performance levels, and if necessary adjust its system to ensure that it makes clear distinctions.

    Agency Affected: Department of Veterans Affairs

 

Explore the full database of GAO's Open Recommendations »

Sep 19, 2014

Sep 18, 2014

Sep 12, 2014

Sep 10, 2014

Aug 25, 2014

Aug 7, 2014

Jul 30, 2014

Jul 29, 2014

Looking for more? Browse all our products here