VA Student Financial Aid:
Management Actions Needed to Reduce Overlap in Approving Education and Training Programs and to Assess State Approving Agencies
GAO-07-384: Published: Mar 8, 2007. Publicly Released: Mar 8, 2007.
In fiscal year 2006, the Department of Veterans Affairs (VA) paid approximately $2.1 billion in education assistance benefits to more than 470,000 beneficiaries and about $19 million to state approving agencies (SAA) to assess whether schools and training programs offer education of sufficient quality for veterans to receive VA education assistance benefits when attending them. Qualified individuals--veterans, service persons, reservists, and certain spouses and dependents--receive benefits through a number of education assistance programs for the pursuit of various types of programs, such as a degree program, vocational program, apprenticeship, or on-the-job training. The Departments of Education (Education) and Labor (Labor) also assess education and training programs for various purposes, primarily for awarding student aid and providing apprenticeship assistance. In 2006, under Title IV of the Higher Education Act, Education provided nearly $77 billion in student aid in the form of both grants and loans. The Department of Education assesses and certifies postsecondary institutions for participation in Title IV programs through various oversight functions to ensure that these schools meet federal administrative and financial requirements and that they are accredited and licensed. Similarly, under the National Apprenticeship Act of 1937, the Department of Labor is authorized to formulate and promote the furtherance of labor standards to safeguard the welfare of apprentices. Given each agency's role, the potential of duplicative efforts among federal agencies has been a congressional concern. In 1995, GAO reported on this matter and concluded that there was a substantial amount of overlap between the efforts of SAAs and the other federal agencies. In light of continued congressional interest in this issue, we have now answered the following questions: (1) What changes have occurred in state approving agencies' duties and functions since 1995? (2) To what extent does the SAA approval process overlap with efforts by the Departments of Education and Labor? (3) What, if any, additional value do the SAA approval activities bring to VA education benefit programs?
Since 1995, legislative changes effective in 2001 created additional responsibilities for SAAs, including promoting the development of apprenticeship and on-the-job training programs, providing outreach services, and approving tests for occupational licensing. From fiscal years 2003 to 2006, SAA funding increased from $13 million to $19 million to expand services and support the additional responsibilities. However, funding is scheduled to decrease beginning in fiscal year 2008. Many education and training programs approved by SAAs have also been approved by Education or Labor, and VA and SAAs have taken few steps to coordinate approval activities with these agencies. In addition, information is not available to determine the amount of resources spent on SAA duties and functions, including those that may overlap with other agencies and programs. SAAs reportedly add value to the approval process for education and training programs through (1) a focus on student services for veterans and on the integrity of VA benefits, (2) more frequent on-site monitoring of education and training programs than provided by Education or Labor, and (3) assessments and approval of a small number of programs that are not reviewed by other agencies. However, VA's lack of outcome-oriented performance measures for evaluating SAAs makes it difficult to assess the significance of these efforts. In conclusion, while VA spends $19 million (less than 1 percent of the total benefit amount) to fund SAA duties and functions, it does not track the amount it spends on specific SAA activities, especially those that may also be performed by other agencies. Without knowing the amount of resources spent on specific duties and functions, VA does not have all relevant information for making resource allocation decisions and cannot determine if it is spending its federal dollars efficiently and effectively. While we have identified some overlap in approval efforts across agencies, the full extent of the overlap between SAA duties and other agencies' oversight efforts is unknown. It is important that VA work with other federal agencies to determine how the scope of the approval process could be streamlined to reduce overlap and ensure that federal dollars are spent efficiently. Finally, it is difficult to assess the effectiveness of SAA activities, in part because VA does not have outcome measures in place to fully evaluate SAA performance. Evaluating the effectiveness of VA's approval process is vitally important in order to manage the program and improve program results.
Recommendations for Executive Action
Status: Closed - Implemented
Comments: VA prepared a recommendation for concurrence by VA's General Counsel that adds language to the FY08 State Approving Agency (SAA) contract that addresses this recommendation. This language adds provisions that: (1) SAAs furnish written reports of outreach visits; (2) all visit reports must include travel time and time spent on the visit; (3) outreach expenses claimed on invoices will be supported by visit reports, travel vouchers, and receipts; and (4) SAAs shall include in the notice of approval the length of time spent reviewing and evaluating official school publications. In May 2007, legislation (S.1290) was introduced to address concerns this recommendation raises, then referred to the Committee on Veterans' Affairs for further action. In July 2009, VA reported that it had worked with the National Association of State Approving Agencies to revise the SAA contracts to include a reporting system for SAA approval activities such as tracking their their time and expenses related to outreach efforts. These changes were incorporated into the SAA FY 2009 final contracts and were sent to them on October 24, 2008.
Recommendation: To help ensure that federal dollars are spent efficiently and effectively, the Secretary of the Department of Veterans Affairs should take steps to monitor SAA spending and identify whether any resources are spent on activities that duplicate the efforts of other agencies. The extent of these actions should be in proportion to the total resources of the program. Specifically, VA should require SAAs to track and report data on resources spent on approval activities such as site visits, catalog review, and outreach in a cost-efficient manner.
Agency Affected: Department of Veterans Affairs
Status: Closed - Implemented
Comments: VA met with representatives from the Department of Labor in March 2007 to discuss areas of possible overlap between the DOL and VA. Talks will continue with DOL as needed to insure that no unnecessary overlap is occurring. VA is in the process of establishing a meeting with the appropriate staff at the Department of Education to discuss areas of possible overlap. In May 2007, Senate Bill, S. 1290 was introduced to address concerns this recommendation raises. The legislation was referred to the Committee on Veterans' Affairs for further action. In July 2009, VA reported that it developed a legislative proposal for 2011 to change section 3672 of title 38 U.S.C. to give VA more discretion in utilizing the SAAs, such as to direct the SAAs approval activities to areas that are not currently reviewed by other government agencies. On January 4, 2011, P.L. 111-377 was passed, which eliminates VA overlap with the Department of Education, accrediting agencies, Federal Aviation Administration, Department of Labor, and other State apprenticeship agencies in regard to program approval issues.
Recommendation: To help ensure that federal dollars are spent efficiently and effectively, the Secretary of the Department of Veterans Affairs should take steps to monitor SAA spending and identify whether any resources are spent on activities that duplicate the efforts of other agencies. The extent of these actions should be in proportion to the total resources of the program. Specifically, VA should collaborate with other agencies to identify any duplicative efforts and use the agency's administrative and regulatory authority to streamline the approval process.
Agency Affected: Department of Veterans Affairs
Status: Closed - Not Implemented
Comments: VA reported that the agency is developing measures to assess the effectiveness of the State Approving Agency (SAA) efforts and they expect to implement the measures in FY 2008. In July 2009, VA reported that it had collaborated with the National Association of State Approving Agencies (NASAA) to develop outcome-oriented performance measures to assess the effectiveness of SAA efforts, and these measures were incorporated into the SSA FY 2009 contracts. However, when we reviewed their business plan, we determined that the measures still focused on outputs rather than outcome-oriented measures, such as completion rates of beneficiaries that would measure the impact of SSA approval activities. In response, VA noted that SAA performance is being measured by outreach to veterans, including the type of outreach and number of veterans reached. SAAs must complete a report with the name, date, number of attendees, length of time spent at the event, address of the event and a description of the activities performed at the event. This report is not specifically included in the business plan because the SAA has no control over the number of attendees. Outreach performance measures are assessed by review of the quarterly report completed by SAAs, review of their self-assessment and the Education Liaison representative (ELR) evaluation at the annual Joint Peer Review Group (JPRG). The JPRG assigns a rating based on achievement of the performance measures. The SAAs do place outreach activities such as development of outreach materials and sending letters to veterans in their business plans. Performance outcomes cannot be measured for these areas. Their business plan also tracks the work processes in which they plan to engage for the fiscal year. However, SAAs do not have the capability to track outcome measures such as completion rates for beneficiaries because school officials do not keep records of completion rates based on only the veteran population. GAO believes that measuring the attendance of the outreach activities is an output rather than an outcome. The SAAs have six core duties/functions and outreach is just one of those functions. Our report provided several examples of their output measures and how those measures could be modified to be more performance-oriented. The completion rates of beneficiaries was one suggestion of how to measure performance for their approval function. VA may encounter some limitations on measuring SAAs' performance for some of the six duties/functions, but it's not clear how the agency has have addressed other SAA functions/duties. In 2011, VA reported that NASAA completed its review of all potential outcome measures and determined they do not have the ability to track detailed outcome-based performance measures such as beneficiary data. VA reviewed all related information and determined that although SAA approval work is valuable, it does not provide a direct correlation to measurable outcomes such as beneficiary graduation rates. In 2011, Congress enacted P.L. 111-377, which allows for specific programs to be deemed already approved and will allow VA to utilize SAAs for compliance activities at schools. VA believes this added focus on compliance, training, and oversight of schools in relation to benefit payments will help ensure quality education for veterans. We acknowledge VA's efforts, however, as previously noted in our report, it is difficult to assess the effectiveness of SAA performance without outcome measures, such as outcomes of their supervisory and inspection visits or technical assistance. While P.L. 111-377 will be reducing SAAs approval activities, one of SAAs six core duties, without outcome measures it is unclear whether VA has sufficient information for making resource allocation decisions to fund SAA duties and functions and to determine if it is spending its federal dollars efficiently and effectively. As such, we consider this recommendation as not implemented.
Recommendation: The Secretary of the Department of Veterans Affairs should establish outcome-oriented performance measures to assess the effectiveness of SAA efforts.
Agency Affected: Department of Veterans Affairs