Small Business Administration:

Actions Needed to Improve the Usefulness of the Agency's Lender Risk Rating System

GAO-10-53: Published: Nov 6, 2009. Publicly Released: Dec 7, 2009.

Additional Materials:

Contact:

William B. Shear
(202) 512-4325
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Small Business Administration (SBA) guarantees individual loans that lenders originate. The agency uses its Loan and Lender Monitoring System (L/LMS) to assess the individual risk of each loan, and SBA's contractor developed a lender risk rating system based on L/LMS data. However, questions have been raised about the extent to which SBA has used its lender risk rating system to improve its oversight of lenders. The Government Accountability Office (GAO) was asked to examine (1) how SBA's risk rating system compares with those used by federal financial regulators and lenders and the system's usefulness for predicting lender performance and (2) how SBA uses the lender risk rating system in its lender oversight activities. To meet these objectives, GAO reviewed SBA documents; interviewed officials from three federal financial regulators and 10 large SBA lenders; analyzed SBA loan data; and interviewed SBA officials.

SBA's lender risk rating system uses some of the same types of information that federal financial regulators and selected large lenders use to conduct off-site monitoring, but its usefulness has been limited because SBA has not followed common industry standards when validating the system--that is, assessing the system's ability to accurately predict outcomes. Like the federal financial regulators and 10 large lenders GAO interviewed, SBA's contractor developed lender risk ratings based on loan performance data and prospective, or forward-looking, measures (such as credit scores). Using SBA data, GAO undertook a number of evaluative steps to test the lender risk rating system's predictive ability. GAO found that the system was generally successful in distinguishing between higher- and lower-risk lenders, but it better predicted the performance of larger lenders. However, the system's usefulness was limited because the contractor did not follow validation practices, such as independent and ongoing assessments of the system's processes and results, consistent with those recommended by federal financial regulators and GAO's internal control standards. For example, the agency did not require a party other than the one who developed the system to perform the validation, and SBA's contractor did not routinely reassess the factors used in the system as part of its validations. Further, SBA does not use its own data to develop alternate measures of lender performance that could be used to independently assess or supplement the risk ratings, citing resource constraints. Because SBA does not follow sound validation practices or use its own data to independently assess the risk ratings, the effectiveness of its lender risk rating system--the primary system SBA relies on to monitor and predict lender performance--may deteriorate as economic conditions and industry trends change over time. Although SBA's lender risk rating system has enabled the agency to conduct some off-site monitoring of lenders, the agency does not use the system to target lenders for on-site reviews or to inform the scope of the reviews. Unlike the Federal Deposit Insurance Corporation and the Federal Reserve, which use their off-site monitoring tools to target lenders for on-site reviews, SBA targets for review those lenders with the largest SBA-guaranteed loan portfolios. As a result of this approach, 97 percent of the lenders that SBA's risk rating system identified as high risk in 2008 were not reviewed. Further, GAO found that the scope of the on-site reviews that SBA performs is not informed by the lenders' risk ratings, and the reviews do not include an assessment of lenders' credit decisions. The federal financial regulators use the results of off-site monitoring to identify which areas of a bank's operations they should review more closely. Moreover, their reviews include an assessment of the quality of the lenders' credit decisions. Federal financial regulators are able to use review results to update their off-site monitoring systems with data on emerging lending trends. Regardless of the lender's risk rating, SBA relies on a standard on-site review form that includes an assessment of lenders' compliance with SBA policies and procedures but not an assessment of lenders' credit decisions. According to SBA officials, it is not the agency's role to assess lenders' credit decisions. Without targeting the most risky lenders for on-site reviews or gathering information related to lenders' credit decisions, SBA cannot effectively assess the risk posed by lenders or ensure that its lender risk rating system incorporates updated information on emerging lending trends.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To make better use of the lender risk rating system in SBA's oversight of lenders, the Administrator should develop a strategy for targeting lenders for on-site reviews that relies more on SBA's lender risk ratings.

    Agency Affected: Small Business Administration

    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To ensure that the lender risk rating system effectively evaluates risk, when validating the system and undertaking any redevelopment efforts, the Administrator should use SBA's own data to assess how well the lender risk ratings predict individual lender performance.

    Agency Affected: Small Business Administration

    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To ensure that the lender risk rating system effectively evaluates risk, when validating the system and undertaking any redevelopment efforts, the Administrator of the Small Business Administration (SBA) should ensure that SBA's contractor follows sound model validation practices. These practices should include (1) testing of the lender risk rating system data, processes, and results, including a routine reassessment of which factors are the most predictive of lender performance; (2) utilizing an independent party to conduct validations; and (3) maintaining complete documentation of the validation process and results.

    Agency Affected: Small Business Administration

    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To make better use of the lender risk rating system in SBA's oversight of lenders, the Administrator should consider revising SBA policies and procedures for conducting on-site reviews. These revised policies and procedures could require staff to (1) use lender risk ratings to tailor the scope of file reviews performed during on-site reviews to areas that pose the greatest risk, (2) incorporate an assessment of lenders' credit decisions in file reviews, and (3) use the results of expanded file reviews to identify information, such as emerging lending trends, that could be incorporated into its lender risk rating system.

    Agency Affected: Small Business Administration

    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Jul 28, 2014

    Jul 23, 2014

    Jul 22, 2014

    Jul 21, 2014

    Jun 26, 2014

    Jun 20, 2014

    Jun 6, 2014

    Jun 4, 2014

    May 30, 2014

    Looking for more? Browse all our products here