Small Business Administration: Actions Needed to Improve Confidence in Small Business Procurement Scorecard
Fast Facts
The federal government is required to allocate at least 23% of its direct contract spending to small businesses. The Small Business Administration's annual scorecards track progress toward that goal.
SBA changed how it calculates scores for fiscal year 2017 but didn't initially plan to evaluate the effect of the changes.
We recommended SBA evaluate its new scoring methodology to assess how effectively the scorecards measure agency performance and promote contracting opportunities for small businesses. SBA could use the evaluation results to inform Congress in its upcoming required report.
A graphic showing part of a blank Small Business Procurement Scorecard
Highlights
What GAO Found
For fiscal year 2017, the Small Business Administration (SBA) revised the methodology for its Small Business Procurement Scorecard, which is used to assess federal agencies' progress toward small business procurement goals. SBA made revisions to address requirements specified in the National Defense Authorization Act for Fiscal Year 2016 (2016 NDAA). SBA (1) reduced the share of the total scorecard grade devoted to prime contracting achievement, which is the dollar amount of contracts awarded directly to small businesses, and (2) added an element calculating changes in the number of small businesses receiving prime contracts. SBA made two additional revisions—with input from other agencies' representatives—to increase the share of subcontracting achievement results and peer review of required activities designed to facilitate small business procurement (see figure). In July 2018, officials said they had begun developing a plan to evaluate the effects of the revised scorecard methodology but did not provide a draft plan. Conducting a well-designed and comprehensive evaluation could aid SBA in determining whether the scorecard is an effective tool for helping to achieve the agency's strategic goals.
Changes to Small Business Procurement Scorecard Methodology by Scorecard Element, Fiscal Years (FY) 2016 and 2017
(Scorecard elements are expressed as a percentage of total scorecard grade.)
Note: Prime contracting involves direct federal awards to a contractor. Subcontracting involves decisions by prime contractors to allocate certain activities and payments to other businesses. The peer review process assesses federal agencies' compliance with required activities designed to facilitate small business procurement.
The published fiscal year 2017 scorecards originally contained errors, including an incorrect grade and numeric score for one agency, and SBA does not have a process to ensure that scorecard results are published accurately. Although SBA later corrected the errors, the agency did not initially document that scorecards had been changed, which is inconsistent with SBA's policy on information quality. SBA officials said that errors occurred in the process of formatting scorecards for publication. Errors in the published scorecards—and the initial lack of disclosure about corrections—weaken data reliability and may undermine confidence in scorecard data.
Agency officials and representatives of small business groups that GAO interviewed generally expected the scorecard revisions to have little impact on small business procurement opportunities. However, one agency's officials said they would focus more on tracking subcontracting activity as a result of changes to the scorecard.
Why GAO Did This Study
Each year SBA produces a scorecard measuring federal contract spending allocated to small businesses. The 2016 NDAA included a provision for SBA to revise the scorecard's methodology and for GAO to evaluate the effects of those revisions for fiscal year 2017. This report discusses, among other things, (1) SBA's changes to the scorecard methodology and plans, if any, to evaluate the effects of these changes, (2) the extent to which SBA has processes to disseminate reliable information, and (3) views of selected stakeholders on the scorecard's effects on small business procurement opportunities.
GAO analyzed SBA's prior and revised scorecard methodology and results and interviewed officials from SBA, four other federal agencies selected based on small business procurement volume and other attributes, and three groups representing the interests of small businesses.
Recommendations
GAO is recommending that SBA (1) design and implement a comprehensive evaluation to assess scorecard revisions and (2) institute a process for reviewing scorecards for accuracy prior to publication and a mechanism for disclosing corrected information. SBA generally agreed with GAO's recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Small Business Administration | The SBA Administrator or her designee should complete the design and implementation of a comprehensive evaluation of the Small Business Procurement Scorecard aligned with key attributes of effective program evaluations to assess the effectiveness of the revised scorecard in measuring agency performance and promoting small business procurement opportunities. (Recommendation 1) |
SBA hired a third-party contractor to conduct an evaluation of the Small Business Procurement Scorecard that assessed the effectiveness of the revised scorecard. The contractor completed its evaluation in March 2021. We found that the design and implementation of the evaluation was aligned with the attributes of effective program evaluations. The evaluation report made eight recommendations to SBA to create more opportunities for small businesses and improve agencies' ability to meet the scorecard goals.
|
| Small Business Administration | The SBA Administrator or her designee should institute a process to review Small Business Procurement Scorecards for accuracy prior to publication and a mechanism for publicly identifying when issued scorecards have been revised. (Recommendation 2) |
SBA finalized a data quality guide in March 2021 that describes the process for reviewing scorecards for accuracy prior to publication and a mechanism for publicly identifying when issued scorecards have been revised. To complete the review of accuracy prior to publication, the data guide describes a process involving two people who reviewed final scorecards before they are publicly released. The mechanism for publicly identifying when issued scorecards have been revised includes a process for identifying the type of error--for example, whether the error is grammatical or one that affects the scorecard results--and the steps that should be taken for each type of error.
|