Tax Exempt Organizations: IRS Increasingly Uses Data in Examination Selection, but Could Further Improve Selection Processes
Fast Facts
Tax-exempt organizations often provide charitable services or membership benefits—and they generally don’t pay federal income taxes. However, they do file an annual return to report their tax-exempt activities.
In 2016, IRS started using analytical models as part of an approach to use data to select which of these returns to review for compliance with tax-exempt laws. The models score each return on the likelihood that it isn’t in compliance.
But the scores may not be reliable because of problems with the models, documentation errors, and incomplete procedures.
IRS Form 990
Highlights
What GAO Found
The Internal Revenue Service (IRS) used data to select almost 70 percent of its examinations of Form 990 returns in fiscal year 2019. Almost half of these examinations were selected using models that score returns for potential noncompliance (see figure).
Figure: Increased Use of Data in Examination Selection, Fiscal Years 2016-2019
Of the returns examined that were selected using the model, 87 percent resulted in a change to the return, indicating that IRS identified noncompliance. GAO found that the model did not improve change rates compared to prior selection methods and a higher model score is not associated with a higher change rate.
IRS has not fully implemented or documented internal controls in its established processes for analyzing data for examination selection. For example:
IRS has not defined measurable objectives for using data to select returns for examination . Without measurable objectives, IRS cannot assess how well it is doing or fully implement other internal controls.
IRS's models have deficiencies affecting the validity and reliability of return scoring and selection . IRS has incomplete definitions and procedures and did not always follow its definitions when assigning point values for identifying potential noncompliance for examination. As a result, return scoring by the models is not always consistent.
IRS did not consistently document the processing and use of data in decision-making on examination selection . Without such documentation, IRS cannot support its use of data in examination selection in all cases.
IRS does not regularly evaluate examination selection. Examination data were inconsistent across years and IRS only tracks one prior year of data. IRS also did not save data on all returns that the models scored. Without data and regular evaluations, IRS cannot assure that its models are selecting returns as intended and that deficiencies are identified and corrected.
Why GAO Did This Study
Exempt organizations often provide charitable services, or in some instances, membership benefits in furtherance of an exempt purpose. They generally do not pay federal income tax. IRS examines exempt organization returns (Form 990 and others) to address noncompliance, which may promote confidence in the tax exempt sector. In 2016, IRS started using three analytical models using Form 990 data to identify potential noncompliance and select returns for examination.
GAO was asked to review IRS's use of Form 990 data. This report assesses (1) IRS's use of data to select returns for examination and, (2) the process IRS has established for selecting returns. GAO analyzed (1) examination data from fiscal years 2016 through 2019 including results from the largest Form 990 model, and (2) model documentation for a generalizable sample. GAO interviewed IRS officials and assessed IRS policies and procedures using relevant standards for internal control.
Recommendations
GAO makes 13 recommendations, including that IRS establish objectives, revise model documentation, fully document processing and using data in decisions, and regularly evaluate examination selection. IRS agreed with all recommendations except one related to evaluating examination selection methods using consistent historical data over time. GAO continues to believe that this recommendation is valid as discussed in the report.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Internal Revenue Service | The Commissioner of Internal Revenue should document measurable objectives for using data in selecting exempt organization returns for examination. (Recommendation 1) |
In March 2022, IRS provided two objectives for using data to select returns for examination. An IRS official said they will use the two objectives for evaluation. However, only one of the objectives is measurable and IRS has not yet provided documentation on where these objectives are stated in official documents. For instance, measurable objectives need to connect data, including for returns selected for audit, and analyses to IRS's mission and strategic plan. Also, measuring progress must include documentation of analyses conducted and subsequent decisions made, beyond lists of what will be collected, analyzed, and revised. In September 2023, IRS provided strategic goals, which they related to exempt organization examinations. IRS was unable to show that these were objectives that were being used in the examination program, and they did not link them to specific measures. IRS officials said that they will address our feedback. As of October 2024, IRS did not have new information on measurable objectives.
|
| Internal Revenue Service | The Commissioner of Internal Revenue should document and consistently use clear criteria and decision rules on assigning point values to queries, using categories and sliding scales. (Recommendation 2) |
In September 2023, IRS officials said that they suspended the use of assigned point values and the scoring approach used in the compliance query sets discussed in our report. IRS shared plans for new approaches to using data to score and select exempt organizations returns for examination. IRS worked with a contractor to identify queries that can lead to productive examinations. In September 2024, provided further details and results from case selection query testing. Specifically, they provided documentation on query testing results that will be regularly assessed and measured using criteria defined in a new Compliance Query Set (CQS) Evaluation Dashboard. Queries are tracked using priority levels, number of hits, and number of observations (compliance outcomes). A performance review will be conducted once a query reaches a certain number of testing observations. The review will assess query performance and determine whether to keep, modify, or remove a query from the set. According to IRS officials, this information may enable TE/GE to utilize machine learning methods and identify the most effective query combinations in future iterations.
|
| Internal Revenue Service | The Commissioner of Internal Revenue should require a regular review of query descriptions and programming to ensure their accuracy and minimize queries that flag the same or similar compliance issue. (Recommendation 3) |
As a result of our recommendation, in April 2021, IRS released a draft desk guide requiring annual reviews of the queries and documentation of any changes. Although the desk guide will not be finalized until fiscal year 2022, relevant sections have been implemented, according to IRS officials and documents. For example, the documents showed that the first review of the queries was conducted in March 2021. The query reviews could help TE/GE ensure valid and consistent scoring from its models, thereby assisting with the selection of the most noncompliant returns for examination.
|
| Internal Revenue Service | The Commissioner of Internal Revenue should develop procedures and criteria to test new queries prior to implementation in the models. (Recommendation 4) |
In October 2021, TE/GE implemented testing of new model queries and provided documentation of the procedures and testing to date. The results of testing will eventually be used to inform a new design for the model. A TE/GE official said they plan to use several metrics for assessing each query. Although there are no standard criteria, they will use the measures together to determine whether a query is effective. TE/GE provided a list of these metrics, which include change rate and cycle time. We determined these actions are consistent with addressing the spirit of the recommendation. Testing queries on reliable data, and making adjustments based on criteria can help TE/GE implement queries that produce reasonable numbers of questionable Forms 990 that are worth pursing through examinations.
|
| Internal Revenue Service | The Commissioner of Internal Revenue should more fully document how TE/GE processes data and uses data to make examination selection decisions for sources outside of the model such as research projects and other projects that use queries. (Recommendation 5) |
In October 2021, IRS revised the charter for its Governance Board. The charter has objectives, including one to review and approve workload selection methods, such as projects that rely on data. All data driven examination selection methods are to be submitted through a Tax Exempt and Government Entities (TE/GE) portal for review and consideration by the Governance Board. This can help TE/GE ensure that it is analyzing quality information in selecting examinations.
|
| Internal Revenue Service | The Commissioner of Internal Revenue should conduct an analysis to identify the optimal interval between model runs. (Recommendation 6) |
IRS provided us with the results of an analysis conducted in June 2021 to identify the optimal interval between model runs, which concluded that the two primary times in September and March that Form 990 returns post to the system were optimal. As such, IRS recommended TE/GE's update cycle months be September and March, continuing the two-times-a-year update cycle while remaining flexible to ad hoc requests for updates. This analysis could help TE/GE maximize the use of model scores to guide decisions on examination selection and allow adequate time to examine returns under the statute of limitations.
|
| Internal Revenue Service | The Commissioner of Internal Revenue should establish a process for regularly evaluating selection decisions and related outcomes for the models and other processes that use data to select returns for examinations. (Recommendation 7) |
In May and June 2021, IRS developed desk guides with procedures for evaluating its Form 990 examination selection models and its compliance strategies. Compliance strategies are to be monitored and evaluated at various stages along several variables, and the Form 990 models are to be evaluated quarterly and annually, according to the desk guides . Evaluating the examination outcomes can help TE/GE ensure that its use of data to select returns is working as intended.
|
| Internal Revenue Service | The Commissioner of Internal Revenue should document consideration or action on recommendations from its 2018 and 2020 contractor assessments. (Recommendation 8) |
As a result of our recommendation, in May and August 2021, IRS provided a spreadsheet with status information and implementation plans for contractor recommendations made in 2018 and 2020 . This could help ensure that TE/GE does not overlook possible improvements to its models.
|
| Internal Revenue Service | The Commissioner of Internal Revenue should document how score and query data for all returns in the models will continue to be saved over the long term. (Recommendation 9) |
In October 2020, TE/GE distributed procedures for saving documentation from its Form 990 model runs. Specifically, staff are instructed to save programming documentation, query definitions, total hits for each query, and query result data set for all scored organizations. Retaining these data can help TE/GE to assess the full performance of its models. Such data would also facilitate an analysis of the queries, and whether they identified returns with changes or related pick-up returns.
|
| Internal Revenue Service | The Commissioner of Internal Revenue should ensure that historical data on examination outcomes are consistently defined and used when doing analysis of examination outcomes. (Recommendation 10) |
In September 2023, as a result of our work, IRS provided exempt organization data that maintained the three portfolios of examination work and reporting that existed at the time of our June 2020 report. TE/GE added a trend analysis for each portfolio that included current year and five years of historical data and notes to explain differences in certain years' data. TE/GE also developed an addendum to their second quarter 2023 Business Performance Review report to explain different data groupings over time. Providing data in a format that facilitates analysis of multi-year data can help with the evaluation of examination selection and the examination outcomes by portfolio.
|
| Internal Revenue Service | The Commissioner of Internal Revenue should routinely analyze the reasons for not examining selected returns and identify any necessary actions to address the reasons. (Recommendation 11) |
In October 2021, IRS issued an alert to TE/GE staff with a new requirement to enter a reason code into the case inventory management system when a selected return is not examined. Further, staff are required to include remarks and comments. In addition to providing us with this alert, TE/GE provided results of an analysis of fiscal year 2021 non-examined returns. The analysis resulted in changes to more precisely document returns that were not examined because of approaching statute of limitations dates. TE/GE also provided additional data on non-examined returns in monthly reports shared with management, and TE/GE provided an example quarterly report on the reasons for not examining selected returns. This provides management a process to monitor non-examined cases. Routinely analyzing reasons for non-examined returns, as well as related data, could help TE/GE identify actions to reduce the number of returns that are sent to the field but are then declined for examination by a manager or examiner.
|
| Internal Revenue Service | The Commissioner of Internal Revenue should annually review and update procedures as needed in relevant IRM sections on examination selection and issue interim guidance until the affected IRM sections are updated. (Recommendation 12) |
In July 2022, TE/GE provided documentation showing interim guidance expected to remain effective into 2023 for processing examination selection by its Compliance Planning and Classification Issue ID Research office specifically and TE/GE more generally. They also provided documentation of how they regularly track and review updates to the Internal Revenue Manual (IRM). Out-of-date IRM and related guidance increases the risk that staff a) follow incorrect procedures, b) use guidance that is not transparent to the public, c) administer tax laws inconsistently, and, d) misinform taxpayers. Consequently, timely documentation of new procedures and responsibilities improves the accuracy and reliability of IRM content.
|
| Internal Revenue Service | The Commissioner of Internal Revenue should document why TE/GE has not identified any risks in its risk register for using data to select exempt organization returns for examination. If risks are subsequently identified, TE/GE should document how it plans to analyze and address them. (Recommendation 13) |
In May 2020, IRS issued a Risk Assessment Guide and Toolkit to help facilitate risk assessment at the business unit and program levels. We reviewed the document and found it provides numerous steps for identifying objectives, risks, and mitigating activities as well as guidance for such items as reevaluating and updating risk assessment methodology. As of September 2020, TE/GE's Risk Liaison worked with its Compliance Planning and Classification office to properly scope and document risk for TE/GE. They provided documentation of a new data-related risk involving examination selection decisions added to its risk register. This also included identifying three areas of mitigation of risks. TE/GE also shared documentation of its process for cataloguing their efforts and evidence for identifying mitigation activities and the results.
|