This is the accessible text file for GAO report number GAO-03-486 entitled 'Tax Administration: IRS Should Reassess the Level of Resources for Testing Forms and Instructions' which was released on April 11, 2003. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products’ accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. Report to the Chairman and Ranking Minority Member, Committee on Finance, U.S. Senate: United States General Accounting Office: GAO: April 2003: TAX ADMINISTRATION: IRS Should Reassess the Level of Resources for Testing Forms and Instructions: GAO-03-486: GAO Highlights: Highlights of GAO-03-486, a report to Chairman and Ranking Member of the Senate Committee on Finance Why GAO Did This Study: Taxpayers rated the Internal Revenue Service’s (IRS) ability to provide clear and easy-to-use forms and instructions among the lowest of 27 indicators of service in 1993. Due to continuing concerns about unclear forms and instructions, GAO was asked to determine * whether and how often IRS tests the clarity of new and revised individual income tax forms and instructions; * the benefits, if any, of testing forms and instructions for clarity prior to their use; and whether any factors limit IRS’s ability to do more tests and if so, how they can be addressed. What GAO Found: IRS used taxpayers and its employees to test revisions to five individual income tax forms and instructions from July 1997 through June 2002. According to IRS officials, they revised about 450 tax forms and instructions in 2001, many of which were for individual income tax returns. Testing forms and instructions can help ensure their clarity and thereby benefit taxpayers and IRS by, for instance, reducing taxpayers’ time to understand and complete tax forms, reducing calls to IRS for assistance, and reducing taxpayer errors. Due to similar benefits, federal agencies we contacted that routinely collect information from the public test their questionnaires. Quantifying benefits due to testing is difficult, but IRS’s experience in revising and testing Earned Income Credit and Child Tax Credit forms and instructions suggests that benefits of testing in some cases can considerably exceed the cost of testing. If taxpayers who did their own tax returns needed 1 less minute to understand these two credits due to testing, their time saved, valued at the minimum wage, would be worth $1.2 million; IRS’s contracting cost for the two tests was $56,000. Although IRS officials recognized that testing could be beneficial, they cited tight time frames and constrained resources as limiting their ability to do more tests. While IRS faces time constraints when making some changes to forms and instructions due to the passage of new laws, not all changes are time constrained. IRS does not have procedures specifying which versions of draft forms and instructions should be tested with taxpayers or when in its annual forms development process testing should occur. Resources currently available for testing are limited but the office responsible for testing has not developed data on missed testing opportunities and has limited data on the benefits that have been realized when testing occurred. IRS’s planning and budgeting process uses such data to support resource allocation decisions. What GAO Recommends: This report makes a series of recommendations that, if fully implemented, would improve IRS’s annual process for creating and revising individual income tax forms and instructions by helping to ensure that its scarce testing resources are consistently applied to the highest priorities, impediments to testing are lessened, and appropriate information is developed so IRS management can better ensure that adequate resources are available to support testing in view of the potential benefits to taxpayers and IRS. IRS agreed with our recommendations and plans to implement all but one of them in time for the 2004 forms development cycle. www.gao.gov/cgi-bin/getrpt?GAO-03-486. To view the full report, including the scope and methodology, click on the link above. For more information, contact Michael Brostek at (202) 512-9110 or brostekm@gao.gov. Contents: Letter: Results in Brief: Background: IRS Tested Five Tax Forms and Instructions Over 5 Years: Testing Written Documents Helps Ensure Clarity and Benefits Taxpayers and IRS: IRS Has Not Addressed Constraints to Increased Testing: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Objectives, Scope, and Methodology: Appendix II: Use of Testing in NCHS, Census, and BLS: Appendix III: Comments from the Internal Revenue Service: Tables: Table 1: Forms and Instructions IRS Tested Using Individual Taxpayers Between July 1997 and June 2002: Table 2: Percentage Decrease in the Number of EIC and Child Tax Credit Errors before and after Revising and Testing: Table 3: Potential Benefits if Testing Helps Ensure the Clarity of Tax Forms and Instructions: Table 4: Illustration of Potential Savings to IRS Alone in 2000 from Testing EIC and Child Tax Credit Forms and Instructions: Table 5: Illustration of Potential Savings to Taxpayers and IRS in 2000 from Error Reductions Alone from Testing EIC and Child Tax Credit Forms and Instructions: Table 6: Illustration of Potential Benefit to Taxpayers in 2000 if Testing Reduced the Time Needed to Understand and Complete EIC and Child Tax Credit Forms: Table 7: Illustration of Potential Cost Savings to IRS in 2002 if the Rate Reduction Credit Instructions Had Been Tested and Errors Were Reduced: Table 8: Illustration of Potential Savings to Taxpayers and IRS in 2002 from Potential Error Reductions if Testing the Rate Reduction Credit Had Reduced Taxpayer Errors: Table 9: Illustration of Potential Benefit to Taxpayers in 2002 if Testing Reduced the Time Needed to Understand and Complete the Rate Reduction Credit: Figures: Figure 1: Simplified Overview of IRS’s Annual Tax Forms Development Process: Figure 2: Simplified Overview of Where Testing Can Occur in IRS’s Annual Tax Forms Development Process: This is a work of the U.S. Government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. It may contain copyrighted graphics, images or other materials. Permission from the copyright holder may be necessary should you wish to reproduce copyrighted materials separately from GAO’s product. Abbreviations: BLS: Bureau of Labor Statistics: EIC: Earned Income Credit: ERS: Error Resolution System: FTE: full-time equivalent: IRS: Internal Revenue Service: NCHS: National Center for Health Statistics: OMB: Office of Management and Budget: TIGTA: Treasury Inspector General for Tax Administration: W&I: Wage and Investment: United States General Accounting Office: Washington, DC 20548: April 11, 2003: The Honorable Charles E. Grassley Chairman, Committee on Finance United States Senate: The Honorable Max Baucus Ranking Minority Member, Committee on Finance United States Senate: Internal Revenue Service (IRS) tax forms and instructions that are not clear and understandable have been a long-standing frustration of taxpayers.[Footnote 1] For example, respondents to a 1993 customer satisfaction survey rated IRS’s ability to provide clear and easy-to- use tax forms and instructions and to minimize taxpayer’s burden as the lowest of 27 indicators of service.[Footnote 2] More recently, we cited the instructions for the tax year 2001 rate reduction credit as confusing and one factor contributing to millions of taxpayer errors.[Footnote 3] To be fair, IRS faces a challenge in clearly communicating with taxpayers due to the complexity of the Internal Revenue Code. However the code’s complexity also underscores the importance of tax forms and instructions that are as clear and understandable as possible. One approach with recognized potential for aiding in the development of clearer forms and instructions is testing. By testing draft forms and instructions with taxpayers, IRS might be able to detect and revise language that is unnecessarily confusing. The Treasury Inspector General for Tax Administration (TIGTA) and we have both previously recommended that IRS do more to, for instance, identify what individual taxpayers find difficult to understand about tax forms or publications.[Footnote 4] Because of your interest in ensuring that taxpayers have clearer, understandable tax forms and instructions and because of the potential for testing to contributing to that end, you asked us to determine: * whether and how often IRS used taxpayers to test the clarity of new or revised individual income tax forms and instructions; * the benefits, if any, of having taxpayers test forms and instructions for clarity prior to their use by the public; and * whether any factors limit IRS’s ability to test more forms and instructions for clarity and if so, how these factors can be addressed. To address these objectives, we interviewed officials (1) in IRS’s Wage and Investment (W&I) Division’s Tax Forms and Publications Division, (2) from three federal agencies that use private citizens extensively in testing research and data collection activities, and (3) a private research firm that specializes in testing written documents, such as forms and surveys, for a variety of public and private sector clients. We also developed illustrations of the potential benefits to taxpayers and IRS of testing forms and instructions for clarity. Our scope and methodology are discussed in greater detail in appendix I to this report. Results in Brief: From July 1997 through June 2002, IRS used taxpayers or its employees, primarily in focus groups, to test 5 individual income tax forms or instructions. According to IRS officials, they revised about 450 forms and instructions in 2001, many of which were for individual income tax returns. Testing written documents, such as forms and instructions, prior to their use helps ensure that they communicate clearly. Testing thereby helps (1) lessen the burden to respondents of understanding and completing forms and (2) reduce errors made by respondents. To the extent that taxpayers find IRS’s tax forms and instructions that were tested easier to understand, IRS is less likely to be contacted by taxpayers for assistance in fulfilling their tax obligations and is likely to have fewer taxpayer errors to detect and correct. Benefits like these may be difficult to quantify, but form the basis for several agencies we contacted, as well as ourselves, to make testing a standard practice when developing or modifying documents used to collect information from the public. IRS’s recent, limited experience with testing forms and instructions used by individuals suggests that testing is beneficial. At least in some cases the potential benefits of testing IRS’s forms and instructions are likely to be substantially greater than the costs of testing to IRS. Although IRS officials said that making greater use of testing to improve clarity of forms and instructions could be beneficial, they cited tight time frames for making changes and limited resources in the Tax Forms and Publications Division as the primary factors that limit their ability to conduct more tests. Although IRS does face time constraints when making some changes, especially when changes are due to new tax laws, time constraints do not apply to all changes IRS makes to forms and instructions for individual taxpayers. Further, IRS does not have clear procedures specifying which draft versions of forms and instructions should be tested with taxpayers or when testing should occur during the annual forms update process. Resources available within the division responsible for developing forms and instructions are limited. For example, one employee has responsibility for organizing testing efforts. However, the division has not developed information that IRS uses when making resource allocation decisions. The office has not developed data on the universe of changes for which testing is most likely to yield significant benefits to taxpayers and IRS and it has only limited data on the benefits that have been realized when testing has occurred. This report makes a series of recommendations to improve IRS’s annual process for creating and revising individual income tax forms and instructions through ensuring that (1) its scarce testing resources are consistently applied to the highest priorities, (2) impediments to testing are lessened, and (3) appropriate information is developed so IRS managers can ensure that adequate resources are available for testing in view of the potential benefits to taxpayers and IRS. IRS agreed with our recommendations and plans to implement all but one of them in time for the 2004 forms development cycle. The remaining recommendation--to ensure that an appropriate range of evaluations is conducted of tests--would take more time to put into practice. Background: Although most organizational components within IRS are involved, the Tax Forms and Publications Division within IRS’s Media and Publications Division of the W&I Division is primarily responsible for creating and improving tax forms, instructions, and other documents. One goal of the Tax Forms and Publications Division is to make tax forms and instructions as clear and understandable as possible. It is divided into three branches--Individual Forms and Publications, Business Forms and Publications, and Tax Exempt/Government Entities and Specialty Forms and Publications. As of late January 2003, 103 persons were assigned to the Tax Forms and Publication Division, including about 15 persons whose primary responsibility was creating, revising, and reviewing individual income tax forms and instructions. Many tax forms and instructions are revised annually, often with short turnaround times and in response to tax law changes. IRS also periodically reviews tax forms and, when appropriate, schedules them for revision. According to IRS’s estimate, it revised about 450 tax forms and instructions in 2001 that affected individual and business tax returns. In addition to tax law changes, revisions to tax forms and instructions generally reflected procedural changes, legal rulings, and feedback from internal and external stakeholders about the understandability of forms and instructions. As illustrated in figure 1, the annual tax forms development process generally starts with a review of the current tax forms. IRS’s tax law specialists review the existing forms and instructions to determine what changes, if any, may be needed to reflect tax law changes and other requirements. The tax law specialists consider comments from a variety of sources both within and outside IRS. For example, comments may be obtained from IRS customer service staff in toll-free call centers who answer calls from taxpayers and have firsthand knowledge of particular forms or instructions that were confusing to taxpayers. IRS’s Taxpayer Advocate Service staff may also provide comments useful to the tax law specialists. The Tax Forms Coordinating Committee, comprised of representatives from all of IRS’s key components, the Department of the Treasury, and IRS’s Chief Counsel, reviews draft forms to help ensure that they are not overly burdensome and that they conform to legal and technical requirements. Draft forms are generally posted to IRS’s external website so that external stakeholders and taxpayers may review and comment on them. The Office of Management and Budget (OMB) is responsible for approving each form once every 3 years. The purpose of OMB’s approval is to assess IRS’s compliance with the Paperwork Reduction Act, which, among other things, requires agencies to assess the extent of burden the information they collect imposes on the public. Under the Paperwork Reduction Act, OMB must approve new forms and major revisions to existing ones. After OMB’s approval, the forms and instructions are sent to IRS’s vendors to be printed. Generally, IRS needs to have approved forms ready for printing by early October to ensure that they can be printed and distributed to the public when the tax filing season starts the following January. Figure 1: Simplified Overview of IRS’s Annual Tax Forms Development Process: [See PDF for image] Note: GAO analysis of IRS data. [End of figure] IRS Tested Five Tax Forms and Instructions Over 5 Years: As shown in table 1, IRS used taxpayers or IRS employees to test the clarity of five individual income tax forms and instructions from July 1997 through June 2002.[Footnote 5] IRS relied primarily on focus groups to do the testing. Contractors using private citizens did three of the tests (i.e., Earned Income instructions, Child Tax Credit instructions, and Schedule D) and IRS did the other two using IRS employees. Table 1: Forms and Instructions IRS Tested Using Individual Taxpayers Between July 1997 and June 2002: Forms and instructions: Earned Income Credit schedule; Year tested: 1998; Number of participants: 50; Testing method: 5 focus groups. Forms and instructions: Earned Income Credit instructions; Year tested: 1999; Number of participants: 104; Testing method: 10 focus groups. Forms and instructions: Child Tax Credit instructions; Year tested: 1999; Number of participants: 104; Testing method: 10 focus groups. Forms and instructions: Schedule D Form-Capital Gains and Losses; Year tested: 2001; Number of participants: 48; Testing method: 6 focus groups; 8 one-on-one interviews. Forms and instructions: Innocent Spouse Application form; Year tested: 2002; Number of participants: 52; Testing method: 6 focus groups. Source: IRS. Note: Forms and instructions may also refer to schedules and worksheets. [End of table]: The two testing methods that IRS used--focus groups and one-on-one interviews--are among the commonly used methods to obtain data from individuals on whether documents such as forms and instructions are clear and understandable. Focus groups generally consist of a small number of participants--about 8 to 12 persons---and are usually selected and organized around the focus group topic. Focus groups are a form of group interviewing that relies on interaction within the group to obtain the impressions of a group of people but not necessarily the impressions of each participant. One-on-one interviews, which aim to obtain individual attitudes, beliefs, and feelings, are used to probe individuals about specific difficulties they may have with completing a form or reading instructions. In some instances, these methods may be used in combination depending upon the particular circumstances of the test. As noted in table 1, IRS used both focus groups and one-on-one interviews in testing the Schedule D form. Testing Written Documents Helps Ensure Clarity and Benefits Taxpayers and IRS: Testing written documents such as tax forms and instructions helps ensure they are clear, thereby benefiting taxpayers and IRS. Researchers from three federal agencies and a private research firm said testing leads to clearer documents as well as more accurate responses. Our guidance on developing and using questionnaires also recommends testing prior to distribution.[Footnote 6] IRS’s experience indicates that testing likely improves the clarity of tax forms and instructions and may therefore help reduce the number of errors taxpayers make. Although limited data were available on the costs and benefits of testing IRS forms and instructions, recent changes to Earned Income Credit (EIC) and Child Tax Credit forms illustrate the potential for benefits to significantly exceed the costs of testing. Experiences of Three Federal Agencies, a Private Research Firm, and IRS Support Use of Testing: Researchers from the National Center for Health Statistics (NCHS), the Census Bureau, and the Bureau of Labor Statistics (BLS) said that testing helps ensure that their documents are clear, and therefore users are more likely to understand them and complete them correctly. Consequently, researchers from the agencies said they routinely test written documents, such as forms or surveys, prior to public distribution. A representative of a private research firm also said that testing ensures that documents communicate clearly and that his firm and many others perform such testing for a wide variety of private and public clients. The researchers from the federal agencies told us that the benefits of testing are difficult to quantify and, in some instances, may not be quantifiable. NCHS officials stated that the experience they have gained over time from testing and revising documents has helped them develop clearer forms from the outset, a benefit that may not be quantifiable. A Census researcher also agreed that testing is beneficial but difficult to quantify. Nevertheless, the Census researcher said that testing documents prior to public use helps ensure that they are clear and understandable, which gives the agency a greater chance of receiving accurate responses and which lessens the need for follow-up interviews. Similarly, a BLS researcher noted that, though difficult to quantify, testing benefits the agency by reducing errors made by respondents and those reductions can result in savings of time, money, and effort for BLS. See appendix II for additional information on the use of testing in these three agencies. The representative of the private research firm also provided some perspective of its experiences in conducting tests for public and private sector customers. This firm, like many similar companies, provides a variety of testing and data collection services to public and private sector customers. The firm arranges and conducts focus groups, one-on-one interviews, and other tests in order to ensure the clarity of forms, instruction manuals, surveys, and Web sites among other things. Focus groups generally involve 12 participants and cost around $6,700 to $7,600, excluding costs to develop the item to be tested but including incentive pay for participants that could range from $25 to $50 per participant. These costs may be higher, the representative said, if the participants come from special groups. For example, if the participants are medical doctors, incentive pay could be as high as $250 per participant. The researcher also said that private firms vary in how much they spend to test their documents. He estimated that firms generally spend between $300,000 and $500,000 to ensure that a form or document is clear and will meet the firm’s needs. In some cases, firms spend more than $500,000 to do a series of tests, making revisions between each test, before they arrive at a final version of a form or document. He also added that most firms do not spend at levels that would allow them to test all their forms and documents. While maintaining that testing is beneficial, researchers also stated that testing is not fail-safe. It can help identify particular parts of a form that are not clear, researchers said, but it cannot ensure that subsequent changes to the form will entirely resolve the clarity issue. In addition, testing may identify problems participants have in completing written documents, but the participants’ problems may be related to other issues, such as poor math skills, rather than confusing or unclear documents. Testing questionnaires before distribution is also recommended as a quality assurance measure in our guidance on developing and using questionnaires. According to our guidance, testing questionnaires before they are used is one of the best ways to ensure that the document actually communicates what it was intended to communicate and that users will uniformly interpret it. Testing increases the likelihood that respondents will provide the information needed and helps to alleviate inaccurate responses. Our guidance is also consistent with professional literature on survey design. According to professional literature, “reducing measurement error through better question design is one of the least costly ways to improve survey estimates. For any survey, it is important to attend to careful question design and pretesting.”[Footnote 7] IRS’s recent limited experience with testing indicates that testing may help ensure the clarity of tax forms and instructions. In 1999, IRS revised the forms and instructions related to the EIC and the Child Tax Credit, tested the revised forms and instructions, and revised them again based on the test results. The following year, when the revised and tested forms and instructions were used by taxpayers, the error rates for the EIC and the Child Tax Credit decreased by 28 and 35 percent, respectively, as shown in table 2.[Footnote 8] Table 2: Percentage Decrease in the Number of EIC and Child Tax Credit Errors before and after Revising and Testing: Forms/Instructions: Earned Income Credit; Number of errors on tax returns filed in 1999 (before testing): 1,797,162; Number of errors on tax returns filed in 2000 (after testing): 1,296,095; Percentage decrease in errors from 1999 to 2000: 28. Forms/Instructions: Child Tax Credit; Number of errors on tax returns filed in 1999 (before testing): 1,430,394; Number of errors on tax returns filed in 2000 (after testing): 934,289; Percentage decrease in errors from 1999 to 2000: 35. Source: IRS data. [End of table]: IRS officials told us they attribute part of the decrease in EIC errors to a new approach officials developed for structuring EIC forms and instructions and part to the improvements in the draft documents that resulted from testing the revised forms and instructions. Before the EIC forms and instructions were revised, the instructions included a definition and example describing a qualifying child that taxpayers had to interpret.[Footnote 9] Incorrectly claimed qualifying children have been a major source of EIC errors. IRS revised the instructions so that taxpayers would answer a series of “yes/no” questions to determine if they have a qualifying child instead of relying on their interpretations of the definition or example of a qualifying child. They then tested the old format and the new format. The number of errors decreased substantially when taxpayers used the new format. IRS then made some final changes to clarify the instructions based on test results. In IRS officials’ opinions, the “yes/no” format made it clearer for taxpayers to determine if they had a qualifying child. Benefits of Testing Forms and Instructions Can Far Exceed IRS’s Costs: The benefits of testing some changes to IRS’s forms and instructions can considerably exceed IRS’s costs to do tests, especially because so many taxpayers can be affected by improvements in clarity that may result from testing. IRS’s contract costs, including travel, for testing changes to the EIC and Child Tax Credit forms and instructions were about $56,000 and these costs may have been offset within IRS alone in the year that the change was implemented. More significantly, if testing changes to forms and instructions for these credits led to a 1-minute reduction, on average, in the time taxpayers needed to understand and complete the forms during the 2000 tax filing season, affected taxpayers would have saved 240,000 hours worth $1.2 million valued at minimum wage. Taxpayers and IRS Can Realize a Wide Range of Benefits and Costs From Testing: Testing has the potential to yield a wide range of benefits to taxpayers and IRS. Table 3 summarizes some of the potential benefits that could result if testing helps clarify tax forms and instructions. If the form or instruction that has been revised and tested remains unchanged, some potential benefits could recur annually for the life of the form or instruction. Table 3: Potential Benefits if Testing Helps Ensure the Clarity of Tax Forms and Instructions: IRS’s benefits: * Fewer errors that need to be corrected before processing returns; * Reduction in audits due to clarity-related taxpayer errors and redirection of audit resources to other noncompliant taxpayers; * Reduction in demand for taxpayer assistance at local IRS offices may allow IRS to provide better service to other taxpayers; * Reduction in demand for assistance via IRS’s toll-free telephone service related to clarity issues may enable IRS to answer some calls that would otherwise have not been answered; * Experience and knowledge gained through testing may lead to creation of clearer forms and instructions in the future; Taxpayers’ benefits: * Increased clarity of IRS’s forms and instructions reduces; -time to complete tax forms; -expenses associated with obtaining assistance in completing tax forms; * Reduced burden from not having to deal with IRS notices, such as reduced time to; -open and read IRS’s notices; -decide what to do; -research tax records; -prepare response to IRS; -copy and mail response, if necessary, to IRS; -call IRS for assistance. Source: GAO. : [End of table] From the taxpayer’s perspective, benefits from testing could include avoiding the burdens associated with (1) interacting with IRS if they make mistakes due to unclear forms and instructions and (2) understanding and complying with unclear forms and instructions. From IRS’s perspective, benefits are generally in the form of opportunities to use its resources better serving other taxpayers and enforcing the tax laws.[Footnote 10] Because IRS makes many changes to forms and instructions affecting individual taxpayers every year, ranging from very simple to more complex changes, the benefits can vary according to the type of changes made. Some forms or instructions may change simply to update certain dollar thresholds based on inflation and these changes may be unlikely to be confusing or unclear to taxpayers. However, in other cases changes may introduce new requirements or concepts to taxpayers, such as when new rules are established through legislation or regulation. Changes intended to address situations like these may be more likely to be confusing or unclear to taxpayers, which could result in a burden on taxpayers to understand their obligations in preparing their tax forms and, possibly, to errors that lead to subsequent interactions with IRS to correct their returns. To the extent that a form or instruction is unclear and the lack of clarity leads to taxpayer errors, the method IRS uses to detect the errors can affect the costs IRS incurs as a consequence. If a taxpayer’s error can be detected by IRS and corrected under its “math error” procedures, which rely extensively on automated processes, the cost to IRS to correct the error is likely to be small.[Footnote 11] On the other hand, if unclear forms or instructions lead to compliance errors that are detected and addressed through audits conducted through the mail, in IRS offices, or in the taxpayer’s location, the costs to IRS are likely to be higher in part because these processes are more labor intensive. The burden and costs taxpayers might avoid if testing helps clarify forms and instructions, and thereby helps taxpayers avoid errors, can vary substantially just as IRS’s costs can vary. In general, because taxpayers need only respond if they disagree with an IRS notice stating that it has corrected an error under its math error procedures, the taxpayer’s burden and cost are likely to be lower than if IRS contacts the taxpayer as part of an audit since audits require taxpayer responses and reviews of taxpayers’ books and records.[Footnote 12] Testing Benefits Can Far Exceed IRS’s Costs: Illustrations we developed of the potential benefits and costs of testing forms and instructions show that at least in some cases benefits can be substantially greater than the costs to IRS to do tests. The benefits of testing to IRS alone can potentially exceed its testing costs in the first year a change is implemented. But, primarily because a small change in the time required of taxpayers to understand their tax obligations can total to a large aggregate benefit, taking taxpayers’ benefits into account can yield total benefits substantially above IRS’s costs. IRS officials have not attempted to develop quantitative estimates of the benefits to taxpayers and IRS that may result from testing forms and instructions and the costs IRS incurs to achieve those benefits. IRS officials did believe that because taxpayers made fewer errors when using the revised EIC and the Child Tax Credit forms and instructions as shown in table 2, IRS spent less time and money correcting errors related to them.[Footnote 13] The officials said they could not quantify the cost savings because IRS does not track error correction costs by type of error. To provide some perspective on the potential magnitude of benefits and costs that may be realized due to testing changes to forms and instructions, we analyzed the changes IRS made to EIC and Child Tax Credit forms and instructions. Our analyses are illustrations and not actual assessments of benefits and costs that were associated with testing these forms and instructions because complete data were not available on the potential benefits and costs. Further, in constructing our illustrations we sought to be conservative in estimating benefits, in part because we did not have information on the full range of costs IRS incurred to test forms and instructions. Our illustrations focus on (1) a narrow set of benefits to IRS alone due to potential reductions in taxpayer errors, (2) those benefits plus certain benefits to the taxpayers from reduced errors, and (3) potential benefits to taxpayers in reduced time to do their taxes. See appendix I for details on the methodology we used in developing our illustrations. Our first illustration quantifies a narrow set of benefits to IRS alone from testing EIC and Child Tax Credit forms and instructions--that is, the benefits IRS may have realized due to reduced numbers of errors that are handled under its math error procedures. It is likely that to the extent testing contributed to better taxpayer understanding of these two credits, IRS would have obtained other benefits. For instance, because improperly claimed qualifying children is one of the leading causes of the EIC’s high noncompliance rate,[Footnote 14] if clarified EIC forms and instructions lead fewer taxpayers to improperly claim the EIC, IRS would likely be able to free some of its EIC-related audit resources for other audits or to audit EIC returns that it might otherwise have had insufficient resources to cover. In fiscal year 2002, IRS used about 1,400 full-time equivalent (FTE) staff years[Footnote 15] for correspondence audits of EIC issues. Our analysis in table 4 shows the amount by which IRS’s potential cost savings from not having to correct EIC and Child Tax Credit errors may have exceeded its testing costs given differing assumptions about how much testing alone may have contributed to reduced taxpayer errors. As illustrated, IRS would have saved more in cost avoidance (thereby freeing resources to work elsewhere) in the first year of the change alone than it spent on the contracted testing of the forms and instructions if half of the reduction in errors was due to testing.[Footnote 16] If only 10 or 25 percent of the reduction was due to testing, then IRS would not have saved more than it spent on the testing contract in the first year. However, some of the benefits of a change in forms or instructions continue to be realized in future years. Again, the illustration does not consider other benefits IRS may have realized. Table 4: Illustration of Potential Savings to IRS Alone in 2000 from Testing EIC and Child Tax Credit Forms and Instructions: Form and instruction: EIC; Differing assumed levels of errors avoided: 50%; Assumed number of errors eliminated due to testing: 250,534; Estimated direct labor cost to correct an error[A]: $0.36; Estimated cost to correct eliminated errors: $90,000; Potential IRS cost savings assuming $56,000 cost to perform test[B]: $34,000. Differing assumed levels of errors avoided: 25%; Assumed number of errors eliminated due to testing: 125,267; Estimated direct labor cost to correct an error[A]: 0.36; Estimated cost to correct eliminated errors: 45,000; Potential IRS cost savings assuming $56,000 cost to perform test[B]: (11,000). Differing assumed levels of errors avoided: 10%; Assumed number of errors eliminated due to testing: 50,107; Estimated direct labor cost to correct an error[A]: 0.36; Estimated cost to correct eliminated errors: 18,000; Potential IRS cost savings assuming $56,000 cost to perform test[B]: (38,000). Form and instruction: Child tax credit; Differing assumed levels of errors avoided: 50%; Assumed number of errors eliminated due to testing: 248,053; Estimated direct labor cost to correct an error[A]: 0.36; Estimated cost to correct eliminated errors: 89,000; Potential IRS cost savings assuming $56,000 cost to perform test[B]: 33,000. Differing assumed levels of errors avoided: 25%; Assumed number of errors eliminated due to testing: 124,026; Estimated direct labor cost to correct an error[A]: 0.36; Estimated cost to correct eliminated errors: 45,000; Potential IRS cost savings assuming $56,000 cost to perform test[B]: (11,000). Differing assumed levels of errors avoided: 10%; Assumed number of errors eliminated due to testing: 49,611; Estimated direct labor cost to correct an error[A]: 0.36; Estimated cost to correct eliminated errors: 18,000; Potential IRS cost savings assuming $56,000 cost to perform test[B]: (38,000). Source: IRS and GAO. Note: GAO analysis of IRS data. [A] IRS officials provided (1) the number of FTEs, including labor- related overhead costs such as training and leave, for operating the Error Resolution System (ERS) and (2) the number of errors corrected by ERS during fiscal year 2002. According to IRS officials, ERS personnel are hired at GS-4 and the journeyman level is GS-6. Using the number of ERS FTEs, the number of errors corrected by ERS staff, and the GS-5, step 1 salary effective January 2002, as the salary for ERS staff members, we estimated that IRS incurred $0.36 in costs per error. (IRS officials were unable to provide the actual costs for ERS staff. We chose to use GS-5, step 1, because beginning ERS staff are GS-4s and more experienced ERS staff are GS-6s.) Total error correction costs may be higher because the $0.36 estimate does not include various other costs such as nonlabor-related overhead for equipment and supplies. [B] IRS contracted with a firm to conduct focus groups to test EIC and Child Tax Credit forms and instructions. The contract was for $54,000 and IRS spent an additional $2,000 on related travel costs. Because one contractor conducted the tests for both credits, IRS officials said it would be difficult to allocate the $56,000 between the tests for the two credits. [End of table]: To provide some perspective on how the potential benefits to taxpayers from testing EIC and Child Tax Credit forms could affect the overall benefits and costs of testing, we next looked at potential reduced burden from credit claimants receiving fewer notices due to reduced errors. First, we assumed that on average all taxpayers receiving an error notice from IRS take 2 or 5 minutes to deal with the notice. Based on those assumptions, we calculated the value to taxpayers of the time saved (using minimum wage levels) from not having to deal with IRS error notices.[Footnote 17] We used the same assumed reductions in errors due to testing that we made for table 4 and we netted taxpayers’ savings with the savings shown in table 4 for IRS alone. As table 5 shows, including testing-related benefits to taxpayers from decreased errors suggests that in the first year following testing of EIC and Child Tax Credit forms and instructions, the net benefit to taxpayers and IRS combined could have been positive except for our lowest assumption about the degree to which testing may have reduced taxpayer errors--our 10 percent assumption. Table 5: Illustration of Potential Savings to Taxpayers and IRS in 2000 from Error Reductions Alone from Testing EIC and Child Tax Credit Forms and Instructions: Form and instruction: EIC; Assumed number of errors eliminated due to testing: 250,534; Value to taxpayers in time saved (2 minutes per notice): $43,000; Value to taxpayers in time saved (5 minutes per notice): $108,000; Potential IRS cost savings assuming $56,000 cost to perform test: $34,000; Potential net savings to taxpayers and IRS from error reductions: 2 minutes: $77,000; Potential net savings to taxpayers and IRS from error reductions: 5 minutes: $142,000. Assumed number of errors eliminated due to testing: 125,267; Value to taxpayers in time saved (2 minutes per notice): 22,000; Value to taxpayers in time saved (5 minutes per notice): 54,000; Potential IRS cost savings assuming $56,000 cost to perform test: (11,000); Potential net savings to taxpayers and IRS from error reductions: 2 minutes: 11,000; Potential net savings to taxpayers and IRS from error reductions: 5 minutes: 43,000. Assumed number of errors eliminated due to testing: 50,107; Value to taxpayers in time saved (2 minutes per notice): 9,000; Value to taxpayers in time saved (5 minutes per notice): 22,000; Potential IRS cost savings assuming $56,000 cost to perform test: (38,000; Potential net savings to taxpayers and IRS from error reductions: 2 minutes: (29,000); Potential net savings to taxpayers and IRS from error reductions: 5 minutes: (16,000). Form and instruction: Child tax credit; Assumed number of errors eliminated due to testing: 248,053; Value to taxpayers in time saved (2 minutes per notice): 43,000; Value to taxpayers in time saved (5 minutes per notice): 106,000; Potential IRS cost savings assuming $56,000 cost to perform test: 33,000; Potential net savings to taxpayers and IRS from error reductions: 2 minutes: 76,000; Potential net savings to taxpayers and IRS from error reductions: 5 minutes: 139,000. Assumed number of errors eliminated due to testing: 124,026; Value to taxpayers in time saved (2 minutes per notice): 21,000; Value to taxpayers in time saved (5 minutes per notice): 53,000; Potential IRS cost savings assuming $56,000 cost to perform test: (11,000; Potential net savings to taxpayers and IRS from error reductions: 2 minutes: 10,000; Potential net savings to taxpayers and IRS from error reductions: 5 minutes: 42,000. Assumed number of errors eliminated due to testing: 49,611; Value to taxpayers in time saved (2 minutes per notice): 9,000; Value to taxpayers in time saved (5 minutes per notice): 21,000; Potential IRS cost savings assuming $56,000 cost to perform test: (38,000); Potential net savings to taxpayers and IRS from error reductions: 2 minutes: (29,000); Potential net savings to taxpayers and IRS from error reductions: 5 minutes: (17,000). Source: GAO. Note: GAO analysis. [End of table]: Finally, to illustrate the potential benefits if testing EIC and Child Tax Credit forms and instructions made them clearer and thereby reduced taxpayers’ time needed to understand and complete the credit forms, we calculated the value of time saved by taxpayers (using minimum wage levels) in understanding and completing EIC and Child Tax Credit forms assuming the time saved was 1 minute. Unlike for tables 5 and 6, all taxpayers who used the form or instructions to determine whether they qualified for either credit may have saved time if testing contributed to clearer EIC and Child Tax Credit forms and instructions. However, because we did not know how many taxpayers might have used the forms and instructions for this purpose, in calculating the value of time taxpayers may have saved we used only the number of taxpayers who claimed these credits and did not use paid preparers to prepare their tax returns. Table 6 shows that if testing the credits’ forms and instructions helped clarify them and that led taxpayers to take 1 minute less to understand and complete the forms, credit claimants would have saved a total of about 240,000 hours worth $1.2 million at minimum wage levels. Table 6: Illustration of Potential Benefit to Taxpayers in 2000 if Testing Reduced the Time Needed to Understand and Complete EIC and Child Tax Credit Forms: Form and Instruction: EIC; Number of returns claiming the credit (millions)[A]: 6.2; Total hours saved assuming 1 minute less per taxpayer: 100,000; Potential value to taxpayers of time saved (millions): $0.5. Form and Instruction: Child Tax Credit; Number of returns claiming the credit (millions)[A]: 8.4; Total hours saved assuming 1 minute less per taxpayer: 140,000; Potential value to taxpayers of time saved (millions): 0.7. Form and Instruction: Totals; Number of returns claiming the credit (millions)[A]: 14.6; Total hours saved assuming 1 minute less per taxpayer: 240,000; Potential value to taxpayers of time saved (millions): 1.2. Source: GAO. Note: GAO analysis. [A] This is the number of tax returns prepared by taxpayers claiming the credits. Since taxpayers who used a preparer might not need to read and understand the forms and instructions, we did not include them in this table. Both the number of EIC and Child Tax Credit returns are based on the percentage of returns prepared by taxpayers who claimed EIC since data on the percentage of returns prepared by taxpayers who claimed the Child Tax Credit were not available. [End of table]: To provide another perspective on the potential magnitude of benefits and costs associated with testing changes to forms and instructions, we also looked at IRS’s experience with the rate reduction credit. This one-time credit was enacted in June 2001.[Footnote 18] When 2001 tax returns were processed during 2002, over 8 million returns had errors related to the credit.[Footnote 19] IRS did not test the instructions for computing the rate reduction credit that was included on the Form 1040 for tax year 2001. According to IRS officials, they did not test the instructions because the credit was a one-time event, and in their judgment, they had insufficient time to test it. We reported that some of the taxpayers’ errors were probably due to taxpayers not understanding IRS’s instructions on how to compute the credit. We also reported that the demand for telephone assistance related to the credit was significant during the 2002 filing season, and that some of these calls, based on Taxpayer Advocate Service information, were made because taxpayers did not understand how to compute the credit.[Footnote 20] Using the same approach to illustrate whether IRS alone may have realized benefits in excess of its testing costs as we did for EIC and Child Tax Credit changes, we developed the illustration shown in table 7.[Footnote 21] As shown, considering only IRS’s cost and assuming that all errors were corrected by IRS using its math error procedures and assuming IRS would have spent the same amount to test the rate reduction credit instructions as it did for EIC and Child Tax Credit tests, IRS may have been able to save between $233,000 and $666,000. Although this case is somewhat atypical since the rate reduction credit affected essentially all individual taxpayers and the number of errors related to the credit was unusually high, these figures illustrate that the potential for savings to IRS alone from testing instructions at times can substantially exceed its testing costs. Table 7: Illustration of Potential Cost Savings to IRS in 2002 if the Rate Reduction Credit Instructions Had Been Tested and Errors Were Reduced: Number of returns with rate reduction credit errors: 8,025,851; Differing assumed levels of errors avoided: 25%; Assumed number of errors eliminated by testing: 2,006,646; Estimated direct labor cost to correct an error: $0.36; Estimated cost to correct eliminated errors: $722,000; Potential IRS cost savings assuming $56,000 cost to perform test: $666,000. Differing assumed levels of errors avoided: 15%; Assumed number of errors eliminated by testing: 1,203,878; Estimated direct labor cost to correct an error: 0.36; Estimated cost to correct eliminated errors: 433,000; Potential IRS cost savings assuming $56,000 cost to perform test: 377,000. Differing assumed levels of errors avoided: 10%; Assumed number of errors eliminated by testing: 802,585; Estimated direct labor cost to correct an error: 0.36; Estimated cost to correct eliminated errors: 289,000; Potential IRS cost savings assuming $56,000 cost to perform test: 233,000. Source: IRS and GAO. Note: GAO analysis of IRS data. [End of table]: However, just as with EIC and Child Tax Credits, taxpayers would have benefited if testing had been done, led to clearer instructions and consequently led to fewer taxpayer errors for the rate reduction credit. Using the same approach we used for EIC and Child Tax Credits, table 8 shows the potential taxpayers’ savings from dealing with fewer rate reduction credit error notices and the net savings to taxpayers and IRS. Table 8: Illustration of Potential Savings to Taxpayers and IRS in 2002 from Potential Error Reductions if Testing the Rate Reduction Credit Had Reduced Taxpayer Errors: Number of returns with rate reduction credit errors: 8,025,851; Assumed number of errors eliminated by testing: 2,006,646; Value to taxpayers of time saved (2 minutes per notice): $344, 000; Value to taxpayers of time saved (5 minutes per notice): $861,000; Potential IRS cost savings assuming $56,000 cost to perform test: $666,000; Potential net savings to taxpayers and IRS from error reductions: 2 minutes: $1,010,000; Potential net savings to taxpayers and IRS from error reductions: 5 minutes: $1,527,000. Assumed number of errors eliminated by testing: 1,203,878; Value to taxpayers of time saved (2 minutes per notice): 207,000; Value to taxpayers of time saved (5 minutes per notice): 517,000; Potential IRS cost savings assuming $56,000 cost to perform test: 377,000; Potential net savings to taxpayers and IRS from error reductions: 2 minutes: 584,000; Potential net savings to taxpayers and IRS from error reductions: 5 minutes: 894,000. Assumed number of errors eliminated by testing: Assumed number of errors eliminated by testing: 802,585; Value to taxpayers of time saved (2 minutes per notice): Value to taxpayers of time saved (2 minutes per notice): 138,000; Value to taxpayers of time saved (5 minutes per notice): Value to taxpayers of time saved (5 minutes per notice): 344,000; Potential IRS cost savings assuming $56,000 cost to perform test: Potential IRS cost savings assuming $56,000 cost to perform test: 233,000; Potential net savings to taxpayers and IRS from error reductions: 2 minutes: 2 minutes: 371,000; Potential net savings to taxpayers and IRS from error reductions: 5 minutes: 5 minutes: 577,000. Source: GAO. Note: GAO analysis. [End of table]: Table 9 shows that if testing had been done and it improved the clarity of the instructions enough to save taxpayers, on average, 30 seconds in understanding whether and how they needed to complete the credit line on their tax returns, the savings would have been larger than savings to IRS and taxpayers from avoided errors alone.[Footnote 22] Table 9: Illustration of Potential Benefit to Taxpayers in 2002 if Testing Reduced the Time Needed to Understand and Complete the Rate Reduction Credit: Number of returns affected by the credit[A]: 50,000,000; Total hours saved assuming 30 seconds less per taxpayer: 415,000; Potential value to taxpayers of time saved (millions): $2.1. Source: GAO. Note: GAO analysis. [A] This is the number of paper returns prepared by taxpayers. Since taxpayers who used a preparer might not need to read and understand the instructions, we did include them in this table. We only included the number of paper returns because information was not readily available on the number of returns filed electronically that were completed by taxpayers rather than a preparer. [End of table] IRS Has Not Addressed Constraints to Increased Testing: Although IRS officials said that making greater use of testing to improve clarity of forms and instructions could be beneficial, officials have not addressed the two constraints--time and resources-- that they state limit their ability to do more testing of changes to forms and instructions. Time constraints are not binding for some changes IRS considers to forms and instructions, although IRS cannot realistically test the unknown portion of the changes that are due to laws passed shortly before, or even after, the effective dates for the forms. Also, IRS’s procedures for developing and revising forms (1) do not clearly specify which draft version of forms and instructions should be tested with taxpayers or (2) when in the annual forms development cycle testing should occur. In addition to tight time frames, officials also say that limited resources, such as only one person responsible for coordinating all testing efforts in the Forms and Publications Division, preclude them from increasing tests of forms and instructions. However, IRS has not documented which changes to forms and instructions likely would benefit from testing or demonstrated the benefits that are gained when testing is done. IRS’s planning and budgeting process uses such information in determining the level of resources to be allocated to various units. Tight Time Frames Do Not Always Exist and IRS’s Procedures Do Not Always Facilitate Testing: IRS officials told us that when new tax laws are enacted during the year that require IRS to create or revise tax forms and instructions in time to distribute them to taxpayers by January 1, the start of the tax-filing season, they lack time to test the forms and instructions before distributing them to taxpayers. However, not all changes to forms and instructions are time constrained and IRS’s procedures lack a clear target for which version of forms and instructions should be tested with taxpayers. While sufficient data were not available to determine the portion of changes IRS makes to forms and instructions that cannot be tested due to time constraints, not all changes are time constrained. Due to the variability in the time that may be required to test a form or instruction and in the amount of time IRS needs to develop the initial form or instruction to be tested, we cannot say definitively when IRS may or may not have sufficient time to conduct tests. In some cases, IRS likely could have sufficient time to do testing when it identifies a needed change to forms or instructions itself since it largely controls the scheduling of this work. Similarly, when the Congress passes a law that is not effective until a future tax year, or that contains provisions that are not effective until a future tax year, IRS may have sufficient time to conduct tests. For example, the Economic Growth and Tax Reconciliation Act of 2001 was passed on June 7, 2001, with some provisions effective for tax year 2001, but others with later effective dates. The provisions modifying education Individual Retirement Accounts were effective for taxable years beginning after December 31, 2001. This gave IRS approximately 16 months to develop and test any modifications to tax forms and instructions and make final revisions before those forms and instructions needed to go to printing for distribution by January 2003. When a law affects the current tax year, i.e., changes how taxpayers will need to calculate their taxes in the next tax-filing season, IRS is less likely to have sufficient time to test. Even in such a case, however, the new law may be passed early enough to allow testing. IRS’s current procedures for developing and revising forms and instructions do not clearly specify which draft version of forms and instructions should be tested with taxpayers or when in the annual forms development cycle testing should occur. Officials said that draft forms may be tested with taxpayers either before or after they are posted to IRS’s website for external comments by the public, tax practitioners, software developers, and others. Tax Forms and Publications Division officials said that they consider the particular circumstances surrounding the development of each form and instruction when deciding which version of a draft form or instruction they should test. However, because IRS does not have a clear targeted time for testing, IRS’s ability to plan and conduct tests maybe constrained. If IRS’s procedures defined a point in the annual forms development cycle where a version of a draft form or instruction would be available for testing, IRS would be able to establish processes and deadlines designed to ensure that the opportunity for testing is realized. To the extent that a draft version of a form or instruction is available for testing early in the process, it would give IRS a fuller range of options for testing. For example, if IRS tested draft versions of forms and instructions before or during the approximately 3-week period that the form is available on its Web site, this would minimize any additional calendar time that testing might otherwise add to IRS’s forms development process. Figure 2 shows the points in IRS’s annual forms development process where testing can occur. As illustrated, testing may be conducted early in the process and late in the process. Figure 2: Simplified Overview of Where Testing Can Occur in IRS’s Annual Tax Forms Development Process: [See PDF for image] Note: GAO analysis of IRS data. [End of figure]: Testing earlier drafts of forms and instructions would also enable officials to select from various testing alternatives depending on how early a draft is available for testing. We did not find a uniform amount of time needed to test a change to a form or instruction. At the low end of the spectrum, an official from NCHS said that it takes about 7 weeks to test that agency’s questionnaires using one-on-one interviews. IRS officials estimated that when IRS employees are used as focus group participants it requires about 8 to 12 weeks to schedule and conduct the tests, analyze the data, and prepare a report summarizing the results. IRS officials estimated that when they contract with a private firm to conduct focus groups using private citizens, 24 to 32 weeks are required to obtain a contract, recruit participants, conduct the tests, analyze the results, and prepare a report. This time frame is based on using regular contracting processes involving developing a statement of work, soliciting bids, and selecting a contractor. Contract options exist that enable agencies to identify a firm or group of firms qualified to undertake work so that an expedited task order procedure can be used to select a firm for when needs arise. According to IRS officials, they recently entered into a multiyear contract with two vendors that will enable them to issue task orders when work is needed. Data Are Not Collected That Could Be Used by IRS Management to Determine the Proper Allocation of Resources to Support Testing: Although IRS’s Tax Forms and Publications Division officials believe current resources are insufficient to support more testing of forms and instructions, they do not have some of the information needed to determine whether to allocate additional resources. This information is not available at least in part because division guidelines and policies do not require that it be gathered. Officials said that because they have so few staff available to conduct tests and have a limited budget to contract for testing, they could not increase the number of tests they perform. According to the officials, currently only 1 of 103 persons in the division is trained in testing methods. In addition to other duties, this person coordinates the tests for the division, such as the test of EIC forms and instructions completed in 1999 by a private vendor and the test of the innocent spouse application form completed by IRS in 2002. Officials also told us some staff who are primarily responsible for creating and revising tax documents may occasionally assist in conducting tests, such as the three persons involved in testing the innocent spouse form. Officials also said the total budget for contract support for the division was $150,000 in fiscal year 2002, $185,000 in fiscal year 2001, and $130,000 in fiscal year 2000. As part of its annual planning and budgeting process, IRS management determines what resources will be needed to accomplish strategies and implement programs. IRS’s planning and budget guidance requires that each operating unit prepare a business plan that, among other things, clearly defines priorities and resource requirements. Requests in the business plan for resources must be substantiated with evidence that allocating additional resources is justified. However, the division does not systematically identify when testing would be beneficial and does not routinely demonstrate the benefits to taxpayers and IRS that have been gained from such testing. Officials do not identify which of the many changes it makes to forms and instructions each year would most likely benefit from testing. Thus, the officials cannot tell IRS management how many opportunities to improve forms and instructions may be lost due to current resource levels. Further, when tests are performed, officials do not identify, quantitatively or qualitatively, the benefits that taxpayers and IRS may have realized. One reason that IRS does not have data on forgone testing opportunities is that the division lacks formal, written guidelines and procedures for determining when testing would be beneficial. Currently, testing is an optional step in the process for developing forms. IRS’s Tax Forms and Publications officials said that they decide which forms to test based on informal guidelines and procedures and input from officials in IRS’s four operating division program offices and the Taxpayer Advocate Service. The informal guidelines and procedures call for officials to weigh, among other things, whether a form or instruction (a) affects a large number of taxpayers, (b) has a high error rate based on taxpayers’ prior use of the form, (c) is perceived as complex, and (d) will be used for several filing seasons. Also, according to IRS, the amount of time available to perform tests is factored into testing decisions. These informal guidelines do not require officials to consider in all cases whether testing would be beneficial and to document the decisions made. Accordingly, even if the informal guidelines are applied, and officials judge that some forms or instructions could benefit from testing but cannot be tested due to scarce resources, those decisions are not made systematically and documented. Further, although the factors the guidelines suggest taking into account appear to have evolved from officials’ experience and therefore should be useful, they do not consider some pertinent factors that could affect the benefits likely to be realized from testing. For instance, the guidelines suggest taking the number of affected taxpayers into account but not the likely amount of burden they would face due to unclear forms or instructions. They also do not clearly call for officials to consider the costs to test forms and instructions and the benefits that may accrue throughout IRS, such as in telephone service centers. In addition, these informal guidelines and procedures automatically exclude testing forms and instructions that will be used only one time. Also, according to IRS officials, the time frame between the passage of new tax laws and when the newly created or revised forms and instructions must be finalized may preclude some forms and instructions from being tested. Even if one-time-use forms meet other testing criteria, such as affecting a large number of taxpayers who may perceive them as complex, IRS will not consider testing them. As the rate reduction credit situation discussed earlier illustrates, such automatic exclusions may not be appropriate in all situations. IRS officials do not have information on the results achieved when forms and instructions are tested in part because the division does not have policies that require such evaluations. When IRS obtained information on the reduction in error rates following testing of EIC and Child Tax Credit forms and instructions, the studies did not include collecting other information on the benefits that may have resulted for taxpayers and for IRS. For instance, the studies did not estimate the savings IRS may have realized in its telephone and walk-in service due to increased form clarity. Capturing fuller information on the results of testing would be consistent with IRS’s strategic planning and budgeting process, which emphasizes assessing the impact of current programs to efficiently allocate resources. Further, by evaluating results of testing decisions, IRS officials would be able to determine if their testing guidelines and procedures lead to good decisions about when testing is most likely to be beneficial. They may also be able to see if the methods they use to test--for example, focus groups formed by IRS employees or one-on-one interviews with individuals--yield the most effective test results. Conclusions: IRS continually faces the daunting task of developing and revising tax forms and instructions to administer our ever-changing set of federal tax laws. Taxpayers rely on IRS for forms and instructions that are as clear and easy to understand as possible given the complexity of the tax laws and providing clear materials is a key goal of IRS’s Tax Forms and Publications Division. In attempting to meet this goal, IRS has tested an average of one set of forms and instructions each year over the last 5 years. In contrast, officials from three federal agencies that routinely collect information from the public say that testing documents for clarity before using them is their standard practice. They do so because they believe testing will ensure that their data collection documents are clear and that individuals will understand them and complete them accurately. Although it is difficult to gauge how much testing alone contributes to the clarity of tax forms and instructions, IRS officials believe testing has contributed to significant declines in taxpayer errors. Illustrations we developed based on IRS’s experience in testing forms and instructions suggest that IRS can completely recover its testing costs in the first year following testing in some circumstances and that when savings to taxpayers from more understandable forms and instructions are considered, total benefits even in the first year following tests can be several times IRS’s testing costs. Although they recognize that testing is beneficial, officials say time constraints and limited resources preclude more testing. However, IRS’s procedures do not clearly specify when draft versions of forms and instructions should be available for testing. Having a clearly defined point where testing would be performed would facilitate establishing procedures and deadlines to better ensure that testing could be done even within IRS’s annual forms update cycle. Further, IRS officials do not have information that would help IRS management to determine whether to allocate additional resources to support enhanced testing. Because IRS lacks standard written procedures for testing, officials have not documented cases where testing would likely be beneficial and have not demonstrated the benefits that are gained from testing. Recommendations for Executive Action: Because testing could potentially yield clearer and more understandable tax forms and instructions, thereby producing benefits both to taxpayers and IRS, we recommend that the Acting Commissioner of Internal Revenue take the following actions. * Develop written criteria for determining which changes to tax forms and instructions should be tested with taxpayers before publication. : * Develop official written guidance that incorporates those criteria and ensure that the guidance requires staff that develop new or revised forms and instructions to document which changes would merit testing and why. : * Clarify procedures by designating when in the annual forms development process that a draft version of forms and instructions should be available for testing with taxpayers. : * Ensure that an appropriate range of evaluations are conducted of tests that are performed to better establish the costs and benefits of performing tests and to refine IRS’s approach to testing on the basis of lessons learned. : * Use information gained from documenting when changes to forms or instructions likely would be beneficial and from evaluations of tests to reassess an appropriate level of resources to perform testing. Agency Comments and Our Evaluation: The Acting Commissioner of Internal Revenue provided written comments on a draft of this report in an April 7, 2003, letter, which is reprinted in appendix III. The Acting Commissioner agreed with our recommendations. We are encouraged that IRS plans to implement all but one of our recommendations in time for the 2004 forms development cycle. Understandably, the remaining recommendation to ensure that an appropriate range of evaluations is conducted of tests would take more time to put into practice. The Acting Commissioner also provided additional comments and observations on our draft report. The Acting Commissioner commented that the crux of our report is that we do not believe IRS has performed adequate testing on new and revised tax forms and instructions due to a lack of resources. He said that resources for testing forms and instructions have been adequate for the testing IRS wanted to perform. While not questioning whether resources were adequate for the testing IRS performed, we concluded that IRS officials do not have information needed to determine the level of resources that should be allocated to testing forms and instructions. Accordingly, we recommended that IRS systematically identify opportunities to improve forms and instructions through testing and to evaluate the costs and benefits when testing is done. Although agreeing that testing is beneficial, the Acting Commissioner also said that there are significant staff costs associated with testing that are not included in our cost analysis. We recognize that our analysis excluded staff costs and as stated in our draft report we sought to be conservative in estimating the benefits of testing, in part because we did not have information on the full range of costs IRS incurs when undertaking projects to tests forms and instructions. During the course of our work, we requested estimates of staff costs for testing but IRS was unable to provide them. Nevertheless, at least in the cases we illustrated the potential benefits of testing were so much greater than the costs that including staff costs likely would not have substantially changed the results of our illustrations. The Acting Commissioner expressed concern about whether IRS could have forms and instructions ready for the filing season if testing was done late in the forms development cycle as shown in our figure 2 depicting IRS’s process. We agree with the Acting Commissioner’s concern; however, our figure shows the various points at which testing can occur in IRS’s current processes based on interviews with IRS officials and the documentation they provided us. As IRS implements our recommendation to clarify when testing should be done, selecting a point as early as possible would help maximize the number of changes that can be tested during the annual forms update cycle. The Acting Commissioner also said that he disagreed with our conclusion that IRS’s experience with obtaining feedback on its products is limited or recent. He said IRS uses various methods to obtain customer feedback. We agree that IRS uses methods other than testing to obtain feedback on its forms and instructions. However, our report describes the potential benefits and costs of testing as a feedback method. In terms of testing, IRS has only tested five forms and instructions during July 1997 through June 2002; in our view, this is a limited number of tests that were conducted during the recent past. The Acting Commissioner also disagreed that testing would result in reduced demand for walk-in and toll-free assistance. He said that IRS lacks data to support such a conclusion and, based on its experience, new forms generate requests for assistance and error rates on them tend to be higher. We recognize that there will always be a demand for taxpayer customer assistance. However, we believe that reduced demand for assistance is a potential benefit of testing. We note, for example, that IRS officials seek input from telephone assistors when deciding which forms or instructions need to be clarified, apparently believing that clarifying the forms and instructions may help reduce calls to assistors. Finally, testing is one means for ensuring that even for new forms and instructions requests for assistance and errors made by taxpayers will be minimized. We are sending copies of this report to the Chairman and Ranking Minority Member of the House Committee on Ways and Means and its Subcommittee on Oversight; the Secretary of the Treasury; the Acting Commissioner of Internal Revenue; the Director of the Office of Management and Budget; and other interested parties. We will make copies available to others on request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov. This report was prepared under the direction of Charlie Daniel, Assistant Director. If you have any questions regarding this report, please contact him or me at (202) 512-9110. Key contributors to this report were David Alexander, Christopher Currie, Ronald La Due Lake, Anne Laffoon, Veronica Mayhand, Edward Nannenhorn, and Shellee Soliday. Michael Brostek: Director, Tax Issues: Signed by Michael Brostek: [End of section] Appendix I: Objectives, Scope, and Methodology: To determine how often IRS has used taxpayers to test the clarity of new or revised individual income tax forms and instructions, we interviewed officials in its W&I Division’s Tax Forms and Publications Division in Washington, D.C. We sought to obtain (a) an understanding of the process IRS used to develop and revise individual income tax forms and instructions and (b) gather information on the forms and instructions IRS tested using taxpayers for the 5-year period between July 1997 and June 2002. We requested information for a 5-year span to help ensure that the information collected would reflect the amount of testing usually done by IRS. Our work did not include assessing IRS’s processes for developing and revising notices or publications or assessing the clarity of any specific tax forms or instructions. To obtain insights on the benefits of testing written documents, we interviewed officials from three federal agencies in the metropolitan Washington, D.C., area that perform extensive research and data collection using private citizens. We contacted NCHS, BLS, and the Census Bureau because they have broad experience in conducting tests of the clarity of written documents, such as forms, surveys, and instructions. We also contacted a private research firm that specializes in testing forms and surveys for a variety of public and private sector clients. We also reviewed our own guidance for developing and using questionnaires.[Footnote 23]3: To determine the benefits to taxpayers and IRS of testing tax forms and instructions with taxpayers prior to their use by the public, we took several steps. First, we interviewed IRS officials including Tax Forms and Publications Division officials to obtain information on whether they perceived testing as being beneficial to taxpayers and IRS. We also interviewed IRS’s W&I Research Division officials in Indianapolis to obtain their views on whether testing tax forms and instructions benefits taxpayers and IRS. The research division officials, among other things, collect information on taxpayer errors that the Tax Forms and Publications Division uses when deciding which forms and instructions to test. These officials provided us with data on changes in error rates for EIC and Child Tax Credit forms and instructions before and after IRS revised and tested them. Second, we developed illustrations of the potential benefits and costs of IRS’s testing EIC and Child Tax Credit forms and instructions by analyzing the data on the changes in error rates obtained from IRS research officials. To construct our illustrations, we used readily available data and made certain assumptions. Data were not available on many of the potential benefits of the changes and on the full range of costs IRS incurred to conduct the tests. We developed similar illustrations of potential benefits to taxpayers and IRS if the rate reduction credit instructions had been tested. In all cases, the illustrations we developed are not actual assessments of the costs and benefits that were associated with testing forms and instructions, or that would have resulted if testing had occurred. In developing the illustrations we sought to be conservative in estimating benefits, in part because we did not have information on the full range of costs IRS incurred to test forms and instructions. Because we had to make various assumptions, our illustrations undoubtedly vary from actual costs and benefits. To determine the potential cost savings to IRS of testing these forms and instructions, we first estimated IRS’s costs to correct a taxpayer error. Our estimates of IRS’s cost to correct errors were limited to the labor cost associated with correcting errors in IRS’s math error program. IRS provided us the number of FTE staff years for operating the ERS that was used to detect and correct math errors and the number of errors corrected by ERS during fiscal year 2002. We used this information to calculate an average labor cost to correct math errors that we then applied to reductions in EIC and Child Tax Credit errors as well as to error reductions that might have resulted from testing the rate reduction credit. Total error correction costs may be higher because we did not include a number of other costs associated with correcting errors. For example, we excluded printing and postage costs for the notices sent to taxpayers. IRS notices often cover more than one issue associated with a tax return and data were not readily available to determine what portion of the postage cost might be attributable to EIC or Child Tax Credit issues alone. In addition, the estimate does not include costs such as equipment and rent. We did not specifically test the accuracy of the cost information provided; however, our audits of IRS’s annual financial statements have raised concerns regarding IRS’s ability to identify all costs associated with a given program or activity.[Footnote 24]4: Because data were not available on the extent that testing reduced errors related to EIC and Child Tax Credit forms and instructions, we assumed different percentage reduction rates in errors due to testing these forms and instructions. Our illustrations show potential savings based on assumed percentage reduction in errors attributable to testing of 50, 25, and 10 percent. Using our estimate of IRS’s costs to correct an error and the assumed number of errors eliminated by testing, we arrived at the costs that would have been incurred to correct those errors. The difference between costs to correct eliminated errors and the costs to test is the potential cost savings from testing. Cost savings due to reduced errors likely would not mean reductions to IRS’s budget. Further, savings likely would mean that IRS would provide services to other taxpayers or would pursue other compliance or tax collection activities that it would otherwise have been unable to do. To determine IRS’s costs to test forms and instructions, we used IRS’s actual contract costs for testing EIC and Child Tax Credits in 1999. IRS spent a total of about $56,000 to test both EIC and Child Tax Credit forms and instructions with focus groups. Although the cost of contracted support for testing each form and instruction individually likely would have been somewhat lower than this, we applied the total cost in each case.[Footnote 25]5 Because IRS could not provide data, we did not include the costs associated with IRS letting and managing the contract or the cost for Tax Forms and Publications Division staff to work with the contractor in conducting and managing the tests or the cost of any other IRS staff that were involved in these tests. To determine the potential benefits to taxpayers from testing EIC and Child Tax Credit forms and instructions, we calculated estimated values to taxpayers of the time saved if testing improved form and instruction clarity, thereby reducing taxpayer errors and the burden of dealing with IRS error notices. We also estimated taxpayers’ time saved if testing reduced the time needed to understand and complete tax forms and instructions. We developed similar illustrations for potential benefits to taxpayers if the rate reduction credit had been tested. Our benefit illustrations were based on a series of assumptions. For example, to estimate the value to taxpayers of time saved from not having to deal with an IRS notice, we applied the minimum wage rate that was in effect after IRS revised and tested the forms and instructions to estimates of time taxpayers might save by not having to deal with an IRS notice.[Footnote 26]6 For these illustrations we assumed, on average, that taxpayers who received an IRS error notice might spend either 2 or 5 minutes to deal with it. Data were unavailable on how much time taxpayers actually spend dealing with IRS’s notices; however, according to Taxpayer Advocate Service information, IRS’s notices are difficult for taxpayers to understand. Further, taxpayers who decide to contest an IRS notice may take time to call or write letters to IRS or to contact and work with a tax preparer. Data were not readily available to determine what portion of taxpayers who received an EIC or Child Tax Credit error notice contested IRS’s change to their tax returns. For our illustrations of the estimated value of time saved by taxpayers if testing reduced the time needed to understand and complete EIC and Child Tax Credit forms and instructions, we assumed that taxpayers would save on average 1 minute by using clearer forms and instructions. We excluded from our illustrations those tax returns prepared by preparers because the taxpayers might not have had to read and understand the forms and instructions. Because data were not available on the number of tax returns claiming the Child Tax Credit that were prepared by paid preparers, we reduced the number of returns claiming the credit by the same percentage of EIC returns that were prepared by preparers. The percentage of EIC claimants using paid preparers exceeds the average for all taxpayers. For the rate reduction credit, our illustration is based on a 30-second time savings and the number of taxpayers who filed on paper and did not use preparers. We chose a 30- second potential savings for this illustration because many taxpayers would have had to read only part of the instructions to determine what to do. We aggregated the times for all taxpayers and multiplied the total hours saved by the prevailing minimum wage rate to arrive at estimated benefits to taxpayers. To determine whether any factors limited IRS’s ability to use individual taxpayers to test forms and instructions and, if so, how these factors can be addressed, we interviewed IRS’s Tax Forms and Publications Division’s officials and analyzed supporting data they provided us. Regarding officials’ view that they lacked sufficient time to do more testing, we reviewed information on the amount of time IRS, NCHS, and the private research firm we contacted took to perform various types of tests. We also reviewed IRS’s process for developing new and revised forms and instructions and determined how many weeks were available between the dates that various laws were enacted or their provisions became effective and IRS’s normal October 1st deadline for printing. Finally, regarding officials’ view that they lacked sufficient resources to do more testing, we obtained information on the resources available within IRS for testing. We performed our work from May 2002 through March 2003 in accordance with generally accepted government auditing standards. [End of section] Appendix II: Use of Testing in NCHS, Census, and BLS: Like IRS, the three federal agencies we contacted create written documents to be completed by the public. The agencies create documents such as forms, surveys, and questionnaires that they use to collect information from the public to fulfill their missions. Unlike IRS, described below, these agencies routinely test their forms, surveys, and questionnaires prior to distribution to the public. * The National Center for Health Statistics, the nation’s principal health statistics agency, compiles statistical information to guide actions and policies relevant to public health and health policy. According to NCHS, obtaining accurate and usable health information is crucial to successfully fulfilling its mission to provide reliable information to the Centers for Disease Control and Prevention and the Department of Health and Human Services. NCHS collects information through various sources including questionnaires that it develops and administers. Researchers at NCHS told us that in support of their research they administer surveys and questionnaires each year in addition to developing questionnaires used by the Centers for Disease Control and Prevention. They test each questionnaire using one-on-one interviews. When documents pertain to a particular rather than a general population, the researchers recruit participants with characteristics similar to those persons who might be completing the forms or questionnaires. For example, researchers recruited asthmatics to test a questionnaire related to asthma. NCHS tests forms in one-on- one settings in which a participant may be asked to work through a form while a moderator observes and then later interviews the participant. This approach allows the researcher to identify specific points at which the forms were confusing or problematic and learn why the participant had difficulty. NCHS prefers to use one-on-one interviews when conducting tests because this method closely resembles the ways in which individuals will be completing the documents since individuals will likely complete the documents by themselves. : * The Census Bureau is the principal agency responsible for collecting and providing data about the people and the economy of the United States. An accurate census is important because census results are used to reapportion seats in the House of Representatives, redraw congressional districts and other political boundaries, and address countless other public and private data needs. The Census Bureau collects information through short-form and long-form questionnaires that it develops, tests, and administers. In preparation for the 2000 Census, the Congress budgeted millions of dollars to develop and test questionnaires during the 1990s. The Census Bureau’s policy requires that demographic survey questionnaires be tested. It has used focus groups and one-on-one interviews to test its questionnaires and forms. For example, in fiscal year 1996, the Census Bureau decided to make fundamental changes to the traditional census design such as shortening census questionnaires. In that year, it budgeted funds to test, among other things, respondents’ understanding of race and ethnicity questions. The Census Bureau has also conducted detailed cost-benefit analyses of alternative designs; in 1992 it tested the simplified questionnaire in order to gauge whether the new form would increase response rates and reduce costly follow up with households that did not respond to the census.[Footnote 27]7 : * The Bureau of Labor Statistics, the principal fact-finding agency for the federal government in the broad field of labor economics and statistics, also depends on clear and understandable written documents to collect accurate information from the public. According to BLS’s policy, testing documents such as forms and surveys prior to use by the general public should be undertaken to help identify factors that may impede users’ ability to understand forms or surveys. Then these factors can be addressed in order to improve the clarity of written documents and increase the accuracy of responses. Testing should be done in the early stages of document development so that any problems with clarity can be identified early. BLS routinely tests its written documents using focus groups and one-on-one interviews, and uses the results of the tests to make improvements to the documents. : [End of section] Appendix III: Comments from the Internal Revenue Service: DEPARTMENT OF THE TREASURY INTERNAL REVENUE SERVICE WASHINGTON, D.C. 20224: COMMISSIONER: April 7, 2003: Mr. Michael Brostek Director, Strategic Issues United States General Accounting Office 441 G Street, N.W. Washington, D.C. 20548: Dear Mr. Brostek: We appreciate the opportunity to review and provide comments on the draft report titled “Tax Administration: IRS Should Reassess the Level of Resources for Testing Forms and Instructions (GAO-03-486).” We value the input of the General Accounting Office on our tax forms and publication development process. As you know, providing clear and understandable tax forms and instructions is a goal the IRS has always pursued to help simplify taxpayers’ tax-related obligations and reduce taxpayer burden in complying with the tax laws. Along with the general comments we provided on the report to your staff, we would like to offer the following observations: * We appreciate that changes were made to the draft report as a result of preliminary comments we provided. The crux of the report is that GAO does not believe IRS has performed adequate testing on new and revised tax forms and instructions due to lack of resources. We would like for the record to reflect the resources for testing forms and instructions have been adequate for the testing we wanted to do. However, as stated in our response to the recommendations, we agree to establish a means of documenting how decisions are made about testing. We also agree that developing a cost-benefit analysis to assess the results of testing will help address this issue. * We fully agree it is beneficial to test certain new or redesigned products. However, there are significant costs in staff resources (including training, travel, salaries, etc.) for creating or redesigning the tested product that are not included by GAO in the cost analysis. In the future, we will assess possible cost analysis methods that take all of the associated factors into account when designing new products. We agree with GAO that other issues outside of our control, such as a taxpayer’s poor math skills, will continue to cause errors. These all contribute to the difficulty in developing a cost-benefit analysis on the value of any testing. * We currently use a number of sources to identify materials to be tested. In conjunction with Wage and Investment Research Group 3, Tax Forms and Publications can identify the products on which the greatest number of errors occur and those impacting the largest number of taxpayers. These criteria are of great value in selecting materials to be redesigned and tested. There are situations when late legislation severely limits our time to develop materials for public use. The rate reduction credit in 2001 is a prime example. However, these situations are relatively uncommon. * We recognize the benefits of obtaining feedback from customers on our products and strive to obtain feedback through a variety of methods in addition to conducting focus groups. We agree that we conducted five tests of forms and instructions in focus groups, and one-on-one interviews between July 1997 and June 2002. We also obtain customer feedback on our products (forms, instructions, and publications) in other ways as well. We conduct town hall meetings with the public, participate in annual Tax Forums, solicit e-mail suggestions from taxpayers and IRS employees, and regularly meet with professional groups such as the American Payroll Association, and the American Institute of Certified Public Accountants. We use feedback from these sources to make many changes to improve the clarity and understandability of our products. Our use of these tools to obtain customer feedback dates back to the late 1970’s. Therefore, we would like to go on record as disagreeing with GAO’s conclusion that IRS’ experience with obtaining feedback on its products is limited or recent. The report reflects GAO’s conclusion that testing a product will result in reduced demand for IRS’ walk-in or toll-free telephone assistance. Since we currently do not have the data to support such a conclusion, we disagree with it. While testing a product may indicate where problems may occur, it cannot predict the number of contacts that will actually result. Our experience has shown that a new product is likely to generate requests for assistance because taxpayers are unfamiliar with it. Similarly, error rates on new forms tend to be higher the first year a product is issued. We still are concerned with the box in Figure 2 for considering a test as changes are made to finalize a form. The amount of time in that stage of the development process is quite limited. We feel it would not be wise to conduct testing at the “Make changes and finalize form” stage of development. It is too late in the forms development process to test the form, review the suggested changes, and then incorporate them in the final product. The printing and distribution of the form in time for the filing season would be compromised. My responses to your recommendations for Executive Action are enclosed. If you need any additional information, please let me know or contact Floyd Williams, Director, Office of Legislative Affairs at 202-622- 3720. Sincerely, Bob Wenzel: Acting Commissioner: Signed by Bob Wenzel: Enclosure: Enclosure: Recommendation: Develop written criteria to be used in determining which changes to tax forms and instructions should be tested with taxpayers before publication. Response: We agree with the recommendation and will develop written criteria for determining which changes to tax forms and instructions should be tested. These written criteria will be in place for the 2004 development cycle. Recommendation: Develop official written guidance that incorporates those criteria and ensure that the guidance requires staff that develop new or revised forms and instructions to document which changes would merit testing and why. Response: We agree with the recommendation and will provide official written guidance for using criteria when developing new and revised forms and instructions so that documentation is available for decisions made. The written guidance will be in place for the 2004 development cycle. Recommendation: Clarify procedures by designating when in the annual forms development process that a draft version of forms and instructions should be available for testing with taxpayers. Response: We agree with the recommendation and are revising our operating procedures to designate when in the annual forms development process a draft version of forms and instructions should be available for testing. The revised operating procedures will be in place for the 2004 development cycle. Recommendation: Ensure that an appropriate range of evaluations are conducted of tests that are performed to better establish the costs and benefits of performing tests and to refine IRS’ approach to testing on the basis of lessons learned. Response: We agree with the recommendation and will proceed with plans to establish a means for cost-benefit analyses to be performed to assess, to the extent possible, the benefits from testing tax forms and instructions. This will require a major project to assess the reduction in errors and the impact on processing, customer assistance, compliance activities, etc., to isolate the benefits from testing tax products from other assessment: tools that create benefits (e.g., practitioner feedback). The scope of the recommendation will most likely require a phased-in approach to implement the processes that ultimately are agreed to by all affected organizations within the IRS. We anticipate that analyses can begin during 2004. Recommendation: Use information gained from documenting when changes to forms or instructions likely would be beneficial and from evaluations of tests that are performed to reassess an appropriate level of resources to perform testing. Response: We agree with the recommendation. Once the written criteria, official written guidance, and revised operating procedures are implemented, we will use the information gathered from testing to reassess the level of resources available to perform testing. We plan to implement this recommendation starting in 2004. [End of section] FOOTNOTES [1] Tax forms and instructions as referred to in this report also include related schedules and worksheets. [2] 1993 IRS Customer Satisfaction Survey: Public Perceptions of IRS Service Quality, Contract number TIR 90-0002 (Aug. 27,1993). [3] U.S. General Accounting Office, Tax Administration: Advance Tax Refund Program Was a Major Accomplishment, but Not Problem Free, GAO- 02-827 (Washington, D.C.: Aug. 2, 2002). [4] The Process of Developing Tax Forms for Individual Taxpayers Should be Further Improved, TIGTA 2000-40-060 (March 2000); U.S. General Accounting Office, Tax Administration: IRS Efforts to Improve Forms and Publications, GAO/GGD-95-34 (Washington, D.C.: Dec. 7, 1994). [5] Because our review was limited to forms and instructions used by individual taxpayers, table 1 includes only tests IRS did on these documents during this period. IRS conducted some other tests during this period. For example, IRS used focus groups to test 5 publications. Of the 5 publications, 3 related to individual taxpayers, 1 to business taxpayers, and one related to both individual and business taxpayers. IRS estimates that it will revise over 90 publications during 2003. [6] U.S. General Accounting Office, Developing and Using Questionnaires, GAO/PEMD-10.1.7 (Washington, D.C.: October 1993). [7] Floyd J Fowler, 2002; “Survey Research Methods”; Third Edition; Sage Publications, Applied Social Research Methods Series, volume 1 (2002). Also see R. Tourangeau, L.J. Rips, and K. Rasinski, The Psychology of Survey Response (2000). [8] As noted in table 1, IRS tested five forms during July 1997 through June 2002. IRS did not determine the effect of testing on error rates after testing the EIC Schedule in 1998 according to IRS Research officials. IRS tested Schedule D in 2001. Any change in error rates would be based on the 2002 filing season, and IRS has not yet analyzed 2002 figures. IRS tested the Innocent Spouse form in 2002 and any change in error rates will not occur until the 2003 filing season. [9] Most taxpayers eligible for the EIC have a qualifying child. [10] Generally, because IRS has more demand for assistance from taxpayers than it can handle and identifies more potentially noncompliant tax returns than it can address, benefits from testing are unlikely to include reductions in IRS’s overall budget. [11] In the math error program, IRS uses computer comparisons and calculations to correct clerical and mathematical errors on tax returns. These corrections involve addition and subtraction errors; incorrect social security numbers, filing status and exemptions; and missing schedules or forms. [12] Although in many cases taxpayers would not need to respond to a math error notice, in some cases, such as when IRS has recalculated EIC or other benefits due to its detection of an apparently invalid social security number, the taxpayer may be able to provide IRS additional information to substantiate the original return. [13] Error correction costs relate to the error resolution process. When a taxpayer makes an error, in some cases, IRS will correct the error. For example, if a taxpayer transposes information from the Schedule D to an incorrect line on the form 1040, IRS can correct this mistake for the taxpayer. IRS cannot correct certain errors made by taxpayers such as claiming EIC for a child whose social security number on the tax return does not match Social Security Administration records. In this case, because it cannot determine the correct social security number for the child, IRS will disallow the claim and send an error notice to the taxpayer that explains what IRS did and what the taxpayer can do if he/she disagrees with IRS’s actions. [14] Department of the Treasury, Internal Revenue Service: Compliance Estimates for Earned Income Tax Credit Claimed on 1999 Returns (Washington, D.C.: Feb. 28, 2002). [15] An FTE consists of one or more employees who collectively work 1 year. For example, one full-time employee or two half-time employees equal one FTE. [16] Because IRS officials attributed reduced errors both to their new approach to structuring EIC guidance and to improvements to the new approach that resulted from testing, we chose to illustrate reductions due to testing alone that accounted for half or less of the decline in errors. [17] We used minimum wage to value taxpayers’ time to be conservative in estimating potential benefits because EIC is targeted to low-income taxpayers. Minimum wage is a conservative estimate of taxpayers’ time even for EIC in that some taxpayers can qualify for EIC at incomes that are three times minimum wage. For purposes of the Paperwork Reduction Act, OMB values time for dealing with paperwork requirements like tax forms at $30 per hour. [18] Taxpayers who were eligible to receive an advance tax refund in 2001 but who (1) did not receive a check because IRS did not have their current addresses or (2) did not have enough taxable income in 2000 to qualify for the maximum amount allowable, may have been entitled to a rate reduction credit when filing their tax year 2001 returns. In addition, taxpayers who were not eligible for an advance tax refund, such as those who did not have taxable income in 2000, may have been entitled to a rate reduction credit provided they had taxable income in 2001. [19] This total only includes rate reduction credit errors corrected by IRS under its math error authority as of September 27, 2002. Some electronically filed tax returns that IRS rejected during screening for processing also had rate reduction credit errors. [20] GAO-02-827. [21] Consistent with our effort to be conservative in estimating testing benefits, we assumed that testing would have reduced errors by no more than 25 percent because many taxpayers only needed to review a sentence or two of the instructions and did not need to do any calculations. [22] We assumed an average taxpayer time savings of 30 seconds because many taxpayers only needed to review a sentence or two of the instructions and did not need to do any calculations. [23] [3] GAO/PEMD-10.1.7. [24] [4] U.S. General Accounting Office, Major Management Challenges and Program Risks: Department of the Treasury, GAO-03-109 (Washington, D.C.: January 2003). [25] [5] IRS contracted with a firm to conduct focus groups to test EIC and Child Tax Credit forms and instructions. The contract was for $54,000 and IRS spent an additional $2,000 on related travel costs. Because one contractor conducted both tests, IRS officials said it would be difficult to allocate the $56,000 between the two credits. [26] [6] Our estimates of the value to taxpayers of time saved from using clearer forms and instructions are conservative estimates. The minimum wage rate of $5.15 per hour that we used in our illustrations is substantially less than OMB’s estimate of $30 per hour as the average value of time for individuals and entities to provide information to the government. Since EIC claimants are lower-income taxpayers, we used the minimum wage rate. However, some taxpayers would qualify for EIC even at three times the income of a full time minimum wage level employee. [27] [7] U.S. General Accounting Office, Census Reform: Questionnaire Test Shows Simplification Holds Promise, GAO/T-GGD-92-59 (Washington, D.C.: July 1, 1992) and U.S. General Accounting Office, Decennial Census: 1990 Results Show Need for Fundamental Reform, GAO/GGD-92-94 (Washington, D.C.: June 9, 1992) address recommendations that the Census Bureau simplify its questionnaire in order to improve its response rate and thereby reduce the number of follow-ups with nonrespondents and costs associated with follow-up activities. GAO’s Mission: The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily E-mail alert for newly released products” under the GAO Reports heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: