Tax Administration: IRS Needs to Further Refine Its Tax Filing Season Performance Measures

During their fiscal year 2003 Leadership Conference, IRS managers reviewed the agencywide and operating division goals and measures, ensured there was alignment among them, and documented their relationship--beginning in the fiscal year 2005 Strategy and Program Plan. According to the IRS, the Strategy and Program Plan was shared throughout the organization to ensure communication at all levels. By implementing this recommendation, the IRS has created a bridge between long-term strategic goals and short-term daily operational goals, which can, among other things, be used for holding the IRS and the key filing season program areas accountable for achieving its goals. Further, the link to specific measures provides additional information needed to clearly communicate the alignment of goals and measures throughout the agency, as well as the level of performance the operating division hopes to achieve.

In its 2003 report, GAO recommended that the IRS make the name and definition of several field assistance measures (i.e., "return preparation contacts," "return preparation units," "TACS total contact," "form contacts," "tax law contacts," "other contacts," "tax law accuracy," "accounts/notices accuracy," and "return preparation accuracy") clearer to indicate what is and is not included in the formula. Field Assistance implemented GAO's recommendation by updating its "data dictionary," which is used in reporting measures to all levels of the organization and specifically identifies what is or is not included in the formulas. In addition, the data dictionary now includes the following categories: operating division, measure name, type of measure, program category, related strategic goal, definition, reporting level(s), reporting data source, reports, formula/methodology, data source/measurement tools, reliability of data, frequency of data availability/reporting, purpose of measure, data limitations, calculation changes, complete description of the process, critical path, and management controls for items on critical path. The measures also include alternate sites in the definition of a Taxpayer Assistance Center, such as a mobile unit. These modifications are substantial and provide significant additional information about the measures. As a result, the measures have clarity and can be meaningful to stakeholders.

Initially, IRS disagreed with this recommendation, stating that the formula accurately reflected the percentage of customers that wanted to speak to a Customer Service Representative (CSR) and subsequently received service. However, automated services have increased as an alternative to CSR services and taxpayers have selected automated services before entering a CSR queue. Thus, while automated calls answered remains part of the CSR Level of Service formula, the number of these calls included in the computation is minimal, if any, according to a December 2006 interview with IRS officials. As a result, the measure's clarity has improved because the name is more closely aligned to the measure's definition since the number of automated calls is minimal. Also, transparency to users and stakeholders has improved, leading to fewer misinterpretations of results or failures to take proper action to resolve performance problems.

IRS initially disagreed with this recommendation because, according to officials, the Customer Service Representative (CSR) Response Level measure was in accordance with industry standards. However, IRS took alternative corrective action by eliminating the measure from Strategy and Program plan and limited its use to an internal diagnostic tool. IRS officials confirmed this change in a December 2006 interview stating that they have no plans to put it back in the Strategy and Program Plan. By removing the measure from this important document, which is presented or available to external stakeholders such as the Congress, IRS has effectively limited external reliance or possible misinterpretation of this information, while still making it available to IRS.

IRS agreed with our recommendation, but instead of revising the Customer Service Representative (CSR) Services Provided measure, it took alternative corrective action by making scripting changes that have reduced transfer rates between fiscal years 2002 and 2006 by an average of almost 10 percent. IRS also better explained how transferred calls were counted in its fiscal year 2005 data dictionary. As a result, the measure is clearer and IRS has been able to reduce inefficiencies associated with transferred calls.

IRS has implemented two new indicators to gauge Website traffic: "Number of visits" shows Website access by counting as "visits" a series of actions that begins when a visitor views his/her first page on IRS.gov and ends when the visitor leaves the site or remains idle beyond 30 minutes, while "Number of page views" counts whenever any given web page is accessed by a visitor to IRS.gov. Information on Website hits is still collected, but is used by IRS only for load balancing of servers.

While IRS initially agreed that there was a need to revise this measure, the Service eventually eliminated the measure under IRS's Taxpayer Assistance Center Closure Project. The FY 2006-2007 Wage and Investment Strategy and Program Plan and 2006 data dictionary do not contain this measure. By eliminating the measure from these important external documents, IRS has effectively addressed GAO's underlying concern that it could be providing misleading information about agency performance to members of Congress or other external stakeholders.

IRS agreed that the Submission Processing productivity measure needed improvement. While IRS has not revised the measure as GAO specifically recommended, Submission Processing officials reported that they have taken steps to address the underlying problems with this measure, which were that field managers did not understand it and therefore did not use it. They did this by holding multiple productivity meetings and conference calls with managers during fiscal year 2006. As a result, field managers have a better understanding of the measure and can better use it to help alert them to problems and/or respond when problems arise.

The IRS concurred with GAO's recommendation that it should only make changes to official targets under extenuating circumstances. Further it agreed that disclosing such changes is appropriate.

GAO found that the customer satisfaction survey, which is the basis of the telephone customer service measure, was not administered properly. Specifically, potential bias existed because the survey administrators did not follow the sampling procedures, thus distorting the sample's randomness and the overall objectivity of the measure. The IRS has taken corrective action by (1) instructing all administering sites to use the sampling procedures and (2) revising the procedures to make them easier to use. Further, GAO recommended that the IRS modify its procedures for the toll-free customer satisfaction survey, possibly by requiring that administrators listen to the entire call, to better ensure that administrators (1) notify Customer Service Representatives that their call was selected for the survey as close to the end as possible and (2) can accurately answer the questions they are responsible for on the survey. In August 2003, the IRS notified GAO that it had instructed the administrators to listen to each call from its beginning to as close the conclusion as practical. They concurred that formalizing this practice would enable the administrators to accurately answer the survey questions.

In 2003, IRS revised four of its five external quality measures for telephone assistance into two measures: "customer accuracy for toll-free tax law" and "customer accuracy for toll-free accounts." The fifth measure, a timeliness measure, was changed to an internal measure. At the time of these changes, the Service also put in place numerous checks and controls for the revised measures to ensure the accuracy of the data collected. Controls include daily and weekly validity checks, a "rebuttal system whereby call sites can challenge the review and coding of individual calls, and unique consistency reviews. IRS has, therefore, addressed GAO's underlying concern behind this recommendation, that there could be problems with the reliability of the data collected to report on the measures.

In fiscal year 2003, IRS officials reviewed all measures to verify that they were relevant and specific. Of the 10 overlapping measures GAO identified, four were removed from the Strategy and Program Plan, including "return preparation units," "automated completion rate," " toll-free tax law correct response rate," and "toll-free account correct response rate." Based on IRS management's judgment, measures that provided basically the same information as another measure were removed. Consequently, managers no longer have to sort through redundant, costly information that might not add value or cloud the bottom line.

IRS responded that the development of cost of service measures was dependent on the Servicewide deployment of the Integrated Financial System (IFS). In 2007, IRS was still developing cost estimates for the Chief Financial Officer's IFS. However, as part of its 2007 Taxpayer Assistance Blueprint, IRS prepared unit cost per contact estimates based on fiscal year 2005 data for the major IRS taxpayer services and channels. While IRS reported that these estimates do not fully allocate all indirect overhead and support costs such as headquarters staffing, information technology modernization projects, and centralized real estate facilities procurement and employee services, they represent the best and most comprehensive estimates currently available. As a result of these initial steps, IRS has better achieved governmentwide priorities and balance of its measures.

Although IRS did not put the measure back into the Strategy and Program Plan (SPP), it made enhancements to TeleTax, IRS's automated telephone information system, which resulted in the ability to capture the number of automated calls that were completed. IRS tracks this enhancement as part of its Automated Calls Answered measure, which is in the SPP. The fiscal year 2005 through 2006 SPP confirms that this enhancement was effective. As a result, taxpayer behavior is tracked and reported in an important document available to Congress and external stakeholders, which better gauges the automated service IRS provides to taxpayers.

By the 2006 tax filing season, IRS had a system in place for monitoring and managing timeliness of customer service at walk-in sites, which provided coverage for 81 percent of all walk-in site traffic. Managers at sites equipped with the "full" Q-Matic system can monitor traffic in real-time, specifically wait-times. With this additional information and capability, IRS is better able to balance customers' need for accurate information against customers' needs for timely service at walk-in sites. IRS has, therefore, addressed GAO's underlying concern behind this recommendation, that field assistance could be emphasizing customers' need for accurate service at the expense of their need for timely service.

IRS implemented a diagnostic tool to monitor Field Assistance's efficiency. We still believe that an organization performance measure that would be visible throughout IRS and to Treasury and congressional stakeholders would be preferable. However, IRS officials confirmed in September 2007 that they find the efficiency diagnostic tool useful and that they monitor frequently. As a result, we agree that IRS's implementation of an efficiency diagnostic tool is a good step for direct mangers to monitor Field Assistance's productivity and to use to identify its strengths, weakness, and areas for improvement.