This is the accessible text file for GAO report number GAO-10-963 
entitled 'Tax Debt Collection: IRS Could Improve Future Studies by 
Establishing Appropriate Guidance' which was released on October 26, 2010. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Ranking Member, Committee on Finance, U.S. Senate: 

United States Government Accountability Office:
GAO: 

September 2010: 

Tax Debt Collection: 

IRS Could Improve Future Studies by Establishing Appropriate Guidance: 

GAO-10-963: 

GAO Highlights: 

Highlights of GAO-10-963, a report to the Ranking Member, Committee on 
Finance, U.S. Senate. 

Why GAO Did This Study: 

In September 2006, the Internal Revenue Service (IRS) started the 
private debt collection (PDC) program for using private collection 
agencies (PCA) to help collect some unpaid tax debts. Aware of 
concerns that PCAs might cost more than using IRS staff, IRS began 
studying the collection costs and performance of PCAs and IRS. In 
March 2009, IRS announced that it would not renew its PCA contracts 
based on the study and announced plans for increasing collection 
staffing. As requested, GAO is reporting on whether (1) the study was 
sound as primary support for IRS’s PDC decision and (2) IRS has 
planned or made changes to its collection approach based on its PCA 
experience and PDC study. GAO compared IRS’s study to federal and 
other guidance on what should be included in analyses to support 
program decisions and analyzed IRS’s changes given expectations that 
IRS would consider PCAs’ best practices. 

What GAO Found: 

IRS’s comparative study of the PDC program was not soundly designed to 
support its decision on whether to continue contracting out debt 
collection. Although the study was not originally intended or designed 
as primary support for the decision, IRS officials nonetheless used it 
as such. IRS did not have guidance for program managers on the type of 
analysis that should be done to support decisions to create, renew, or 
expand programs. IRS had not retained sufficient documentation on the 
sample used in the study or documented some analyses that would have 
been helpful if performed. The study results may be overstated or 
understated because the study sample was not generalizable to the 
program as a whole. The study had a narrow objective of comparing 
results for IRS working the same cases as PCAs had, and as a result, 
the study design did not consider other factors recommended by Office 
of Management and Budget and other guidance on conducting program 
analysis. For example, the study did not analyze alternatives to 
program scale, such as expanding it or scaling it back. Program 
analysis guidance states that to the extent possible all costs and 
benefits should be counted and alternative means of achieving a program’
s goals should be considered. But the study did not identify important 
costs and benefits, such as whether taxpayers’ compliance costs would 
be different if IRS or PCAs work debt cases. Nevertheless, neither GAO 
nor IRS officials know whether the study results and decision on the 
program would have differed significantly if it had been designed to 
be primary support for IRS’s PDC program. In commenting on a draft of 
GAO’s report, IRS disagreed that the PDC study was not soundly 
designed. GAO stands by its analysis detailing the study’s errors, 
narrow scope, and lack of adherence to guidance. These design and 
methodology deficiencies limited the study’s usefulness in supporting 
IRS’s decision. 

IRS has not made or planned changes to its collection approach based 
on its PCA experience and study. In authorizing the use of PCAs, 
Congress required IRS to report to Congress its measurement plan to 
identify any of the PCAs’ best practices that IRS could adopt to 
improve its own collection operations. IRS did not continue to report 
to Congress as required. In an unpublished draft report, IRS asserted 
that it had reviewed a number of PCA practices and found no immediate 
opportunities to change its collection approach. IRS did not provide 
GAO documentation on the study to support that conclusion. In part 
because PCA-type cases had previously been considered low priority, 
IRS officials were surprised by the PDC study results, which indicated 
that IRS staff might have better results working PCA-type cases than 
some of the cases IRS normally works. IRS officials said that they 
initiated a pilot study in 2009 to help them decide whether to use IRS 
staff to work selected types of PCA cases. As GAO concluded its 
review, IRS provided conflicting information on the role of the pilot 
study. On one hand, IRS said a collection selection system to be 
implemented in January 2011 overtook the need for the study. On the 
other hand, an IRS official said that the results from PCA-type cases 
were not used in the development of the new case selection system. 

What GAO Recommends: 

GAO recommends that IRS (1) establish guidance on analyses to support 
program decisions, (2) establish a policy requiring documentation of 
program studies, and (3) ensure that PCA-type case results are 
considered for IRS’s new case selection model. IRS agreed with the 
first two recommendations and agreed in principle with the third, 
which GAO revised to reflect updated information that IRS provided. 

View [hyperlink, http://www.gao.gov/products/GAO-10-963] or key 
components. For more information, contact Michael Brostek at (202) 512-
9110 or brostekm@gao.gov. 

[End of section] 

Contents: 

Letter: 

Scope and Methodology: 

Background: 

Methodological Errors and Narrow Scope Limit the Study's Usefulness in 
Supporting the PDC Decision: 

IRS Has Not Changed Its Collection Approach Based on Its PDC 
Experience and Study, and Whether IRS Will Work Certain PCA-Type Cases 
Is Unclear: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Comments from the Internal Revenue Service: 

Appendix II: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: IRS's PDC Study Results Comparing IRS and PCAs Working 
Collection Inventory from the Same Pool of Cases Assigned to PCAs: 

Table 2: Summary of the Types of Costs Included in IRS's PDC Study: 

Abbreviations: 

ACS: Automated Collection System: 

CDA: Consolidated Decision Analytics: 

IRM: Internal Revenue Manual: 

IRS: Internal Revenue Service: 

OMB: Office of Management and Budget: 

OPERA: Office of Program Evaluation and Risk Analysis: 

PCA: private collection agency: 

PDC: private debt collection: 

[End of section] 

United States Government Accountability Office: 

Washington, DC 20548: 

September 24, 2010: 

The Honorable Charles E. Grassley: 
Ranking Member: 
Committee on Finance: 
United States Senate: 

Dear Senator Grassley: 

In October 2004, Congress authorized the Internal Revenue Service 
(IRS) to create the private debt collection (PDC) program to collect 
some portion of the tax debt inventory. The program enabled IRS to 
collect tax debts by contracting with private collection agencies 
(PCA) and paying them out of a revolving fund of the revenue 
collected.[Footnote 1] Some in Congress and elsewhere in the tax 
community, including the National Taxpayer Advocate, expressed 
concerns that using PCAs might result in higher costs as compared to 
increasing IRS resources to collect the debts.[Footnote 2] Aware of 
these concerns, IRS officials said that if Congress authorized the use 
of PCAs, IRS would study the comparative performance of PCAs and IRS 
in collecting unpaid taxes. The law authorizing the program also 
required that IRS create a measurement plan to identify and capture 
information on the best practices used by PCAs compared with IRS's 
practices to identify any that IRS could adopt to improve its own 
collection operations.[Footnote 3] IRS began assigning cases to PCAs 
and began the PDC study in September 2006. 

In March 2009, IRS released the PDC study and announced that IRS would 
not renew expiring contracts with PCAs. The announcement stated that 
IRS had reviewed the PDC program and specifically cited the results of 
the study, an independent review of it, and its conclusion that IRS is 
more cost effective than PCAs when working similar inventory.[Footnote 
4] The announcement also noted that IRS anticipated hiring additional 
collection staff in fiscal year 2009 and quoted the Commissioner of 
Internal Revenue as saying, "I believe this work is best done by IRS 
employees, and I believe we have strong support from the 
Administration and the Congress for increased IRS enforcement 
resources going forward." 

You asked us to review the soundness of IRS's PDC study and address 
questions related to changes IRS made since it stopped using PCAs. 
Specifically, our objectives were to determine the following: 

* How sound was IRS's PDC study as primary support for IRS's decision 
to discontinue contracting out tax debt collection? 

* What changes has IRS planned or made to its collection approach 
based on its PCA experience and the PDC study? 

Scope and Methodology: 

To determine the soundness of IRS's PDC study as primary support for 
IRS's decision to discontinue contracting out tax debt collection, we 
reviewed the study report and supporting documents and other data. We 
interviewed IRS officials and contractors involved in the study. We 
also reviewed the report IRS commissioned to validate the PDC study 
conclusion and interviewed contractor staff involved in that effort. 
[Footnote 5] We also interviewed officials from one of the PCA firms 
IRS contracted with to obtain their views on the PDC study.[Footnote 
6] We reviewed and summarized program analysis guidance from various 
sources in developing our criteria. Such guidance came from Office of 
Management and Budget (OMB) Circular A-94,[Footnote 7] a previous GAO 
publication on evaluating federal programs,[Footnote 8] an academic 
research paper on costs and benefits that should be considered in 
making decisions on resources for tax enforcement programs,[Footnote 
9] and accepted quantitative analysis criteria on sampling cases and 
projecting results. We compared IRS's study methodology and report to 
the criteria from the various guidance. We also reviewed IRS's 
Internal Revenue Manual (IRM) and interviewed officials to determine 
if IRS had guidance on whether and how to conduct and document 
economic analyses to support decisions to initiate, renew, or expand 
programs. We also reviewed the types of costs and benefits that IRS 
included in the PDC study. 

To determine what changes IRS has planned or made to its collection 
approach based on its PCA experience and the PDC study, we reviewed 
program documents and interviewed IRS officials on their processes and 
procedures for collecting tax debt. We also compared IRS's plans for 
studying whether to work on PCA-type cases in the future to guidance 
in our previous publication on evaluating federal programs[Footnote 
10] and in our report reviewing IRS's study of the earned income tax 
credit.[Footnote 11] 

We conducted this performance audit from November 2009 to September 
2010 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit 
to obtain sufficient, appropriate evidence to provide a reasonable 
basis for our findings and conclusions based on our audit objectives. 
We believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 

Background: 

In developing the PDC program, IRS officials said that contracting 
with PCAs was needed because Congress was unlikely to provide IRS 
sufficient staff to attempt collection on the inventory of cases with 
a lower priority.[Footnote 12] Also, officials said that PCAs would 
pursue cases that IRS staff would not because IRS had other higher-
priority collection cases. 

As IRS planned the PDC program, we issued two reports with findings 
and recommendations to improve the program. 

* In May 2004, we reported that IRS's PDC study approach--comparing 
PCA and IRS performance for the same type of simpler cases that would 
be sent to PCAs--would provide limited information to judge whether 
using PCAs is the best use of resources.[Footnote 13] In sum, the 
approach conflicted with IRS officials' position that these simpler 
cases would not be worked by IRS employees given the higher-priority 
cases in IRS's workload. We recommended instead that IRS compare the 
use of PCAs to a collection strategy that officials determine to be 
the most effective and efficient overall way of achieving collection 
goals. 

* In September 2006, we reported that IRS's planned PDC study approach 
would not meet the intent of our 2004 recommendation.[Footnote 14] 
Because the study would not count the fees paid to PCAs as program 
costs, it would not compare the results of using PCAs with the results 
IRS could get if it was given the same amount of resources, including 
the fees to be paid to the PCAs. We recommended instead that IRS 
ensure that the study methodology and the reports on the study include 
the full costs of the PDC program, including the fees paid to PCAs and 
the best use of those federal funds. 

IRS agreed with the recommendations in both reports, and IRS's actions 
to implement them generally are the topic of this report. IRS's PDC 
study did not meet the intent of our 2004 recommendation because it 
did not compare using PCAs to what IRS officials determined to be the 
most efficient and effective overall strategy. Although IRS met part 
of our 2006 recommendation in that the PDC study included fees paid to 
PCAs as program costs, the PDC study did not fully implement our 
recommendation to include the best use of federal funds because, as 
discussed further below, the study had methodological errors and a 
narrow scope. 

IRS's PDC study report results were released on March 5, 2009, along 
with IRS's announcement that it was ending the program.[Footnote 15] 
The study tracked selected cases assigned to IRS's Automated 
Collection System (ACS) versus PCAs and measured them in terms of cost 
per dollar collected, percentage of balance due collected, and 
percentage of cases in payment status at the end of the study period. 
[Footnote 16] For both PCAs and IRS, the PDC study counted the dollars 
collected from full payments or estimated dollars collected through 
installment agreements. The study results, summarized in table 1, 
showed that IRS performed better in each of the measures. 

Table 1: IRS's PDC Study Results Comparing IRS and PCAs Working 
Collection Inventory from the Same Pool of Cases Assigned to PCAs: 

IRS: 
Dollars collected (estimated): $775,302; 
Cost: $53,545; 
Cost per dollar collected: $0.07; 
Balance due: $6.8 million; 
Percentage of balance due collected: 11; 
Modules[A]: assigned to be worked: 1,341; 
Modules[A] in payment status: 369; 
Percentage of modules[A] in payment status: 28. 

PCAs: 
Dollars collected (estimated): $443,438; 
Cost: $105,621; 
Cost per dollar collected: $0.24; 
Balance due: $11.7 million; 
Percentage of balance due collected: 4; 
Modules[A]: assigned to be worked: 2,133; 
Modules[A] in payment status: 284; 
Percentage of modules[A] in payment status: 13. 

Source: IRS. 

[A] A module in this case is a tax year for which there was an amount 
of unpaid tax due from a taxpayer. Quarterly employment taxes were not 
included. 

[End of table] 

On February 3, 2009, legislation was introduced in the House to amend 
the Internal Revenue Code of 1986 to repeal the authority of the 
Secretary of the Treasury to enter into PDC contracts.[Footnote 17] 
The last major action on the legislation was on February 3, 2009, when 
it was referred to the House Committee on Ways and Means. 

The Omnibus Appropriations Act of 2009, enacted March 11, 2009, 
prevented IRS from using fiscal year 2009 appropriated funds "to enter 
into, renew, extend, administer, implement, enforce, or provide 
oversight of any qualified tax collection contract (as defined in 
section 6306 of the Internal Revenue Code of 1986)."[Footnote 18] 
Absent the access to appropriated funds, IRS funded the administrative 
costs of the PDC program through its user fees[Footnote 19] until IRS 
was able to end all PDC activities. The Consolidated Appropriations 
Act of 2010 placed the same prohibition on IRS's use of fiscal year 
2010 funds.[Footnote 20] 

Methodological Errors and Narrow Scope Limit the Study's Usefulness in 
Supporting the PDC Decision: 

According to IRS officials, the PDC study was not originally intended 
or designed to be primary support for IRS's PDC program decision. Even 
though other factors, such as potential increases in IRS collection 
staffing, were considered, based on our interviews with IRS officials 
and IRS's announcement of the program's termination, the study results 
played a primary role in supporting the decision. IRS officials noted 
that the difference in cost per dollar collected using IRS staff and 
PCAs was so pronounced that in their view, additional analyses would 
have been unlikely to change the decision that was made. Nevertheless, 
neither we nor IRS officials know whether the PDC study results and 
decision on the program would have differed significantly if the study 
had been designed to be primary support for IRS's PDC program. 
However, errors in the study sampling methodology and the study's 
narrow scope limit its usefulness in supporting the PDC decision. 

IRS does not have guidance on whether and how to conduct and document 
economic analyses to support decisions to initiate, renew, or expand 
programs. The IRM, which includes authoritative guidance for IRS 
managers, does not include guidance on economic analyses that should 
support decisions to initiate, renew, or expand a program. 

The Study Result May Be Over-or Understated Because the Sample Was Not 
Generalizable to the PDC Population: 

Because IRS did not perform certain analyses and documentation is not 
available to do those analyses, it is unclear whether the study's 
results are accurate. Guidance for statistical surveys requires that a 
study's sampling procedures be sufficiently described such that 
policymakers can assess whether the study's results can be generalized 
to the settings and times of interest.[Footnote 21] The PDC study 
documentation mentioned a design that could represent the full 
population of PCA-type cases based on sample cases drawn from various 
groupings of PCA-type cases.[Footnote 22] However, IRS did not 
generalize the study results back to the full PCA case population, 
limiting its analysis to the sample cases. It is common for study 
results to differ significantly between results for the sampled cases 
and results when generalized back to the full population.[Footnote 23] 
IRS did not retain sufficient documentation on the sample selection 
and analysis to enable it or others at this point to calculate 
estimates for the population, measure the margin of error, or 
otherwise ensure the soundness of the sample approach.[Footnote 24] 
Even though IRS documentation referred to a sampling approach, IRS 
officials said the study was not intended to provide generalizable 
results. The sample cases assigned to the PCAs and IRS for comparative 
purposes may or may not have been similar. IRS was concerned about 
potential differences between the groups and tested the difference in 
the average balance due amount between the cases to be assigned to 
PCAs and IRS. The test did not find a difference. However, this was a 
limited test of only one variable. IRS could have tested other 
variables, such as taxpayer filing history, adjusted gross income, 
filing status, geographic region, and type of forms completed by the 
taxpayer. Differences in any of these variables could affect taxpayer 
compliance and payment. Performance differences between the PCAs and 
IRS could be due to differences in the composition of the groups. 

Different Study Objectives and Design Would Have Better Supported the 
Program Decision-Making Process: 

Because the PDC study had a narrow objective of comparing the results 
of collection efforts by IRS and PCAs for the PCA-type cases, its 
design did not consider other factors included in federal and other 
guidance on conducting program analyses. An objective more directly 
focused on providing information to serve as primary support for IRS's 
decision on the PDC program could have led IRS to conduct additional 
analyses relevant to the decision. The findings from those analyses 
could have affected the program decision. 

The Study Provides No Statement of the Program's Rationale and 
Analysis of Selected Data Related to It: 

Guidance on conducting program analysis to determine whether to 
continue a program states that the rationale for the government 
program being examined should be clearly stated in an analysis. 
[Footnote 25] Clearly stating the rationale for the program can help 
ensure that the analysis includes relevant factors and outcome 
measures, particularly if facts related to the rationale have changed 
and therefore need to be included in the analysis. Stating the 
rationale of the program and conducting related analyses can help 
ensure that variables measured in the study answer the questions 
decision makers have about the basic reasons the program exists, the 
continuing need for it, and the costs and benefits of continuing, 
revising, or ending it. For instance, one rationale for starting the 
program was that Congress was unlikely to provide funds for staff to 
work these cases. The PDC study did not cover whether this had changed 
and, if so, whether IRS would use any portion of additional staffing 
for PCA-type cases. As discussed later, the number of staff who would 
normally work PCA-type cases has increased in recent years. 

Alternatives Were Considered, but Scaling Back the PDC Program Was Not 
Among Them: 

To inform a decision on whether to continue a program and ensure the 
best use of federal resources, government officials need information 
on the range of alternatives for achieving a program's objectives at 
the lowest costs. Guidance states that program analyses should include 
alternative means of achieving program objectives by examining 
different program scales, methods of provision, and degrees of 
government involvement.[Footnote 26] 

The PDC study included three alternatives: (1) PCAs attempting to 
collect unpaid tax debts for the inventory assigned to them, (2) IRS 
taking collection action on the types of cases that were assigned to 
PCAs, and (3) IRS taking collection action on another type of 
inventory normally not worked.[Footnote 27] However, beyond not 
addressing whether to continue the PDC program, the study did not 
analyze alternatives for program scale, such as expanding the PDC 
program or scaling it back to a segment of cases that might be more 
cost effective for PCAs to work than IRS. For example, in deciding 
whether to scale back rather than eliminate the PDC program, IRS could 
have analyzed the types of cases, if any, where PDC performance was 
better than IRS performance. Assuming that this analysis pointed to 
favorable types of cases for PCAs, IRS then could have limited the 
program to working those cases, unless the benefit of doing this was 
so limited that running a separate PDC program with fixed costs would 
not be cost beneficial. According to IRS officials, IRS had 
effectively scaled back the program during its course because after 
the initial wave of cases assigned to PCAs, IRS had difficulties in 
obtaining a sufficient volume of cases appropriate for PCAs to work. 

In addition, IRS did not compare the PDC program to what it judged to 
be the best overall strategy for improving tax collections, as we had 
recommended IRS do in our 2004 report, even though the Commissioner of 
Internal Revenue agreed with our recommendation.[Footnote 28] In 2004, 
we concluded that IRS should do a study in line with federal guidance, 
such as comparing the results of using PCAs to the results from using 
the same amount of funds to be paid to PCAs in an unconstrained manner 
that IRS determined to be the most effective overall way of achieving 
its collection goals. After our report, Congress authorized the PDC 
program and included the provision making 25 percent of PDC 
collections available to IRS. Conceptually, Congress effectively made 
available funds for the PDC program totaling the amounts paid to PCAs 
plus the up to 25 percent of PDC collections that IRS could use for 
enforcement purposes. We continue to believe that a comparison of 
results of the PDC program--as authorized by Congress--to results IRS 
would achieve if given the same funds to use in what it judged to be 
the best possible manner would have better supported a decision on the 
PDC program. 

Not All Benefits and Costs Were Included: 

To inform a decision on whether to continue a program, government 
officials need complete and reliable information on all the program's 
benefits and costs.[Footnote 29] Guidance for doing economic analyses 
states that to the extent possible all benefits and costs should be 
monetized to provide a standard unit of comparison.[Footnote 30] If it 
is not feasible to assign monetary values, other quantification of 
costs and benefits should be done. If quantification is not possible, 
at a minimum, analyses should include a comprehensive listing of the 
different types of benefits and costs to identify the full range of 
program effects. Furthermore, analyses should be explicit about the 
underlying assumptions used to estimate future benefits and costs. 

The PDC study measured the government's cost per dollar of direct tax 
revenue collected, percentage of balance due collected, and percentage 
of cases in payment status at the end of the study period. As cited in 
IRS's announcement of the decision to not renew PCAs' contracts, the 
study showed that when working the same types of cases as PCAs, IRS 
had better results than PCAs. Specifically, IRS's cost was $0.07 to 
collect a dollar itself while the government's cost using PCAs was 
$0.24 for each dollar collected.[Footnote 31] According to IRS 
officials, IRS's results on the cases it worked were comparable to or 
better than the results of working its normal cases. 

In addition to direct revenues, the study did not estimate or 
otherwise discuss indirect revenue (for example, determine whether 
other taxpayers are more or less likely to pay their due taxes when 
IRS works the cases than when PCAs do). Indirect revenues are 
difficult to estimate, but the study could have explored whether any 
logical reasons exist to indicate that indirect revenue would have 
varied based on which party worked the PCA-type cases. 

As discussed further below, the PDC study also did not include 
potentially important program costs and benefits for a tax enforcement 
program, such as: 

* costs to taxpayers to comply with requirements to pay their 
delinquent taxes (are costs to taxpayers lower under IRS or PCAs); 

* equity (is IRS or are PCAs more effective in collecting taxes from 
taxpayers in different income groups, for example); and: 

* economic efficiency (is IRS or are PCAs more effective at working 
cases involving taxpayers in different industries, for example, 
resulting in more or less distortion of activity across types of 
businesses).[Footnote 32] 

IRS officials said that these benefits and costs were not addressed in 
the study because they are difficult to measure. However, IRS could 
have followed OMB guidance to, at a minimum, list and discuss these 
omitted costs and benefits in the final report. As a result, the 
decision to end the program rests, in part, on the assumption that 
important differences do not exist between IRS and PCA handling of 
these cases related to these costs and benefits. If IRS assumed that 
no such differences existed, this should have been stated in the 
report with the supporting rationale. 

Some important differences might exist. For example, it is not clear 
that taxpayer costs would be the same regardless of which party worked 
on collecting their tax debt. Taxpayer compliance costs could be lower 
when IRS collects taxes because 90 percent of the cases IRS worked 
were eligible for systemic actions, such as systemic levies of 
taxpayer's assets. Such actions might incur little cost for the 
taxpayer beyond reading IRS's notification of intent to levy assets, 
as compared to a PCA case, which required that a taxpayer first make a 
payment or answer or return a phone call in response to a PCA's letter 
or call. Policymakers do not know whether costs to taxpayers differed 
between IRS and PCA collections because IRS did not consider this and 
other circumstances that could affect taxpayers' costs. 

The study also did not include a type of collected revenue--which IRS 
called noncommissionable revenue--that IRS officials tracked in a 
measure to assess the PDC program's performance and, in some cases, 
might have gone uncollected otherwise. For example, IRS did not pay 
PCAs a commission when debt was collected within a 10-day window after 
being assigned to a PCA. In establishing the program, an IRS official 
told us that IRS expected that many dollars would be collected within 
this 10-day window because taxpayers would send in payment after 
receiving notification that their debts were being assigned to a PCA. 
To the extent this occurred, not taking these revenues into account in 
the study may have led to an underestimate of the collections 
attributable to the PDC program. Other noncommissionable revenue 
included collections through actions IRS systemically took regardless 
of whether a case was assigned to a PCA, such as refund offsets. 
According to IRS officials, such revenues were not included in the PDC 
study for either PCAs or IRS collections. IRS data through fiscal year 
2007[Footnote 33] show that total collections for the PDC program were 
about $32.1 million with $7 million (22 percent) of the total being 
noncommissionable; for fiscal year 2008, the total was $37.3 million 
with $9.6 million (26 percent) noncommissionable; and for fiscal year 
2009, the total was $28.8 million with $11.5 million (40 percent) 
noncommissionable. 

For the Types of Government Costs IRS Chose to Include, the Study Was 
Consistent with Selected OMB Guidance: 

IRS's PDC study focused on just a few types of government costs. Those 
costs included IRS's costs as well as the commissions paid to PCAs. 
Table 2 identifies these types of costs. 

Table 2: Summary of the Types of Costs Included in IRS's PDC Study: 

IRS costs: 
Direct labor (includes number of direct hours worked); 
Indirect labor (such as annual and sick leave, training, and 
administrative time); 
Estimated telephone and mailing/postage costs; 

PCA costs: 
Commissions on dollars collected[A]; 
IRS support costs (includes salary and benefits costs for employees) 
for: 
* referral units and; 
* the Taxpayer Advocate Service. 

Source: IRS. 

[A] Including the commissions as costs was consistent with our 
recommendation in GAO-06-1065. 

[End of table] 

IRS used a variable costing methodology rather than a full costing 
approach to compare the costs of PCAs and IRS working inventory that 
they determined to be similar.[Footnote 34] According to IRS, the 
variable costing approach included expenses that would vary depending 
on an increase or decrease in the inventory assigned to either the 
PCAs or IRS. The study did not include previous costs, such as costs 
in setting up the PDC program. In general, this overall approach to 
including costs is consistent with OMB guidance. That guidance says 
that analyses should be based on incremental costs and benefits and 
sunk costs should be ignored. 

IRS did not include management oversight or information technology 
since such costs would not vary with the volume of cases handled. For 
example, IRS did not include the costs of a PDC program oversight unit 
that among other things was responsible for monitoring PCA 
performance. Although these costs likely would not vary greatly with 
changes in the volume of cases handled by IRS or PCAs, if the absolute 
costs were significantly greater either for IRS or PCAs, the 
difference may have affected the cost comparison, especially over 
relatively small numbers of cases. IRS did not document whether these 
costs varied significantly based on who handled the cases. 

To determine the cost for the PCAs, IRS used historical cost data of 
the PDC program for fiscal year 2008 that were generated from IRS's 
Integrated Financial System. IRS apportioned those costs to the number 
of cases in its study. 

Unlike the PCAs cost data, the IRS costs were not system generated but 
were calculated based on estimates and assumptions. For example, IRS 
identified the transactions (such as phone calls made or notices sent 
to taxpayers) taken on each case in the sample by manually reviewing 
individual data posted in the taxpayers' case files in the Integrated 
Data Retrieval System. To then calculate direct labor hours per case, 
IRS used information from inventory handling time reports and call 
handling time reports. IRS determined an average minutes per 
transaction, which it applied to the sampled cases in the study. 
Because the PCA-type cases were at least somewhat different than cases 
ACS normally handled, applying the average minutes from typical 
transactions may or may not have accurately reflected the time 
actually taken on the PCA-type cases. Other estimated costs were for 
telephone calls and postage costs for mailing a single installment 
agreement notice to taxpayers. 

Study Assessed Past Performance, Not Likely Future Performance: 

Another fundamental program analysis principle is to estimate expected 
results of the program. Because decisions about the future of a 
program depend on the expected future results of it or any 
alternatives that are under consideration, future results are to be 
included on a discounted basis in either benefit-cost analysis (the 
value of expected net benefits, i.e., benefits minus costs) or cost-
effectiveness analysis (to determine which alternative has the lowest 
costs for a given amount of benefits or which has the greatest 
benefits for a given amount of costs). Past performance can be 
relevant in helping to estimate the value of future results.[Footnote 
35] 

IRS's PDC study included no estimates of future costs and revenues for 
the alternatives studied. The study presented data and related ratios, 
including costs per dollar collected, only for cases worked in the 
past. Although the study text states that the cases studied were like 
those PCAs worked during the program and would work in the future, it 
provided no analytical basis for this assumption. 

Documenting any analyses of future costs and revenues was especially 
relevant for any decision on the future of the PDC program because IRS 
had previously revised the criteria for selecting PDC cases several 
times in order to provide the volume of work to PCAs that had been 
anticipated. Accordingly, reasonable questions existed about whether 
IRS would be able to continue providing a sufficient stream of work to 
PCAs and how that might affect the scope of the program and the nature 
of the cases PCAs would work. Absent any such documented analysis, 
decision makers and those overseeing the agency had a limited basis on 
which to assess whether PCAs likely would have been as effective at 
working the cases IRS could deliver in the future as they had been in 
working past cases. 

IRS officials said that they considered future results, but only 
outside the study, in deciding to end the PDC program. IRS officials 
said that during management meetings on deciding whether to continue 
the PDC program, they considered past program performance compared to 
expectations and how that might affect future performance. For 
example, they said that they considered gaps between IRS's original 
expectations, what the program actually realized, and what the program 
would likely achieve in the future given actual program experience. 
Among other things, IRS officials noted the declining available case 
inventory levels for the program. They said that the program's actual 
performance compared to expectations was a reason that IRS ended the 
program. IRS was unable to provide documentation of any analyses of 
future expected costs and results. Without such analyses, it is 
unclear to what extent the program did not meet expectations, whether 
IRS determined the underlying reason, and whether the program's future 
performance could have been improved in a manner that could have 
affected the decision on the program. 

IRS Has Not Changed Its Collection Approach Based on Its PDC 
Experience and Study, and Whether IRS Will Work Certain PCA-Type Cases 
Is Unclear: 

Although the Basis Was Not Documented, IRS Concluded That No PCA 
Practices Should Be Adopted: 

In authorizing the PDC program in 2004, Congress required IRS to 
create a measurement plan to capture information on successful 
collection techniques used by the contractors that IRS could 
adopt.[Footnote 36] The PCAs' best practices were to be compared with 
IRS's collection practices.[Footnote 37] IRS was to report on this 
measurement plan and its results in a mandated biennial report that 
was to include specific types of information.[Footnote 38] 

In an unpublished draft biennial report, IRS said it reviewed PCA best 
practices and concluded that none of them were sufficiently better 
than IRS's practice to merit adoption. IRS officials provided us a 
draft version of the biennial report for 2007. IRS neither finalized 
the report nor released it. IRS officials pointed to significant 
transition in the PDC office during this time, as well as transitions 
in the Deputy Commissioner's and the Commissioner's offices. This 
draft report described IRS's steps to identify lessons learned from 
PCAs.[Footnote 39] It said that IRS did not find any immediate 
opportunities to adopt PCA practices, but provided no details beyond 
this sentence. The draft report also said IRS would continue to try to 
identify lessons learned. 

Conflicting Information on Whether or How IRS Will Work PCA-Type Cases: 

IRS officials said that IRS had not changed its criteria to start 
regularly selecting PCA-type cases to work because the PDC study 
results were not sufficient to identify which of the PCA-type cases 
could be productively worked. However, IRS officials said that they 
were surprised by the study results, which indicated that IRS staff 
might have better results working these cases than some of the cases 
IRS normally works. IRS officials said that the types of cases sent to 
PCAs previously had been considered low priority because of low 
potential collection return. In establishing the PDC program, IRS 
officials indicated that the PCA contractors would be working inactive 
collection cases that IRS collection staff would not be working. 
[Footnote 40] For example, in May 2007, after IRS had sent the initial 
inventory of cases to PCAs to work, the Acting Commissioner of 
Internal Revenue testified that if the money invested in the PDC 
program was not used for it, IRS would not use the funds to work cases 
that would have been assigned to PCAs.[Footnote 41] Rather, the funds 
would be used to work other cases considered higher priority in the 
tax debt inventory. 

Because of the surprising result, IRS began a pilot study of the types 
of cases that had been worked by PCAs. According to IRS officials, the 
pilot study's goals were to (1) provide coverage for a segment of the 
unpaid tax debt inventory that would not be worked because of the end 
of the PCA contract and (2) provide information on which types of 
cases IRS could fruitfully work in the future. 

IRS placed approximately 19,000 pilot cases with ACS over a 4-month 
period beginning in September 2009.[Footnote 42] IRS officials said 
that the agency randomly drew the cases from the database of cases 
that IRS had used to assign cases to the PCA staff. IRS employees 
cannot identify which of their assignments are PCA-type cases. As of 
July 2010, the status of these cases was as follows: 

* 8,559 cases (45 percent) had been closed, for example, through some 
form of payment or other action; 

* 4,954 cases (26 percent) were being worked; and: 

* 5,552 cases (29 percent) had yet to be worked. 

As of June 2010, IRS officials said that they expected that the work 
on these pilot study cases would be finished by December 2010. For 
these cases, IRS collected data on its activities (such as liens and 
levies) and the results (such as dispositions and dollars collected). 
Of the 8,559 cases closed as of July 1, 2010, IRS had received payment 
in full in 931 cases (11 percent) and had entered into installment 
agreements in 2,878 cases (34 percent). 

IRS officials said they had planned to use the collected data to 
determine if changes should be made to the case selection criteria to 
assign certain types of PCA-type cases to the IRS active collection 
inventory. According to IRS officials, after the pilot cases were 
worked, IRS's Office of Program Evaluation and Risk Analysis (OPERA) 
was to study the collected data to see what happened with the cases. 
IRS expected that the OPERA study would take a year to complete (i.e., 
until about December 2011). IRS officials said that the agency was 
then to determine whether more PCA-type cases should be routinely 
worked in IRS and, if so, which ones. 

Following guidance on conducting program analyses helps ensure that 
relevant information is considered and a sound methodology is used, 
including stated study objectives and a clear and appropriate study 
design for answering the objectives. As we have reported in the past, 
these principles are essential when developing an analysis or 
evaluation plan.[Footnote 43] For example, this plan should not only 
be written but have sufficient detail to answer questions such as the 
following: 

* Are the test goals and objectives of IRS's pilot study clearly 
stated? 

* Has IRS identified the types of data to be collected? 

* What specific types of analysis will be performed? 

The documentation IRS had available describing the pilot study--a 
draft plan prepared in December 2009--addressed summarizing and 
providing data on the sampled cases (such as collection action taken, 
e.g., liens and levies) and the results (such as the disposition or 
current status of the cases and the dollars collected). But as of June 
24, 2010, IRS had not produced an approved plan on how it intended to 
analyze the pilot study data even though it had worked about three-
fourths of the cases selected for the pilot. According to IRS 
officials, progress in developing an analysis plan for the pilot study 
had been delayed by efforts to implement Consolidated Decision 
Analytics (CDA). Among other things, CDA is to implement new models to 
better predict the collection potential of unpaid tax debt cases to 
help ensure the best use of IRS's collection resources. Officials said 
that any plan for analyzing the pilot study's data would need to 
ensure that the pilot study results will be useful given IRS's plans 
to more fully implement CDA in January 2011. 

Beyond not knowing how the pilot study data will be analyzed, IRS had 
not clarified its criteria on how it would use the results of any 
analyses to make a decision about assigning more PCA-type cases to be 
worked in IRS. Specifically, IRS had no documentation on the criteria 
that officials would use in making this decision and what factors, if 
any, beyond the analyses of the pilot study data would contribute to 
those criteria or that decision. Developing criteria would be 
important to ensure that the variables measured in the study would be 
useful in supporting the decision on whether to change case selection 
criteria to regularly pursue such cases. 

As we concluded our review, the status of the pilot study and whether 
or how PCA-type cases would be included in active collection case 
inventory became less clear. IRS provided conflicting information 
about determining whether PCA-type cases have sufficient collection 
potential to be included in its collection inventory. On one hand, 
officials said that CDA models were built by tracking the 
characteristics and collection potential of actual collection cases. 
On the other hand, PCA-type cases generally have not been worked by 
collection staff, which is why IRS began its pilot study of PCA-type 
cases to determine their collection potential. In its comments on our 
draft report, IRS said that the CDA models that would be implemented 
in January 2011 had overtaken the need to complete the pilot study. 
Examining the CDA models was outside the scope of our review. 
Therefore, we had not reviewed documentation on what types of data 
were used in developing the CDA models. However, in response to our 
direct question about whether PCA-type case results were used in 
developing the CDA models to be implemented in January 2011, an IRS 
official said that they had not been used. 

Staff Who Could Work PCA-Type Cases Have Increased, but IRS Has Not 
Committed to Working the Cases: 

IRS's March 2009 announcement ending the PDC program said that IRS 
anticipated hiring more collection staff in fiscal year 2009 and 
referred to support from the administration and Congress for increased 
IRS enforcement resources. According to IRS officials, IRS's ACS staff 
could work on the PCA-type cases. Based on IRS data, ACS staffing 
levels--as measured by full-time equivalent (FTE) positions--have 
increased since fiscal year 2008.[Footnote 44] The ACS staffing levels 
increased by 308 FTEs from fiscal years 2008 to 2009 (i.e., 3,395 to 
3,703) and were to increase by 94 FTEs during fiscal year 2010 (i.e., 
to 3,797). These staffing levels account for hiring done to offset 
attrition as well as additional hiring beyond attrition. 

The PDC program announcement was unclear on whether additional staff 
to be hired would be used for PCA-type cases. On the one hand, the 
announcement said that "IRS determined the work is best done by IRS 
employees," which appears to refer to PCA-type work. On the other 
hand, the announcement also said that "new employees would give the 
IRS the flexibility to make assignments based on the areas of greatest 
need rather than filtering which cases can be worked using contractor 
resources." IRS officials told us that the announcement did not imply 
that the collection staff to be hired in fiscal year 2009 would be 
used to work PCA-type cases. 

Conclusions: 

In comparing IRS and PCAs working the same types of cases in the PDC 
study, IRS was responsive to concerns of some in Congress, the 
National Taxpayer Advocate, and others on the study's comparison. 
Although a study that adhered to federal guidance on analyzing 
programs would have better informed IRS's decision on the fate of the 
PDC program, it is not possible to know whether such a study would 
have had materially different results or changed IRS's decision on the 
program. However it would have provided more complete information for 
policymakers to consider. To the extent possible, and especially for 
significant program decisions, IRS should adhere to OMB and other 
guidance for economic analyses to better ensure that policymakers have 
adequate information to support their decisions. IRS does not have 
guidance for managers on the types of analyses that should be done and 
documented to support program decisions. Such analyses can yield 
significant benefits by helping inform decision making, but they also 
incur costs. Therefore, careful consideration of the potential risks 
and positive impacts of various study designs is necessary to select 
an appropriate study design and scope to answer the relevant questions 
in a methodologically sufficient manner. 

IRS's PDC study suggested that at least certain PCA-type cases, which 
IRS had not been working, may be worth including in ACS's inventory 
for collection action. To the extent that these results are valid and 
reliable, IRS may be able to make a relatively low-cost investment in 
certain PCA-type cases to collect tax debts. However, as we concluded 
our audit, IRS provided conflicting information about determining 
whether PCA-type cases have sufficient collection potential to be 
included in its collection inventory. Given the conflicting 
information available to us, we believe it is important that PCA-type 
case results are considered and incorporated as appropriate into the 
CDA models. If IRS determines that completing the pilot study is the 
best method to do so, a documented methodology and criteria for the 
study's analysis could help IRS make a better decision on which PCA-
type cases, if any, should be added routinely to active collection 
status. 

Recommendations for Executive Action: 

We recommend that the Commissioner of Internal Revenue take the 
following three actions: 

* Establish guidance on the types of analyses that should be done to 
support decisions to initiate, renew, or expand programs. The guidance 
might refer to OMB Circular A-94 and, if needed, provide any 
supplementation specific to IRS. 

* Establish a policy requiring documentation for the design, analyses, 
and conclusions of studies supporting program changes. 

* Ensure that PCA-type case results are considered and incorporated as 
appropriate into the CDA model. 

If IRS determines completing the pilot study is the best means to 
ensure that PCA-type case results are considered for the CDA models, 
the Commissioner should ensure that the pilot study has a documented 
methodology and criteria to guide IRS's analysis and decision. 

Agency Comments and Our Evaluation: 

The IRS Deputy Commissioner for Services and Enforcement provided 
written comments on a draft of this report in a September 8, 2010, 
letter, which is reprinted in appendix I. IRS staff also provided 
technical comments, which we incorporated into the report as 
appropriate. 

IRS disagreed with our finding that its PDC study was not soundly 
designed to support its decision on whether to continue contracting 
out debt collection. IRS said the study's comparison of the cost- 
effectiveness of PCAs and IRS working similar cases provided 
meaningful data that aided its decision making. IRS cited an 
independent review of the PDC study that found the results to be 
reasonable, even though the study had limitations and constraints. We 
continue to believe that the study was not a soundly designed cost-
effectiveness comparison for supporting IRS's decision. Our report 
discusses our reasoning in detail, focusing on the study's 
methodological errors, narrow scope, and lack of adherence to guidance 
for doing such studies. For example, IRS did not do the analysis 
necessary to generalize the study results to the full PCA case 
population even though study results could differ significantly when 
generalized to the full population. Our meetings with staff who 
performed the independent review and our analyses of their 
documentation did not change our finding about IRS's study. 

IRS agreed with our two recommendations dealing with establishing 
guidance on analyses to support decisions to initiate, renew, or 
expand a program and policies to ensure documentation of such studies. 
More specifically, IRS said it would review current guidance and 
policies and develop additional guidance where needed. 

IRS agreed in principle with our third draft recommendation on 
ensuring that a documented methodology and criteria guide IRS's 
analysis and decision on whether to include selected PCA-type cases in 
its collection inventory, but said events have overtaken the need to 
complete the ongoing study, citing IRS's plans to implement CDA models 
in fiscal year 2011. These models are intended to select cases with 
the best potential for collection action in one of IRS's work streams. 
IRS said that to measure the impact of the PCA-type cases, as was the 
plan when the PCA project was terminated, is no longer necessary. 

We had discussed with IRS officials the continued need for the pilot 
study when IRS told us in July 2010 that it planned to implement CDA 
in January 2011. IRS officials, including the Acting Director, 
Collection Business Reengineering, said that while CDA selection would 
focus on collection potential and not type of case (i.e., PCA-type), 
the pilot study of approximately 19,000 PCA-type cases might provide 
data useful for improving CDA models. Officials affirmed that they 
initiated the pilot study because the PDC study showed that PCA-type 
cases might have high collection potential at low cost. Accordingly, 
our draft report recommended that IRS document the methodology and 
criteria for its pilot study. Information provided in IRS comments on 
the report and in response to our subsequent questions suggests that 
whether and how PCA-type cases may be selected for active collection 
inventory is uncertain. Although IRS's comments on the draft report 
said that the need for completing the pilot case study was overtaken 
by the development of the CDA models, in separate technical comments 
IRS officials said they were continuing to work the pilot cases and 
provided no indication that they would stop working them before CDA is 
implemented in January 2011. Further, in response to our question 
about whether PCA-type case results were used in developing the CDA 
models, an IRS official said that they had not been used. In response 
to IRS's comments and absent evidence that CDA will be implemented as 
planned and that its models will include IRS's experience in 
attempting collection of PCA-type cases, we revised the third 
recommendation to better focus on ensuring that PCA-type case results 
are considered and incorporated as appropriate into the CDA models. 
Further, if IRS determines completing the pilot study is the best 
means to ensure that PCA-type case results are considered for the CDA 
models, we maintained our recommendation that IRS ensure that the 
study has a documented methodology and criteria to guide IRS's 
analysis and decision. 

As agreed with your office, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies to the 
Secretary of the Treasury, the Commissioner of Internal Revenue, and 
other interested parties. In addition, the report will be available at 
no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-9110 or brostekm@gao.gov. Contact points for 
our Offices of Congressional Relations and Public Affairs may be found 
on the last page of this report. Key contributors to this report are 
listed in appendix II. 

Sincerely yours, 

Signed by: 

Michael Brostek: 
Director, Tax Issues: 
Strategic Issues: 

[End of section] 

Appendix I: Comments from the Internal Revenue Service: 

Department Of The Treasury: 
Deputy Commissioner
Internal Revenue Service: 
Washington, D.C. 20224: 

September 8, 2010: 

Mr. Michael Brostek: 
Director, Tax Issues: 
U.S. Government Accountability Office: 
Washington, D.C. 20548: 

Dear Mr. Brostek: 

Thank you for the opportunity to review the Government Accountability 
Office's (GAO) draft report entitled, "Tax Debt Collection: IRS Could 
Improve Ongoing and Future Studies by Establishing Appropriate 
Guidance" (GA0-10-963). 

As the report acknowledges, the IRS's study of Private Debt Collection 
(PDC) program operations was designed to compare the cost of working 
similar cases in IRS Automated Collection System (ACS) call centers 
with Private Collection Agencies (PCAs). While the study's objectives 
were limited to comparing the cost-effectiveness of PCAs and ACS, the 
study provided meaningful data to help make the overall decision
on whether to continue the PDC program. Data from the study along with 
other resource, inventory, and operational factors were considered in 
the overall decision. 

The IRS disagrees with the report's contention that the study was not 
soundly designed. An independent review of the cost-effectiveness 
study confirmed that while the study contained limitations and 
constraints, the findings presented in the study were reasonable. The 
independent review found that it was unlikely that any of the 
constraints identified would reverse the comparative positions of ACS 
and PDC operations. 

The report provides recommendations for improving the guidance 
available to managers on how to perform program analysis. We agree 
that clearer guidance would improve consistency, documentation, and 
the overall quality of future program reviews. 

The enclosed response addresses each recommendation separately. 

If you have questions, please contact me, or a member of your staff 
may contact Nikole Flax, Assistant Deputy Commissioner for Services 
and Enforcement, at (202) 622-6860. 

Sincerely, 

Signed by: 

Steven T. Miller: 

Enclosure: 

[End of letter] 

Enclosure: 

GAO Recommendations and IRS Response to GAO Draft Report: 

Tax Debt Collection: IRS Could Improve Ongoing and Future Studies by 
Establishing Appropriate Guidance: GA0-10-963: 

Recommendation 1: 

Establish guidance on the types of analyses that should be done to 
support decisions to initiate, renew, or expand programs. The guidance 
might refer to OMB Circular A-94 and, if needed, provide any 
supplementation specific to IRS. 

Comments: 

We agree with the recommendation that the IRS explore clearer guidance 
on the types of analyses that should be done to support decisions that 
will have a significant impact on major programs. IRS's Office of 
Program Evaluation and Risk Analysis (OPERA) will lead a review of 
current IRS policies for performing program analysis and develop 
additional guidance where needed. 

Recommendation 2: 

Establish a policy requiring documentation for the design, analyses, 
and conclusions of studies supporting program changes. 

Comments: 

We agree with the recommendation that the IRS explore clearer guidance 
on the documentation requirements for analysis conducted to support 
significant program changes. IRS's Office of Program Evaluation and 
Risk Analysis (OPERA) will lead a review of current IRS policies for 
documenting program analysis and develop additional guidance where 
needed. 

Recommendation 3: 

Ensure that IRS's ongoing study on whether to include selected PCA-
type cases in the collection inventory includes a documented 
methodology and criteria to guide IRS's analysis and decision. 

Comments: 

We agree with this recommendation in principle, but events have 
overtaken the need to complete this study. In FY 2011, the IRS will 
implement Consolidated Decision Analytics (CDA) to select balance due 
cases to be worked and route the cases to the appropriate work stream. 
The CDA system will use decision analytics and modeling to select 
cases with the best potential for collection and will put these cases, 
in priority order, into the work streams of ACS or field collection, 
as indicated by the models. The models and their output will be 
regularly reviewed for accuracy and performance. As such, we believe 
that using these models will force the best cases into our work 
streams, and to measure the impact of the PCA-type cases, as was the 
plan when the PCA project was terminated, is no longer necessary. 

[End of section] 

Appendix II: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Michael Brostek, (202) 512-9110 or brostekm@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, Tom Short, Assistant Director; 
Ray Bush; George Guttman; Lois Hanshaw; Ronald W. Jones; Veronica 
Mayhand; Ed Nannenhorn; Karen O'Conor; and Cynthia Saunders made key 
contributions to this report. 

[End of section] 

Footnotes: 

[1] 26 U.S.C. § 6306(c)(1). 

[2] The National Taxpayer Advocate is the head of the Taxpayer 
Advocate Service within IRS, which works to assist taxpayers in 
resolving individual problems and propose administrative and 
legislative changes to mitigate problems affecting groups of taxpayers. 

[3] 26 U.S.C. § 6306 note (American Jobs Creation Act of 2004, Pub. L. 
No. 108-357, title VIII, § 881(e), 118 Stat. 1418 (Oct. 22, 2004)). 

[4] When this report refers to working cases or inventory, we are 
using IRS's term for any case action taken to collect unpaid tax debt. 

[5] According to the contractor's staff, they had approximately 3 
weeks to validate the study's conclusion that IRS was more cost 
effective in collecting this debt than PCAs. The study's conclusion 
was limited to the measures and data IRS chose to study. 

[6] We requested comments about IRS's study from the two PCAs with IRS 
contracts at the time IRS released the study, of which one provided 
comments to us. 

[7] OMB Circular A-94, Guidelines and Discount Rates for Benefit-Cost 
Analysis of Federal Programs (Washington, D.C., Oct. 29, 1992). The 
goal of Circular A-94 is to promote efficient federal resource 
allocation through well-informed decision making. It applies to 
analyses used to support federal decisions to initiate, renew, or 
expand programs or projects resulting in a series of measurable 
benefits or costs extending 3 or more years into the future. The 
circular is to serve as a checklist for whether an agency has 
considered and properly dealt with all the elements for sound benefit- 
cost and cost-effectiveness analyses. The circular's guidelines are 
suggested for use in the internal planning of executive branch 
agencies. 

[8] GAO, The Evaluation Synthesis, [hyperlink, 
http://www.gao.gov/products/GAO/PEMD-10.1.2] (Washington, D.C.: March 
1992). 

[9] Joel Slemrod and Shlomo Yitzaki, The Costs of Taxation and the 
Marginal Efficiency Cost of Funds, International Monetary Fund Staff 
Papers (March 1996). 

[10] [hyperlink, http://www.gao.gov/products/GAO/PEMD-10.1.2]. 

[11] GAO, Earned Income Tax Credit: Implementation of Three New Tests 
Proceeded Smoothly, But Tests and Evaluation Plans Were Not Fully 
Documented, [hyperlink, http://www.gao.gov/products/GAO-05-92] 
(Washington, D.C.: Dec. 30, 2004). 

[12] GAO, Tax Debt Collection: IRS Is Addressing Critical Success 
Factors for Contracting Out but Will Need to Study the Best Use of 
Resources, [hyperlink, http://www.gao.gov/products/GAO-04-492] 
(Washington, D.C.: May 24, 2004). 

[13] [hyperlink, http://www.gao.gov/products/GAO-04-492]. 

[14] GAO, Tax Debt Collection: IRS Needs to Complete Steps to Help 
Ensure Contracting Out Achieves Desired Results and Best Use of 
Federal Resources, [hyperlink, 
http://www.gao.gov/products/GAO-06-1065] (Washington, D.C.: Sept. 29, 
2006). 

[15] Internal Revenue Service, IRS Private Debt Collection Cost 
Effectiveness Study (Washington, D.C., March 2009). 

[16] ACS handles IRS collections by telephone whereas its Collection 
Field Function handles more complex and in-person collection contacts. 
The study also included taxpayer satisfaction scores, but the study 
report states that the scores reflect cases beyond those in the study 
and should not be used to compare the performance of PCAs and IRS. 

[17] See [hyperlink, http://www.congress.gov/cgi-
lis/query/z?c111:H.R.796]: for H.R. 796, 111th Cong. (2009). 

[18] Omnibus Appropriations Act, 2009, Pub. L. No. 111-8, 123 Stat. 
524 (Mar. 11, 2009). 

[19] 26 U.S.C. § 7801 note (Treasury, Postal Service and General 
Government Appropriations Act, 1995, Pub. L. No. 103-329, 108 Stat. 
2382 (Sept. 30, 1994), as amended). 

[20] Consolidated Appropriations Act, 2010, Pub. L. No. 111-117, 123 
Stat. 3034 (Dec. 16, 2009). 

[21] See White House, Standards and Guidance for Statistical Surveys, 
[hyperlink, http://www.whitehouse.gov/omb/inforeg_statpolicy/#pr] 
(Washington, D.C., September 2006) (accessed Aug. 3, 2010). According 
to Standard 7.3, agencies must produce survey documentation that 
includes those materials necessary to understand how to properly 
analyze data from each survey, as well as the information necessary to 
replicate and evaluate each survey's results (see also Standard 1.2). 
Survey documentation must be readily accessible to users, unless it is 
necessary to restrict access to protect confidentiality. 

[22] The documentation that is available suggests that IRS drew a 
stratified random sample. Such a sample design involves first 
classifying the population into several groups--strata--and then 
taking a random sample from each stratum. 

[23] In a stratified sample, sample cases in different strata can 
represent a substantially different number of cases. For example, if 
the cases in the population are divided into two strata based on 
characteristics that are of interest, a case in one stratum might 
represent 20 cases in its stratum whereas a case in the other stratum 
might represent 100 cases in its stratum. In order to generalize such 
a stratified sample to the full population, cases must be "weighted" 
based on the numbers of cases in the population they represent. 
Largely for this reason, analytic results--as in the PDC study--that 
are based only on the sampled cases themselves may be significantly 
different than they would be when properly generalized back to the 
full universe from which the sample was drawn. 

[24] Sampling errors are errors associated with survey estimates that 
are due to sampling some and not all of the units in the sampling 
frame. 

[25] OMB Circular A-94. 

[26] OMB Circular A-94. 

[27] The PDC study called the third alternative the "next best use of 
funds." However, IRS officials said that they did not decide to study 
these cases because they believed working such cases would be the most 
effective and efficient strategy for achieving overall collection 
goals. Instead, IRS officials restricted the types of strategies that 
could be selected. For example, IRS officials considered attempting 
collection only on cases suitable for IRS's ACS, which handles IRS 
collections by telephone and not other collection strategies like 
information systems investments, or assigning more cases to IRS's 
Collection Field Function (which handles more complex and in-person 
collection contacts). In deciding which types of ACS cases to study, 
IRS officials also gave higher priority to cases that--like the PCA- 
type inventory--otherwise would not have been worked. For example, IRS 
officials considered the alternative of including more of the normal 
ACS inventory but instead chose to study cases not normally in ACS's 
active inventory, in part because the officials wanted to learn more 
about the potential costs and collection results for them. IRS 
normally does not take collection action on such cases because they 
require more manual processing than is deemed worthwhile. Regardless, 
IRS did not cite the results of these cases in its decision to end the 
PDC contracts or ask the independent reviewers of the PDC study to 
address the validity of the study's results for these cases. 

[28] [hyperlink, http://www.gao.gov/products/GAO-04-492]. 

[29] From an economic perspective, tax revenues are not a benefit to 
society in that they represent simply a transfer of funds from 
taxpayers to the government. Social benefits and social costs, not 
just benefits and costs to the government, are the relevant benefits 
and costs. 

[30] OMB Circular A-94. 

[31] For the third alternative IRS studied, government costs to 
collect a dollar were $0.17 and $0.25, depending on which IRS division 
worked the cases (Small Business/Self-Employed Division and Wage and 
Investment Division, respectively). 

[32] Noncompliance can lead to economic distortions as more resources, 
such as workers, are attracted to areas of the economy where 
noncompliance is relatively high. If PCAs were more effective than IRS 
in collecting taxes in particular industries or from particular types 
of business that are highly noncompliant, it might be possible to 
justify keeping PCAs on economic efficiency grounds, even if IRS was 
more effective across all the studied cases. 

[33] Beginning September 7, 2006, when IRS turned over the first cases 
to PCAs. 

[34] Variable costs fluctuate in proportion with changes in production. 

[35] OMB guidance states that retrospective studies are potentially 
valuable in determining the necessary corrections in existing programs 
and improving future estimates of results. 

[36] 26 U.S.C. § 6306 note (American Jobs Creation Act of 2004, Pub. 
L. No. 108-357, title VIII, § 881(e), 118 Stat. 1418 (Oct. 22, 2004)). 

[37] We previously reported that part of Congress's intent in 
approving the 1997 IRS PDC pilot program was to learn more about 
private sector collection techniques. Additionally, we stated that the 
program's measurement plan did not include a comparison of such best 
practices and the written design lacked a mechanism to capture 
information on the best collection practices used by PCAs that could 
be adopted by IRS. See GAO, Internal Revenue Service: Issues Affecting 
IRS' Private Debt Collection Pilot, [hyperlink, 
http://www.gao.gov/products/GAO/GGD-97-129R] (Washington, D.C.: July 
18, 1997). 

[38] In addition to the information on PCA best practices, Congress 
required that the biennial report include a cost-benefit analysis, the 
impacts of PCA contracts on IRS collection enforcement staffing and on 
unpaid assessments and assessments collected by IRS after initial PCA 
contacts, the amounts collected and collection costs incurred by IRS, 
an evaluation of contractor performance, and a disclosure safeguard 
report. 

[39] Among other things, the draft biennial report for 2007 also 
included an evaluation of contractor performance. 

[40] Inactive cases were cases for which IRS deferred or suspended 
collection action because of (1) IRS resource limitations, (2) the 
cases not meeting IRS tolerance levels for active inventory, or (3) 
IRS not being able to locate or contact the taxpayers. 

[41] Statement of Acting Commissioner of Internal Revenue Kevin M. 
Brown before the House Ways and Means Committee on May 23, 2007. 

[42] The figures IRS provided for the total cases ranged from 18,134 
to 19,065. IRS provided no explanation for the discrepancies in 
response to our requests. 

[43] [hyperlink, http://www.gao.gov/products/GAO-05-92]. 

[44] According to IRS, a FTE is the equivalent of one person working 
full-time for 1 year without overtime. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: