This is the accessible text file for GAO report number GAO-13-435 
entitled 'IRS Website: Long-Term Strategy Needed to Improve 
Interactive Services' which was released on April 16, 2013. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Chairman, Committee on Finance, U.S. Senate: 

April 2013: 

IRS Website: 

Long-Term Strategy Needed to Improve Interactive Services: 

GAO-13-435: 

GAO Highlights: 

Highlights of GAO-13-435, a report to the Chairman, Committee on 
Finance, U.S. Senate. 

Why GAO Did This Study: 

U.S. taxpayers visited the IRS’s website over 375 million times during 
2012. IRS officials noted there are over 110,000 web pages and 
downloadable documents plus some basic interactive tools, such as 
calculators, on IRS.gov. In December 2011, GAO reported that taxpayers 
benefit from the increased web services, and IRS could realize 
substantial savings by transferring taxpayers away from costly 
telephone interactions. GAO was asked to review IRS’s efforts to offer 
more interactive services. Among other things, this report describes 
the interactivity of IRS’s online services and assesses its strategy 
for expanding the interactivity of its website. To address these 
objectives, GAO reviewed IRS’s website to determine the services it 
provided as of March 2013, compared the IRS Online Strategy to federal 
guidance such as Howto.gov, and interviewed IRS officials in the 
Offices of Online Services and Information Technology. 

What GAO Found: 

The Internal Revenue Service’s (IRS) existing online services are 
limited when compared to many other government and private sector 
organizations. Static web pages make up the majority of www.IRS.gov; 
however, IRS highlights 18 basic interactive tools from its homepage. 
IRS does not offer dynamic account access to taxpayers. Other federal 
and state taxing authorities provide a broader range of online 
services to their customers, including interactive account access. 

Figure: Current and Future Online Services on IRS.gov: 

[Refer to PDF for image: illustration] 
        
Present services: 

Static web pages: 
* Forms and publications; 
* General information.   

Basic interactive tools: 
Non-secure tools: 
* Volunteer Income Tax Assistance (VITA) site locator; 
* Authorized e-file provider locator; 
Secure tools: 
* Order a Transcript; 
* Where's my Refund? 
* Online Payment Agreement. 

Future plans: 

Dynamic account access: 
* Personalized online individual account access; 
* No planned implementation date for these services. 

Source: GAO analysis of IRS information. 

[End of figure] 

To improve its website, IRS has been simultaneously (1) focusing on 
multiple short term projects to deliver new basic interactive tools 
and (2) longer term efforts to invest in foundational infrastructure, 
such as security. 

IRS does not, however, have a long term strategy for enhancing its 
website that explains how its ongoing and new efforts fit together. No 
overall cost estimate exists and there are not enough details on 
goals, deliverables, future online services, and timeframes to be able 
to assess progress. Several fundamental elements for a website 
strategy, as described on www.Howto.gov and in other guidance, are 
missing. Among others actions, IRS is not: 

* using leading practices learned to help formulate a long-term 
strategy; 

* setting a measurable taxpayer satisfaction goal to help ensure 
taxpayers’ needs are being met; 

* prioritizing the development of new online services based on 
business cases that outline the benefits and costs; and 

* linking investments in security to a long term plan. 

While IRS’s efforts to date have already benefited taxpayers and hold 
the promise of additional benefits in the future, a long term strategy 
could help managers have a common understanding of IRS’s plans, and 
better assist Congress in understanding what it is being asked to fund 
and holding IRS accountable for progress. 

What GAO Recommends: 

GAO recommends that IRS develop a long-term strategic plan for its web 
services that includes, among other things, studies of leading 
practices at a strategic level; a measurable goal for taxpayer 
satisfaction; business cases for new online services that describe the 
potential benefits and costs and prioritized projects; and links to 
investments in security. IRS agreed with two of these recommendations, 
did not state whether it agreed or disagreed on two others, and 
partially agreed with developing business cases because it believes 
other criteria should be considered. GAO believes this recommendation 
remains valid as discussed in this report. 

View [hyperlink, http://www.gao.gov/products/GAO-13-435]. For more 
information, contact James R. White at (202) 512-9110 or 
whitej@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

IRS Provides Limited Interactive Services and Is Tracking More 
Performance Metrics: 

Taxpayers' Rating of IRS's Website Is below Average and Oversight 
Entities Recommend Enhancements: 

IRS Is Following Some but Not All Fundamental Elements for a Website 
Strategy: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: IRS's Basic Interactive Tools: 

Appendix III: IRS's Component Scores for Customer Satisfaction: 

Appendix IV: Comments from the Department of Treasury: 

Appendix V: GAO Contact and Staff Acknowledgment: 

Tables: 

Table 1: Number of Visits and Downloads on IRS.gov: 

Table 2: IRS's Use of Howto.gov's Recommended Performance Metrics for 
Federal Websites, as of March 7, 2013: 

Table 3: New and Planned Self Service Tools Fiscal Years 2013 and 2014: 

Table 4: IRS e-Authentication Procedures and Identity Elements 
Required: 

Table 5: Interactive Tools for Taxpayers Available on IRS.gov as of 
March, 2013: 

Table 6: IRS's ACSI Scores for Website Elements: 

Figures: 

Figure 1: IRS.gov Homepage: 

Figure 2: Summary of Current and Future Online Services on IRS.gov: 

Figure 3: IRS's American Customer Satisfaction Index (ACSI) Scores 
2005 through 2012 Compared to Selected Private Sectors: 

Abbreviations: 

ACSI: American Customer Satisfaction Index: 

AGI: Adjusted Gross Income: 

AMT: Alternative Minimum Tax: 

DOB: Date of Birth: 

EIN: Employer Identification Number: 

EITC: Earned Income Tax Credit: 

GPRA: Government Performance and Results Act: 

GPRAMA: GPRA Modernization Act of 2010: 

IRS: Internal Revenue Service: 

IT: Information Technology: 

NIST: National Institute of Standards and Technology: 

OLS: Office of Online Services: 

OMB: Office of Management and Budget: 

PIN: Personal Identification Number: 

SSN: Social Security Number: 

VITA: Volunteer Income Tax Assistance: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

April 16, 2013: 

The Honorable Max Baucus: 
Chairman: 
Committee on Finance: 
United States Senate: 

Dear Mr. Chairman: 

U.S. taxpayers visited the Internal Revenue Service's (IRS) website 
over 375 million times during 2012 to obtain tax-related information, 
download forms, and check on the status of a refund, among other 
things.[Footnote 1] IRS officials noted that there are over 110,000 
web pages and downloadable documents available on IRS.gov. Enhancing 
IRS.gov with additional online services is cited by the Department of 
Treasury as part of how it intends to achieve its priority goals of 
improving service to taxpayers and increasing compliance with tax 
laws.[Footnote 2] Taxpayers benefit from online services because they 
can research large amounts of tax guidance, the services are available 
24 hours a day, and there is no waiting in a queue to speak to a 
telephone representative. In December 2011, we reported that IRS could 
realize savings by transferring taxpayers from costly telephone and 
walk-in services to much cheaper online services.[Footnote 3] 

While IRS's website is heavily used and provides access to vast 
amounts of information, concerns have been expressed about the 
sophistication of the services provided and the extent to which 
taxpayers' needs are met. Much of the content on IRS's website is read-
only. There are some basic tools that allow users to request 
personalized information, such as the status of refunds, but IRS's 
website does not give taxpayers interactive personal account access. 
The IRS Oversight Board, the National Taxpayer Advocate, and the 
Electronic Tax Administration Advisory Committee have all recently 
made recommendations that IRS should provide taxpayers with online 
access to their accounts, including ways to resolve compliance 
problems or other issues.[Footnote 4] In 2011, we recommended that IRS 
complete a strategy that includes an assessment of providing online 
interactive tools that allow taxpayers to access and update elements 
of their accounts online.[Footnote 5] IRS stated it would consider 
online access for taxpayers, but as of March 2013, IRS has not yet 
taken action to fully address the recommendation. 

You asked us to review IRS's efforts to offer more interactive 
services. This report (1) describes the interactivity of IRS's online 
services and assesses how IRS measures use of the services, (2) 
describes taxpayers' satisfaction with these services, and (3) 
assesses IRS's strategy for expanding the interactivity of its 
website, including how it learns from other organizations, sets goals 
for taxpayer satisfaction for its website, prioritizes the development 
of new services, and addresses security risks. 

To accomplish our objectives, we reviewed IRS.gov to determine the 
different types of online services IRS offered as of March 2013, and 
analyzed the performance metrics IRS uses to monitor taxpayer 
satisfaction with the website and its interactive tools. We compared 
IRS's performance metrics to the recommended performance metrics on 
Howto.gov--a website managed by the General Services Administration to 
guide federal agencies to provide better customer experiences through 
their websites.[Footnote 6] We reviewed IRS's average American 
Customer Satisfaction Index (ACSI) scores from 2005 through 2012 as 
gathered by ForeSee via optional online surveys.[Footnote 7] We 
compared IRS's score to the average scores of other federal agencies 
and private organizations.[Footnote 8] We analyzed the September 2012 
IRS Online Strategy and updates of the strategy provided March 19, 
2013. We reviewed IRS's records of contact with other organizations to 
determine the steps taken to learn about leading practices when 
implementing interactive tools, and we compared IRS's website 
management practices to best practices outlined on Howto.gov. We 
interviewed Office of Online Service (OLS) officials to determine 
IRS's plans to set a taxpayer satisfaction goal. We assessed how IRS 
prioritized new interactive tools for implementation in fiscal year 
2013 by comparing the planned time frames listed in the IRS Online 
Strategy to actual implementation dates of the interactive tools and 
reviewing OLS's documentation used to determine prioritization, 
including business cases. We compared the business cases to benefit 
cost criteria, including the Office of Management and Budget's (OMB) 
Circular A-94. We obtained and reviewed IRS's security risk 
assessments for two interactive tools--Send My Transcript to Banks and 
First Time Homebuyers Credit--and its planned actions to address any 
risks it identified. We also reviewed IRS's plans to authenticate 
taxpayer identities for new interactive tools and compared them to the 
National Institute of Standards and Technology's (NIST) guidance on 
authentication.[Footnote 9] For all objectives we interviewed relevant 
IRS officials to collect information on IRS's online services and its 
plans to implement additional online services in the future. We 
reviewed documents and interviewed ACSI officials and determined that 
the ASCI data presented in our report for years 2005 through 2012 were 
sufficiently reliable for our purposes. (See appendix I for more 
information on our scope and methodology.) 

We conducted this performance audit from June 2012 to April 2013 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

Background: 

IRS.gov offers users access to a wide range of guidance, forms, and 
sources for help, as indicated by the screen shot of the home page 
shown in figure 1. According to the IRS Oversight Board's annual 
taxpayer attitude survey, 86 percent of taxpayers were likely to use 
technology-based services, like the IRS website, and this percentage 
has grown in recent years.[Footnote 10] 

Figure 1: IRS.gov Homepage: 

[Refer to PDF for image: web page illustration] 

Source: Internal Revenue Service. 

Note: IRS.gov homepage as of March 6, 2013. 

[End of figure] 

Based on our review of literature on interactive online services, we 
identified three categories of such services, which we call (1) static 
web pages, (2) basic interactive tools, and (3) dynamic account access. 

* Static web pages: web pages that provide the same information for 
all users. These pages are mostly read-only, although users may be 
able to use hyperlinks to move to other related content. The pages do 
not have the capacity to return any personalized information to the 
user. 

* Basic interactive tools: more complex web pages that allow users to 
perform routine tasks online and, in some cases, obtain personalized 
data. These tools can be non-secure or secure. 

- Non-secure tools: basic interactive tools that do not require users 
to confirm their identity in order to use the tool. There is also no 
interaction between the user and the user's tax account at IRS. The 
tools may allow the taxpayer to input some data and in return get back 
personal information based on the information provided. A calculator 
for estimating tax is one example. 

- Secure tools: basic interactive tools that allow users to obtain 
some information from their account after confirming their identity. 
While users can request some information about their account, the 
information is limited and users do not have the ability to make 
changes to the account. For example, one of IRS's secure tools allows 
taxpayers to request a copy of a previously filed tax return. 

* Dynamic account access: the most complex web pages that provide 
users with real-time account access and the ability to make account 
changes after confirming their identity. For example, the Social 
Security Administration allows users to start or change direct deposit 
benefit payments. 

The Office of Online Services: 

Created in 2011, OLS is charged with developing IRS's overall website 
strategy and operations and creating new services to increase website 
use. Officials developed the IRS Online Strategy, a PowerPoint slide 
presentation, to help govern the steps OLS plans to take to develop 
future online services. Officials noted that the strategy is a living 
document that was meant to evolve as priorities changed. OLS has an 
operating budget for the management of its staff, but it does not have 
a specific funding source to develop online services and generally 
must rely on off-budget cycle funding requests presented to the IRS 
Technical Investment Review Board.[Footnote 11] 

Since OLS's establishment, officials have been focused mainly on 
modernizing IRS.gov, including updating the technical infrastructure 
that supports the website and also lays the groundwork for the next 
phase of online services. In addition to technical upgrades, OLS's 
main goals have been to improve website content and design and to 
optimize the search capability on the website. OLS officials pursued a 
limited set of new interactive tools during their first two years to 
realize short-term improvements for taxpayers. Prior to the 
establishment of OLS, IRS business units, primarily Wage and 
Investment, developed online services for the IRS website based on 
specific needs. 

Guidance for Websites: 

While there is no single authoritative source or governance board for 
federal website development and management, several sources of 
guidance exist. 

* Howto.gov: A website managed by the General Services Administration 
and designed to be a resource to help improve how agencies communicate 
and interact with customers and use innovative tools and technologies 
to improve service. Howto.gov offers best practices, training, and 
guidance on strategic planning; federal web requirements and policies; 
applications, data, and web infrastructure tools; web content 
management, usability, and design; and performance metrics. 

* Digital Government-Building a 21st Century Platform to Better Serve 
the American People: A directive issued by the President on May 23, 
2012, aimed at guiding agencies to deliver better digital services. 
Its main objectives are to (1) enable the American people to access 
high-quality digital government information and services anywhere, 
anytime, on any device; (2) ensure that the government adjusts to this 
new digital world; and (3) spur innovation. 

* National Institute of Standards and Technology (NIST): Part of the 
U.S. Department of Commerce, NIST is charged with providing guidance 
and assistance to agencies, defining minimum security requirements, 
identifying methods of assessing the effectiveness of security 
requirements, and evaluating security policies and technologies from 
the private sector and national security systems to potentially 
implement in federal agencies, among others. 

* IRS's Internal Revenue Manual: IRS publishes its information 
security policies and guidance and project assurance standards and 
procedures in the Internal Revenue Manual.[Footnote 12] 

* OLS's internal guidance: According to the guidance, before new 
interactive tools are released for taxpayer use, they should go 
through multiple stages of a product management process including 
evaluating new product ideas; baselining the current state; 
identifying a potential set of online solutions; developing business 
cases and securing funding; and building, piloting, and launching new 
tools. According to the Director of OLS, the product management 
process identifies future online opportunities, assesses the value of 
a product, and develops a business case to justify the need for an 
interactive tool. 

Other guidance is also available to federal agencies to help managers 
make resource decisions when implementing new programs, including 
websites. The OMB Circular A-94 provides federal program managers with 
guidance to help promote efficient resource allocation through well-
informed decision making.[Footnote 13] It provides guidance for 
conducting benefit cost and cost-effectiveness analyses, and can serve 
as a checklist of all the elements to be considered. 

IRS Provides Limited Interactive Services and Is Tracking More 
Performance Metrics: 

IRS's Website Does Not Offer Dynamic Account Access: 

IRS's existing online services are limited when compared to many other 
organizations. According to IRS officials, static web pages make up 
the majority of content on IRS.gov. IRS offers some basic interactive 
tools, 18 of which are highlighted from its homepage as of March 2013. 
(See appendix II for a description of the 18 basic interactive tools.) 
IRS is not currently positioned to offer dynamic account access to 
taxpayers.[Footnote 14] Figure 2 provides some examples. 

Figure 2: Summary of Current and Future Online Services on IRS.gov: 

[Refer to PDF for image: illustration] 
        
Present services: 

Static web pages: 
* Forms and publications; 
* General information.   

Basic interactive tools: 
Non-secure tools: 
* Volunteer Income Tax Assistance (VITA) site locator; 
* Authorized e-file provider locator; 
Secure tools: 
* Order a Transcript; 
* Where's my Refund? 
* Online Payment Agreement. 

Future plans: 

Dynamic account access: 
* Personalized online individual account access; 
* No planned implementation date for these services. 

Source: GAO analysis of IRS information. 

[End of figure] 

* Static web pages: Most of IRS's web pages are static. According to 
IRS officials, there are over 30,000 static web pages on IRS.gov 
(i.e., general information for taxpayers) and more than 80,000 static 
files (i.e., downloadable documents such as tax forms) with more 
content being added daily. These pages only change when the agency 
updates the information, such as a revised tax form or publication. 

* Basic interactive tools: IRS offers a few basic interactive tools, 
both non-secure and secure. IRS highlights a number of basic 
interactive tools for taxpayers on its website (see appendix II). 

- Non-secure tools: IRS offers several non-secure tools. For example, 
a taxpayer who would like to determine the potential deduction amount 
for a state or local sales tax can use the Sales Tax Deduction 
Calculator and answer a series of questions to determine the deduction 
estimate. The website also offers a number of calculators to help 
taxpayers determine either the correct deduction amount or amount of 
tax owed. 

- Secure tools: IRS offers a few basic secure tools for taxpayers. For 
example, to receive updates on refund status using the Where's My 
Refund? tool, a taxpayer must provide his or her Social Security 
number, filing status, and refund amount. In another example, the 
Order a Transcript tool allows taxpayers to request that a copy of a 
prior year tax return be mailed to the taxpayer's address on file. 

* Dynamic account access: IRS is not currently positioned to offer 
dynamic account access to taxpayers that would allow, for example, a 
taxpayer to access his or her tax account balance, view recent 
payments, or correspond digitally with IRS to respond to notices 
similar to other federal and private sector organizations that provide 
a broad range of services to their customers. 

Other federal agencies, states, and countries provide their taxpayers 
with dynamic account access features. For example, the Social Security 
Administration offers users the ability to create a personalized My 
Social Security account. Through their individual accounts, users can 
get their Social Security statements to review estimates of 
retirement, disability, and survivors' benefits; earnings records; and 
estimated Social Security and Medicare taxes paid. Those receiving 
benefits can also use their dynamic account access to get benefit 
verification letters; check benefit and payment information and 
earnings records; change address and phone number; and start or change 
direct deposit benefit payments. Medicare allows beneficiaries to 
access their account through MyMedicare.gov. This secure site allows 
users to check their eligibility status, enrollment status, and 
Medicare benefits. Users can also access claims information and 
receive preventative health information. 

In addition, the New York and California state tax agencies provide 
dynamic account access allowing taxpayers to view tax account balances 
and recent payments; respond to notices; and change 
addresses.[Footnote 15] Finland encourages accurate withholding of 
taxes from taxpayers' income through an online system called the Tax 
Card that covers the national tax, municipality tax, social security 
tax, and church tax.[Footnote 16] Taxpayers can access the Tax Card 
through a taxpayer's secured bank system or by a government issued 
access card. Taxpayers can also update withholding rates and revise 
government-prepared income tax returns online. 

IRS Management Is Tracking More Recommended Website Performance 
Metrics and Is Considering Others, but Has Not Yet Determined a Final 
Suite: 

For years, IRS has tracked and reported on a limited number of website 
metrics such as those shown in table 1. 

Table 1: Number of Visits and Downloads on IRS.gov: 

Total visits[A]; 
2009: 299.0 million; 
2010: 307.5 million; 
2011: 322.5 million; 
2012: 375.5 million; 
2009--2012 percent change: 25.6%; 
2011--2012 percent change: 16.4%. 

Downloads[B]; 
2009: 203.8 million; 
2010: 216.8 million; 
2011: 247.5 million; 
2012: 358.7 million; 
2009--2012 percent change: 76.0%; 
2011--2012 percent change: 44.9%. 

Source: IRS data, based on calendar years. 

[A] Total visits is a count of the number of times taxpayers visited 
IRS.gov. 

[B] Downloads is a count of the number of taxpayer forms, 
publications, or instructions that were downloaded to a taxpayer's 
computer from IRS.gov. 

[End of table] 

Within the last 3 years, IRS has started to track additional 
performance metrics. As of early March 2013, OLS management tracked 
and used 18 of the 24 metrics recommended by Howto.gov. Specifically, 
OLS fully tracked 15 of the 18 measures and partially tracked 3 
measures (i.e., for some web pages). Six of the 24 metrics were not 
tracked or used by management (see table 2). Howto.gov calls the 
measures in table 2 a minimum baseline set of performance and customer 
satisfaction measures, which allow agency officials to get a holistic 
view of how well online services and information are delivered and to 
prioritize improvements. 

Table 2: IRS's Use of Howto.gov's Recommended Performance Metrics for 
Federal Websites, as of March 7, 2013: 

Website performance: 

Performance metric: 1.Total visits; 
Description: An interaction by an individual viewing one or more pages 
on a website; 
IRS tracking: IRS fully tracks this performance metric. 

Performance metric: 2. Total page views; 
Description: The number of times a page was viewed during the 
designated time period (e.g., monthly); 
IRS tracking: IRS fully tracks this performance metric. 

Performance metric: 3. Unique visitors; 
Description: One person (or a computer/IP address) who visits a website 
at least once during a designated time period (e.g., monthly); 
IRS tracking: IRS does not track this performance metric. 

Performance metric: 4. Page views per visit; 
Description: The number of page views in a reporting period divided by 
the number of visits in the same reporting period; 
IRS tracking: IRS fully tracks this performance metric. 

Performance metric: 5. Average visit duration; 
Description: The length of time in a session (activity on a website 
during a specific time period); 
IRS tracking: IRS fully tracks this performance metric. 

Performance metric: 6. Time on page; 
Description: The time visitors spend on individual pages; 
IRS tracking: IRS tracks this performance metric and uses it for only 
some pages. 

Performance metric: 7. Bounce rate; 
Description: The percentage of visitors who looked at only one page 
and immediately left the site; 
IRS tracking: IRS tracks this performance metric and uses it for only 
some pages. 

Performance metric: 8. New vs. returned visitor; 
Description: The ratio between first-ever site visitors to returning 
visitors (i.e., a user that visited the site prior to the reporting 
period); 
IRS tracking: IRS fully tracks this performance metric. 

Performance metric: 9. Visits per visitor for a given time frame; 
Description: The number of times a visitor visited 
a website during the designated time period (e.g., monthly); 
IRS tracking: IRS does not track this performance metric. 

Performance metric: 10. Total number of on-site search queries; 
Description: The total number of times the website's search function 
was used; 
IRS tracking: IRS fully tracks this performance metric. 

External search metrics: 

Performance metric: 11. Top referring commercial search engines; 
Description: The search engines (e.g., Google, Bing, Yahoo) that send 
the most traffic to the site; 
IRS tracking: IRS fully tracks this performance metric. 

Performance metric: 12. Top referring search terms; 
Description: The most-used words and phrases people type into 
commercial search engines to find content; 
IRS tracking: IRS does not track this performance metric. 

Performance metric: 13. Top referring search terms with low click 
through rates; 
Description: The most common links followed for specific search terms 
that don't lead to clicks on the website; 
IRS tracking: IRS does not track this performance metric. 

Internal/site search: 

Performance metric: 14. Top search terms/phrases; 
Description: The most-used words and phrases people type into the 
website's search box; 
IRS tracking: IRS fully tracks this performance metric. 

Performance metric: 15. Top "no results" queries; 
Description: Most popular, valid searches that don't return any results 
on the website's search results page (e.g., because the agency does not 
have the content on the website, or the content is not findable by the 
person's search term); 
IRS tracking: IRS fully tracks this performance metric. 

Performance metric: 16. Top searches with low click through rates; 
Description: Most popular search terms that people generally don't act 
(click) on; 
IRS tracking: IRS does not track this performance metric. 

Performance metric: 17. Top changing search terms (movement up/down); 
Description: Shows trending topics, what's hot or not; 
IRS tracking: IRS fully tracks this performance metric. 

Performance metric: 18. Percentage of visitors using site search; 
Percentage of people who visited the website and used the search box 
on the site; 
IRS tracking: IRS tracks this performance metric and uses it for only 
some pages. 

Performance metric: 19. Top clicked URLs for specific queries; 
Description: Most common results that searchers found most relevant or 
appealing, for specific search terms; 
IRS tracking: IRS does not track this performance metric. 

Speed: 

Performance metric: 20. Page load time; 
Description: How quickly pages (including the search results page) load 
on the website; 
IRS tracking: IRS fully tracks this performance metric. 

Customer satisfaction: 

Performance metric: 21. Overall customer experience; 
Description: Customer's perception of the experience of his or 
her visit; 
IRS tracking: IRS fully tracks this performance metric. 

Performance metric: 22. Completion rate of intended task; 
Description: The customer's perceived ability to get the information 
or service he or she came to accomplish; 
IRS tracking: IRS fully tracks this performance metric. 

Performance metric: 23. Percent of visitors likely to return; 
Description: A customer's perceived willingness to choose the service 
for the same or similar task in the future; 
IRS tracking: IRS fully tracks this performance metric. 

Performance metric: 24. Percent of visitors likely to recommend; 
Description: A customer's perceived willingness to recommend the 
product or service; 
IRS tracking: IRS fully tracks this performance metric. 

Source: GAO analysis of IRS data from IRS.gov Site Usage Report 
(January 2013) and customer satisfaction survey questions. 

[End of table] 

In addition to increasing the number of website metrics they track, 
OLS managers told us they are making or considering other changes to 
measuring performance. OLS managers noted that they changed the 
program used to collect performance data in August 2012, and the new 
program provides the capability to obtain a more comprehensive view of 
IRS's online performance.[Footnote 17] They also said that some OLS 
staff recently started tracking 4 of the remaining 6 metrics in table 
2, although the metrics are not monitored by OLS management and they 
have not determined if all of the existing measures as well as the 4 
they are considering will be tracked long-term.[Footnote 18] 
Furthermore, OLS management stated that not all the Howto.gov metrics 
may be relevant, in their entirety, for major federal websites. They 
said they plan to focus on the metrics that help improve the 
taxpayer's experience with the website. We agree that learning more 
about taxpayers' experience with the website is important. However, 
the documents OLS provided did not discuss how IRS intends to do this, 
what measures would be included in the complete final set of 
performance metrics, or when the decisions of what to include would be 
made. Because IRS's website is highly used and enhancing it with 
additional online services is a key goal to improve taxpayer service 
and compliance, it is important to have a full set of performance 
metrics as a baseline. As noted by Howto.gov, without a baseline set 
of performance metrics in place, managers will not have a complete 
picture of the extent to which taxpayers' needs are being met and will 
lack information useful for making decisions about website 
improvements. 

Taxpayers' Rating of IRS's Website Is below Average and Oversight 
Entities Recommend Enhancements: 

According to one survey, IRS's website has consistently ranked below 
the average of websites from other federal agencies and many sectors 
of the private economy including banks, Internet retailers, and 
Internet brokerage firms in customer satisfaction as shown in figure 
3.[Footnote 19] IRS uses the American Customer Satisfaction Index 
(ACSI) to measure taxpayers' satisfaction with IRS.gov. ACSI is an 
independent national model of customer satisfaction measured by 
website users who volunteer to complete a survey and answer questions 
determining elements of satisfaction such as customer expectations, 
perceived quality, and value.[Footnote 20] While ACSI survey results 
are not generalizable to all website users and subject to the 
limitations of a convenience sample, IRS has scored below the average 
of other federal government websites since 2008. A score of 80 or 
above (on its 100 point scale) refers to top performing websites, 
according to ACSI's model.[Footnote 21] The score for IRS.gov has 
ranged from an annual average high of 73 in 2007 to an annual average 
low of 67 in 2012. Judgments about the quality of IRS's website should 
not be based on ACSI scores alone--other organizations have different 
missions and users. OLS officials told us that given its new website 
content and design, they expected their scores to fall in 2012, as 
taxpayers had to adapt to the new site.[Footnote 22] Further, 
officials noted that they also track 10 individual components of the 
ACSI customer satisfaction scores, such as taxpayers' views on ease of 
navigation and search, to gain more insights into taxpayers' 
experience. These component scores ranged from 68 to 84 in 2012 with 
the majority of scores in the 70's (see appendix III). 

Figure 3: IRS's American Customer Satisfaction Index (ACSI) Scores 
2005 through 2012 Compared to Selected Private Sectors: 

[Refer to PDF for image: multiple line graph] 

Year: 2005; 
Government portal: 73; 
ACSI Internet Retail: 81; 
ACSI Internet Brokerage Firms: 76; 
IRS.gov: 69; 
Banks: 75. 

Year: 2006; 
Government portal: 75; 
ACSI Internet Retail: 83; 
ACSI Internet Brokerage Firms: 78; 
IRS.gov: 72; 
Banks: 77. 

Year: 2007; 
Government portal: 73; 
ACSI Internet Retail: 83; 
ACSI Internet Brokerage Firms: 79; 
IRS.gov: 73; 
Banks: 78. 

Year: 2008; 
Government portal: 73; 
ACSI Internet Retail: 82; 
ACSI Internet Brokerage Firms: 74; 
IRS.gov: 72; 
Banks: 75. 

Year: 2009; 
Government portal: 75; 
ACSI Internet Retail: 83; 
ACSI Internet Brokerage Firms: 78; 
IRS.gov: 72; 
Banks: 75. 

Year: 2010; 
Government portal: 75; 
ACSI Internet Retail: 80; 
ACSI Internet Brokerage Firms: 78; 
IRS.gov: 71; 
Banks: 76. 

Year: 2011; 
Government portal: 75; 
ACSI Internet Retail: 81; 
ACSI Internet Brokerage Firms: 76; 
IRS.gov: 70; 
Banks: 75. 

Year: 2012; 
Government portal: 74; 
ACSI Internet Retail: 82; 
ACSI Internet Brokerage Firms: 78; 
IRS.gov: 67; 
Banks: 77. 

Source: GAO analysis of ACSI data. 

Note: These estimates are subject to sampling error, but the precise 
confidence level is not known. It is also unknown whether these 
estimates are statistically significant. The overall score represents 
users' responses to three questions about their overall satisfaction 
with the web sites being surveyed. The three questions are: "What is 
your overall satisfaction with this site?"; "How well does this site 
meet your expectations?"; and "How does this site compare to your idea 
of an ideal website?" 

[End of figure] 

The reviews of external oversight organizations provide evidence 
corroborating the message in the ACSI scores for IRS.gov. In the last 
2 years, three separate oversight entities have found that IRS's 
website, and especially the interactive services offered, could be 
improved. The Electronic Tax Administration Advisory Committee, IRS 
Oversight Board, and the National Taxpayer Advocate have all recently 
stated that IRS needs to develop a more modern website and provide 
more interactive services such as dynamic account access to meet the 
needs of taxpayers.[Footnote 23] 

IRS Is Following Some but Not All Fundamental Elements for a Website 
Strategy: 

Since its inception, OLS has been simultaneously (1) focusing on short 
term projects to deliver new basic interactive tools (what OLS 
managers described as "low hanging fruit") and (2) longer term efforts 
to stand up the organization and invest in foundational 
infrastructure, such as security. In 2012 and 2013, OLS has delivered 
seven interactive tools, including First Time Homebuyer Credit, Offer 
in Compromise Pre-qualifier, and e-Transcripts for Banks. OLS 
officials said they are planning to deliver four additional tools in 
2014. Efforts to stand up the organization have included establishing 
a governance structure and hiring staff. Foundational infrastructure 
improvements recently implemented include the redesign of the static 
website, which improved content, design, and search capabilities. Two 
foundational infrastructure investments underway are projects to 
develop more secure internet portals for accessing information and 
improve IRS's ability to authenticate taxpayers' identities. In 2009, 
IRS estimated that it would spend $320 million to upgrade and maintain 
its website over a 10-year period. 

IRS does not have a long-term strategy for its website that explains 
how all of these ongoing and new efforts over the next several years 
fit together. IRS has a strategy document (IRS Online Strategy), but 
it does not contain any overall cost estimates or enough detail on 
goals, deliverables, future online services, or timeframes so that 
Congress can hold IRS accountable for making progress over the long-
term. The document itself is labeled "draft/pre-decisional," and much 
of the content is described as illustrative. Information about new 
services and benefits to taxpayers beyond 2014 is at times referred to 
as an aspiration or "to be determined." OLS provided us with a large 
number of documents providing a range of detail about IRS's website 
improvement efforts. For example, in March 2013, IRS provided us an 
Excel spreadsheet describing "multi-year themes" for new services it 
plans to implement through 2015, but it is not consistent with the IRS 
Online Strategy. 

As we have previously reported, the Government Performance Results Act 
of 1993 (GPRA), as amended by the GPRA Modernization Act of 2010 
(GPRAMA) can serve as leading practices for planning at lower levels 
within federal agencies, such as individual programs or initiatives. 
[Footnote 24] Among other things, GPRAMA states that strategic plans 
should contain the goals and objectives of a program along with 
measures of performance so that an agency can be held accountable. It 
also describes what is needed to carry out the strategy including 
resource needs and milestones over the long term. We recognize that a 
strategy is necessarily high level and that proposals for out year 
efforts may not be fully developed. This is especially true for 
technology-related investments. However, without a strategy that 
provides an overall picture of what IRS is investing in, it will be 
difficult for Congress and other decision makers to understand up 
front what they are funding and what benefits they can expect. Because 
some of the costs being incurred today are for foundational efforts, 
it is even more important that a strategy explain the long-term 
benefits. 

To be more specific, IRS is missing important parts of the following 
four elements that Howto.gov and other guidance describe as key parts 
of a website strategy: 

* using leading practices to inform strategic decision making; 

* setting goals for improving customer satisfaction; 

* developing business cases with estimates of benefits and costs and 
using the information to prioritize new investments; and: 

* linking security investments to the rest of the strategy . 

IRS Reviews Other Organizations' Interactive Tool Designs, but Has Not 
Used Leading Practices to Inform Its Strategic Plan: 

IRS has taken steps to learn about website design, such as layout, 
page flow, and usability for specific interactive tools. For example, 
OLS staff attended a presentation given by the New York State 
Department of Taxation and Finance to learn about the online services 
account page. OLS officials also reviewed the websites of the 
California Franchise Tax Board and the Internal Revenue Authority of 
Singapore, which both utilize secure email technologies. Additionally, 
IRS has used a private research firm to help explore new options for 
online tools, such as improving taxpayers' ability to make online 
payments on IRS.gov. Also, Where's My Refund? and Where's My Amended 
Return? are IRS tools that were derived from private sector tracking 
tools (e.g., online tools that track the location of mailed packages). 
IRS officials told us they are also part of a Government Services 
Administration community of practice which focuses on website content. 

OLS officials did not provide us with analysis showing how they used 
leading practice reviews to help make long-term strategic decisions 
about, for example, how much and what type of dynamic account access 
to offer. In addition, IRS is not using leading practice reviews to 
help inform prioritization and has had difficulty adhering to 
timeframes based on its implementation decisions. For example, IRS 
rated the Online Payment Agreement tool highest of all new tools 
scheduled for implementation in fiscal year 2012 to fiscal year 2014 
based on projected impact and ease of implementation. However, the 
business case was in draft form as of March 2013, and implementation 
is now scheduled for fiscal year 2014, whereas other tools with a 
lower level of priority have been implemented. Federal guidance, such 
as OMB's Streamlining Service Delivery and Improving Customer Service, 
states that government managers should learn what is working in the 
private sector and apply these leading practices to deliver services 
better, faster, and cheaper, including improvements to interactive 
tools.[Footnote 25] Howto.gov and OLS's Product Management Process 
provide similar guidance. 

OLS officials told us that because IRS is so far behind on the web 
services it offers on IRS.gov, they have enough ideas for website 
improvements and do not always need to look to other organizations for 
a list of improvements. OLS officials further stated that as an 
organization, IRS is unique and that the lessons learned from other 
organizations cannot always be directly applied to online services for 
IRS.gov. In addition, officials told us that staff responsible for 
developing the interactive tools may have prior private industry 
experience. Therefore, IRS relied on staff knowledge. 

However, learning from other organizations about their strategy for 
providing online services could help IRS improve its own services. 
Looking at other organizations' experiences, including those of state 
and foreign tax agencies, and lessons learned could help IRS 
prioritize future projects by identifying which tools are providing 
users, including other taxpayers, with the most benefit. 

IRS Does Not Have a Long-Term Measurable Goal for Taxpayer 
Satisfaction with Its Website: 

IRS does not have a long-term numerical or other measurable goal 
including a related timeframe to increase its below average taxpayer 
satisfaction with its website. A key attribute of a successful 
performance metric is to have a measurable target with a numerical 
goal.[Footnote 26] Numerical or measurable targets facilitate future 
assessments of whether overall goals and objectives were achieved 
because comparisons can be easily made between projected performance 
and actual results. OLS officials stated that their long-term goal is 
"to increase satisfaction over a period of time." They stated the time 
period is not specific because they want to look at trends that 
pertain to the redesigned IRS.gov before making meaningful long-term 
goals.[Footnote 27] We understand that examining trends can be helpful 
in determining meaningful long-term goals; however, it is difficult to 
measure progress without establishing a period of time for which to 
measure such progress. Without a numerical goal or other measurable 
target and a related timeframe for achieving the goal, IRS will have 
difficulty determining whether its long-term taxpayer satisfaction 
plan is successful. 

Business Cases for New Interactive Tools Are Missing Useful 
Information and Are Not Being Used to Set Priorities: 

The September 2012 IRS Online Strategy outlined eight new interactive 
tools to implement in fiscal year 2013. (See table 3.) Subsequent to 
the issuance of this strategy, IRS modified the implementation date of 
four of these tools to fiscal year 2014. In a more recent document, 
IRS shows most of these tools as being implemented in multiple phases 
over the next 2 years. However, the documentation is not clear as to 
how many phases will be needed for each tool until it is complete. 

Table 3: New and Planned Self Service Tools Fiscal Years 2013 and 2014: 

Implementation in fiscal year 2013: 

Online interactive tool: Where's My Refund (2013); 
Description: Improves taxpayers ability to check refund status across 
the filing process. 

Online interactive tool: Taxpayer Protection Unit with Out of Wallet 
Authentication; 
Description: Enables online authentication of taxpayers who have been 
identified as potential fraud victims. 

Online interactive tool: Offer in Compromise Pre-qualifier; 
Description: Assists taxpayers in determining eligibility for offers in 
compromise. 

Online interactive tool: Where's My Amended Return; 
Description: Allows taxpayers the ability to check the status of 1040-
X Amended returns. 

Projected implementation in fiscal year 2014: 

Online interactive tool: Automated Clearing House Debit; 
Description: Allows taxpayers to electronically debit fund from bank 
accounts. 

Online interactive tool: e-Transcripts View and Print; 
Description: Enables taxpayers to view and print their tax transcript 
online. 

Online interactive tool: Online Payment Agreement Enhancements; 
Description: Improves the usability of exiting online installment 
agreement tool. 

Online interactive tool: Direct Authorization of Power of Attorney; 
Description: Allows individuals with power of attorney to request 
transcripts. 

Source: IRS Online Strategy (September 2012). 

Note: The Fiscal Year 2014 President's Budget for IRS, which was 
released on April 10, 2013, requests funding for e-Transcripts View 
and Print, Online Payment Agreement Enhancements, and Direct 
Authorization of Power of Attorney. 

[End of table] 

IRS provided us with business cases for some of its new interactive 
tools, but these cases lacked enough information about benefits and 
costs to be useful for setting priorities--either across potential 
investments IRS-wide (both web related and non-web related) or across 
potential interactive tool investments.[Footnote 28] Benefits were 
often described on a per taxpayer basis without any indication of how 
many taxpayers might be expected to use a tool. In some cases 
descriptions about potential benefits were inconsistent across 
documents. For example, for the Automated Clearing House Debit tool, 
one document states that the tool will provide a free payment option 
for taxpayers, but benefits such as this are not listed in another 
document. Another document describing the features of the e-
Transcripts View and Print tool states taxpayers will be able to 
export data to Excel, but this is not mentioned in the business case. 
Costs were also not listed consistently. In some cases costs were 
provided for one year, in others for multiple years. Some estimates 
covered only developmental costs; others covered operating costs as 
well. In no case was there any link to overall costs for website 
modernization, including to the $320 million dollars mentioned above 
for upgrading website infrastructure. Consequently, it is not clear 
what the overall costs including infrastructure and tool development 
will be. Furthermore, it was not clear from the business cases if the 
costs were for the first or later phases. Because of these 
inconsistencies and other issues about the benefits and costs, we are 
not listing in this report any of the estimates we found in the IRS 
documentation. 

IRS is not prioritizing which interactive tools it will develop based 
on benefits and costs. There are two reasons for this. First, as just 
mentioned, basic information about benefits and costs is not provided 
or is inconsistent. Second, IRS established priorities before 
developing the full set of business cases. For example, IRS originally 
planned to implement all of the tools in table 3 in 2013, but by mid-
March officials had provided us with business cases for moving forward 
with only four of the tools. According to the Director of OLS, the 
eight new interactive tools listed in the September 2012 IRS Online 
Strategy were a "wish list" for fiscal year 2013, and not all of them 
had been approved for development at the time the strategy was 
developed.[Footnote 29] 

The IRS Online Strategy shows OLS prioritizes the implementation of 
new interactive tools based on projected impact (improving taxpayer 
service; reducing use of other high costs services such as telephone; 
and increasing collection revenue) and ease of implementation 
(technical, organizational, dependence on other IRS projects, and 
risks). However, it is unclear whether IRS followed this 
prioritization methodology. For example, the Online Payment Agreement 
tool was rated the highest priority of all new tools scheduled for 
implementation in fiscal years 2012 to 2014 based on projected impact 
and ease of implementation. However, the business case was in draft 
form as of March 2013, and implementation is now scheduled for fiscal 
year 2014. OLS officials stated that they must work with multiple IRS 
divisions and management to prioritize new information technology (IT) 
demands, while balancing among investing in new systems, maintaining 
existing systems, and operating a large technology infrastructure. 
However, situations where multiple proposed investment projects have 
to compete with each other for limited funds are exactly when using 
business cases to rank projects can be useful.[Footnote 30] 

OMB's Circular A-94 provides guidance to agencies on how to develop 
benefit and cost data. This guidance suggests that analyses be used to 
support decisions to initiate, renew, or expand programs or projects 
which would result in measurable benefits or costs. Without business 
cases that explicitly outline the benefit cost analysis for proposed 
interactive tools, it will be difficult for IRS to know which 
additional services would benefit taxpayers the most and the costs of 
those services.[Footnote 31] Our previous work has shown that results 
of benefit cost analyses help agencies decide which new projects to 
start in a manner that maximizes the benefits derived from agency 
resources.[Footnote 32] 

IRS's Current Security Capabilities Limit Its Ability to Increase 
Interactive Web Services, and Investments in Security Upgrades Are Not 
Clearly Linked to a Long-Term Plan: 

New online services go through IRS processes to assess security before 
being released for taxpayer use. While IRS has upgraded its 
capabilities to authenticate taxpayers' identities, it is missing some 
of the capabilities needed to ensure the security of either new, basic 
interactive tools, or dynamic account access. While investments in 
security upgrades are underway, they are not linked to long-term plans 
that describe security needs. 

IRS's Security Risk Assessment Process Is Not Consistently Followed 
When Implementing Interactive Tools: 

IRS has processes in place to identify and assess information security 
risks for its new interactive tools in accordance with NIST guidance. 
However, for the two interactive tools we reviewed, the implementation 
was inconsistent. IRS uses a detailed methodology for conducting 
information security risk assessments, which consider internal and 
external threats to the confidentiality and integrity of the tools and 
data.[Footnote 33] The methodology also requires IRS to develop a 
mitigation plan to reduce or eliminate the risks identified. 

IRS followed its methodology for conducting the risk assessment for 
the Send My Transcripts to Banks tool, which allows taxpayers to 
request a transcript be sent to their bank for income verification 
when applying for a mortgage. The risk assessment included key 
elements such as threat and vulnerability identification. The risks 
were also documented in a mitigation plan, which included the 
resources required to address the risk and the timeframe for 
completion. 

IRS did not follow its risk assessment methodology for First Time 
Homebuyer Credit tool, which allows taxpayers to determine if they 
have a repayment requirement and a remaining balance on the credit. 
The risk assessment documentation included some key elements, such as 
identification of risks to the tool and recommendations to mitigate 
those risks. However, all five identified risks were not included in 
the mitigation plan. After we informed IRS of this error, the Director 
of Applications Development Customer Service Domain acknowledged that 
the identified risks were not included and immediately took steps to 
address four of the five. IRS plans to address the other risk by June 
2013. IRS did not indicate why this oversight occurred, but the fact 
that it did occur raises questions about the reliability of the risk 
assessment process for its other six interactive tools. Reviewing 
mitigation plans for other interactive tools to help ensure that all 
risks are included could provide IRS with reasonable assurance that it 
is addressing those risks, and that its processes are reliable. 

IRS Has Taken Some Steps to Upgrade Its Capability to Authenticate 
Taxpayers' Identities, but Additional Steps Are Required to Provide 
Dynamic Account Access: 

As of March 2013, IRS does not have the capability to authenticate a 
taxpayer's identity with the level of security required for dynamic 
account access. e-Authentication is the process of establishing 
confidence in the taxpayer's identity electronically over the Internet 
for the purpose of electronic government and commerce. NIST provides e-
Authentication standards.[Footnote 34] IRS plans to verify taxpayers' 
identities by confirming various elements of their identity with 
either IRS or third party databases (see table 4). 

Table 4: IRS e-Authentication Procedures and Identity Elements 
Required: 

Level of authentication based on NIST guidance[A]: Level 1; 
e-Authentication procedure: Credential not required; 
Identity elements required: User name and password not required; 
Source of confirmation data: N/A. 

Level of authentication based on NIST guidance[A]: Level 2; 
e-Authentication procedure: Single factor identity validation; 
Identity elements required: User name and password required, and 
taxpayer must be able to provide taxpayer's name, Social Security 
Number, date of birth, address, and filing status; 
Source of confirmation data: IRS databases. 

Level of authentication based on NIST guidance[A]: Level 2; 
e-Authentication procedure: Knowledge based authentication (out of 
wallet); 
Identity elements required: User name and password required, and 
taxpayer must answer some personal knowledge based questions, such 
as previous known address, name of mortgage lender, or prior telephone 
number; 
Source of confirmation data: IRS databases; Third party information 
(e.g. credit bureaus). 

Level of authentication based on NIST guidance[A]: Level 2; 
e-Authentication procedure: Financial/utility information validation; 
Identity elements required: User name and password required, and 
taxpayer must confirm a financial or utility account number; 
Source of confirmation data: IRS databases; Third party information 
(e.g. financial institutions or utility companies). 

Level of authentication based on NIST guidance[A]: Level 3; 
e-Authentication procedure: Multi-factor identity validation; 
Identity elements required: User name and password required, and 
taxpayer is mailed a one-time password though U.S. Postal Service 
which is needed to complete registration process; 
Source of confirmation data: IRS databases; 
Third party information (e.g. financial institutions or utility 
companies). 

Level of authentication based on NIST guidance[A]: Level 4; 
e-Authentication procedure: In person; 
Identity elements required: Taxpayer must be authenticated in person; 
Source of confirmation data: N/A. 

Source: GAO analysis of IRS information. 

Note: N/A means not applicable. 

[A] NIST Special Publication 800-63-1. 

[End of table] 

In August 2012, IRS implemented the first stage of a Level 2 
authentication procedure--single factor identity validation.[Footnote 
35] By the end of 2013, IRS IT officials expect to rollout the 
remaining stages of the Level 2 authentication plus Level 3. Level 3 
would provide IRS with the highest level of confidence that the 
taxpayer is in fact the person attempting authentication, without the 
need for in-person verification. According to IRS officials, at Level 
3 the security processes would be in place to allow for the 
possibility of dynamic account access for taxpayers. 

IRS Investments in Upgrading Security and Developing Authentication 
Capabilities Are Not Linked to a Long-Term Plan: 

The IRS Online Strategy does not describe how security issues, 
including how investing in security upgrades, such as the e-
Authentication procedures, will fit into an overall plan for improving 
the interactivity of IRS's website. IRS's investment in security 
upgrades have been a multiyear effort and required resources from 
several offices to meet federal requirements. For example, in 2010, 
IRS started upgrading security because it had outdated technology, and 
federal guidance required secure access to account information. 
[Footnote 36] Officials in the IRS offices of Privacy, Cybersecurity, 
Online Services, and others had to coordinate many aspects of the 
security upgrades, including determining what information would be 
used to confirm taxpayers' identities. IT officials stated that more 
interactive tools will use the e-Authentication procedure in the 
future and that officials plan to transfer older interactive tools to 
the new procedures as well.[Footnote 37] 

Upgrading IRS's systems is an important and required step to ensure 
IRS provides a secure environment for taxpayers to interact with IRS, 
and to ultimately offer dynamic account access. GPRA Modernization Act 
of 2010 and OMB Circular A-94 guidance suggests that agencies should 
include information on benefits and costs of security investments in 
their strategic plans to ensure IRS managers have the most relevant 
available information on which to base critical investment decisions. 
[Footnote 38] OLS officials did not state why security investments 
were not included in the IRS Online Strategy. However, linking 
security investments to the strategic plan helps to align activities, 
core processes, and resources to support the mission. 

Conclusions: 

IRS provides online services to millions of taxpayers and is engaged 
in a multi-year effort to improve its website. Since 2011 when OLS was 
created, it has focused on improving IRS's static web pages, realizing 
short-term gains by developing new basic interactive tools, and 
building an infrastructure foundation for future improvements. These 
efforts have already benefited taxpayers and hold the promise of 
additional benefits in the future. 

However, IRS does not have a long-term strategy for enhancing its 
website. While IRS has a large amount of documentation for its ongoing 
website improvement efforts, we could not piece together a clear 
understanding of the intended benefits to be provided to website 
users, when those benefits would be realized, how IRS would accomplish 
its goals, or what it would cost. A strategy is necessarily at a high 
level without many details, but ensuring it contains enough 
information could help managers throughout IRS have a common 
understanding of the strategy, better assist Congress in understanding 
what it is being asked to fund and holding IRS accountable for 
progress over time. 

We identified several areas where improvements could help develop a 
long-term strategic plan. One area is measuring performance, where IRS 
has not finalized a full suite of metrics. Another area is learning 
from other organizations, and IRS has not made use of them to inform 
strategic decision making. IRS also has not set a measurable goal for 
improving taxpayer satisfaction, nor consistently developed business 
cases based on benefit and costs or used business cases to set 
priorities. Finally, IRS has not ensured that all risks are mitigated 
and security investments are linked to a long-term plan. Improvements 
in all of these areas would help IRS develop a strategic plan that 
would provide the information that managers and Congress need for 
improving website services to taxpayers. 

Recommendations for Executive Action: 

The Acting Commissioner of the Internal Revenue Service should direct 
appropriate officials to develop a long-term strategy to improve web 
services provided to taxpayers, in accordance with Howto.gov and other 
federal guidance outlined in our report. To accomplish this, we 
further recommend that IRS take the following six actions. 

* Decide on a complete suite of performance metrics, as soon as 
practical. 

* Study leading practices of other organizations to understand how web 
improvement strategies were developed and new services prioritized. 

* Establish a numerical or other measurable goal to improve taxpayer 
satisfaction and a timeframe for achieving it. 

* Develop business cases for all new online services, describing the 
potential benefits and costs of the project, and use them to 
prioritize future projects. 

* Review risk mitigations plans for interactive tools to ensure all 
risks are addressed. If risks are not addressed, IRS should take 
corrective action and identify the root cause to help prevent similar 
occurrences in future mitigation plans. 

* Link investments in security to the long-term plan. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to the Acting Commissioner of the 
IRS. In written comments, which are reprinted in appendix IV, the 
Acting Commissioner agreed with four of the seven recommendations, did 
not state agreement or disagreement with two, and partially agreed 
with one. IRS also provided technical comments on our draft report, 
which we incorporated as appropriate. 

Specifically, IRS stated that it agreed with deciding on a complete 
suite of performance metrics, studying leading practices of other 
organizations to understand how web improvement strategies were 
developed and new services priorities, reviewing risk mitigation plans 
for interactive tools to ensure all risks are addressed, and linking 
investments in security to the long-term plan. 

IRS did not state whether it agreed with the recommendation to develop 
a long-term strategy in accordance with federal guidance. Rather, IRS 
stated it has a long-term strategy to improve online services and 
would make improvements to the plan as appropriate. Thus, it is not 
clear to what extent IRS's actions will fully address the intent of 
this recommendation. As we note throughout the report, we were unable 
to identify a number of fundamental elements for a website strategy, 
as described on [hyperlink, http://www.Howto.gov] in the documentation 
IRS provided us. Thus, we continue to believe that a strategy should 
be developed that includes these elements. Without a strategy that 
clearly outlines the elements described in this report and provides an 
overall picture of what IRS is investing in, it will be difficult for 
Congress and other decision makers to understand up front what they 
are funding and what benefits they can expect. 

IRS also did not state whether it agreed with the recommendation to 
establish a numerical or other measurable goal to improve taxpayer 
satisfaction and a timeframe for achieving it. IRS stated it 
recognized the need for setting specific goals for taxpayer 
satisfaction, but that a high satisfaction score by itself is not 
necessarily as meaningful as other measures, such as reviewing 
satisfaction scores with responses to open ended questions. IRS 
reported it will consider defining a set of measurable goals that 
demonstrates taxpayer satisfaction. As discussed in this report, we 
acknowledge that reviewing other measures of satisfaction and 
examining trends in the data have merit, but without setting a 
numerical or other measurable goal IRS will have difficulty 
determining whether its long-term taxpayer satisfaction plan is 
successful. 

IRS stated that it partially agreed with the recommendation to develop 
business cases for all new online services. Specifically, IRS stated 
that it will continue to develop business cases, including potential 
benefits and costs, for all new online services and use them to help 
prioritize future projects. However, IRS stated that it will also need 
to consider additional criteria, such as risk, time-to-market (i.e., 
how quickly projects can be developed), and the ability for 
stakeholder organizations to incorporate the project into existing 
systems or processes. As noted in the report, these additional 
criteria are part of the same criteria IRS reported already using to 
set priorities for implementing new tools. However, as we also note in 
the report, it was unclear whether IRS used this prioritization 
methodology and the benefit and cost information in the business cases 
lacked enough information to be useful for setting priorities. Thus, 
it is unclear whether IRS's actions will fully meet the intent of the 
recommendation. 

We plan to send copies to the Chairmen and Ranking Members of other 
Senate and House committees and subcommittees that have appropriation, 
authorization, and oversight responsibilities for IRS. We will also 
send copies to the Acting Commissioner of Internal Revenue, the 
Secretary of the Treasury, the Chairman of the IRS Oversight Board, 
and the Deputy Director for Management of the Office of Management and 
Budget. In addition, the report will be available at no charge on the 
GAO website at [hyperlink, http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-9110 or whitej@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made key contributions to 
this report are listed in appendix V. 

Sincerely yours, 

Signed by: 

James R. White: 
Director, Tax Issues Strategic Issues: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

We were asked to review the interactivity of the Internal Revenue 
Service's (IRS) website. This report (1) describes the interactivity 
of IRS's online services and assesses how IRS measures use of the 
services, (2) describes taxpayers' satisfaction with these services, 
and (3) assesses IRS's strategy for expanding the interactivity of its 
website, including how it learns from other organizations, sets goals 
for taxpayer satisfaction for its website, prioritizes the development 
of new services, and addresses security risks. 

To describe the interactivity of IRS's online services and assess how 
IRS measures use of the services, we: 

* reviewed IRS.gov to compile a list of all interactive tools 
available to individual taxpayers as of March 2013. We also 
interviewed Office of Online Services (OLS) officials and reviewed 
supplemental documentation they provided on their tools to complete 
our list. For each tool, we obtained information on its purpose, 
information requirements for use, and determined whether it was a 
secure or non-secure interactive tool. 

* reviewed two federal websites that had dynamic account access 
available to users, Social Security Administration's and Medicare's 
websites. We also cited examples used in prior reports on the tax 
agencies of New York, California, and Finland's websites that also 
provide dynamic account access to users. 

* reviewed IRS's performance metrics for the website and compared them 
against criteria established by Howto.gov, a leading source for 
guidance and best practices for government websites. We also 
interviewed officials to determine their plans to track additional 
performance measures as recommended by Howto.gov: 

To describe taxpayer satisfaction with IRS's online services, we: 

* reviewed IRS's American Customer Satisfaction Index (ACSI) scores as 
gathered by ForeSee via optional online surveys and compared IRS's 
average annual scores from 2005 to 2012 to the average scores of (1) 
approximately 30 other federal agencies and (2) private organizations 
with transactions similar to IRS including in the Internet brokerage, 
Internet retail, and banks sectors.[Footnote 39] Both the federal 
agencies and private organizations used ACSI to measure overall 
customer satisfaction with their websites. 

To assess IRS's strategy for expanding the interactivity of its 
website, including how it learns from other organizations, sets goals 
for taxpayer satisfaction for its website, prioritizes the development 
of new services, and addresses security risks, we: 

* analyzed the September 2012 IRS Online Strategy and updates to the 
strategy provided March 19, 2013 and compared them to federal guidance. 

* identified and reviewed documentation on outreach OLS conducted with 
other organizations to learn about leading practices when implementing 
interactive tools, such as attending a presentation from the New York 
State Department of Taxation and Finance. We compared the steps IRS 
takes to conduct outreach to OLS's product management process and 
other federal guidance, such as Executive Order 13571: Streamlining 
Service Delivery and Improving Customer Service, OMB M-11-24: 
Implementing Executive Order 13571 on Streamlining Service Delivery 
and Improving Customer Service, and Howto.gov, to determine if there 
are any deficiencies. 

* interviewed OLS officials to determine what goals IRS plans to set 
for customer satisfaction on IRS.gov and how officials planned to 
achieve those goals. 

* assessed how IRS prioritized new interactive tools for 
implementation, by comparing the time frames listed in the IRS Online 
Strategy to actual implementation dates of the interactive tools. We 
reviewed business cases for new tools scheduled to be implemented in 
fiscal year 2013, outlined in the September 2012 IRS Online Strategy. 
We asked for business cases for all new tools that were shown to be 
implemented in fiscal year 2013. We reviewed the business cases for 
the interactive tools, Where's My Refund? (2013), Taxpayer Protection 
Unit with Out of Wallet Authentication, Offer in Compromise Pre-
qualifier, and Where's My Amended Return? We compared the business 
cases to benefit cost criteria, including OMB's Circular A-94: 
Guidelines and Discount Rates for Benefit-Cost Analysis of Federal 
Programs. We interviewed OLS officials about the prioritization 
process and the listed time frames for implementation of new 
interactive tools. 

* reviewed IRS's security risk assessments for two interactive tools 
and IRS's risk mitigation plans used to overcome security challenges 
it identified in the security risk assessments. We selected the First 
Time Homebuyer Credit and Send My Transcripts to Banks tools, because 
they were the most recently implemented interactive tools for fiscal 
year 2012. 

* reviewed IRS's recently implemented and planned e-Authentication 
procedures and compared them to National Institute of Standards and 
Technology (NIST) Electronic Authentication Guideline, Special 
Publication 800-63-1 and OMB's E-Authentication Guidance for Federal 
Agencies (OMB M-04-04). 

For all objectives we interviewed relevant IRS officials to collect 
information on IRS's online services and its plans to implement 
additional online services in the future. We reviewed documents and 
interviewed ACSI officials to determine that the ACSI data presented 
in our report for years 2005 to 2012 were sufficiently reliable for 
our purposes. We conducted our work primarily at IRS's National Office 
where relevant IRS staffs are located. 

We conducted this performance audit from June 2012 to April 2013 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: IRS's Basic Interactive Tools: 

As of March 2013, IRS's website highlighted the following 18 basic 
interactive tools from its homepage. Table 5 describes the interactive 
tools, if the tool was secure or not, and the types of personal 
information required to use the tool. 

Table 5: Interactive Tools for Taxpayers Available on IRS.gov as of 
March, 2013: 

1. 
Name of tool: Order A Transcript; 
Description: Allows taxpayers to 
order a transcript to be sent to their address on record (the taxpayer 
will not be able to view the transcript online); 
Secure tool (Y/N): Yes; 
Personal information required: Social Security number (SSN), date of 
birth (DOB), address, ZIP code. 

2. 
Name of tool: Where's My Refund?; 
Description: Allows taxpayers to check the status of their refund; 
Secure tool (Y/N): Yes; 
Personal information required: SSN, filing status, refund amount. 

3. 
Name of tool: Online Payment Agreement; 
Description: Allows taxpayers or their authorized representatives to 
apply for installment agreements if taxes are unable to be paid in 
full; 
Secure tool (Y/N): Yes; 
Personal information required: SSN, DOB, personal identification 
number (PIN) or adjusted gross income, bank address, employer address, 
estimated repayment amount, and time. 

4. 
Name of tool: Electronic Filing PIN Request; 
Description: Allows taxpayers to request an electronic filing PIN to 
verify their identity to e-file tax returns and other electronic tax 
documents; 
Secure tool (Y/N): Yes; 
Personal information required: SSN, name, DOB, filing status, 
address on prior year tax return. 

5. 
Name of tool: First Time Homebuyer Credit Account Look-up; 
Description: Allows taxpayers to get information such as the total 
amount of the credit or repayment amount; 
Secure tool (Y/N): Yes; 
Personal information required: SSN, DOB, address, ZIP code. 

6. 
Name of tool: Send My Transcript to Banks[A]; 
Description: Allows taxpayers to order a transcript to be sent to 
their bank or vendor (the taxpayer will not be able to view the 
transcript online); 
Secure tool (Y/N): Yes; 
Personal information required: SSN, tax year, bank information. 

7. 
Name of tool: Where's My Amended Return; 
Description: Tracks the status of a taxpayer's amended return after it 
was filed with the IRS; 
Secure tool (Y/N): Yes; 
Personal information required: SSN, DOB, and ZIP code. 

8. 
Name of tool: IRS Withholding Calculator; 
Description: Helps taxpayers determine whether they should submit a 
new Form W-4 to their employer to avoid having too much or too little 
tax withheld from their pay; 
Secure tool (Y/N): No; 
Personal information required: Filing status, credits, income and 
withholding, adjustments, deductions. 

9. 
Name of tool: Authorized e-File Provider (For Individuals); 
Description: Allows taxpayers to search for nearby authorized e-file 
providers to electronically file their tax return; 
Secure tool (Y/N): No; 
Personal information required: ZIP code. 

10. 
Name of tool: Volunteer Income Tax Assistance (VITA) Site Locator; 
Description: Allows taxpayers who generally make $50,000 or less to 
find free assistance in preparing their own tax returns. It also 
includes Association of American Retired Persons Tax Aide locator; 
Secure tool (Y/N): No; 
Personal information required: ZIP code. 

11. 
Name of tool: Taxpayer Assistance Center Locator; 
Description: Allows taxpayers to find a walk-in Taxpayer Assistance 
Center to get help with questions that cannot be handled by phone; 
Secure tool (Y/N): No; 
Personal information required: ZIP code. 

12. 
Name of tool: Earned Income Tax Credit (EITC) Assistant; 
Description: Helps taxpayers determine whether they are eligible for 
the EITC and estimate the amount of the credit through a series of 
questions (Spanish version available); 
Secure tool (Y/N): No; 
Personal information required: General information to determine 
eligibility. 

13. 
Name of tool: Alternative Minimum Tax (AMT) Assistant; 
Description: Helps taxpayers determine whether they are subject to the 
AMT through a series of questions; 
Secure tool (Y/N): No; 
Personal information 
required: General information to determine eligibility from past Form 
1040. 

14. 
Name of tool: Sales Tax Deduction Calculator; 
Description: Helps taxpayers determine the amount of optional state 
and local sales tax they can claim on Schedule A of Form 1040; 
Secure tool (Y/N): No; 
Personal information required: Answers to general questions to 
determine deduction estimate based on income and residency. 

15. 
Name of tool: Exempt Organizations Select Check; 
Description: Allows taxpayers to find an exempt organization and 
determine the amount of their contributions that are tax deductible as 
well as other information on an organization's federal tax status and 
filings; 
Secure tool (Y/N): No; 
Personal information required: Database search based on organization's 
employer identification number (EIN), name, and address. 

16. 
Name of tool: Interactive Tax Assistant; 
Description: Provides taxpayers with a resource for tax law and 
provides responses to tax law questions. Categories covered are 
credits, deductions, general filing questions, and income; 
Secure tool (Y/N): No; 
Personal information 
required: Answers to general questions to provide personalized answers. 

17. 
Name of tool: Tax Trails; 
Description: Provides taxpayers with answers to many common tax 
questions, such as deductions and exemptions, tax credits, taxable 
income, and other topics; Secure tool 
(Y/N): No; 
Personal information required: General and some personalized 
information to provide answers to questionnaires. 

18. 
Name of tool: Subscription Services (IRS E-mail Services); 
Description: Allows taxpayers to sign up for one or more of IRS's 
subscription e-mail services to receive tax-related news; 
Secure tool (Y/N): No; 
Personal information required: Email address. 

Source: GAO analysis of IRS.gov. 

Note: This table primarily focuses on tools provided for individual 
taxpayer uses, but other taxpayers, such as businesses or exempt 
organizations, could use some of these interactive tools as well. 

[A] OLS started a pilot program for the Send My Transcript to Banks 
tool at the end of 2012, but it has not been rolled out to the public, 
as of March 2013. 

[End of table] 

[End of section] 

Appendix III: IRS's Component Scores for Customer Satisfaction: 

IRS tracks additional customer service performance metrics using the 
American Customer Satisfaction Index (ACSI). According to Office of 
Online Services officials, the component scores for customer 
satisfaction are considered valuable since they provide more insights 
to customers' experiences. 

Table 6: IRS's ACSI Scores for Website Elements: 

Elements of Customer Satisfaction: Search; 
The relevance, organization, and quality of search results available 
on the site. (Although this element is not applicable universally, it 
is often extremely impactful for sites where it is relevant); 
2005: 69; 
2006: 72; 
2007: 73; 
2008: 73; 
2009: 71; 
2010: 69; 
2011: 69; 
2012: 68. 

Elements of Customer Satisfaction: Functionality; 
The usefulness, convenience, and variety of online features and tools 
available on the website; 
2005: 73; 
2006: 76; 
2007: 77; 
2008: 74; 
2009: 75; 
2010: 75; 
2011: 76; 
2012: 74. 

Elements of Customer Satisfaction: Online transparency; 
How thoroughly, quickly, and accessibly the website discloses 
information about what the agency is doing; 
2005: N/A; 
2006: N/A; 
2007: N/A; 
2008: N/A; 
2009: N/A; 
2010: N/A; 
2011: 75; 
2012: 73. 

Elements of Customer Satisfaction: Navigation; 
The organization of the site and options for navigation; 
2005: 67; 
2006: 71; 
2007: 72; 
2008: 72; 
2009: 71; 
2010: 69; 
2011: 69; 
2012: 68. 

Elements of Customer Satisfaction: Look and feel; 
The visual appeal of the site and its consistency throughout the site; 
2005: 73; 
2006: 76; 
2007: 77; 
2008: 77; 
2009: 77; 
2010: 75; 
2011: 76; 
2012: 76. 

Elements of Customer Satisfaction: Content; 
The accuracy, quality and relevancy of news, information and content 
on the website; 
2005: 78; 
2006: 80; 
2007: 81; 
2008: 78; 
2009: 80; 
2010: 79; 
2011: 80; 
2012: 77. 

Elements of Customer Satisfaction: Site performance; 
The speed, consistency, and reliability of loading pages on the 
website; 
2005: 80; 
2006: 82; 
2007: 83; 
2008: 82; 
2009: 83; 
2010: 83; 
2011: 84; 
2012: 82. 

Elements of Customer Satisfaction: Primary resource; 
Likelihood to use website as primary resource for tax information?; 
2005: 74; 
2006: 77; 
2007: 78; 
2008: 76; 
2009: 78; 
2010: 77; 
2011: 77; 
2012: 75. 

Elements of Customer Satisfaction: Recommend; 
Likelihood to recommend website to someone else? 
2005: 78; 
2006: 81; 
2007: 82; 
2008: 80; 
2009: 81; 
2010: 81; 
2011: 80; 
2012: 77. 

Elements of Customer Satisfaction: Return; 
Likelihood to return to the website? 
2005: 82; 
2006: 85; 
2007: 86; 
2008: 83; 
2009: 86; 
2010: 86; 
2011: 86; 
2012: 84. 

Source: IRS data from ForeSee's ACSI survey results. 

Note: ACSI survey results are subject to the limitations of a 
convenience sampling methodology. We did not independently verify IRS's 
scores. Individual components scores do not necessarily sum to the 
final score because only three questions are used to determine the 
overall customer satisfaction score. The three questions are "What is 
your overall satisfaction with this site?"; "How well does this site 
meet your expectations?"; and "How does this site compare to your idea 
of an ideal website?" 

N/A: Data are not available for those years. 

[End of table] 

[End of section] 

Appendix IV: Comments from the Department of Treasury: 

Department of The Treasury: 
Internal Revenue Service: 
Washington, D.C. 20224: 

April 9, 2013: 

Mr. James R. White: 
Director, Tax Issues: 
Strategic Issues Team: 
U.S. Government Accountability Office: 
441 G Street NW: 
Washington, DC 20548: 

Dear Mr. White: 

Thank you for the opportunity to review and comment on your draft 
report entitled, "IRS Website: A Long Term Strategy Needed to Improve 
Interactive Services" (GAO-13-435, Job Code 450999). We appreciate 
GAO's recognition of the progress we have made in this area as well as 
the review of our efforts and plans for the future. 

Since its inception in July 2011, the IRS Office of Online Services 
has partnered with key IRS business and technology groups to develop 
and launch many new products and services, and also to build 
foundational web capabilities while simultaneously standing up the 
Online Services organization. 

A significant accomplishment has been the relaunch of IRS.gov, an 
enterprise-wide effort that features new design, navigation and Google-
powered search for the public as well as a new platform for future 
content and functionality upgrades. Concurrent with the deployment of 
IRS.gov, IRS also launched a number of other taxpayer service tools 
that include: 

* A revamped Where's My Refund tool; 

* A new Where's My Amended Return tool; 

* New versions in both 2012 and 2013 of our IRS2Go app on iOS and 
Android; 

* Public-facing proof-of-concept applications in the e-authentication, 
electronic payments and online account product lines; 

* Additions to our suite of basic interactive tools including locators 
and calculators. 

As mentioned above, since the inception of Online Services, IRS has 
also focused on building web capabilities for the long-term to support 
greater online customer service delivery. Among other things, the IRS 
has established online governance, program management, product and 
enterprise lifecycle processes; instituted content publishing best 
practices to hundreds of publishers within the Service; and partnered 
with other parts of Treasury to enhance our digital capabilities and 
drive several of the launches mentioned above. The IRS has also begun 
an effort to partner with the tax industry to explore ways of 
delivering enhanced services to taxpayers online. 

A fundamental element of our approach is to launch, learn, and 
continually make improvements in order to improve online customer 
service options in the short-as well as medium-terms. At the same time 
we are developing plans, processes and talent to drive the conversion 
to digital over the long term. In order to keep up with rapidly 
evolving technology and consumer preferences, our iterative approach 
allows us to deliver functionality and content more rapidly, measure 
its impact, and adjust our roadmap accordingly. We believe this 
approach is consistent with industry best practices and allows us to 
combine secure and scalable services for taxpayers with digital best 
practices. 

One example of this approach is in the area of dynamic account access. 
While the IRS does not currently have online account access, we are 
working toward this goal in the future. We are putting in place 
fundamental components such as e-authentication, online transcript 
access, as well as several other applications, to move toward the 
adoption of an online account on IRS.gov. 

We agree that business cases are an important element to prioritize 
projects. In addition to business cases, we also use other criteria 
for prioritization, including time-to-market, the current roadmap, 
compatibility with other business priorities, operating capabilities 
of the systems that a new tool may rely on, and the legal requirements 
related to protecting taxpayer data. 

We agree that further refinement of our long-term plan is key to 
building on the successful momentum created to date. Planned 
refinement includes further articulating our strategic roadmap; 
updating and better aggregating our cost and risk estimates; and 
extending our use of analytics to measure our progress. 

We recognize that continued improvement to our online resource is 
important in order to assist taxpayers to effectively meet their 
compliance responsibilities as well as to improve taxpayer 
satisfaction and achieve budget efficiency. We appreciate your 
continued assistance as we work to fulfill this vision. 

Our responses to your recommendations are enclosed. If you have any 
questions, please contact Rajive Mathur, Director, Office of Online 
Services, at 202-927-0145. 

Sincerely, 

Signed by: 

Steven T. Miller: 
Acting Commissioner: 

Enclosure: 

The Acting Commissioner of Internal Revenue Service should direct 
appropriate officials to: 

Recommendation: 

Develop a long term strategy to improve web services provided to 
taxpayers, in accordance with Howto.gov and other federal guidance 
outlined in our report. 

Comment: 

While the IRS has already developed a long term strategy on improving 
web services for taxpayers, we will continue to expand the strategy 
and make improvements as appropriate. As stated previously, this 
strategy is a living document that will evolve as priorities and 
progress are made over time. Based on the recommendations below, we 
will make necessary changes to continue to improve our long term 
strategy. 

To accomplish this, we further recommend that IRS take the following 
six actions: 

Recommendation: 

Decide on a complete suite of performance metrics, as soon as 
practical; 

Comment: 

We agree with this recommendation. We currently already track the vast 
majority of recommended metrics. We will continue to identify and 
evaluate other relevant performance metrics for potential value. 

Recommendation: 

Study leading practices of other organizations to understand how web 
improvement strategies were developed and new services prioritized; 

Comment: 

We agree with your recommendation and we will continue to use industry 
best practices for web improvement. We will continue to review online 
private and public sector approaches in order to build upon the work 
already completed. We will use such information to make strategic 
decisions and help inform prioritization of services whenever 
practicable. 

Recommendation: 

Establish a numerical or other measurable goal to improve taxpayer 
satisfaction and a timeframe for achieving it; 

Comment: 

IRS recognizes the need for setting specific goals and will consider 
the development of a numerical or other set of measurable goals that 
can be used to improve taxpayer satisfaction. IRS already engages in a 
continuous improvement process with regard to its website, and 
satisfaction scores are part of that overall improvement. We 
recognize, however, that a high satisfaction score, by itself, is not 
necessarily as meaningful as other measures. By reviewing satisfaction 
scores with responses to open ended questions and conversion rates on 
new applications, a more balanced scorecard can be created. As the IRS 
continues to leverage capabilities of the new IRS.gov, we will 
consider defining a set of measurable goals that demonstrates taxpayer 
satisfaction. 

Recommendation: 

Develop business cases for all new online services, describing the 
potential benefits and costs of the project, and use them to 
prioritize future projects; 

Comment: 

We partially agree with this recommendation. We will continue to 
develop business cases for all new online services and include 
potential benefits and costs of the project. We will use online 
business cases to help prioritize future products/projects, but will 
also need to balance the prioritization using broader criteria which 
includes elements such as time-to-market, ability for stakeholder 
organizations to incorporate the online program into existing business 
processes, and risk to existing business systems or processes. 

Recommendation: 

Review risk mitigation plans for interactive tools to ensure all risks 
are addressed. If risks are not addressed, IRS should take corrective 
action and identify the root cause to help prevent similar occurrences 
in future mitigation plans; 

Comment: 

We agree with this recommendation and will continue to collaborate 
with Information Technology and the business customers to conduct 
electronic risk assessments of all planned online tools. We will also 
continue to ensure all risks are addressed and take the necessary 
corrective actions, as needed. 

Recommendation: 

Link investments in security to the long term plan. 

Comment: 

We agree with this recommendation and will integrate the future plans 
for the use of authentication into the overall online strategy, 
including the dependency on "eAuthentication" capabilities. 

[End of section] 

Appendix V: GAO Contact and Staff Acknowledgment: 

GAO Contact: 

James R. White, (202) 512-9110, or Whitej@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, Libby Mixon, Assistant 
Director; Amy Bowser, Emily Gruenwald, Nicole Jarvis, Jeffrey Knott, 
Ed Nannenhorn, Cynthia Saunders, and Robyn Trotter made key 
contributions to this report. 

[End of section] 

Footnotes: 

[1] [hyperlink, http://www.IRS.gov], accessed March 15, 2013. 

[2] As cited on [hyperlink, http://www.performance.gov], accessed 
March 26, 2013. 

[3] GAO, 2011 Tax Filing: Processing Gains, but Taxpayer Assistance 
Could be Enhanced by More Self-Service Tools, [hyperlink, 
http://www.gao.gov/products/GAO-12-176] (Washington, D.C.: Dec. 15, 
2011). 

[4] IRS Oversight Board, Electronic Filing 2011: Annual Report to 
Congress, December, 2011; National Taxpayer Advocate Annual Report to 
Congress, December 31, 2012; and Electronic Tax Administration 
Advisory Committee, Annual Report to Congress, June 2012, Publication 
3415. These three organizations oversee IRS's operations. 

[5] [hyperlink, http://www.gao.gov/products/GAO-12-176]. 

[6] Accessed February 26, 2013, [hyperlink, http://www.howto.gov]. 

[7] IRS uses ForeSee, a company that uses customer experience 
analytics to continuously measure satisfaction and provide insights to 
clients to administer the ACSI optional pop up survey to users on 
IRS.gov. 

[8] We compared IRS's average ACSI scores (2005 to 2012) to the 
average scores of (1) approximately 30 other federal agencies and (2) 
private organizations with transactions similar to IRS including in 
the Internet brokerage, Internet retail, and banks sectors. Both the 
federal agencies and private organizations used ACSI to measure 
overall customer satisfaction with their websites. 

[9] NIST is charged with providing guidance and assistance to agencies 
and defining minimum security requirements, among other things. 

[10] IRS Oversight Board, 2012 Taxpayer Attitude Survey, February, 
2013. 

[11] The IRS Technical Investment Review Board is a group of 
executives including the Chief Financial Officer, Chief Technology 
Officer, Deputy Commissioners, and the Chief of Staff from the 
Commissioner Office, that decides what investments to fund from off-
budget cycle proposals. The Director of OLS told us he expects funding 
for fiscal year 2014 and beyond to go through the congressional 
justification budget process. The Fiscal Year 2014 President's Budget 
was released on April 10, 2013, and requests $24.1 million for online 
services. 

[12] The Internal Revenue Manual section on Information Technology 
Security, Policy and Guidance has been aligned with NIST guidance. 

[13] OMB Circular A-94, Memorandum For Heads of the Executive 
Departments and Establishments on Guidelines and Discount Rates for 
Benefit Cost Analysis of Federal Programs (Oct. 29, 1992). 

[14] IRS cannot offer dynamic account access until it can securely 
authenticate a taxpayer's identity electronically, which is planned 
for the end of 2013. IRS.gov facilitates electronic filing for 
taxpayers through two programs, E-file and Free File. Officials 
consider these programs components of account services; however, they 
were not included in our review because the services are not hosted on 
IRS.gov and taxpayers can only send information to IRS. 

[15] [hyperlink, http://www.gao.gov/products/GAO-12-176]. 

[16] GAO, Tax Administration: Information on Selected Foreign 
Practices That May Provide Useful Insights, [hyperlink, 
http://www.gao.gov/products/GAO-11-439], (Washington, D.C.: May 24, 
2011). 

[17] Since August 30, 2012, IRS has used Google Analytics to collect 
performance metrics data on IRS.gov. Prior to August 30, 2012, IRS 
used WebTrends 8.7 on Demand to analyze website usage. IRS also tracks 
performance metrics using surveys. According to IRS, Google Analytics 
is more comprehensive because it gives IRS the ability to track more 
measures. 

[18] The two metrics that IRS has not begun to track as of March 18, 
2013 are visits per visitor for a given time frame and top searches 
with low click through rates. 

[19] The average customer satisfaction score for other federal 
agencies websites ranged from 74 to 75 for 2009 to 2012 and select 
sectors of the private economy ranged from 75 to 81 for the same time 
period. 

[20] ACSI was developed at the University of Michigan as a multi-
equation econometric model to determine the satisfaction of customers. 
Internet industries include web-based businesses such as Internet 
retailers and Internet brokerage firms. 

[21] While the model has limitations, all of the federal agencies and 
private organizations surveyed by ACSI are subject to the same 
limitations. 

[22] IRS's scores for January and February 2013 have increased to 69 
and 70, respectively. IRS officials attribute this increase to the 
redesigned home page and search functions. 

[23] Electronic Tax Administration Advisory Committee, Annual Report 
to Congress, June 2012, Publication 3415; IRS Oversight Board, 
Electronic Filing 2011: Annual Report to Congress, December, 2011; and 
National Taxpayer Advocate, Annual Report to Congress, December 31, 
2012. 

[24] Pub L. No. 103-62, 107 Stat. 285 (1993); Pub. L. No. 111-352, 124 
Stat. 3866 (Jan. 4, 2011); and GAO, Veteran-Owned Small Businesses: 
Planning and Data System for VA's Verification Program Needs 
Improvement, [hyperlink, http://www.gao.gov/products/GAO-13-95] 
(Washington, D.C.: Jan.14, 2013). 

[25] See OMB M-11-24, Implementing Executive Order 13571 on 
Streamlining Service Delivery and Improving Customer Service (June 13, 
2011); 25 Point Implementation Plan to Reform Federal Information 
Technology Management by the U.S. Chief Information Officer (Dec. 9, 
2010); and Executive Order 13571, Streamlining Service Delivery and 
Improving Customer Service ( April 27, 2011). 

[26] In our prior work, we developed nine key attributes of successful 
performance measures by reviewing, among other things, key legislation 
and performance management literature. GAO, Tax Administration: IRS 
Needs to Further Refine Its Tax Filing Season Performance Measures, 
[hyperlink, http://www.gao.gov/products/GAO-03-143] (Washington, D.C.: 
Nov. 22, 2002). A numerical goal does not have to be a number, but 
could be qualitative, such as "above average." 

[27] IRS has set numerical taxpayer satisfaction goals for other areas 
of strategic importance, such as achieving an ASCI score of 72 for 
individual income tax filers' experience with paper and electronic 
filing in fiscal year 2013. 

[28] The depth and rigor of a benefit cost analysis should depend on 
the magnitude of the project and where the agency is in the planning 
process. 

[29] As of the end of February 2013, IRS revised its time frames to 
only implement in fiscal year 2013 the four interactive tools for 
which business cases existed. As of mid-March IRS provided three 
additional draft business cases for the projects to be implemented in 
fiscal year 2014. IRS officials are in the process of developing the 
business case for the last tool. 

[30] GAO, IRS Guidance on Economic Analysis in Investment Business 
Cases, [hyperlink, http://www.gao.gov/products/GAO-02-234R] 
(Washington, D.C.: May 9, 2002). 

[31] The depth and rigor of a benefit cost analysis should depend on 
the magnitude of the project and where the agency is in the planning 
process. 

[32] GAO, Tax Gap: IRS Could Significantly Increase Revenues by Better 
Targeting Enforcement Resources, [hyperlink, 
http://www.gao.gov/products/GAO-13-151] (Washington, D.C.: Dec. 5, 
2012). 

[33] Threat and vulnerability identification is the process of 
identifying (1) threats (both internal and external) to the 
confidentiality, integrity, and availability of systems and data 
supporting critical operations and assets and (2) vulnerabilities 
(i.e., flaws or weaknesses) that could be exploited by such threats. 

[34] NIST Special Publication 800-63-1 provides technical guidance to 
agencies to allow an individual to remotely authenticate his or her 
identity to a federal information technology system. 

[35] According to IT officials, the Send My Transcript to Banks tool, 
which allows taxpayers to send a copy of their transcript to a bank of 
vendor of a financial institution, is the only interactive tool using 
the Level 2 authentication. Static web pages are considered to have 
Level 1 authentication. 

[36] According to NIST Special Publication 800-63 and OMB Circular 04-
04, federal agencies are required to develop security features that 
allow people to access information based on the level of risk 
associated with information being accessed. 

[37] Some of the future interactive tools that would require e-
authentication include the Online Payment Agreement and Automated 
Clearing House Debit tools. 

[38] GPRA Modernization Act of 2010, Pub. L. No. 111-352, 124 Stat. 
3866 (Jan. 4, 2011), OMB Circular A-94, and [hyperlink, 
http://www.gao.gov/products/GAO-02-234R]. 

[39] IRS uses ForeSee, a company that uses customer experience 
analytics to continuously measure satisfaction and provide insights to 
clients to administer the ACSI optional pop up survey to users on 
IRS.gov. 

[End of section] 

GAO’s Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO’s commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
“E-mail Updates.” 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO’s 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548. 

Public Affairs: 
Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548. 

[End of document]