This is the accessible text file for GAO report number GAO-04-470 
entitled '2010 Census: Overseas Enumeration Test Raises Need for Clear 
Policy Direction' which was released on June 21, 2004.

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov.

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately.

Report to the Subcommittee on Technology, Information Policy, 
Intergovernmental Relations, and the Census, Committee on Government 
Reform, House of Representatives: 

May 2004: 

2010 CENSUS: 

Overseas Enumeration Test Raises Need for Clear Policy Direction: 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-470] 

GAO Highlights: 

Highlights of GAO-04-470, a report to the Subcommittee on Technology, 
Information Policy, Intergovernmental Relations, and the Census, 
Committee on Government Reform, House of Representatives

Why GAO Did This Study: 

In the 1990 and 2000 Censuses, U.S. military and federal civilian 
employees overseas were included in the numbers used for apportioning 
Congress. Currently, the U.S. Census Bureau (Bureau) is assessing the 
practicality of counting all Americans abroad by holding a test census 
in France, Kuwait, and Mexico. GAO was asked to (1) assess the 
soundness of the test design, and (2) examine what past court decisions 
have held about Americans’ rights and obligations abroad.

What GAO Found: 

Although the overseas enumeration test was designed to help determine 
the practicality of counting all Americans abroad, because of various 
methodological limitations, the test results will only partially answer 
the Bureau’s key questions concerning feasibility, data quality, and 
cost. For example, one research questions asks, “How good is the 
quality of the data?” However, the Bureau will only measure item 
nonresponse, which indicates whether a person completed a particular 
question. As a measure of quality, it is far from complete. Similarly, 
although a key research objective was to determine the cost of counting 
Americans overseas, the Bureau’s data will not inform the cost of 
conducting future tests or an overseas enumeration in 2010. Overall, 
the Bureau overstated the test’s ability to answer its key research 
objectives.

Counting Americans Abroad Would be a Monumental Challenge: 

[See PDF for image]

[End of figure]

Overseas Americans have various rights and obligations to federal 
programs and activities. For example, Americans abroad are generally 
taxed on their worldwide income and can vote in federal elections, but 
are generally not entitled to Medicare benefits. There is nothing in 
the Constitution, federal law, or court decisions that would either 
require the Bureau to count overseas Americans, or not count this 
population group. As a result, Congress would need to enact legislation 
if it wanted to require the Bureau to include overseas Americans in the 
2010 Census. Counting Americans abroad as part of the census would add 
new risks to an enterprise that already faces an array of challenges. 
Therefore, it will be important for Congress to decide whether overseas 
Americans should be counted as part of the census or counted as part 
of a separate survey or whether there are so many obstacles to a 
successful count regardless of the approach that the Bureau should 
shelve any plans for further research and testing. To the extent a 
second test is required, the Bureau will need to take steps to develop 
a more rigorous design.

What GAO Recommends: 

In order to give the Bureau as much planning time as possible, Congress 
may wish to consider coming to an early decision on whether the Bureau 
should be required to enumerate all overseas Americans, and if so, 
whether they should be counted as part of the decennial census or by 
some other, separate data collection effort. Should Congress desire an 
overseas count, it should consider telling the Bureau how the data 
would be used. Further, the Bureau should address the shortcomings of 
the 2004 test design, in part by being more transparent about what the 
test can and cannot measure. The Bureau generally concurred with our 
findings and recommendations, but believes it has already been clear 
about the test’s limitations. Still, the Bureau agreed it will be 
critical to point out the various limitations that may affect 
congressional deliberations on this matter. 

www.gao.gov/cgi-bin/getrpt?GAO-04-470.

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Patricia A. Dalton at 
(202) 512-6806 or daltonpj@gao.gov.

[End of section]

Contents: 

Letter: 

Results in Brief: 

Background: 

Scope and Methodology: 

Design Limitations Could Undermine the Usefulness of the Overseas Test: 

Overseas Test Design Has Other Shortcomings: 

The Rights and Obligations of Americans Overseas Vary: 

Conclusions: 

Matters for Congressional Consideration: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendixes: 

Appendix I: Comments from the Department of Commerce: 

Appendix II: Criteria for Evaluating Census Research Design for 
Overseas Enumeration Test: 

Appendix III: 2004 Overseas Test Census Questionnaire: 

Tables: 

Table 1: Treatment of Certain Population Groups Living or Working 
Overseas in the Decennial Censuses, 1900-2000: 

Table 2: Research Questions Have Methodological Limitations That Could 
Reduce the Bureau's Ability to Address Research Objectives: 

Table 3: The Applicability of Federal Programs to Americans Overseas 
Varies: 

Figure: 

Figure 1: Census Bureau Poster Advertising the Overseas Census: 

Letter May 21, 2004: 

The Honorable Adam H. Putnam: 
Chairman: 
The Honorable Wm. Lacy Clay: 
Ranking Minority Member: 
Subcommittee on Technology, Information Policy, Intergovernmental 
Relations and the Census: 
Committee on Government Reform: 
House of Representatives: 

One of the thornier issues facing the U.S. Census Bureau (Bureau) as it 
prepares for the 2010 Census is whether to count American citizens 
residing abroad, and if so, how to use the results. Under federal law, 
the Bureau has discretion over whether to count this population group. 
Thus, in prior censuses, the Bureau has generally included "federally 
affiliated" groups--members of the military and federal employees and 
their dependents--but has excluded private citizens residing abroad 
from all but the 1960 and 1970 Censuses.[Footnote 1] Moreover, when 
these overseas groups were included in the census, their population 
totals were generally not included with the counts used for 
apportioning Congress.

According to the Bureau, no accurate estimate exists of the total 
number of Americans living abroad.[Footnote 2] Further, while it is 
unclear how counting these individuals would affect congressional 
apportionment and other purposes for which census data are used, in 
some instances it could be significant. For example, in January 2001, 
Utah sued the U.S. Department of Commerce claiming that it lost a 
congressional seat because the 2000 Census excluded the state's 11,000 
Mormon missionaries and other private citizens living abroad.[Footnote 
3] Although the suit was unsuccessful, according to a report by the 
Congressional Research Service, Utah would have gained a congressional 
seat had an additional 855 people been counted in the census.[Footnote 
4]

In response to congressional direction and the concerns of various 
stakeholder groups representing overseas Americans, the Census Bureau 
launched a research and evaluation program to assess the practicality 
of counting both private and federally affiliated U.S. citizens 
residing abroad, as well as their dependents. The first step of this 
effort, a test enumeration of Americans residing in France, Kuwait, and 
Mexico, is currently underway. A sound feasibility test is essential 
because the Bureau has already identified a number of operational, 
conceptual, and policy issues that would make an accurate count of 
overseas Americans even more difficult than the stateside enumeration, 
an endeavor that is a daunting challenge in its own right. As a second 
step, the results of the 2004 test will be used to provide information 
and recommendations for another overseas enumeration test planned for 
2006.

To facilitate Congress's oversight of the test and provide information 
for future decision making on this issue, you asked us to review the 
overseas test design. As agreed with your offices, we (1) assessed the 
soundness of the Bureau's test design and its suitability for 
addressing the Bureau's specific research questions, and (2) examined 
what past court decisions have held about Americans' rights and 
obligations abroad that could help inform whether and how they should 
be included in the census.

To meet these objectives, we interviewed knowledgeable Bureau 
officials; reviewed pertinent documents such as test plans; and 
examined relevant statutes, court decisions, and legal analyses. We 
also systematically rated the soundness of the Bureau's test design 
using a checklist of over 30 design elements that, based on our review 
of program evaluation literature, are important for a sound study plan. 
We conducted our work from June 2003 through February 2004, in 
accordance with generally accepted government auditing standards.

This report is the second in our ongoing series on the planning and 
development of the 2010 Census. In a study issued earlier this year, 
we described the various challenges facing the 2010 Census, and the 
need for the Bureau to address cost and design issues.[Footnote 5]

Results in Brief: 

The Bureau designed the 2004 test to help determine the practicality of 
collecting data from U.S. citizens living overseas. However, because of 
various methodological limitations, the test results will only 
partially answer the Bureau's key objectives concerning feasibility (as 
measured by such indicators as participation and number of valid 
returns), data quality, and cost. Although some of the methodological 
limitations stem from the inherent challenges of enumerating Americans 
abroad, others are due to the way in which the Bureau is implementing 
the test. Overall, the Bureau overstated the test's ability to answer 
its key research objectives and, as a result, congressional decision 
making on this issue will be that much more difficult.

For example, one of the Bureau's research questions asks, "How good is 
the quality of the data?" However, the only measure of quality 
available to address this is the percentage of respondents who did not 
complete a particular question. As the Bureau acknowledges, this 
measure of nonresponse is an incomplete measure of the quality of the 
data. Similarly, although a key research objective was to determine the 
cost of counting Americans overseas, the Bureau did not develop a 
research question to address this variable and, more importantly, the 
cost information that the Bureau will collect will be limited to the 
test sites. It will not inform the cost of conducting future tests or 
an overseas enumeration in 2010.

Americans residing abroad do not have the same rights and obligations 
under federal programs and activities as compared to their stateside 
counterparts. On the one hand, overseas citizens are generally taxed on 
their worldwide income, can vote in federal elections, and can receive 
Social Security benefits. On the other hand, they are generally not 
entitled to Medicare benefits, or, if they reside outside of the United 
States for more than 30 days, Supplemental Security Income.

With respect to the census, the Bureau has discretion over whether or 
not to count Americans overseas. There is nothing in the Constitution, 
federal law, or court decisions that would either require the Bureau to 
count or not count Americans abroad. As a result, Congress would need 
to enact legislation if it wanted to require the Bureau to include 
overseas Americans in the 2010 Census.

The full results of the overseas enumeration test will not be known 
until 2005, when the Bureau anticipates it will complete a series of 
evaluations. However, the Bureau's experience thus far makes it clear 
that an accurate count of U.S. citizens abroad as part of the 2010 
Census would be extremely difficult, and would introduce new resource 
requirements, risks, and uncertainties to an endeavor that already 
faces a host of difficulties.

At the same time, to the extent that better demographic data on 
overseas Americans might be useful for various policymaking and other 
nonapportionment purposes, it does not necessarily need to come from 
the decennial census. Such information could be acquired through a 
separate survey or alternative data collection effort, although it 
would still be a difficult task.

It will be important for Congress to come to an early decision on 
whether the Bureau should be required to count Americans abroad as part 
of the 2010 Census or as part of a separate data collection effort or 
whether there are so many hurdles to a successful overseas enumeration 
regardless of the approach that the Bureau should abandon any plans for 
further research and testing. Should Congress desire an overseas count, 
be it part of the decennial census or a separate effort, it should 
consider providing the Bureau with input on whether the data would be 
used for purposes of apportionment, redistricting, allocating federal 
funds, or a tally of the U.S. overseas population. Armed with this 
information, the Bureau would be better positioned to design a test 
that would more accurately assess the specific resources and 
methodology needed to accomplish the type of census that Congress 
desires. Moving ahead with a second test without this information would 
be an imprudent use of the Bureau's resources.

Further, to the extent that a second test of enumerating Americans 
abroad is needed in 2006, we recommend that the Secretary of Commerce 
direct the Bureau to resolve the shortcomings of the design of the 2004 
test and better address the objectives of an overseas enumeration. At a 
minimum, specific steps should include (1) being more transparent with 
Congress and other stakeholders on what variables and research 
questions the Bureau can and cannot assess, (2) explore developing 
broader measures of data quality, and (3) developing a cost model to 
provide the Bureau and Congress with better estimates of the budgetary 
impact of conducting an overseas census under different methodological 
and other scenarios.

The Secretary of Commerce forwarded written comments from the U.S. 
Census Bureau on a draft of this report. The comments are reprinted in 
appendix I.

The Bureau generally agreed with our conclusions and recommendations 
but took exception to our finding and related recommendation that the 
Bureau needs to be more transparent with Congress and other 
stakeholders regarding the variables and research questions it can and 
cannot answer. The Bureau maintains that it has always been clear that 
the test is only a "most basic assessment of feasibility." Still, the 
Bureau agreed that as it completes its evaluations and documents its 
findings from the test, it will be "critical" to highlight the various 
limitations that could affect congressional deliberations on this 
subject.

Background: 

The Census Bureau has discretion under the Constitution and federal 
statutes to decide whether to count Americans residing overseas. The 
federal decennial census is conducted pursuant to the requirement 
imposed by Article I, Section 2, Clause 3 of the Constitution, and 
Section 2 of the Fourteenth Amendment, that Congress enumerate "the 
whole number of persons in each State" as the basis for apportionment 
of seats in the United States House of Representatives. Under the 
Constitution, the census is to be conducted every 10 years "in such 
Manner as [Congress] shall by Law direct." Congress has exercised its 
authority under the Constitution by passing the Census Act, which 
assigns to the Secretary of Commerce the responsibility of "tak[ing] a 
decennial census of population as of the first day of April" of each 
census year.[Footnote 6] The Secretary does so with the assistance of 
the Census Bureau and its Director.[Footnote 7]

The statutes governing the earliest censuses provided that enumerators 
should record all persons reported to them within their respective 
districts as having a usual place of abode there or as usually residing 
within that district, even though such persons might be "occasionally 
absent at the time of enumeration."[Footnote 8] Statutes governing 
later censuses, including the current provisions in Title 13 of the 
United States Code, contain no similar provision, or any provision 
specifically governing the enumeration of inhabitants of the United 
States who are outside of its borders on the enumeration date.

To determine who should be included in the census, the Bureau applies 
its "usual residence rule," which has been defined as the place where a 
person lives and sleeps most of the time. People who are temporarily 
absent from that place are still counted as residing there.[Footnote 9] 
One's usual residence is not necessarily the same as one's voting 
residence or legal residence. Noncitizens living in the United States 
are counted in the census, regardless of their immigration status.

Historically, the census has focused primarily on the domestic 
population and typically has not included any procedures designed to 
enumerate Americans residing outside of the United States. The first 
attempts to count Americans residing overseas were in the 1830 and 1840 
censuses, which included procedures for counting the "crews of naval 
vessels at sea."[Footnote 10] The naval personnel included in those 
censuses, however, were not allocated to any individual state, and thus 
were not included in the apportionment population.[Footnote 11]

As shown in table 1, various overseas population groups were included 
in the census at different times. For example, while federally 
affiliated personnel were typically included in the enumerations that 
took place from 1900 through 2000, only the 1970, 1990, and 2000 
censuses used the numbers for purposes of apportioning Congress. At the 
same time, private citizens living abroad were included only in the 
1960 and 1970 censuses, but not for purposes of apportionment.

Table 1: Treatment of Certain Population Groups Living or Working 
Overseas in the Decennial Censuses, 1900-2000: 

Population group[A]: U.S. military personnel stationed abroad or at 
sea and their dependents; 
1900: Counted in the census, but not included in the population totals 
used for apportionment; 
1910: Counted in the census, but not included in the population totals 
used for apportionment; 
1920: Counted in the census, but not included in the population totals 
used for apportionment; 
1930: Counted in the census, but not included in the population totals 
used for apportionment; 
1940: Counted in the census, but not included in the population totals 
used for apportionment; 
1950: Counted in the census, but not included in the population totals 
used for apportionment; 
1960: Counted in the census, but not included in the population totals 
used for apportionment; 
1970: Included in the population count used for congressional 
apportionment; 
1980: Counted in the census, but not included in the population totals 
used for apportionment; 
1990: Included in the population count used for congressional 
apportionment; 
2000: Included in the population count used for congressional 
apportionment.

Population group[A]: Federal civilian employees stationed abroad and 
their dependents; 
1900: Counted in the census, but not included in the population totals 
used for apportionment; 
1910: Counted in the census, but not included in the population totals 
used for apportionment; 
1920: N/A; 
1930: N/A; 
1940: N/A; 
1950: Counted in the census, but not included in the population totals 
used for apportionment; 
1960: Counted in the census, but not included in the population totals 
used for apportionment; 
1970: Included in the population count used for congressional 
apportionment; 
1980: Counted in the census, but not included in the population totals 
used for apportionment; 
1990: Included in the population count used for congressional 
apportionment; 
2000: Included in the population count used for congressional 
apportionment.

Population group[A]: Persons abroad working for the American Red Cross 
or in the consular service and their dependents; 
1900: N/A; 
1910: N/A; 
1920: Counted in the census, but not included in the population totals 
used for apportionment; 
1930: Counted in the census, but not included in the population totals 
used for apportionment; 
1940: Counted in the census, but not included in the population totals 
used for apportionment; 
1950: N/A; 
1960: N/A; 
1970: N/A; 
1980: N/A; 
1990: N/A; 
2000: N/A.

Population group[A]: Private U.S. citizens abroad for an extended 
period; 
1900: N/A; 
1910: N/A; 
1920: N/A; 
1930: N/A; 
1940: N/A; 
1950: N/A; 
1960: Counted in the census, but not included in the population totals 
used for apportionment; 
1970: Counted in the census, but not included in the population totals 
used for apportionment; 
1980: N/A; 
1990: N/A; 
2000: N/A. 

Source: U.S. Bureau of the Census, Americans Overseas in the U.S. 
Censuses, Technical Paper 62 (Washington, D.C.: November 1993).

Notes: 

[A] This table excludes the officers and crew of merchant marine 
vessels because available data were unclear as to whether these groups 
were included in the overseas enumerations or the stateside counts in 
the decennial censuses.

[End of table]

In response to congressional direction and the concerns of various 
business, political, and other groups that represent overseas 
Americans, the Census Bureau embarked on a research and evaluation 
program aimed at determining the feasibility, quality, and cost of 
counting both federally affiliated and private citizens living abroad. 
The test enumeration began February 2004 and is to run through July 
2004 at three sites: France, Kuwait, and Mexico.

The Bureau selected these countries based on several criteria including 
their geographic diversity, the fact that large numbers of U.S. 
citizens reside there, and because of the existence of administrative 
records that can be used to compare to the test census counts for 
evaluation purposes. The Bureau estimated the implementation costs for 
the 2004 test at approximately $2.5 million in fiscal year 2004. 
Further, the Bureau estimates that by the end of fiscal year 2004, it 
will have spent an additional $3.5 million for planning and preparation 
during fiscal years 2003 and 2004.

Americans can participate in the test census by completing a short-form 
paper questionnaire that is available at embassies, consulates, and 
other organizations that serve overseas Americans, or by completing the 
form on the Internet. The Bureau hired a public relations firm to 
develop a communications strategy to inform and motivate respondents 
living in the selected countries to answer the census. Responses from 
the paper and the Internet returns will be captured in order to 
analyze, among other things, the demographic characteristics of 
respondents and patterns of item nonresponse. The Bureau plans to 
conduct a 2006 overseas test if Congress appropriates requested funds 
in fiscal years 2005, 2006, and 2007. If Congress then indicates its 
desire that the Census Bureau conduct a general overseas enumeration in 
2010, the Bureau will seek a supplementary appropriation in calendar 
year 2007 for that purpose and to conduct a 2008 overseas dress 
rehearsal beginning in 2007.

A sound test is essential in order for the Bureau, Congress, and other 
stakeholders to resolve the numerous logistical, conceptual, policy, 
and other questions that surround the counting of overseas Americans. 
They include: 

* Who should be counted? U.S. citizens only? Foreign-born spouses? 
Children born overseas? Dual citizens? American citizens who have no 
intention of ever returning to the United States? Naturalized citizens?

* How should overseas Americans be assigned to individual states? For 
certain purposes, such as apportioning Congress, the Bureau would need 
to assign overseas Americans to a particular state. Should one's state 
be determined by the state claimed for income tax purposes? Where one 
is registered to vote? Last state of residence before going overseas? 
These and other options all have limitations that would need to be 
addressed.

* How should the population data be used? To apportion Congress? To 
redistrict Congress? To allocate federal funds? To provide a count of 
overseas Americans only for general informational purposes? The answers 
to these questions have significant implications for the level of 
precision needed for the data and ultimately, the enumeration 
methodology.

* How can the Bureau verify U.S. citizenship? Administrative records 
such as passports and Social Security data have limitations. For 
example, Americans can reside in Mexico and Canada without a passport 
and many Americans overseas do not have Social Security numbers, 
especially dependents.

* How can the Bureau ensure a complete count without a master address 
list? The foundation of the stateside decennial census is a master 
address list. Because the list is essentially the universe of all known 
living quarters in the United States, the Bureau uses it to deliver 
questionnaires, follow up with nonrespondents, determine vacancies, and 
determine individuals the Bureau may have missed or counted more than 
once. The Bureau lacks a complete and accurate address list of overseas 
Americans. Consequently, these operations would be impossible and the 
quality of the data would suffer as a result.

* Can administrative records be used to help locate and count overseas 
Americans? Administrative records such as passport and visa files, 
voter registration forms, as well as records held by private companies 
and organizations have the potential to help the Bureau enumerate 
Americans abroad. However, the accuracy of these records, the Bureau's 
ability to access them, confidentiality issues, and the possibility of 
duplication all remain open questions.

* Do certain countries have requirements that could restrict the 
Bureau's ability to conduct a count? According to the Bureau, in 
planning the overseas test, the Bureau was informed that French privacy 
laws prohibit asking about race and ethnicity, two questions that are 
included on the U.S. census questionnaire. Although the Bureau worked 
with French officials to address this problem, the extent to which the 
Bureau will encounter restrictions in other countries, or whether other 
countries will cooperate with the Bureau at all, is unknown.

Scope and Methodology: 

As agreed with your offices, our objectives for this report were to (1) 
assess the soundness of the Bureau's test design and its suitability 
for addressing the Bureau's specific research questions, and (2) 
examine what past court decisions have held about Americans' rights and 
obligations abroad that could help inform whether and how they should 
be included in the census.

To assess the soundness of the Bureau's 2004 overseas enumeration test 
design, we interviewed knowledgeable Bureau officials and reviewed 
existing documents that described the Bureau's test objectives, 
research questions, and test plans. We then systematically rated the 
Bureau's approach using a checklist of over 30 design elements that, 
based on our review of program evaluation literature, are relevant to a 
sound study plan. For example, we reviewed the Bureau's approach to 
determine, among other things, (1) how clearly the Bureau presented the 
research objectives, (2) whether the research questions matched the 
research objectives, (3) whether potential biases were recognized and 
addressed, and (4) the appropriateness of the data collection strategy 
for reaching the intended sample population. See appendix II for a 
complete list of the 30 design elements.

We supplemented our ratings on the suitability of the test by gathering 
additional information through telephone and in-person interviews with 
representatives of several stakeholder organizations that represent 
various groups of Americans residing abroad. The organizations included 
Democrats Abroad, Republicans Abroad, Association of Americans Resident 
Overseas, and the American Business Council of the Gulf Countries. In 
addition, we interviewed representatives of the Mexican American Legal 
Defense and Educational Fund, National Coalition for an Accurate Count 
of Asians and Pacific Islanders, and California Rural Legal Assistance, 
Inc., to get their perspectives on the Bureau's plans for counting 
American citizens living in Mexico, particularly migrant and seasonal 
farm workers, a group that the Bureau had trouble counting during the 
2000 Census. These three organizations, while not actively involved in 
the planning of the overseas enumeration test, are members of the 
Secretary of Commerce's Decennial Census Advisory Committee, a panel 
that advises the Bureau on various census-related issues.

To examine what past court decisions have held about Americans' rights 
and obligations living abroad, including their right to be counted in 
the census, we reviewed a judgmental selection of five federal laws 
and/or programs that cover large numbers of Americans stateside, in 
order to determine how those laws and programs treat U.S. citizens 
should they live overseas. We examined federal election law, federal 
income tax law, and federal laws relating to Social Security benefits, 
Supplemental Social Security Income, and Medicare. For each of these 
laws and programs, we reviewed relevant statutes, court decisions, and 
legal analyses.

We requested comments on a draft of this report from the Secretary of 
Commerce, which were sent to us April 13, 2004 (see app. I). We address 
them in the Agency Comments and Our Evaluation section of this report.

Design Limitations Could Undermine the Usefulness of the Overseas Test: 

According to the Bureau, its objectives for the 2004 overseas test are 
"to determine the feasibility, quality, and cost of collecting data 
from U.S. citizens living overseas." To meet those objectives, the 
Bureau developed eight research questions designed to gather data on 
such salient factors as participation levels, data quality, and the 
relative response from the two enumeration modes (Internet and paper 
questionnaire; see appendix III for a sample of the paper 
questionnaire). To assess the overseas test the Bureau is planning to 
complete a series of evaluations due in early 2005. The test objectives 
and related research questions are appropriate as written, but, as 
shown in table 2, because of various methodological limitations, the 
data that will result from the test will not fully answer key questions 
concerning feasibility, data quality, and cost. In short, the Bureau 
overstated the research test's ability to answer its key research 
objectives and, as a result, congressional decision making on this 
issue will be that much more difficult.

Table 2: Research Questions Have Methodological Limitations That Could 
Reduce the Bureau's Ability to Address Research Objectives: 

Census Bureau research question: 1. What is the level of participation 
from U.S. citizens overseas? 
Indicators the Bureau will measure: Participation counts from the three 
test sites; 
Methodological limitations: 
* The universe of overseas Americans is unknown so participation rates 
cannot be determined; 
* The Bureau has not developed residence rules for overseas 
enumeration; 
therefore, who should participate is unclear; 
* Dual nationals and certain other groups may have difficulty 
accessing Internet and other response options.

Census Bureau research question: 2. How good is the quality of the 
data? 
Indicators the Bureau will measure: Item nonresponse; 
Methodological limitations: 
* The Bureau acknowledges that item nonresponse does not provide a 
comprehensive measure of quality.

Census Bureau research question: 3. To what level of geography can the 
Bureau geocode the overseas population? 
Indicators the Bureau will measure: Ability to geocode addresses; 
Methodological limitations: 
* People with noncity-style addresses such as post office boxes will be excluded.

Census Bureau research question: 4. How effective is the overseas 
marketing program? 
Indicators the Bureau will measure: Participation and public awareness; 
Methodological limitations: 
* Participation data are not available for measuring effectiveness; 
* Public awareness of marketing will be difficult to measure.

Census Bureau research question: 5. Can administrative records be used 
effectively? 
Indicators the Bureau will measure: Estimates from administrative 
records compared to the enumeration results; 
Methodological limitations: 
* A large disparity between administrative records counts will make it 
difficult to determine effectiveness; 
* Administrative records were developed for different purposes, 
therefore, comparing against overseas data may yield inconclusive 
results.

Census Bureau research question: 6. What is the relative response from 
the two enumeration modes (Internet and paper return) and their 
effectiveness? 
Indicators the Bureau will measure: Responses by mode and site (France, 
Kuwait, Mexico); 
Methodological limitations: 
* Because the universe of Americans is unknown, analysis is limited to 
providing counts.

Census Bureau research question: 7. Can the Bureau successfully 
implement an appropriate invalid return detection (IRD) system in an 
overseas enumeration? 
Indicators the Bureau will measure: Number of valid returns (a valid 
return is a questionnaire where at least one person in the household 
checked the U.S. citizen box, provided a valid Social Security or 
passport number, and was subject to and passed an algorithm. All other 
returns are invalid.); 
Methodological limitations: 
* The Bureau was unable to negotiate use of passport files with the 
State Department and they will not be used for IRD; 
* The Bureau cannot verify a questionnaire's point of origin received 
via Internet and therefore cannot determine whether respondents are 
truly living in the test sites; 
* Some people may be reluctant to provide a Social Security or passport 
number.

Census Bureau research question: 8. What are the primary barriers of 
integrating an overseas enumeration with the stateside enumeration 
process? 
Indicators the Bureau will measure: Lessons learned; 
Methodological limitations: 
* Overseas and stateside data will not be integrated in this test; 
* Unforeseen problems could exist that will not be revealed until the 
data are actually integrated. 

Source: GAO analysis.

[End of table]

Research Question #1: Participation Data Could be Unreliable: 

For the decennial census, the Bureau uses participation data as a key 
indicator of public cooperation with the census. The Bureau measures 
participation levels by what it calls the "return rate," which it 
calculates as a percentage of all forms in the mailback universe 
(excluding vacant and nonexistent housing units) from which it receives 
a questionnaire.

Stateside, the Bureau is able to perform this calculation because, as 
noted above, it has a master address list of all known housing units in 
the United States, an inventory that takes the Bureau several years and 
considerable resources to compile.

However, no such address list exists for overseas Americans. 
Consequently, participation rates for overseas Americans cannot be 
calculated and the Bureau will only be able to tally the number of 
responses it receives (both overall for each site and within several 
demographic categories) and compare the results to counts obtained from 
administrative records. The sources for the records include a 
combination of tax, Medicare, and State Department data, as well as 
foreign census data if available. For a variety of reasons--some of 
which the Bureau has already acknowledged--the information generated 
from this exercise may not be relevant.

First, the administrative records were developed for different 
purposes, and as a result, are not well suited as a base for comparing 
against overseas counts; thus, their relevance is uncertain. As the 
Bureau has already reported, each of the records it plans to use to 
compare to the census counts has coverage limitations.[Footnote 12] 
Further, as each of these records is associated with particular 
demographic groups, they could introduce systemic biases (we discuss 
potential problems with administrative records in greater detail below, 
under Research Question #5).

Second, the census counts could be problematic because it is unclear 
who should participate in the overseas census, which in turn could 
confuse potential respondents. For the stateside enumeration, to 
determine where an individual should be counted, the Bureau uses the 
concept of "usual residence," which it defines as "the place where a 
person lives and sleeps most of the time." The Bureau has developed 
guidelines, which it prints on the stateside census questionnaire, to 
help people figure out who should and should not be included. However, 
the Bureau has not developed similar guidance for the overseas test. 
According to the Bureau, this was intentional; because this was an 
initial feasibility test, the Bureau did not want to restrict response, 
but rather to encourage the widest possible participation.

Further, the guidance the Bureau has developed, which is available on 
its Web site and promotional literature, is vague and could confuse 
potential respondents. For example, the guidelines inform potential 
respondents that, "All U.S. citizens living in France, Mexico and 
Kuwait, regardless of shared citizenship, can and should participate in 
the test. U.S. citizens on vacation or on short business trips should 
not." Unclear is what constitutes a short or long business trip. Is it 
3 weeks or 3 months? Does it matter whether one stays in a hotel or an 
apartment? Also, should naturalized U.S. citizens, some of whom may not 
return to the United States, participate? What about children born in 
the United States to noncitizens, but who only lived in the United 
States a short time? Should students spending a semester abroad but who 
maintain a permanent residence stateside be included? Without clear 
residence rules and appropriate guidance indicating who should be 
counted, it is quite possible that some people might inappropriately 
opt in or out of the census, which would reduce the quality of the 
data. Bureau officials have told us that they are working to develop 
residence rules that it will apply if there is a second overseas 
enumeration test in 2006.

Participation data might also be problematic because the Bureau's 
enumeration methods strategy might not be as effective with certain 
groups compared to others. To the extent this occurs, it could 
introduce a systemic undercount. This is particularly true for dual 
nationals who, for cultural reasons, may not think of themselves as 
American citizens.

For example, according to representatives of two advocacy groups we 
contacted, Mexican dual nationals include migrant farm workers, a group 
that often consists of poor, less-educated people living in rural 
areas. They noted that this population group has low literacy levels 
and thus might not understand the questionnaire, and is not likely to 
have Internet access. At the same time, they are not likely to pick up 
a copy of the questionnaire at an embassy. Further, the barriers that 
make it difficult to count migrant farm workers in the United States, 
such as a distrust of government and the fact that they may speak 
indigenous languages, also make it difficult to count this group in 
Mexico.

Research Question #2: Information on Data Quality Will be Limited: 

The Bureau plans on measuring the quality of the data collected in the 
overseas test by tabulating item nonresponse, which refers to whether a 
respondent completed a particular question. The Bureau is to calculate 
this information by enumeration mode, test site, and various 
demographic categories. The Bureau also plans to compare this measure 
of nonresponse for key variables to those obtained in an earlier, 
stateside test held in 2003 by tabulating the rate respondents did not 
complete a particular question.

According to the Bureau, patterns of item nonresponse are critical for 
improving question design, training, and procedures. However, as the 
Bureau acknowledges in its study plan for evaluating the quality of the 
overseas enumeration data, item nonresponse by itself does not address 
the quality of the data. Thus, at the end of the test, the Bureau will 
have, at best, only limited information on the quality of the overseas 
data.

By comparison, the Bureau's guidelines for measuring data quality in 
other surveys they conduct use measures such as coverage, unit response 
rates, imputation rates, and data collection errors. Because the Bureau 
lacks information on the universe of overseas Americans it will be 
unable to calculate these indicators. Therefore, it is misleading for 
the Bureau to state in its research objectives that it will determine 
the quality of the overseas data, when in fact it will deliver 
something far more limited. Comprehensive measures of data quality are 
critical because they could help Congress decide whether the data are 
sufficiently reliable to use for specific purposes. If the numbers were 
to be used to obtain a simple count of Americans abroad, absolute 
precision is not as critical. However, for other uses of the data, 
particularly congressional apportionment and redistricting, the 
quality of the data would need to be far higher.

Research Question #3: The Bureau Will be Unable to Geocode Certain U.S. 
Addresses: 

Counting people in their correct locations is essential for 
congressional apportionment, redistricting, and certain other uses of 
census data. With respect to Americans abroad, if the data are to be 
used for apportionment, the Bureau would need to assign respondents to 
a specific state. For purposes of redistricting and allocating federal 
funds, the Bureau would need to assign overseas respondents to specific 
neighborhoods and street addresses in the United States--a far more 
challenging task. Geocoding is the process of linking an address in the 
Bureau's Master Address File (MAF) to a geographic location in the 
Bureau's geographic database, known as the Topologically Integrated 
Geographic Encoding and Referencing (TIGER) mapping system. To obtain 
this information, the overseas enumeration form asks respondents to 
indicate their U.S. state of last residence, and their last street 
address within that state.

Although the Bureau is assessing the level of geography to which it can 
geocode the overseas population, an important limitation is that the 
Bureau will not be able to make this assessment for people who live in 
certain noncity-style U.S. addresses; that is, U.S. addresses without a 
housing number and/or street name. Specifically, people whose U.S. 
addresses consist of a post office box will be excluded from the study. 
For the stateside enumeration, Bureau employees canvass the country, 
identify noncity-style addresses, and mark the locations of those 
residences on a map. During the 2000 Census, around 20 percent of U.S. 
households had noncity-style addresses.

Indeed, the overseas enumeration questionnaire instructs respondents 
not to provide a post office box number for their last stateside 
address. However, if a respondent's address includes a post office box 
or rural route number, it is unclear how they are supposed to complete 
this question. Also unclear is how migrant farm workers, who may not 
have had an address in the United States, would complete this question.

To the extent they leave the question blank, the Bureau would be unable 
to distinguish between those people who did so because they have a 
noncity-style address, or left it blank for privacy or other reasons. 
This could affect the accuracy of the Bureau's assessment. Moreover, 
the construction of the question could introduce a systemic bias 
because those states with large rural areas are more likely to have 
noncity-style addresses.

The Bureau's evaluation plan recognizes that respondents in the 2004 
overseas enumeration test may provide noncity-style addresses which 
cannot be geocoded by the TIGER system. The Bureau intends to provide 
data on how many city-style and noncity-style addresses could be 
geocoded.

Research Question #4: Measuring the Marketing Program's Effectiveness 
Will be Difficult: 

Although the Bureau has taken a number of steps to publicize the 
overseas enumeration, evaluating the effectiveness of that effort will 
present a challenge. The Bureau awarded a $1.2 million contract to a 
public relations firm to develop a promotion strategy for the overseas 
enumeration test. As part of that effort, the public relations firm 
identified a number of stakeholder organizations that represent U.S. 
citizens living overseas in each of the three test countries. The 
organizations included advocacy groups, universities, church groups, 
and corporations.

The Bureau anticipates that these stakeholders will help get the word 
out via e-mail, newsletters, and other media that a test census of 
Americans overseas is underway. In addition, the Bureau is to provide 
copies of the overseas questionnaire to stakeholders so that they can 
distribute them to their members and constituents. As noted earlier, 
questionnaires will also be available on the Internet, as well as at 
public places that Americans may visit, such as embassies and 
consulates. The Bureau has produced posters and pamphlets to promote 
the test (see fig. 1). The Bureau also plans to have articles about the 
census test placed in newspapers and magazines and stories run on local 
television and radio. Although paid advertising was not part of its 
original plans, the Bureau later decided to run a limited amount of 
paid advertising in Mexico and France.

Figure 1: Census Bureau Poster Advertising the Overseas Census: 

[See PDF for image] 

Source: U.S. Census Bureau.

[End of figure] 

The Bureau will attempt to gauge the effectiveness of the marketing 
program by measuring participation (as measured by the number of 
responses) and public awareness. According to the Bureau, respondents 
who submit a questionnaire via the Internet will be asked to complete a 
short survey eliciting information on how they learned about the census 
test and what motivated them to participate. No similar survey is 
planned for people who pick up their surveys at an embassy or other 
distribution site. Thus, the Bureau will not have a parallel set of 
data from a group of respondents that might be demographically or 
behaviorally different from Internet respondents.

The Bureau does, however, expect to conduct focus group interviews and 
debriefings to obtain feedback from mail respondents and stakeholder 
organizations. Focus group interviews targeting nonrespondents are 
planned as well.

Yet, as the Bureau acknowledges, participation, or the final count of 
U.S. citizens living in the selected countries, will only be an 
indicator of the number of people that heard about the test, completed 
the questionnaire, and submitted it to the Census Bureau. It will not 
be able to measure the Bureau's success in getting Americans to respond 
because the universe of Americans overseas is unknown.

Public awareness will also be difficult to measure because it includes 
an unknown number of people who were aware that a test census was being 
conducted but chose not to respond. Nevertheless, as noted above, the 
Bureau intends to interview both respondents and nonrespondents in an 
effort to determine their awareness and motivation for responding or 
not responding to the census test. To the extent the Bureau conducts 
these interviews, it will be important for it to include hard-to-count 
groups, such as dual nationals and migrant farm workers, that may have 
been outside the reach of the Bureau's marketing campaign.

Research Question #5: Use of Administrative Records Needs to be 
Thoroughly Tested: 

The Bureau plans on using administrative records such as Medicare and 
passport data to provide comparison information to assess (1) 
participation, (2) an invalid return detection system, and (3) the 
records' potential use for building an address list. Specifically, the 
Bureau plans to compare the number of people counted at each site to 
federal tax, Medicare, U.S. Department of State, and foreign census 
records. While it is important for the Bureau to assess the utility of 
administrative records, it is unclear from the Bureau's study plans how 
it will make this determination given what Bureau officials have said 
is a large disparity between administrative record counts on the number 
of Americans living overseas at the three test sites.

Further, because these administrative records were developed for 
different purposes and as a result are not well suited as a base for 
comparing against overseas counts, their relevance is uncertain. For 
example, not all American citizens who live abroad file tax returns; 
dependents are not always listed on tax returns; and dependents that 
are included in the tax form may not be U.S. citizens. After living 
abroad 30 days, Americans are no longer eligible for Medicare benefits; 
therefore, Medicare records may not be the most useful and only apply 
to U.S. citizens over 65 years old. U.S. State Department records are 
nonofficial and according to the Bureau inaccurate because the 
Department of State does not officially track either the number or 
location of U.S. citizens living in other countries. Finally, the type 
of administrative records kept by each country is unknown and earlier 
Bureau research found that census data from foreign countries do not 
contain the detailed information required for apportionment, 
redistricting, or other census uses.[Footnote 13]

Research Question #6: Effectiveness of Enumeration Methods Will be 
Difficult to Assess: 

To determine the relative response from the two enumeration modes and 
their relative effectiveness, the Bureau will look at such data as the 
timing of returns by mode and site, and whether one mode provided more 
valid returns than the other. Respondents can either fill out a paper 
questionnaire or complete the form on the Internet. The Bureau will 
deploy an invalid return detection system to determine whether a form 
is valid and responses will be tallied by mode and by country.

However, the Bureau recognizes that its analysis will contain several 
limitations. Key among them is that it will not be able to determine 
participation rates because the universe of Americans overseas is 
unknown. As a result, the analysis is limited to a count of the total 
number of forms returned at each site. In light of this and other 
limitations, it is unclear what conclusions the Bureau will be able to 
draw about the effectiveness of the two response modes.

Indeed, just because one enumeration mode results in a larger number of 
returns from a particular demographic group--older Americans, for 
example--may not necessarily have anything to do with the mode itself. 
As a result, it would be inappropriate to say that one enumeration 
method was more effective than the other in counting senior citizens. 
This is because there are other factors that can influence the mode 
such as advertising or accessibility to the Internet or the paper 
questionnaire. Overall, while it will be important to collect 
information on the returns by method of enumeration, this information 
should not be construed as a measure of the effectiveness of that mode.

Research Question #7: System to Detect Invalid Returns Has 
Shortcomings: 

Part of the processing of overseas returns involves validating that the 
respondents are within the scope of the enumeration; that is, that they 
in fact reside in one of the three overseas test areas. Thus, the 
Bureau needs a better method to detect invalid returns.

To determine whether responses are within scope, the overseas 
enumeration questionnaire asks respondents to provide their Social 
Security and U.S. passport numbers. Although we agree with the 
importance of determining whether respondents reside overseas, the 
Bureau's analysis will not assess this. Rather, the Bureau defines a 
valid return as one where at least one person in the household checked 
the U.S. citizen box on the questionnaire, provided a valid Social 
Security or U.S. passport number, and has been subject to and passes an 
algorithm that analyzes data from the questionnaire. All other returns 
are invalid. Thus, what the Bureau is really measuring is whether a 
questionnaire is eligible for further processing, and not whether the 
respondent lives abroad. Put another way, anyone who completes a 
questionnaire with valid data, including Social Security numbers, would 
be considered a valid return, regardless of whether the individual 
lived in the test areas.

This is not an unlikely scenario given the way the Bureau set up its 
Internet site. Indeed, anyone--even if they live outside of the three 
test sites---can be included in the overseas count, so long as they 
provide the required information. The reason they can slip through the 
invalid return detection system is because the Bureau is unable to 
confirm the point of origin for questionnaires completed on the 
Internet. The Bureau is aware of this gap in the invalid return 
detection system, but has been unable to resolve this condition.

Another potential problem is the Bureau's sole reliance on Social 
Security numbers to validate returns for the 2004 test. The Bureau had 
also wanted to use passport numbers to validate returns. Although the 
Bureau has been negotiating with the Department of State for access to 
the passport database, the Bureau does not expect this to occur in time 
for use in the 2004 test. Bureau officials said they were aware that 
there was a strong possibility that they would not be able to gain 
access to the passport file because the Bureau had not worked with 
Department of State data and that a memorandum of understanding would 
first need to be in place before the data would be released. 
Nevertheless, the Bureau believes that the impact of this would be low 
if it could be shown that using Social Security numbers alone to 
validate returns was sufficient.

A third potential problem is that some people may be reluctant to 
provide their Social Security and passport numbers for privacy reasons. 
Based on the Bureau's research, requesting this information could 
reduce participation levels.

Research Question #8: Overseas Enumeration Will Not be Integrated With 
the Stateside Enumeration: 

The Bureau, based on its earlier research, has already identified a 
list of barriers to integrating overseas with stateside data. They 
include different questionnaire content for the overseas form, 
detecting and eliminating duplication within and between overseas and 
stateside enumerations, timely geocoding of addresses, and limited 
resources.

The Bureau plans to document the lessons learned from the 2004 overseas 
test and how they might apply to a more integrated test in 2006. In 
particular, the Bureau is to focus on the issues encountered or 
associated with collecting, capturing, and processing overseas data. 
While it will be important for the Bureau to thoroughly document these 
issues and their implications for integrating the two data sets, the 
Bureau does not intend to actually integrate any data from the overseas 
test with data being collected from a parallel stateside test it's 
conducting at three Georgia counties and in Queens, New York. Without 
an actual integration, the Bureau may miss problems that will not 
likely be detected until a next test in 2006.

Overseas Test Design Has Other Shortcomings: 

In addition to the limitations noted above, the overseas enumeration 
test has other limitations that will affect the Bureau's ability to 
answer its key research objectives.

Cost Data Will Not Identify the Cost of Conducting Future Tests or an 
Overseas Enumeration in 2010: 

Although one of the Bureau's objectives for the overseas enumeration 
test is to determine the cost of collecting data from overseas 
citizens, the Bureau's test design lacked a specific research question 
aimed at obtaining this information. More importantly, the cost 
information that the Bureau will collect will be of limited value 
because it will not be used to estimate the costs of future tests, nor 
model the costs of conducting a broader overseas enumeration in 2010.

The Bureau developed a cost model for the 2000 Census that provided the 
agency with an automated means to estimate staffing and budget 
requirements. The Bureau used the cost model to support the budget 
process, as well as to answer questions from Congress, the Office of 
Management and Budget (OMB), and our office. The cost model could also 
estimate the budgetary impact of certain assumptions and alternative 
census-taking scenarios.

The Bureau maintains that it would need more data than those that are 
now available to develop a cost model for an overseas enumeration in 
2010. Nevertheless, while only at the beginning stages, the Bureau has 
some data points to begin developing a cost model for overseas 
enumeration or, at a minimum, for identifying major cost components. 
For example, Bureau officials told us that their agency will track and 
report the marketing, printing, postage, data capture, and processing 
costs for conducting the overseas enumeration at the three test sites. 
All of the costs provided by the Bureau are direct costs and could be 
specifically and uniquely attributed to a cost model for the overseas 
enumeration. Indirect costs, however--those that are not easily 
attributable to the overseas enumeration such as executive management 
or technical labor--would still need to be determined and captured.

Cost will be an important factor to consider when making a decision on 
whether to enumerate Americans overseas. The cost of the 2010 Census, 
now estimated at more than $11.3 billion in current dollars, is the 
most expensive enumeration in the nation's history. Consequently, it 
will be critical for the Bureau to have a mechanism for accurately and 
quickly estimating overseas enumeration costs so that Congress, other 
oversight groups, and the Bureau itself can have reliable information 
on which to base or advise decisions.

Testing Questionnaire Would Help Ensure Clarity and Accurate Responses: 

Testing a questionnaire with a sample of intended respondents before it 
is distributed is a standard approach that survey organizations employ 
to ensure questions are clear and understandable, and that respondents 
will be able to provide accurate information. In short, testing is an 
important quality assurance procedure that increases the likelihood 
that respondents will provide the information needed and help reduce 
the likelihood of inaccurate responses.

The Bureau is aware of the benefits of testing questionnaires. In 
preparation for the 2000 Census, Congress budgeted millions of dollars 
for the Bureau to develop and test questionnaires during the 1990s, 
which it did using focus groups and one-on-one interviews. Moreover, 
the Bureau's policy requires that demographic survey questionnaires be 
tested.

However, the Bureau did not test the overseas enumeration 
questionnaire. Instead, the Bureau gave stakeholders the opportunity to 
review and comment on the questionnaire before it went to OMB for final 
approval. Stakeholder feedback was generally positive. One change that 
was made in response to stakeholder feedback was that "United States" 
was added to "passport" in the question that asked for a respondent's 
passport number. While sharing the questionnaire with stakeholders is 
important, it should not be seen as a replacement of questionnaire 
testing.

According to Bureau officials, the Bureau developed the overseas 
questionnaire by modifying the Census 2000 short form to collect data 
needed for the overseas count. New questions asked respondents for 
their employment status, Social Security number, and passport number. 
The Bureau believed that the new questions did not require testing. 
However, as stated earlier, collecting Social Security numbers and 
passport numbers could be problematic. While some stakeholders believed 
this to be acceptable, other stakeholders believe it could reduce 
participation, especially in Mexico where dual nationals reside. In 
addition, the questionnaire requests data on everyone in the household 
even if a person is not a U.S. citizen. These questions could be seen 
as too intrusive and potentially could stop someone from completing the 
form, thus resulting in an undercount. Since there was no testing of 
the questionnaire, the Bureau cannot be certain of the impact of these 
questions.

Data Analysis Will Not Provide Congress with Requested Information: 

In a 2001 report to Congress, "Issues of Counting Americans Overseas in 
Future Censuses," the Bureau indicated that it would provide Congress 
with data on the number of people in the military, federal, private 
business, nonprofit, and other categories. However, the Bureau is only 
collecting data on the number of military and federal workers; people 
working in other sectors will be grouped in an "other" category. The 
reason that the Bureau is unable to provide the additional breakdown is 
because the length of the questionnaire did not allow for additional 
check boxes. Therefore, the Bureau collapsed people working in the 
private sector and nonprofit organizations and others into one 
category.

The Bureau Has Identified a Number of New Challenges to Counting 
Overseas Americans: 

The soundness of the test design notwithstanding, the Bureau has 
already identified several country-specific challenges to counting 
American citizens at each of the test sites. Together, they suggest 
that an enumeration of Americans on a more global scale in 2010 would 
introduce a number of unforeseen obstacles that the Bureau would need 
to address on a country-by-country basis. For example, shortly before 
the test was to begin in France, the French government contacted the 
Census Bureau indicating that French law prohibited the collection of 
race and Hispanic origin data. Furthermore, they were also opposed to 
the U.S. government asking for information on persons who were not 
American citizens. The Bureau worked with the French government and it 
was agreed that an advisory would be posted on the Internet site 
explaining that under French law it was not mandatory to respond to the 
questionnaire.

Problems have also surfaced in Mexico and Kuwait. Stakeholders and 
Bureau officials have told us that the mail system in Mexico is not 
always reliable. The concern is that the questionnaires may not make it 
to the Census Bureau, or arrive too late to be processed. In Kuwait, 
security concerns have prevented the Census Bureau from posting the 
location of sites where Americans can pick up the questionnaire. While 
the impact of these problems is difficult to quantify, it may prevent 
some Americans from completing the questionnaire and being counted.

The Rights and Obligations of Americans Overseas Vary: 

Americans residing abroad do not have the same rights and obligations 
under federal programs and activities as Americans living in the United 
States. In order to determine the rights and obligations of Americans 
residing abroad, one must examine the specific statutes governing each 
program. For this study, we examined whether overseas Americans can 
vote in federal elections; are subject to federal income tax; and can 
receive Social Security, Supplemental Security Income, and Medicare 
benefits (see table 3). Stateside, these programs cover millions of 
Americans; whether and how they extend to Americans living overseas 
could help inform the treatment of U.S. expatriates in the decennial 
census, to the extent there are any patterns.

Table 3: The Applicability of Federal Programs to Americans Overseas 
Varies: 

Federal program: Voting; 
Applicability to U.S. citizens residing overseas: U.S. citizens 
residing outside of the country have the right to vote in federal 
elections in their last state of residence as outlined in federal 
statute.

Federal program: Federal income tax; 
Applicability to U.S. citizens residing overseas: U.S. citizens are 
taxed on their worldwide income, subject to certain exclusions in the 
Internal Revenue Code.

Federal program: Social Security payments; 
Applicability to U.S. citizens residing overseas: Americans living 
abroad are entitled to receive benefits, subject to a few exceptions.

Federal program: Supplemental Security Income; 
Applicability to U.S. citizens residing overseas: Individuals are not 
entitled to benefits once they reside outside of the United States for 
30 days.

Federal program: Medicare; 
Applicability to U.S. citizens residing overseas: No coverage for 
Americans outside of the United States, with a limited exception for 
services occurring near U.S. borders.

Source: GAO analysis of applicable statutes and court decisions.

[End of table]

Voting Rights: 

American citizens who reside outside of the United States have the 
right to vote in federal elections under the 1986 Uniformed and 
Overseas Citizens Absentee Voting Act (UOCAVA).[Footnote 14] Under this 
law, U.S. citizens residing on foreign soil can vote in federal 
elections as absentee voters of their last state of residence, even if 
they have no intention of ever returning to the United States. 
(American citizens residing in U.S. territories, however, cannot so 
vote. The territories include Puerto Rico, the Northern Mariana 
Islands, the U.S. Virgin Islands, Guam, and American Samoa.): 

UOCAVA repealed legislation enacted in 1955 that was designed to 
prevent members of the Armed Forces and their families from being 
denied their voting rights when absent from their home or in a far-off 
place. The goal was to make it easier for military personnel to cast 
votes in any federal primary, general, or special election through 
absentee balloting procedures.[Footnote 15] While the 1955 law was 
amended several times to, among other purposes, specifically include 
other Americans living overseas, in 1986 Congress acknowledged that 
there was a legitimate need for further legislation. UOCAVA's main 
purpose was to facilitate absentee ballot voting, while also providing 
"for a write-in absentee ballot that may be used in Federal general 
elections by overseas voters who, through no fault of their own, fail 
to receive a regular [State] absentee ballot in sufficient time to vote 
and return the ballot prior to the voting deadline in their 
State."[Footnote 16]

Federal Income Tax: 

U.S. citizens are taxed on their worldwide income, subject to certain 
exclusions specified in the Internal Revenue Code. In 1913, the 
enactment of the Sixteenth Amendment to the U.S. Constitution gave 
Congress "the power to lay and collect taxes on income, from whatever 
source derived, without apportionment among the several States, and 
without regard to any census or enumeration." After ratification of 
this constitutional provision, Congress imposed a tax on the net income 
of every U.S. citizen, wherever they lived.[Footnote 17]

That the Constitution vests Congress with the power to tax Americans 
living overseas on their income earned outside of the United States was 
reinforced by the Supreme Court in 1924 when it ruled: "[g]overnment, 
by its very nature, benefits the citizen and his property wherever 
found, and therefore has the power to make the benefit complete. Or, to 
express it another way, the basis of the power to tax was not and 
cannot be made dependent upon the situs of property in all cases, it 
being in or out of the United States, nor was not and cannot be made 
dependent upon the domicile of the citizen, that being in or out of the 
United States, but upon his relation as citizen to the United States 
and the relation of the latter to him as citizen."[Footnote 18]

There are statutory exclusions, however. Generally, a U.S. citizen or 
resident may exclude a portion (ascending each year to $80,000 during 
or after 2002) of his earned income if he is a resident of a foreign 
country for a full calendar year or is physically present in a foreign 
country for 330 days of any 12 consecutive months and otherwise meets 
certain requirements.[Footnote 19] This foreign earnings exclusion 
principally aims to encourage U.S. citizens and residents to work 
abroad without worrying about how it might increase their tax 
liability. Indeed, in view of the nation's continuing trade deficits, 
Congress deemed it important to allow Americans working overseas to 
contribute to the effort to keep American business 
competitive.[Footnote 20]

Social Security Payments: 

The Social Security Act provides individuals over the age of 62 who 
have worked for a minimum number of years with monthly benefit 
payments.[Footnote 21] Social Security checks generally follow 
recipients wherever they go around the world, subject to only a few 
exceptions.[Footnote 22] In fact, Social Security payments continue no 
matter how long a beneficiary stays outside of the United States--even 
if the individual retires overseas.[Footnote 23] Thus, if you are a 
U.S. citizen residing abroad you generally continue to receive your 
monthly benefits.[Footnote 24]

Supplemental Security Income: 

Supplemental Security Income (SSI) is a "need-based" program designed 
to help individuals who are over the age of 65, blind, or 
disabled.[Footnote 25] SSI benefits cease once a recipient remains 
outside the United States for a period of at least 30 days.[Footnote 
26] If the recipient returns to this country within 30 days, SSI 
benefits are to continue as before.[Footnote 27]

Medicare Coverage: 

Since Medicare became effective almost 35 years ago, the program has 
excluded coverage for health care services received outside of the 
United States, even if those services are medically necessary.[Footnote 
28] There is a limited exception for services occurring near U.S. 
borders.[Footnote 29]

The constitutionality of foreign exclusion was raised in a 1986 court 
decision.[Footnote 30] In that case, plaintiffs argued that by leaving 
Medicare beneficiaries without health care coverage once they leave the 
United States, the foreign exclusion deters these same individuals from 
traveling overseas and consequently infringes on their constitutional 
right to travel abroad. The district court, however, found the Medicare 
foreign exclusion to pass constitutional muster. The court reasoned 
that the difficulties of administering medical services abroad and the 
concern that Medicare funds be spent within the United States were not 
particularly compelling, but were rationally based. It thus concluded 
that the foreign exclusion satisfied the rational basis test and 
summarily dismissed the plaintiff's claim.[Footnote 31]

Legislation Would be Needed To Require the Census Bureau to Enumerate 
Americans Abroad: 

The Bureau has the discretion to decide whether to count American 
citizens abroad. Indeed, there is nothing in the Constitution, the 
Census Act, or court decisions that would require the Bureau to count 
Americans living overseas, or to not count such individuals. 
Consequently, if Congress wanted to require the Census Bureau to 
include this population in the 2010 Census, legislation would be 
needed.

The issue of whether, and if so how, to count Americans living overseas 
was raised in federal court after both the 1990 and 2000 censuses. 
Massachusetts challenged the results of the 1990 Census claiming that 
it lost a seat in the House of Representatives as a result of the 
Secretary of Commerce's decision to enumerate and apportion federal 
employees stationed abroad.[Footnote 32] Conversely, Utah challenged 
the results of the 2000 Census maintaining that it lost a congressional 
seat because no overseas Americans other than federally affiliated 
groups were included in the apportionment numbers.[Footnote 33] In both 
cases, the courts determined that the Census Bureau has discretion 
under the Constitution and the Census Act to decide whether to count 
Americans residing overseas.

Conclusions: 

The design of the Bureau's overseas enumeration test falls short in 
many respects, and the data that the Bureau will collect as a result of 
this exercise may only partially answer key questions relating to 
feasibility, cost, and data quality. The Bureau overstated the test's 
ability to answer its key research objectives and, as a result, 
congressional decision making on this issue will be that much more 
difficult.

The full results of the overseas enumeration test will not be available 
until early 2005, when the Bureau expects to issue the last of a series 
of evaluations. However, its experience thus far has made it clear that 
counting Americans abroad as an integral part of the 2010 Census would 
be an extraordinary challenge, one that would introduce new resource 
demands, risks, and uncertainties to an endeavor that is already 
costly, complex, and controversial.

That said, to the extent that better data on the number and 
characteristics of Americans abroad might be useful for various 
policymaking and other nonapportionment purposes, such information does 
not necessarily need to come from the decennial census. This data 
could, in fact, be acquired through a separate survey or some other 
type of data collection effort, although it would still be a difficult 
undertaking.

The Bureau is unlikely to decide--and in fact should not decide--on its 
own, whether or not to enumerate Americans abroad, and will need 
congressional guidance on how to proceed. Therefore, to give the Bureau 
as much planning time as possible, it will be important for Congress to 
soon decide whether the Bureau should be required to count this 
population group as part of the 2010 Census or as part of a separate 
data collection effort or whether there are so many obstacles to a 
successful overseas count regardless of the approach that the Bureau 
should shelve any plans for further research and testing.

Should Congress desire an overseas count--as part of the decennial 
census or a separate effort--it should consider providing the Bureau 
with input on how it expects to use the data on overseas Americans. The 
purposes of the data drive the design of the enumeration; therefore, 
once the Bureau has a clear idea on how the data will be used, it would 
be better positioned to plan a test that will more accurately assess 
the feasibility of the procedures, methodology, and resource 
requirements needed to accomplish the type of count that Congress 
desires. Moreover, if a second test of enumerating Americans abroad is 
needed in 2006, it will be important for the Bureau to address the 
shortcomings of the design of the 2004 overseas test. Conducting a 
second test without this information and a sound design to fulfill it 
would not be cost-effective.

Matters for Congressional Consideration: 

In order to give the Bureau as much planning time as possible, Congress 
may wish to consider coming to an early decision on whether the Bureau 
should be required to enumerate overseas Americans, and if so, whether 
they should be counted as part of the decennial census or by some 
other, separate data collection effort. Should Congress desire an 
overseas count--be it part of the decennial census or a separate data 
collection effort--it should consider telling the Bureau how the data 
would be used (e.g., for purposes of apportionment, redistricting, 
allocating federal funds, or a tally of the U.S. overseas population). 
This information would enable the Bureau to more thoroughly evaluate 
procedures and resources needed to meet Congress's specific 
requirements, and ultimately provide Congress with better information 
with which to gauge the feasibility of such an approach.

Recommendations for Executive Action: 

To the extent that the Bureau proceeds with plans to conduct a second 
feasibility test in 2006, we recommend that the Secretary of Commerce 
direct the Bureau to take appropriate steps to improve the soundness of 
the test design and better address the objectives of an overseas 
enumeration. Specific steps should include the following 12 actions: 

* Be more transparent with Congress and other stakeholders on what 
variables the Bureau is able to quantitatively measure, as well as what 
research questions the Bureau can and cannot answer.

* Develop and pretest clear residence rules and appropriate guidelines 
on who should be included in the count to prevent confusing prospective 
respondents.

* Ensure that its outreach and promotion strategy, data collection 
methods, and other aspects of the design are free from cultural and 
other biases that could introduce systemic errors.

* Explore the possibility of developing more comprehensive measures of 
data quality.

* Test the Bureau's ability to geocode certain noncity-style addresses 
such as those with post office box numbers.

* Research how best to market the overseas census to hard-to-count 
groups and other, less visible, segments of a country's overseas 
American population.

* Develop procedures to determine whether a return is within the scope 
of the enumeration--i.e., that it is truly from an overseas source.

* Actually integrate overseas data with stateside data to more 
thoroughly test this activity.

* Develop a cost model to provide the Bureau and Congress with better 
estimates of the costs of conducting an overseas census under different 
methodological and other scenarios.

* Thoroughly pretest the overseas questionnaire with various groups of 
potential respondents to ensure the questions are clear, appropriate, 
and can be accurately answered by the unique population groups that are 
found overseas.

* Add more response options to the questionnaire item concerning 
respondents' primary activity. Specific information on whether an 
individual is retired, a student, or working for a private company, 
etc., could provide the Bureau with valuable data on the 
characteristics of overseas Americans that could be important for some 
of the purposes for which the data might be used, and just as 
important, could provide the Bureau with invaluable marketing data that 
the Bureau could use to develop a more effective outreach and promotion 
campaign.

* Work with Congress and other stakeholders to develop a broad 
consensus on what would be acceptable levels of accuracy, completeness, 
participation, and other key measures of performance.

Agency Comments and Our Evaluation: 

The Secretary of Commerce forwarded written comments from the U.S. 
Census Bureau on a draft of this report on April 13, 2004, which are 
reprinted in appendix I. The Bureau generally agreed with our key 
findings, conclusions, and recommendations, and suggested some 
additional context, technical corrections, and clarifications, which we 
have incorporated.

The Bureau disagreed with our recommendation that it be more 
transparent with Congress and other stakeholders on what variables the 
Bureau is able to quantitatively measure, as well as what research 
questions the Bureau can and cannot answer. The Bureau believes that it 
has always been transparent about its plans and the likely limitations 
of this first test.

Although the Bureau's test plan describes the limitations associated 
with answering its various research questions, nowhere does it disclose 
that its fundamental research objective to "determine the feasibility, 
quality, and cost of collecting data from U.S. citizens living 
overseas" will only be partially answered. Indeed, none of the 
documentation we reviewed, including the test project plan or briefing 
slides provided to congressional staff in April 2003, indicated either 
explicitly or implicitly that this test was, as the Bureau maintains, 
"only the most basic assessment of feasibility," or that the Bureau 
would not be able to fully answer its key questions regarding 
feasibility, data quality, and cost. For example, as noted in our 
report, even though one of the principal objectives of the test was to 
determine the cost of collecting data from U.S. citizens living 
overseas, the Bureau's test design lacked a research question for this 
objective.

The Bureau agreed, however, that as it completes its evaluations and 
documents its findings from the test, it will be "critical" to 
highlight the various qualitative and quantitative limitations that 
could affect congressional deliberations on this subject.

As agreed with your offices, unless you release its contents earlier, 
we plan no further distribution of this report until 30 days from its 
issue date. At that time we will send copies to other interested 
congressional committees, the Secretary of Commerce, and the Director 
of the U.S. Census Bureau. Copies will be made available to others upon 
request. This report will also be available at no charge on GAO's Web 
site at [Hyperlink, http://www.gao.gov].

If you or your staff have any questions concerning this report, please 
contact me at (202) 512-6806 or by e-mail at [Hyperlink, 
daltonp@gao.gov] or Robert Goldenkoff, Assistant Director, at (202) 
512-2757 or [Hyperlink, goldenkoffr@gao.gov]. Key contributors to this 
report were Lisa Pearson, Charlesetta Bailey, Betty Clark, Ellen 
Grady, Ronald La Due Lake, Andrea Levine, and Daniel Messler.

Signed by: 

Patricia A. Dalton, 
Director Strategic Issues: 

[End of section]

Appendixes: 

Appendix I: Comments from the Department of Commerce: 

THE SECRETARY OF COMMERCE 
Washington, D.C. 20230:

April 13, 2004:

Ms. Patricia A. Dalton 
Director, Strategic Issues 
U.S. General Accounting Office 
Washington, DC 20548:

Dear Ms. Dalton:

The U.S. Department of Commerce appreciates the opportunity to comment 
on the:

U.S. General Accounting Office draft report entitled 2010 Census. 
Overseas Enumeration Test Raises Need for Clear Policy Direction. The 
Department's comments on this report are enclosed.

Sincerely,

Signed by: 

Donald L. Evans: 

Enclosure:

Comments from the U.S. Department of Commerce, U.S. Census Bureau, 
Regarding the U.S. General Accounting Office (GAO) Draft Report 
Entitled "2010 Census: Overseas Enumeration Test Raises Need for Clear 
Policy Direction":

Thank you for providing us the opportunity to comment on your draft 
report, "2010 Census: Overseas Enumeration Test Raises Need for Clear 
Policy Direction."

Comments on Overall Conclusions:

We agree with all of the general conclusions stated at the end of this 
draft report:

We agree that the 2004 test will not fully answer all key questions 
regarding feasibility, data quality, and cost, but disagree that the 
Census Bureau has overstated the test. The Bureau has always been clear 
that the test is only a first and most basic assessment of feasibility.

We also agree that our 2004 test "experience thus far has made it clear 
that counting Americans abroad as an integral part of the 2010 census 
would be an extraordinary challenge, one that would introduce new 
resource demands, risks, and uncertainties to an endeavor that is 
already costly, complex, and controversial."

We concur with the GAO's observation that data about the number and 
characteristics of Americans abroad could be collected as part of a 
separate, albeit still difficult, effort from the decennial census.

We believe we have always made it clear that the Census Bureau will not 
decide on its own whether to pursue such an effort for the 2010 census, 
and we fully agree that Congressional guidance is needed on whether and 
how to proceed. We join GAO in urging Congress to make this decision as 
soon as possible.

We also support GAO's conclusion that if the Congress desires an 
overseas count, as part of or separate from the 2010 census, it will be 
important for Congress to indicate the purposes for which these data 
will be used. This information will indeed be critical in developing 
any such effort for 2010 and in designing any further tests.

Comments on Recommendations for Executive Action:

Be more transparent with Congress and other stakeholders on what 
variables the Bureau is able to quantitatively measure, as well as what 
research questions the Bureau can and cannot answer. We believe we have 
been very transparent about our plans and about the likely limitations 
of these initial test results. As we complete our evaluations and 
document our findings from this test, we agree it will be critical to 
point out both quantitative and qualitative limitations that may affect 
Congressional deliberations on this matter.

Develop and pretest clear residence rules and appropriate guidelines on 
who should be included in the count to prevent confusing prospective 
respondents. We did not include residence rule guidance on the 
questionnaire for the 2004 test on purpose, as it was an initial 
feasibility test and we did not wish to restrict response, but rather 
to encourage the widest possible participation. We concur that 
developing and pretesting clear residence rules and guidelines is an 
important goal, and we intend to explore these issues in the 2006 
Overseas Enumeration Test.

Ensure that its outreach and promotion strategy, data collection 
methods, and other aspects of the design are free from cultural and 
other biases that could introduce systematic errors. We note that, as 
stated, these goals would be virtually impossible to achieve. We do 
agree that these are important goals however, and as a statistical 
agency, we strive to prevent such biases in all our census and survey 
efforts. The initial 2004 test results suggest, however, that it will 
be more difficult to reach and include some segments of the overseas 
population, that this differential may vary by country, and that these 
factors may lead to differential coverage rates in the results.

Explore the possibility of developing more comprehensive measures of 
data quality. We agree we need to continue to explore ways to measure 
data quality, but we note that without a known (or even knowable) 
denominator, all measures of data quality will remain of limited 
usefulness.

Test the Bureau's ability to geocode certain non-city-style addresses 
such as those with post office box numbers. We agree this will be an 
important objective insofar as the Congress indicates these data will 
be used for purposes that require data to be tabulated below the 
national or state level.

Research how best to market the overseas census to hard-to-count groups 
and other, less visible, segments of a country's overseas American 
population. We note that test results to date indicate that the entire 
overseas population is going to be extremely hard to count. We agree 
that the goal of future marketing strategies should be to continue 
efforts to find ways to increase participation of all U.S. citizens 
living overseas.

Develop procedures to determine whether a return is within the scope of 
the enumeration - that is, that they are truly from an overseas source. 
We agree that this is a critical goal, but note that our testing may 
well show that ensuring such validity will be all but impossible for 
any reasonable level of effort and funding.

Actually integrate stateside data with overseas data to more thoroughly 
test this activity. Agreed. The Census Bureau plans to study this as 
part of the 2006 Overseas Enumeration Test.

Develop a cost model to provide the Bureau and Congress with better 
estimates of the costs of conducting an overseas census under different 
methodological and other scenarios. We agree we will need to provide 
these types of data to the Congress as we gain such knowledge from our 
testing efforts.

Thoroughly pretest the overseas questionnaire with various groups of 
potential respondents to ensure the questions are clear, appropriate, 
and can be accurately answered by the unique population groups that are 
found overseas. We agree with this recommendation. We note, however, 
that pretesting a single questionnaire for use in every Nation of the 
globe presents unique challenges.

Add more response options to the questionnaire item concerning 
respondents' primary activity. Specific information on whether an 
individual is retired, a student, or working for a private company, 
etc., could provide the Bureau with valuable data on the 
characteristics of overseas Americans that could be important for some 
of the purposes for which the data might be used, and just as 
important, could provide the Bureau with invaluable marketing data that 
the Bureau could use to develop a more effective outreach and promotion 
campaign. While we generally agree with this recommendation, we note 
that if the Congress desires an overseas count, as part of or separate 
from the:

2010 census, it will be important for them to indicate the purposes for 
which these data will be used. This information will be critical in 
designing any further tests.

Work with Congress and other stakeholders to develop a broad consensus 
on what would be acceptable levels of accuracy, completeness, 
participation, and other key measures of performance. We agree that 
developing such a consensus is critical to all future efforts.

Comments on Specific Statements in the Draft Report:

The Census Bureau believes that several factual points in this draft 
report should be clarified or corrected.

Page 11. "The Bureau estimates the 2004 test will cost around $2.5 
million." A more correct statement is that "last year the Census Bureau 
estimated the implementation costs for the 2004 test at approximately 
$2.5 million in FY 2004. We also estimate that by the end of FY 2004, 
we will have spent an additional $3.5 million for planning and 
preparation during FY 2003 and FY 2004."

Page 11. "The Bureau plans to conduct an additional test in 2006 and a 
dress rehearsal in 2008." The correct statement is that "the Census 
Bureau plans to conduct a 
2006 overseas test if the Congress appropriates requested funds in FY 
2005, 2006, and 2007. If Congress then indicates its desire that the 
Census Bureau conduct a general overseas enumeration in 2010, we will 
seek a supplementary appropriation in calendar year 2007 for that 
purpose and to conduct a 2008 overseas dress rehearsal beginning in 
2007."

Page 22. "Research Question #3: The Bureau Will be Unable to Geocode 
Certain U.S. Addresses." We agree that some of the U.S. addresses we 
receive from U.S. citizens living overseas will not be geocodable. For 
example, some addresses will be incomplete, including those that 
provide only a town name or post office name (which would include P. O. 
Box addresses). This means the actual location of the living quarters 
cannot be geocoded to the block level required to include it in 
redistricting tabulations. It is also likely that some reported U.S. 
addresses will be old addresses that no longer exist. The impact of 
these problems will be more critical if the Congress indicates these 
data will be used for purposes that require data to be tabulated below 
the national or state level.

Page 24. "Advertisements are to run in newspapers, magazines, local 
television and radio and the Bureau has produced pamphlets to promote 
the test as well." The correct statement is that the Census Bureau is 
attempting to have articles about the census test placed in newspapers, 
magazines, and stories run on local television and radio. Paid 
advertising was not part of the original plan for the test.

Page 30. "Another potential problem is that the Bureau has so far been 
unable to negotiate access to the State Department's passport database, 
and must instead rely solely on Social Security numbers to validate 
returns." A clearer statement would be that "Another potential problem 
is the sole reliance on Social Security Number to validate returns for 
the 2004 test. The Census Bureau is in ongoing negotiations with the 
State Department for access to the passport database, but such access 
is not expected in time for use in the 2004 test." 

[End of section]

Appendix II: Criteria for Evaluating Census Research Design for 
Overseas Enumeration Test: 

1. How clearly are the objectives of the research design presented?

* Are the test/research objectives and/or questions specified clearly 
in the design?

* Are concepts defined where necessary?

2. How sound is the research design?

* Do the research questions match the research objectives?

* Is the rationale for the determination of the size and type of sample 
explained?

* Are potential biases recognized and addressed (e.g., cultural bias, 
question item bias, or sample bias)?

* Does the sample selection reflect the full range and full cycle of 
entities or processes that should be considered?

3. How appropriate is the data collection strategy?

* Is the mode of data collection stated clearly?

* Is the timing and frequency of data collection considered?

* Is the data collection method appropriate for reaching the intended 
sample population?

* Is the data collection instrument appropriate for the sample 
population?

* Is a plan for administering and monitoring the data collection 
discussed in the design?

* How well does the design address factors that may interfere with data 
collection?

* How well are methods for assuring adequate response rates addressed?

4. How thoroughly did Census test the survey instrument(s)?

* Were any new survey items cognitively tested?

* Were field tests conducted to identify the best question wording and 
determine whether changes in questions are likely to achieve the change 
objectives?

* Were research studies used to address possible changes in the 
questionnaire?

5. How involved were relevant stakeholders in the research planning?

* Were relevant stakeholders for the research objectives identified?

* Were stakeholders involved in the planning or review of the methods 
of data collection?

* Were appropriate stakeholders participants in the review and testing 
of the questionnaire?

6. How sound is the design's plan for reaching the target sample?

* Are the goals of the outreach strategy feasible?

* Are the methods of the outreach strategy viable?

7. How sound is the plan for implementing test site activities?

* Does the design consider possible mistakes and their consequences 
(including their seriousness)?

* Does the design assure that sufficient evidence will be gathered to 
answer the research questions?

* Does the design consider the level of difficulty in obtaining the 
data?

8. How good is the relationship between the research design and the 
time and resources allocated?

* Does the execution of the design appear feasible within the stated 
time frame?

* Do the proposed resources for the execution of the design appear 
feasible?

* Are the roles and responsibilities of key parties specified?

9. How well does the design develop a data analysis plan?

* Is the method of enumeration clearly presented?

* Are the proposed analytical techniques for analysis presented?

* Does the design provide a basis for comparing the results of the 
research?

* Does the design discuss the possible limitations of the findings/test 
results?

[End of section]

Appendix III: 2004 Overseas Test Census Questionnaire: 

[See PDF for image] 

Source: U.S. Census Bureau.

Note: This appendix does not include the entire census questionnaire. 
The same questions are asked of person 1 through person 6, as such, 
only questions for person 1 have been included.

[End of figure] 

(450236): 

FOOTNOTES

[1] U.S. Bureau of the Census, Americans Overseas in U.S. Censuses, 
Technical Paper 62 (November 1993) hereinafter Americans Overseas. 

[2] According to the Bureau, the apportionment counts from the 2000 
Census included only federally affiliated groups (226,363 military 
personnel, 30,576 civilian employees, and 319,428 dependents of both 
groups). The Bureau did not collect data on other Americans living 
abroad for 2000.

[3] Utah v. Evans, 143 F. Supp. 2d 1290 (D. Utah 2001), aff'd, by Utah 
v. Evans, 534 U.S. 1038 (2001).

[4] Congressional Research Service, House Apportionment: Could Census 
Corrections Shift a House Seat?, RS21638 (Washington, D.C.: Oct. 8, 
2003).

[5] U.S. General Accounting Office, 2010 Census: Cost and Design Issues 
Need to Be Addressed Soon, GAO-04-37 (Washington, D.C.: Jan. 15, 2004).

[6] 13 U.S.C. § 141(a).

[7] Id. §§ 2, 21.

[8] See 41 U.S. Op. Att'y. Gen. 31, 32 (1949), which gives citations to 
early statutes.

[9] District of Columbia v. United States Department of Commerce, 789 
F. Supp. 1179, 1180 (D.D.C. 1992).

[10] Thomas R. Lee & Lara J. Wolfson, The Census and the Overseas 
Population, 2 Election L. J. 343, 344 (2003) (hereinafter referred to 
as Overseas Population), citing Americans Overseas at 10.

[11] Id.

[12] U.S. Census Bureau, Issues of Counting Americans Overseas in 
Future Censuses (Washington, D.C.: Sept. 27, 2001) hereinafter Issues 
of Counting Americans Overseas.

[13] Issues of Counting Americans Overseas, U.S. Census Bureau.

[14] 42 U.S.C. §§ 1973ff-1 to 1973ff-6.

[15] Bush v. Hillsborough County Canvassing Bd., 123 F. Supp. 2d 1305, 
1310-12 (N.D. Fla. 2000).

[16] H.R. Rep. No. 99-765 at 5 (1986), cited in Bush at 1310-11.

[17] Revenue Act of 1913, ch. 16, § IIA, 38 Stat. 114, 166.

[18] Cook v. Tait, 265 U.S. 47, 56 (1924).

[19] 26 U.S.C. § 911.

[20] Tobias M. Lederberg, The Qualified Individual Under I.R.C. 
Section 911 (d) After Jones v. Commissioner: Rethinking Characteristics 
of Eligibility for the Foreign Earned Income Exclusion, 10 B.U. 
International Law Journal 143, 145 (1992).

[21] See 42 U.S.C. § 402; see also Sarah H. Bohr, Overview of Social 
Security Insurance Benefits and Supplemental Security Income, 40 Soc. 
Sec. Rep. Ser. 685, 687 (1993).

[22] U.S. citizens living in certain specified countries, for example 
Cuba and North Korea, cannot receive social security payments while 
they reside there. Once they leave those countries, they can receive 
all of their withheld payments.

[23] James R. Whitman, Venturing Out Beyond the Great Wall of 
Medicare: A Proposal to Provide Medicare Coverage Outside the United 
States, 8 Elder L.J. 181, 208 (2000) hereinafter (Venturing Out).

[24] See Your Payments While You Are Outside the United States, SSA 
Publication #05-10137, ICN 480085, July 2002.

[25] 42 U.S.C. §§ 1381 - 1383c.

[26] The Social Security Administration considers individuals as being 
abroad if they have been away from the United States for at least 30 
consecutive days, and until they return and stay in the United States 
for at least 30 consecutive days.

[27] 42 U.S.C. § 1382(f); see Califano v. Aznavorian, 439 U.S. 170 
(1978).

[28] Medicare became effective in July 1966. See Social Security 
Amendments of 1965, Pub. L. No. 89-97, 79 Stat. 286 (codified in 
scattered sections of 42 U.S.C.). At that time, it already included the 
foreign exclusion. Id. at §1862(a)(4) (codified at 42 U.S.C. § 
1395y(a)(4)).

[29] Medicare's foreign exclusion generally denies payment for all 
medical services received outside the United States, subject only to 
the Canadian travel exception and the exceptions for emergency 
inpatient care where a foreign hospital is closer to the site of an 
emergency occurring within the United States and nonemergency inpatient 
care where the foreign hospital is closer to the patient's U.S. 
residence than a domestic hospital. 42 U.S.C. § 1395y(a)(4).

[30] Milkson v. Secretary of Health and Human Services, 633 F. Supp. 
836 (E.D.N.Y. 1986).

[31] Both Social Security and Medicare are premised upon the idea that 
"you get what you put in" because funds for each program are paid out 
of employee and employer payroll withholding taxes. Both utilize the 
same criteria to determine eligibility. Yet, Social Security payments 
follow beneficiaries overseas, no matter how long they remain outside 
the Untied States, while Medicare benefits stop at the border. And 
while SSI is entirely funded by the federal government without the help 
of payroll withholding taxes, the government allows beneficiaries to 
receive benefits if they remain outside the United States for up to 30 
days. See Venturing Out at 207-208.

[32] Franklin v. Massachusetts, 505 U.S. 788 (1992).

[33] Utah v. Evans, 143 F. Supp. 2d 1290 (D. Utah 2001).

GAO's Mission: 

The General Accounting Office, the investigative arm of Congress, 
exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics.

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading.

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. General Accounting Office

441 G Street NW,

Room LM Washington,

D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.

General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.

20548: