This is the accessible text file for GAO report number GAO-07-24 
entitled 'Darfur Crisis: Death Estimates Demonstrate Severity of 
Crisis, but Their Accuracy and Credibility Could Be Enhanced' which was 
released on December 11, 2006. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Requesters: 

United States Government Accountability Office: 

GAO: 

November 2006: 

Darfur Crisis: 

Death Estimates Demonstrate Severity of Crisis, but Their Accuracy and 
Credibility Could Be Enhanced: 

GAO-07-24: 

GAO Highlights: 

Highlights of GAO-07-24, a report to congressional requesters 

Why GAO Did This Study: 

In 2003, violent conflict in Darfur, Sudan, broke out between rebel 
groups and government troops and government-supported Arab militias. 
While few would dispute that many thousands of Darfur civilians have 
died, less consensus exists about the total number of deaths 
attributable to the crisis. Estimates by the Department of State 
(State) and other parties report death tolls up to about 400,000 for 
varying populations and periods of time between February 2003 and 
August 2005. Based on the views of experts convened by GAO and the 
National Academy of Sciences, interviews with estimate authors, and a 
review of relevant literature, this report (1) evaluates six Darfur 
death estimates, (2) identifies general challenges to estimating deaths 
in such crises, and (3) discusses measures to improve death estimates. 

What GAO Found: 

The experts GAO consulted did not consistently rate any Darfur death 
estimate as having a high level of accuracy; moreover, they noted that 
all the studies had methodological strengths and shortcomings. Most of 
the experts had the highest overall confidence in estimates by the 
Centre for Research on the Epidemiology of Disasters (CRED) and had a 
slightly lower level of confidence in State’s estimate. Many experts 
believed State’s lower-end estimate was too low. Additionally, the 
published documents describing State’s estimate lacked sufficient 
information about its data and methods to allow it to be replicated and 
verified by external parties. 

Estimating deaths in a humanitarian crisis such as that in Darfur 
involves numerous challenges. For example, in Darfur, difficulties in 
collecting mortality data, such as lack of access to particular 
geographical regions, impacted the data’s quality and led to data gaps. 
Because of such data gaps, some Darfur death estimates relied on 
potentially risky assumptions and limited contextual information. 
Further, limitations in estimates of Darfur’s population before and 
during the crisis may have led to over- or underestimates of the death 
toll. Finally, varying use of baseline mortality rates—the rate of 
deaths that would have occurred without the crisis—may have led to 
overly high or low death estimates. 

The experts proposed and rated a wide range of measures that U.S. 
agencies could take to improve the quality and reliability of death 
estimates for Darfur and future humanitarian crises. Among these 
measures, the most highly rated was ensuring that public documentation 
of the data and methods used contain sufficient information to enable 
external replication and verification of the estimates. Other very 
highly rated measures include collecting and maintaining data for 
specific periods of time and geographic areas and housing the 
responsibility for making estimates in a reputable independent body. 

Figure: Darfur Village Burning: 

[See PDF for Image] 

Source: United States Holocaust Museum, Photograph by Brian Steidle. 

[End of Figure] 

What GAO Recommends: 

To safeguard the U.S. government’s credibility as a source of reliable 
death estimates, GAO recommends ensuring greater transparency regarding 
the data and methods used for such estimates. GAO also recommends that 
the U.S. government consider other measures suggested by the experts to 
help address gaps in data and improve the quality of any future death 
estimates. State and the U.S. Agency for International Development 
agreed with GAO’s recommendations. 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-24]. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact David Gootnick at (202) 
512-3190 or gootnickd@gao.gov. 

[End of Section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Some Death Estimates Judged More Accurate and Methodologically 
Stronger, Despite Shortcomings: 

Estimating Deaths in Humanitarian Crises Involves Many Challenges: 

Wide Range of Measures Could Improve Death Estimates for Darfur and 
Future Humanitarian Crises: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: List of Experts and Meeting Agenda: 

Appendix III: List of Death Estimates and Mortality Surveys Provided to 
Experts and Additional Bibliographical References: 

Appendix IV: Summary Description of Death Estimates Reviewed: 

Appendix V: Additional Follow-Up Survey Results: 

Appendix VI: Comments from the Department of State: 

GAO Comments: 

Appendix VII: Comments from the U.S. Agency for International 
Development: 

Appendix VIII: Summary of Authors' Comments: 

Appendix IX: GAO Contacts and Staff Acknowledgments: 

Tables: 

Table 1: Accuracy of Darfur Death Estimates Rated by Experts: 

Table 2: Measures Rated by Experts as Likely to Greatly Improve Death 
Estimates for Future Crises, in Order of Ranking and Number of 
Endorsements: 

Figures: 

Figure 1: Map of Sudan: 

Figure 2: Map of Destroyed and Damaged Darfur Villages, as of February 
2005: 

Figure 3: Example of Generation of a Death Estimate: 

Figure 4: Death Estimates Based on Reported Figures and Time Included: 

Abbreviations: 

CDC: Centers for Disease Control and Prevention: 
CE-DAT: Complex Emergency Database: 
CIJ: Coalition for International Justice: 
CMR: crude mortality rate: 
CRED: Centre for Research on the Epidemiology of Disasters: 
IDP: internally displaced persons: 
JEM: Justice and Equality Movement: 
NGO: nongovernmental organization: 
SLM/A: Sudan Liberation Movement/Army: 
SMART: Standardized Monitoring and Assessment of Relief and 
Transitions: 
USAID: U.S. Agency for International Development: 
UN: United Nations: 
WHO: World Health Organization: 

United States Government Accountability Office: 
Washington, DC 20548: 

November 9, 2006: 

The Honorable Tom Lantos: 
Ranking Minority Member: 
Committee on International Relations: 
House of Representatives: 

The Honorable Mike DeWine: 
The Honorable Richard J. Durbin: 
United States Senate: 

In 2003, violent conflict broke out in the Darfur region of western 
Sudan when rebel groups, believing that the region was marginalized by 
the Sudanese government, led attacks against the government. In 
response, the government armed and supported local Arab and tribal 
militias, commonly known as the Janjaweed, to defeat the rebels. 
Attacks on the civilian population by these militias, sometimes in 
conjunction with the Sudanese army, have resulted in widespread death 
and disruption. The crisis has affected an estimated 3.76 million 
people in Darfur, including approximately 1.85 million--"internally 
displaced persons" (IDPs)--who now live in camps. To aid these 
populations, the United States has provided more than $1 billion in 
assistance for Darfur since fiscal year 2004, largely through the U.S. 
Agency for International Development (USAID).[Footnote 1] 

While few would dispute that many civilians have died in Darfur owing 
to violence, disease, and malnutrition, less consensus exists about the 
total number of deaths during, or directly resulting from, the crisis. 
The U.S. Department of State (State) reported that a total of 98,000 to 
181,000 people died between March 2003 and January 2005. Five other 
studies--conducted for varying purposes by international institutions, 
academics, and individual researchers--have produced estimates ranging 
up to about 400,000 deaths for various periods of time between February 
2003 and August 2005.[Footnote 2] 

Because policymakers require an accurate estimate of the death toll in 
Darfur to understand the dimensions of the crisis and determine the 
U.S. response, we were asked to examine the estimates, the methods used 
to produce them, and their relative accuracy. This report (1) evaluates 
the relative accuracy and methodological strengths and shortcomings of 
the six death estimates for Darfur;[Footnote 3] (2) identifies general 
challenges to estimating the total death toll in Darfur and similar 
humanitarian crises; and (3) discusses measures that the U.S. 
government could take to improve its death estimates for Darfur and any 
similar, future crisis. 

To evaluate the estimates, we reviewed and analyzed public information 
on the estimates and interviewed the estimate authors regarding their 
studies' data, methods, and objectives. We provided this information 
and summaries of the interviews to a group of 12 experts in 
epidemiology, demography, statistics, and the Darfur crisis convened in 
April 2006 in collaboration with the National Academy of 
Sciences.[Footnote 4] These experts discussed their review of this 
information and evaluation of the estimates during an all-day session 
and also assessed the estimates in a follow-up survey. State's Bureau 
of Intelligence and Research, which conducted the department's death 
estimate for Darfur, declined to speak with us or provide additional 
information, limiting the experts' ability to fully understand State's 
methods of analysis. However, despite this limitation, the experts were 
able to discuss State's estimate in detail and assess its accuracy and 
methodologies. To identify challenges of estimating total deaths in 
Darfur, we asked the group of experts to highlight key problems, and we 
reviewed literature related to death estimates and mortality data in 
crises such as the one in Darfur. To identify ways in which the U.S. 
government could improve death estimates for that and any future 
humanitarian crisis, we asked the experts to provide suggestions during 
their discussion, and we solicited further opinions on these 
suggestions in the follow-up survey. Additionally, to learn about 
current practices related to the experts' suggestions for improving 
death estimates, we spoke with officials from USAID, State, and the 
Centers for Disease Control and Prevention (CDC) in the U.S. Department 
of Health and Human Services. Appendix I provides additional details of 
our objectives, scope and methodology. We conducted our work from 
September 2005 to November 2006 in accordance with generally accepted 
government auditing standards. 

Results in Brief: 

The experts we consulted did not consistently rate any of the death 
estimates as having a high level of accuracy and noted that all of the 
studies had methodological strengths and shortcomings. Most of the 
experts had the highest overall confidence in the estimates by the 
Centre for Research on the Epidemiology of Disasters (CRED), which 
relied primarily on a statistical analysis of about 30 mortality 
surveys, and they rated the CRED estimates' accuracy and methodological 
strengths highest among the six.[Footnote 5] The experts had a slightly 
lower level of confidence in the State estimate and gave it slightly 
lower ratings for accuracy and methodological strengths. Further, many 
experts believed that the lower end of State's estimate was too low and 
found that published documents describing State's estimate lacked 
sufficient information about its data and methods to allow it to be 
replicated and verified by external researchers. Most experts rated an 
estimate by the World Health Organization (WHO), which primarily 
extrapolated findings from its 2004 mortality survey, somewhat lower 
than the estimates by CRED and State. Finally, most of the experts 
expressed the least confidence in three estimates that reported the 
highest number of deaths. They cited several shortcomings in these 
estimates, such as a reliance on unrealistic assumptions regarding 
fixed levels of mortality for all populations and time periods included 
in the estimate.[Footnote 6] 

Estimating deaths in a humanitarian crisis such as that in Darfur 
involves numerous challenges. In Darfur, difficulties related to the 
collection of survey data, including a lack of access to particular 
geographical regions, challenging survey conditions, and limited 
resources and training for research staff, impacted the data's quality 
and resulted in data gaps. Because of the lack of complete mortality 
data, some of the estimates rely on potentially risky assumptions and 
limited contextual information. For example, some estimates assume that 
mortality rates in accessible areas can be applied to inaccessible 
areas, without accounting for differences between the two due to 
factors such as patterns of violence and humanitarian relief efforts. 
In addition, limitations in estimates of Darfur's population before and 
during the crisis may have led to over-or underestimations of the death 
toll. Finally, varying use of baseline mortality rates--the rate of 
deaths that would have occurred regardless of the crisis--may also have 
led to overly high or low death estimates. 

The group of experts proposed and rated a wide range of measures that 
U.S. agencies could take, directly or by supporting other 
organizations, to improve the quality and reliability of death 
estimates and relevant data for Darfur and any future such humanitarian 
crises. The most highly rated measure was ensuring that publicly 
available documentation of U.S. government estimates contains 
sufficient information on data and methods, so that external 
researchers can replicate the estimates and verify their credibility 
and objectivity. Other very highly rated measures that the government 
could consider included collecting and maintaining temporal and spatial 
data (i.e., data covering specific periods of time and geographic 
areas); housing the responsibility for making estimates in a reputable 
independent body; improving the training of nongovernmental 
organization (NGO) staff who collect mortality survey data; and 
promoting an interdisciplinary approach to estimating mortality. 
Through various initiatives already under way, U.S. agencies are 
carrying out efforts that may address some of the experts' suggestions, 
for example, providing guidance on the design and implementation of 
survey instruments and supporting the collection and maintenance of 
temporal and spatial data. 

To safeguard State's credibility as a source of accurate and reliable 
death estimates, we recommend that the Secretary of State provide, for 
this and any future estimates of mortality that State conducts, 
sufficiently detailed descriptions of its data and methodology to 
enable other parties to assess and replicate its findings.[Footnote 7] 
Additionally, to enhance the U.S. government's capacity to assess and 
respond to any future humanitarian crisis, we recommend that the 
Secretary of State and the Director of U.S. Foreign Assistance and 
USAID Administrator consider the experts' other suggestions to help 
address gaps in data and improve any future death estimates. 

We provided a draft of this report to the Department of State and 
USAID. State and USAID responded with formal comments, agreeing with 
our recommendations, and State provided additional perspectives on 
reporting and documentation regarding its death estimate. Reproductions 
of these letters, as well as our responses to the letters, can be found 
in appendixes VI and VII. We also provided a draft to the CDC for 
technical review, and we received technical comments from both the CDC 
and State, which we incorporated in the report as appropriate. Finally, 
we provided the authors of the other five estimates the portions of the 
report pertaining to their individual estimates. They provided 
comments, which we incorporated in the report as appropriate. These 
authors' comments and our responses are summarized in appendix VIII. 

Background: 

Sudan is the largest country in Africa, with a population estimated at 
about 40 million (see fig. 1). Darfur is in the western region of Sudan 
and comprises three states, with an estimated preconflict population of 
around 6 million.[Footnote 8] 

Figure 1: Map of Sudan: 

[See PDF for image] 

Source: GAO based on Map Resources (map). 

[End of figure] 

Conflict in Darfur: 

In early 2003, Darfur rebels attacked Sudanese police stations and the 
airport in El Fasher, the capital of North Darfur. In El Fasher, the 
rebels destroyed numerous military aircraft, killed many Sudanese 
soldiers, and kidnapped a Sudanese general. In response, the government 
armed and supported the local tribal and Arab militias (the Janjaweed). 
Fighting between the principal rebel groups--the Sudan Liberation 
Movement/Army (SLM/A) and the Justice and Equality Movement (JEM)--and 
the Sudan military and Janjaweed intensified during late 2003. In 
addition to displacing approximately 1.85 million Darfurians in the 
region, attacks on civilians by the Sudan military and Janjaweed have 
forced an estimated 220,000 Darfur refugees to take shelter in 
Chad[Footnote 9] and so affected approximately 1.91 million additional 
Darfur residents that they require humanitarian assistance. (Fig. 2 
shows the locations of Darfur villages destroyed or damaged in the 
conflict.) 

Figure 2: Map of Destroyed and Damaged Darfur Villages, as of February 
2005: 

[See PDF for image] 

Source: GAO based on a map from State's Humanitarian Information Unit 
and map Resources (map). 

[End of figure] 

Peace negotiations under the mediation of the Chad government in 2003 
led to a September 2003 cease-fire agreement between the SLM/A and the 
Sudan government; however, the agreement collapsed in December of that 
year. A second cease-fire agreement was signed by the Sudanese 
government, the SLM/A, and the JEM in April 2004.[Footnote 10] At this 
point, the African Union was brought in to monitor compliance with the 
cease-fire agreement between the three parties, and peace negotiations 
continued under African Union auspices with Chadian 
participation.[Footnote 11] After a relatively calm 2005, cease-fire 
violations and violent incidents reportedly began to increase in the 
final months of the year and into 2006. 

On May 5, 2006, the government of Sudan and one faction of the SLM/A 
signed the Darfur Peace Agreement, which establishes agreements in key 
areas such as power sharing, wealth sharing, and security 
arrangements.[Footnote 12] The U.S. government and other parties 
support a transition of peacekeeping responsibilities from the African 
Union to the UN. In August 2006, the UN Security Council adopted a 
resolution expanding the mandate of the UN Mission in the Sudan and 
calling for the mission's deployment to Darfur.[Footnote 13] As of 
October 2006, the Sudanese president had rejected a UN transition but 
expressed support for a September offer by the UN Secretary-General to 
assist the African Union Mission in Sudan by providing equipment and 
dedicated personnel. Meanwhile, the environment in Darfur remained 
insecure, with attacks and displacement continuing and, during some 
periods, worsening over time. 

U.S. Response to Darfur Crisis: 

The U.S. government has been active in addressing the conflict. On July 
22, 2004, the U.S. House and the Senate each passed separate 
resolutions citing events in Darfur as acts of genocide.[Footnote 14] 
Further, on September 9, 2004, in testimony before the Senate Foreign 
Relations Committee, the U.S. Secretary of State announced that 
genocide had been committed in Darfur and that the Sudanese government 
had supported the Janjaweed, directly and indirectly, as they carried 
out a "scorched earth" policy toward the rebels and the African 
civilian population in Darfur.[Footnote 15] 

According to State officials, the administration's declaration of 
genocide was influenced by findings from interviews with 1,136 Darfur 
refugees in eastern Chad in July and August 2004, which demonstrated a 
pattern of abuse against members of Darfur's non-Arab communities. The 
interviews were conducted by an "Atrocities Documentation Team" 
assembled by State and consisting of officials from State and USAID, as 
well as members of the Coalition for International Justice[Footnote 16] 
and the American Bar Association. State reported that 61 percent of 
those interviewed said that they had directly witnessed the killing of 
a family member in addition to other crimes.[Footnote 17] 

For fiscal years 2004 through 2006, the United States obligated $996 
million in humanitarian assistance for Darfur.[Footnote 18] Although 
more than 68 percent of this assistance has consisted of food aid, U.S. 
assistance to Darfur has also supported other vital needs, such as 
water and sanitation, shelter, and primary health care services. Partly 
as a result of U.S. assistance, NGOs and UN agencies have made 
significant progress in increasing the number of internally displaced 
persons and affected residents receiving aid. The number of 
international and national humanitarian aid workers in Darfur expanded 
from 202 in April 2004 to 13,500 staff members of 84 NGOs and 13 UN 
agencies in July 2006. In 2005 and early 2006, malnutrition and 
mortality rates in Darfur had dropped since 2004, a trend that U.S. and 
other officials attribute in part to international humanitarian 
assistance efforts. Further, according to State sources, the U.S. 
government, via private contractors, provided about $280 million, 
between June 2004 through September 2006, primarily to build and 
maintain facilities throughout Darfur to house African Union troops 
assigned to monitor compliance with the April 2004 cease-fire 
agreement.[Footnote 19] 

Data Sources and Methods for Darfur Death Estimates: 

A key data source for the Darfur death estimates is health, nutrition, 
and mortality surveys conducted in the field by NGOs delivering 
humanitarian services (e.g., Médecins Sans Frontières, known in English 
as Doctors without Borders, and Save the Children)[Footnote 20] as well 
as by UN or governmental agencies (e.g., the WHO, the World Food 
Program, and the CDC). These surveys are discrete data collection 
exercises carried out at a specific time with a particular sample of 
the affected population, such as people in a certain IDP camp. 

Surveys that ask about mortality are often combined with those 
collecting health and nutrition data. "Retrospective" mortality data 
are collected by asking a sample of respondents to recall the number of 
deaths that occurred in their household during an earlier defined 
period of time.[Footnote 21] Interviewers may also ask respondents 
questions that allow them to categorize the cause of death (e.g., 
deaths due to violence, disease, or malnutrition). Households surveyed 
may be located in a single area (e.g., displacement camp) or multiple 
areas (e.g., multiple camps or sites within a region.) Because an 
absolute number of deaths is difficult to interpret, organizations 
conducting surveys calculate mortality rates for the population sampled 
and for the time included in the "recall period." These mortality rates 
allow for comparison among different population groups and with 
different time periods, such as when no humanitarian crisis is 
occurring. In acute emergencies, when mortality may change 
significantly during a short time interval, mortality rates are often 
expressed as the number of deaths per 10,000 people per day. Typically, 
a crude mortality rate--that is, the rate of death for the entire 
population, including both sexes and all ages[Footnote 22]--is 
reported, as well as mortality rates for specific groups (such as those 
younger than 5 years or of a specific sex). Data from CRED's Complex 
Emergency Database (CE-DAT) show that at least 68 surveys conducted 
with IDPs and affected residents reporting crude mortality rates were 
conducted in Darfur between 2004 and 2006.[Footnote 23] Two 
retrospective mortality studies used in most Darfur death estimates 
were conducted, respectively, by Médecins Sans Frontières and the WHO. 

* Médecins Sans Frontières Mortality Surveys in West Darfur. The 
organization conducted retrospective mortality surveys with 3,175 
households in four sites in West Darfur.[Footnote 24] The surveys, 
conducted between April and June 2004, asked respondents about the 
number and cause of deaths in their household up to 6 months 
earlier,[Footnote 25] before they fled their villages, as well as after 
they arrived at their IDP camps or settlements. Médecins Sans 
Frontières reported crude mortality rates ranging from 5.9 to 9.5 
deaths per 10,000 per day during the period when respondents were in 
their villages and in flight and rates ranging from 1.2 to 5.6 deaths 
per 10,000 per day during the camp period. Médecins Sans Frontières 
also reported that violence caused 68 to 93 percent of deaths during 
the village and flight periods. 

* 2004 World Health Organization Retrospective Mortality Survey. The 
WHO surveyed 3,140 households in accessible IDP settlements in the 
three regions of Darfur to determine mortality rates between June and 
August of 2004.[Footnote 26] The organization reported crude mortality 
rates of 1.5 per 10,000 per day for North Darfur; 2.9 per 10,000 per 
day in West Darfur; and 3.8 per 10,000 per day for Kalma camp in South 
Darfur for the defined recall period. Diarrhea accounted for most of 
the reported deaths in all three regions, while violence or injury 
accounted for a smaller percentage of deaths, ranging from 10 to 21 
percent. 

Researchers estimating death tolls in humanitarian crises such as that 
in Darfur generally extrapolate mortality surveys' various mortality 
rates to longer time periods and larger populations. In addition, to 
estimate an "excess" number of deaths directly attributable to the 
conflict, some researchers subtract a baseline mortality rate--that is, 
an expected number of deaths that would have occurred absent the 
conflict--from the total number of deaths estimated for the time period 
and population. (Fig. 3 illustrates the generation of a death 
estimate.) 

Figure 3: Example of Generation of a Death Estimate: 

[See PDF for image] 

Source: GAO. 

[End of figure] 

The death estimates that we and the group of experts reviewed were 
produced for varying purposes, according to their authors. 

* The authors of the CRED estimates[Footnote 27] said that their aim in 
conducting their estimate was to develop a method that, rather than 
extrapolating mortality from a single survey to the entire region and 
conflict period, took into account variations over time and space. 

* Dr. Coebergh[Footnote 28] noted that his estimate was intended as a 
political statement to increase public awareness of the crisis. He also 
stated that the estimate was produced as a response to earlier total 
death tolls quoted in the media, which he believed were too low and did 
not adequately capture violent deaths. 

* According to Dr. Hagan,[Footnote 29] his estimate in the spring of 
2005 responded to earlier total death figures, quoted in the media, 
that he believed were too low and did not adequately capture violent 
deaths. He stated that one of his goals was to provide greater 
transparency about the available estimates. 

* Dr. Reeves[Footnote 30] said that he produced his first estimate of 
the Darfur death toll, in June 2004, because he believed that the 
figures being quoted by the UN significantly understated the Darfur 
death toll and were not supported by the data. 

* According to State, its purpose was to provide information for 
internal policymakers. 

* A WHO[Footnote 31] official told us that the organization sought to 
assess the order of magnitude of deaths in Darfur to assist it in 
planning humanitarian relief for IDPs in camps. 

The estimates that we and the experts reviewed vary quantitatively and 
in the time that they cover--from 35,000 excess deaths or 45,000 total 
deaths for IDPs in camps over 7 months of the conflict, to almost 
400,000 total deaths for Darfur over 26 months of the conflict. (See 
fig. 4 for a summary description of the estimates and their findings, 
and see app. IV for more details.) 

Figure 4: Death Estimates Based on Reported Figures and Time Included: 

[See PDF for image] 

Source: GAO analysis of estimates from above organizations. 

[A] CRED conducted two death estimates: one for the period of September 
2003-January 2005 and another for the period of February-June 2005. The 
bar on the left for "estimated deaths" is the combined total of these 
two estimates. 

[B] This estimate was conducted by John Hagan and colleagues and 
released by the CIJ. 

[End of figure] 

We and the group of experts also reviewed a preliminary version of a 
death estimate for West Darfur by John Hagan and Alberto Palloni; 
however, this estimate is not discussed in our report, because the 
version that the experts reviewed was preliminary and not publicly 
available when they convened in April 2006. In the estimate's final 
version, which became publicly available in September 2006, Hagan and 
Palloni estimated that a range of 57,506 to 85,346 deaths had occurred 
in West Darfur over 31 months. Using the same ratio of deaths to 
displacement, they estimated a range of 170,000 to 255,000 deaths in 
all three states of Darfur over the same time period.[Footnote 32] 

In addition, the estimates differ methodologically, incorporating 
varying source data, mortality rates and causes of death, assumptions 
regarding variable or fixed mortality rates, estimates of the affected 
population, consideration of baseline mortality rates, and reporting 
methods. (See app. IV for more information.) 

* Source data. The estimates' respective sources of data range from 
about 30 mortality surveys to a single mortality survey. The estimates 
also use nonsurvey or contextual information, such as the retrospective 
reporting of attacks or displacement patterns, to varying degrees. In 
addition, the amount of source data used varies according to the 
availability of such data when the estimates were conducted. For 
example, fewer data were available for estimates conducted in 
2004[Footnote 33] than for those conducted in 2005. 

* Mortality rates and cause of death. The estimates differ in whether 
they applied mortality rates that include all causes of death or 
calculated and applied mortality rates due to violence and nonviolence 
separately. However, most of the estimates, as well as the mortality 
surveys we reviewed, express mortality rates in deaths per 10,000 
people per day. 

* Assumptions of variable or fixed mortality rates. The estimates 
include different assumptions regarding whether mortality rates varied 
over the period of the conflict or according to the affected region or 
remained constant for all populations in all areas over the entire 
period of conflict. 

* Affected population estimates. The estimates rely primarily on 
information from the UN Humanitarian Profiles in determining the total 
affected population, but they differ in their assumptions about whether 
the affected population changed over time or remained constant. 
Additionally, one estimate also includes only IDPs in camps and 
excludes affected residents in Darfur and refugees in Chad. 

* Baseline mortality rates. The estimates varied in their use of a 
baseline mortality rate--that is, the number of expected deaths absent 
the conflict--from using no baseline to using a baseline of 0.5 deaths 
per 10,000 per day. 

* Reporting methods. Some estimates report a range of possible total 
deaths, others report point estimates --that is, single numbers. 
Additionally, some estimates are precise figures, such as 396,563, 
while others are rounded to fewer significant digits, such as 134,000. 

Some Death Estimates Judged More Accurate and Methodologically 
Stronger, Despite Shortcomings: 

Although none of the death estimates was consistently considered 
accurate or methodologically strong, the experts we consulted rated 
some of the estimates more highly than others. Overall, the experts 
expressed the highest level of confidence in CRED's estimates and 
slightly lower levels of confidence in State's and the WHO's estimates. 
They expressed the lowest level of confidence in the three estimates 
that report the highest number of deaths, citing multiple shortcomings, 
such as a reliance on unrealistic assumptions about populations' level 
of risk over periods of time. 

No Estimate Consistently Considered Accurate: 

Although the experts clearly had greater confidence in some estimates 
than in others, they did not consistently express a high level of 
confidence in any of the estimates or consistently rate any of them as 
accurate.[Footnote 34] 

The experts expressed a slightly higher level of confidence in the CRED 
estimates than the State estimate. However, only 2 of the 12 experts 
expressed a high level of confidence, and most had a moderate level of 
confidence, in the CRED estimates. Experts had a moderate to low level 
of confidence in the WHO estimate. Almost all experts expressed a low 
level of confidence in the estimates by Coebergh, Hagan, and 
Reeves.[Footnote 35] (See app. V for additional survey results.) 

Similarly, the experts did not consistently rate any of the estimates 
as accurate; the majority of experts rated almost all of the reported 
figures as either too high or too low. Only CRED's second estimate-- 
36,000 total deaths for February-June 2005--was viewed by half of the 
experts as "about right." (See table 1.) 

Table 1: Accuracy of Darfur Death Estimates Rated by Experts: 

Estimate (number of months in estimate): Coebergh: 218,449 excess 
deaths (21); 
Too high: 7; 
About right: 1; 
Too low: 2. 

Estimate (number of months in estimate): Coebergh: 253,573 excess 
deaths (21); 
Too high: 8; 
About right: 0; 
Too low: 2. 

Estimate (number of months in estimate): Coebergh: 306,130 excess 
deaths (21); 
Too high: 8; 
About right: 1; 
Too low: 1. 

Estimate (number of months in estimate): CRED's two estimates: Sept. 
2003-Jan. 2005: 134,000 total deaths (17); 
Too high: 1; 
About right: 3; 
Too low: 6. 

Estimate (number of months in estimate): CRED's two estimates: Feb. 
2005-July 2005: 36,237 total deaths (5); 
Too high: 2; 
About right: 5; 
Too low: 2. 

Estimate (number of months in estimate): Hagan et al: 396,563 total 
deaths (26); 
Too high: 10; 
About right: 0; 
Too low: 0. 

Estimate (number of months in estimate): Reeves: Over 370,000 excess 
deaths (31); 
Too high: 10; 
About right: 0; 
Too low: 0. 

Estimate (number of months in estimate): State's lower-and higher-end 
estimates: Lower-end estimate: 98,000 total deaths (23); 
Too high: 0; 
About right: 1; 
Too low: 9. 

Estimate (number of months in estimate): State's lower-and higher-end 
estimates: Higher-end estimate: 181,000 total deaths (23); 
Too high: 4; 
About right: 4; 
Too low: 2. 

Estimate (number of months in estimate): WHO's lower-and higher-end 
estimates: Lower-end estimate: 45,000 total deaths (7); 
Too high: 1; 
About right: 2; 
Too low: 7. 

Estimate (number of months in estimate): WHO's lower-and higher-end 
estimates: Higher-end estimate: 80,000 total deaths (7); 
Too high: 3; 
About right: 1; 
Too low: 6. 

Source: GAO. 

Note: We asked experts to rate the accuracy of total deaths when these 
were reported in the estimate. Coebergh and Reeves reported excess 
deaths only. Additionally, in responding to this question, one expert 
said that there was no basis to judge or that he or she was not sure 
about the accuracy of the estimates; one expert chose not to respond to 
this question; another expert chose two responses for the second CRED 
estimate, and his or her responses are excluded. 

[End of table] 

In some cases, experts tended to agree as to whether the reported 
figures were too high or too low; in other cases, there was less 
agreement. For example, 9 of 10 experts rated the lower-end of State's 
estimate as too low, and the majority of the experts viewed the 
estimates by Coebergh, Hagan, and Reeves as too high. In contrast, 
experts did not agree as to whether State's higher estimate was too 
high, too low, or about right. For example, a few experts believed the 
higher end of State's estimate was likely to be closer to a reasonable 
midrange estimate. These experts also thought that the mortality 
surveys State used for the estimate may have been conducted at places 
with higher levels of aid and subsequently lower levels of mortality, 
or that disease outbreaks may have been missed. However, one expert 
believed that State may have overestimated mortality by applying 
elevated mortality rates for too many months of the crisis. 

Experts Found Methodological Shortcomings in Each Estimate: 

The experts' overall assessment of the estimates' methodological 
strengths in terms of their data, methods, objectivity, and reporting 
did not produce any consistently high ratings, and experts noted 
shortcomings in each estimate.[Footnote 36] 

CRED Estimates: 

Overall, the experts rated CRED's estimates most highly in terms of 
data, methods, objectivity, and reporting of limitations. However, 
several experts found shortcomings in the CRED estimates' data and 
methods and thought that CRED could have provided more information and 
clarity in its reporting. 

* Source data. Most experts said that the data CRED used for its two 
estimates (drawing from a total of about 30 mortality surveys in Darfur 
and Chad[Footnote 37]) were generally sound.[Footnote 38] CRED reported 
checking the reliability and validity of the surveys included in its 
estimates, which experts found to be a strength. However, several 
experts cited some shortcomings in CRED's data sources. For example, 
some said that CRED could have better articulated the criteria used to 
select the survey data or weighted the surveys it used by sample size. 
A few experts also thought that CRED should have considered other 
sources besides mortality surveys, such as surveillance data on 
morbidity or nutrition. 

* Methods used, including extrapolations and assumptions. Some experts 
found CRED's method of using disparate data sources to estimate total 
deaths to be innovative and logical.[Footnote 39] Additionally, more 
than half of the experts rated CRED's assumptions and extrapolations as 
somewhat appropriate or reasonable.[Footnote 40] For example, several 
experts found the Sudan baseline mortality rate that CRED used more 
accurate than the baseline mortality rates derived from a larger region 
of sub-Saharan Africa used in some of the other estimates. However, 
some assumptions and extrapolations were questioned. For example, 
several experts thought that CRED's assumption regarding a generally 
stable rate for nonviolent mortality during much of 2004 may have been 
inaccurate, owing to possible changes in the affected population, camp 
formations, the level of aid, and the outbreak of disease. 
Additionally, some experts considered the method that CRED used to 
estimate deaths among refugees in Chad to be somewhat unsystematic. 

* Level of objectivity. Overall, experts viewed CRED's death estimates 
as having the highest level of objectivity.[Footnote 41] Two authors of 
other estimates also concluded that the CRED estimates were likely to 
be more reliable and more scientific than other Darfur death estimates, 
including their own. 

* Sufficiency of reporting. Experts noted that, among the estimates 
they evaluated, CRED most sufficiently reported the limitations and 
potential sources of over-or underestimation in its estimates. However, 
several experts believed that better descriptions of the methods used, 
including information on specific formulas and calculations, could have 
been provided. An author of another estimate also noted that the 
mortality surveys used in CRED's estimates lacked complete citation 
information. In our review of CRED's first estimate, we were able to 
replicate it to some degree only after the authors provided a 
substantial amount of information, such as specific mortality rates and 
formulas used and citations for source studies, in addition to the 
information in the published document. 

State Estimate: 

In assessing State's estimate, the experts identified methodological 
strengths related to each of the four elements but also noted some 
shortcomings. Strengths included its use of multiple types of 
information, including contextual data from other sources besides 
surveys, such as reporting of attacks. Many experts also believed the 
estimate had a high level of objectivity. However, the experts, as well 
as authors of other estimates, cited several shortcomings in State's 
estimate. For example, many believed that the lower end of the estimate 
was too low, owing to several factors including the use of some data 
that underestimated mortality rates. Additionally, experts and other 
authors thought that the published documents containing the estimate 
lacked sufficient information, such as a clear description of the 
mortality rates used for all populations and time periods included in 
the estimate. 

* Source data. Many experts cited as one strength the estimate's use of 
different types of data, including mortality survey data and contextual 
information, to triangulate findings and estimate mortality, and one 
expert deemed this approach a "pioneering attempt" in the field of 
death estimates in humanitarian crises.[Footnote 42] However, just over 
half of the experts thought that the data used were methodologically 
sound.[Footnote 43] Some experts said that several of the mortality 
surveys used in State's estimate may have had methodological 
limitations in areas such as survey design, implementation, or 
accessibility to insecure regions, resulting in unrealistically low 
mortality rates. These experts believed that such limitations in source 
data, in addition to other problems--for example, the estimate's lack 
of clarity regarding how missing populations are accounted for and use 
of a relatively higher baseline mortality rate--may have pulled down 
State's estimate, in particular, its lower end. 

* Methods used, including extrapolations and assumptions. About half of 
the experts felt that State's estimate applied somewhat appropriate 
extrapolations, and a similar proportion thought that it made 
reasonable assumptions. A few experts thought that its depiction of 
varying levels of mortality over time and affected regions was 
appropriate to estimate total deaths throughout the Darfur crisis. 
However, several experts thought that the assumptions used, in some 
cases, were based on insufficient rationale and evidence and that 
additional sensitivity analysis could have been conducted. For example, 
the State estimate assumes that mortality rates in Darfur are 20 
percent lower for affected residents than for IDPs, an assumption that 
some experts believed lacked sufficient rationale. 

* Level of objectivity. Nine of the 12 experts rated the State 
estimate's level of objectivity as high.[Footnote 44] Several experts 
generally believed that the estimate represented a "good faith effort" 
to use available evidence in an unbiased way. 

* Sufficiency of reporting. Many of the experts found that the 
published documents containing State's estimate lacked sufficient 
information to allow them to replicate the estimate and verify the 
accuracy and reliability of the data and methods. For example, some 
experts noted that the data used were not sufficiently described and 
that more specific citations, as well as a description of the criteria 
applied to select the data, would have been useful. Additionally, some 
experts felt that they did not have a good sense of the reliability and 
validity of the contextual data used in the estimate and the 
application of these data to determine mortality rates or total deaths. 
Moreover, several experts said that State should have included the 
specific formulas or mortality rates used for all populations, time 
periods, and regions. Similar comments were made by several authors of 
other estimates. Our review of the State estimate also showed that it 
could not be replicated with the information contained in the report. 
Further, one expert noted that the published document available on 
State's Web site particularly lacked sufficient description of the 
estimate's methods, data, and potential limitations. 

WHO Estimate: 

Several experts found strengths in the data and level of objectivity of 
the WHO's estimate, which it presented in a short briefing in October 
2004. However, several experts observed shortcomings in the WHO's 
reporting of its estimate. 

* Source data. The WHO estimate of IDP deaths in Darfur for 7 months in 
2004 relied primarily on findings from the organization's 2004 
mortality survey. Several experts noted that this survey followed 
standard methods and was generally reliable. However, a few experts and 
estimate authors said that the 2004 WHO survey may have underestimated 
mortality, owing to local government restrictions that prevented 
researchers from asking respondents detailed questions about mortality. 

* Methods used, including extrapolations and assumptions. Experts 
provided mixed ratings on the appropriateness of the extrapolations and 
the reasonableness of the assumptions used in the WHO estimate. For 
example, one expert believed that because the methodology of the 
organization's 2004 survey was appropriate to gauge levels of mortality 
among a limited IDP population for 2 months, but not aggregate 
mortality, the survey findings should not have been extrapolated to 
generate a death estimate for the total IDP population for 7 months. 

* Level of objectivity. Half of the experts rated the level of 
objectivity of the WHO estimate as high, three rated it as moderate, 
and two rated it as low.[Footnote 45] About half of the experts thought 
that the WHO estimate's level of objectivity was equal to CRED's and 
State's. 

* Sufficiency of reporting. Several experts thought that the WHO's 
reporting of its estimate had several shortcomings. For example, they 
thought that the briefing document describing the death estimate did 
not make it sufficiently clear that the estimate included deaths of 
IDPs from all causes--both violent and nonviolent--but excluded deaths 
of affected residents in Darfur and refugees in Chad. The experts found 
that the lack of a clear description of such issues allowed for 
misinterpretation by readers, including the media and academia. 

Three Other Estimates: 

The experts cited several methodological shortcomings in the Coebergh, 
Hagan, and Reeves estimates, including the use of problematic data and 
application of unrealistic assumptions about the levels of mortality 
over time and affected populations. 

* Source data. Many experts found shortcomings in each of the three 
estimates' use of certain survey data.[Footnote 46] A number of experts 
noted problems in the design, sampling, and data collection in the 
Atrocities Documentation Team's survey of Chad refugees on which all 
three estimates based, at least in part, their numbers of violent 
deaths. Experts also pointed out that, because the survey's intended 
purpose was to document levels and types of victimization, the 
estimates by Dr. Coebergh, Dr. Hagan, and Dr. Reeves should not have 
extrapolated the survey findings to a broader population or time period 
in order to estimate total deaths. In addition, many experts observed 
that the estimates by Coebergh and Hagan inappropriately used findings 
from the 2004 WHO survey to calculate only nonviolent deaths without 
taking into account the fact that some deaths reported by the WHO were 
due to violence or injury.[Footnote 47] 

* Methods used, including extrapolations and assumptions. Most experts 
found that the Coebergh, Hagan, and Reeves estimates used unrealistic 
extrapolations and assumptions to fill information gaps and estimate 
total deaths. For example, many experts thought that each of the three 
estimates relied on too few data points extrapolated to an excessive 
degree. As a result of this type of extrapolation, the experts 
observed, a sensitivity analysis changing one or two assumptions could 
swing the total number of deaths from 100,000 to half a million, making 
the estimates unreliable. Moreover, several experts believed that some 
of the authors had inappropriately assumed constant rates of mortality 
for different population groups in Darfur at different periods in the 
conflict.[Footnote 48] Two of the estimates (Coebergh's and Hagan's) 
also used "fixed" estimates of the affected population over time, a 
method that some experts thought was inappropriate because the affected 
population grew over the course of the conflict. Additionally, the 
three estimates involve assumptions that some experts viewed as 
questionable, such as using unsupported numbers to estimate the number 
of deaths among populations inaccessible to aid or assuming all missing 
persons were likely to be dead. 

* Level of objectivity. Most experts rated the level of objectivity of 
the three estimates as low, particularly those by Drs. Coebergh and 
Reeves. The experts thought that the estimates were more characteristic 
of advocacy or journalistic material than of objective analysis. 

* Sufficiency of reporting. One expert noted that the estimate by Dr. 
Hagan was very straightforward and one could follow the logic of the 
data and methods applied. However, most of the experts found that the 
three estimates did not sufficiently describe limitations that may have 
resulted in under-or overestimation of total deaths. In reviewing the 
estimates, we found we were able to replicate Dr. Hagan's entire 
estimate based on its description in public documents. We were also 
generally able to replicate Reeves' estimate after receiving additional 
information about, among other things, his calculation of some numbers 
and the sources of his data. We had more difficulty in replicating Dr. 
Coebergh's estimate based on the information in the published article 
and were able to do so only after the author provided, at our request, 
details about the specific data, methods, and formulas that were used. 

Estimating Deaths in Humanitarian Crises Involves Many Challenges: 

Estimating the total deaths in a humanitarian crisis such as that in 
Darfur involves a number of challenges, most notably related to 
collecting data in the field and extrapolating from limited data. 
Difficulties in collecting sound, consistent survey data--including 
lack of access to particular geographical regions, the conditions under 
which the surveys are conducted, and limited resources and training for 
field staff conducting surveys--affect the quality of the data 
collected and result in data gaps. Because of such limitations, as well 
as the unavailability of data from other sources, the death estimates 
that we reviewed rely on potentially risky assumptions and limited 
contextual information. Limitations in estimates of Darfur's population 
before and during the crisis also impacted the quality of the death 
estimates. Additionally, varying approaches to the use of baseline 
mortality rates may have somewhat affected their accuracy. 

Challenges in Collecting Source Data Affected Data Quality and Led to 
Data Gaps: 

Difficulties in conducting health, nutrition, and mortality surveys in 
a crisis such as Darfur's can affect survey data's quality and 
completeness and, thus, the soundness of death estimates based on the 
data. In Darfur, these difficulties include lack of access to certain 
geographical regions, difficult survey conditions, researchers' limited 
resources and training, and lack of consensus over sampling methods. In 
addition, the findings from the surveys in Darfur are not always 
publicly available, and few other reliable sources of mortality data 
exist in Darfur. 

* Lack of access to some geographical regions. Researchers' lack of 
access to some geographic areas in Darfur raises concerns about the 
completeness of the data collected. Because of security problems, 
humanitarian aid organizations that typically conduct the surveys, such 
as Médecins Sans Frontières, and other researchers found it difficult 
to access all areas of Darfur, particularly South Darfur, according to 
several experts. In one instance, a survey conducted by the WHO in 2005 
in South Darfur was suspended because of security concerns. Other 
surveys were also curtailed because of security concerns or attacks on 
NGO staff. In general, surveys were conducted primarily in camps where 
humanitarian relief was being provided, which could lead to 
underestimates of actual mortality. For example, the WHO and World Food 
Program note in their respective reports that their 2004 surveys may 
underestimate mortality, owing to their samples' exclusion of people in 
inaccessible areas. Additionally, one expert reported that the Sudanese 
government at times placed restrictions on the relief organizations 
collecting data and limited their access to particular camps and 
regions. 

* Challenging survey conditions. Challenges in conducting household- 
based mortality surveys during and after humanitarian crises such as 
that in Darfur can affect the data's accuracy, consistency, and 
completeness and lead to over-or underreporting of mortality. Such 
challenges include linguistic and other cultural factors, difficulties 
resulting from the surveys' conflict or postconflict environment, and 
issues of recall and precision. 

- Accurate translation of surveys into foreign languages can be 
difficult. 

- Surveys might not be conducted in all local languages.[Footnote 49] 

- Definitions of a household vary.[Footnote 50] 

- Some people are reluctant to talk about death. 

- Some cultures will not report the deaths of infants.[Footnote 51] 

- If all household members die, none remain to be surveyed.[Footnote 
52] 

- The presence of government employees or parties to the conflict can 
lead to over-or underreporting. 

- A reluctance to forgo food rations may lead to underreporting of 
deaths in the household. 

- Dating deaths that occurred months prior to survey can be 
difficult.[Footnote 53] 

- The length of the survey's recall period may lead to under-or 
overreporting of deaths and affect the precision of estimated mortality 
rates.[Footnote 54] 

- Identifying some causes of death can be difficult.[Footnote 55] 

* Insufficient training and resources among research staff. 
Difficulties in training staff and assembling resources may also have 
affected the quality of the surveys conducted. Mortality surveys 
require staff who are proficient in matters such as designing 
questionnaires, selecting samples of the local population,[Footnote 56] 
and conducting interviews in local languages. Although larger 
organizations that routinely conduct surveys generally have staff that 
are experienced in designing and implementing surveys, other groups may 
not have as much knowledge and experience in collecting data, according 
to experts and the research literature. Although the NGOs provide some 
training, high turnover rates make it difficult to retain the levels of 
knowledge and skill that are required. 

* Challenges related to sampling methods. The research literature notes 
that samples drawn in IDP or refugee camps may not provide an accurate 
count of deaths that occurred in attacks on villages and when IDPs were 
fleeing to the camps.[Footnote 57] However, although it recognizes the 
difficulty of selecting and implementing an appropriate sampling method 
to measure violent deaths outside camps, the literature does not offer 
any definitive solutions. 

In addition, the NGOs that conduct mortality surveys in Darfur do so 
primarily to monitor conditions in the camps they serve, and they 
generally do not disseminate the survey results. To address this 
problem, CE-DAT was established in 2003, under the Standardized 
Monitoring and Assessment of Relief and Transitions (SMART) 
initiative,[Footnote 58] to provide quick access to accurate and 
reliable data needed by humanitarian decision makers. However, the 
usefulness of the database is limited, because NGOs are not reporting 
the survey results systematically and because the quality of many of 
the reported surveys is problematic. 

Moreover, mortality data from other sources, such as prospective or 
ongoing surveillance systems[Footnote 59] that systematically record 
deaths, are generally unavailable in Darfur. The research literature 
notes that existing systems for registering or reporting deaths 
generally collapse when crises occur and that prospective surveillance 
systems established in crisis situations have limitations.[Footnote 
60]However, the literature also notes that data from graveyard counts 
and regular reporting and surveillance systems have been used in some 
crises, despite such limitations, in conjunction with survey-based data 
to estimate mortality in crisis situations.[Footnote 61] In Darfur, 
data from surveillance systems were generally not available, although 
one expert reported that systems had been set up in some of the larger 
camps by the middle of 2004. However, the expert also reported that 
these systems were capturing data in ways that would allow them to be 
used only qualitatively, rather than quantitatively, for death toll 
estimates. 

Gaps in Data Leads to Reliance on Assumptions and Limited Information: 

Gaps in data on mortality can lead to reliance on extrapolations based 
on potentially risky assumptions and limited contextual information. 
Because the available data for Darfur cover certain geographic areas 
and time periods, the estimates that we examined assume that mortality 
rates for surveyed locations and specific periods can apply to 
unsurveyed locations and longer periods. Most of the experts we 
consulted voiced concerns about such extrapolations. Some experts noted 
that factors such as patterns of attack and displacement, humanitarian 
relief efforts, and the incidence of disease might cause mortality 
rates in the surveyed areas to differ from rates in the unsurveyed 
areas. One expert stated that he could only speculate on conditions and 
mortality rates in the areas that had not been surveyed. In addition, 
several experts expressed concern about extrapolations from limited 
time periods to longer periods, noting that mortality rates can change 
rapidly. 

Some of the estimates' assumptions are informed by contextual 
information in the absence of data; however, this information also had 
limitations. For example, some estimates relied on anecdotal reports of 
conditions in Darfur, satellite imagery on attacks of villages, and 
information about weather conditions to determine appropriate mortality 
rates. While the experts generally approved of the use of contextual 
information, they did not believe that the information used could 
compensate for the gaps in the data. 

Limitations in Population Data Affect Reliability of Death Estimates: 

Limitations in the population data before and during a crisis such as 
Darfur's can also impact researchers' ability to produce reliable death 
estimates. No definitive estimate of Darfur's pre-crisis population 
exists, and estimates of the current population vary considerably, from 
around 4 to 7 million. The difficulty of estimating the region's 
population is compounded by the fact that migration was widespread in 
Darfur before the crisis, making it difficult to get accurate counts. 
Because the death estimates essentially extrapolate the mortality rates 
from the surveys to the entire population affected by the crisis, an 
estimate of the affected population that is too high or too low will 
lead to an over-or underestimate of the death toll. The estimates we 
reviewed generally used data for the affected population that were 
reported in the UN Humanitarian Profiles; however, the collection and 
reliability of the profile data had limitations.[Footnote 62] For 
example, the profiles estimated a population of more than 160,000 IDPs 
in Kalma camp, the largest in Darfur, prior to October 2005. Yet, a 
subsequent count by a relief agency in October 2005 reported only 
87,000 IDPs in the camp. An expert also noted that figures based on 
registration can be unreliable and depend greatly on the agency 
responsible for registration. He stated that sources of bias include 
poor coverage by the agency (underestimation), as well as problems with 
populations' being "double-counted" to increase the amount of aid 
delivered (overestimation). 

Varying Use of Baseline Mortality Rates Can Affect Estimates' Accuracy: 

Varying approaches to the selection of a baseline for normal mortality 
can raise or lower death estimates, possibly making the totals overly 
high or low. All but one of the Darfur death estimates that we examined 
subtract baselines for normal or expected mortality from the total 
deaths, based on the assumption that some deaths from disease, old age, 
or malnutrition would have occurred without the crisis. For example, 
CRED's use of a baseline mortality rate of 0.3 subtracted about 16,000 
deaths from the total estimate of 134,000, resulting in 118,000 
"excess" deaths. If CRED had used a baseline of 0.5, it would likely 
have subtracted about 26,000 deaths, obtaining a somewhat lower 
estimate of deaths from the crisis.[Footnote 63] 

The experts we convened proposed various baseline mortality rates, 
ranging from 0.3 to 0.75 deaths per 10,000 affected persons per 
day.[Footnote 64] One reason for the difference in the suggested rates 
is the range of methods that the experts used to select them--for 
example, a rate that prevailed in the country before the crisis, a 
standard based on prior humanitarian crises, or a rate from a 
comparable country in the region. However, each of these methods has 
acknowledged limitations that could result in inaccurate estimates of 
expected rates and, therefore, in over-or underestimations of mortality 
due to the crisis. 

In addition, the experts debated whether a baseline of any sort was 
justified for a humanitarian crisis such as Darfur, arguing ethical and 
philosophical, rather than technical, considerations. About half of the 
experts said that deaths that would have occurred regardless of the 
crisis should be subtracted from the death toll attributed to the 
crisis. However, two experts took a contrary position, arguing that the 
concept of expected or normal levels of mortality was not appropriate 
in the presence of genocide or ethnic cleansing because the 
perpetrators of those crimes against humanity should be considered 
culpable for all deaths that resulted from the crises they 
instigated.[Footnote 65] Using a baseline to estimate mortality would 
lead to a somewhat smaller excess death toll than not using a baseline. 
For example, State's estimate of total deaths ranged from 98,000 to 
181,000, minus 35,000 expected deaths; thus, State estimated 63,000 to 
146,000 excess deaths directly resulting from the crisis. 

Wide Range of Measures Could Improve Death Estimates for Darfur and 
Future Humanitarian Crises: 

The group of experts proposed and rated a wide range of measures that 
U.S. agencies could take directly, or support other entities in taking, 
to improve the quality and reliability of death estimates and relevant 
data for Darfur and future humanitarian crises. The measure rated most 
likely to produce the most improvements was ensuring sufficient public 
documentation of estimates' data and methods to allow replication of 
the methods, verification of the findings, and confirmation of the 
estimates' credibility and objectivity. Other highly rated measures 
included collecting and maintaining temporal and spatial data, housing 
responsibility for making the estimates in a reputable independent 
body, improving the training of nongovernmental organizations' staff 
who collect survey mortality data, and promoting an interdisciplinary 
approach to estimating mortality. U.S. agencies are engaged in several 
initiatives that address some of these measures. 

Proposed Measures Cover Wide Range of Activities: 

The experts suggested a series of measures covering a broad range of 
activities that U.S. agencies or other, U.S.-funded or -supported 
entities could take to improve death estimates for future humanitarian 
crises.[Footnote 66] (See table 2.) Although the experts acknowledged 
the importance and necessity of estimating mortality--for example, to 
help hold perpetrators accountable and to have a complete historical 
record--they generally believed that death estimates should be 
conducted with caution. To assist in this endeavor, the experts 
proposed and rated 19 measures as likely to produce some improvements 
in data collection and mortality estimation. Some experts differed as 
to whether government agencies or other entities would be best placed 
to implement particular suggestions. 

Table 2: Measures Rated by Experts as Likely to Greatly Improve Death 
Estimates for Future Crises, in Order of Ranking and Number of 
Endorsements: 

Measure: Ensure that publicly available documents on U.S. government 
estimates provide sufficient information on methods, data, assumptions, 
and limitations; 
Number of endorsements: 11. 

Measure: Support the collection and maintenance of temporal and spatial 
data; 
Number of endorsements: 10. 

Measure: House responsibility for mortality estimates in a reputable, 
independent body or group; 
Number of endorsements: 9. 

Measure: Improve training of NGO staff who collect mortality survey 
data; 
Number of endorsements: 9. 

Measure: Promote an interdisciplinary approach to estimating mortality 
(include epidemiologists and demographers); 
Number of endorsements: 9. 

Measure: Create technical teams, under the auspices of an international 
body, that can conduct mortality estimates as needed; 
Number of endorsements: 8. 

Measure: Report mortality and morbidity information more routinely and 
systematically to provide an ongoing sense of the situation; 
Number of endorsements: 8. 

Measure: Promote data collection by NGOs on the ground at routine 
service points in addition to periodic assessments; 
Number of endorsements: 7. 

Measure: Promote the use of other measures of a conflict's severity 
(e.g., displaced persons, number of attacks) in addition to mortality 
estimates; 
Number of endorsements: 7. 

Measure: Improve existing surveying techniques (e.g., cluster sampling) 
by incorporating spatial or temporal information; 
Number of endorsements: 7. 

Measure: Provide guidance (minimum standards) on how to design and 
implement survey instruments in the affected region (e.g., pretesting 
or translation techniques); 
Number of endorsements: 6. 

Measure: Provide guidance on amalgamating existing mortality/morbidity 
surveys; 
Number of endorsements: 5. 

Measure: Define criteria for selecting and using data; 
Number of endorsements: 4. 

Measure: Make satellite tapes and imagery available to researchers; 
Number of endorsements: 3. 

Measure: Provide guidance on triangulating different types of data; 
Number of endorsements: 3. 

Measure: Provide assistance to local statistical agencies to improve 
data collection; 
Number of endorsements: 3. 

Measure: Develop algorithms to track population change over time; 
Number of endorsements: 3. 

Measure: Create a statistical unit under appropriate agency (possibly 
under auspices of the Committee on National Statistics) to be 
responsible for these types of estimates; 
Number of endorsements: 3. 

Measure: Tap other resources, such as pro bono groups of statisticians, 
that could provide assistance; 
Number of endorsements: 1. 

Source: GAO. 

Note: The table is based on analysis of 12 experts in epidemiology, 
demography, statistics, and the Darfur crisis. Numbers shown under 
"Number of endorsements" represent the number of experts who rated the 
measure as likely to "very greatly" or "greatly" to improve the 
estimates. In several instances, one or two experts noted problems with 
suggestions that had been highly rated by others. The experts rated an 
additional measure for the Darfur crisis, namely, whether an 
independent agency should conduct a retroactive assessment. 

[End of table] 

Majorities of the experts rated 10 of the 19 measures, and 9 of the 12 
experts rated 5 of the measures, as likely to "very greatly" or 
"greatly" improve the estimates for future crises.[Footnote 67] 
Following are the 10 most highly rated measures: 

* Ensure that publicly available documents on the U.S. government 
estimates provide sufficient information on methodology. Sufficient 
documentation and transparency of data and methods are needed to allow 
independent researchers to verify the reliability and validity of 
estimates. Although this suggestion would not improve the estimates per 
se, experts felt that it was important for establishing the credibility 
of the estimates. 

* Support the collection and maintenance of temporal and spatial data. 
Temporal and spatial databases would allow researchers to track 
mortality over time and across regions and could improve researchers' 
ability to estimate mortality. Experts suggested that temporal and 
spatial data could be collected in a number of ways, including through 
the use of satellite imagery. 

* House responsibility for mortality estimates in a reputable, 
independent entity or group. Experts indicated that an independent 
entity could be perceived as free of bias, political goals, or both. 
However, such an entity would need adequate funding, and experts had 
differing views as to where it should be housed. One expert felt it 
could be placed under a U.S. organization such as the CDC, while 
another felt it should be under an international organization such as 
the WHO. 

* Improve training of NGO staff who collect mortality survey data. 
Because NGO staff conduct most of the surveys, improving their training 
would help improve the quality of data collected. Several experts 
reported the existence of initiatives to improve and standardize NGOs' 
data collection in emergencies, such as the SMART initiative, and of 
guidance on the topic from the World Food Program and the WHO. Some 
experts suggested expanding these efforts with additional U.S. 
government support. 

* Promote an interdisciplinary approach to estimating mortality 
(include epidemiologists and demographers). Because epidemiologists and 
demographers bring different expertise and perspectives to mortality 
estimation, collaboration between the disciplines could lead to greater 
understanding of each other's approaches and, ultimately, to more 
integrated methodologies for death estimates. 

* Create technical teams, under the auspices of an international body, 
that can conduct mortality estimates as needed. Technical teams, 
assembled as needed and operating under independent organizations, 
could--rather than a single, permanent independent entity--be 
responsible for conducting death estimates. Such teams might be 
perceived as free of bias and political goals, which would increase the 
credibility of their findings. However, some experts reported that such 
technical teams already exist, citing as examples the WHO, the 
International Rescue Committee, and Médecins Sans Frontières, among 
others.[Footnote 68] 

* Report mortality and morbidity information more routinely and 
systematically to provide an ongoing sense of the situation. This 
suggestion is intended to allow researchers and governments to monitor 
crises as they develop. Although this suggestion was relatively highly 
rated, experts had some questions about who would do the reporting, how 
the data would be gathered, and how the estimates would be made. One 
expert noted that groups like CRED are currently reporting such 
mortality and morbidity information somewhat routinely and 
systematically. 

* Promote data collection by NGOs on the ground at routine service 
points, in addition to periodic assessments. Proponents stated that 
humanitarian relief organizations could collect data on famine-related 
deaths and that human rights organizations could collect narrative 
testimonies about political violence and conflict-related displacement. 
Some experts believed that the NGOs could collect these data with 
little additional time or expense and that the data collection could 
help create an early warning system for famine and diseases. However, 
several experts were concerned that data collected at routine service 
points would provide biased estimates because certain segments of the 
affected population would not be likely to pass through these points. 

* Promote the use of other measures of a conflict's severity (e.g., 
displaced persons, number of attacks) in addition to mortality 
estimates. Such alternative estimates could provide insights into 
crises and help check the accuracy of the mortality data. These 
estimates could include conflict-related displacement, property 
destruction, property looting, exposure to violence as a marker for 
psychological trauma, incidence of sexual and gender-based violence, 
forced displacement, and data from security incidence reports. One 
expert indicated that some of these data are probably being collected 
by various human rights agencies and organizations and suggested 
creating a body to collate and report their data. 

* Improve existing surveying techniques (e.g., cluster sampling) by 
incorporating spatial or temporal information. A few experts felt that 
improvements in surveying techniques were needed because of limitations 
in certain aspects of cluster sampling, the most commonly used 
technique; however, another expert stated that cluster 
sampling[Footnote 69] was extremely useful. One expert argued that 
existing surveying techniques should take into account the spatial and 
temporal distribution of the affected population. 

Existing U.S. Initiatives May Improve Some Aspects of Data and Death 
Estimates: 

Several current U.S. initiatives may enhance the availability and 
quality of mortality data and produce more accurate death estimates for 
Darfur and other similar crises in the future. These initiatives embody 
several of the measures rated by the group of experts as likely to 
improve death estimates; however, several of the experts, as well as 
U.S. officials knowledgeable about the initiatives, whom we spoke with 
indicated that more can be done. 

* CE-DAT may improve death estimates through its promotion of more 
routine reporting of mortality and morbidity information and defining 
some criteria based on methodological requirements for assessing the 
quality of data. In addition, CE-DAT includes a cartographical database 
to link indicators to maps, which aligns with the experts' 
recommendation for collecting and maintaining spatial and temporal 
data. CE-DAT staff discussed some limitations, with respect to the data 
in the database, including the fact that information is not provided to 
CE-DAT routinely and CE-DAT is not widely known. CE-DAT staff also said 
that there is currently no systematic means of verifying data 
reliability and validity of data and making data publicly available in 
a timely manner. 

* SMART, an interagency initiative coordinated by USAID, may improve 
death estimates through its promotion of a standard methodology to 
design and implement survey instruments that measure crude mortality, 
as well as other indicators of need.[Footnote 70] According to a USAID 
official, SMART is also intended to help build the technical capacity 
of NGO field workers and host government partners to collect and report 
more reliable data. To this end, the SMART initiative aligns with some 
of the recommendations made by our group of experts, including, 
improving the training of NGO staff that collect mortality data and 
reporting mortality and morbidity data more routinely and 
systematically. Experts and officials whom we spoke with generally 
agreed that the idea behind SMART represents positive steps toward 
improving mortality data collection. However, they also discussed 
various limitations of the initiative. For example, an official 
involved with the effort noted that that funding has been at times 
sporadic and that limited resources have stymied outreach efforts. 
Others stated that some of SMART's methodological principles need 
further research and testing in the field and that no consensus 
currently exists regarding some of these principles. Officials from 
USAID indicated that SMART's major components, including mortality, 
nutrition, and food security, have been piloted in several countries 
and that the SMART methodology is meant to be iterative with continuous 
upgrading based on further research and best practices. 

* The Humanitarian Information Unit, housed in State's Bureau of 
Intelligence and Research, may enhance the quality and availability of 
spatial and temporal information used in future death estimates through 
its current efforts to develop of maps and other visual images of 
humanitarian crises. For example, for the Darfur crisis, it has 
developed a series of maps showing the number of destroyed villages or 
reported attacks in the region at different points in time based on 
data from satellite imagery or reports from on the ground organizations 
such as the African Union, the UN, and USAID. The unit's work also 
addresses two additional suggested measures--promoting the use of other 
measures of conflict and improving existing surveying techniques by 
incorporating spatial and temporal data. However, the Humanitarian 
Information Unit, in some cases, has had difficulty obtaining 
standardized, reliable, or complete data. Additionally, the unit does 
not consistently and systematically collect data, such as preconflict 
population estimates, in part because the unit performs its work upon 
request. 

* The International Emergency and Refugee Health Branch of the CDC may 
improve death estimates and the collection of mortality data through 
its provision of technical assistance to UN and U.S. agencies 
collecting data in complex emergencies and its support of the 
collection of spatial and temporal data.[Footnote 71] However, 
officials from the International Emergency and Refugee Health Branch 
stated that because it often works as a consultative body and its 
resources are constrained, the scope of its work is limited. 

Several international initiatives also address problems with data 
collection and analysis. The WHO plans to implement a Health and 
Nutrition Tracking Service that would routinely monitor mortality and 
malnutrition during major crises and provide a central help desk, 
remotely located, for field staff conducting surveys. Additionally, the 
World Food Program has created a manual on survey techniques to measure 
health and nutrition indicators, including crude mortality. 

The group of experts we convened were generally aware of, or involved, 
with these U.S. government and international initiatives, as well as 
some NGO initiatives, that addressed certain aspects of their 
suggestions. For example, three experts assisted in the development of 
the SMART guidelines, and two of the three were also technical advisors 
for the CE-DAT initiative. Some experts felt that certain suggestions 
could be met by investing more in these existing initiatives rather 
than by creating new ones. U.S. officials responsible or aware of these 
initiatives also acknowledged limitations with several of these or 
thought that more could be done with respect to the measures suggested 
by the experts. 

Conclusions: 

Despite variance among the death estimates we reviewed, each of the 
estimates shows, tragically and unquestionably, that many thousands of 
civilians died in Darfur between February 2003 and August 2005. Given 
the continuing conflict, insecurity, and displacement in the region, 
many more have undoubtedly died since the estimates were conducted, as 
a direct result of violence or because of increased vulnerability to 
disease and malnutrition. However, despite the importance of the death 
estimates in showing the severity of the crisis, none of the estimates 
consistently received high ratings in terms of accuracy or 
methodological strengths from the experts we convened. Some of the 
shortcomings in the estimates' source data, methods, and objectivity 
may be attributed to challenges in mortality data collection and 
extrapolation, characteristic of a humanitarian crisis such as Darfur. 
However, in certain cases, a lack of transparent reporting of an 
estimate's data, methods, assumptions, or limitations hindered the 
experts from replicating it and thus verifying its accuracy and 
credibility. In particular, many experts noted the unavailability of 
these published details--and the resulting lack of transparency--for 
State's estimate. 

Several ongoing U.S. initiatives may produce enhancements that align 
with the experts' suggested measures for addressing gaps in data and 
improving death estimates conducted for Darfur and any similar 
humanitarian crises in the future. For example, the SMART initiative 
and CE-DAT, respectively, provide guidance on how to design and 
implement surveys and promote more routine reporting of mortality and 
morbidity data. However, experts and U.S. government officials observed 
that gaps in these areas still exist. Among the measures that have not 
been addressed, the one that the experts rated most highly--ensuring 
the public availability of information on estimate methods, data, 
assumptions, and limitations--is essential to protect the credibility 
of U.S. government death estimates for Darfur and any future 
humanitarian crises. 

Recommendations for Executive Action: 

To safeguard the Department of State's credibility as a source of 
accurate and reliable death estimates, we recommend that the Secretary 
of State promote greater transparency in any of its future death 
estimates for Darfur or other humanitarian crises by ensuring that 
publicly available documents contain sufficient detail on the 
estimates' data, methods, assumptions, and limitations to allow 
external researchers to replicate and verify the estimates. 

Additionally, to enhance the U.S. government's capacity to assess the 
dimensions of, and respond appropriately to, any future humanitarian 
crises, we recommend that the Secretary of State and the Director of 
U.S. Foreign Assistance and USAID Administrator consider the experts' 
other suggested measures to help address gaps in data and improve the 
quality and reliability of any future death estimates. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to the Department of State and 
USAID. State and USAID responded with formal comments, agreeing with 
our recommendations, and State provided additional perspectives on 
reporting and documentation regarding its death estimate. Reproductions 
of these letters, as well as our responses to the letters, can be found 
in appendixes VI and VII. We also provided a draft to the CDC for 
technical review, and we received technical comments from both the CDC 
and State, which we incorporated in the report as appropriate. 

We provided the authors of the other five estimates the portions of the 
report pertaining to their individual estimates. The authors of the 
CRED estimates agreed with the experts' evaluation. Jan Coebergh did 
not provide any major comments regarding the relevant portion of the 
report and indicated that he was aware of the limitations of his 
estimate. John Hagan expressed concerns regarding the experts' 
relatively positive evaluation of the estimates by CRED and State and 
believed we should have included the estimate he did with Alberto 
Palloni in our evaluation. David Nabarro from the WHO disagreed with 
our inclusion of his estimate with the others in the report and said 
that we had not clearly conveyed the estimate's purpose. Eric Reeves 
disagreed with the experts' criticism of the Atrocities Documentation 
Team's survey of Chad refugees, stating that the survey was well 
conducted and a critical source of data to estimate violent deaths, 
particularly early on in the conflict when little data was available. 
More details on the authors' comments and our responses can be found in 
appendix VIII. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies of this report 
to the Secretary of State, the Director of U.S. Foreign Assistance and 
USAID Administrator, the Director of the CDC, relevant congressional 
committees, and other interested parties. We will also make copies 
available to others on request. In addition, the report will be 
available on GAO's Web site at [Hyperlink, http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact either David Gootnick at (202) 512-3149 or gootnickd@gao.gov, 
or Nancy Kingsbury at (202) 512-2700 or kingsburyn@gao.gov. Contact 
points for our Offices of Congressional Relations and Public Affairs 
may be found on the last page of this report. Key contributors are 
listed in appendix IX. 

Signed by: 

David Gootnick, Director: 
International Affairs and Trade: 

Signed by: 

Nancy Kingsbury, Managing Director: 
Applied Research and Methods: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

This report (1) evaluates the relative accuracy and methodological 
strengths and shortcomings of six death estimates for Darfur, including 
one by the Department of State (State); (2) identifies general 
challenges to estimating the total death toll in Darfur and similar 
humanitarian crises; and (3) discusses measures that the U.S. 
government could take to improve its death estimates for Darfur and any 
similar, future crises. 

To evaluate the relative accuracy and methodological strengths and 
shortcomings of the Darfur death estimates, we selected estimates that 
had been made public prior to March 2006. We imposed this cutoff point 
so that we could interview the authors, prepare materials for the 
experts, and give the experts time to review the materials. Because our 
cutoff point was March 2006, the convening experts' discussion was of 
death estimates that spanned various time periods between February 2003 
and August 2005. Our findings do not, therefore, discuss any estimates 
that were made after March 2006. 

Additionally, to address these objectives, we convened, in 
collaboration with the National Academy of Sciences, 12 experts for an 
all-day meeting on April 7, 2006, in Washington, D.C., to discuss the 
six death estimates. (Two additional experts also participated by phone 
for parts of the day.) The academy proposed lists of experts, and we 
approved their final selection. The selected experts had extensive 
knowledge of estimating mortality rates and death totals in conflict 
and postconflict situations or other types of humanitarian crises and 
were familiar with the sorts of data and estimates that have been 
produced for the Darfur crisis or other such crises in Africa. 
Additionally, the selected experts represented a range of professional 
experiences and backgrounds, including epidemiologists, demographers, 
statisticians, and directors of aid programs currently in Darfur. They 
were affiliated with various organizations, including universities, 
U.S. government and United Nations (UN) agencies, and humanitarian aid 
and nonprofit organizations based in the United States, as well as 
overseas. All of the experts signed a form from the National Academy of 
Sciences certifying that they had no conflicts of interest that could 
compromise their ability to assess the death estimates objectively. 
Some experts reported they had had professional contact with several of 
the authors. 

During the meeting, the experts discussed their evaluation of each of 
the Darfur death estimates, as well as challenges to estimating total 
deaths for Darfur and similar crises and measures to improve such 
estimates. The meeting was recorded and transcribed to ensure that we 
accurately captured the experts' statements, and we reviewed the 
transcripts as a source of evidence. (See app. II for a list of the 
experts' names and affiliations and a summary of the meeting agenda and 
discussion questions.) 

In addition, prior to the meeting, we reviewed and analyzed public 
information on the estimates, including documents describing the 
estimates and source data for the estimates and provided this 
information to the experts.[Footnote 72] The source data documents that 
we reviewed included 15 reports of health, nutrition, and mortality 
surveys; State publication describing the July-August 2004 survey with 
refugees in Chad; and volumes of the UN Nutrition Information in Crisis 
Situations containing descriptions of additional health, nutrition, and 
mortality surveys of which we were unable to obtain full reports. (See 
app. III for a complete list of published documents that were provided 
to the experts.) We also provided the experts supplemental information 
on the design, sampling and implementation of the Atrocities 
Documentation Team's survey of Chad refugees and a copy of the survey 
instrument, which we obtained from representatives of the Coalition of 
International Justice. Additionally, we interviewed the estimates' 
authors regarding the data, methods, and objectives of their work and 
replicated their estimates when we had sufficient information. We 
provided summaries of these interviews to the experts, as well as 
replications of the estimates, reviewed and approved by the authors, in 
advance of the meeting. We spoke with all of the estimate authors 
except State's Bureau of Intelligence and Research, which declined to 
speak with us or provide additional information. Although this limited 
the ability of our experts to fully understand State's methods of 
analysis as noted during their discussion, the experts were able to 
discuss State's estimate in detail and provide assessments of its data, 
methods, objectivity, limitations, and accuracy. 

Following the meeting, we also asked the experts to answer additional 
questions on each estimate, as well as questions on suggested measures 
to improve estimates through a follow-up data collection instrument. We 
developed the instrument with the help of survey specialists and based 
the questions in the instrument on the meeting agenda and points that 
arose during the meeting deliberations. We pretested the instrument 
with two experts and made changes based on their input. We administered 
the instrument via e-mail and received responses from all 12 of the 
experts who attended the meeting in Washington, D.C. In some instances, 
we contacted the respondents by e-mail or phone to obtain greater 
clarity or details regarding their answers. 

Further, to identify challenges involved in estimating total deaths in 
humanitarian crises, such as that in Darfur, we asked the group of 
experts to highlight key challenges during their discussion, and we 
reviewed literature related to death estimates and mortality data for 
humanitarian crises. We summarized the parts of the experts' discussion 
that most directly addressed challenges, and we identified themes that 
were raised by the experts. The literature we reviewed included 
articles that we identified through databases, such as ProQuest, Lexis 
Nexis, and Medline, using various search terms, such as mortality 
estimates, death estimates, humanitarian crise(s), conflict(s), or 
(complex) emergency/ emergencies. We also reviewed literature cited in 
these articles and on Web sites related to humanitarian assistance or 
data analysis in humanitarian crises, as well as literature recommended 
by the group of experts[Footnote 73] and the authors of the estimates. 
In total, we reviewed about 20 articles. We organized the individual 
challenges identified by the experts and in the literature according to 
the two overarching themes we identified from the experts' discussion 
and from the literature: (1) challenges affecting the source data used 
for the death estimates and (2) challenges affecting the generation of 
the death estimates. We also provided portions of the draft pertaining 
to the challenges to conducting death estimates as well as the 
background regarding mortality surveys to several of the experts we 
convened and incorporated their technical comments into the final 
version of the report. 

Finally, to identify measures that the U.S. government could take to 
improve death estimates for Darfur and in future humanitarian crises, 
we asked the experts to provide suggestions during the meeting and 
solicited further opinions on these suggestions in the follow-up 
instrument. The instrument listed 19 measures that the experts had 
suggested as likely to improve death estimates for Darfur and such 
crises that may occur in the future and asked the experts to rate them 
with a five-point scale, ranging from "Very greatly improve" to "Not 
improve."[Footnote 74] We ranked the measures according to the numbers 
of respondents that rated them "Very greatly improve" and "Greatly 
improve." The instrument also asked for experts' comments on each of 
the suggested measures, and we followed-up with some experts to get 
additional information regarding comments on suggested measures, 
particularly with respect to current efforts. Additionally, we spoke 
with officials from the U.S. Agency for International Development, 
State, and the Department of Health and Human Services' Centers for 
Disease Control and Prevention to learn of any current U.S. government 
initiatives related to the suggested measures. We also reviewed public 
information and documents provided by the officials and available on 
the Internet that describe efforts by the U.S. government, as well as 
other international initiatives. 

We conducted our work in San Francisco and Washington, D.C., from 
September 2005 to November 2006 in accordance with generally accepted 
government auditing standards. 

[End of section] 

Appendix II: List of Experts and Meeting Agenda: 

This appendix provides the names and affiliation of the experts 
participating in the meeting held on April 7, 2006, in Washington, 
D.C., and a summary of the day's discussion questions. 

List of Experts: 

The following experts attended the meeting: 

* Jana Asher, American Association for the Advancement of Science: 

* Richard Brennan, Health Unit, International Rescue Committee: 

* Francesco Checchi, London School of Hygiene and Tropical Medicine: 

* Allan Hill, Harvard School of Public Health, Harvard University: 

* Arif Husain, Vulnerability Analysis and Mapping Unit, United Nations 
World Food Program: 

* Mark Myatt, University College of London: 

* W. Courtland Robinson, Bloomberg School of Public Health, Johns 
Hopkins University: 

* William Seltzer, Department of Sociology and Anthropology, Fordham 
University: 

* Romesh Silva, Human Rights Data Analysis Group, The Benetech 
Initiative: 

* Michael VanRooyen, Program on Humanitarian Crises and Human Rights, 
Harvard University: 

* Ronald Waldman, Mailman School of Public Health, Columbia University: 

* Bradley Woodruff, Maternal and Child Nutrition Branch, U.S. Centers 
for Disease Control and Prevention: 

The following experts participated in the meeting by phone for parts of 
the day: 

* Bushra Gamar Hussein, Darfur Region, Sudan Social Development 
Organization: 

* Jennifer Leaning, Harvard School of Public Health, Harvard 
University: 

Summary of Meeting Discussion Questions: 

Darfur Death Estimates: 

* Are the source data used in each of the estimates sufficiently 
representative and reliable? 

* To what extent do the methods used in each of the estimates follow 
principles that are generally accepted in the field? 

* Do the key assumptions made in the estimates seem reasonable? 

* How accurate or inaccurate do you think each of these estimates are 
in terms of representing the actual number of deaths that occurred in 
Darfur during the time period and regions under consideration? 

* For each of the estimates, is sufficient information presented in the 
reports to adequately assess the strengths and weaknesses and/or the 
reasonableness of the estimates? Do you understand the processes by 
which the estimates were derived? 

* What are the major strengths and limitations of each of these 
estimates for Darfur? 

Challenges to Estimating Total Deaths for Darfur and Similar Crisis: 

* In your opinion, what sources of data for what time periods and 
locations would be necessary to produce reasonably precise estimates of 
death totals for Darfur? 

* What general observations do you have concerning the challenges of 
estimating total deaths in Darfur? 

Implications for the U.S. Government: Ways to Improve Death Estimates 
for Darfur and Future Crises: 

* In your view, are there any lessons learned about conducting death 
estimates in Darfur for the U.S. government? If so, what are these? 

* Are there measures the U.S. government could take or promote to 
improve estimates for Darfur or similar crises in the future? 

[End of section] 

Appendix III: List of Death Estimates and Mortality Surveys Provided to 
Experts and Additional Bibliographical References: 

Death Estimates[Footnote 75] 

Estimate by the U.S. Department of State: 

* State Department's Fact Sheet, "Sudan: Death Toll in Darfur," March 
25, 2005. [Hyperlink, 
http://www.state.gov/s/inr/rls/fs/2005/45105.htm](Viewed, August 24, 
2006): 

* M. Phelan in D. Guha-Sapir and O. Degomme, "Darfur: Counting the 
Deaths" (Method 2). Centre for Research on the Epidemiology of 
Disasters (CRED). Brussels: May 26, 2005. [Hyperlink, 
http://www.cred.be/docs/cedat/DarfurCountingTheDeaths-
withClarifications.pdf] (Viewed October 23, 2006): 

Estimate by the Centre for Research on the Epidemiology of Disasters: 
Debarati Guha-Sapir and Olivier DeGomme: 

* Guha-Sapir, D., O. Degomme, and M. Phelan, "Darfur: Counting the 
Deaths" (Method 1). Centre for Research on the Epidemiology of 
Disasters (CRED). Brussels: May 26, 2005. [Hyperlink, 
http://www.cred.be/docs/cedat/DarfurCountingTheDeaths-
withClarifications.pdf] (Viewed October 23, 2006): 

* Guha-Sapir, D. and O. Degomme, "Darfur: Counting the Deaths (2): What 
are the trends?" Centre for Research on the Epidemiology of Disasters 
(CRED). Brussels: December 15, 2005. [Hyperlink, 
http://www.cred.be/docs/cedat/DarfurCountingTheDeaths2.pdf] (Viewed 
October 23, 2006): 

Estimate by Jan Coebergh: 

* Coebergh, J. "Sudan: genocide has killed more than the tsunami." 
Parliamentary Brief, vol. 9, no. 7, pp. 5-6 (February 2005). 
[Hyperlink, http://www.thepolitician.org/] (Viewed, August 24, 2006): 

Estimate by John Hagan, Wenona Rymond-Richmond, and Patricia Parker and 
announced by the Coalition for International Justice: 

* Coalition for International Justice Press Release, "New Analysis 
Claims Darfur Deaths Near 400,000." April 21, 2005. 

* Hagan, J., W. Rymond-Richmond, and P. Parker, "The Criminology of 
Genocide: The Death and Rape of Darfur," Criminology, vol. 43, no. 3, 
pp. 525-561 (July 2005). 

Estimate by Eric Reeves: 

* Portions of mortality updates from www.sudanreeves.org. (Reviewed by 
author.) Dr. Reeves began reporting his mortality updates in June 11, 
2004. The most recent update reviewed by the experts was from August 
31, 2005. 

Estimate by the World Health Organization: 

* Nabarro, D. "Mortality Projections for Darfur," World Health 
Organization Briefing presented October 15, 2004. [Hyperlink, 
http://www.who.int/mediacentre/news/briefings/2004/en] (Viewed, August 
24, 2006): 

Surveys Used in Estimates as Data Sources: 

State or regionwide estimates: 

* U.S. Centers for Disease Control and Prevention and UN World Food 
Program, "Emergency Nutrition Assessment of Crisis Affected Populations 
Darfur Region, Sudan, August-September 2004." [Hyperlink, 
http://www.reliefweb.int/library/documents/2004/wfp-sdn-26ocr.pdf]. 
(Viewed, August 24, 2006): 

* UN World Food Program, "Emergency Food Security and Nutrition 
Assessment in Darfur, Sudan." (Rome, Italy: World Food Program, October 
2004). [Hyperlink, 
http://www.wfp.org/country_brief/africa/sudan/assessments/041025_food_as
sessment.pdf] (Viewed, August 24, 2006): 

* World Health Organization, "Retrospective Mortality Survey Among the 
Internally Displaced Population Greater Darfur, Sudan 2004." September 
15, 2004. [Hyperlink, 
http://www.reliefweb.int/library/documents/2004/who-sdn-15sep.pdf] 
(Viewed, August 24, 2006): 

* World Health Organization, "Mortality Survey among Internally 
Displaced Persons and Other Affected Populations in Greater Darfur, 
Sudan." September 2005. [Hyperlink, 
http://www.emro.who.int/sudan/pdf/CMS%20Darfur%202005%20final%20report_1
1%2010%2005.pdf] (Viewed, August 24, 2006): 

North Darfur: 

* Dubray, C., "Health assessment in emergencies: Serif Umra, North 
Darfur, Sudan." Epicentre and Medecins Sans Frontiers. November 2004. 

* Sibson, V., "Findings of a nutrition survey of Kutum town, Kasab camp 
and Fata Borno, North Darfur." GOAL. May 2005.[Footnote 76] 

West Darfur: 

* Abadallah, S., "Report of rapid baseline survey Azirni, Sanidadi and 
Um Tagouk." Center for International Emergency, Refugee and Disaster 
Studies, Johns Hopkins University and World Relief. October 12, 2004. 

* Deconinick, H. and O. Karouri, "Nutrition assessment; Fur Baranga, 
Administrative Unit, Habila Locality, West Darfur, Sudan." Save the 
Children, USA. January 14-17, 2005. [Hyperlink, 
http://www.humanitarianinfo.org/darfur/uploads/assessments/fb%20nutritio
n%20survey%20report%20jan%2005.doc] (Viewed, August 24, 2006): 

* Depoortere, E., F. Checchi, F. Broillet, S. Gerstl, A. Minettia, O. 
Gayraud et al., "Violence and mortality in West Darfur, Sudan (2003- 
2004): epidemiological evidence from four surveys" Lancet, vol. 364, 
no. 9442, pp. 1,315-1,320 (2004). [Hyperlink, 
http://www.msf.fr/documents/base/2004-10-01-Depoortere.pdf] (Viewed, 
August 24, 2006): 

* Depoortere, E., "Health assessment in emergencies: Murnei and 
Zalingei, West Darfur, Sudan." Epicentre and Médecins Sans Frontières. 
(Paris, France: Médecins Sans Frontières, June 2004). [Hyperlink, 
http://www.doctorswithoutborders.org/publications/reports/2004/epicentre
_report_darfur_05-2004.pdf] (Viewed, August 24, 2006): 

* Hearns, A. and A. Agar, "Nutritional Survey, Mornei Camp, January- 
February 2005." Concern Worldwide in collaboration with State Ministry 
of Health and UNICEF. 

* Rio, D., "Nutrition and Food Security Assessment, Wade Saleh and 
Mukjar Provinces, West Darfur, North-Sudan." Médecins Sans Frontières 
Holland. April 2004. 

* West Darfur State, Ministry of Health in collaboration with UNICEF, 
Concern and Save the Children-US. "Nutrition and Mortality Survey in 
Ardamat, Dorti, Riyad and Abu-zar IDP camps." June 14-18, 2005. 

South Darfur: 

* Grandesso, F., F. Sanderson, J. Kruijit, T. Koene, and V. Brown, 
"Mortality and Malnutrition among Populations Living in South Darfur, 
Sudan." JAMA, vol. 293, no. 12, pp. 1,490-1,494 (March 2005). 
[Hyperlink, http://jama.ama-assn.org/cgi/content/abstract/293/12/1490] 
(Viewed, August 24, 2006): 

Chad: 

* United Nations High Commissioner for Refugees et al., "Emergency 
Nutrition and Mortality Surveys Conducted among Sudanese Refugees and 
Chadian Villagers, Northeast Chad, June 2004." [Hyperlink, 
http://www.cdc.gov/nceh/ierh/ResearchandSurvey/Chad_report04.pdf] 
(Viewed, August 24, 2006): 

Other Information Provided: 

* UN Nutrition Information in Crisis Situations, volumes 3, 4, 6, 7 
from 2004 and 2005. [Hyperlink, 
http://www.unsystem.org/scn/publications/RNIS/countries/sudan.htm] 
(Viewed, August 24, 2006): 

* Bureau of Democracy, Human Rights and Labor and Bureau of 
Intelligence and Research, Department of State, "Documenting Atrocities 
in Darfur," Publication 11182 (Washington, D.C.: Bureau of Democracy, 
Human Rights, and Labor and the Bureau of Intelligence and Research, 
September 2004). [Hyperlink, http://www.state.gov/g/drl/rls/36028.htm] 
(Viewed, August 24, 2006): 

Additional Bibliographical References: 

* Boss, L., M. Toole, and R. Yip, "Assessment of Mortality, Morbidity 
and Nutritional Status in Somalia During the 1991-1992 Famine: 
Recommendations for Standardization of Methods," JAMA, vol. 272, no. 5 
(1994). 

* Checchi, F. and L. Roberts. "Interpreting and using mortality data in 
humanitarian emergencies: A primer for non-epidemiologists," 
Humanitarian Practice Network, Network Paper no. 52. (London, UK, 
2005). 

* Coghlan, B., R. Brennan, P. Ngay, D. Dofara, B. Otto, M. Clements, 
and T. Stewart, "Mortality in the Democratic Republic of Congo: a 
nationwide survey," The Lancet, vol. 367, pp. 44-51 (2006). 

* Guha-Sapir, D. and W. van Panhuis, "Conflict Mortality: Comparing the 
data." CE-DAT Working Paper, series no. CD/2004/001. (2004). 

* Hofman, C., L. Roberts, J. Shoham, P. Harvey, "Measuring the impact 
of humanitarian aid: A review of current practice," Humanitarian Policy 
Group. (London, UK, 2004). 

* Leaning, J. and M. VanRooyen. "An assessment of mortality studies in 
Darfur, 2004-2005," Humanitarian Exchange, vol. 30, pp. 23-26 (2005). 

* Myatt M., A. Taylor, and W. Courtland Robinson, "A method for 
estimating mortality rates using previous birth history," Field 
Exchange, p. 13 (2002). 

* Reed, H. (Rapporteur),"Demographic Assessment Techniques in Complex 
Humanitarian Emergencies, Summary of a Workshop." National Academy 
Press, Washington, D.C. (2002). 

* Reed, H. and C. B. Keeley (eds.), Forced Migration and Mortality. 
National Academy Press, Washington, D.C. (2001). 

* Salama, P., P. Spiegel, L. Talley, and R. Waldman, "Lessons Learned 
from complex emergencies over past decade," The Lancet, vol. 364, 
pp.1,801-1,813 (2004).p: 

* Silva, R. and P. Ball, "The Demography of Conflict-Related Mortality 
in Timor-Leste (1974-1999): Empirical Quantitative Measurement of 
Civilian Killings, Disappearances & Famine-Related Deaths." In Human 
Rights and Statistical Objectivity. J. Asher, D. Banks, F. Schueren, 
eds. New York: Springer (forthcoming). 

* Spiegel, P., P. Salama, S. Maloney, and A. van der Veen, "Quality of 
Malnutrition Assessment Surveys Conducted During Famine in Ethiopia," 
JAMA, vol. 292, no. 5 (2004). 

* Thieren, M., "Health information systems in humanitarian 
emergencies," Bulletin of the World Heath Organization, vol. 83, no. 8 
(2005). 

* Toole, M. and R. Waldman, "The Public Health Aspects of Complex 
Emergencies and Refugee Situations," Annual Review of Public Health, 
vol. 18, pp. 283-312 (1997). 

* U.S. Centers for Disease Control and Prevention and The World Food 
Program, A Manual: Measuring and Interpreting Malnutrition and 
Mortality. Washington and Rome. (2005). 

* Woodruff, B., "Violence and mortality in West Darfur." The Lancet, 
vol. 364, pg. 1,290 (2004). 

* Woodruff, B., "Interpreting mortality data in humanitarian 
emergencies," The Lancet, vol. 367, pg. 9 (2006). 

* World Food Program, "Emergency Food Security and nutrition assessment 
in Darfur, Sudan 2005." Rome. (December 2005). [Hyperlink, 
http://documents.wfp.org/stellent/groups/public/documents/ena/wfp089682.
pdf] (Viewed October 23, 2006): 

* World Health Organization, "Module 4. Studying Health Status and 
Health Needs." [Hyperlink, 
http://www.who.int/hac/techguidance/tools/disrupted_sectors/module_04/en
/index2.html] (Viewed October 23, 2006): 

[End of section] 

Appendix IV: Summary Description of Death Estimates Reviewed: 

Table 3: Estimate by Jan Coebergh: 

Reported figures; 
Three possible point estimates for excess deaths: 218,449; 253,573;or 
306,130 represent a range of death estimates all above 200,000. 

Time period covered; 
April 2003 through December 2004. 

Total number of months; 
21. 

Data sources; 
* Uses varied sources including Atrocities Documentation Team's survey 
of Chad refugees based on number of people who reported seeing family 
member killed, 2004 mortality surveys by World Health Organization 
(WHO) and Médicins Sans Frontiers and estimates by WHO officials; 
* Does not use any contextual or nonsurvey information. 

Mortality rates applied and cause of death; 
* Calculates violent and nonviolent deaths separately; 
* 306,130: Includes nonviolent and violent deaths. About 170,000 from 
violence, 108,000 from health causes, and 25,000 in inaccessible areas; 
* 218,449: Includes nonviolent and violent deaths. About 73,700 from 
violence and 126,000 from health causes; 
* 253,573: Includes nonviolent and violent deaths. About 111,000 from 
health causes and about 143,000 from violence; 
* Applies some daily mortality rates; other mortality rates are monthly 
or for longer periods of time. 

Mortality changes over time and/or region; 
Assumes the same mortality levels over time and per region. 

Baseline crude mortality rate assumed; 
0.5. 

Affected population included in estimate; 
* Uses one "fixed" population estimate of 400,000 which is the average 
of an assumed starting point of 0 in February 2003 and 800,000 in April 
2004. (Affected population does not change over time.); 
* Estimate includes affected population from all regions in Darfur and 
refugees in Chad. 

[End of table] 

Table 4: Centre for Research on the Epidemiology of Disasters (CRED): 
Two Estimates: 

Reported figures; 
* 134,000 total deaths; 
* 118,142 excess deaths; 
* 36,237 total deaths; 
* 23,658 excess deaths. 

Time period covered; 
* September 2003 through January 2005; 
* February 2005 through June 2005. 

Number of months; 
* 17; 
* 5. 

Data sources; 
* Examines data from about 20 mortality surveys. About half of these 
were used to estimate nonviolent mortality rates, and half were used to 
analyze the proportion due to violence. (Surveys used depended on what 
surveys reported the proportion of deaths due to violence.); 
* Some steps were taken to examine the reliability of methods used in 
the surveys and exclude outliers; 
* Examines about 10 surveys conducted between January and August 2005; 
however, primarily uses findings from the 2005 statewide WHO survey 
because these results did not differ with other smaller surveys that 
covered the same period. 
* Relies mainly on mortality survey findings and uses other sources of 
contextual or nonsurvey information to refine both estimates. 

Mortality rates applied and cause of death; 
* Estimates nonviolent and violent deaths separately. Total crude 
mortality is nonviolence crude mortality divided by (1 minus the 
proportion due to violence). 
* Nonviolent mortality is generally assumed stable except for the 
hunger (or rainy season) during June through August 2004; 
* Proportion due to violence is based on a regression analysis of 
survey data; 
* 2005 WHO survey findings include crude mortality rates and proportion 
of deaths due to violence or injury. 

Mortality changes over time and/or region; 
* Nonviolent mortality rate is assumed stable at 1.1 for all regions 
except June through August 2004 for which rates are higher due to 
impact of rainy season. (Rates used are based on results from the 2004 
WHO survey for each region.); 
* Based on regression analysis, proportion of violence generally 
decreases over time beginning December 2003 to January 2005. Prior to 
December 2003, assumes steady rate of violence; 
* Assumes some difference in violence between regions; 
* Assumes different mortality levels per region but no difference over 
time from February to June 2005; 
* Shows increase in violence for North Darfur from last estimate. 

Baseline crude mortality rate assumed; 
* 0.3 based on national average from the United Nations Children's Fund 
(UNICEF). 

Affected population included in estimate; 
* Assumes different monthly estimates of the affected population based 
on information from the United Nations Humanitarian Profiles for each 
Darfur state from September 2003 till January 2005; 
* Excludes deaths from isolated areas with high violence rates in South 
Darfur after September 2004; 
* Adds additional 200,000 for Chad refugees; their deaths estimated as 
10% of the total excess deaths estimated; 
* Assumes different monthly estimates separated by internally displaced 
persons (IDP) and affected residents based on information from the 
United Nations Humanitarian Profiles; 
* Includes all regions in Darfur; 
* Applies different mortality rates to groups of affected residents, 
IDPs in camps, IDPs outside camps (did not do this in first estimate). 

[End of table] 

Table 5: Estimate by John Hagan and Others and Released by the 
Coalition for International Justice: 

Reported figures; 
396,563 total deaths. 

Time period covered; 
February 2003 through March 2005. 

Total number of months; 
26. 

Data sources; 
* Uses Atrocities Documentation Team's survey of Chad refugees to 
estimate violent deaths based on number of people who reported seeing a 
family member killed; 
* Uses 2004 WHO mortality survey findings for North and West Darfur to 
estimate nonviolent deaths; 
* Does not use any contextual or nonsurvey information. 

Mortality rates applied and cause of death; 
* Nonviolent mortality rate is on a combination of findings from North 
and West Darfur WHO survey; 
* Violent mortality rate is calculated based on survey of Chad refugees 
and assumptions include that all missing persons have died and that the 
average family size is five. 

Mortality changes over time and/or region; 
* Assumes the same mortality levels over time and per region. 

Baseline crude mortality rate assumed; 
* 0 (Did not apply a baseline mortality rate because believed it was 
not appropriate.) 

Affected population included in estimate; 
* Uses a single estimate of 1.5 million, as of April 2005, to include 
IDP population from all regions in Darfur and refugees in Chad. 1.5 
million is the midpoint between 1.2 and 1.8 million. (1.2 comes from 
State's Documenting Atrocities Report, and 1.8 was stated as widely 
cited as the affected population in early 2005.) 

[End of table] 

Table 6: Estimate by Eric Reeves: 

Reported figures; 
Over 370,000 excess deaths. 

Time period covered; 
February 2003 through August 2005. 

Number of months; 
31. 

Data sources; 
* Uses varied sources such as the Atrocities Documentation Team's 
survey of refugees in Chad, the 2005 WHO mortality survey, 
communication from David Nabarro, and projections from the Department 
of U.S. Agency for International Development; 
* Uses contextual or nonsurvey data in various calculations, such as 
mortality rates for inaccessible populations. 

Mortality rates applied and cause of death; 
* Generally estimates nonviolent and violent deaths separately; 
* Survey of Chad refugees to estimate violent deaths based on number of 
people who reported seeing family member killed; 
* Estimates some mortality rates per day; estimates other rates per 
month or longer periods of time. 

Mortality changes over time and/or region; 
* Assumes some changes in mortality over time. Generally does not 
assume different mortality rates per region except for populations in 
inaccessible areas. 

Baseline crude mortality rates; 
* 0.3 based on UNICEF. 

Affected population included in estimate; 
* Assumes affected population changes somewhat over time, although not 
on a monthly basis. Uses information from UN profiles, as well as other 
sources. For example, in some cases, assumes some groups are excluded 
from UN profiles and increases numbers accordingly; 
* Includes affected population from all regions in Darfur and Chad 
refugees. 

[End of table] 

Table 7: Department of State Estimate: 

Reported figures; 
* 98,000 to 181,000 total deaths; 
* 63,000 to 146,000 excess deaths. 

Time period covered; 
March 2003 through January 2005. 

Total number of months; 
23. 

Data sources; 
* Uses data deemed sound from available mortality surveys in the 
Complex Emergency Database at that time for Darfur and refugees in Chad 
(refers to list of about 20 surveys included in report); 
* Provides some information regarding which surveys and contextual data 
sources were used and why; 
* Applies contextual information to fill in data gaps and determine if 
existing mortality rates should be applied to a broader population 
within that region or if a higher or lower rate was warranted. Refers 
to information such as displacement patterns, fire mapping remote 
sensing data, reporting of attacks, historical trends on humanitarian 
intervention, and disease outbreaks. 

Mortality rates applied and cause of death; 
* Estimates high and low mortality rates that are assumed to include 
mortality due to all causes (nonviolent and violent); 
* Reports that some mortality rates were adjusted based on contextual 
information (e.g., report of attacks). For example, reports using West 
Darfur survey data as a basis for estimating mortality in North Darfur 
are based on nonsurvey data showing similar patterns of conflict in 
2003 and early 2004, but estimates of higher total deaths in North 
Darfur are due to larger displaced population and more incidents of 
fighting. 

Mortality changes over time and/or region; 
* Assumes differences in mortality over time and per region; 
* Divides conflict into four time periods: (1) the initial outbreak of 
violence (March through September 2003), (2) the breakdown of cease-
fire/escalation of conflict (October 2003 through March 2004), (3) the 
second cease-fire (April through June 2004) and (4) increased 
international humanitarian response (July 2004 through January 2005); 
* Reports that highest mortality and level of violence occurred during 
December 2003 to February 2004. Mortality began falling in March 2004. 

Baseline crude mortality rate assumed; 
Uses 0.5 deaths per 10,000 per based on estimates from the WHO of sub-
Saharan African populations. 

Affected population included in estimate; 
* Assumes different monthly estimates (separated by displaced and 
affected populations) based on UN profiles for each Darfur state from 
September 2003 until January 2005; 
* Assumes an incremental increase prior to September 2003 because no 
source of reliable information was available. Uses data from the United 
Nations High Commissioner for Refugees (UNHCR) on spontaneous 
settlements and camps to estimate Chad population; 
* Assumes affected residents experienced mortality rates that were 20% 
lower than displaced populations. 

[End of table] 

Table 8: WHO Estimate: 

Reported figures; 
* 45,000 to 80,000 total deaths; 
* 35,000 to 70,000 excess deaths. 

Time period covered; 
March 2004 through September 2004. 

Total number of months; 
7. 

Data sources; 
* Primarily uses findings from 2004 WHO mortality survey, although 
looked at other surveys conducted; 
* Uses contextual information, such as the occurrence of disease 
outbreaks and the level of humanitarian assistance, to help refine 
estimates and define best and worst case scenarios. 

Mortality rates applied and cause of death; 
Estimates mortality rates include all causes of death (violent and 
nonviolent). 

Mortality changes over time and/or region; 
Estimates an average mortality rate for the entire region of Darfur. 
Does not assume different mortality rates over time or per region. 

Baseline crude mortality rate assumed; 
0.5. 

Affected population included in estimate; 
* Assumes different monthly estimates of IDPs based on UN profile 
reports; 
* Estimate includes only IDPs in camps and excludes affected residents 
from all Darfur regions and refugees in Chad. 

Source: GAO analysis based on published documents of estimates and 
interviews with estimate authors. 

Note: The summaries of these estimates are based on the published 
information, as well as interviews with and additional information 
provided by estimate authors with the exception of State who declined 
to speak with us. Additionally, mortality rates discussed in table are 
crude mortality rates per 10,000 per day. 

[End of table] 

[End of section] 

Appendix V: Additional Follow-Up Survey Results: 

This is a summary of responses to additional questions from our follow- 
up survey from the 12 experts we convened with the National Academy of 
Sciences in April 2006. 

(1) In your view, are the mortality estimates by the listed authors 
based on methodologically sound source data? Mark only one response in 
each row. 

Authors/estimates: Coebergh; 
Definitely yes: 0; 
Generally yes: 1; 
Based as much on sound as unsound data: 4; 
Generally no: 5; 
Definitely no: 2; 
No basis to judge/not sure: 0. 

Authors/estimates: CRED(Guha-Sapir and Degomme); 
Definitely yes: 0; 
Generally yes: 9; 
Based as much on sound as unsound data: 2; 
Generally no: 1; 
Definitely no: 0; 
No basis to judge/not sure: 0.  

Authors/estimates: Hagan, Rymond-Richard, and Parker; 
Definitely yes: 0; 
Generally yes: 0; 
Based as much on sound as unsound data: 2; 
Generally no: 7; 
Definitely no: 2; 
No basis to judge/not sure: 0. 

Authors/estimates: Reeves; 
Definitely yes: 0; 
Generally yes: 0; 
Based as much on sound as unsound data: 2; 
Generally no: 7; 
Definitely no: 3; 
No basis to judge/not sure: 0. 

Authors/estimates: U.S. Department of State (Phelan); 
Definitely yes: 0; 
Generally yes: 7; 
Based as much on sound as unsound data: 2; 
Generally no: 1; 
Definitely no: 1; 
No basis to judge/not sure: 1. 

Authors/estimates: WHO (Nabarro); 
Definitely yes: 1; 
Generally yes: 6; 
Based as much on sound as unsound data: 3; 
Generally no: 0; 
Definitely no: 2; 
No basis to judge/not sure: 0. 

[End of table] 

(2) In your view, do the authors make appropriate or inappropriate 
extrapolations from the source data to the affected population ? Mark 
only one response in each row. 

Authors/estimates: Coebergh; 
Very appropriate: 0; 
Somewhat appropriate: 0;  
About as appropriate as inappropriate: 1; 
Somewhat inappropriate: 5; 
Very inappropriate: 5; 
No basis to judge/not sure: 1. 

Authors/estimates: CRED (Guha-Sapir and Degomme); 
Very appropriate: 1; 
Somewhat appropriate: 7;  
About as appropriate as inappropriate: 2; 
Somewhat inappropriate: 1; 
Very inappropriate: 0; 
No basis to judge/not sure: 1. 

Authors/estimates: Hagan, Rymond-Richmond, and Parker; 
Very appropriate: 0; 
Somewhat appropriate: 0;  
About as appropriate as inappropriate: 1; 
Somewhat inappropriate: 5; 
Very inappropriate: 6; 
No basis to judge/not sure: 0. 

Authors/estimates: Reeves; 
Very appropriate: 0; 
Somewhat appropriate: 0;  
About as appropriate as inappropriate: 0; 
Somewhat inappropriate: 5; 
Very inappropriate: 7; 
No basis to judge/not sure: 0. 

Authors/estimates: U.S. Department of State (Phelan); 
Very appropriate: 0; 
Somewhat appropriate: 5; 
About as appropriate as inappropriate: 3; 
Somewhat inappropriate: 2; 
Very inappropriate: 0; 
No basis to judge/not sure: 2. 

Authors/estimates: WHO (Nabarro); 
Very appropriate: 0; 
Somewhat appropriate: 4; 
About as appropriate as inappropriate: 3; 
Somewhat inappropriate: 2; 
Very inappropriate: 2; 
No basis to judge/not sure: 1. 

[End of table] 

(3) In your view, are the assumptions made by the authors reasonable or 
unreasonable? (Mark only one response in each row. 

Authors/estimates: Coebergh; 
Very reasonable: 0; 
Somewhat reasonable: 0; 
As many reasonable as unreasonable assumptions: 2; 
Somewhat unreasonable: 4; 
Very unreasonable: 5; 
No basis to judge/Not sure: 1. 

Authors/estimates: CRED (Guha-Sapir and Degomme); 
Very reasonable: 1; 
Somewhat reasonable: 9; 
As many reasonable as unreasonable assumptions: 0; 
Somewhat unreasonable: 1; 
Very unreasonable: 1; 
No basis to judge/Not sure: 0. 

Authors/estimates: Hagan, Rymond-Richmond, and Parker; 
Very reasonable: 0; 
Somewhat reasonable: 2; 
As many reasonable as unreasonable assumptions: 0; 
Somewhat unreasonable: 4; 
Very unreasonable: 6; 
No basis to judge/Not sure: 0. 

Authors/estimates: Reeves; 
Very reasonable: 0; 
Somewhat reasonable:0;  
As many reasonable as unreasonable assumptions: 1; 
Somewhat unreasonable: 4; 
Very unreasonable: 7; 
No basis to judge/Not sure: 0. 

Authors/estimates: U.S. Department of State (Phelan); 
Very reasonable: 1; 
Somewhat reasonable: 6;  
As many reasonable as unreasonable assumptions: 1; 
Somewhat unreasonable: 2; 
Very unreasonable: 0; 
No basis to judge/Not sure: 2. 

Authors/estimates: WHO (Nabarro); 
Very reasonable: 0; 
Somewhat reasonable: 5;  
As many reasonable as unreasonable assumptions: 3; 
Somewhat unreasonable: 2; 
Very unreasonable: 1; 
No basis to judge/Not sure: 1. 

[End of table] 

(4) In your view, do the authors sufficiently or insufficiently 
describe appropriate limitations, including sources of possible over or 
under estimation? (Mark only one response in each row.) 

Authors/estimates: Coebergh; 
Very sufficiently: 0; 
Somewhat sufficiently: 3;  
As sufficiently as insufficiently: 0; 
Somewhat insufficiently: 3; 
Very insufficiently: 6; 
No basis to judge/Not sure: 0. 

Authors/estimates: CRED (Guha-Sapir and Degomme); 
Very sufficiently: 2; 
Somewhat sufficiently: 6;  
As sufficiently as insufficiently: 2; 
Somewhat insufficiently: 2; 
Very insufficiently: 0; 
No basis to judge/Not sure: 0. 

Authors/estimates: Hagan, Rymond-Richmond, and Parker; 
Very sufficiently: 0; 
Somewhat sufficiently: 0;  
As sufficiently as insufficiently: 3; 
Somewhat insufficiently: 1; 
Very insufficiently: 8; 
No basis to judge/Not sure: 0. 

Authors/estimates: Reeves; 
Very sufficiently: 0; 
Somewhat sufficiently: 1;  
As sufficiently as insufficiently: 0; 
Somewhat insufficiently: 4; 
Very insufficiently: 7; 
No basis to judge/Not sure: 0. 

Authors/estimates: U.S. Department of State (Phelan); 
Very sufficiently: 1; 
Somewhat sufficiently: 4;  
As sufficiently as insufficiently: 2; 
Somewhat insufficiently: 4; 
Very insufficiently: 0; 
No basis to judge/Not sure: 1. 

Authors/estimates: WHO (Nabarro); 
Very sufficiently: 0; 
Somewhat sufficiently: 7;  
As sufficiently as insufficiently: 1; 
Somewhat insufficiently: 2; 
Very insufficiently: 2; 
No basis to judge/Not sure: 0. 

[End of table] 

(5) In your view, how high or low would you rate the level of 
objectivity in the authors' mortality estimates? (Mark only one 
response in each row. 

Authors/estimates: Coebergh; 
Very high: 0; 
High: 0; 
Moderate: 2; 
Low: 8; 
Very Low: 2; 
No basis to judge/Not sure: 0. 

Authors/estimates: CRED (Guha-Sapir and Degomme); 
Very high: 2; 
High: 8; 
Moderate: 1; 
Low: 0; 
Very Low: 0; 
No basis to judge/Not sure: 1. 

Authors/estimates: Hagan, Rymond-Richmond, and Parker; 
Very high: 1; 
High: 0; 
Moderate: 5; 
Low: 3; 
Very Low: 2; 
No basis to judge/Not sure: 1. 

Authors/estimates: Reeves; 
Very high: 0; 
High: 0; 
Moderate: 0; 
Low: 8; 
Very Low: 4; 
No basis to judge/Not sure: 0. 

Authors/estimates: U.S. Department of State (Phelan); 
Very high: 0; 
High: 9; 
Moderate: 2; 
Low: 0; 
Very Low: 0; 
No basis to judge/Not sure: 1. 

Authors/estimates: WHO (Nabarro); 
Very high: 0; 
High: 6; 
Moderate: 3; 
Low: 1; 
Very Low: 1; 
No basis to judge/Not sure: 1. 

[End of table] 

(6) Overall, what is your level of confidence in the estimates made by 
each of the authors? (Mark only one response in each row. 

Authors/estimates: Coebergh; 
Very high: 0; 
High: 0; 
Moderate: 0; 
Low: 7; 
Very Low: 5; 
No basis to judge/Not sure: 0. 

Authors/estimates: CRED (Guha-Sapir and Degomme); 
Very high: 0; 
High: 2; 
Moderate: 9; 
Low: 1; 
Very Low: 0; 
No basis to judge/Not sure: 0. 

Authors/estimates: Hagan, Rymond-Richmond, and Parker; 
Very high: 0; 
High: 0; 
Moderate: 1; 
Low: 3; 
Very Low: 8; 
No basis to judge/Not sure: 0. 

Authors/estimates: Reeves; 
Very high: 0; 
High: 0; 
Moderate: 0; 
Low: 4; 
Very Low: 8; 
No basis to judge/Not sure: 0. 

Authors/estimates: U.S. Department of State (Phelan); 
Very high: 0; 
High: 2; 
Moderate: 7; 
Low: 2; 
Very Low: 1; 
No basis to judge/Not sure: 0. 

Authors/estimates: WHO (Nabarro); 
Very high: 0; 
High: 0; 
Moderate: 7; 
Low: 3; 
Very Low: 1; 
No basis to judge/Not sure: 1. 

[End of Table] 

[End of section] 

Appendix VI: Comments from the Department of State: 

Note: GAO comments supplementing those in the report text appear at the 
end of this appendix. 

United States Department of State: 
Assistant Secretary for Resource Management and Chief Financial 
Officer: 
Washington, D. C. 20520: 

Ms. Jacquelyn Williams-Bridgers: 
Managing Director: 
International Affairs and Trade: 
Government Accountability Office 441 G Street, N. W. 
Washington, D.C. 20548-0001: 

Oct 4 2006: 

Dear Ms. Williams-Bridgers: 

We appreciate the opportunity to review your draft report, "Darfur 
Crisis: Death Estimates Show Crisis Severity, but Their Accuracy and 
Credibility Could Be Enhanced," GAO Job Code 320420. 

The enclosed Department of State comments are provided for 
incorporation with this letter as an appendix to the final report. 

If you have any questions concerning this response, please contact Jim 
Gray, Congressional Liaison, Bureau of Intelligence and Research, at 
(202) 647-2921. 

Sincerely, 

Signed by: 

Bradford R. Higgins: 

cc: GAO - Leslie Holen: 
INR - Randall M. Fort: 
State/OIG - Mark Duda: 

Department of State Comments on GAO Draft Report: 

Darfur Crisis: Death Estimates Show Crisis Severity, but Their Accuracy 
and Credibility Could be Enhanced (GAO-7-24, GAO Code 320420): 

Thank you for allowing the Department of State the opportunity to 
comment on the draft report Darfur Crisis: Death Estimates Show Crisis 
Severity, but Their Accuracy and Credibility Could be Enhanced. 

GAO Recommendations: 

To safeguard the Department of State's credibility as a source of 
accurate and reliable death estimates, we recommend that the Secretary 
of State promote greater transparency in any of its future death 
estimates for Darfur or other humanitarian crises by ensuring that 
publicly available documents contain sufficient detail on the 
estimates' data, methods, assumptions, and limitations to allow 
external researchers to replicate and verify the estimates. 

Additionally, to enhance the U.S. government's capacity to assess the 
dimensions of, and respond appropriately to, any future humanitarian 
crises, we recommend that the Secretary of State and the Director of 
U.S. Foreign Assistance and USAID Administrator consider the experts' 
other suggested measures to help address gaps in data and improve the 
quality and reliability of any future death estimates. 

Department of State Comment: 

The Department of State endorses these recommendations and supports 
efforts to increase transparency, address gaps in data, and improve the 
quality of future death estimates. As the GAO points out, humanitarian 
crises present difficult challenges in collecting and presenting 
mortality data. 

In the case of the Darfur Mortality Estimate, the State Department 
declassified an internal assessment of the Bureau of Intelligence and 
Research (INR) originally intended for use by Department policymakers. 
INR's source documentation and description of methodology were provided 
to the Centre for Research on the Epidemiology of Disasters (CRED) for 
use in compiling the CRED study titled Darfur: Counting the Deaths, a 
source for this GAO report. 

Following are GAO's comments on the Department of State's letter dated 
October 4, 2006. 

GAO Comments: 

1. State asserts that its death estimate was intended for internal 
purposes. We maintain that because State's estimate was publicly 
available and discussed by State officials, sufficient detail on this 
and any future such estimates is necessary to safeguard State's 
credibility as a source of accurate and reliable death estimates, 
particularly where such a serious topic is concerned. 

2. When asked, the CRED authors indicated that State provided them with 
the text on the methodology as included in the published report, but 
did not provide source information on the data used in State's 
estimate. 

[End of section] 

Appendix VII: Comments from the U.S. Agency for International 
Development: 

USAID: 
From The American People:  

Oct 17 2006: 

David Gootnick: 
Director: 
International Affairs and Trade: 
U.S. Government Accountability Office: 
441 G Street, N.W. 
Washington, D.C. 20548: 

Dear Mr. Gootnick: 

I am pleased to provide the U.S. Agency for International Development's 
(USAID) formal response on the draft GAO report entitled Darfur Crises: 
Death Estimates Show Crisis Severity, but Their Accuracy and 
Credibility Could Be Enhanced (GAO-07-24). 

Thank you for the opportunity to respond to the GAO draft report and 
for the courtesies extended by your staff in the conduct of this 
review. We appreciate GAO's comprehensive efforts in addressing this 
important subject and we support your recommendation that the U.S. 
Government should work to ensure greater transparency regarding the 
data and methods used for death estimates. USAID also concurs with your 
second recommendation that the Secretary of State and the Director of 
U.S. Foreign Assistance and USAID Administrator consider the experts' 
other suggested measures to help address gaps in data and improve the 
quality and reliability of future death estimates. 

USAID looks forward to working to help address gaps in data and improve 
the quality of future death estimates. Thank you again for the 
opportunity to comment. 

Sincerely, 

Signed by: 

Mosina H. Jordan: 
Counselor to the Agency: 

[End of section] 

Appendix VIII: Summary of Authors' Comments: 

To the five other estimate authors, we provided portions of the report 
pertaining to their individual estimates. We summarize their comments 
and our responses below. 

* The authors of the Centre for Research on the Epidemiology of 
Disasters (CRED) estimates agreed with the experts' evaluation. 

* Jan Coebergh did not provide any major comments regarding the 
relevant portion of the report and indicated that he was aware of the 
limitations of his estimate. 

* John Hagan expressed concern regarding the experts' relatively 
positive evaluations of the estimates by CRED and the Department of 
State (State), stating that the respective estimates' published 
documentation does not provide sufficient information on the data and 
methodologies used. As discussed in the report, the experts noted 
limitations in the sufficiency of reporting for both these estimates, 
but they were able to discuss these estimates in detail and assess 
their respective levels of accuracy and methodological strengths and 
shortcomings. In addition to providing the experts the published 
reports containing the estimates, we provided them information 
regarding the source mortality surveys used in the estimates, as listed 
in appendix III. For the CRED estimate, we also provided the experts 
with additional details about the methodology based on our interviews 
with the authors. Dr. Hagan also believed that we should have included 
in our evaluation the estimate he did with Alberto Palloni published in 
the September 2006 issue of Science. We added information on the key 
findings of this estimate to the report, but as we note in the report, 
it does not include an analysis of the estimate because a final 
publicly available version was not available when the experts convened 
in April 2006. 

* Eric Reeves disagreed with the experts' criticism of the Atrocities 
Documentation Team's survey of Chad refugees. He stated this survey was 
well conducted and a critical source of data to estimate violent deaths 
especially early in the conflict when little information was available. 
In the report, we note that the literature acknowledges difficulties in 
capturing deaths due to violence. Nevertheless, some of the estimates 
reviewed by the experts used other techniques to account for violent 
deaths, such as statistical analysis or contextual information. 
Moreover, many experts felt that Atrocities Documentation Team's survey 
was not a reliable or appropriate source of data to estimate violent 
deaths for a cumulative death estimate on Darfur, based on public 
documentation on the survey, as well as supplemental information on the 
survey's design, implementation, and sampling we obtained from 
representatives at the Coalition for International Justice. In 
addition, Dr. Reeves also expressed a concern regarding the 2004 survey 
by WHO and believed it did not sufficiently capture violent mortality 
in areas outside camps. To address this concern, we added information 
regarding the issue of accessibility as a potential source for 
underestimation of mortality and specifically discuss this survey. Dr. 
Reeves also stated that in a draft version of the report, we had 
incorrectly characterized his estimate's use of the 2004 WHO survey to 
account for only nonviolent deaths and, accordingly, we removed this 
characterization from the report. 

* Regarding the WHO estimate, David Nabarro emphasized in his comments 
that the estimate was undertaken to provide a rough order of magnitude 
of deaths in order to facilitate humanitarian relief efforts. He also 
pointed out that it applied to a shorter time period than the other 
estimates and was restricted to IDPs located in camps, without 
attempting to account for deaths in the larger population. We provided 
additional clarification in the report to emphasize the more restricted 
scope and coverage of the WHO estimate. Further, we stated that the 
purpose of the WHO estimate was to provide an order of magnitude 
estimate to assist in humanitarian relief planning. 

[End of section] 

Appendix IX: GAO Contacts and Staff Acknowledgments: 

GAO Contacts: 

David Gootnick, (202) 512-3149, gootnickd@gao.gov Nancy Kingsbury, 
(202) 512-2700, kingsburyn@gao.gov: 

Staff Acknowledgments: 

In addition to the individuals named above, Emil Friberg, Assistant 
Director; Jim Ashley; Martin de Alteriis; Etana Finkler; Leslie Holen; 
Theresa Lo; Reid Lowe; Grace Lui; John F. Miller; and Chhandasi Pandya 
made key contributions to this report. 

FOOTNOTES 

[1] Simultaneously with the release of this report, we are issuing a 
separate report on U.S. humanitarian assistance to relieve the crisis 
in Darfur; challenges that have affected the delivery of the 
assistance; the African Union's efforts to fulfill a mandate to support 
peace in Darfur; and factors that have affected its implementation of 
this mandate. See GAO, Darfur Crisis: Progress in Aid and Peace 
Monitoring Threatened by Ongoing Violence and Operational Challenges, 
GAO-07-9 (Washington, D.C.: Nov. 9, 2006). 

[2] The five estimates that were publicly available in March 2006 were 
conducted by, respectively, (1) Jan Coebergh, a medical doctor who has 
worked in Darfur; (2) the Centre for Research on the Epidemiology of 
Disasters (CRED) in Brussels, Belgium; (3) John Hagan, Wynnona Rymond- 
Richmond, and Patricia Parker (released by the Coalition of 
International Justice); (4) Eric Reeves, an researcher and advocate for 
Sudan-related issues; and (5) the World Health Organization (WHO). 

[3] This report's assessment of the estimates' methodological strengths 
and shortcomings is based on experts' opinions of the estimates' data, 
methods, objectivity, and sufficiency of reporting. 

[4] Two additional experts participated by phone for parts of the day. 

[5] CRED conducted two death estimates: one for the period of September 
2003-January 2005 and another for the period of February-June 2005. 

[6] The three highest estimates reviewed by the experts ranged from 
253,573 to 396,563 deaths over varying periods of time. 

[7] We recognize that when such estimates draw from classified data, 
external researchers reviewing these data would require appropriate 
levels of security clearance. 

[8] Estimates for the preconflict population of Darfur range from about 
4 million to close to 7 million. Experts we interviewed have noted 
uncertainty regarding the population estimate for Darfur due to the 
lack of a current census and the fact that migration in this region 
occurs even during nonconflict times. 

[9] U.S. law considers any person who is outside the country of such 
person's nationality and who is unable or unwilling to return to, and 
is unable or unwilling to avail himself or herself of the protection 
of, that country because of persecution or a well-founded fear of 
persecution on account of race, religion, nationality, membership in a 
particular social group, or political opinion to be a refugee. 8 U.S.C. 
§ 1101 (a) (42)A. 

[10] "Agreement on Humanitarian Ceasefire on the Conflict in Darfur," 
signed on April 8, 2004, in N'Djamena, Chad. 

[11] Additional interim agreements were also reached, including the 
July 5, 2005, "Declaration of Principles for the Resolution of the 
Sudanese Conflict in Darfur" signed by the Sudanese government, the 
SLM/A, and the JEM. This declaration contains 17 principles to guide 
future deliberations, such as respect for the diversity of the people 
of the Sudan, democracy, a federal system of government, effective 
representation in all national government institutions, and equitable 
distribution of national wealth. 

[12] Darfur Peace Agreement" signed on May 5, 2006, at Abuja, Nigeria. 

[13] UN Security Council Resolution 1706, adopted August 31, 2006. The 
expanded UN Mission in the Sudan mandate includes, among other things, 
supporting the implementation of the 2006 Darfur Peace Agreement and 
the 2004 Humanitarian Ceasefire Agreement. 

[14] H. Con. Res. 467, 108th Cong. (2004); S. Con. Res. 133, 108th 
Cong. (2004). 

[15] The Convention on the Prevention and Punishment of the Crime of 
Genocide of 1948 defines genocide as any of the following acts 
committed with intent to destroy, in whole or in part, a national, 
ethnical, racial or religious group, as such: (a) killing members of 
the group, (b) causing serious bodily or mental harm to members of the 
group, (c) deliberately inflicting on the group conditions of life 
calculated to bring about its physical destruction in whole or in 
part,(d) imposing measures intended to prevent births within the group, 
or (e) forcibly transferring children of the group to another group. 
The U.S. government ratified the convention in 1988. According to a 
State official, the key factor in the U.S. government's genocide 
determination was the intent of the Sudanese government regarding its 
actions in Darfur (i.e., its intent to destroy, in whole or in part, a 
specific group of people); the number of deaths attributable to the 
crisis was not a critical factor. 

[16] The Coalition for International Justice (CIJ) was an 
international, nonprofit organization that supported international war 
crimes tribunals and justice initiatives. It closed its operations in 
March 2006. 

[17] Department of State, Documenting Atrocities in Darfur, Publication 
11182 (Washington, D.C.: Bureau of Democracy, Human Rights, and Labor 
and the Bureau of Intelligence and Research, September 2004). Available 
at [Hyperlink, http://www.state.gov/g/drl/rls/36028.htm]. 

[18] In addition to the U.S. assistance provided for Darfur, an 
additional $197 million has supported Darfur refugees located in 
Eastern Chad for fiscal years 2004 through 2006. 

[19] See GAO-07-9. 

[20] The estimates by Coebergh, Hagan, and Reeves used the Atrocities 
Documentation Team's survey of Chad refugees as an additional key 
source of data. 

[21] Respondents are also asked to report the number of births and the 
numbers of people in their household during the recall period. 

[22] According to the Sphere Project, the crude mortality rate can be 
expressed with different standard population denominators and for 
different time periods; however, the daily crude mortality rate is the 
most specific and useful health indicator to monitor in a disaster 
situation. [See The Sphere Project, Humanitarian Charter and Minimum 
Standards in Disaster Response (Geneva: Switzerland, 2004); also known 
as the Sphere Handbook.] Sphere, launched in 1997 by a group of 
humanitarian NGOs and the Red Cross, developed minimum standards to be 
attained in disaster assistance in five sectors: water supply and 
sanitation, nutrition, food aid, shelter, and health services. The 
Sphere Handbook reports that a doubling of the baseline crude mortality 
rate indicates a significant public health emergency, requiring an 
immediate response. If the baseline rate is unknown, health agencies 
should aim to maintain the crude mortality rate at below 1.0 per 10,000 
per day. According to data from the U.S. National Center for Health 
Statistics, the normal crude mortality rate for 2003 in the United 
States is about 0.23 deaths per 10,000 per day. 

[23] CE-DAT is a searchable database of complex emergencies that 
includes information on health and mortality indicators. The database 
which receives funds from the State's Bureau of Population, Refugees, 
and Migration, is housed in CRED, a WHO Collaborating Center located 
within the School of Public Health of the Université Catholique de 
Louvain in Brussels. (See [Hyperlink, 
http://www.cred.be/cedat/index.htm.]) According to CRED, the database 
contains 1,155 surveys covering 36 countries; half of these surveys are 
drawn from original reports, and the other half are drawn from 
secondary sources. Research staff from CE-DAT we interviewed have told 
us that, because they rely on voluntary reporting from organizations 
conducting surveys to populate their database, not all surveys that 
have been conducted in Darfur may be included in their database. 

[24] Across the four sites, Médecins Sans Frontières surveyed 3,175 
households representing a total of about 17,500 people and drawn from 
an estimated IDP population of 215,400. See Depoortere, Evelyn et al. 
"Violence and Mortality in West Darfur, Sudan (2003-2004): 
Epidemiological Evidence from Four Surveys." Lancet, 364 (2004): 1,315- 
1,320. Médecins Sans Frontières produced this study in collaboration 
with Epicentre, a nonprofit organization created in 1987 by Médecins 
Sans Frontières, which groups health professionals specialized in 
public health and epidemiology. (See Hyperlink, 
http://www.epicentre.msf.org/.) 

[25] The recall period varied in the surveys conducted across the four 
sites. The longest was 183 days for the survey conducted at Zalingei, 
and the shortest was 39 days for the survey at El Geneina. Médecins 
Sans Frontières reported using a calendar of locally important events 
to facilitate recall. 

[26] The WHO reported surveying a total of 3,140 households 
representing about 21,000 people. The WHO surveyed IDPs from 43 
locations in North Darfur and 43 locations in West Darfur, but due to 
security problems surveyed only IDPs residing in Kalma camp in South 
Darfur. See World Health Organization. Retrospective Mortality Survey 
among the Internally Displaced Population Greater Darfur, Sudan 2004, 
2004. [Hyperlink, http://www.who.int/disasters/repo/14656.pdf] 

[27] CRED is a nonprofit research institution and a World Health 
Organization Collaborating Centre based in the School of Public Health 
of the Université Catholique de Louvain in Brussels. 

[28] Jan Coebergh is a medical doctor in the Netherlands who has worked 
in Darfur. 

[29] John Hagan, a professor of sociology at Northwestern University, 
conducted his estimate with his colleagues, Wynnona Rymond-Richmond and 
Patricia Parker; the estimate was released by the Coalition of 
International Justice. Dr. Hagan stated that the 2005 estimate used an 
assumption of constant levels of mortality over time, because he 
believed other estimates had relied on that assumption. He also noted 
that he subsequently modified his assumptions and estimating methods in 
his estimate with Alberto Palloni. [See John Hagan and Alberto Palloni, 
"Social Science: Death in Darfur," Science, vol. 313 (2006): 1,578- 
1,579.] 

[30] Eric Reeves is a professor of English language and literature at 
Smith College and has spent the past 7 years working full-time on 
research and advocacy issues related to Sudan. When the experts 
convened in April 2006, they reviewed and discussed Reeves's latest 
available estimate of 375,000 through August 2005. At the end of April 
2006, he reported that total excess mortality in Darfur was greater 
than 450,000 deaths. 

[31] The WHO estimate was presented by David Nabarro, a senior WHO 
official. 

[32] See John Hagan and Alberto Palloni, "Social Science: Death in 
Darfur," Science, vol. 313 (2006): 1578-1579. In addition, the experts 
did not review a study of the Darfur conflict by Bloodhound, a Danish 
advocacy group, because it was published after the experts met in April 
2006. The study estimates that deaths in Darfur ranged from 57,000 to 
128,000 deaths between April 2003 and September 2005 due to attacks on 
villages throughout the region. [See Andreas Höfer Petersen and Lise- 
Lotte Tullin, The Scorched Earth of Darfur: Patterns in Death and 
Destruction Reported by the People of Darfur. January 2001-September 
2005 (Copenhagen: Bloodhound, 2006). Available at Hyperlink, 
http://www.bloodhound.se/rap_uk.html.] 

[33] The WHO conducted an initial estimate in July 2004 and a 
subsequent estimate in October 2004. Eric Reeves began his estimates in 
June 2004 and has reported ongoing "mortality updates" since that time. 
As of October 2006, his last update was conducted at the end of April 
2006. 

[34] Although they questioned the accuracy of the estimates and had 
differing views about the actual number of deaths that have occurred in 
Darfur, the experts did not question the severity of the crisis. In 
discussing the importance of the estimates' accuracy, several experts 
noted that estimates can be used in war crime proceedings, and one 
stated that it was important to account for all those who died so that 
this could become part of the historical record. 

[35] One expert rated having a moderate level of confidence in the 
estimate by Hagan. 

[36] In evaluating each estimate's methodological strength, the experts 
rated the source data and whether such data were methodologically 
sound; the methods, including whether extrapolations were appropriate, 
assumptions were reasonable, and shortcomings that could result in over-
or underestimation were sufficiently described; the level of 
objectivity, based on whether a particular bias appears to be part of 
the estimating procedure; and the sufficiency of reporting and 
information contained in published documents describing the estimate. 

[37] CRED selected or excluded surveys for reasons such as whether 
findings from a regional survey differed statistically from findings 
for statewide surveys or whether a survey's findings were viewed as 
outliers. CRED used the selected surveys to analyze violent or 
nonviolent mortality rates depending on, for example, whether the 
surveys reported the proportion of deaths due to violence. Although 
CRED primarily relied on mortality survey data, it also used contextual 
information to refine its estimates and make some assumptions. 

[38] Nine experts rated the data CRED used as generally sound, two 
rated the data as equally sound and unsound, and two rated the data as 
generally unsound. 

[39] CRED analyzed findings from the mortality surveys mentioned above 
to separately estimate nonviolent and violent mortality rates and 
combined these to get overall mortality rates for different regions and 
periods of the conflict. 

[40] Eight experts rated CRED's extrapolations as very or somewhat 
appropriate, two experts rated them about as appropriate as 
inappropriate, one expert rated them as somewhat inappropriate, and one 
expert found no basis to judge or was not sure. Additionally, 10 
experts rated CRED's assumptions as very or somewhat reasonable, and 
two experts rated them as somewhat or very unreasonable. 

[41] Ten experts rated CRED's level of objectivity as very high or 
high, one expert rated it as moderate, and one expert indicated that he 
or she had no basis to judge or was not sure. 

[42] State reports using various sources of information from sources 
other than surveys, such as displacement patterns, village destruction, 
and retrospective reporting of attacks to, among other things, fill in 
data gaps and determine whether mortality rates derived from existing 
surveys could be applied as an overall rate to a broader population or 
if higher or lower rates were warranted. 

[43] Seven of the 12 experts rated the State's data as generally 
methodologically sound, 2 rated the data as equally sound and unsound, 
2 rated the data as generally not sound, and 1 indicated that he or she 
was not sure or had no basis to judge. 

[44] Two experts rated the objectivity as moderate, and one said that 
he or she had no basis to judge or was not sure. 

[45] One indicated no basis to judge or not sure. 

[46] Each of the three estimates based, at least in part, its numbers 
of violent deaths on the "Atrocities Documentation Team's" survey of 
Chad refugees and its numbers of nonviolent deaths on the 2004 WHO 
survey. The Coebergh and Reeves estimates also used additional data 
sources. 

[47] The 2004 WHO survey reported that violence or injury accounted for 
10 to 21 percent of deaths in North and West Darfur and Kalma camp in 
South Darfur for a 2-month recall period in the summer of 2004. Jan 
Coebergh and John Hagan told us they knew that the WHO survey reported 
a percentage of deaths due to violence but believed the survey did not 
sufficiently capture violent deaths, particularly during the period 
prior to camp arrival. 

[48] Reeves's estimate assumes some change in mortality levels over 
time. 

[49] In Darfur, for example, some surveys were conducted in Arabic and 
not in other local languages. 

[50] One expert stated that in Darfur, a household was defined as 
everyone who ate at the same table the previous night. This definition 
is important because household size is a factor in calculating crude 
mortality rates. 

[51] One expert noted that certain cultures do not report the deaths of 
unnamed infants and, in some of these cultures, naming does not occur 
until an infant has lived for 40 days. Because the first 40 days of 
life present the highest risk of mortality, unreported infant deaths 
could compromise mortality estimates. 

[52] Known as "survivor bias" in the research literature. For example, 
see World Health Organization, "Module 4: Studying Health Status and 
Health Needs." (Available at Hyperlink, 
http://www.who.int/hac/techguidance/tools/disrupted_sectors/module_04/en
/index2.html p.3) 

[53] Known as "recall bias" in the research literature. For example, 
see World Health Organization, "Module 4: Studying Health Status and 
Health Needs." (Available at Hyperlink, 
http://www.who.int/hac/techguidance/tools/disrupted_sectors/module_04/en
/index2.html p.3.) 

[54] For example, a survey conducted by the World Food Program in the 
fall of 2004 had a recall period of 7 months. However, because the 
average length of displacement was 7.5 months, it is likely that the 
survey did not capture all mortality that occurred prior to 
displacement, some of which could have been due to violence. 

[55] For example, identifying death from particular illnesses and 
diseases can be difficult for respondents. 

[56] The research literature also questions whether appropriate 
guidance and training in sampling are routinely provided to field staff 
and whether guidance is always followed when provided. See Paul B. 
Spiegel, Peter Salama, Susan Maloney, and Albertine van der Veen, 
"Quality of Malnutrition Assessment Surveys Conducted During Famine in 
Ethiopia," JAMA, vol. 292, no. 5 (2004). 

[57] See Jennifer Leaning and Michael VanRooyen, "An Assessment of 
Mortality Studies in Darfur, 2004-2005," Humanitarian, vol. 30 (June 
2005). Also see Michel Thieren, "Health Information Systems in 
Humanitarian Emergencies," Bulletin of the World Heath Organization, 
vol. 83, no. 8 (2005). For a discussion of possible approaches to 
sampling in these situations, see Holly Reed (Rapporteur), "Demographic 
Assessment Techniques in Complex Humanitarian Emergencies," Summary of 
a Workshop (Washington, D.C.: National Academy Press, 2002). 

[58] SMART coordinated by USAID, involves experts from U.S. government 
agencies, multilateral organizations, universities, and nongovernmental 
organizations. SMART has issued guidance on how to measure mortality, 
nutrition, and food security in crisis situations. CE-DAT serves as 
SMART's primary data source for trend analysis and monitoring and 
reporting. 

[59] Mortality surveillance systems require teams of trained home 
visitors to record deaths in the population as they occur. The home 
visitors are assigned a specific sector of the camp or village and 
instructed to visit their assigned areas regularly. During their 
visits, they record the population size of the area and all deaths that 
have occurred. 

[60] See World Health Organization, pp. 2-3, and Bradley Woodruff, 
"Violence and Mortality in West Darfur," The Lancet, vol. 364 (2004), 
p. 1,290. 

[61] See Bradley Woodruff. "Interpreting mortality data in humanitarian 
emergencies," The Lancet, vol. 367 (2006), p. 9. Also see Romesh Silva 
and Patrick Ball, "The Demography of Conflict-Related Mortality in 
Timor-Leste (1974-1999): Empirical Quantitative Measurement of Civilian 
Killings, Disappearances and Famine-related Deaths," in Human Rights 
and Statistical Objectivity, J. Asher, D. Banks, F. Schueren, eds. New 
York: Springer (forthcoming). Also see Francesco Checchi and Les 
Roberts, Interpreting and Using Mortality Data in Humanitarian 
Emergencies: A Primer for Non-Epidemiologists, Network Paper no. 52 
(London: Humanitarian Practice Network, 2005). 

[62] Estimates of the affected population reported in the UN 
Humanitarian Profiles are based on data provided by international 
humanitarian agencies and their implementing partners; the majority of 
information comes from the UN World Food Program food registration. 
These estimates do not include residents in the three state capitals of 
Darfur, Nyala, El Fasher, and Geneina because their number is 
relatively large compared with the IDP population they are hosting. 
Officials from the UN Office for the Coordination of Humanitarian 
Affairs told us that they had had difficulty verifying the reliability 
of data reported and insuring the uniformity of such data. 

[63] The Darfur death estimates used different baseline mortality 
rates. For example, CRED used a baseline of 0.3 because UNICEF reported 
that as a national average prior to the conflict. State used 0.5 
because that was the WHO baseline mortality rate for sub-Saharan 
Africa. 

[64] In addition, some experts felt that it was appropriate to use two 
rates, one for the "normal" period and one for the "emergency" period 
of the crisis. For example, one expert proposed 0.3 for the normal 
period and 0.6 for the emergency period. 

[65] The remaining experts either presented views for or against or did 
not comment on this issue. An expert who made this argument also noted 
that the countries with higher rates of normal mortality would 
attribute relatively more deaths to "normal" factors than in countries 
with lower mortality rates. 

[66] The experts suggested that the U.S. agencies best able to 
implement these suggestions are the CDC, USAID, and State. Other, 
nongovernment entities include academic institutions, such as the Johns 
Hopkins University; multilateral organizations, such as the WHO; and 
NGOs such as Médecins Sans Frontières. 

[67] Experts rated the measures using the following 5-point scale: 
"Very greatly improve," "Greatly improve," Moderately Improve," 
"Somewhat Improve," and "Not improve." 

[68] One expert felt that it would be preferable for the U.S. 
government to support the existing technical teams. 

[69] Cluster sampling is a simple method that can be used to draw a 
representative sample even where there is no listing, or known total 
number, of households (that is, no individual household sampling frame) 
and where households are arranged in a chaotic pattern, as if often the 
case in IDP camps. See Francesco Checchi and Les Roberts, Interpreting 
and Using Mortality Data in Humanitarian Emergencies: A Primer for Non- 
Epidemiologists, Network Paper no. 52 (London: Humanitarian Practice 
Network, 2005). 

[70] SMART initiative is aimed at improving the monitoring, reporting, 
and evaluation of humanitarian assistance interventions. (See 
www.smartindicators.org.) 

[71] For example, members from the International Emergency and Refugee 
Health Branch provided technical training on data collection to the 
World Food Program for two mortality surveys conducted in Darfur. 

[72] As noted in the report, we and the group of experts also reviewed 
a preliminary death estimate for West Darfur by John Hagan and Alberto 
Palloni, but we do not discuss the estimate in this report because the 
estimate had not been published. The authors told us that the version 
of the estimate they gave us was preliminary and that they were working 
on a more refined version of the estimate using different methods. 

[73] Over half of our experts were also authors or coauthors on some of 
the articles we reviewed. 

[74] The experts also rated an additional measure for the Darfur crisis 
regarding whether an independent agency should conduct a retrospective 
assessment. 

[75] As noted in the report, the experts reviewed a death estimate of 
West Darfur by John Hagan and Alberto Palloni, but this estimate is not 
discussed in this report because it was not publicly available. 

[76] Survey included in CRED December 2005 update. Note: the report 
included here was only publicly available in HTML format. Therefore, 
some of the appendixes and formatting did not come through. 

[77] This figure is from an August 2005 estimate that was the latest 
available at the time the group of experts convened, and this was 
reviewed. At the end of April 2006, Eric Reeves provided a new estimate 
of over 450,000 people have died from violence, disease, and 
malnutrition. 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts 
newly released reports, testimony, and correspondence on its Web site. 
To have GAO e-mail you a list of newly posted products every afternoon, 
go to www.gao.gov and select "Subscribe to Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 441 G Street NW, Room LM 
Washington, D.C. 20548: 

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 
512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. 
Government Accountability Office, 441 G Street NW, Room 7125 
Washington, D.C. 20548: 

Public Affairs: 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, D.C. 20548: