This is the accessible text file for GAO report number GAO-06-758 
entitled 'No Child Left Behind Act: Education Actions Needed to Improve 
Local Implementation and State Evaluation of Supplemental Educational 
Services' which was released on August 4, 2006. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Requesters: 

United States Government Accountability Office: 

GAO: 

August 2006: 

No Child Left Behind Act: 

Education Actions Needed to Improve Local Implementation and State 
Evaluation of Supplemental Educational Services: 

No Child Left Behind Act: 

GAO-06-758: 

GAO Highlights: 

Highlights of GAO-06-758, a report to congressional requesters 

Why GAO Did This Study: 

The No Child Left Behind Act of 2001 (NCLBA) requires districts with 
schools that have not met state performance goals for 3 consecutive 
years to offer their low-income students supplemental educational 
services (SES), such as tutoring, if these schools receive Title I 
funds. SES are provided outside of the regular school day by a state-
approved provider, with responsibility for implementation shared by 
states and districts. GAO examined (1) how SES participation changed 
between school years 2003-2004 and 2004-2005; (2) how SES providers are 
working with districts to deliver SES; (3) how states are monitoring 
and evaluating SES; and (4) how the Department of Education (Education) 
monitors and supports state implementation of SES. To collect data on 
SES, GAO surveyed all states and a nationally representative sample of 
districts with schools required to offer SES. We also visited 4 school 
districts, interviewed 22 SES providers, reviewed SES-related research, 
and interviewed Education staff. 

What GAO Found: 

SES participation among eligible students increased from 12 to 19 
percent between school years 2003-2004 and 2004-2005, and the number of 
recipients also increased, due in part to a rise in the number of 
schools required to offer services. Districts have used some promising 
practices to inform parents and encourage participation, such as 
offering services on school campuses and at various times. However, 
challenges remain, including timely and effective notification of 
parents and attracting providers to serve certain areas and students, 
such as rural districts or students with disabilities. To promote 
improved student academic achievement, SES providers took steps to 
align their curriculum with district instruction and communicate with 
teachers and parents, though the extent of their efforts varied. A 
majority of the 22 providers we interviewed worked to align SES and 
district curriculum by hiring teachers familiar with the district 
curriculum as tutors. However, at least some providers did not have any 
contact with teachers in about 40 percent of districts. Both providers 
and district officials experienced challenges related to contracting 
and coordination of service delivery. Providers, districts, and schools 
reported that greater involvement of schools would improve SES delivery 
and coordination, as it has in some places where this is occurring. 
While state monitoring of district and provider efforts to implement 
SES has been limited in past years, more states reported conducting on-
site reviews and other monitoring activities during 2005-2006. In 
addition, districts have taken a direct role in monitoring providers, 
and their monitoring efforts have similarly increased. Although states 
are required to withdraw approval from providers that fail to increase 
student academic achievement for 2 years, many states struggle to 
develop meaningful SES evaluations. While a few states have completed 
evaluations, none provides a conclusive assessment of SES providers’ 
effect on student academic achievement. Several Education offices 
monitor SES activity across the country and provide SES support to 
states and districts through written guidance, grants, and technical 
assistance. However, states and districts reported needing additional 
SES evaluation support and technical assistance. For example, 85 
percent of states reported needing assistance with methods for 
evaluating SES. Many also voiced interest in Education’s pilot programs 
that increase SES flexibility, including the one that allowed certain 
districts identified as in need of improvement to act as providers. 

Figure: SES Is Often Delivered after School and on School Campuses: 

[See PDF for Image] 

Source: GAO. 

[End of Figure] 

What GAO Recommends: 

GAO recommends that Education disseminate information on promising 
practices used to improve SES implementation, provide states with 
technical assistance to improve evaluation of SES’s effect on student 
achievement, and expand program flexibility where appropriate. 
Education generally supported GAO’s recommendations. 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-758]. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Marnie Shaul at (202) 512-
7215 or shaulm@gao.gov. 

[End of Section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

SES Participation Has Increased as Districts Have Taken Steps to Ease 
Access, but Challenges Remain: 

Providers Have Taken Steps to Deliver Quality Services, but Contracting 
and Coordination Remain Challenges to Working with Districts and 
Schools: 

States' SES Monitoring Has Been Limited Though Their Efforts Are 
Increasing, and Many States Struggle to Develop Meaningful Evaluations: 

Several Education Offices Monitor and Support SES Implementation, but 
States and Districts Cite the Need for Additional Assistance and 
Flexibility: 

Conclusions: 

Recommendations: 

Agency Comments: 

Appendix I: Scope and Methodology: 

Appendix II: Comments from the Department of Education: 

Appendix III: GAO Contacts and Staff Acknowledgments: 

Related GAO Products: 

Tables: 

Table 1: NCLBA Interventions for Schools Not Meeting Yearly Performance 
Goals over Time: 

Table 2: SES Stakeholder Roles and Responsibilities: 

Table 3: District Actions Taken to Encourage SES Participation (2005- 
2006): 

Table 4: State Actions Taken to Encourage SES Participation (2005- 
2006): 

Table 5: Percentage of States and Districts That Reviewed Specified 
Program Elements to Monitor Providers in 2005-2006: 

Table 6: Percentage of States and Districts in Need of Different Types 
of Information or Assistance with SES Implementation: 

Table 7: State and District Opinion on the Ordering of School Choice 
and SES: 

Table 8: Description of the Population and Sample of Districts: 

Table 9: Sampling Error Calculations: 

Table 10: SES District Survey Response Rates: 

Figures: 

Figure 1: Number of Students Receiving SES Nationwide (2002-2003 to 
2004-2005): 

Figure 2: School Districts Required to Offer SES in 2004-2005: 

Figure 3: Estimated Percentage of Districts Experiencing Certain Issues 
with None, Few, Some, Most, or All of Their Providers in 2004-2005: 

Figure 4: Percentage of States That Reviewed Specific Program Elements 
to Monitor District SES Implementation in 2005-2006: 

Figure 5: Percentage of States Reviewing Information Collected from 
Different Sources to Monitor SES Providers in 2004-2005 and 2005-2006: 

Figure 6: Estimated Percentage of Districts Reviewing Information 
Collected from Different Sources to Monitor SES Providers in 2004-2005 
and 2005-2006: 

Figure 7: U.S. Department of Education Offices Monitoring and 
Supporting SES: 

Abbreviations: 

AYP: adequate yearly progress: 
CSPR: Consolidated State Performance Report: 
ESEA: Elementary and Secondary Education Act: 
NCLBA: No Child Left Behind Act: 
NSLP: National School Lunch Program: 
OESE: Office of Elementary and Secondary Education: 
OIG: Office of the Inspector General: 
OII: Office of Innovation and Improvement: 
SES: supplemental educational services: 
SESQC: Supplemental Educational Services Quality Center: 

United States Government Accountability Office: 
Washington, DC 20548: 

August 4, 2006: 

Congressional Requesters: 

Title I of the No Child Left Behind Act (NCLBA) supports the academic 
achievement of 16.5 million low-income students nationwide by providing 
them with additional opportunities, such as supplemental educational 
services (SES), if their schools are low performing. SES includes 
tutoring and remediation that are provided outside of the regular 
school day by a state-approved provider, such as a for-profit company 
or a community-based organization. When a Title I school does not meet 
state performance goals for 2 years, including goals set under the 
Elementary and Secondary Education Act (ESEA) before the enactment of 
NCLBA, the district must offer students the choice of transferring to 
another school in the district that is not in improvement status. When 
a Title I school does not meet state performance goals for 3 or more 
years, the district must offer SES to all of the low-income students 
enrolled in the school. Districts with schools required to offer school 
choice and SES must set aside an amount equal to 20 percent of their 
Title I funds to provide choice-related transportation and SES for 
eligible students in these schools. In school year 2005-2006, $12.7 
billion in federal Title I funding was distributed to nearly all school 
districts and approximately half of the public schools nationwide in 
order to improve the education of low-income students. 

While states set NCLBA performance goals and schools are judged on the 
performance of their students, responsibility for SES implementation is 
primarily shared by states and districts under the law. States are 
responsible for approving SES providers to serve students in their 
states and, therefore, review provider applications to assess each 
provider's record of effectiveness and program design, including its 
instructional strategies and service costs. States are also responsible 
for monitoring SES providers and evaluating their services. Districts 
are responsible for notifying parents of their child's eligibility for 
SES and contracting with the state-approved providers parents select 
for services. At the federal level, the U.S. Department of Education 
(Education) oversees SES implementation nationwide and provides 
guidance and technical assistance. 

Although some districts were first required to offer SES in 2002-2003, 
others did not have to offer SES until 2003-2004 or the year after, and 
therefore, states and districts are at different stages of implementing 
the SES provisions. Further, research suggests that many face 
challenges to increasing student participation, working with providers 
to ensure students receive quality services, and monitoring and 
evaluating these services. In response to congressional interest in 
these and other issues related to early implementation of SES, and in 
anticipation of reauthorization of the ESEA, we agreed to answer the 
following questions: (1) How has SES participation changed between 
school years 2003-2004 and 2004-2005, and what actions have been taken 
to increase participation? (2) How are providers working with districts 
and schools to provide services that increase student achievement? (3) 
To what extent are states monitoring and evaluating SES implementation? 
(4) How does Education monitor state SES implementation and assist 
state and district efforts? 

To answer our four research questions, we collected data through 
several methods. First, to gather state-level information on SES, we 
collected information through a Web-based survey of state SES 
coordinators in all 50 states, the District of Columbia, and Puerto 
Rico. Further, to gather district-level information on SES, we 
conducted a mail survey of district SES coordinators from a nationally 
representative sample of districts with schools required to offer SES. 
To ensure that we obtained information from the largest districts with 
schools required to offer SES, we included all 21 districts with 
100,000 or more enrolled students in our sample. In designing our 
surveys, we reviewed survey questions used by Education. The district 
and state surveys were both administered between January and March 
2006. Seventy-seven percent of district SES coordinators, including all 
coordinators from districts with 100,000 or more enrolled students, and 
all state SES coordinators responded to the surveys. All percentage 
estimates presented from the district survey have margins of error of 
plus or minus 8 percentage points, unless otherwise noted. While we did 
not validate specific information that states and districts reported 
through our surveys, we reviewed the information to determine that 
their responses were complete and reasonable and found the information 
to be sufficiently reliable for the purposes of this report. To gather 
federal-level information on SES implementation, we interviewed 
Education officials and analyzed Education's data on SES available from 
state reports and ongoing studies. 

To collect additional information from states and districts, as well as 
information from schools and providers, we conducted site visits to 
four school districts that had experience with SES implementation 
(Woodburn, Ore; Newark, N.J; Chicago, Ill; and Hamilton County, Tenn.) 
During the visits, we spoke with state and district officials, as well 
as officials from a total of 12 schools and 15 providers. When viewed 
as a group, the site visit districts provided variation across 
characteristics such as geographic location, district size, student 
ethnicity, and the percentage of students with limited English 
proficiency or disabilities. In addition to the 15 providers we spoke 
with during the site visits, we also interviewed 7 for-profit SES 
providers operating in multiple states, for a total of 22 providers. In 
our surveys and other data collection efforts, we asked questions about 
SES implementation during specific school years, and, therefore, all 
years cited refer to school years. 

See appendix I for detailed information on both of our surveys as well 
as our other data collection methods. We conducted our work from August 
2005 through July 2006 in accordance with generally accepted government 
auditing standards. 

Results in Brief: 

Nationally, SES participation increased from 12 percent of eligible 
students receiving services in 2003-2004 to 19 percent in 2004-2005, 
but challenges to increasing participation remain. The total number of 
students receiving services rose from approximately 117,000 in 2002- 
2003 to 430,000 in 2004-2005, due in part to a rise in the number of 
schools required to offer services. An estimated 20 percent of 
districts required to offer SES had no students receiving services, and 
the majority of those districts were rural or had less than 2,500 
enrolled students. While these districts, therefore, spent no funds for 
SES, some other districts that were providing services reported needing 
more than the 20 percent Title I set-aside to serve all eligible 
students that had requested services. While a student's eligibility for 
SES is based on income and not academic achievement, most students 
receiving services were among the lower achieving students in school. 
Districts have taken multiple actions to encourage participation, such 
as using several methods to contact parents and offering services on 
school campuses and at various times. For example, 90 percent or more 
of districts provided parents with written information, held individual 
meetings with parents, encouraged school staff to talk with parents 
about SES, and offered services in locations easily accessible to 
students, such as on or near the school campus. Although some districts 
have used promising practices to increase SES participation, notifying 
parents in a timely and effective way continues to be a challenge for 
districts. About half of the districts did not notify parents before 
the beginning of the current school year, due in part to delays in 
receipt of school improvement results from their state. Further, 
several providers indicated that confusing parental notification 
letters and enrollment forms may discourage participation. States and 
districts have also been challenged to ensure there are providers to 
serve students from rural areas or students with limited English 
proficiency or disabilities. Encouraging student attendance has also 
been a challenge, in part because students may participate in other 
after-school activities, such as sports or work. 

Providers reported that they are working with districts and schools to 
deliver quality services, but contracting with districts and 
coordinating with schools remain challenges, and all parties reported 
that implementation would be improved with greater school involvement. 
To help improve student academic achievement, providers align their 
curriculum with district instruction primarily by hiring teachers 
familiar with the district curriculum and communicating with the 
teachers of participating students. Providers reported communicating 
with teachers in person as well as mailing information and progress 
reports to them, which are similar to the methods reported for 
communicating with parents. However, we estimate that some, most, or 
all providers did not contact teachers in about 40 percent of districts 
and did not contact parents in about 30 percent of districts during the 
2004-2005 school year. In addition, contracting for services was 
sometimes a challenge for providers and districts. Some of the 
providers we interviewed said certain districts impose burdensome 
contract requirements, such as limiting provider marketing and use of 
school facilities. About 40 percent of districts also considered 
contracting to be a challenge, and officials in three of the four 
districts we visited expressed concerns about their lack of authority 
under the law to set parameters around costs and program design, such 
as the number of service hours per student and the tutor-to-student 
ratio. While officials from one state we visited expressed interest in 
additional federal guidance on setting such parameters, another state 
helped to address some of its district's concerns and improve 
transparency and accountability by requiring approved providers to 
submit cost information and sharing this information with districts. 
About 70 percent of the states also reported that coordination among 
providers, districts, and schools is a moderate to very great 
challenge. For example, services were delayed or withdrawn in three of 
the four districts we visited because not enough students signed up to 
meet the providers' enrollment targets and districts were not aware of 
these targets. Because SES may be delivered in school facilities, 
providers and officials in the districts and schools we visited 
reported that involvement of school administrators and teachers can 
ease SES delivery and coordination with providers. 

Prior to 2005-2006, state SES monitoring of both districts and 
providers was limited, and over the last few years, many states have 
struggled to develop meaningful SES evaluations. In regard to 
monitoring districts, more states reported conducting on-site reviews 
in 2005-2006, and many states also collected information from other 
sources, such as school principals and parents. Further, nearly all 
states were collecting information on districts' expenditures for SES 
and their efforts to notify parents of available services. In regard to 
monitoring providers, though states were challenged by their oversight 
capacity, many states reported using a more active monitoring approach 
in 2005-2006. For example, approximately three-fourths indicated they 
were conducting on-site reviews, and many were also collecting 
information from districts, schools, and parents to monitor SES 
providers in that year. In addition to states, districts have also 
taken an active role in monitoring providers, and similarly increased 
their efforts in 2005-2006. While states are required to withdraw 
approval from providers that fail to increase student academic 
achievement for 2 years, many are struggling to develop meaningful SES 
evaluations that assess this outcome. Approximately three-fourths of 
states reported that they are experiencing challenges evaluating SES, 
including designing methods for determining sufficient academic 
progress of students, having the time and knowledge to analyze SES 
data, and developing data systems to track SES information. For 
example, during our site visits to Illinois and New Jersey, state 
officials noted that they were currently in the process of improving 
their data collection systems to more effectively capture and analyze 
data for SES evaluations. At the time of our survey, only a few states 
had drafted or completed an evaluation report addressing SES providers' 
effect on student academic achievement, and no state had produced a 
report that provided a conclusive assessment of this effect. In 
addition, over half of the states reported that they were in the 
process of conducting an SES evaluation in order to meet the federal 
requirement. 

Several Education offices monitor SES and support implementation 
through written guidance and technical assistance grants, but states 
and districts cite the need for additional assistance and flexibility. 
Two Education offices are primarily responsible for SES monitoring. The 
Office of Innovation and Improvement (OII) monitors SES implementation 
by reviewing SES-related reports and responding to state, district, and 
provider concerns brought to its attention, while the Office of 
Elementary and Secondary Education (OESE) visits states and districts 
as part of its overall Title I monitoring. The Office of the Inspector 
General (OIG) also conducts monitoring through on-site SES audits. 
Education shares monitoring results internally and has used them to 
help states and districts address implementation challenges. For 
example, OII has updated the SES non-regulatory guidance several times 
since 2002, and OESE has recommended actions needed to improve state 
compliance with federal regulations. Education also issues grants that 
provide SES implementation support. For example, OESE recently funded 
the Center on Innovation and Improvement, which provides technical 
assistance to regional centers to help them work with states to improve 
Education's programs, including SES. Despite Education's efforts, many 
states and districts reported that they need additional information or 
assistance with certain aspects of SES implementation to better comply 
with SES requirements. For example, 85 percent of states and an 
estimated 70 percent of districts needed assistance with methods for 
evaluating SES. Further, states and districts both indicated needing 
additional capacity to implement SES. Officials also reported needing 
additional or clearer guidance from Education on certain SES 
provisions, such as crafting a notification letter that is both 
complete and easy for parents to understand. Although Education has 
cited several states for districts' incomplete notification letters, 
the department's current SES guidance provides a sample parental 
notification letter that does not clearly specify all of the required 
elements. Many states also reported needing a forum to share promising 
practices with peers, such as meetings and conferences. In addition, 
many states and districts expressed interest in expansion of 
Education's pilot programs. For example, some officials suggested 
expanding Education's pilot that enables low-achieving districts 
typically restricted from being SES providers to provide these services 
on the condition that they evaluate the effect of their services on 
student achievement. Officials noted this may ease access to SES and 
potentially reduce costs. 

To help states and districts implement SES more effectively, we are 
recommending that Education collect and disseminate information on 
promising practices used by districts, including examples of how 
districts have successfully involved school officials in SES 
implementation, and sample parental notification letters that meet 
federal requirements and are easy for parents to understand. Further, 
to improve states' and districts' ability to provide services to the 
maximum number of students, we are recommending that Education consider 
expanding its current SES pilot program allowing selected districts in 
need of improvement to serve as providers and clarify state authority 
to set parameters around service design and costs. To improve federal 
and state monitoring of SES, we are recommending that Education require 
states to collect and submit information on the amount and percent of 
Title I funds spent on SES by districts and provide states with 
technical assistance and additional guidance on how to evaluate the 
effect of SES on student academic achievement. 

In written comments on a draft of this report, Education supported our 
recommendations and cited actions the department has already initiated 
or plans to take in addressing them. Education also provided technical 
comments, which were incorporated into the report as appropriate. 

Background: 

Under NCLBA, SES primarily includes tutoring provided outside of the 
regular school day that is designed to increase the academic 
achievement of economically disadvantaged students in low-performing 
Title I schools. These services must consist of high-quality, research- 
based instruction that aligns with state educational standards and 
district curriculum. 

Title I: 

Title I[Footnote 1] of ESEA, as amended and reauthorized by NCLBA, 
authorizes federal funds to help elementary and secondary schools 
establish and maintain programs that will improve the educational 
opportunities of economically disadvantaged children. Title I is the 
largest federal program supporting education in kindergarten through 
12th grade, supplying $12.7 billion in federal funds in fiscal year 
2006. According to Education, during the 2005-06 school year, nearly 
all U.S. school districts and approximately half of public schools 
received some Title I funding. In addition, the latest national data 
available from Education counted 16.5 million students as Title I 
participants in the 2002-2003 school year. 

Title I funds are distributed by formula to state education agencies, 
which retain a share for administration and school improvement 
activities before passing most of the funds on to school districts. 
Districts are required to distribute Title I funds first to schools 
with poverty rates over 75 percent, with any remaining funds 
distributed at their discretion to schools in rank order of poverty 
either districtwide or within grade spans. A school's Title I status 
can change from year to year because school enrollment numbers and 
demographics may vary over time. 

NCLBA: 

Enactment of NCLBA strengthened accountability by requiring states and 
schools to improve the academic performance of their students so that 
all students are proficient in reading and math by 2014. Under NCLBA, 
each state creates its own content standards, academic achievement 
tests, and proficiency levels. In 2005-2006, states were required to 
test all children for reading and mathematics achievement annually in 
grades 3-8 and once in high school to determine whether schools are 
making adequate yearly progress (AYP).[Footnote 2] 

In addition to meeting the state's performance goals by grade, subject, 
and overall student population, schools are responsible for meeting 
those goals for designated groups. These groups are students who (1) 
are economically disadvantaged, (2) are part of a racial or ethnic 
group that represents a significant proportion of a school's student 
population, (3) have disabilities, or (4) have limited English 
proficiency. To make AYP, each school must also show that each of these 
groups met the state proficiency goals for both reading and math. In 
addition, schools must show that at least 95 percent of students in 
grades required to take the test have done so. Schools must also 
demonstrate that they have met state targets for at least one other 
academic indicator, including graduation rate in high schools and a 
state-selected measure in elementary or middle schools.[Footnote 3] 

For Title I schools that do not meet state AYP goals, NCLBA requires 
the implementation of specific interventions, and these interventions 
must continue until the school has met AYP for 2 consecutive years. 
Table 1 outlines the interventions applied after each year a Title I 
school misses state performance goals. At their discretion, states may 
also implement interventions for public schools that do not receive 
Title I funds and do not make AYP. Although districts are not required 
to offer SES until a Title I school has missed performance goals for 3 
years, because some schools had not met state goals set under ESEA 
before the enactment of NCLBA, some Title I schools were first required 
to offer SES in 2002-2003, the first year of NCLBA implementation. 

Table 1: NCLBA Interventions for Schools Not Meeting Yearly Performance 
Goals over Time: 

Number of years school misses performance goals: First year missed; 
School status in the next year: N/A; 
NCLBA interventions for Title I schools: None. 

Number of years school misses performance goals: Second year missed; 
School status in the next year: Needs Improvement - First Year; 
NCLBA interventions for Title I schools: Required to offer school 
choice. 

Number of years school misses performance goals: Third year missed; 
School status in the next year: Needs Improvement - Second Year; 
NCLBA interventions for Title I schools: Required to offer school 
choice and SES. 

Number of years school misses performance goals: Fourth year missed; 
School status in the next year: Corrective Action[A]; 
NCLBA interventions for Title I schools: Required to offer school 
choice and SES. 

Number of years school misses performance goals: Fifth year missed; 
School status in the next year: Planning for Restructuring[B]; 
NCLBA interventions for Title I schools: Required to offer school 
choice and SES. 

Number of years school misses performance goals: Sixth year missed; 
School status in the next year: Implementation of Restructuring; 
NCLBA interventions for Title I schools: Required to offer school 
choice and SES. 

Source: GAO analysis of NCLBA. 

Note: N/A = not applicable. 

[A] Corrective action is a significant intervention in a school that is 
designed to remedy the school's persistent inability to make adequate 
progress toward all students becoming proficient in reading and 
mathematics. 

[B] Restructuring is a major reorganization of a school, involving 
fundamental reforms, such as significant changes in the school's 
staffing and governance. For example, some schools may be converted to 
charter schools during restructuring. 

[End of table] 

States are also required to establish and implement AYP standards for 
school districts based on the performance of all of the schools in the 
district. If districts fail to meet these standards for 2 consecutive 
years, states may classify districts as needing improvement. A district 
identified for improvement must develop and implement an improvement 
plan and remain in this status until it meets AYP standards for 2 
consecutive years. If a district remains in improvement status for 2 or 
more years, it may be identified for corrective action as deemed 
necessary by the state. 

SES under NCLBA: 

Students are eligible for SES if they attend Title I schools that have 
missed AYP for 3 consecutive years and are from low-income families. 
School districts must determine family income on the same basis they 
use to make allocations to schools under Title I, for which many have 
historically used National School Lunch Program (NSLP) data. The NSLP 
is a federally funded program that annually collects family income data 
from students' parents to determine student eligibility for free and 
reduced-priced lunch.[Footnote 4] A student's state assessment scores, 
grades, and other academic achievement information are generally not 
considered when determining SES eligibility. However, if sufficient 
funds are not available to provide SES to all eligible children, school 
districts must give priority to the lowest-achieving eligible students. 

SES providers may include nonprofit entities, for-profit entities, 
school districts, public schools, public charter schools, private 
schools, public or private institutions of higher education, 
educational service agencies, and faith-based organizations. Under the 
Title I regulations that govern SES, a district identified as in need 
of improvement or corrective action may not be an SES provider, though 
its schools that are not identified as needing improvement may. In 
addition, individual teachers who work in a school or district 
identified as in need of improvement may be hired by any state-approved 
provider to serve as a tutor in its program. 

A district must set aside an amount equal to 20 percent of its Title I 
allocation to fund both SES and transportation for students who elect 
to attend other schools under school choice. This set-aside cannot be 
spent on administrative costs for these activities, and the district 
may reallocate any unused set-aside funds to other Title I activities 
after ensuring all eligible students have had adequate time to opt to 
transfer to another school or apply for SES. Funding available for SES 
is, therefore, somewhat dependent on costs for choice-related 
transportation, though as we found in our 2004 report on NCLBA's school 
choice provisions, few students are participating in the school choice 
option.[Footnote 5] If a district does not incur any choice-related 
transportation costs, it must use the full 20 percent set-aside amount 
to pay for SES if sufficient demand for services exists. In addition, 
if the Title I set-aside is not sufficient to fund SES for interested 
students, both states and districts may direct other funds for these 
services at their discretion. For each student receiving SES, a 
district must spend an amount equal to its Title I per-pupil allocation 
or the actual cost of provider services, whichever is less.[Footnote 6] 

Education oversees SES implementation by monitoring states and 
providing technical assistance and support. OII leads SES policy 
development and coordinates the publication of SES guidance, and OESE 
oversees and monitors Title I, including SES. NCLBA and the Title I 
regulations and SES guidance outline the roles and responsibilities 
states, school districts, parents, and service providers have in 
ensuring that eligible students receive additional academic assistance 
through SES (see table 2). 

Table 2: SES Stakeholder Roles and Responsibilities: 

Stakeholder: State; 
Roles and responsibilities: Set criteria and standards for approving 
providers; 
Identify, approve, and maintain public list of providers; 
Ensure that the list of approved providers includes organizations that 
are able to serve students with disabilities and limited English 
proficiency; 
Monitor and evaluate the effectiveness of provider services; 
Monitor district SES implementation; 
Develop and use policy criteria for withdrawing providers from state-
approved list, including if; 
* provider fails for 2 consecutive years to increase student 
proficiency relative to state academic content and achievement 
standards; 
* provider fails to adhere to applicable health, safety, and civil 
rights requirements. 

Stakeholder: School district; 
Roles and responsibilities: Provide an annual notice to parents, which 
must identify available providers, describe the enrollment process and 
timeline, describe the services, qualifications, and demonstrated 
effectiveness of each provider, and be easily understandable; 
Help parents choose a provider, if requested; 
Protect the privacy of students eligible for and receiving services; 
Calculate and establish the SES per pupil allocation if not determined 
by the state; 
Determine which students should receive services if more students apply 
for SES than can be served with available funds; 
Enter into contracts with providers; 
Ensure eligible students with disabilities and eligible students with 
limited English proficiency may participate in SES; 
At the discretion of the state, may be involved in collecting data from 
providers to assist state monitoring and evaluation activities. 

Stakeholder: Providers; 
Roles and responsibilities: Provide services in accordance with 
district agreements; 
Enable students to attain their individual achievement goals; 
Measure student progress and inform parents and teachers of progress 
made by students; 
Ensure non- disclosure of student data to the public; 
Provide services consistent with applicable health, safety, and civil 
rights laws; 
Provide services that are secular, neutral, and non-ideological. 

Stakeholder: Parents; 
Roles and responsibilities: Choose a provider from the state-approved 
list; 
Are encouraged to be actively involved in their child's SES program. 

Source: GAO, per P.L.107-110, 34 C.F.R. Part 200, or the U.S. 
Department of Education, Supplemental Educational Services Non- 
Regulatory Guidance, June 2005. 

[End of table] 

During the 2005-2006 school year, Education announced the 
implementation of two pilot programs intended to increase the number of 
eligible students receiving SES and generate additional information 
about the effectiveness of SES on students' academic achievement. In 
the first, Education permitted four districts in Virginia to offer SES 
instead of school choice in schools that are in their first year of 
needs improvement. In the second, Education entered into flexibility 
agreements with the Boston and Chicago school districts, enabling them 
to act as SES providers while in improvement status. OII and OESE 
coordinated implementation of the pilots for the department. Both 
pilots were subject to review at the end of the 2005-2006 school year, 
at which time Education planned to evaluate their effect on student 
academic achievement. 

SES Participation Has Increased as Districts Have Taken Steps to Ease 
Access, but Challenges Remain: 

SES participation increased between 2003-2004 and 2004-2005, and most 
students receiving services were among the lower achieving students in 
school. Districts have taken multiple actions to encourage 
participation, such as offering services on or near the school campus 
or at various times. Despite these efforts, challenges to increasing 
participation remain, including notifying parents in a timely and 
effective manner, ensuring there are providers to serve certain areas 
and students, and encouraging student attendance. 

Participation among Eligible Students Increased from 12 to 19 Percent 
between 2003-2004 and 2004-2005, and Most Participants Were Low 
Achieving: 

Nationally, the participation rate increased substantially from 12 
percent of eligible students receiving SES in 2003-2004 to 19 percent 
in 2004-2005.[Footnote 7] In addition, the number of students receiving 
services almost quadrupled between 2002-2003 and 2004-2005 from 
approximately 117,000 to 430,000 students nationwide, based on the best 
available national data (see fig. 1). This increase may be due in part 
to the increase in the number of schools required to offer SES over 
that time period. Specifically, between 2004-2005 and 2005-2006 the 
number of schools required to offer SES increased from an estimated 
4,509 to 6,584.[Footnote 8] 

Figure 1: Number of Students Receiving SES Nationwide (2002-2003 to 
2004-2005): 

[See PDF for image] 

Source: Education's NCLBA Consolidated State Performance Reports and 
GAP state survey. 

Note: Certain states did not submit SES recipient information to 
Education through their NCLBA Consolidated State Performance Reports 
for all years. Specifically, 2002-2003 data from Kansas and North 
Dakota, 2003-2004 data from Pennsylvania, and 2004-2005 data from New 
Jersey are not included in the figure. In addition, 2002-2003 data from 
New York only include information from New York City.  

[End of figure] 

Although nationally SES participation is increasing, some districts 
required to offer SES have no students receiving services. 
Specifically, we estimate that no students received services in about 
20 percent of the approximately 1,000 districts required to offer SES 
in 2004-2005. A majority of these districts were rural or had a total 
enrollment of fewer than 2,500 students. Our survey did not provide 
sufficient information to explain why these districts had no students 
receiving services in 2004-2005; therefore, it is unclear whether their 
lack of participation was related to district SES implementation or 
other issues. 

Nationwide, we estimate that districts required to offer SES spent the 
equivalent of 5 percent of their total Title I funds for SES in 2004- 
2005 excluding administrative expenditures. Districts set aside an 
amount equal to 20 percent of their Title I funds for SES and choice- 
related transportation at the beginning of the school year, and the 
proportion of the set-aside spent on SES varied by district. 
Specifically, in 2004-2005, about 40 percent of districts spent 20 
percent or less of the set-aside to provide SES and almost one-fifth of 
districts spent over 80 percent.[Footnote 9] Nationwide, of the total 
amount districts set-aside for SES, we estimate they spent 42 percent 
on SES, excluding administrative expenditures. 

Further, an estimated 16 percent of districts reported that the 
required Title I set-aside was not sufficient to fund SES for all 
eligible students whose parents requested services. For example, during 
our site visit to Newark, N.J., district officials reported budgeting 
the entire 20 percent Title I set-aside to fund SES in 2004-2005, but 
with this amount of funding, the district was only able to fund SES for 
17 percent of the students eligible for services. In addition, 
according to Chicago, Ill., district officials, the district budgeted 
the entire 20 percent Title I set-aside to fund SES in 2005-2006, and 
because parents' demand for services significantly exceeded the amount 
of funding available, the district also allocated $5 million in local 
funds to provide SES. 

While approximately 1,000 of the over 14,000 districts nationwide were 
required to offer SES in 2004-2005, SES recipients are concentrated in 
a small group of large districts, as 56 percent of recipients attended 
school in the 21 districts required to offer SES with more than 100,000 
total enrolled students (see fig. 2). Further, states ranged from 
having 0 districts to 257 districts required to offer SES in 2004-2005, 
with most states having fewer than 10 districts required to offer SES. 
State differences in the number of districts required to offer SES may 
have resulted from differences in performance or differences in state 
proficiency standards and methods used to measure adequate yearly 
progress. 

Figure 2: School Districts Required to Offer SES in 2004-2005: 

[See PDF for image] 

Source: GAO. 

[End of figure] 

Students receiving SES in 2004-2005 shared certain characteristics, as 
districts reported that most students receiving services were among the 
lower achieving students in school. Specifically, an estimated 91 
percent of the districts that reviewed the academic records of students 
receiving SES classified most or all of the students receiving SES as 
academically low achieving.[Footnote 10] For example, Hamilton County, 
Tenn., school officials said that students receiving SES are frequently 
behind grade level in their skills and require special attention to 
increase their academic achievement. 

Further, we estimate that over half of SES recipients were elementary 
school students in the majority of districts and about 60 percent of 
schools required to offer SES in 2004-2005 were elementary 
schools.[Footnote 11] Districts varied in the percentage of students 
with limited English proficiency receiving services. In about one-third 
of districts, less than 5 percent of SES recipients were students with 
limited English proficiency; however, in about one-fifth of districts, 
over half of SES recipients were students with limited English 
proficiency. Students with disabilities made up less than 20 percent of 
students receiving services in about two-thirds of districts. Finally, 
in some districts, the majority of SES recipients were African-American 
or Hispanic. In about 40 percent of districts, over half of SES 
recipients were African-American, and in about 30 percent of districts, 
over half of SES recipients were Hispanic. Because we were unable to 
obtain comparable data on the characteristics of Title I students 
enrolled in these districts in 2004-2005, we were unable to determine 
whether certain groups of students were underserved. 

We estimate that about 2,800 providers delivered services to students 
nationwide in 2004-2005, and more providers were available to deliver 
services in the districts with the largest student 
enrollments.[Footnote 12] Specifically, about 80 percent of districts 
had between 1 and 5 providers delivering services in 2004-2005. 
However, the number of providers delivering services in the 21 
districts with more than 100,000 total enrolled students ranged from 4 
to 45, and averaged 15 providers per district in 2004-2005. 

Districts Used Several Methods to Contact Parents and Offered Services 
on School Campuses and at Various Times to Increase Participation: 

Districts have taken multiple actions to encourage participation, as 
shown in table 3. In line with the federal statutory requirement that 
districts notify parents in an understandable format of the 
availability of SES, over 90 percent of districts provided written 
information in English, held individual meetings with parents, and 
encouraged school staff to talk with parents about SES. Some districts 
collaborated with providers to notify parents. For example, during our 
site visit, Illinois state officials described a provider and district 
sharing administrative resources to increase participation, which 
involved the provider printing promotional materials and the district 
addressing and mailing the materials to parents. In addition, we 
estimate that over 70 percent of districts lengthened the period of 
time for parents to turn in SES applications, held informational events 
for parents to learn about providers, and provided written information 
to parents in languages other than English. During our site visit to 
Woodburn, Ore., district officials reported extending the time parents 
had to sign up their children for SES and hosting an event where 
providers presented their programs to parents in English and Spanish. 
Further, Newark, N.J., district officials told us during our site visit 
that the district provided transportation for parents to attend 
informational events to increase participation. Also to encourage 
participation, an estimated 90 percent of districts offered services at 
locations easily accessible to students, such as on or near the school 
campus, and almost 80 percent of districts offered services at a 
variety of times, such as before and after school or on weekends. For 
example, Hamilton County, Tenn., worked with providers to offer an 
early morning tutoring program located at the school site in addition 
to providing services after school. Providers also reported delivering 
SES on school campuses and at various times. Specifically, over three- 
fourths of the 22 providers we interviewed reported delivering services 
at the school site, although providers also offered services off-site, 
such as in the home, online, or at the provider's facility. In 
addition, providers generally delivered SES after school and some also 
offered SES at alternative times, such as before school, on weekends, 
or during the summer. Finally, about one-third of districts provided or 
arranged for transportation for participating students or worked with a 
local community partner to raise awareness of the services. For 
example, in Newark, N.J., the district worked with a local community 
organization to inform parents and students living in public housing 
and homeless shelters about SES. 

Table 3: District Actions Taken to Encourage SES Participation (2005- 
2006): 

Action taken during the 2005-2006 school year: Provided written 
information in English to parents; 
Estimated percentage of districts: 99. 

Action taken during the 2005-2006 school year: Held individual meetings 
and/or phone conversations with interested parents; 
Estimated percentage of districts: 95. 

Action taken during the 2005-2006 school year: Encouraged principals, 
teachers, or other school staff to talk with parents; 
Estimated percentage of districts: 93. 

Action taken during the 2005-2006 school year: Offered supplemental 
services in locations that are easily accessible to students after 
school (e.g., on or near the school campus); 
Estimated percentage of districts: 90. 

Action taken during the 2005-2006 school year: Offered SES at a variety 
of times (e.g., after school, weekends, summer break); 
Estimated percentage of districts: 79. 

Action taken during the 2005-2006 school year: Lengthened the period of 
time parents have to submit applications for SES; 
Estimated percentage of districts: 79. 

Action taken during the 2005-2006 school year: Held events where 
parents of eligible students can learn about providers; 
Estimated percentage of districts: 78. 

Action taken during the 2005-2006 school year: Provided written 
information in language(s) other than English about SES to parents; 
Estimated percentage of districts: 72. 

Action taken during the 2005-2006 school year: Made public 
announcements (e.g., television, billboards, newspaper ads, school 
newsletters); 
Estimated percentage of districts: 67. 

Action taken during the 2005-2006 school year: Worked with a local 
community partner to raise awareness of SES (e.g., Parent Information 
Resource Center); 
Estimated percentage of districts: 39. 

Action taken during the 2005-2006 school year: Provided or arranged for 
transportation of students receiving SES to off-site providers; 
Estimated percentage of districts: 33. 

Source: GAO. 

[End of table] 

States also reported taking actions to increase participation in 2005- 
2006, as shown in table 4. Regarding parent notification, all states 
encouraged district staff to communicate with parents about SES. In 
addition, almost 90 percent of states provided guidance to districts on 
the use of school campuses for service delivery to encourage 
participation. 

Table 4: State Actions Taken to Encourage SES Participation (2005- 
2006): 

Action taken during the 2005-2006 school year: Encouraged district 
superintendents, supplemental educational services coordinators, or 
other district staff to talk with parents about supplemental 
educational services; 
Percentage of states: 100. 

Action taken during the 2005-2006 school year: Increased the number of 
approved providers; 
Percentage of states: 94. 

Action taken during the 2005-2006 school year: Provided guidance to 
districts on the use of school sites to deliver supplemental 
educational services; 
Percentage of states: 87. 

Action taken during the 2005-2006 school year: Increased the number of 
approved providers that offer supplemental educational services online; 
Percentage of states: 73. 

Action taken during the 2005-2006 school year: Worked with a local 
community partner to raise awareness of SES (e.g., Parent Information 
Resource Center); 
Percentage of states: 40. 

Action taken during the 2005-2006 school year: Made public 
announcements (e.g., television, billboards, newspaper ads); 
Percentage of states: 23. 

Source: GAO. 

[End of table] 

Remaining Challenges Include Notifying Parents in a Timely and 
Effective Manner and Attracting More Providers for Certain Areas and 
Students: 

Despite some districts' promising approaches to encourage 
participation, notifying parents in a timely manner remains a challenge 
for some districts. An estimated 58 percent of districts did not notify 
parents that their children may be eligible to receive SES before the 
beginning of the 2005-2006 school year, which may be due in part to 
delays in states reporting which schools were identified for 
improvement.[Footnote 13] Specifically, about half of districts that 
did not notify parents before the beginning of the 2005-2006 school 
year did not receive notification from the state of the schools 
identified for improvement by that time.[Footnote 14] Moreover, 
district officials in three of the states we visited experienced delays 
in receiving school improvement information, and state officials agreed 
that providing timely information about whether schools have met state 
performance goals has been a challenge. Almost all of the districts 
that did not notify parents before the beginning of the 2005-2006 
school year did so within the first 2 months of the year.[Footnote 15] 

Effectively notifying parents is also a challenge for some districts. 
For example, officials in all four districts we visited reported 
difficulties contacting parents to inform them about SES in part 
because some families frequently move and do not always update their 
mailing address with districts. In addition, some providers we 
interviewed indicated that confusing parental notification letters do 
not effectively encourage SES participation. For example, some of the 
providers we interviewed said some districts use confusing and poorly 
written letters to inform parents of SES or send letters to parents of 
eligible children but conduct no further outreach to encourage 
participation in SES. Four of the providers we interviewed also 
indicated that complicated district enrollment processes can discourage 
participation. For example, one provider said certain districts send 
parents multiple documents to complete in order for their child to 
receive SES, such as an enrollment form to select an SES provider and a 
separate contract and learning plan. 

Another challenge to increasing SES participation is attracting more 
SES providers for certain areas. Some rural districts surveyed 
indicated that no students received services last year because of a 
lack of providers in the area.[Footnote 16] We estimate that the 
availability of transportation for students attending supplemental 
services was a moderate, great or very great challenge for about half 
of rural districts.[Footnote 17] For example, one rural district 
commented in our survey that there are no approved providers within 200 
miles of its schools. A few other rural districts commented in our 
survey that it was difficult to attract providers to their area because 
there were few students to serve or providers had trouble finding staff 
to serve as tutors. 

In addition, ensuring there are providers to serve students with 
limited English proficiency or disabilities has been a challenge for 
some districts. There were not enough providers to meet the needs of 
students with limited English proficiency in an estimated one-third of 
districts, and not enough providers to meet the needs of students with 
disabilities in an estimated one-quarter of districts. Many states also 
indicated that the number of providers available to serve these groups 
of students was inadequate. While over half of the providers we 
interviewed reported serving students with limited English proficiency 
or disabilities, some providers served these students on a limited 
basis and reported difficulties meeting their needs. For example, one 
provider reported serving few students with limited English proficiency 
and disabilities because the amount of funding the provider receives 
for SES was not sufficient to pay for specialized tutors to serve these 
students. Another provider said it was difficult to find tutors to meet 
the needs of students with limited English proficiency and its program 
was not designed for students with disabilities. Another provider said 
that it was difficult to serve students with disabilities because it 
required significantly modifying the tutoring lessons to meet their 
needs. 

Encouraging student attendance has also been a challenge, in part 
because students may participate in other afterschool activities, such 
as sports or work. Low parent and student demand for SES has been a 
challenge in about two-thirds of districts. For example, about one- 
quarter of districts reported that both competition from other 
afterschool programs and the availability of services that are engaging 
to students were challenges to implementing SES. In addition, 
providers, district and school officials in all four districts we 
visited told us that SES is competing for students with extracurricular 
and other activities. For example, a Chicago, Ill., high school 
official indicated that student attendance at SES sessions declined 
significantly as the school year progressed. To address this problem, 
providers sometimes offer students incentives for participation. For 
example, while 2 of the 22 providers we interviewed offered incentives 
for students to sign up for services, 19 providers used incentives to 
encourage student attendance, such as school supplies and gift 
certificates. 

Providers Have Taken Steps to Deliver Quality Services, but Contracting 
and Coordination Remain Challenges to Working with Districts and 
Schools: 

To promote improved student academic achievement, providers aligned 
their curriculum with district instruction primarily by hiring district 
teachers and communicating with the teachers of participating students. 
Providers reported communicating with teachers and parents in person as 
well as mailing information and progress reports to them; however, 
districts indicated the extent of provider efforts varied, as some did 
not contact teachers and parents in 2004-2005. In addition, both 
providers and districts experienced contracting and coordination 
difficulties. In part because SES is often delivered in school 
facilities, providers and officials in the districts and schools we 
visited reported that involvement of school administrators and teachers 
can improve SES delivery and coordination. 

Providers Worked to Align Curriculum and Communicate with Parents, 
though the Extent of Their Efforts Varied: 

In order to increase student academic achievement, providers took steps 
to align their curriculum with school instruction by hiring and 
communicating with teachers, though the extent of their efforts varied. 
A majority of the 22 providers we interviewed hired certified teachers 
in the district as tutors. Some providers said hiring district teachers 
promoted curriculum alignment, in part because district teachers were 
apt to draw on district curriculum during tutoring sessions. School 
officials in three of our site visits also said providers' use of 
district teachers as tutors helped to align the tutoring program with 
what the student learned in the classroom. In addition, some providers 
reported aligning curriculum by communicating with the teachers of 
participating students to identify student needs and discuss progress. 
The frequency of contact between tutors and teachers ranged from 
mailing teachers information once prior to the beginning of the program 
to contacting teachers at least weekly, according to the providers we 
interviewed. A few providers also used other methods to align their 
curriculum with district instruction, such as using the same tests to 
evaluate student progress and allowing principals to choose components 
of the tutoring curriculum for students receiving SES in their schools. 
However, not all providers worked with schools to align curriculum, as 
we estimate that some, most, or all providers did not contact teachers 
to align curriculum with school instruction in almost 40 percent of 
districts in 2004-2005. For example, Woodburn, Ore., district and 
school officials indicated during our site visit that instead of 
aligning their services with the district curriculum, certain providers 
openly questioned the district's curriculum and teaching methods, which 
caused confusion among some parents and students. 

Providers reported mailing information as well as meeting with parents 
over the phone and in-person to communicate about student needs and 
progress; 
however, the frequency of communication with parents varied by 
provider. A majority of the providers we interviewed communicated with 
parents about student progress repeatedly, sometimes by sending home 
progress reports on a monthly basis or holding parent-tutor 
conferences. The frequency of contact between tutors and parents 
reported by the 22 providers we interviewed ranged from meeting with 
parents twice during the tutoring program to giving parents a weekly 
progress report. A few providers also reported communicating with 
parents by holding workshops for parents to learn about the SES program 
and in some cases having the parents sign their students' learning 
plans. For example, one provider conducted workshops where parents 
received reading materials to share with their children and a parent 
guide in English and Spanish that explained the program and how to use 
the materials to enhance student learning. Some providers also reported 
hiring staff dedicated in part to maintaining communication with 
parents. However, some providers faced difficulties when communicating 
with parents about SES, such as language barriers or incorrect contact 
information. Districts confirmed that the degree to which providers 
communicated with parents varied, as we estimate that some, most, or 
all providers did not contact parents to discuss student needs and 
progress in about 30 percent of districts in 2004-2005. 

Despite these challenges, most districts had positive relationships 
with providers. Specifically, an estimated 90 percent of districts 
indicated that their working relationships with providers during 2004- 
2005 were good, very good, or excellent. In addition, many of the 
providers we interviewed during our site visits also reported having 
positive working relationships with district officials. Although other 
studies have found that districts reported certain difficulties working 
with providers, relatively few districts reported that their providers 
signed up ineligible students or billed for services not performed in 
2004-2005, as shown in figure 3. 

Figure 3: Estimated Percentage of Districts Experiencing Certain Issues 
with None, Few, Some, Most, or All of Their Providers in 2004-2005: 

[See PDF for image] 

Source: GAO. 

[End of figure] 

Generally, states did not hear about these provider issues very often. 
Almost half of states said the issue of providers not showing up for 
SES sessions was rarely brought to their attention.[Footnote 18] 
Similarly, half of states said the issue of providers billing the 
district for services not performed was rarely brought to their 
attention. In addition, about 40 percent of states said the issue of 
providers using excessive incentives was rarely brought to their 
attention. Further, about 40 percent of states said the issue of 
providers signing up ineligible students rarely arose. Almost one-third 
of states heard about each of these issues sometimes, while few states 
had these issues brought to their attention very often. For example, 
during our site visits, state officials provided examples of issues 
that had been brought to their attention regarding provider practices, 
but these issues were often isolated incidents particular to one or a 
few providers in certain districts. 

Providers and Districts Experienced Contracting and Coordination 
Difficulties: 

While providers have taken steps to deliver quality services, both 
providers and districts reported experiencing difficulties during the 
contracting process. For example, some of the providers we interviewed 
said certain districts imposed burdensome contract requirements, such 
as requiring substantial documentation to be submitted with invoices, 
limiting the marketing they could do to parents and students, or 
restricting the use of school facilities to deliver services. 
Specifically, 7 of the 22 providers we interviewed experienced 
difficulties with districts restricting provider access to school 
facilities, by for example, not allowing providers to deliver services 
in school buildings or by charging providers substantial fees to do so. 
A few providers also said contracting with districts was a resource- 
intensive process, in part because contract requirements vary by 
district and state. Some of the multi-state providers we interviewed 
reported dedicating a team of staff to help them finalize and manage 
contracts with districts. These providers commented that, while they 
have the administrative capacity to manage this process, smaller 
providers may not have such capacity. In addition, one provider that 
delivered services exclusively online commented that contracting with 
districts across the country was a challenge, particularly because some 
states and districts require provider representatives to attend 
meetings in-person and be fingerprinted in their states. 

Contracting with providers was also a challenge for some districts. 
Specifically, negotiating contracts with providers was a moderate, 
great, or very great challenge in about 40 percent of districts 
nationwide. For example, Woodburn, Ore., district officials described 
having contractual discussions with providers around whether the 
district would charge fees for the use of school facilities, the types 
of incentives providers could use to encourage students to sign up, and 
whether the district would pay for services when students did not 
attend SES sessions. While states may review and define program design 
parameters as part of the provider approval process, district officials 
in three of our site visits expressed concern about their lack of 
authority to set parameters in provider contracts around costs and 
program design, such as tutor-to-student ratios and total hours of 
instruction. For example, during our site visit, a Hamilton County, 
Tenn., district official expressed frustration with providers that 
charged the maximum per-pupil amount but varied in the level of 
services provided, such as the number of instructional hours and tutor- 
to-student ratio. Chicago, Ill., district officials also expressed 
concern about the variation among providers in the hours of instruction 
and cost of services because the district does not have sufficient 
funds to serve all eligible students and officials would like to 
maximize the number of students they can serve. In part to help address 
district concerns, in 2005-2006, Illinois required approved providers 
to submit information on the cost of providing services in each of the 
districts they served and made this information available to districts 
and the public in order to improve transparency and accountability. 
While Tennessee state officials told us they were hesitant to set 
restrictions on providers and would like more federal direction on this 
issue, other states have set restrictions on the cost and design of SES 
programs. For example, Georgia set a maximum tutor-to-student ratio of 
1:8 for non-computer based instruction and 1:10 for computer based 
instruction, and New Mexico adopted a sliding fee scale based on the 
educational level of tutors. 

Coordination of service delivery has also been a challenge for 
providers, districts, and schools. About 70 percent of states reported 
that the level of coordination between providers, districts, and 
schools implementing SES was a moderate to very great challenge. 
Sometimes these coordination difficulties have resulted in service 
delays. For example, services were delayed or withdrawn in three of the 
districts we visited because not enough students signed up to meet the 
providers' enrollment targets and districts were not aware of these 
targets.[Footnote 19] In one district we visited, services were delayed 
because school teachers hired to be tutors did not provide evidence of 
their background checks and teaching certificates to providers in a 
timely manner. Coordination difficulties also occurred during the 
enrollment process. Though districts are responsible for arranging SES 
for eligible students, in two districts we visited, both the district 
and providers sent parents enrollment forms, which caused confusion 
among parents as well as additional work for the district staff 
processing the forms. In addition, a few providers told us they do not 
know how many students they will serve until enrollment forms are 
returned to district officials, which hinders planning and may delay 
services since they do not know how many tutors they will need to hire 
and train to deliver SES in each district. 

Providers, District and School Officials Reported That a Greater Role 
for Schools Would Improve Local SES Implementation: 

In part because SES can be delivered in school facilities, providers 
and officials in the districts and schools we visited reported that 
involvement of school administrators and other staff improves SES 
implementation. Although schools do not have federally-defined 
responsibilities for administering SES, many officials said SES 
implementation is hindered when school officials are not involved. Some 
providers we interviewed said that a lack of involvement of school 
principals can make it difficult for them to coordinate with schools to 
encourage student participation. In addition, a few providers said 
certain districts contributed to this problem by restricting 
communication with school officials or not defining a role for schools 
in SES implementation. Officials in one of the districts we visited 
also told us that encouraging participation and administering the 
program was more difficult when they did not designate school staff to 
assist the district with SES implementation. School officials from all 
four of our site visits also said the lack of a clear role for school 
officials, including principals, in administering SES has been a 
challenge. For example, Illinois and Oregon school principals told us 
they found it difficult to manage afterschool activities because they 
didn't have sufficient authority to oversee SES tutors operating in 
their buildings at that time. Further, problems can arise when school 
officials are not fully informed about the SES program. For example, 
Woodburn, Ore., school officials told us that although the school was 
not provided SES tutoring schedules for students, parents and students 
have come to school officials for help when they were unclear about the 
schedule or when tutors failed to show up. 

A majority of the providers we interviewed told us that involvement of 
school principals can improve participation and make it easier to 
deliver services, in part because principals are familiar with the 
students and manage school staff. For example, certain providers 
reported providing principals with information about the tutoring 
program so that school staff can assist with the enrollment process, 
involving principals in selecting the curriculum used in their schools, 
and sending principals student progress reports. In addition, all four 
districts we visited had school site coordinators to assist with SES, 
such as helping with the enrollment process and assisting with the day- 
to-day administration of the SES program in the schools. For example, 
Woodburn, Ore., district officials told us implementation improved when 
the district designated school site coordinators to assist with 
parental notification and events where providers present their 
programs, and meet with parents and providers to help them complete 
individual student learning plans. A few providers we interviewed also 
assigned a staff person at the school site to communicate with teachers 
and parents. While helping to administer the SES program adds 
additional administrative burden on schools, school officials in all 
four of the districts we visited said they welcomed a stronger or more 
clearly defined role. 

States' SES Monitoring Has Been Limited Though Their Efforts Are 
Increasing, and Many States Struggle to Develop Meaningful Evaluations: 

While state monitoring of SES had been limited, more states reported 
taking steps to monitor both district and provider efforts to implement 
SES in 2005-2006. In addition, districts have taken a direct role in 
monitoring providers, and their monitoring activities similarly 
increased during this time. Although states are required to withdraw 
approval from providers that fail to increase student academic 
achievement for 2 years, many states reported challenges evaluating SES 
providers. In addition, the few states that have completed an 
evaluation have not yet produced reports that provided a conclusive 
assessment of SES providers' effect on student academic achievement. 

More States Reported Conducting On-Site Reviews of Districts in 2005- 
2006, and Many also Collected Information from Other Sources to Monitor 
District SES Implementation: 

State monitoring of district SES implementation, which is sometimes 
performed as part of state Title I monitoring, had been limited prior 
to 2005-2006, though more states reported conducting on-site reviews of 
districts in that year. While less than one-third of states conducted 
on-site reviews of districts to monitor their implementation of SES in 
2004-2005, almost three-fourths reported conducting such reviews in 
2005-2006. This increase reflects both those states that had already 
begun monitoring district SES implementation for 2005-2006 at the time 
of our survey and those states planning to conduct monitoring 
activities before the end of that school year. Because our data were 
collected during the middle of the 2005-2006 school year, we do not 
know whether all of the states that planned to complete these 
activities before the end of the year did so. In both years, a majority 
of the states that conducted on-site reviews visited few or some of 
their districts. Therefore, while more states reported conducting such 
reviews in 2005-2006 than in 2004-2005, the number of districts per 
state receiving reviews does not appear to have increased. 

In addition to on-site reviews, many states also reported reviewing 
information collected from several other sources to assess district SES 
implementation in 2005-2006. The most common source used by states was 
district officials, as almost all states reported reviewing or planning 
to review information collected from district officials to monitor 
their implementation of SES in 2005-2006. Further, many states were 
also collecting information from school principals, parents, and 
providers to monitor districts, with between 67 and 81 percent of 
states reviewing or planning to review information collected from these 
sources in 2005-2006. 

States also reported reviewing or planning to review information 
related to several aspects of district SES implementation in 2005-2006. 
For example, almost all states reported reviewing district notification 
of parents and SES expenditures, as shown in figure 4. 

Figure 4: Percentage of States That Reviewed Specific Program Elements 
to Monitor District SES Implementation in 2005-2006: 

[See PDF for image] 

Source: GAO. 

Note: The percentage of states that did not review or plan to review 
these program elements to monitor district SES implementation in 2005-
2006 and the percentage of states that did not answer these survey 
questions are not shown in this figure. 

[End of figure] 

States we visited reported that some districts have had difficulties 
implementing SES, in part because of district staff capacity 
limitations and the complexities of administering SES at the local 
level. When states find that a district is having difficulty 
implementing SES, most hold a meeting with the district and provide or 
arrange for assistance, including consultations or training. Half of 
the states also develop an action plan and time line with the district 
to help improve its efforts. During our site visits, state officials 
reported that notifying parents, maintaining a fair and competitive 
environment for providers, ensuring providers understand their SES 
roles and responsibilities, and determining an appropriate role for 
schools continue to challenge some districts as they implement SES. 

Though States and Districts Are Challenged to Monitor Providers, They 
Are Moving from Limited Monitoring of Providers to a More Active 
Approach: 

Although states and districts reported increasing their efforts to 
monitor SES providers between 2004-2005 and 2005-2006, over two-thirds 
of states reported that on-site monitoring of providers has been a 
challenge. In addition, several districts commented in our survey that 
more provider monitoring is needed. During all four of our site visits, 
state and district officials expressed concerns about their capacity to 
fully administer and oversee all required aspects of SES 
implementation, including provider monitoring. Officials explained that 
state and district capacity to implement SES is limited, because there 
is typically one staff person at each level coordinating all of SES, 
and sometimes that person may also oversee implementation of additional 
federal education programs. Further, several states commented in our 
survey that additional training, technical assistance, and national 
monitoring protocols from the federal government would assist their 
efforts to monitor providers. 

During our site visits, state officials noted that while they did not 
initially have structured plans or procedures in place to monitor SES 
providers, they took steps to develop and formalize procedures starting 
with the 2004-2005 and 2005-2006 school years. Nationally, in 2004- 
2005, states monitored providers primarily by collecting data from 
district officials, though many states reported using a more active 
monitoring approach in the next year. For example, while less than one- 
third of states conducted on-site reviews of providers in 2004-2005, 
over three-fourths had conducted or planned to conduct such reviews in 
2005-2006, as shown in figure 5. In addition, while one-third or fewer 
states reviewed information collected from school staff, parents, and 
students in 2004-2005, the percentage that reported reviewing or 
planning to review information collected from these sources more than 
doubled the next year. Similar to 2004-2005, many states continued to 
rely on information collected from district officials to monitor 
providers in 2005-2006, with almost all states reviewing or planning to 
review information collected from districts in that year. 

Figure 5: Percentage of States Reviewing Information Collected from 
Different Sources to Monitor SES Providers in 2004-2005 and 2005-2006: 

[See PDF for image] 

Source: GAO. 

Note: The percentage of states that did not review or plan to review 
information collected from these sources to monitor providers in each 
year and the percentage of states that did not answer these survey 
questions are not shown in this figure. 

[End of figure] 

Federal guidance suggests states may request district assistance in 
collecting data from providers to assist state monitoring activities. 
While the state is ultimately responsible for monitoring providers, 
most states reported that districts have taken a direct role in 
monitoring providers. Similar to states, although district monitoring 
of providers was limited in 2004-2005, districts used a more extensive 
and active approach in the next year, as shown in figure 6.[Footnote 
20] For example, while we estimate that less than half of districts 
collected information from on-site reviews, school staff, parents, and 
students to monitor providers in 2004-2005, 70 percent or more were 
collecting or planning to collect information from these sources in 
2005-2006. 

Figure 6: Estimated Percentage of Districts Reviewing Information 
Collected from Different Sources to Monitor SES Providers in 2004-2005 
and 2005-2006: 

[See PDF for image] 

Source: GAO. 

[End of figure] 

Although states and districts collected information from similar 
sources to monitor providers, districts collected information from more 
providers than states. Specifically, while a majority of the states 
that conducted on-site reviews observed only some or few providers, we 
estimate that a majority of districts that conducted on-site reviews 
observed most or all of their providers in 2004-2005.[Footnote 21] 
While states and districts may both have capacity limitations that 
affect their ability to conduct on-site reviews to monitor providers, 
conducting such reviews is likely easier for districts because services 
are often delivered on or near school campuses. 

States and districts collected information on several aspects of SES 
programs to monitor providers, as shown in table 5. While federal 
regulations provide states flexibility to design their own SES 
monitoring systems, over two-thirds or more of states and districts 
monitored or planned to monitor all program elements listed, including 
those related to service delivery and use of funds. For example, 94 
percent of states monitored or planned to monitor parent or student 
satisfaction with providers, and 93 percent of districts monitored or 
planned to monitor billing and payment for services and student 
attendance records. 

Table 5: Percentage of States and Districts That Reviewed Specified 
Program Elements to Monitor Providers in 2005-2006: 

Program element: Parent/student satisfaction with a provider; 
Percentage of states: Monitored: 27; 
Percentage of states: Planned to monitor: 67; 
Percentage of states: Monitored or planned to monitor: 94; 
Estimated percentage of districts: Monitored: 34; 
Estimated percentage of districts: Planned to monitor: 57; 
Estimated percentage of districts: Monitored or planned to monitor: 91. 

Program element: Provider communication with teachers and parents; 
Percentage of states: Monitored: 37; 
Percentage of states: Planned to monitor: 56; 
Percentage of states: Monitored or planned to monitor: 92; 
Estimated percentage of districts: Monitored: 46; 
Estimated percentage of districts: Planned to monitor: 43; 
Estimated percentage of districts: Monitored or planned to monitor: 89. 

Program element: Extent to which a provider's program, as enacted, 
reflects its program design, as outlined in its application to your 
state; 
Percentage of states: Monitored: 19; 
Percentage of states: Planned to monitor: 73; 
Percentage of states: Monitored or planned to monitor: 92; 
Estimated percentage of districts: Monitored: 30; 
Estimated percentage of districts: Planned to monitor: 41; 
Estimated percentage of districts: Monitored or planned to monitor: 70. 

Program element: Evidence of meeting academic achievement goals as 
stated on student learning plan; 
Percentage of states: Monitored: 23; 
Percentage of states: Planned to monitor: 65; 
Percentage of states: Monitored or planned to monitor: 88; 
Estimated percentage of districts: Monitored: 28; 
Estimated percentage of districts: Planned to monitor: 60; 
Estimated percentage of districts: Monitored or planned to monitor: 88. 

Program element: Evidence of improved student achievement based on any 
statewide assessment; 
Percentage of states: Monitored: 15; 
Percentage of states: Planned to monitor: 71; 
Percentage of states: Monitored or planned to monitor: 87; 
Estimated percentage of districts: Monitored: 26; 
Estimated percentage of districts: Planned to monitor: 65; 
Estimated percentage of districts: Monitored or planned to monitor: 91. 

Program element: Alignment of provider curriculum with district/school 
curriculum or instruction; 
Percentage of states: Monitored: 25; 
Percentage of states: Planned to monitor: 62; 
Percentage of states: Monitored or planned to monitor: 87; 
Estimated percentage of districts: Monitored: 35; 
Estimated percentage of districts: Planned to monitor: 39; 
Estimated percentage of districts: Monitored or planned to monitor: 74. 

Program element: Student attendance records; 
Percentage of states: Monitored: 27; 
Percentage of states: Planned to monitor: 56; 
Percentage of states: Monitored or planned to monitor: 83; 
Estimated percentage of districts: Monitored: 67; 
Estimated percentage of districts: Planned to monitor: 25; 
Estimated percentage of districts: Monitored or planned to monitor: 93. 

Program element: Evidence of improved student achievement based on 
provider assessments; 
Percentage of states: Monitored: 27; 
Percentage of states: Planned to monitor: 56; 
Percentage of states: Monitored or planned to monitor: 83; 
Estimated percentage of districts: Monitored: 39; 
Estimated percentage of districts: Planned to monitor: 52; 
Estimated percentage of districts: Monitored or planned to monitor: 91. 

Program element: Protection of student privacy; 
Percentage of states: Monitored: 33; 
Percentage of states: Planned to monitor: 50; 
Percentage of states: Monitored or planned to monitor: 83; 
Estimated percentage of districts: Monitored: 55; 
Estimated percentage of districts: Planned to monitor: 28; 
Estimated percentage of districts: Monitored or planned to monitor: 82. 

Program element: Adherence to applicable health, safety, and civil 
rights laws; 
Percentage of states: Monitored: 29; 
Percentage of states: Planned to monitor: 48; 
Percentage of states: Monitored or planned to monitor: 77; 
Estimated percentage of districts: Monitored: 48; 
Estimated percentage of districts: Planned to monitor: 26; 
Estimated percentage of districts: Monitored or planned to monitor: 74. 

Program element: Provider financial stability (e.g., audits, financial 
statements); 
Percentage of states: Monitored: 31; 
Percentage of states: Planned to monitor: 42; 
Percentage of states: Monitored or planned to monitor: 73; 
Estimated percentage of districts: Monitored: N/ A; 
Estimated percentage of districts: Planned to monitor: N/A; 
Estimated percentage of districts: Monitored or planned to monitor: N/ 
A. 

Program element: Evidence of improved student achievement based on 
grades, promotion, and/or graduation; 
Percentage of states: Monitored: 12; 
Percentage of states: Planned to monitor: 58; 
Percentage of states: Monitored or planned to monitor: 69; 
Estimated percentage of districts: Monitored: 23; 
Estimated percentage of districts: Planned to monitor: 57; 
Estimated percentage of districts: Monitored or planned to monitor: 80. 

Program element: Billing and payment for services; 
Percentage of states: Monitored: N/A; 
Percentage of states: Planned to monitor: N/A; 
Percentage of states: Monitored or planned to monitor: N/A; 
Estimated percentage of districts: Monitored: 72; 
Estimated percentage of districts: Planned to monitor: 21; 
Estimated percentage of districts: Monitored or planned to monitor: 93. 

Source: GAO. 

Note: The percentage of states that did not review or plan to review 
these program elements to monitor providers in 2005-2006 and the 
percentage of states that did not answer these survey questions are not 
shown in this table. In addition, we did not ask states if they 
monitored billing and payment for services, and we did not ask 
districts if they monitored provider financial stability. 

[End of table] 

Many States Struggle to Develop Meaningful Evaluations, and the Few 
State Evaluations Completed to Date Were Inconclusive: 

Many states struggle to develop evaluations to determine whether SES 
providers are improving student achievement, though states are required 
to evaluate and withdraw approval from providers that fail to do so 
after 2 years. Specifically, federal law requires states to develop 
standards and techniques to evaluate the services delivered by approved 
providers, but it does not require states to use specific evaluation 
methods or criteria for determining provider effectiveness.[Footnote 
22] Through our survey, states reported several challenges to 
evaluating SES providers. Specifically, over three-fourths of states 
reported that determining sufficient academic progress of students, 
having the time and knowledge to analyze SES data, and developing data 
systems to track SES information have been challenges. Further, during 
our site visits to Illinois and New Jersey, state officials noted they 
were currently in the process of improving their data collection 
systems to more effectively capture and analyze data for SES 
evaluations. In addition, several state officials reported that while 
they have collected some information to assess provider effectiveness, 
they have done little with that data. Others noted that they have not 
received sufficient federal guidance on effective models for SES 
provider evaluations, and because developing and implementing such 
evaluations can be both time-consuming and costly, additional 
assistance from Education would improve their efforts. 

At the time of our survey in early 2006, only a few states had drafted 
or completed an evaluation report addressing individual SES provider's 
effects on student academic achievement, and we found that no state had 
produced a report that provided a conclusive assessment of this effect. 
New Mexico and Tennessee were the only two states that had completed 
final or draft SES evaluation reports that attempted to assess the 
impact of all SES providers serving students in their states in 
previous years.[Footnote 23] To measure student academic achievement, 
New Mexico's report analyzed students' grades as well as their scores 
on state assessments and provider assessments, which often differ by 
provider and are administered both before SES sessions begin and at the 
end of SES sessions each year. However, the report noted that these 
assessments produced different results related to student academic 
achievement gains. While Tennessee also planned to review students' 
state assessment scores, the draft available at the time of our 
analysis did not include this information. In addition, both reports 
drew on information obtained through other sources, such as teacher 
surveys, to assess provider effectiveness. Due to their limitations, 
neither evaluation provided a conclusive assessment of SES providers' 
effect on student academic achievement. 

In addition, at the time of our survey, over half of the states 
reported that they were in the process of conducting an evaluation of 
SES providers in order to meet the federal requirement of assessing 
each provider's effect on student academic achievement.[Footnote 24] 
Similar to the state evaluations already undertaken, officials reported 
using different methods and criteria to evaluate SES providers. For 
example, some states were collecting each provider's pre-and post-SES 
assessments of students while others were collecting student 
achievement data from districts or students' state assessment scores. 
Further, while one state defined adequate student progress as 80 
percent of a provider's students making one-grade level of improvement 
after a year of SES, another state defined adequate student progress as 
50 percent or more of a provider's students having any positive 
academic achievement gain after a year of SES. While these states have 
yet to produce final results from their SES provider evaluations, it is 
unclear whether any of these efforts will produce a conclusive 
assessment of SES providers' effect on student academic achievement. 

Likely because states are struggling to complete evaluations to assess 
SES providers' effect on student academic achievement, states did not 
report that they have withdrawn approval from providers because their 
programs were determined to be ineffective at achieving this 
goal.[Footnote 25] Rather, though over 40 percent of states reported 
that they had withdrawn approval from some providers, they most 
frequently reported withdrawing provider approval because the provider 
was a school or district that had entered needs improvement status, the 
provider asked to be removed from the state-approved provider list, or 
because of provider financial impropriety. 

Several Education Offices Monitor and Support SES Implementation, but 
States and Districts Cite the Need for Additional Assistance and 
Flexibility: 

Several offices within Education monitor various aspects of SES 
activity across the country and provide support, but states and 
districts reported needing additional assistance and flexibility with 
program implementation. Education conducts SES monitoring in part 
through policy oversight and compliance reviews of states and 
districts, and provides SES support through guidance, grants, research, 
and technical assistance. However, many states and districts reported 
needing additional assistance and guidance regarding evaluation and 
administration of SES. Further, some states and districts voiced 
interest in expansion of Education's pilot programs that increase SES 
flexibility, including the pilot that allows certain low-achieving 
districts to serve as SES providers. 

Education Monitors SES and Provides States and Districts with Guidance 
and Technical Assistance: 

OII and OESE are primarily responsible for monitoring and supporting 
state and district SES implementation, and other Education offices also 
contribute to these efforts (see fig. 7). 

Figure 7: U.S. Department of Education Offices Monitoring and 
Supporting SES: 

[See PDF for image] 

Source: GAO analysis. 

Note: This figure reflects the coordination of Education's offices 
rather than the statutory reporting relationship. ] 

[End of figure] 

OII, which leads SES policy development and provides strategic 
direction, monitors SES policy issues primarily through what it calls 
"desk monitoring." This monitoring is performed at its federal office 
and includes the review of SES-related research and media reports. OII 
also conducts more intensive monitoring of specific SES implementation 
challenges when states, districts, and providers bring them to 
Education's attention and keeps a log documenting these issues. For 
example, during 2004-2005, Illinois and New Jersey officials contacted 
OII to clarify guidance regarding providers affiliated with districts 
in need of improvement, and OII staff provided assistance and 
clarification. In addition, several providers we interviewed also 
mentioned that they have contacted OII directly to discuss 
implementation challenges associated with enrollment, district 
contracts, and provider access to school facilities. 

OESE, which oversees and supports NCLBA implementation, is also 
involved in monitoring SES implementation through its overall 
monitoring of state compliance with Title I and NCLBA. To monitor Title 
I, OESE staff visit state departments of education and selected 
districts within each state to interview officials and review relevant 
documents. Once the review is complete, OESE issues a report to the 
state outlining any instances of Title I non-compliance, including 
those related to SES, and actions needed to comply with regulations. As 
of June 2006, OESE had visited and issued reports to over three-fourths 
of the states. 

In addition to its Title I monitoring, OESE also oversees the 
collection of state NCLBA data, including data on SES, through the 
annual Consolidated State Performance Report (CSPR). For the CSPR, each 
state is required to report the number of schools with students 
receiving SES, the number of students eligible for services, and the 
number that received services.[Footnote 26] However, although almost 
all states reported that they are collecting information on district 
SES expenditures as part of their oversight, Education does not require 
states to submit information on the percent or amount of Title I funds 
districts spent for SES through the CSPR or other means. Therefore, 
while Education tracks the extent to which states are providing SES to 
eligible students, the department does not collect data on the relative 
costs of serving them. Further, under NCLBA, Education is required to 
present an annual summary of the CSPR data to Congress. As of June 
2006, the most recent report to Congress was for the 2002-2003 school 
year, though Education officials indicated they expect to submit 
reports for 2003-2004 and 2004-2005 in the near future. 

While OII and OESE monitoring of SES has been either limited to desk 
monitoring or combined with general Title I monitoring, OIG has 
conducted audits specifically focused on SES. During 2003-2004 and 2004-
2005, the office performed a series of SES implementation audits in six 
states, which involved site visits to state offices and selected 
districts within each state.[Footnote 27] Also during 2004-2005, OIG 
performed audits of five California districts and one SES provider 
within each district.[Footnote 28] These audits included an examination 
of district SES contracts with providers, provider services, and 
provider compliance with state SES regulations. 

Several Education officials reported coordinating internally to share 
information, including SES monitoring results. To facilitate 
coordination, OII leads an internal group comprised of staff members 
from other Education offices, who meet bi-weekly to exchange 
information. OESE shares its state Title I monitoring results and CSPR 
data with other Education offices. In addition, OIG makes 
recommendations to both OII and OESE in its state and district SES 
auditing reports and disseminates the reports throughout Education and 
on the department's Web site. Since 2002, OII has coordinated the 
publication of four versions of non-regulatory SES guidance, each 
updated to address ongoing challenges with SES implementation. The 
latest and most comprehensive version of non-regulatory SES guidance 
was published in June 2005. In May 2006, Education issued a separate 
supplement to the guidance containing additional information on private 
school participation in providing SES and a policy letter clarifying 
the definition of a district-affiliated provider. 

In addition to its monitoring efforts, OII also provides SES 
implementation assistance, in part through presentations at 
conferences, and through grants to external organizations that assist 
states and districts. For example, OII staff have presented information 
on SES policy and promising practices at national meetings attended by 
SES coordinators and others involved in SES implementation. In 
addition, the office has issued grants to the Black Alliance for 
Educational Options, the Hispanic Council for Reform and Educational 
Options, and through the Star Schools Program to promote SES to 
minority students and those in rural areas. Further, OII funded the 
Supplemental Educational Services Quality Center (SESQC), which offered 
SES technical assistance through tool-kits, issue briefs, and a Web 
site containing SES information for state and district officials, 
schools, parents, and providers. SESQC also periodically convened 
representatives of organizations working on education issues to discuss 
SES national coordination, challenges, and promising practices. 
However, those meetings and all SESQC activities were discontinued in 
December 2005 when SESQC's grant period ended. 

Other Education offices also provide SES support through various means. 
For example, OESE funded the Comprehensive Centers Program through 
grants that established technical assistance centers across the country 
to help low-performing schools and districts close achievement gaps and 
meet the goals of NCLBA. Of these, the Center on Innovation and 
Improvement provides support to regional centers that assist states 
with Education's programs, including SES. In addition, Education's 
Policy and Program Studies Service, within the Office of Planning, 
Evaluation and Policy Development, oversees several research studies 
that examine SES, either in whole or in part. These reports, such as 
the National Assessment of Title I: Interim Report and Case Studies of 
Supplemental Services under the No Child Left Behind Act, provide 
states and districts with information on SES implementation, 
challenges, and promising practices. Further, Education's Center for 
Faith-Based and Community Initiatives offers technical assistance to 
faith-and community-based organizations interested in becoming state- 
approved SES providers. 

Given the technical assistance and support Education has already 
provided to states and districts for implementation of SES and school 
choice, and the department's view that implementation of these 
provisions has been uneven throughout the country, in May 2006, 
Education issued a policy letter announcing the department's plans to 
take significant enforcement action. Specifically, Education plans to 
use the data collected through its monitoring and evaluation efforts to 
take enforcement actions such as placing conditions on state Title I 
grants, withholding federal funds, or entering into compliance 
agreements. In the letter, the department noted that its various 
monitoring activities have identified several areas of noncompliance 
with SES requirements. For example, the OIG's audits found that each of 
the six states reviewed failed to adequately monitor their districts 
for compliance. Consequently, nearly all of the parental notification 
letters reviewed failed to include the required key components, and 
several districts failed to budget sufficient funding for services. 
Through our own analysis of Education's monitoring reports, we also 
found that some of the states reviewed were found to have inadequate or 
incomplete processes for monitoring and evaluating SES providers. 

Many States and Districts Reported Needing SES Evaluation Support and 
Additional Technical Assistance, and Some Suggested Increasing 
Flexibility through Education's Pilot Programs: 

Despite Education's efforts, many states and districts reported needing 
more information and assistance to better comply with certain aspects 
of SES implementation, including SES evaluation (see table 6). 
Specifically, 85 percent of states and an estimated 70 percent of 
districts needed additional assistance with methods for evaluating SES, 
and over 60 percent also needed assistance with developing data 
systems. Many districts also needed more information on provider 
quality and effectiveness. Although OESE and OIG monitoring results 
have also continually indicated that states and districts struggle with 
SES evaluation, Education has yet to provide comprehensive assistance 
in this area, and during our site visits, officials mentioned that they 
have been relying on other states, organizations, or individuals for 
evaluation assistance. States and districts also indicated a need for 
more support and technical assistance to help them administer SES. 
Specifically, approximately three-fourths of states and two-thirds of 
districts reported needing funding to increase their capacity to 
implement SES. Many states also reported needing tool kits and model/ 
sample documents, as well as training from Education, and a majority of 
districts needed effective parent outreach strategies. Further, most 
states reported a need for conferences or meetings where they could 
share lessons learned and promising practices with other states. A few 
Tennessee officials mentioned that conferences hosted by national 
organizations have been an effective means of allowing SES officials to 
gather and share knowledge. 

Table 6: Percentage of States and Districts in Need of Different Types 
of Information or Assistance with SES Implementation: 

Evaluation information or assistance needed: Methods for evaluating 
SES; 
Percentage of states: Evaluation information or assistance needed: 85; 
Information on provider quality or effectiveness; 
Estimated percentage of districts: 78. 

Evaluation information or assistance needed: Developing data systems; 
Percentage of states: Evaluation information or assistance needed: 71;  
Methods for evaluating SES; 
Estimated percentage of districts: 70. 

Evaluation information or assistance needed: Developing data systems; 
Percentage of states: Evaluation information or assistance needed: 71; 
Developing data systems for monitoring and evaluation; 
Estimated percentage of districts:  64. 

Other support or technical assistance needed: Conferences/meetings with 
other states to share lessons learned/ promising practices; 
Percentage of states: 90; 
Funding to increase district capacity to implement SES; 
Estimated percentage of districts: 67. 

Other support or technical assistance needed: Tool kits and 
model/sample documents; 
Percentage of states: 85; 
Information on effective parent outreach strategies; 
Estimated percentage of districts: 61. 

Other support or technical assistance needed: Training; 
Percentage of states:  85; 
Assistance with contract negotiation; 
Estimated percentage of districts: 35. 

Other support or technical assistance needed: Funding to increase state 
capacity to implement SES; 
Percentage of states: 77; 
[Empty]; 
Estimated percentage of districts: [Empty]. 

Other support or technical assistance needed: Approving and monitoring 
providers; 
Percentage of states: 71; 
 [Empty]; 
Estimated percentage of districts: [Empty]. 

Other support or technical assistance needed: Allowances for SES to be 
offered before school choice provisions; 
Percentage of states: 58; 
[Empty]; 
Estimated percentage of districts: [Empty]. 

Other support or technical assistance needed: Clarification of roles 
for states, districts, and providers in implementing SES; 
Percentage of states: 46; 
[Empty]; 
Estimated percentage of districts: [Empty]. 

Other support or technical assistance needed: Waivers for states for 
exclusion from specific SES provisions; 
Percentage of states: 44; 
[Empty]; 
Estimated percentage of districts: [Empty]. 

Other support or technical assistance needed: Clarification of the 20-
percent set-aside requirements and carryover allowances; 
Percentage of states: 44; 
[Empty]; 
Estimated percentage of districts: [Empty]. 

Source: GAO. 

[End of table] 

While three-fourths of states reported that the most recent version of 
Education's SES guidance, published in June 2005, has been very or 
extremely useful, several states commented through our survey that they 
needed additional or clearer guidance on certain SES provisions. This 
included guidance on managing SES costs and fees, implementing SES in 
rural areas, and handling provider complaints. During three of our site 
visits, officials also expressed some concern about the lack of clarity 
in the SES guidance with regards to student eligibility requirements 
and how to craft a parental SES notification letter that is both 
complete and easy for parents to understand. 

Regarding parental notification letters, though both OESE and OIG found 
many states and districts to be non-compliant with the federal 
requirement that district SES parental notification letters include 
several specific elements,[Footnote 29] Education's SES guidance, which 
is coordinated by OII, provides a sample that does not clearly specify 
all of the key elements required by SES law and regulations. For 
example, the sample letter does not include information on provider 
services, qualifications, and effectiveness. Furthermore, a few state 
and district officials commented that, when followed, the Title I 
regulations governing SES yield a letter that is unreasonably long and 
complex, which may be difficult for parents to understand. 

Many states and districts expressed interest in the flexibility offered 
through two pilot programs that Education implemented during 2005-2006. 
The department designed these pilots to increase the number of eligible 
students receiving SES and to generate additional information about the 
effect of SES on student academic achievement. For example, several 
state and district SES coordinators expressed interest in Education's 
pilot program that allowed two districts in needs improvement status to 
act as SES providers in exchange for their expansion of student access 
to SES providers and collection of achievement data to determine SES 
program effectiveness. During three of our four site visits, state and 
district officials expressed concern that districts identified for 
needs improvement are excluded from delivering SES, and one state 
official noted that removing districts from the state approved provider 
list may result in lower SES participation. Further, in our surveys, 
three state SES coordinators and 17 district SES coordinators wrote in 
comments that permitting districts in needs improvement status to 
provide services would assist their efforts. Through both our surveys 
and site visits, officials suggested that allowing districts to act as 
providers may ease student access to SES for rural districts that do 
not have providers located nearby, allow more students to participate 
in SES because district costs to provide services are sometimes lower 
than other providers' costs, and enable districts to continue their 
existing tutoring programs that they feel are effective and meet the 
same goals as SES. Overall, we estimate that 15 percent of districts 
were state-approved providers in 2004-2005. However, another national 
survey recently found the percentage of urban and suburban districts 
that are state-approved SES providers is declining.[Footnote 30] 

The other SES pilot allowed four districts in Virginia to offer SES 
instead of school choice in schools that have missed state performance 
goals for 2 years and are in their first year of needs improvement. 
During our site visits and through our surveys, many states and 
districts expressed interest in adjusting the order of the SES and 
school choice interventions. Specifically, half of states and over 60 
percent of districts suggested that SES should be made available before 
school choice (see table 7). Further, approximately three-quarters of 
both states and districts indicated that SES should be offered either 
before or simultaneously with choice. As we found through our previous 
work on school choice, few students are opting to transfer schools in 
the first year of needs improvement, and therefore this change would 
provide students with another option to receive additional academic 
support in that year. Further, in a recent national study, district and 
school officials noted that parents and students are often not 
interested in changing schools, in part because of potential long 
commutes and satisfaction with their current schools, which suggests 
that parents and students would likely prefer to receive SES in their 
own schools and neighborhoods rather than school choice.[Footnote 31] 
In line with interest in increased flexibility with these 
interventions, in May 2006, Education announced that due to the 
positive results in Virginia districts under the pilot, the department 
plans to extend and expand this pilot in 2006-2007. 

Table 7: State and District Opinion on the Ordering of School Choice 
and SES: 

In percent: Order of school choice and SES: SES should precede school 
choice; 
States: 48; 
District: 62. 

In percent: Order of school choice and SES: Both school choice and SES 
should be offered at the same time; 
States: 27; 
District: 15. 

In percent: Order of school choice and SES: School choice should 
precede SES; 
States: 15; 
District: 23. 

Source: GAO. 

Note: 10 percent of states did not respond or were not sure. In 
addition, district percentages are estimates. 

[End of table] 

Conclusions: 

Over the last few years, almost all states and approximately 1,000 
districts have been required to implement SES across the country and, 
if current trends continue, more schools will be required to offer 
services in the future. Although some states and districts are 
beginning to gain experience in implementing SES and use promising 
approaches to increase SES participation, many students are still not 
receiving services, in part because providers are sometimes not 
available to serve certain areas and groups. In addition, some 
districts are unsure how to involve school officials in facilitating 
local coordination of SES implementation and providing effective 
parental notification. While Education has provided support to states 
and districts through guidance and technical assistance, many report 
needing additional assistance to address these challenges. Further, the 
lack of clarity between policy guidance issued by OII and criteria used 
by OESE in their compliance reviews of states' implementation efforts 
creates additional challenges in meeting the federal requirements for 
parental notification letters. Providing states with clear guidance 
that has been coordinated across Education offices is especially 
important now that the department has announced plans to take 
significant enforcement actions to ensure states comply with federal 
SES requirements. 

While some districts do not have any students receiving services and, 
therefore, are not spending any Title I funds for SES, other districts 
are spending more than their entire set-aside and still have students 
on waiting lists to receive services. Two districts have been able to 
participate in Education's pilot program waiving federal regulations 
that preclude districts in need of improvement from providing SES, 
which may help them provide services to more students at a lower cost. 
However, extending this flexibility to more districts depends on the 
evaluation of the quality of these services to determine if SES is 
having a positive impact on student academic achievement. In addition, 
the absence of strategies that states can use to set parameters around 
program design and costs further hinders their ability to stretch 
available funding to serve more students. Federal and state oversight 
of district efficiency in using federal funds to provide SES services 
is hindered by incomplete reporting requirements that require states to 
report on the number of eligible children receiving SES, but not the 
data they collect on the amount of Title I funding used to serve them. 
This information gap limits Education's ability to track state and 
district compliance in spending funds for SES. Further, Education's 
ability to ensure that federal dollars are effectively spent to improve 
student academic achievement is limited until states increase their 
capacity to monitor and evaluate provider performance. In the absence 
of additional federal technical assistance and access to information 
about state and district promising practices, some states may continue 
to struggle with implementation and evaluation of SES. 

Recommendations: 

To help states and districts implement SES more effectively, we 
recommend that the Secretary of Education use the department's Web site 
and the Center on Innovation and Improvement, as well as other means of 
communication, to: 

* Provide federal guidance on SES parental notification letters that is 
clear and has been coordinated internally between OII and OESE to 
provide additional assistance to states and districts to help them 
comply with federal requirements and ensure that letters are easy for 
parents to understand. Education might consider providing several 
samples of actual district notification letters that meet these 
criteria. 

* Collect and disseminate information on promising practices used by 
states to attract more providers for certain areas and groups and 
promising practices used by districts to improve parental notification 
of SES services and providers' ability to serve specific groups of 
students and to encourage student attendance. 

* Provide examples of how districts can involve schools and school 
officials to facilitate local coordination with providers. 

To improve states' and districts' ability to make the most of funding 
for SES and provide services to the maximum number of students, we 
recommend that the Secretary of Education: 

* Consider expanding the 2005-2006 pilot that allowed two districts in 
need of improvement to enter into flexibility agreements to serve as 
SES providers if evaluation results show that these districts can 
provide quality SES services. 

* Clarify what states can do through the provider approval process to 
set parameters around program design and costs. For example, Education 
could issue guidance to states that clarifies their authority to set 
parameters on issues such as minimum hours of SES per student, minimum 
tutor qualifications, and cost ranges. In addition, Education could 
suggest to states that they coordinate these discussions with districts 
to address their concerns about program design and costs. 

To improve federal and state monitoring of SES, we recommend that the 
Secretary of Education: 

* Require states to report information necessary to determine the 
amount of funds spent by districts to provide SES and the percentage of 
their Title I allocations that this amount represents. Because almost 
all states reported that they are planning to monitor district SES 
expenditures, Education could require the states to submit these data 
through the annual NCLBA Consolidated State Performance Reports. 

* Provide states with technical assistance and guidance on how to 
evaluate the effect of SES on student academic achievement. For 
example, Education might require the Center on Innovation and 
Improvement to focus its SES assistance on providing states with 
suggested evaluation methods, measures to assess the impact of SES on 
achievement, and criteria for using this information to monitor and 
withdraw state approval from providers. Further, lessons learned and 
promising practices on evaluation could also be shared with states on 
the Center on Innovation and Improvement's Web site or during national 
or regional meetings, trainings, or conferences. 

Agency Comments: 

We provided a draft of this report to Education for review and comment. 
Educations' written comments are reprinted in appendix II, and the 
department's technical comments were incorporated into the report as 
appropriate. In its written comments, Education expressed appreciation 
for the report's recommendations and cited actions the department has 
already initiated or plans to take in addressing them. Specifically, 
Education noted several projects under development that might assist in 
carrying out our recommendations to provide more assistance to states 
on notifying parents, attracting providers for certain areas and 
groups, and involving schools. The department also said that is 
currently considering expanding the pilot allowing districts in need of 
improvement to apply to become SES providers, per our recommendation. 
Regarding our recommendation that Education clarify what states can do 
to set parameters around program design and costs, Education said it 
would consider addressing this issue further in the next set of 
revisions to the SES non-regulatory guidance. In addition, Education 
said it would address our recommendations to improve federal and state 
monitoring of SES by proposing that districts report on their SES 
spending and by providing more SES evaluation assistance to states 
through an updated issue brief as well as technical assistance provided 
by the Comprehensive Center on Innovation and Improvement and at a 
conference this fall. 

We are sending copies of this report to appropriate congressional 
committees, the Secretary of Education, and other interested parties. 
Copies will also be made available upon request. In addition, the 
report will be available at no charge on GAO's Web site at [Hyperlink, 
http://www.gao.gov]. If you or your staff have any questions about the 
report, please contact me at (202) 512-7215. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made major contributions to 
this report are listed in appendix III. 

Signed by: 

Marnie S. Shaul: 
Director, Education, Workforce, and Income Security Issues: 

List of Requesters: 

The Honorable Edward M. Kennedy: 
Ranking Minority Member: 
Committee on Health, Education, Labor, and Pensions: 
United States Senate: 

The Honorable Christopher J. Dodd: 
Ranking Minority Member: 
Subcommittee on Education and Early Childhood Development: 
Committee on Health, Education, Labor, and Pensions: 
United States Senate: 

The Honorable Howard "Buck" McKeon: 
Chairman: 
The Honorable George Miller: 
Ranking Minority Member: 
Committee on Education and the Workforce: 
House of Representatives: 

The Honorable Dale E. Kildee: 
Ranking Minority Member: 
Subcommittee on 21st Century Competitiveness Committee on Education and 
the Workforce: 
House of Representatives: 

The Honorable Lynn C. Woolsey: 
Ranking Minority Member: 
Subcommittee on Education Reform Committee on Education and the 
Workforce: 
House of Representatives: 

The Honorable Hillary Rodham Clinton: 
The Honorable Richard J. Durbin: 
The Honorable Charles E. Schumer: 
United States Senate: 

The Honorable Robert E. Andrews: 
The Honorable John A. Boehner: 
The Honorable Rahm Emanuel: 
The Honorable Betty McCollum: 
House of Representatives: 

[End of section] 

Appendix I: Scope and Methodology: 

To obtain nationally representative information on supplemental 
educational services (SES) participation, state and local 
implementation, and federal oversight, we conducted a Web-based survey 
of state SES coordinators and a mail survey of district SES 
coordinators from a nationally representative sample of districts with 
schools required to offer SES. We also conducted site visits during 
which we interviewed state, district, and school officials representing 
four school districts, and we conducted interviews with 22 SES 
providers both during the site visits and separately. In addition, we 
spoke with staff at Education involved in SES oversight and 
implementation and reviewed Education's data on SES. We conducted our 
work from August 2005 through July 2006 in accordance with generally 
accepted government auditing standards. 

Survey of States: 

To better understand state SES implementation, particularly how states 
are monitoring and evaluating SES, we designed and administered a Web- 
based survey of state SES coordinators in all 50 states, the District 
of Columbia, and Puerto Rico. The survey was conducted between January 
and March 2006 with 100 percent of state SES coordinators responding. 
The survey included questions about student participation in SES, 
actions taken to increase participation, SES funding and expenditures, 
methods used to monitor and evaluate implementation, implementation 
challenges, and assistance received from Education. 

Because this was not a sample survey, there are no sampling errors. 
However, the practical difficulties of conducting any survey may 
introduce nonsampling errors, such as variations in how respondents 
interpret questions and their willingness to offer accurate responses. 
We took steps to minimize nonsampling errors, including pre-testing 
draft instruments and using a Web-based administration system. 
Specifically, during survey development, we pre-tested draft 
instruments with officials in Oregon, Maryland, and Washington between 
October and November 2005. In the pre-tests, we were generally 
interested in the clarity of the questions and the flow and layout of 
the survey. For example, we wanted to ensure definitions used in the 
surveys were clear and known to the respondents, categories provided in 
closed-ended questions were complete and exclusive, and the ordering of 
survey sections and the questions within each section was appropriate. 
On the basis of the pre-tests, the Web instrument underwent some slight 
revisions. A second step we took to minimize nonsampling errors was 
using a Web-based survey. By allowing respondents to enter their 
responses directly into an electronic instrument, this method 
automatically created a record for each respondent in a data file and 
eliminated the need for and the errors (and costs) associated with a 
manual data entry process. To further minimize errors, programs used to 
analyze the survey data and make estimations were independently 
verified to ensure the accuracy of this work. 

While we did not fully validate specific information that states 
reported through our survey, we took several steps to ensure that the 
information was sufficiently reliable for the purposes of this report. 
For example, after the survey was closed, we made comparisons between 
select items from our survey data and other national-level data 
sets.[Footnote 32] We found our survey data were reasonably consistent 
with the other data sets. On the basis of our checks, we believe our 
survey data are sufficient for the purposes of our work. 

Survey of School Districts: 

To obtain national-level information on district implementation of SES, 
we administered a mail survey to a nationally representative sample of 
districts that had schools required to offer SES in school year 2004- 
2005. The survey was conducted between January and March 2006. To 
obtain the maximum number of responses to our survey, we sent a 
reminder postcard to nonrespondents approximately 1 week after the 
initial mailing of the survey instrument, a follow-up mailing with the 
full survey instrument to nonrespondents approximately 3 weeks after 
the initial mailing, and a second follow-up mailing with the full 
survey instrument approximately 4 weeks later. The survey included 
questions about student participation in SES, characteristics of 
students receiving SES, actions taken to increase participation, SES 
funding and expenditures, methods used to monitor and evaluate 
implementation, implementation challenges, and assistance received and 
still needed. 

Population: 

The target population of 1,095 districts consisted of public school 
districts with at least one school in each of their jurisdictions 
required to provide SES in the 2004-2005 school year. To define our 
population, we collected school improvement information from state 
education agency Web sites and the NCLBA Consolidated State Performance 
Reports: Part I for Reporting on School Year 2003-2004 that states 
submitted to Education. When available, we checked both sources for 
school improvement information and used the source that provided the 
most recently updated data, as this data is often updated by states 
over the course of the school year. After constructing our population 
of districts, we used Education's Common Core of Data Local Education 
Agency (School District) preliminary file and the Public Elementary/ 
Secondary School preliminary file for the 2003-2004 school year to 
further define the characteristics of our population. On the basis of 
our review of these data, we determined these sources to be adequate 
for the purposes of our work. 

Sample Design and Errors: 

The sample design for the mail survey was a stratified random sample of 
districts with one certainty stratum containing all of the districts 
with 100,000 students or more and one stratum containing all other 
districts in the universe. We included the 21 districts with 100,000 or 
more students with certainty in the sample to ensure we gathered 
information from the largest districts required to offer SES. 

We selected a simple random sample of districts in the non-certainty 
stratum and calculated the sample size to achieve a precision of plus 
and minus 7 percent at the 95 percent confidence level for an expected 
proportion of 50 percent. To ensure the sample sizes were adequate, we 
increased the sample size assuming we would obtain a 70 percent 
response rate. The total sample size for this stratum was 237 
districts. In the sample, each district in the population had a known, 
nonzero probability of being selected. Each selected district was 
subsequently weighted in the analysis to account statistically for all 
the schools in the population, including those that were not selected. 
Table 8 provides a description of the universe and sample of districts. 

Table 8: Description of the Population and Sample of Districts: 

Stratum: District enrollment >= 100,000; 
Population: 21; 
Sample size: 21. 

Stratum: District enrollment < 100,000; 
Population: 1,074; 
Sample size: 237. 

Stratum: Total; 
Population: 1,095; 
Sample size: 258. 

Source: GAO. 

[End of table] 

Because we surveyed a sample of districts, our results are estimates of 
a population of districts and thus are subject to sampling errors that 
are associated with samples of this size and type. Our confidence in 
the precisions of the results from this sample is expressed in 95 
percent confidence intervals, which are expected to include the actual 
results in 95 percent of the samples of this type. We calculated 
confidence intervals for this sample based on methods that are 
appropriate for a stratified random sample. 

We determined that 10 of the sampled districts were out of scope 
because they did not have any schools required to provide SES in the 
2004-2005 school year. All estimates produced from the sample and 
presented in this report are for the estimated target population of 
1,034 districts with at least one school required to provide SES in the 
2004-2005 school year. All percentage estimates included in this report 
have margins of error of plus or minus 8 percentage points or less, 
except for those shown in table 9. All other numerical estimates, such 
as the total number of schools required to offer SES in 2004-2005, 
included in this report have margins of error of plus or minus 18 
percent or less. 

Table 9: Sampling Error Calculations: 

Page: 15; 
Description: Percentage of districts that spent 20 percent or less of 
the amount set aside for SES in 2004-2005; 
Estimate: 38; 
Lower bound: 30; 
Upper bound: 47. 

Page: 15; 
Description: Percentage of districts that spent over 80 percent of the 
amount set aside for SES in 2004-2005; 
Estimate: 18; 
Lower bound: 11; 
Upper bound: 26. 

Page: 17; 
Description: Percentage of districts where over half of SES recipients 
in 2004-2005 were elementary school students; 
Estimate: 56; 
Lower bound: 48; 
Upper bound: 65. 

Page: 17; 
Description: Percentage of districts where less than 5 percent of SES 
recipients in 2004-2005 were students with limited English proficiency; 
Estimate: 33; 
Lower bound: 22; 
Upper bound: 43. 

Page: 17; 
Description: Percentage of districts where over half of SES recipients 
in 2004-2005 were students with limited English proficiency; 
Estimate: 21; 
Lower bound: 13; 
Upper bound: 32. 

Page: 17; 
Description: Percentage of districts where less than 20 percent of SES 
recipients in 2004-2005 were students with disabilities; 
Estimate: 69; 
Lower bound: 59; 
Upper bound: 79. 

Page: 17; 
Description: Percentage of districts where over half of SES recipients 
in 2004-2005 were African-American students; 
Estimate: 41; 
Lower bound: 31; 
Upper bound: 50. 

Page: 17; 
Description: Percentage of districts where over half of SES recipients 
in 2004-2005 were Hispanic students; 
Estimate: 30; 
Lower bound: 21; 
Upper bound: 39. 

Page: 20; 
Description: Percentage of the districts not notifying parents before 
the beginning of the school year that did not receive final school 
improvement data from the state prior to beginning of the school year; 
Estimate: 51; 
Lower bound: 42; 
Upper bound: 61. 

Page: 20; 
Description: Percentage of the districts not notifying parents before 
the beginning of the school year that notified parents within 2 months 
following the beginning of the school year; 
Estimate: 91; 
Lower bound: 82; 
Upper bound: 96. 

Page: 21; 
Description: Percentage of rural districts where availability of 
transportation for students attending SES was a moderate to very great 
challenge; 
Estimate: 50; 
Lower bound: 37; 
Upper bound: 63. 

Page: 32; 
Description: Percentage of the districts that observed most or all 
providers during on-site reviews conducted in 2004-2005; 
Estimate: 83; 
Lower bound: 72; 
Upper bound: 91. 

Source: GAO. 

[End of table] 

We took steps to minimize nonsampling errors that are not accounted for 
through statistical tests, like sampling errors. In developing the mail 
survey, we conducted several pretests of draft instruments. We 
pretested the survey instrument with district officials in Woodburn, 
Ore; Tacoma, Wash; Baltimore, Md; and Alexandria, Va., between October 
and November 2005. These pre-tests were similar to the state Web survey 
pre-tests in design and content. On the basis of the pre- tests, the 
draft survey instrument underwent some slight revisions. 

While we did not fully validate specific information that districts 
reported through our survey, we took several steps to ensure that the 
information was sufficiently reliable for the purposes of this report. 
For example, data from the surveys were double-keyed to ensure data 
entry accuracy, and the information was analyzed using statistical 
software. After the survey was closed, we also made comparisons between 
select items from our survey data and other national-level data 
sets.[Footnote 33] We found our survey data were reasonably consistent 
with the external sources. On the basis of our checks, we believe our 
survey data are sufficient for the purposes of our work. 

Response Rate: 

We received survey responses from 73 percent of the 258 district Title 
I/SES coordinators in our sample. The response rate, adjusted for the 
known and estimated districts that were out of scope, was 77 percent. 

Table 10: SES District Survey Response Rates: 

Stratum: District enrollment >= 100,000; 
Districts in population: 21; 
Districts in sample: 21; 
In-scope districts in sample: 21; 
Districts responding: 21. 

Stratum: District enrollment < 100,000; 
Districts in population: 1,074; 
Districts in sample: 237; 
In-scope districts in sample: 227; 
Districts responding: 167. 

Stratum: Total; 
Districts in population: 1,095; 
Districts in sample: 258; 
In-scope districts in sample: 248; 
Districts responding: 188. 

Source: GAO. 

[End of table] 

After the survey was closed, we analyzed the survey respondents to 
determine if there were any differences between the responding 
districts, the nonresponding districts, and the population. We 
performed this analysis for three characteristics--total number of 
students enrolled, total number of special education students, and 
total number of English language learner students. We determined 
whether sample-based estimates of these characteristics compared 
favorably with the known population values. The population value for 
all of the characteristics we examined fell within the 95 percent 
confidence intervals for the estimates from the survey respondents. On 
the basis of the 77 percent response rate and this analysis, we chose 
to include the survey results in our report and produce sample-based 
estimates to the population of districts required to provide SES in the 
2004-2005 school year. 

Site Visits: 

To understand SES implementation at the local level, we conducted site 
visits to four districts between October 17, 2005, and February 16, 
2006. The districts visited included Woodburn School District 
(Woodburn, Ore.), Hamilton County Schools (Chattanooga, Tenn.), Newark 
Public Schools (Newark, N.J.), and Chicago Public Schools (Chicago, 
Ill.) The four districts visited were selected because they had 
experience implementing SES in their schools and were recommended by 
stakeholders as having promising parent outreach and/or monitoring 
practices. When viewed as a group, the districts also provided 
variation across characteristics such as geographic location, district 
size, student ethnicity, and the percentage of students with limited 
English proficiency or disabilities. 

During the site visits, we interviewed state officials, including the 
state SES coordinator, and district officials, including the 
superintendent and SES coordinator. We also interviewed officials 
representing 12 schools, including principals and other school staff 
involved with SES. In total, we visited several schools of each level, 
from elementary to high, and though district officials selected the 
schools we visited, all of the schools had experience implementing SES. 
Through our interviews with state, district, and school officials, we 
collected information on district efforts to notify parents and fulfill 
implementation responsibilities, student participation, providers, 
local implementation challenges, and implementation assistance received 
and needed. During the visits, we also interviewed providers and 
observed tutoring sessions in order to better understand 
implementation. During our visit to Woodburn, Ore., we also observed a 
provider fair. 

In addition to our site visits to four districts, we also visited the 
Rhode Island Department of Elementary and Secondary Education in March 
2006 to gather additional information on state efforts to monitor and 
evaluate SES. Rhode Island invited us to attend two meetings the state 
held with districts implementing SES and providers serving students in 
Rhode Island, during which SES challenges, ways to improve 
implementation, and state efforts to evaluate providers were discussed. 

Provider Interviews: 

In total, we conducted interviews with 22 providers, including 15 
providers during the site visits and 7 providers operating in multiple 
states. The Education Industry Association assisted our efforts to 
contact multi-state providers, and most of the multi-state providers we 
interviewed were association members.[Footnote 34] Multi-state provider 
interviews were conducted between November and December 2005. 

Through all of our provider interviews, we collected information on 
provider efforts to increase participation in SES, align services with 
state standards and district curriculum, and communicate with parents 
and schools to ensure students are receiving needed services. We also 
collected information on students served, tutor and program 
characteristics, and provider challenges to SES implementation. While 
the providers we interviewed reflect some variety in provider 
characteristics, our selections were not intended to be representative. 
Thus, the findings from our interviews cannot be used to make 
inferences about all providers. 

Education Data Analyses: 

We analyzed state data submitted to Education through the NCLBA 
Consolidated State Performance Reports (CSPR) for school years 2002- 
2003, 2003-2004, and 2004-2005. State reports from all 3 years included 
the number of students receiving SES and the number of schools those 
students attended, and state reports from 2003-2004 and 2004-2005 also 
included the number of students eligible for SES. Data from the 2003- 
2004 CSPRs were used to assist our analysis of SES participation. To 
assess the reliability of the 2003-2004 data, we performed a series of 
tests, which included checking to ensure that data were consistent, 
that subtotals added to overall totals and that data provided for 1 
year bore a reasonable relationship to the next year's data and to data 
reported elsewhere, including state education reports. We also spoke 
with Education officials about their follow-up efforts to verify the 
data. At the time of our review, Education was in the process of 
completing efforts to verify the 2003-2004 data. 

While we compared the 2004-2005 CSPR data to data obtained through our 
state and district surveys to further verify our data, we generally did 
not use the 2004-2005 CSPR data for our analysis.[Footnote 35] During 
this comparison analysis, where we found discrepancies or sought 
clarification, we followed up with state officials. In several states, 
officials revised the numbers that they had initially reported to us or 
to Education. On the basis of our review of these data, we determined 
these sources to be adequate for the purposes of our work. 

We also considered SES-related findings from Education studies, 
including the Evaluation of Title I Accountability Systems and School 
Improvement Efforts: Findings From 2002-03 (2005) and the National 
Assessment of Title I: Interim Report (2006). To ensure the findings 
from these studies were generally reliable, we reviewed each study's 
methodology, including data sources and analyses, limitations, and 
conclusions. In addition, in designing our state and district surveys, 
we reviewed SES-related survey questions used by Education in these 
studies. 

[End of section] 

Appendix II: Comments from the Department of Education: 

United States Department Of Education: 
Office Of Innovation And Improvement: 

July 14, 2006: 

Ms. Marnie S. Shaul: 
Director, Education, Workforce, and Income Securities Issues: 
Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Ms. Shaul: 

I am writing in response to your request for comments on the Government 
Accountability Office (GAO) draft report (GAO-06-758) entitled, 
"Education Actions Needed to Improve Local Implementation and State 
Evaluation of Supplemental Educational Services." I appreciate the 
opportunity to respond to the recommendations made in the report and to 
provide you with additional information on how the U.S. Department of 
Education (Department) is supporting effective implementation of the 
supplemental educational services (SES) provisions of Title I of the 
Elementary and Secondary Education Act of 1965, as amended by the No 
Child Left Behind Act of 2001 (NCLB). 

As you know, the Department takes very seriously the need for effective 
SES implementation at all levels. In a letter to all Chief State School 
Officers on May 15, 2006, Secretary Spellings stressed the need for the 
Department and States to work together to expedite the implementation 
of further improvements of SES. For example, we expect districts to 
notify eligible families about SES in a way that is timely, clear, and 
complete, as well as to spend the full amount of funds necessary to 
meet demand for SES from families. In the May 15 letter, the Secretary 
also directed States to help their districts become fully compliant 
with SES in 2006-07 by closely monitoring districts' actions and 
providing them with requisite technical assistance. In such cases where 
districts are not complying with the SES provisions, the Secretary made 
clear that we are fully prepared to take actions such as placing 
conditions on Title I grants, withholding funding, and, if appropriate, 
entering into compliance agreements. The Department believes that these 
actions, as well as other strategies and actions we are undertaking, 
explained in more detail below, will provide significant motivation and 
resources to help all districts and States be successful with their SES 
implementation. We appreciate the specific recommendations you make in 
the report and respond to each of them below. 

Your first three recommendations concern additional assistance to help 
States and districts implement SES more effectively, which is a primary 
goal of the Department and one we pursue with full commitment and 
dedication. Specifically, you recommend that the Department: (1) 
provide better guidance on SES parental notification letters, (2) 
collect and disseminate information on promising practices used by 
States to attract more providers and used by districts to improve 
parental notification letters and provider services, and (3) provide 
examples of how districts can involve schools and school officials in 
coordinating activities with providers. 

As you state in your report, the Department has provided States and 
districts with a sample parent notification letter in the June 2005 
edition of the SES Non-Regulatory Guidance. A team of Department staff, 
including staff from both the Office of Elementary and Secondary 
Education (OESE) and the Office of Innovation and Improvement (OII), 
developed this letter, in direct response to the need we saw in the 
field for more informative and higher-quality parent letters. We 
drafted a sample letter not only to contain all required information, 
but also to be as "parent-friendly" and easy to use as possible. 
However, we recognize that States and districts may need more 
information on parent outreach and communication. The Department has 
assigned to our Comprehensive Center on Innovation and Improvement the 
task of developing a technical assistance effort to help respond to the 
needs of States, districts, and community-based organizations to 
conduct effective outreach on SES issues. The Center will be developing 
this effort this summer and fall and will implement it in sites around 
the country during the 2006-07 school year. The Center's effort will 
include technical assistance in the areas of planning and implementing 
outreach, as well as providing sample tools for educators to use to 
effectively reach parents. Department officials also will be meeting 
with State Title I Directors at the end of July 2006, and in that 
forum, we will solicit further information on the types of technical 
assistance that would be most useful in improving parent outreach and 
communication, including on how the sample letter in the SES guidance 
might be improved. 

We will also continue to collect and share examples of districts that 
are effectively reaching out to parents and working well with 
providers. We believe that there are lessons to be learned from the 
districts that participated in the two SES pilots in 2005-06. For 
example, we invited Newport News Public Schools in Virginia to 
participate in a recent production of the Department's Education News 
Parents Can Use program that focused on SES topics. Newport News shared 
with a national audience of viewers the activities and actions it had 
taken to implement SES successfully in that district. 

We will also be disseminating promising practices that States use to 
attract providers through our State SES directors meeting, to be held 
this October. This is the third meeting that the Department has 
sponsored, in conjunction with the Council of Chief State School 
Officers (CCSSO) and with the support of the C.S. Mott Foundation, to 
assist State SES directors with SES implementation. (The meeting also 
supports successful collaboration between SES and the 21ST Century 
Community Learning Centers Program.) As part of this October meeting, 
we will feature a session on successful State actions that help ensure 
a diversity of providers and on-going, high-quality communication 
between States and providers. During this meeting, we will also feature 
a session exploring successful partnerships between districts, schools, 
and providers. This session will build on guidance that the Department 
has already offered in the area of school involvement in SES. In the 
Department's June 2005 revisions to the SES Non-Regulatory Guidance, we 
encouraged districts to enlist the help of their schools in reaching 
parents. For example, we suggested that districts could use back-to-
school nights to explain SES to parents and that districts should 
educate their teachers and principals about the SES provisions to make 
sure they are able to answer parents' questions. We will continue to 
highlight the important role that schools can play in the SES process 
as we provide technical assistance to the field on SES. 

Your second set of recommendations involves States' and districts' 
ability to fund SES sufficiently and provide services to the maximum 
number of students. As noted above, these matters were addressed in 
various ways in the May 15, 2006, letter. For example, the Secretary 
announced in this letter that the Department was expanding the SES 
pilot program that began last year that allowed four districts in 
Virginia to provide SES to students in schools in year one of 
improvement. We requested that States interested in participating in 
this program submit proposals by June 19. In the proposals, each State 
had to demonstrate that it met three key conditions related to NCLB 
implementation: timely adequate yearly progress notification, SES 
evaluation in progress, and State assessment systems in one of the top 
three review categories. States could propose up to seven districts to 
participate in the program. The Department is currently reviewing these 
proposals and anticipates making a decision about this pilot. 
Additionally, we are currently considering expanding the SES pilot that 
allows districts in need of improvement to apply to become SES 
providers. 

You also recommend that the Department clarify what States can do 
through the provider approval process to set parameters around program 
design and costs. In the Department's June 2005 revisions to the SES 
Non-Regulatory Guidance, we provide information to States about their 
role in establishing certain program design criteria for providers to 
meet. We discuss, for example, that a State could establish a range for 
student/teacher ratios or for the rates that providers charge for their 
programs. We also instruct States that they may develop a policy with 
regard to providers' use of financial incentives and other gifts. We 
note in the guidance that school districts may not impose requirements 
on providers' program designs because doing so would undermine States' 
authority to establish standards for provider approval. While, at this 
time, the Department has not provided additional guidance to all States 
on their authority in this area, we continue to respond to individual 
State questions on the matter. Additionally, it is a topic we will 
consider further addressing in our next set of revisions to the SES 
guidance. 

Your final set of recommendations discusses steps the Department can 
take to improve Federal and State monitoring of SES. The Department 
continues to improve its monitoring efforts of the Title I 
requirements, including SES. Additionally, as outlined in the 
Secretary's letter dated May 15, 2006, we urged States to work with 
local educational agencies (LEAs) in their efforts to comply with SES 
requirements and to closely monitor the LEAs' actions in this area. 


With regard to this recommendation, you specifically recommend that the 
Department require States to report information necessary to determine 
the percentage and amount of Title I funds spent by districts to 
provide SES. The Department recognizes the importance of collecting 
information on how much districts are spending on SES (as well as how 
much they are spending on public school choice-related transportation). 
In the 2006-07 data collection for the Education Data Exchange Network 
(EDEN)/EDFacts system, the Department is proposing that districts 
report on their SES and public school choice spending. This will give 
the Department data on spending trends across the country, as well as 
help us identify districts that are doing either particularly well, or 
are under-spending, on SES. 

Finally, you recommend providing States with technical assistance on 
how to evaluate the effects of SES on student achievement. The 
Department has recognized the importance of high-quality State 
evaluations for several years. As part of a grant to the American 
Institutes for Research, the Department supported the development of an 
issue brief to advise States on the evaluation process. (This issue 
brief is available at: [Hyperlink, 
http://www.tutorsforkids.org/documents/SES] Evaluation Issue Brief 
004.pdf). The Department and the authors of this issue brief have 
shared it widely and presented it at several national conferences. 
Additionally, the Department has assigned the Comprehensive Center on 
Innovation and Improvement the task of providing additional assistance 
in this area. The Center will work with the authors of the original 
issue brief to update it by this September and disseminate it to the 
field, and the Center has hired the authors as technical advisors so 
that they can provide ongoing technical assistance to States in the 
area of evaluation. Finally, this fall at the Department's conference 
with the CCSSO and the C.S. Mott Foundation, we will hold several in-
depth sessions on conducting evaluations to provide more technical 
assistance to States. 

Thank you again for taking the time to research and report on the SES 
provisions of NCLB. The Department values the work that you have done 
to provide rich and compelling information about the current status of 
SES around the country. We will use the findings and recommendations 
made in this report to improve our technical assistance to States and 
districts and ultimately to improve the quality of SES implementation 
around the country. 

Sincerely, 

Signed by: 

Christopher J. Doherty: 
Acting Assistant Deputy Secretary: 
Office of Innovation and Improvement: 

[End of section] 

Appendix III: GAO Contacts and Staff Acknowledgments: 

GAO Contacts: 

Marnie S. Shaul (202) 512-7215 shaulm@gao.gov Cornelia M. Ashby (202) 
512-7215 ashbyc@gao.gov: 

Acknowledgments: 

Cindy Ayers, Assistant Director, and Rachel Frisk, Analyst-in-Charge, 
managed this assignment and made significant contributions to all 
aspects of this report. Cathy Roark, Ted Burik, and David Perkins also 
made significant contributions. Kevin Jackson, Jean McSween, Jim 
Ashley, and Jerry Sandau provided methodological expertise and 
assistance; Rachael Valliere assisted with message and report 
development; and Rasheeda Curry made contributions during study design. 
In addition, Jessica Botsford assisted in the legal analysis. 

[End of section] 

Related GAO Products: 

No Child Left Behind Act: Assistance from Education Could Help States 
Better Measure Progress of Students with Limited English Proficiency. 
GAO-06-815. Washington, D.C.: July 26, 2006. 

No Child Left Behind Act: States Face Challenges Measuring Academic 
Growth that Education's Initiatives May Help Address. GAO-06-661. 
Washington, D.C.: July 17, 2006. 

No Child Left Behind Act: Most Students with Disabilities Participated 
in Statewide Assessments, but Inclusion Options Could Be Improved. GAO- 
05-618. Washington, D.C.: July 20, 2005. 

No Child Left Behind Act: Education Needs to Provide Additional 
Technical Assistance and Conduct Implementation Studies for School 
Choice Provision. GAO-05-7. Washington, D.C.: December 10, 2004. 

No Child Left Behind Act: Improvements Needed in Education's Process 
for Tracking States' Implementation of Key Provisions. GAO-04-734. 
Washington, D.C.: September 30, 2004. 

No Child Left Behind Act: Additional Assistance and Research on 
Effective Strategies Would Help Small Rural Districts. GAO-04-909. 
Washington, D.C.: September 23, 2004. 

Disadvantaged Students: Fiscal Oversight of Title I Could Be Improved. 
GAO-03-377. Washington, D.C.: February 28, 2003. 

Title I Funding: Poor Children Benefit Though Funding Per Poor Child 
Differs. GAO-02-242. Washington, D.C.: January 31, 2002. 

FOOTNOTES 

[1] In this report, we refer to Title I, Part A of ESEA as "Title I." 
Other Parts of Title I (Parts B, C, and D) are targeted at specific 
populations or purposes and are commonly referred to by their program 
names, such as Even Start. 

[2] State testing has been phased in, whereby from 2002-2004 states 
were required to administer tests at least once in grades 3-5, 6-9, and 
10-12. Beginning in 2005-2006, states must administer annual tests in 
grades 3-8, and at least once in grades 10-12. Testing for science 
proficiency will begin in 2007-2008. 

[3] Measures for elementary and middle schools may include, but are not 
limited to, attendance, grade-to-grade retention rates, or changes in 
the percentage of students completing gifted and talented programs. 

[4] In addition, NSLP allows certain schools and districts with high 
percentages of low-income families to certify students as eligible for 
free and reduced price lunches about once every 4 years. These 
alternatives to the traditional requirements for annual certification, 
known as "Provision 2" and "Provision 3," reduce local paperwork and 
administrative burden. Education has determined that, for purposes of 
identifying students as "economically disadvantaged" for SES, school 
officials may deem all students in Provision 2 and Provision 3 schools 
as "economically disadvantaged." See policy memo: U.S. Department of 
Agriculture, Food and Nutrition Service, "No Child Left Behind Act 
(NCLB)," School Meals (Washington, D.C.: Feb. 2003), 
http://www.fns.usda.gov/end/governance/policy-memos/2002-02-20.htm 

[5] Only 1 percent of students eligible for school choice elected to 
transfer to another school during the 2003-2004 school year. See GAO, 
No Child Left Behind Act: Education Needs to Provide Additional 
Technical Assistance and Conduct Implementation Studies for School 
Choice Provision, GAO-05-07 (Washington, D.C.: Dec. 10, 2004). 

[6] A district calculates the Title I per pupil allocation by dividing 
its total Title I, Part A allocation by the number of children residing 
within the district aged 5-17 who are from families below the poverty 
level, as determined by the most recent Census Bureau estimates from 
the Department of Commerce. 

[7] Certain states did not submit SES recipient information to 
Education through their NCLBA Consolidated State Performance Reports 
for all years. See notes to figure 1 for details. Further, Education 
did not collect data on the number of students eligible for SES in 2002-
2003, and therefore an estimate of the SES participation rate is 
unavailable for that year. 

[8] In addition to our analysis, the Center on Education Policy 
reported that within districts required to offer SES, the percentage of 
all schools required to offer SES rose from 27 percent of schools in 
2002-2003 to 65 percent of schools in 2005-2006. See the Center on 
Education Policy, From the Capital to the Classroom, Year 4 of the No 
Child Left Behind Act (Washington D.C.: March 2006), for more 
information. 

[9] These district estimates have a margin of error that exceeds plus 
or minus 8 percentage points. See table 9 in appendix I for more 
information. 

[10] We did not review the academic achievement records of students 
receiving SES or independently verify this information obtained through 
the district survey. 

[11] Many of the district estimates included in this paragraph have a 
margin of error that exceeds plus or minus 8 percentage points. See 
table 9 in appendix I for more information. 

[12] In addition to our analysis, the Center on Education Policy 
reported that that as of August 2005, more than half of approved SES 
providers were private, for-profit entities. See the Center on 
Education Policy, From the Capital to the Classroom, Year 4 of the No 
Child Left Behind Act (Washington D.C.: March 2006), for more 
information. 

[13] GAO previously reported that some states have difficulty notifying 
schools of their status in meeting proficiency goals in a timely 
fashion in part because of the time involved in identifying and 
correcting errors in student assessment data. See GAO, No Child Left 
Behind Act: Improvements Needed in Education's Process for Tracking 
States' Implementation of Key Provisions, GAO-04-734 (Washington D.C.: 
Sept. 30, 2004), for more information. 

[14] This district estimate has a margin of error that exceeds plus or 
minus 8 percentage points. See table 9 in appendix I for more 
information. 

[15] This district estimate has a margin of error that exceeds plus or 
minus 8 percentage points. See table 9 in appendix I for more 
information. 

[16] GAO previously reported that geographic isolation created 
difficulties for rural districts in implementing SES. Specifically, 
rural district officials stated that traveling long distances to meet 
providers was not a viable option and use of online providers was 
challenging in some small rural districts where it was difficult to 
establish and maintain Internet service. See GAO, No Child Left Behind 
Act: Additional Assistance and Research on Effective Strategies Would 
Help Small Rural Districts, GAO-04-909 (Washington D.C.: Sept. 23, 
2004), for more details. 

[17] This district estimate has a margin of error that exceeds plus or 
minus 8 percentage points. See table 9 in appendix I for more 
information. 

[18] The other states included five that did not respond to the 
applicable survey question. A similar number of states did not respond 
to the other survey questions discussed in this paragraph. 

[19] In addition to our analysis, the Center on Education Policy case 
studies also found that in some cases, approved providers that 
initially expressed interest in serving a certain district later 
decided not to provide services because too few students enrolled. See 
the Center on Education Policy, From the Capital to the Classroom, Year 
4 of the No Child Left Behind Act (Washington D.C.: March 2006), for 
more information. 

[20] Similar to states, districts reported on monitoring activities 
through our survey in early 2006; 
therefore, while some had already begun to monitor providers, others 
reported on their plans to monitor providers before the end of the 2005-
2006 school year. The increases in monitoring reflect both those 
districts that had already begun monitoring activities and those 
planning monitoring activities for later in 2005-2006. Because our data 
were collected during the middle of the 2005-2006 school year, we do 
not know whether the estimated percentage of districts that planned to 
complete these activities before the end of the year did so. 

[21] The district estimate has a margin of error that exceeds plus or 
minus 8 percentage points. See table 9 in appendix I for more 
information. 

[22] Further, the federal SES guidance suggests that student 
performance can be measured in a variety of ways, including with 
provider assessments or state or district standardized assessments. 
However, the guidance does not discuss the limitations of using 
different approaches to conduct such evaluations. 

[23] At the time of our survey, several additional states, including 
Louisiana and Pennsylvania, were in the process of drafting an SES 
evaluation report that would assess the impact of SES providers serving 
students in their states in previous years, but the reports were not 
yet available to the public. 

[24] In addition, at least two school districts, Chicago and 
Minneapolis, have taken steps to evaluate SES, but we found that their 
evaluation efforts have also not yet produced a conclusive assessment 
of SES's impact on student academic achievement. 

[25] Only one state reported withdrawing approval from one of its 
providers because that provider's program was generally ineffective. 
However, this provider's program was found to be ineffective because 
the provider did not deliver services to all of the students it 
enrolled. This state also indicated that it had not yet completed an 
evaluation of SES's effect on student academic achievement. 

[26] States have only reported the number of students eligible for SES 
since the 2003-2004 CSPR. Also, for the 2004-2005 CSPR, Education gave 
states the option to also report the number of students who applied for 
SES. 

[27] States audited were Delaware, Illinois, Indiana, Michigan, New 
Jersey, and Nevada. 

[28] California districts audited were Los Angeles Unified, Oakland 
Unified, Salinas Union High School, San Diego City Schools, and 
Stockton Unified. 

[29] As discussed earlier, OIG found all six of the states it visited 
to be deficient with respect to parent notifications. In addition, in 
our analysis of OESE Title I monitoring reports issued as of June 2006, 
we found that OESE cited 9 of the 40 states it had visited for SES non- 
compliance with respect to district parent notifications. 

[30] See the Center on Education Policy, From the Capitol to the 
Classroom: Year 4 of the No Child Left Behind Act (Washington D.C.: 
March 2006). 

[31] Ibid. 

[32] We compared our state survey data to data on SES reported by 
Education in the Evaluation of Title I Accountability Systems and 
School Improvement Efforts: Findings From 2002-03 (Washington, D.C.: 
Nov. 2005) and the National Assessment of Title I: Interim Report 
(Washington, D.C.: Feb. 2006). We also compared our survey data to data 
on SES reported by the Center on Education Policy in From the Capital 
to the Classroom: Year 3 of the No Child Left Behind Act (Washington, 
D.C.: March 2005) and From the Capital to the Classroom: Year 4 of the 
No Child Left Behind Act (Washington, D.C.: March 2006). 

[33] We compared our district survey data to data on SES reported by 
Education in the Evaluation of Title I Accountability Systems and 
School Improvement Efforts: Findings From 2002-03 (Washington, D.C.: 
Nov. 2005) and the National Assessment of Title I: Interim Report 
(Washington, D.C.: Feb. 2006). We also compared our survey data to data 
on SES reported by the Center on Education Policy in From the Capital 
to the Classroom: Year 3 of the No Child Left Behind Act (Washington, 
D.C.: March 2005) and From the Capital to the Classroom: Year 4 of the 
No Child Left Behind Act (Washington, D.C.: March 2006). 

[34] The seven multi-state providers we spoke with were Babbage Net 
School, Cambridge Educational Services, Catapult Learning, Huntington 
Learning Center, Newton Learning, Platform Learning, and University 
Instructors. 

[35] At the time of our survey, seven states were unable to provide 
2004-2005 data on SES recipients and students eligible to receive SES. 
By the time Education required states to submit the 2004-2005 CSPR 
reports in March 2006, six of these states had provided this data to 
Education; we, therefore, used this data to supplement their responses 
to our survey. The seventh state, New Jersey, was unable to provide 
this data to Education by the time 2004-2005 CSPRs were due; 
therefore, New Jersey data is not included in the 2004-2005 
participation analysis included in this report. 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; 
evaluates federal programs and policies; 
and provides analyses, recommendations, and other assistance to help 
Congress make informed oversight, policy, and funding decisions. GAO's 
commitment to good government is reflected in its core values of 
accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, 

NelliganJ@gao.gov 

(202) 512-4800 

U.S. Government Accountability Office, 

441 G Street NW, Room 7149 

Washington, D.C. 20548: