This is the accessible text file for GAO report number GAO-03-11 
entitled 'Public Schools: Insufficient Research to Determine 
Effectiveness of Selected Private Education Companies' which was 
released on October 29, 2002.



This text file was formatted by the U.S. General Accounting Office 

(GAO) to be accessible to users with visual impairments, as part of a 

longer term project to improve GAO products’ accessibility. Every 

attempt has been made to maintain the structural and data integrity of 

the original printed product. Accessibility features, such as text 

descriptions of tables, consecutively numbered footnotes placed at the 

end of the file, and the text of agency comment letters, are provided 

but may not exactly duplicate the presentation or format of the printed 

version. The portable document format (PDF) file is an exact electronic 

replica of the printed version. We welcome your feedback. Please E-mail 

your comments regarding the contents or accessibility features of this 

document to Webmaster@gao.gov.



Report to the Ranking Minority Member, Subcommittee on the

District of Columbia, Committee on Appropriations,

House of Representatives:



United States General Accounting Office:



GAO:



October 2002:



Public Schools:



Insufficient Research to Determine Effectiveness of Selected Private 

Education Companies:



Selected Private Education Companies:



GAO-03-11:



Contents:



Letter:



Results in Brief:



Background:



Education Management Companies Have Multifaceted Programs; District 

Schools Varied In Their Implementation:



Limited Research Exists of the Effectiveness of These Companies’ 

Programs:



Agency Comments:



Appendix I: Objectives, Scope, and Methodology:



Objectives:



Scope and Methodology:



Appendix II: Studies and Other Information Sources Considered:



Appendix III: Comments from the Department of Education:



Appendix IV: GAO Contacts and Staff Acknowledgments:



GAO Contacts:



Acknowledgments:



Related GAO Products:



Table:



Table 1: Profiles of Three Education Management Companies Operating in 

the District of Columbia:



United States General Accounting Office:



Washington, DC 20548:



October 29, 2002:



The Honorable Chaka Fattah

Ranking Minority Member

Subcommittee on the District of Columbia

Committee on Appropriations

House of Representatives:



Dear Mr. Fattah:



In recent years, local school districts and traditional public schools 

have taken various initiatives to improve failing schools. For example, 

school districts and charter schools are increasingly contracting with 

private, for-profit companies to provide a range of education and 

management services to schools. To date, there has been debate 

regarding the effectiveness of such companies in managing public 

schools.



These companies generally offer schools services in areas such as 

school organization, instruction, technology, and professional 

development. In the District of Columbia, some public schools currently 

contract with three such companies: Edison Schools, Mosaica Education, 

and Chancellor Beacon Academies. As agreed with your office, we 

(1) identified the characteristics of their programs and determined the 

extent to which District schools managed by them have used their 

programs and (2) determined what is known about the effectiveness of 

these companies’ educational programs, as measured primarily by student 

achievement.



To address these issues, we reviewed relevant research on charter and 

traditional public schools managed by for-profit educational management 

companies as well as documents and materials provided by the companies. 

In addition, we observed an on-site review of one school’s program 

which was conducted for District oversight authorities. We also 

interviewed officials of the companies that manage District public 

schools, officials of the District’s oversight authorities, and 

representatives of the schools, as well as officials of the Department 

of Education, other education experts and advocates. Finally, we 

reviewed evaluations concerning the three companies operating in the 

District that met the following criteria: included comparison groups 

and measurement of performance over time, and focused on academic 

achievement, parental satisfaction, parental involvement, or school 

climate. We assessed the quality of the evaluations’ research designs, 

reviewed them for threats to validity and determined whether we had 

confidence in their conclusions. We conducted our work between January 

and September 2002 in accordance with generally accepted government 

auditing standards.



Results in Brief:



Edison, Mosaica, and Chancellor Beacon have programs that consist of 

both management services, such as personnel, and educational services, 

which they offer to schools across the nation; in the District, most of 

the schools managed by these companies have either adopted selected 

elements of their companies’ programs or chosen other educational 

programs. Each company provides services such as curriculum, 

assessments, parental involvement opportunities, and student and family 

support. They also offer a variety of organizational options, including 

smaller class and school sizes, as well as longer school days and 

school years. All of the companies allow their schools some flexibility 

in adapting their programs to local circumstances. The extent to which 

the District schools implemented all the elements of these companies’ 

educational programs varied. For example, 6 of the 10 schools managed 

by these companies had either partially implemented the company’s 

curriculum or had not implemented that curriculum at all. Some schools 

have opted to customize the company’s educational program; other 

schools have left in place the educational program of a company that 

formerly managed them. In school year 2001-02, all 10 District schools 

managed by these companies were charter schools with predominantly poor 

and minority student populations; most enrolled elementary and middle 

school students.



Little is known about the effectiveness of these companies’ programs on 

student achievement, parental satisfaction, parental involvement, or 

school climate because few rigorous studies have been conducted. While 

the companies publish year-to-year comparisons of standardized test 

scores to indicate that students in schools they manage are making 

academic gains, they do not present data on comparable students who are 

not in their programs, a necessary component of a program effectiveness 

study. An effectiveness study attempts to isolate the effect a program 

has on specific outcomes by, among other things, comparing outcomes for 

students in company managed schools with those of a comparable group of 

traditional public school students, tracking students over time, 

testing students before and after exposure to the company’s program, 

and controlling for differences between these groups. Of five studies 

we identified concerning the three companies operating in the District, 

four focused only on one company. Moreover, of the five studies, one--

based on one school in Florida--was rigorous enough to allow confidence 

in the findings about the program’s effectiveness in that school. This 

particular study, using two analytical techniques, found no difference 

between students in the company’s program and other students. The 

remaining studies had methodological limitations that precluded 

assessments about a company’s effect on student achievement, parental 

satisfaction, parental involvement, and school climate. Additional 

research on one company’s program is planned by an organization with 

experience in conducting educational evaluations.



Background:



During the last decade, a new kind of entity has emerged in public 

education: the for-profit provider of education and management 

services. Historically, school districts have contracted with private 

companies for noninstructional services, such as transportation and 

food service, and have also relied on contractors in some cases to 

provide limited instructional services to specified populations. Until 

recently, public schools have generally not contracted for the 

comprehensive programs of educational and management services that 

these companies typically offer. In recent years, the options available 

to public schools considering contracting with private companies have 

steadily grown. Today, approximately 20 major companies manage public 

schools.[Footnote 1] Nationally, it is estimated that these companies 

as well as other smaller companies serve over 300 schools out of the 

nation’s approximately 92,000 public schools. Although these companies 

manage public schools at all grade levels, most such privately managed 

public schools are elementary and middle schools. In these public 

schools, companies generally provide the same kinds of educational and 

management services that school districts do for traditional public 

schools. Educational services typically include a curriculum as well as 

a range of services designed to enhance or support student achievement, 

such as professional development opportunities for teachers, 

opportunities for parental involvement and school environments that aim 

to facilitate student support. Management services typically include 

personnel, payroll, and facilities management. Although these are the 

services that are typically offered to schools, companies also may 

adapt their services to respond to the preferences or needs of 

individual schools. For example, while some companies offer a 

particular curriculum or educational approach, others appear more 

willing to work with the curriculum the school or school district has 

already adopted.[Footnote 2]



Typically, companies provide their services to public schools in one of 

two ways. First, they can contract directly with school districts to 

manage traditional public schools; such schools are known as “contract 

schools.” Second, they can manage charter schools, which are public 

schools that receive a degree of autonomy and freedom from certain 

school district requirements in exchange for enhanced accountability. 

Generally, charter schools are run by individual boards of trustees, 

which in most states and the District of Columbia have the authority to 

decide whether to contract with a private company.[Footnote 3] Both 

contract schools and charter schools remain public schools, however, 

and are generally subject to federal and state requirements for public 

schools in areas such as the application of standardized tests and 

special education.



While the reasons public schools turn to private companies vary, the 

potential to increase student achievement appears to be one factor. In 

particular, according to certain experts and company officials we spoke 

to, school districts that seek a company’s help often do so with the 

expectation of raising achievement in struggling or failing schools. 

While management services appear to be especially important for charter 

schools that contract with such companies, charter schools also 

consider the potential to raise student achievement or a particular 

educational approach consistent with the school’s mission, according to 

school officials and experts we spoke with. Both types of schools that 

seek these companies’ assistance--struggling schools and charter 

schools--appear concentrated in urban areas. Further, several of the 

major companies reportedly serve a predominantly disadvantaged urban 

and minority student population.



Recent changes in federal law have implications for the role played by 

these companies in public schools. The No Child Left Behind Act of 

2001[Footnote 4] requires that schools that fail to meet state student 

achievement standards for 5 consecutive years must be restructured by 

implementing one or more alternative governance actions. One of the 

alternatives available to states and districts is to contract with an 

education management company.[Footnote 5]



Three companies currently operate in the District of Columbia: Edison 

Schools, Mosaica Education, and Chancellor Beacon Academies. Edison 

began operating its first District school in 1998, and Mosaica and 

Chancellor Beacon first contracted with the District schools they 

manage in 2001. Throughout this report, these companies will generally 

be discussed in this order.



Mergers and acquisitions are common among such companies. In 2001, 

Edison acquired nine schools nationwide through a merger with LearnNow. 

In the same year, Mosaica acquired nine schools nationwide through its 

acquisition of Advantage Schools. In addition, Chancellor and Beacon 

merged into a single company. Such changes can have several outcomes: 

in some cases, the company may operate schools that continue to use the 

educational program of another company; in other cases, the school may 

consider adopting the educational program of the new company or 

terminating the contract.



Education Management Companies Have Multifaceted Programs; District 

Schools Varied in Their Implementation:



The companies that operate public schools in the District of Columbia 

offer management and educational services as part of their programs; 

the extent to which District schools managed by these companies 

implemented all of the components of the companies’ programs varied. 

All of these companies offer programs that include management and 

educational services, such as curricula that integrate technology and 

professional development opportunities for teachers. Of the 10 District 

schools managed by these companies, 4 had completely implemented their 

company’s program. In school year 2001-02, all 10 District schools 

managed by these companies were charter schools with predominantly poor 

and minority student populations; most enrolled elementary and middle 

school students. Similar to traditional public schools, the District 

schools managed by these companies were required to be open to all 

students, up to their enrollment limits, and to meet District standards 

in areas such as health, safety, standardized testing, and compliance 

with federal special education requirements.



All Three Companies Offer Management and Educational Services to 

Schools Nationally:



The three for-profit companies that operate in the District of 

Columbia[Footnote 6]--Edison, Mosaica, and Chancellor Beacon--share 

common elements in terms of the management and educational services 

they offer to schools nationwide as well as those company officials 

described as distinctive. Each of the three companies generally offers 

similar management services. For example, all three offer management 

services such as personnel, payroll and facilities management, services 

that can be important for charter schools. In addition, the three 

companies employ some common approaches designed to improve student 

achievement. All three companies offer an extended school day and year. 

All three integrate technology in their educational programs. For 

example, all three offer students access to classroom computers. 

Similarly, all organize schools into smaller units to facilitate their 

tracking of students’ progress. All three provide summer training to 

teachers as well as other forms of professional development. 

Additionally, all have activities designed to involve and support 

parents and students. For example, each company uses parent 

satisfaction surveys. Experts we spoke to noted that these same 

approaches were being used in some other public schools. Finally, 

officials of all three companies stated that their companies 

contributed positively to school climate--a sense of mission and an 

environment conducive to learning--and cited aspects of school climate 

such as a safe and orderly school environment and teacher motivation. 

In addition to the characteristics they had in common, company 

officials identified others they believed were distinctive. These 

include, for example, their programs’ curriculum and instruction as 

well as the ability to provide economies of scale, develop community 

partnerships, and provide strong administrative support. As Table 1 

shows, all three companies provided their services to schools in 

multiple states in 2001-02.



Table 1: Profiles of Three Education Management Companies Operating in 

the District of Columbia:



Year company was established; Edison: 1992; Mosaica: 1997; Chancellor

Beacon: 2001[A].



Number of states[B] where company operated schools in school year 2001-

02; Edison: 24; Mosaica: 8; Chancellor Beacon: 11.



Number of schools company operated nationwide in school year 2001-02; 

Edison: 136; Mosaica: 21; Chancellor Beacon: 82.



Number of schools company operated in the District of Columbia in 

school year 2001-02; Edison: 6; Mosaica: 2; Chancellor Beacon: 2.



[A] Year of merger of Chancellor Academies (originally founded in 1999) 

and Beacon Education Management (founded in 1993).



[B] Including the District of Columbia.



Source: Edison, Mosaica, and Chancellor Beacon.



[End of table]



According to Edison officials, its program has a number of distinctive 

characteristics.[Footnote 7] The first of these is its curriculum, 

which emphasizes basic skills, especially reading as the basis for 

future learning. It also includes enrichment in areas such as world 

languages (e.g., Spanish) and art. Edison’s basic skills curriculum 

includes components developed by Edison, such as a remedial reading 

program, and other components that Edison states are supported by 

research, such as Chicago Math and the Success for All reading program. 

Instructional methods are a second characteristic of Edison’s program. 

Edison schools use a variety of instructional methods. One of these, 

direct instruction, relies on repetition and drill. Other methods use 

projects, small groups, and individualized lessons. A third 

characteristic of Edison schools is their use of assessments. According 

to Edison officials, their program uses frequent assessments and the 

results of these assessments are promptly provided to teachers to 

assess student needs and provide appropriate additional help. “Systems 

and scale” is another key characteristic of Edison schools according to 

company officials. The company views its schools as part of a national 

system linked by a common purpose, and because of the system’s size, 

the company says it is able to purchase supplies at lower costs.



Mosaica officials also identified certain distinctive characteristics 

of their company’s program.[Footnote 8] The first is the program’s 

curriculum, which has two parts. According to Mosaica officials, its 

morning program features instruction in traditional subjects such as 

reading and math. In the afternoon, students use Paragon--Mosaica’s own 

curriculum. According to company officials, Paragon stresses 

multidisciplinary learning, uses projects to emphasize the humanities, 

and recognizes students’ different learning styles. For example, 

students may use their reading, math, and social studies learning to 

build a pyramid or a Viking ship and thus study a period of history. 

According to company officials, projects accommodate a variety of 

learning styles--for example, some students learn visually, others by 

performing. Community involvement is a second key characteristic of 

Mosaica’s program. Company officials say that Mosaica brings community 

support into the school by networking with various community 

organizations. According to company officials, this provides its 

schools with access to additional resources.



Chancellor Beacon officials also identified distinctive 

characteristics of their program. One is their willingness to customize 

their educational program to meet the needs and preferences of local 

schools. For example, in response to community interest, some 

Chancellor Beacon schools feature a cultural heritage element in the 

curriculum while one of its schools emphasizes the environment. 

Chancellor Beacon’s own curriculum was recently finalized in July 2002 

and is based on an integration of the curricula of Chancellor and 

Beacon before they merged. One component of its curriculum is Core 

Knowledge--a program that expects students to master specific content 

in language arts, history, geography, math, science and fine arts. 

Other components emphasize ethics, morality and community volunteerism. 

A second key characteristic of Chancellor Beacon’s program is its 

operational support, according to company officials. These officials 

told us that in focusing on operational support, Chancellor Beacon 

allows schools to focus on academics.



While the Chancellor Beacon program emphasizes customization as a key 

characteristic, the other two companies also allow schools to modify 

their programs. For example, in its reading program, Edison allows 

schools some flexibility regarding what books to read and in what 

order. In addition, up to one-fourth of its curriculum can be 

determined by the local school. Similarly, Mosaica allows its schools 

to use different approaches or materials in their morning session.



Of the 10 District Schools, 4 Completely Implemented the Company’s 

Educational Program While 6 Schools Selected Elements of Their 

Companies’ Programs or Chose Other Programs:



While all of the 10 District schools managed by the companies during 

the 2001-02 school year obtained management services from these 

companies, the schools were more selective in implementing the 

companies’ educational programs. Of the 10 District schools, 4 have 

completely implemented the companies’ educational programs and 6 have 

adopted selected elements of their companies’ programs or chosen other 

programs, typically those of a previous company. A key factor that 

helps explain the difference between the programs the companies offer 

and what has been implemented by District schools is that recent 

mergers and acquisitions have led to changes in management companies in 

these 6 schools; these schools have generally left in place the 

educational programs of the companies that formerly managed them.



Four schools, all managed by Edison, implemented the company’s 

educational program completely, according to company officials. These 

4 schools all opened in 1998 as the result of a partnership between 

Friendship House, a nonprofit community organization serving District 

children and youth since 1904, and Edison. According to a Friendship 

House official, these schools completely implemented Edison’s program 

because they saw it as complementing their own goals. One of these 

schools--a high school--has supplemented the Edison program by 

developing a program to expose certain students to college through 

campus visits and workshops for parents.



Six District schools adopted selected elements of their companies’ 

educational programs or chose other educational programs. These 

6 schools include 2 schools managed by Edison, 2 by Mosaica, and 2 by 

Chancellor Beacon. All 6 schools have had recent changes in management 

companies as a result of mergers or acquisitions.



The 2 schools that received services from Edison have opted to retain 

the curriculum already in place at the schools, rather than adopt the 

Edison program.[Footnote 9] In 2001, Edison bought LearnNow, the 

company that formerly provided services to the 2 schools. According to 

an Edison official knowledgeable about the schools formerly managed by 

LearnNow, the primary difference between the companies’ curricula was 

in elementary language arts, for which LearnNow preferred a different 

reading program than Success for All, which the Edison program uses in 

its other schools.



The 2 schools managed by Mosaica have adopted some elements of the 

company’s educational program, and have plans to adopt more by 2003. In 

2001, Mosaica bought Advantage, the company that formerly managed these 

schools. Both schools retained an instructional approach put in place 

by the previous company. This approach--direct instruction--emphasizes 

drill and repetition. By school year 2003, both schools expect to use 

direct instruction during the morning session and Paragon in the 

afternoon.



The 2 schools managed by Chancellor Beacon both had distinct curricula 

in place before being managed by this company; one has combined its 

existing curriculum with elements of Chancellor Beacon’s, and the other 

has left its existing curriculum in place. The school that has adopted 

elements of Chancellor Beacon’s curriculum has done so by integrating 

the company’s language arts and math curriculum with the school’s 

existing curriculum, according to company officials.[Footnote 10] This 

school, which serves at-risk youth, had a curriculum called 

expeditionary learning, which focuses on learning through field trips 

and experiences. The other Chancellor Beacon school opted to retain its 

existing basic-skills curriculum, relying instead on the company’s 

management services and selected educational services, such as 

assessments. Chancellor Beacon officials support the schools’ choices 

regarding what company components to adopt.



Company and school officials identified several reasons why these 

6 schools did not completely implement the current company’s 

educational program, opting instead to continue with an existing 

curriculum. These included continuity for students, the company’s 

flexibility with regard to local customization, and the right of 

charter school boards to make broad curriculum decisions.[Footnote 11]



The 10 schools in the District managed by these companies shared 

certain characteristics and served similar student populations in 2001-

02. All were public charter schools governed by their own boards and 

accountable to District oversight authorities. Most (9) were combined 

schools spanning elementary and middle school grades. As public 

schools, they were required to accept any student who applied, up to 

their enrollment limit. Their student populations were substantially 

minority and poor: 92 to 

100 percent African American and 48 to 95 percent receiving free or 

reduced school lunch.[Footnote 12] All served some students with 

special needs, such as learning disabilities: in 9 of the schools, the 

percentage ranged from 5 to 13 percent, and in one school, 32 percent 

of the student population had special needs. All but one served no or 

very few students with limited English proficiency; at the remaining 

school, students with limited English proficiency represented about 12 

percent of all students enrolled.



Limited Research Exists of the Effectiveness of These Companies’ 

Programs:



Little rigorous research exists on the effectiveness of the three 

educational management companies--Edison, Mosaica, and Chancellor 

Beacon--in the schools they manage across the country; as a result, we 

cannot draw conclusions about the effect that these companies’ programs 

have on student achievement, parental satisfaction, parental 

involvement, or school climate. Students in company managed schools 

have demonstrated academic progress, but more research is needed to 

determine if this improvement is directly the result of the companies’ 

programs and if this progress is different from that of comparable 

students in traditional public schools. We reviewed five studies that 

addressed student achievement, but only one was conducted in a way that 

allows an assessment of the effect the company’s program had on student 

achievement in one school. The remaining studies had methodological 

limitations that precluded such assessments. In an effort to learn more 

about effectiveness, Edison has recently commissioned RAND, a nonprofit 

research organization that has evaluated educational reforms, to 

complete a study to assess its program’s impact.



Determining the effect of an educational company’s program can be 

challenging for researchers. Ideally, evaluations of program 

effectiveness should involve a comparison of outcomes for one group 

exposed to a particular program with outcomes of a second group not 

exposed to the program. Some evaluations assign participants randomly 

to one group or the other to increase the likelihood that the two 

groups are roughly equivalent on all characteristics that could affect 

outcomes. This technique of random assignment is often problematic in 

educational research because public school enrollment is generally 

based on residency requirements. Therefore the most common way to 

compare student achievement results from two different groups of 

students is to ensure the groups are similar in a number of ways, 

including socioeconomic status, ethnicity, and performance on prior 

academic assessments. In addition to controlling for the effects of 

these background characteristics, it is critical to follow the 

performance of students over time, preferably before any group has been 

exposed to the program, and at least one point thereafter.[Footnote 13] 

It is also beneficial to analyze individual student data, rather than 

grade or school-level averages, to account for individual differences 

and to factor in the effects of missing data.



Within the context of rigorous educational program evaluations, various 

measurements can be used to capture a student’s performance on 

standardized tests. According to several experts, it is important to 

examine both the percent of students in a particular grade or school 

making yearly gains and the distribution of these gains across ability 

levels to ensure that students of all achievement levels are 

demonstrating academic growth. Another point of interest relates to the 

length of time students participate in a particular program. Some 

experts claim that students will exhibit greater gains the longer they 

participate in a program. However, it is particularly challenging to 

design studies that address this claim, because educational companies 

are still a relatively new phenomenon.



We identified five studies concerning the three companies operating in 

the District that met the criteria for our review: inclusion of 

comparison groups, measurement over time, and focus on academic 

achievement, parental satisfaction, parental involvement, or school 

climate.[Footnote 14] All of the studies addressed the effectiveness of 

schools managed by Edison. One study also addressed the effectiveness 

of schools managed by all three private companies--Edison, Mosaica, and 

Chancellor Beacon.[Footnote 15] We were unable to identify any rigorous 

studies that included analysis of District public schools managed by 

any of these three companies.[Footnote 16] Of the studies included in 

our review, four studies addressed only outcomes related to student 

achievement, while one study addressed student achievement and other 

outcomes such as parental satisfaction and school climate.[Footnote 17]



Only one of the studies, A Longitudinal Study of Achievement Outcomes 

in a Privatized Public School: A Growth Curve Analysis, based on one 

Edison school in Miami-Dade County, Florida, was conducted in a way 

that allows an assessment of the program’s effect on student 

achievement. This study followed individual student standardized test 

scores over a 3-year period and found that Edison students progressed 

at similar rates to those in the traditional Miami-Dade County Public 

Schools (MDCPS); this finding is not generalizable to other schools 

managed by Edison or any other private company. The study was designed 

to ensure that the Edison students were similar to the random sample of 

students drawn from MDCPS in terms of school grade, socioeconomic 
status, 

as indicated by the percent eligible for free/reduced price lunch, 

ethnicity, and achievement levels, as indicated by comparability in 

test scores prior to students enrolling in the Edison school. The study 

employed two different analytical techniques and both resulted in the 

finding that the Edison students progressed at similar rates to the 

traditional public school students. Several methodological techniques 

that would have strengthened its overall findings could have been 

employed. These include controlling more specifically for school-level 

differences between the participating students as well as better 

ensuring the two groups of students remained equivalent despite study 

dropouts (subsequently referred to as attrition). Differences in the 

composition of these groups, after attrition, could affect the test 

score results. This study did not examine the effect of this company’s 

program on parental satisfaction, parental involvement, or school 

climate.



Significant limitations in the other four studies preclude our making 

assessments of the effectiveness of schools managed by Edison, 

Chancellor Beacon, or Mosaica that were included in the 

studies.[Footnote 18] These limitations included use of comparison 

groups that did not adequately control for differences between the 

students in the company’s schools and the students in traditional 

public schools, instances where achievement data were not available for 

all students, and lack of adjustment for high attrition rates.



Company officials report that one way to determine if their programs 

are effective is to assess whether students demonstrate academic growth 

as evidenced by improvement on standardized tests. There is evidence to 

support the assertion that students enrolled in schools managed by 

Chancellor Beacon, Mosaica, and Edison have demonstrated academic 

improvement from one point in time to another, but it is important to 

determine if these gains are specifically the result of company 

programs.



Additional research is in progress. Edison commissioned RAND to 

evaluate Edison schools across the country. Where possible, RAND plans 

to compare the scores of individual Edison students to those of 

traditional public schools students with similar characteristics. Since 

it is often difficult to gather individual level student data, RAND 

will also compare Edison data, either at the grade or school level, to 

publicly available state data at that same level. RAND expects to 

publish its findings in 2004.



Agency Comments:



We received written comments on a draft report from the Department of 

Education. These comments are presented in appendix III. Education 

stated that there are insufficient data on the effectiveness of private 

education companies. Education also stated that it encourages others’ 

evaluation efforts. We also received comments from an expert on private 

education companies, the authors of the MDCPS study that we assessed, 

the District of Columbia Board of Education, the District of Columbia 

Public Charter School Board, as well as Edison Schools, Mosaica 

Education, and Chancellor Beacon Academies. These comments were also 

incorporated where appropriate.



We are sending a copy of this report to the Secretary of Education, the 

District of Columbia Board of Education, the District of Columbia 

Public Charter School Board, Edison Schools, Mosaica Education, and 

Chancellor Beacon Academies. We will make copies available to others on 

request. In addition, the report will be available at no charge on the 

GAO Web site at http://www.gao.gov.



If you or your staff have any questions about this report, please call 

me at (202) 512-7215. Other contacts and contributors to this report 

are listed in appendix IV.



Sincerely yours,



Marnie S. Shaul, Director

Education, Workforce, and Income Security Issues:



Signed by Marnie S. Shaul



[End of section]



Appendix I: Objectives, Scope, and Methodology:



Objectives:



The objectives of our review were to (1) identify the characteristics 

of the for-profit educational management companies operating in the 

District and determine the extent to which District schools managed by 

these companies have used their programs and (2) determine what is 

known about the effectiveness of these programs, as measured primarily 

by student achievement. We conducted our work between January and 

September 2002, in accordance with generally accepted government 

auditing standards.



Scope and Methodology:



To identify the characteristics of the programs offered by for-profit 

companies operating in the District, and determine the extent to which 

District public schools managed by them have used their programs, we 

interviewed company officials, representatives of the 10 schools, as 

well as officials of the District’s chartering authorities. We 

collected information on the companies from their Web sites and 

obtained technical comments from the companies on the descriptions of 

their programs. We also contacted education experts and advocates to 

obtain both their recommendations on research regarding the three for-

profit companies and information on any research they might have 

conducted on the companies. We also acquired information on the 

companies by reviewing relevant research summaries. We also observed an 

on-site review of one school’s program conducted for District oversight 

authorities.



To determine what is known about the effectiveness of these programs, 

we collected, reviewed, and analyzed information from available 

published and unpublished research on the effect on student 

achievement, parental satisfaction, parental involvement, and school 

climate of the three companies managing schools in the District. We 

also spoke with RAND officials about the design and methods of their 

current evaluation of Edison Schools. To identify relevant research, we 

followed three procedures: (1) interviewed experts to find out what 

studies were completed or in the process of being completed on the 

effectiveness of company programs; (2) conducted library and Internet 

searches; and (3) reviewed bibliographies of studies that focused on 

the effectiveness of company programs.



We reviewed studies concerning the three companies operating in the 

District that met the following criteria: included comparison groups 

and measurement over time, and focused on academic achievement, 

parental satisfaction, parental involvement, or school climate. Our 

final list of studies for review consisted of five studies, as listed 

in appendix II. We did not identify any studies that evaluated the 

effect of these three programs in District schools.



Two GAO social scientists examined each study to assess the adequacy of 

the samples and measures employed, the reasonableness and rigor of the 

statistical techniques used to analyze them, and the validity of the 

results and conclusions that were drawn from the analyses. For selected 

studies, we contacted the researchers directly when we had questions 

about their studies.



[End of section]



Appendix II: Studies and Other Information Sources Considered:



In order to identify research that explicitly addresses the effect on 

student achievement, parental satisfaction, parental involvement, or 

school climate of the three companies managing schools in the District, 

we interviewed experts to determine what studies were completed or in 

the process of being completed, conducted library and Internet 

searches, and reviewed bibliographies of studies that focused on the 

effect of these companies’ programs on student achievement. Although 

five studies met our criteria for review (inclusion of comparison 

groups, measurement over time, and focus on academic achievement, 

parental satisfaction, parental involvement, or school climate), we 

cannot draw conclusions, due to methodological weaknesses, from the 

four studies listed below.[Footnote 19] Conclusions from A Longitudinal 

Study of Achievement Outcomes in a Privatized Public School: A Growth 

Curve Analysis were presented in the text.



* Miron, Gary and Brooks Applegate. An Evaluation of Student 

Achievement in Edison Schools Opened in 1995 and 1996. Kalamazoo, 

Michigan: The Evaluation Center, Western Michigan University, December 

2000.



* Miron and Applegate analyzed both individual and aggregate level data 

and compared improvements in the test scores of 10 Edison schools with 

those of comparison schools, districts, states, and national norms, 

where applicable. However, significant weaknesses prevented conclusive 

statements on the effects of Edison schools. These weaknesses included 

limitations in the available data, such as incompleteness and 

inconsistency, high attrition rates, and the lack of corresponding 

adjustments for attrition.



* Horn, Jerry and Gary Miron. An Evaluation of the Michigan Charter 

School Initiative: Performance, Accountability, and Impact. Kalamazoo, 

Michigan: The Evaluation Center, Western Michigan University, July 

2000.



* Horn and Miron examined the percentage of students earning a passing 

grade on achievement tests in individual charter schools in Michigan in 

comparison with the percentage passing in the districts where these 

schools were located.[Footnote 20] The analysis included schools 

managed by Edison, Mosaica, and Beacon. Weaknesses included inadequate 

controls for differences between the students in charter schools and 

their host districts, no consideration of attrition rates, and the 

likelihood that analyses were often based on a small number of 

students.



* American Federation of Teachers. Trends in Student Achievement for 

Edison Schools, Inc.: The Emerging Track Record. Washington, D.C.: 

October 2000.



* Researchers examined school and grade-level achievement data from 40 

Edison schools in eight states and compared it to data gathered from 

school districts and other schools. Weaknesses included insufficient 

information about the methodology employed by the states, including 

construction of comparison groups and matching techniques, and a lack 

of analysis of attrition rates.



* Gomez. Ph.D., Joseph and Sally Shay, Ph.D. Evaluation of the Edison 

Project School. Final Report, 1999-00 (portions related to parental 

satisfaction and involvement, and school climate). Office of Evaluation 

and Research, Miami-Dade County Public Schools (MDCPS), April 

2001.[Footnote 21]



* Gomez and Shay examined responses from surveys MDCPS had administered 

to parents and teachers from both the Edison school and the control 

group. However, the outcomes related to parental satisfaction and 

involvement were measured with single-item survey questions that do not 

seem to capture the full context of the concepts. School climate was 

measured with a single-item question on a teacher survey and with 

school archival data.[Footnote 22] Shay and Gomez did not report 

whether any differences are statistically significant, in part because 

they acknowledged it would be inappropriate to conduct tests of 

significance on single-item questions. Therefore, there is no evidence 

to determine whether Edison school parents were more satisfied or 

involved than those in the control group, or whether the Edison school 

improved school climate.



We are aware of other studies and reports that address the effect of 

Chancellor Beacon Academies, Mosaica Education, and Edison Schools on 

academic achievement, parental satisfaction, parental involvement, or 

school climate; however, the following are examples that did not meet 

the criteria for inclusion in our review.



* District of Columbia Public Charter School Board. School Performance 

Reports. Washington, D.C.: August 2001.



* Department of Research, Evaluation, and Assessment, Minneapolis 

Public Schools. Edison/PPL School Information Report 2000-2001. 

Minneapolis, Minnesota: 2001.



* Department of Administration, Counseling, Educational and School 

Psychology, Wichita State University. An Independent Program Evaluation 

for the Dodge-Edison Partnership School: First Year Interim Report. 

Wichita, Kansas: 1996.



* Missouri Department of Elementary and Secondary Education. Charter 

School Performance Study: Kansas City Charter Schools. Jefferson City, 

Missouri: 2001.



* Company-provided information such as annual reports and school 

performance reports.



Other sources of general information included school district websites 

and other educational services, such as Standard and Poor’s School 

Evaluation Services and the National Association of Charter School 

Authorizers’ Educational Service Provider Information Clearinghouse.



[End of section]



Appendix III: Comments from the Department of Education:



UNITED STATES DEPARTMENT OF EDUCATION:



THE DEPUTY SECRETARY:



October 8, 2002:



Ms. Marnie S. Shaul:



Director, Education, Workforce and Income Security Issues, General 

Accounting Office:



Washington, DC 20548:



Dear Ms. Shaul:



Thank you for the opportunity to review and comment on the draft 

report, “Public Schools: Insufficient Research to Determine 

Effectiveness of Selected Private Education Companies.” Across the 

country, we see how public education is being transformed by expanding 

the options and choices available to parents. For example, when the 

school year opened last month, there were over 2,700 charter schools 

operating in 39 States nationwide, an impressive feat considering the 

charter school movement is just a decade old.



Charter schools, which are public schools of choice that operate free 

from many State and local district rules in exchange for heightened 

accountability, allow a variety of different groups to enter the public 

education market and manage schools. Some of these new managers include 

local community groups and businesses, as well as private education 

management organizations (EMOs), like those identified in your report. 

As you also indicate in your report, many of these EMOs also operate 

traditional public schools on a contract basis. Are these companies 

effective in boosting student achievement? Unfortunately, as your study 

details, there is insufficient data on the performance of EMOs due to 

the relatively small number of schools they manage and other 

methodological constraints.



The only way to learn more about charter schools, alternate governance 

arrangements (such as schools managed by EMOs), and other reforms is 

through further experimentation and research. Those of us in public 

education should embrace these exciting new innovations that hold out 

the promise of boosting student achievement. Unfortunately, we know 

that too many of our schools are not performing at the levels they 

should, so we must explore alternatives while rigorously examining 

their results. Parents too must be free to choose which schools best 

suit their children’s needs.



I applaud the work of States and local communities in broadening the 

choices available to parents and experimenting with new, innovative 

approaches to the delivery of public education. In the future, I know 

there will be a deeper body of research available and I encourage the 

GAO and other organizations to continue their evaluation efforts. 

Together, we will work to ensure that no child is left behind.



Sincerely,



William D. Hansen



Signed by Bill Hansen



Appendix IV: GAO Contacts and Staff Acknowledgments:



GAO Contacts:



Harriet Ganson, (202) 512-7042

Chris Morehouse, (202) 512-7214:



Acknowledgments:



In addition to those named above, Rebecca Ackley and N. Kim Scotten 

made key contributions to this report. Jay Smale, Michele Fejfar, Kevin 

Jackson, Sara Ann Moessbauer, and Shana Wallace provided important 

methodological contributions to the review of research. Patrick 

Dibattista and Jim Rebbe also provided key technical assistance.



[End of section]



Related GAO Products:



School Vouchers: Characteristics of Privately Funded Programs. GAO-02-

752. Washington, D.C.: September 26, 2002.



School Vouchers: Publicly Funded Programs in Cleveland and Milwaukee. 

GAO-01-914. Washington, D.C.: August 31, 2001.



Charter Schools: Limited Access to Facility Financing. GAO/HEHS-00-163. 

Washington, D.C.: September 12, 2000):



Charter Schools: Federal Funding Available but Barriers Exist. HEHS-98-

84. Washington, D.C.: April 30, 1998.



Charter Schools: Recent Experiences in Accessing Federal Funds. 

T-HEHS-98-129. Washington, D.C.: March 31, 1998.



Charter Schools: Issues Affecting Access to Federal Funds. T-HEHS-97-

216. Washington, D.C.: September 16, 1997.



Private Management of Public Schools: Early Experiences in Four School 

Districts. GAO/HEHS-96-3. Washington, D.C.: April 19, 1996.



Charter Schools: New Model for Public Schools Provides Opportunities 

and Challenges. GAO/HEHS-95-42. Washington, D.C.: January 18, 1995.



School-Linked Human Services: A Comprehensive Strategy for Aiding 

Students At Risk of School Failure. GAO/HRD-94-21. Washington, D.C.: 

December 30, 1993.



FOOTNOTES



[1] In this context, “major” refers to those companies that manage at 

least 3 schools and operate in multiple states.



[2] In fact, not all companies offer a particular curriculum or 

educational approach, focusing instead on management services. For 

example, Arizona-based ABS concentrates on management services like 

recruitment, personnel, payroll, and facilities management.



[3] However, in Arizona, private companies may run charter schools 

directly.



[4] P.L. 107-110, Jan. 8, 2002.



[5] Others include reconstituting by replacing teachers, principals, or 

both; converting to public charter school status; and turning over 

school operation to the state educational agency.



[6] In school year 2001-02, a fourth for-profit company, Richard 

Milburn Academies, continued to operate a District public charter 

school, pending a final decision by District oversight authorities to 

revoke its charter. The school closed in June 2002.



[7] Edison officials describe the educational approach as consisting of 

“10 fundamental” characteristics: school organization, extended school 

day and year, curriculum, instructional methods, assessments, 

professional teaching environment (including professional 

development), use of technology, partnership with families, 

instructional programs that reflect community interests, and the 

benefits of “systems and scale.”



[8] Mosaica officials describe the educational approach as consisting 

of “seven pillars” or characteristics: the Paragon curriculum, 

professional development, use of technology, extended school day and 

year, parental involvement, community involvement, and school climate.



[9] According to an Edison official, as of September 2002, Edison and 

one of these 2 schools were planning a transition that would completely 

discontinue Edison’s services to the school and return the school to 

management by its own board of trustees. The services that Edison 

provided to both schools in the 2001-02 school year were limited to 

management services, such as payroll. 



[10] This school’s existing educational approach is tied to a 3-year 

federal comprehensive school reform demonstration grant. The grant 

expires at the end of the 2004 school year.



[11] In addition, according to officials of the agencies that oversee 

District charter schools, substantial curriculum changes must be 

officially approved.



[12] Data were not available for all schools; these ranges are based on 

8 schools (percent African American) and 9 schools (percent receiving 

subsidized lunch).



[13] Experts varied with regard to how many years of data are 

sufficient for analysis. Generally, experts we spoke to indicated that 

3 to 5 years of data would be sufficient for analysis.



[14] While the study, Achievement Performance Report: Dallas-Edison 

Partnership Schools 2001-02 (Dallas Division of Evaluation and 

Accountability, Dallas Independent School District, 2002), met the 

criteria for inclusion, we were unable to review it because it was 

published after completion of the review.



[15] The evaluation included 18 schools managed by Beacon and was 

completed prior to the merger of Chancellor and Beacon.



[16] One study compares academic achievement in District charter 

schools and traditional public schools, but does not distinguish 

charter schools managed by private companies. See Jeffrey Henig et al. 

Growing Pains: An Evaluation of Charter Schools in the District of 

Columbia; 1999-2000. (Washington, D.C.: The Center for Washington Area 

Studies, The George Washington University, 2001). 



[17] We did not review the student achievement portion of the Miami-

Dade County Public Schools Evaluation because the same data were 

analyzed, with very similar results, by the same researcher in another 

study in the review.



[18] Please see appendix II for the studies and other information 

sources considered for our review.



[19] While the study, Achievement Performance Report: Dallas-Edison 

Partnership Schools 2001-02 (Dallas Division of Evaluation and 

Accountability, Dallas Independent School District, 2002), met the 

criteria for inclusion, we were unable to review it because it was 

published after completion of the review.



[20] Analyses of individual schools were included in an appendix 

provided upon request from the authors. We included this study in our 

review because it explicitly addressed the results of schools managed 

by the three companies in this report, unlike other studies that did 

not disaggregate school results by company.



[21] We only examined the portion of this study that relates to the 

effect of the Edison school on parental satisfaction and involvement 

and school climate. We examined the data and analysis that related to 

the academic achievement piece in Dr. Shay’s dissertation, A 

Longitudinal Study of Achievement Outcomes in a Privatized Public 

School: A Growth Curve Analysis, reported in the text.



[22] This includes data on students’ attendance rates, student 

mobility, indoor and outdoor suspensions, and the teacher-student 

ratio.



GAO’s Mission:



The General Accounting Office, the investigative arm of Congress, 

exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability 

of the federal government for the American people. GAO examines the use 

of public funds; evaluates federal programs and policies; and provides 

analyses, recommendations, and other assistance to help Congress make 

informed oversight, policy, and funding decisions. GAO’s commitment to 

good government is reflected in its core values of accountability, 

integrity, and reliability.



Obtaining Copies of GAO Reports and Testimony:



The fastest and easiest way to obtain copies of GAO documents at no 

cost is through the Internet. GAO’s Web site ( www.gao.gov ) contains 

abstracts and full-text files of current reports and testimony and an 

expanding archive of older products. The Web site features a search 

engine to help you locate documents using key words and phrases. You 

can print these documents in their entirety, including charts and other 

graphics.



Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its 

Web site daily. The list contains links to the full-text document 

files. To have GAO e-mail this list to you every afternoon, go to 

www.gao.gov and select “Subscribe to daily E-mail alert for newly 

released products” under the GAO Reports heading.



Order by Mail or Phone:



The first copy of each printed report is free. Additional copies are $2 

each. A check or money order should be made out to the Superintendent 

of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. 

Orders should be sent to:



U.S. General Accounting Office



441 G Street NW,



Room LM Washington,



D.C. 20548:



To order by Phone: 	



Voice: (202) 512-6000:

TDD: (202) 512-2537:

Fax: (202) 512-6061:



To Report Fraud, Waste, and Abuse in Federal Programs:



Contact:



Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov



Automated answering system: (800) 424-5454 or (202) 512-7470:



Public Affairs:



Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.



General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.



20548: