This is the accessible text file for GAO report number GAO-13-655R entitled 'Education Needs to Further Examine Data Collection on English Language Learners in Charter Schools' which was released on August 15,2013. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO-13-655R: GAO: United States Government Accountability Office: 441 G St. N.W. Washington, DC 20548: July 17, 2013: The Honorable George Miller: Ranking Member: Committee on Education and the Workforce: House of Representatives: The Honorable Raúl Grijalva: House of Representatives: Education Needs to Further Examine Data Collection on English Language Learners in Charter Schools: Over the past decade, the number of students who are English Language Learners (ELLs) enrolled in U.S. public schools has grown substantially. At the same time, charter schools have expanded the educational options for ELLs and their families. As the number of charter schools continues to grow, questions have arisen about how charter schools--a relatively new phenomenon in public education that began in the 1990s--are serving special student populations, including ELLs. Following the completion of our study comparing the enrollment of students with disabilities in charter schools and traditional public schools, GAO began a similar study evaluating the enrollment of ELLs in charter schools and traditional public schools.[Footnote 1] As part of this study, we obtained a dataset from the U.S. Department of Education (Education) containing school-level data that could be used to compare ELL enrollment in charter schools to ELL enrollment in traditional public schools for school year 2010-11. This was the only dataset available at the time that could be used for our analysis. Further, while this dataset was designed to contain enrollment data only on ELLs in ELL programs, Education officials initially told us there was no reason to expect that charter schools would be less likely than other schools to report data for the dataset. In accordance with Government Auditing Standards and GAO policy, we assessed the quality of the ELL data as part of our study and found problems with the data's quality that prevented us from completing the study.[Footnote 2] This letter frames our findings concerning Education's data on ELL enrollment around two questions: (1) What is the quality of Education's data on ELLs in charter schools? and (2) What efforts has Education taken to improve its collection of ELL data from charter schools? We conducted this review from November 2012 to July 2013 in accordance with generally accepted government auditing standards. See enclosure I for details on our scope and methodology. Results in Brief: GAO was unable to compare ELL enrollment in charter schools to ELL enrollment in traditional public schools because Education's only available data on school-level ELL enrollment were unreliable and incomplete. Specifically, for over one-third of charter schools, the field for reporting the counts of ELLs enrolled in ELL programs was left blank. These blank fields cannot reliably be interpreted to mean that the charter schools did not have ELLs enrolled. Education officials told us that these school-level ELL data do not have a data steward (that is, an office responsible for overseeing the quality of the data) and that the quality of the data is not examined on a regular basis. A definitional issue may have resulted in states excluding some students from the reported counts, and some charter schools may have failed to submit required data to their states. Education officials said they had not systematically studied charter school nonreporting but acknowledged that states sometimes have difficulty obtaining data from charter schools. Some charter schools with blank ELL counts were also missing data from other important datasets, such as those on school performance, which suggests potential broader problems with charter school reporting. Education has a new ELL school-level data collection effort underway that will address the problem with the definition used in the current school-level ELL dataset. However, Education has not articulated how it will address the broader problem with charter school nonreporting that prevented us from completing our study. Further, Education plans to stop collecting school-level data for the ELL dataset with the problematic definition, but it plans to continue collecting these data for the local educational agency[Footnote 3] (LEA) and state levels. GAO is recommending that Education systemically evaluate the extent of charter school nonreporting across other data sets and explore whether collection of the poorly defined data on ELLs enrolled in ELL programs can be phased out at the LEA and state levels, if the data quality problems are not addressed. In its comments on a draft of this report, Education questioned why data issues prevented GAO from completing its study and stated that GAO could have proceeded with its study by focusing on the factors affecting patterns of ELL enrollment in charter schools and traditional schools and the techniques charter schools use to outreach to ELLs. We disagree with Education because without information showing what the patterns of ELL enrollment in charter schools actually are, a study of the factors affecting those patterns would not be practicable, useful or informative. With respect to our recommendations, Education did not express agreement or disagreement. Education said that it was aware of problems with charter school reporting prior to the GAO study and that it has efforts already underway to address charter school nonreporting, including developing a complete directory of all charter schools in operation. We believe those efforts, while necessary, do not fully address the problems with charter school nonreporting that we found or illuminate the reasons for nonreporting of charter school data. Education also said it is exploring options regarding the LEA-and SEA-level counts, including whether these data can be phased out if the data quality problems are not addressed. Background: By definition, ELLs come from a non-English speaking background and have not yet developed the level of English proficiency needed for academic success. Various federal requirements pertain to ELLs, including the requirement under the Elementary and Secondary Education Act of 1965 (ESEA), as amended, for states to include ELLs in the academic assessments prescribed under the act and to report on their achievement.[Footnote 4] Further, according to Education's Office for Civil Rights, ELLs must be provided with alternative educational services until they are proficient enough in English to participate meaningfully in regular educational programs. In school year 2010-11, 40 states had at least one charter school operating in the state.[Footnote 5] Relative to traditional schools, charter schools served only a small fraction of school age children-- about 1.8 million (4 percent) of the 46 million public school students in those 40 states in school year 2010-11. Charter schools are public schools created to achieve a number of goals, including encouraging innovation in public education and addressing failing schools. Generally, charter schools operate with more autonomy than traditional public schools but are subject to various federal requirements in the same way as traditional public schools, including requirements under ESEA, civil rights laws, and the Individuals with Disabilities Education Act. As part of its mission and oversight, Education collects and reports a wide array of data on public schools, such as student enrollment, demographics, and academic performance. Education also publishes these data in several reports, most notably its Digest of Education Statistics and The Condition of Education that are used by researchers who follow trends in education. While Education often reports these data for each state or the nation as a whole, the data originate from individual schools. As shown in figure 1, generally, data, such as ELL counts, are reported from schools through LEAs and state educational agencies (SEAs) to Education. SEAs submit the data to Education, but, to do this, SEAs must first collect the data from schools and LEAs. Figure 1: Flow of Data from Schools to Education: [Refer to PDF for image: illustration] 1) Traditional and charter schools submit school-level data to a Local educational agency (LEA). 2) LEAs submit school- and LEA-level data to State Educational Agency (SEA). 3) Charter schools acting as their own LEA submit school- and LEA- level data to State Educational Agency (SEA). 4) SEA submits school-, LEA-, and State-level data to U.S. Department of Education. 5) U.S. Department of Education releases data in publications and on publicly available databases. Source: GAO. [End of figure] Depending on the requirements for each dataset, SEAs may report data to Education at the school-level (that is, broken down by each individual school) or aggregated to the LEA-or SEA-level.[Footnote 6] Having school-level data is important to researchers wishing to compare charter schools to traditional schools because once data are aggregated to the LEA or SEA levels, the ability to separate out charter schools from traditional schools is lost. That is why our study comparing enrollment of ELLs in charter schools and traditional schools required school-level data. The only comprehensive school- level data on ELL counts that Education had at the time of our study was of ELLs "enrolled in English language instruction educational programs" designed for ELLs.[Footnote 7] Education periodically collects school-level counts of ELLs in its Civil Rights Data Collection (CRDC), but, at the time of our review, the recent CRDC data only covered a sample of schools that was not designed to be statistically representative of charter schools.[Footnote 8] Education collects other data on numbers of ELLs at the LEA and SEA levels, but not at the school-level. These include data on the number of: * ELLs enrolled,[Footnote 9] * ELLs served by the Title III program,[Footnote 10] and: * former ELLs who have achieved proficiency in English and are no longer receiving Title III services. Unlike traditional public schools that are part of a LEA, some states establish charter schools as their own LEA. Other states require charter schools to be part of a larger LEA, while still others allow charter schools the option to choose between being a distinct LEA or part of a larger LEA.[Footnote 11] When a charter school is part of a larger LEA, its data are supposed to be included along with data from other schools in the LEA's totals reported to Education. Conversely, a charter school that is its own LEA would report only a single school's data even when reporting at the LEA-level.[Footnote 12] Nonreporting and Definitional Issues Undermine the Quality of Education's Data on ELLs in Charter Schools: We were unable to compare ELL enrollment in charter schools to ELL enrollment in traditional public schools nationwide or by state because of problems with Education's only available dataset of school- level ELL enrollment counts. Specifically, in our review of this dataset, we found that a substantial percentage of charter schools--37 percent in school year 2010-11--had blanks in the field capturing ELL enrollment counts.[Footnote 13] While some of these schools may not have any ELL students, our analyses suggested that some schools that have ELLs enrolled did not report them.[Footnote 14] Further, the percentage of schools with blanks in the field for ELL counts varied by state. In 14 states, at least 60 percent of the charter schools had blank fields. In two states generally known to have large numbers of ELLs, New York and New Jersey, over 80 percent of charter schools had blank fields.[Footnote 15] In contrast, in six states all charter schools had numerical values--including zeros--for ELL counts; that is, no charter schools had blank fields for their ELL counts in these states. Figure 2 shows the distribution of charter schools with blank ELL counts in the 40 states with charter schools. Figure 2: Percentage of Charter Schools With Blank ELL Enrollment Counts in School Year 2010-11, by State: [Refer to PDF for image: illustrated table] Percentage left blank: No blanks; Number of states: 6; States: Arkansas, Florida, Georgia, Iowa[A], Oregon, Virginia[A]. Percentage left blank: 1%-19%; Number of states: 5; States: Alaska, California, Colorado, Oklahoma, Wisconsin. Percentage left blank: 20%-39%; Number of states: 6; States: Connecticut, Delaware, Hawaii, Nevada, North Carolina, Texas. Percentage left blank: 40%-59%; Number of states: 9; States: Arizona, District of Columbia, Illinois, Indiana, Massachusetts, New Mexico, Rhode Island, Tennessee, Utah. Percentage left blank: 60%-79%; Number of states: 9; States: Idaho, Louisiana, Maryland, Michigan, Minnesota, Missouri, Pennsylvania, South Carolina, Wyoming[A]. Percentage left blank: 80%-100%; Number of states: 5; States: Kansas, New Hampshire, New Jersey, New York, Ohio. Source: GAO analysis of Education’s school-level ELL enrollment data for SY 2010-11. [A] States with less than 5 charter schools operating in school year 2010-11. [End of figure] While Education's data reporting protocols for this dataset allow blank fields if the ELL counts are zero, these blanks cannot reliably be interpreted as zeros, especially given the large number of charter schools with blank ELL counts. In response to concerns we raised about the data, Education conducted analyses that found some charter schools with blank ELL counts did, in fact, have ELLs enrolled. Specifically, Education found that some of these charter schools were their own LEA and had LEA-level counts for ELLs in a different dataset which is collected at the LEA-level but not at the school-level. Also, some of the charter schools with blank ELL counts in the dataset we reviewed had reported assessment results for ELLs through another Education data collection. The difficulty of interpreting blank ELL counts in Education's dataset is further complicated by how Education defines a key term for the dataset. Specifically, this dataset is a count of ELLs enrolled in "English language instruction educational programs," not a simple count of all ELLs. When we began our study, Education officials told us there was no reason to expect that certain types of schools such as, charter schools or schools without Title III funding, would be less likely than other schools to report data for the dataset. However, after we identified the high percentage of charter schools with blank ELL counts, Education officials told us that the definition of "program" for the dataset is broad and left up to state interpretation. Specifically, they said that charter schools may be less likely to consider their ELLs as participating in a program, as defined by their state and, therefore, may not report their ELLs in this dataset. Further, they said that, in some states, the SEA may decide not to report ELLs for some schools if it determines that the schools did not have programs that met the state's definition of "program" for the dataset. An Education official told us that officials in one state said they consider the term "program" for this dataset to mean the Title III program that provides grants to states to support the education of ELLs, and the state only reports ELLs in schools that receive Title III funding. This definitional issue could affect the reliability of ELL counts across all states, including the six states where no charter schools had blank ELL counts. An additional likely cause for blank ELL counts in Education's dataset is that some charter schools failed to submit required data to their state. Education officials said they had not systematically studied charter school nonreporting but acknowledged that states sometimes have difficulty obtaining data from charter schools. Further, Education officials told us that some state officials identified charter school nonreporting as a contributing factor to the large percentage of blank ELL counts in Education's dataset. In addition, Education's analyses suggest that some of the charter schools with no reported ELL data for the dataset we reviewed may have additional data reporting problems. Specifically, Education's analyses found that some charter schools with blank ELL counts were also missing math or reading performance data or graduation rates for all students, including ELLs, which suggests these schools had broader reporting problems not limited to ELLs. In response to our questions about the problems with charter schools' ELL data, Education officials told us that the school-level ELL data that we reviewed do not have a data steward (that is, an office responsible for overseeing the quality of the data) and that the quality of the data is not examined on a regular basis; instead, any quality reviews of the data are conducted on an ad hoc basis when the data are requested. The problems we identified with Education's school-level ELL dataset coupled with Education's own findings that key information was missing from other databases that collect information on charter schools, indicate potential broader problems with charter school reporting and undermine Education's ability to have reliable, timely information to monitor charter schools.[Footnote 16] In addition, the problems GAO identified with the school-level ELL data appear to affect the quality of the LEA-level counts of ELLs enrolled in "English language instruction educational programs" that Education publishes and makes available to researchers in the Common Core of Data, the agency's dataset that captures a broad range of information on education. Education officials told us that states submit school-level and LEA- level data separately to Education; therefore, the LEA-level data may not have the same problems that we identified with the school-level data. However, when we calculated and compared statewide ELL counts from two datasets--(1) the school-level dataset we received from Education and (2) LEA-level data from the Common Core of Data--we found that, for most states, the ELL counts for both datasets matched almost exactly, as shown in figure 3. This suggests that, even though states submit the school-level and LEA-level data separately to Education, both datasets are derived from the same source--data collected from schools. This further suggests that, as with the school- level data, the LEA-level data are incomplete and unreliable. For example, New Jersey's ELL counts from its LEA-level and school-level data matched almost exactly even though over 80 percent of its charter schools did not have ELL counts in the school-level data. Figure 3: Percentage Difference between Numbers of English Language Learners Included in School-Level Dataset Analyzed by GAO and Local Educational Agency Level Data from Education's Common Core of Data: [Refer to PDF for image: illustrated table] Percentage difference: 0.0; Number of states: 26 States: Alaska, Arizona, Arkansas, Colorado, Connecticut, District of Columbia, Georgia, Hawaii, Idaho, Illinois, Kansas, Louisiana, Maryland, Missouri, Nevada, New Hampshire, New Jersey, North Carolina, Ohio, Pennsylvania, Rhode Island, South Carolina, Texas, Utah, Virginia, Wyoming. Percentage difference: 0.1%-0.9%; Number of states: 9 States: Delaware, Florida, Indiana, Iowa, Massachusetts, Michigan, New Mexico, Oregon, Tennessee. Percentage difference: 1.0%-3.0%; Number of states: 2; States: New York, Oklahoma. Percentage difference: More than 3.0%; Number of states: 2 States: Wisconsin, Michigan. Source: GAO analysis of states’ ELL counts in school year 2010-11; School-level dataset provided by Education and LEA-level data downloaded from Common Core of Data for states with charter schools. Note: California is excluded from the table because its ELL counts for school year 2010-11 were not included in the Common Core of Data. [End of figure] Education is Revising its ELL Data Collection Efforts, but Charter School Nonreporting Remains Problematic: Beginning in school year 2013-14, Education plans to collect new school-level data on all ELLs regardless of whether or not they are enrolled in a "program," thus addressing the issue with the term "program" that limits the usefulness of the current school-level dataset.[Footnote 17] According to Education officials, the new school- level data will, unlike the current school-level data, have a data steward and will receive regular data quality reviews. For the dataset that we analyzed, ELLs enrolled in English language instruction educational programs designed for ELLs, Education plans to halt collection of school-level data but continue collecting LEA-and SEA- level data. Given that the quality problems we found in the current school-level data appear to carry over to and reduce the usefulness of the LEA-and SEA-level data, the cost of continuing to collect these LEA-and SEA-level data may not be justified. In addition, Education officials are developing a workbook to serve as a guide for SEAs in reporting charter school data. Specifically, according to Education officials, the workbook will address issues that affect Education's ability to maintain a complete and accurate directory of charter schools. For example, the workbook is intended to address issues, such as reporting on the opening and closing of charter schools and providing up-to-date information on charter schools' LEAs and authorizers. Education plans to issue this workbook in the summer of 2013 before the school year 2013-14 data collection cycle. However, Education has not articulated how it will address issues with charter school nonreporting that hindered our ability to compare ELL enrollment at charter schools to enrollment at traditional public schools. Conclusions: The population of ELLs is growing and charter schools provide these students and their families an additional educational option. Therefore, it is important that data be available on the characteristics of students enrolled in charter schools in order to assess how these schools are serving ELLs relative to traditional public schools. However, because of problems with the completeness and reliability of Education's current school-level data, we could not determine how the enrollment of ELLs in charter schools and traditional public schools compare. Further, the nonreporting by charter schools that we identified and the subsequent finding by Education that some charter schools had not reported information in other datasets may signal a broader problem with charter school reporting. These data problems could have implications for Education's ability to track key information related to charter schools and interfere with their ability to effectively oversee charter schools' compliance with relevant federal requirements. While Education plans to begin collecting new school-level ELL data that does not have the definitional issue that characterizes its current data collection, the problem of nonreporting by charter schools remains. Further, the problems we identified with Education's current school-level ELL data also appear to affect these data at the LEA and SEA levels. While Education has plans to stop collecting these data at the school-level, it plans to continue its LEA-and SEA-level data collection. The limited usefulness of the LEA- and SEA-level data may not be worth the cost of collecting the data, given the data quality problems we identified. Recommendations for Executive Action: We recommend that the Secretary of Education take the following two actions: 1. To more fully understand and address nonreporting of charter school data, conduct a systematic evaluation of other important data sets such as, datasets on academic achievement and graduates/completers to determine the extent of charter school nonreporting, assess underlying reasons for such nonreporting, and determine the steps needed to address nonreporting. 2. Because the problems we identified with the school-level ELL data also affect the LEA-level data, explore whether the collection of the LEA- and SEA-level counts of ELLs enrolled in English language instruction educational programs can be phased out, if the data quality problems are not addressed. Agency Comments and Our Evaluation: We provided a draft of this report to the U.S. Department of Education for review and comment. The U.S. Department of Education provided written comments which are reproduced in enclosure II. In its comments, Education did not indicate agreement or disagreement with GAO's recommendations. Education questioned why data issues prevented GAO from completing its study. Education posited that GAO could have proceeded with its study focusing on the factors affecting patterns of ELL enrollment in charter schools and traditional schools and the techniques charter schools use to outreach to ELLs. We disagree with Education. Lacking information showing what the patterns of ELL enrollment in charter schools actually are, a study of the factors that affect those patterns would not be practicable, useful or informative. Further, with respect to the data Education provided to GAO for its study, Education in its agency comments reported that they had already determined that the data were of "limited utility." However, Education did not provide information on the problems with the data until after we identified the high percentage of charter schools with blank fields in the dataset. As a result of our review of the data, Education conducted its own analyses and consulted with states to understand the potential causes for the problems. Education said that, other than the dataset that GAO used on "Limited English Proficient students enrolled in language instruction educational programs," it has not had a school-level dataset on ELL enrollment data because, consistent with its program statutes, such a dataset has not been needed by its program offices to date. Moving forward, Education said that it will have two datasets with school-level ELL enrollment data and that one of these, the Civil Rights Data Collection for school year 2011-12, will be available for use by early 2014. Once these data are available, GAO will assess the quality and make a determination about whether the data can be used for a future study of ELL enrollment. In response to GAO's first recommendation that Education conduct a systematic evaluation of other important datasets to determine the extent of charter school nonreporting, Education said that it was aware of problems with charter school reporting and that the department had efforts underway to improve the completeness and quality of charter school data prior to the GAO study. Specifically, Education said that it is developing a complete and accurate directory of charter schools and that this is a first step in ensuring high quality reporting because schools must be included in the directory before student counts can be reported for them. Education also stated that the lessons and outcomes from the directory review are informing its development of a charter school reporting workbook, which will assist states, along with traditional and charter school districts and authorizers, in reporting complete and high quality data. Further, Education said the technical assistance it provides to states and the attention it is giving to the charter school directory will encourage closer examination of the charter school data that states report to Education. We believe Education's efforts to provide technical assistance to states, maintain an up-to-date directory of charter schools, and develop a charter school reporting workbook can help improve the collection of charter school data. However, these efforts are not designed to provide a clear picture of the extent of nor reasons for charter school nonreporting, and we believe a systemic evaluation could be useful for this purpose. In response to GAO's second recommendation that Education explore whether the collection of LEA-and SEA-level counts for ELLs enrolled in "English language instruction educational programs" can be phased out if the data quality problems are not addressed, Education said it is exploring how phasing out this data collection would meet the needs of various data users and impact the Common Core of Data, which is a compilation of information on all public schools and school districts in the United States. Further, its Office of Civil Rights and Title III office are discussing the option of providing instructions and other guidance to states and districts on the definition of "English language instruction educational programs" in the data collection in order to improve the quality and quantity of responses. Education said it plans to continue collecting these data at the LEA and SEA levels during this evaluative phase. Education also provided technical comments on the report, and we have made changes in the report, where appropriate, to address these comments. As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its issue date. At that time, we will send copies of this report to relevant congressional committees, the Secretary of Education, and other interested parties. In addition, the report will be made available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (202) 512-7215 or scottg@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this correspondence. GAO staff who made key contributions to this report are Sherri Doughty, Assistant Director; Kathy Peyman, Analyst-in-Charge; Divya Bali; James Bennett; Holly Dye; Kirsten Lauber; Lise Levie; Ying Long; Luann Moy; and James Rebbe. Signed by: George A. Scott: Director: Education, Workforce, and Income Security Issues: [End of section] Enclosure I: Scope and Methodology: We assessed the quality of Education's enrollment data on English Language Learners (ELLs) in charter schools in order to determine if the data were reliable for the purpose of comparing ELL enrollment in charter schools to ELL enrollment in traditional public schools. We obtained from Education nationwide data for school year 2010-11 on all public schools, including charter schools.[Footnote 18] These data came from a dataset on the "unduplicated number of limited English proficient (LEP) students enrolled in English language instruction educational programs designed for LEP students." (Education's datasets use the term "limited English proficient" to refer to ELLs.) Based on an interview with Education officials and our review of available documentation about Education's datasets, we concluded that this was the only available dataset that could be used for a valid comparison of ELL enrollment in charter schools to ELL enrollment in traditional public schools. Education had other datasets related to ELL enrollment, such as a dataset on the "unduplicated number of LEP students enrolled in an elementary or secondary school at any time during the school year." However, these other datasets did not contain representative school-level data for charter schools that were needed for our review. Specifically, other datasets either (1) only included state-level or local educational agency (LEA) level data but not the school-level data needed for our analysis or (2) included school-level data for a sample of schools that was not designed to be statistically representative of charter schools. To determine the quality of Education's ELL enrollment data, we conducted analyses and reviewed documentation such as data reporting protocols. Our analysis of the dataset included analyzing, at the national and state levels, the number of charter schools and traditional public schools that had a blank field for the number of ELLs enrolled. In addition, for some states with many charter schools with blank fields for ELL enrollment numbers, we reviewed the list of individual charter schools to assess the plausibility that the blanks indicated that no ELLs were enrolled in the schools. To do this, we considered evidence such as the names of charter schools and how their ELL count reporting compared to other schools in nearby zip codes. Further, we calculated statewide ELL counts from two datasets--(1) the school-level dataset we received from Education on the "limited English proficient (LEP) students enrolled in English language instruction educational programs designed for LEP students" and (2) LEA-level data from the Common Core of Data--and we compared these two statewide ELL counts to determine the extent to which they differed. Finally, we interviewed officials at Education about the data and about the analyses they conducted on the data reporting of charter schools that had blanks in the field for the number of ELLs enrolled. To determine the efforts Education has taken to improve its collection of ELL data from charter schools, we interviewed Education officials and reviewed relevant documents. We conducted this review from November 2012 to July 2013 in accordance with generally accepted government auditing standards. [End of section] Enclosure II: Agency Comments and Our Evaluation: United States Department of Education: Assistant Secretary: Office of Planning, Evaluation and Policy Development: 400 Maryland Ave. S.W. Washington, DC 20202: June 26, 2013: Mr. George A. Scott: Director: Education, Workforce, and Income Security Issues: Government Accountability Office: Washington, DC 20548: Dear Mr. Scott: Thank you for the opportunity to review and comment on the Government Accountability Office (GAO) draft report, Education Needs to Further Examine Data Collection on English Language Learners in Charter Schools (GAO-13-655R). The U.S. Department of Education (the Department) appreciates GAO's interest in studying the distribution of English Language Learners (ELLs) in charter schools and traditional public schools, as it is critically important that school districts take steps to help ELL students overcome language barriers and to ensure that they can participate meaningfully in the district's educational programs. We recognize, however, that GAO has concluded it is unable to complete the study at this time. Last July, GAO's preliminary research approach included an outline to move beyond analysis of nationally representative data in order to explore factors affecting patterns of enrollment within charter schools and the techniques being used by charters to provide outreach to the traditionally underserved group of ELLs across our country. We believe that this additional analysis would have been useful to the Department and education field more broadly and could have been conducted at this time. GAO's initial research approach started with analysis of existing data in order to "compare enrollment level of ELL students in charter schools and traditional public schools."[Footnote 1] A school-level dataset of ELL enrollment has not been part of the EDFacts collection because, consistent with our program statutes, this dataset has not been a need or requirement of Department program offices to date. Thus, we had not requested our grantees to take on this additional reporting burden. To help support GAO's first phase of their study, however, the Department provided GAO with a dataset containing the most closely related set of school-level data available within EDFacts at the time-- counts of "Limited English Proficient students enrolled in language instruction educational programs." As discussed in subsequent conversations with GAO, these data are not equivalent to ELL student enrollment counts. Individual schools might not have programs that meet State definitions of a language instruction educational program (LIEP), but they may enroll ELL students whose needs are served through other means, such as supported inclusion within regular classroom settings. In these cases, the Department would not expect data to be reported for that individual school within EDFacts, since no students at the school meet the State's definition of "Limited English Proficient students enrolled in language instruction educational programs." It is important to note that the reason the term "program" in this data collection is left to State interpretation is that this is consistent with Title III program requirements. The Department cannot and does not dictate the type of LIEP States or LEAs are to provide (e.g., ESL (English as a second language), bilingual, etc.), either under Title III or under civil rights laws. Under Title VI of the Civil Rights Act and the Equal Educational Opportunities Act, all school districts, including charter schools that are LEAs and LEAs that do not receive Title III funds, must provide an alternative English language program that is recognized as sound by educational experts in the field, is fully implemented, and is shown to be successful. Case law and memoranda issued by the Department's Office for Civil Rights (OCR) provide guidance for programs, but do not prescribe a particular type of program that a school district must adopt to serve LEP students. School districts that receive Title III subgrants are required to provide services to ELLs that supplement the district's basic English language program. We are hopeful that GAO comments about "data quality" or "completeness" will be viewed in this context and the context set forth below. We understand that in conducting this study, GAO was interested in the total number of ELLs enrolled in schools, not the number served by English language instruction programs. While a school-level dataset of total ELL enrollment is not currently collected or available for the reasons noted above, the Department will have two datasets with this information in the future. The Civil Rights Data Collection for school year (SY) 2011-12 contains ELL enrollment data for all schools, and the Department anticipates that these data will be available for use by early 2014. In addition, the EDFacts data collection has already obtained OMB clearance to expand its SEA- and LEA-level collection of ELL enrollment counts to include school-level data starting in SY 2013- 14. These datasets should facilitate further research into the enrollment of ELL students in charter schools and traditional public schools, and we are hopeful that these datasets will be useful to GAO and the public in the future. As currently drafted, the report includes the following recommendations for Executive action: Recommendation 1: To more fully understand and address nonreporting of charter school data, conduct a systematic evaluation of other important data sets such as, datasets on academic achievement and graduates/completers to determine the extent of charter school nonreporting, assess underlying reasons for such nonreporting, and determine the steps needed to address nonreporting. The Department already had efforts underway to improve the completeness and quality of the charter school data prior to GAO's study. We have been aware of data concerns regarding charter schools, and we began addressing them in July 2011, with a systematic review and reconciliation of directory data across three independent sources: 1) charter school directory data reported through EDFacts, 2) grant and subgrant award data from the Charter Schools Program, and 3) the National Charter School Resource Center's authorizer survey. Maintaining a complete and accurate directory of charter schools is the first step to ensuring high quality reporting of such schools and the students they serve, since schools must first be included in the directory before additional student counts can be reported for them. The Department has taken additional steps to improve charter school inclusion in all aspects of EDFacts data collection as part of our overall commitment to ensuring a complete and accurate data collection across all schools in every State. The lessons and outcomes from the directory review are informing our development of a charter schools reporting workbook, which will assist States, along with traditional and charter school districts and authorizers, in reporting complete and high quality data. This workbook will be discussed during a session at the National Center for Education Statistics' July 2013 STATS-DC Data Conference. The session will emphasize the importance of reporting complete and accurate charter schools data and will provide technical assistance to the SEA staff responsible for EDFacts data submissions. We believe that our directed attention to the charter school directory, as well as technical assistance to SEAs through our EDFacts Partner Support Center, will encourage closer examination of the charter school data that SEAs are reporting to EDFacts. Additionally, we expect that the forthcoming workbook and conference session will foster more open communication between stakeholders who manage charter school data at the State, LEA, and school levels. Recommendation 2: Because the problems we identified with the school- level data also affect the LEA-level data, explore whether the collection of the LEA- and SEA-level counts of ELLs enrolled in English language instruction educational programs can be phased out, if the data quality problems are not addressed. The Department continually reviews its inventory of reporting requirements, and remains open to modifying our reporting requirements when a more useful, and higher quality, alternative can be identified. We must continually weigh the value of alternatives that may produce more useful and higher quality data against the burdens the alternatives impose. Additionally, we routinely assess our regulatory data reporting requirements and seek to streamline them by maintaining items with high utility for supporting improvements in educational outcomes and accountability and eliminating items with high administrative costs and low utility. As discussed above, the EDFacts data collection has been approved by OMB to expand our SEA- and LEA- level collection of total ELL enrollment counts to include school- level data starting with SY 2013-14. The Department's Office of Elementary and Secondary Education serves as the steward for the total ELL enrollment counts across all levels and conducts an annual review of the data submitted by States. Regarding the counts of "Limited English Proficient students enrolled in language instruction educational programs," prior to this GAO study we had already determined that EDFacts data on student participation in LIEP was of limited utility. For example, because the Title III program is limited by statute to overseeing SEA-level and LEA-level compliance, the school-level data is not utilized for Title III enforcement purposes. Therefore, the Department could no longer justify continuing the burden of collecting these school-level data in addition to the ELL enrollment counts being added at the school level for 2013-14. As we consider any changes to our data reporting requirements and data usage, we are taking the necessary steps to meet the needs of various data users. The Department is exploring what the impact would be upon the Common Core of Data (CCD) if reporting was switched from LEA-level counts of ELL students enrolled in language programs to a count of all ELL students enrolled, regardless of program participation. However, there are purposeful definitional differences in these items that may preclude the Department from simply eliminating the LEA and SEA collections as GAO suggests. Both the Title III program officials and the Department's OCR have an interest in the data on students enrolled in English language programs as reported by States and school districts. These two offices are discussing the option of providing instructions and other guidance to States and districts on the definition of "English language instruction educational programs" in the data collection in order to improve the quality and quantity of responses. During this evaluative phase, it is important that the Department continue to collect both data groups in order to assess the likely impact of any change for all users. In addition to these comments, we have included an enclosure with technical points of clarification to the draft report, which I trust GAO will find useful. Please let us know if you have any questions about our comments in this letter or our clarification in the enclosure. Thank you for the opportunity to provide comments and clarification on the draft report. Sincerely, Signed by: Denise Forte: Acting Assistant Secretary: Enclosure — Technical Edits: Enclosure 1 Footnote: [1] GAO, 'English Language Learners in Charter Schools (131193) Entrance Conference July 31, 2012; 11:00 a.m. ET' agenda document. [End of section] Footnotes: [1] GAO, Charter Schools: Additional Federal Attention Needed to Help Protect Access for Students with Disabilities, [hyperlink, http://www.gao.gov/products/GAO-12-543] (Washington, D.C.: June 7, 2012). [2] See section 6.65 of Government Auditing Standards, 2011 Revision, [hyperlink, http://www.gao.gov/products/GAO-12-331G] (Washington, D.C.: December 2011). [3] Generally, local educational agencies are school districts. [4] 20 U.S.C. §§ 6311(b)(3)(C)(ix)(III) and 6311(h)(1)(C)(i). [5] The District of Columbia is included as one of the 40 states. [6] To lessen the reporting burden on SEAs, for many datasets, Education does not require school-level data and instead requires only LEA-or SEA-level data. Requiring school-level data would greatly increase the quantity of data that SEAs must submit because there are many more schools than LEAs and SEAs. [7] In Education's datasets, the term "limited English proficient" (LEP) is used to refer to ELLs. The school-level dataset we used was counts of "LEP students enrolled in English language instruction educational programs designed for LEP students," referred to as data group 123. [8] The CRDC is conducted approximately every other school year and usually covers only a sample of schools. Education is in the process of collecting school year 2011-12 CRDC data for all schools, and Education officials estimated that these data will be available for use in late 2013 or early 2014. [9] This dataset is of limited English proficient students "enrolled in an elementary or secondary school at any time during the school year" and is referred to as data group 678. [10] The Title III program provides grants to SEAs to improve the education of ELLs by helping them learn English and meet challenging state academic content and student academic achievement standards. Title III program grants are authorized by Title III of ESEA. [11] In addition, some states allow charter schools to be their own LEA for some purposes and part of a larger LEA for others, including for purposes of special education. [12] Some charter school LEAs comprise more than one charter school. In such cases, the LEA would submit data for all charter schools within the LEA. [13] We found that 21 percent of traditional public schools had blank ELL enrollment counts. However, the analyses that we and Education conducted on schools with blank ELL counts focused on charter schools because their higher percentage of blanks (37 percent) would likely hinder our ability to make meaningful comparisons between charter schools and traditional public schools. [14] In this report, we refer to schools as leaving fields blank in Education's dataset. As shown in figure 1, school data originate with schools but are reported to Education by SEAs. Because of the SEA's role of compiling and submitting data, it is possible that some SEAs received ELL counts for schools that they did not to submit to Education. [15] According to data from the American Community Survey, New York and New Jersey had the 3RD and 7THhighest numbers of ELLs, respectively, among the 50 states based on 2009 3-year estimates. Specifically, these ACS data are on the number of individuals ages 5 to 21 who speak a language other than English at home and speak English less than very well. [16] Federal agencies are responsible for establishing and maintaining internal control to achieve their objectives such as enforcing compliance with federal laws and regulations, and having relevant, reliable, and timely information is a standard element of internal control. See OMB Circular A-123, "Management's Responsibility for Internal Control", revised December 21, 2004 and GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999). [17] Currently, this dataset (referred to as data group 678) is collected only at the LEA and SEA levels. [18] Education also provided GAO with ELL enrollment data for school year 2009-10, but GAO's detailed analysis, discussed in this letter, focused on the school year 2010-11 data because it was the most recent available data. [End of section] GAO’s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select “E-mail Updates.” Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. GAO Job Code: 131193: [End of document]