This is the accessible text file for GAO report number GAO-09-619 entitled 'District Of Columbia Public Schools: Important Steps Taken to Continue Reform Efforts, But Enhanced Planning Could Improve Implementation and Sustainability' which was released on July 23, 2009. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to the Subcommittee on Oversight of Government Management, the Federal Workforce, and the District of Columbia, Committee on Homeland Security and Governmental Affairs, U.S. Senate: United States Government Accountability Office: GAO: June 2009: District Of Columbia Public Schools: Important Steps Taken to Continue Reform Efforts, But Enhanced Planning Could Improve Implementation and Sustainability: GAO-09-619: GAO Highlights: Highlights of GAO-09-619, a report to the Subcommittee on Oversight of Government Management, the Federal Workforce, and the District of Columbia, Committee on Homeland Security and Governmental Affairs, U.S. Senate. Why GAO Did This Study: In response to long-standing problems with student achievement and the management of the District of Columbia (D.C. or the District) public school system, the D.C. Council approved the Public Education Reform Amendment Act of 2007. This act made major changes to the governance of the D.C. public school system, giving the Mayor authority over public schools. This report follows a GAO testimony in March 2008 and focuses on the primary reform approaches the District has taken. This report examines the steps the District took to: (1) address student academic achievement; (2) strengthen the quality of teachers and principals; (3) develop long-term plans and involve stakeholders; and (4) improve accountability and performance of the D.C. public schools (DCPS) and the state superintendent’s central offices. GAO reviewed documentation on District initiatives, and interviewed District education officials as well as representatives from the teachers’ union, community organizations, and research institutions. GAO also conducted visits to four urban school districts with mayoral governance. What GAO Found: Early efforts to improve student achievement at DCPS have focused on improving student performance, closing underutilized and reorganizing underperforming schools, and creating and enhancing data systems. During the first 2 years of its reform efforts, DCPS implemented many initiatives to improve overall student performance, such as classroom- based initiatives to improve basic skills of students. In addition, under the No Child Left Behind Act, DCPS restructured 22 schools before the fall of 2008, after the schools failed to meet academic targets for 6 consecutive years. Finally, DCPS and the state superintendent’s office are developing new ways to monitor student achievement and school performance. Specifically, a longitudinal database is being developed that is intended to allow DCPS and other key users to access a broad array of data, including student test scores. DCPS is modifying its approach to many of these initiatives such as focusing on effective teaching as opposed to implementing disparate programs. DCPS has focused on improving the quality of its workforce by replacing teachers and principals and by providing professional development, but it has encountered challenges in effectively implementing these changes. After the 2007-2008 school year, about one-fifth of the teachers and one-third of the principals resigned, retired, or were terminated from DCPS. However, because DCPS did not have an effective way to evaluate teacher performance, officials are uncertain if the new staff improved the quality of its workforce. DCPS is currently working on a new teacher evaluation system. In addition, DCPS introduced professional development initiatives for teachers and principals. For example, it began placing teacher coaches at schools to support teachers at their work sites. However, late decisions to hire these teacher coaches led to inconsistent implementation of this initiative during the 2008-2009 school year. The state superintendent’s office and DCPS each developed their 5-year strategic plans and involved stakeholders in developing these plans. The state superintendent plan and the DCPS draft strategic plan each contain many elements of effective plans, such as aligning short-term objectives to long-term goals. DCPS has recently increased its efforts to involve stakeholders in various initiatives; however, it has not always involved stakeholders in key decisions and initiatives. DCPS and the state superintendent’s office have taken steps to improve accountability and performance. For example, both offices have started implementation of new individual employee performance management systems. However, while DCPS has taken some additional steps to improve accountability, it has not yet linked its employee expectations and performance evaluations to organizational goals to improve central office operations. What GAO Recommends: GAO recommends that the Mayor direct DCPS to (1) establish planning processes that include evaluating internal capacity and involving stakeholders at key junctures; and (2) link individual performance evaluations for central office employees to organizational goals to strengthen accountability. The District’s education offices agreed with GAO’s recommendations and provided additional information which was incorporated as appropriate. View [hyperlink, http://www.gao.gov/products/GAO-09-619] or key components. For more information, contact Cornelia Ashby at (202) 512- 7215 or ashbyc@gao.gov. [End of section] Contents: Letter: Background: Early Initiatives Are Focused on Improving Student Achievement and DCPS Is Modifying Its Approach as It Moves Forward: DCPS Replaced Teachers and Principals and Introduced Professional Development Initiatives, but Encountered Challenges in Implementation: The State Superintendent's Office and DCPS Have Developed and Begun Implementing Strategic Plans; However, DCPS Has Not Always Involved Relevant Stakeholders in Planning and Implementing Key Initiatives: DCPS and the State Superintendent's Office Have Taken Steps to Improve Accountability and Performance, and DCPS Has Yet to Align Key Aspects of Its Performance Management System to Organizational Goals: Conclusions: Recommendations to the Mayor of the District of Columbia: Agency Comments and Our Evaluation: Appendix I: Comments from the D.C. Mayor's Office and District Education Offices: Appendix II: GAO Contact and Staff Acknowledgments: Related GAO Products: Tables: Table 1: Status of Major Academic Initiatives during the First 2 Years of DCPS Reform Efforts: Table 2: Options DCPS Selected for Schools Implementing Restructuring, SY 2008-2009: Table 3: Status of Key DCPS Data Initiatives: Table 4: Status of Key SLED Deliverables: Table 5: DCPS's Goals, as Outlined in Its Strategic Plan: Table 6: Key Elements of Strategic Plans and Their Inclusion in the State Superintendent's Office and DCPS Plans: Figures: Figure 1: D.C. Public Schools Governance Structure, prior to the 2007 Reform Act and after the Reform Act: Figure 2: Federally Mandated School Improvement Status for DCPS, SY 2008-2009: Figure 3: The Flow of Teachers into and out of DCPS between the 2007- 2008 and 2008-2009 school years: Abbreviations: AYP: adequate yearly progress: D.C. Council: Council of the District of Columbia: DCPS: District of Columbia public schools: Education: U.S. Department of Education: NCLBA: No Child Left Behind Act: Recovery Act: American Recovery and Reinvestment Act: Reform Act: Public Education Reform Amendment Act of 2007: SLED: Statewide Longitudinal Education Data Warehouse: [End of section] United States Government Accountability Office: Washington, DC 20548: June 26, 2009: The Honorable Daniel K. Akaka: Chairman: The Honorable George V. Voinovich: Acting Ranking Member: Subcommittee on Oversight of Government Management, the Federal Workforce, and the District of Columbia: Committee on Homeland Security and Governmental Affairs: United States Senate: The District of Columbia's (D.C. or the District) public school system has had long-standing problems with student academic performance, the condition of school facilities, and the overall management of the D.C. school system. For example, test scores have lagged behind those of most other urban districts in a nationally administered test. Further, the District's public schools have fallen well behind the District's own targets for demonstrating adequate yearly progress toward meeting the congressionally mandated goal of having 100 percent of students proficient in math, reading, and science by 2014, as outlined in the Elementary and Secondary Education Act of 1965, as amended by the No Child Left Behind Act (NCLBA). Some parents have been enrolling their children in charter schools, and student enrollment in D.C. public schools (DCPS)[Footnote 1] has declined from 65,000 in 2000-2001 to 45,200 in 2008-2009, a decline of about 30 percent.[Footnote 2] Of the nearly $762 million the District spends on DCPS, 16 percent comes from federal sources. There is an increasing urgency to address these persistent problems. In an effort to address the lack of academic progress, declining enrollment, and dilapidated buildings, the Council of the District of Columbia (D.C. Council) approved the Public Education Reform Amendment Act of 2007 (Reform Act), which made major changes to the operations and governance of the school district.[Footnote 3] The act gave the Mayor broad authority over the District's public school system, including curricula, operations, budget, personnel, and school facilities. In doing so, the District joined a growing number of cities to adopt mayoral governance of public school systems in an effort to expedite major reforms. Because of the District's past struggles to reform its public school system and the broad changes in governance, Congress asked GAO to evaluate the District's reform efforts. As part of this evaluation, we testified in March 2008 about the status of the reform efforts. [Footnote 4] In that testimony, GAO recommended that the Mayor direct the D.C. Department of Education to develop a long-term district-wide education strategic plan that would include certain key elements including a mission or vision statement, long-term goals and priorities, and approaches and time frames for assessing progress and achieving goals. To provide further evaluation of the District's primary reform efforts, we addressed the following questions: (1) What steps has the District taken to address student academic achievement? (2) What actions has the District taken to strengthen the quality of teachers and principals? (3) To what extent have the District's education offices[Footnote 5] developed and implemented long-term plans and how has DCPS used stakeholder input in key initiatives? (4) What steps have DCPS and the state superintendent's office taken to improve their accountability and performance? To answer these questions, we reviewed and analyzed relevant documents and research and interviewed officials from the District's education offices, such as DCPS and the Office of the State Superintendent of Education. For example, we interviewed high-ranking officials such as the State Superintendent of Education; the Chancellor of DCPS; and departmental leaders at DCPS, including the Interim Chief Academic Officer, the Deputy Chancellor for Human Resources and External Relations, the Chief of Data and Accountability, and the head of the Transformation Management Office. We also interviewed representatives of education and research associations, and various organizations based in the Washington, D.C. community. Across all our objectives, we measured the progress of ongoing reform efforts against any implementation time frames established by DCPS or the state superintendent's office. We based our evaluation of completed initiatives on relevant recognized standards, such as those established by GAO in past reports. To understand the steps that have been taken to address student achievement, we gathered information about the various academic initiatives DCPS has planned or implemented, and reviewed and analyzed documentation of how schools were restructured. We assessed the reliability of the restructuring data for the 2008-2009 school year by reviewing documentation and interviewing agency officials knowledgeable about the data. We determined that the data were sufficiently reliable for the purposes of this report. We also interviewed eight principals from schools that had been consolidated as a result of the closure of 23 schools prior to the 2008-2009 school year primarily due to declining enrollments.[Footnote 6] We reviewed and analyzed documentation--including state-level contracts and project plans--regarding efforts to improve data collection, quality, and usage. We did not independently verify reported improvements to DCPS and state superintendent's office data systems. To understand how the District was strengthening teacher and principal quality, we reviewed documents regarding teacher and principal recruitment, development, and evaluation, as well as licensure requirements. We also interviewed officials from the Washington Teachers' Union, the American Federation of Teachers, and the Council of School Officers (principals' union). To address the question on strategic planning and stakeholder involvement, we reviewed and analyzed strategic plans, interviewed cognizant officials from the District's education offices, and attended public discussions relevant to strategic planning, as well as D.C. Council hearings. We also interviewed the Chairman of the D.C. Council and representatives from several community and education organizations, including DC VOICE, the Council of the Great City Schools, and Parents United for the D.C. Public Schools.[Footnote 7] To understand the steps that DCPS and the state superintendent's office have taken to improve performance and accountability, we reviewed the alignment of the individual performance management plans to organizational goals and the results of DCPS's internal customer satisfaction survey. To provide a broader national context for our work, we visited four urban school districts with mayoral governance: Boston, Chicago, Cleveland, and New York City. We based our selection of these districts on how long the school district had been under mayoral control and student demographic information. We interviewed high-level officials--including superintendents and former superintendents, school board presidents and members, officials from mayors' offices--as well as union leaders, and representatives from various community and research organizations in these cities. In addition, we asked officials in the District's education offices about their planned and actual use of economic stimulus funds provided under the American Recovery and Reinvestment Act (Recovery Act).[Footnote 8] We performed our work from May 2008 through June 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: The District's Public School System: The District's prekindergarten through grade 12 school system is composed of 128 public schools[Footnote 9] with enrollment for the 2008- 2009 school year around 45,200. Historically, DCPS has had several problems that interfere with the education of its students. One primary problem was the dysfunction of the central office. For example, textbooks were not delivered on time or at all, parents complained about the lack of responsiveness of the central office, and teachers were not always paid on time. In addition, data systems were obsolete and inundated with errors, making it difficult to access basic information, such as the number of students enrolled at a school and student attendance rates. Such problems persisted in the D.C. public school system for several years despite numerous efforts to address them. In 1989, a report by the D.C. Committee on Public Education noted declining achievement levels as students moved through grades, the poor condition of the school system's physical facilities, and the lack of accountability among D.C. agencies for the schools.[Footnote 10] Recent reports have continued to cite these problems. In 2004, the Council of the Great City Schools reviewed the D.C. school system and cited the continued failure to improve student achievement.[Footnote 11] Efforts to improve the District's schools often included new leadership to head the troubled school system. Over the last 20 years, DCPS has employed more than seven superintendents with an average tenure of 2.9 years. Such frequent changes in leadership may have further complicated efforts to improve student achievement, as each leader may have brought a different cadre of initiatives and goals which were not fully developed or implemented with the constant changes in leadership. In 2006, an analysis of the school system's reform efforts by a consulting firm found no progress in student achievement and recommended a change in governance to improve student achievement and system-wide accountability.[Footnote 12] The Reform Act: In response to the problems facing the District's public school system, the D.C. Council (the legislative branch of the D.C. government) approved the 2007 Reform Act, which significantly altered the governance of the D.C. public schools. The Reform Act transferred the day-to-day management of the public schools from the Board of Education to the Mayor and placed DCPS under the Mayor's office as a cabinet- level agency. Prior to the Reform Act, the head of D.C. public schools was selected by and reported to the Board of Education. The Reform Act also moved the state functions into a new state superintendent's office, established a separate facilities office, and created the D.C. Department of Education headed by the Deputy Mayor for Education. The Deputy Mayor's Office and the state superintendent's office are also cabinet-level offices in the D.C. government structure. Figure 1: D.C. Public Schools Governance Structure, prior to the 2007 Reform Act and after the Reform Act: [Refer to PDF for image: illustration] Before the Reform Act of 2007: Top level: Mayor. Second level, reporting to the Mayor: * State Education Office[B]; Second level, no upward reporting: * Board of Education. Third level, reporting to the Board of Education: * District of Columbia Public Schools (DCPS); * State Education Agency[A]; * Local Education Agency; * Office of Facilities Management. After the Reform Act of 2007: Top level: Mayor. Second level, reporting to the Mayor: * Department of Education headed by Deputy Mayor (new entity established by the Reform Act); * District of Columbia Public Schools (DCPS) headed by Chancellor(new entity established by the Reform Act); * Office of the City Administrator[D]. Third level, reporting to the Department of Education headed by Deputy Mayor: * Office of the State Superintendent of Education (new entity established by the Reform Act); * Office of Public Education Facilities Modernization (new entity established by the Reform Act); * Office of the Ombudsman for Public Education (new entity established by the Reform Act); * Interagency Collaboration and Services Integration Commission (new entity established by the Reform Act). Fourth level, reporting to the Office of the State Superintendent of Education: * State Board of Education[C]; * Public Charter Schools[E]. Note: The Office of the State Superintendent of Education provides oversight, monitoring and technical assistance to DCPS for federal and state education programs. Source: GAO analysis based on The Parthenon Group, December 2006 and D.C. government documents. [A] The State Education Agency had responsibility for administering federal grant money, and setting state academic standards, achievement goals, and graduation requirements. The Local Education Agency, typically referred to as the school district, had responsibility for the management and operations of D.C. public schools. [B] The State Education Office administered food nutrition programs and state scholarship grants. [C] The State Board of Education was the Board of Education prior to the Reform Act. [D] The Office of the City Administrator prepares the District's annual operating budget and provides direction to all District agencies, including DCPS. Prior to the Reform Act, the City Administrator did not have this role because the Mayor did not have direct oversight of DCPS. [E] The public charter schools comprised 59 school districts as of March 2009. The charter schools often consist of just one school (some charters have multiple campuses). The Public Charter School Board also has some oversight of the District's public charter schools. [End of figure] Office of the State Superintendent of Education: Although the District of Columbia is not a state, its Office of the State Superintendent of Education serves as the District's state education agency. Prior to the Reform Act, state functions and local functions were conducted in one office which led to problems with oversight and monitoring. Further, the District was and continues to be on the U.S. Department of Education's (Education) high-risk list for its management of federal education grants. The Reform Act addressed such issues by clearly separating the two entities. Along with managing, distributing, and monitoring the use of federal funds across DCPS and the public charter schools,[Footnote 13] the office of the state superintendent has a significant policy role. For example, the state superintendent's office works collaboratively with the State Board of Education to set standards of what students should learn in all the District's public schools. In addition, in carrying out NCLBA, the state superintendent's office is responsible for the state-wide assessment, or standardized test, that measures students' progress in attaining proficiency and sets annual proficiency targets.[Footnote 14] The state superintendent's office also delineates requirements for teacher licensure and, within the guidelines provided by NCLBA, determines the District's definition of "highly qualified teachers." In addition to these policy functions, the state superintendent's office also provides support to D.C. public schools and the public charter schools. For example, the office can offer training and technical assistance on a variety of topics, such as the appropriate use and tracking of federal education funds. NCLBA and the Recovery Act: In January 2002, Congress passed NCLBA which requires states to focus on increased expectations for academic performance and accountability. Under NCLBA, states are required to establish performance goals and hold schools that receive federal funds under Title I of NCLBA accountable for student performance by determining whether or not they have made adequate yearly progress (AYP). The failure to make AYP, or meet academic targets, for 2 or more consecutive years leads to specific actions that schools must take to improve student academic achievement. These actions, such as developing a school improvement plan or extending the school day, are more intensive the longer the school fails to meet academic targets. After 5 or more consecutive years of failing to meet academic targets, a school must make plans to restructure its governance and implement those plans the subsequent year. NCLBA specifies five options for restructuring schools: reopening as a charter school, replacing all or most of the school staff relevant to the failure to make AYP, contracting with another organization to run the school, turning the operation of the school over to the state, or undertaking another action that would result in restructuring the school's governance. NCLBA also establishes a federal requirement for teacher quality. It requires that teachers across the nation be "highly qualified" in every core subject they teach by the end of the 2005-2006 school year. In general, NCLBA requires that teachers have a bachelor's degree, have state certification, and demonstrate subject area knowledge for every core subject they teach. States also have flexibility to set the requirements that teachers need to meet to demonstrate that they are highly qualified.[Footnote 15] In March 2008, the state superintendent's office and the D.C. State Board of Education revised the District's highly qualified teacher definition to better align it with NCLBA's definition and allow more teachers to be considered highly qualified. Officials from the state superintendent's office contend that the District's previous highly qualified definition was more stringent than federal standards and disqualified good teachers from joining the D.C. public school system. The Recovery Act was enacted in February 2009 to promote economic recovery, make investments, and minimize and avoid reductions in state and local government services. About $100 billion of the $787 billion funds included in the Recovery Act are targeted to support education at the state and local level. Some of the Recovery Act funds support existing programs, such as Title I of the Elementary and Secondary Education Act, as amended by NCLBA, and parts of the Individuals with Disabilities Education Act. In addition, the new State Fiscal Stabilization Fund provides funds to restore state support for elementary and secondary education, public higher education, and early childhood education programs and services. The District will receive an estimated $148 million of Recovery Act funds to support its education programs. Pay for Performance Initiatives: The current teacher compensation system used by most school districts in the United States dates back to the 1920s and pays teachers based on their level of education and years of experience. However, many school districts have begun to experiment with alternative methods of compensation that reward teachers on certain elements of performance, such as improving student achievement, filling hard-to-staff positions, and taking on additional responsibilities. Some school districts offer bonuses for all staff or all teachers at schools who have met certain criteria (usually including an increase in student achievement). Other school districts offer differentiated pay to teachers based on characteristics other than education and years of experience. For example, the Denver Public School District has implemented a teacher compensation plan that allows multiple pathways to compensation bonuses. Bonuses can be based on professional evaluations using a standards-based system, progress toward objectives as agreed upon by teachers and their principal, and growth in student achievement on the Colorado Student Assessment Program. Teachers may receive additional incentives for filling hard-to-staff positions. The Denver plan is funded through a tax levy, federal grants, and private funding. National teachers' unions approve of some types of differentiated or incentive pay. Specifically, the American Federation of Teachers, which is the parent union of the Washington Teachers' Union, has taken the position that teacher compensation plans could include financial incentives to teachers who acquire additional knowledge and skills or agree to teach in low-performing and hard-to-staff schools. In addition, the American Federation of Teachers supports incentive pay for school-wide improvement. Early Initiatives Are Focused on Improving Student Achievement and DCPS Is Modifying Its Approach as It Moves Forward: During the first 2 years of its reform efforts, DCPS implemented several classroom-based initiatives to improve students' basic skills in core subjects and implemented a new staffing model designed to give all students access to art, music, and physical education classes. In addition, as required by NCLBA, DCPS restructured 22 schools before the fall of 2008, after the schools failed to meet academic targets for 6 consecutive years. Restructuring will be ongoing as the vast majority of DCPS schools are in some form of school improvement status under NCLBA. In addition, DCPS and the state superintendent's office are planning and developing new ways to use data to monitor student achievement and school performance. DCPS is refocusing or revising its approach to many of these initiatives as it continues to implement them. DCPS Quickly Implemented Many Separate Initiatives to Improve Overall Student Performance and DCPS Is Refocusing Its Approach as It Moves Forward: During the first 2 years of reform, DCPS quickly implemented various initiatives intended to improve student achievement. For example, to improve students' basic skills and standardized test scores in reading and math, DCPS introduced targeted interventions for students struggling in math and reading[Footnote 16] and provided additional instruction and practice to improve students' responses to open-ended questions, including test questions. DCPS also introduced Saturday classes primarily targeted to students in grades 3 through 12 who were on the cusp of meeting academic targets on standardized tests. It also introduced initiatives designed to address student motivation and behavior. For example, DCPS piloted the Capital Gains program with the specific goals of improving student engagement, and ultimately student learning, by offering financial incentives to students for attendance, academic performance, and other positive behaviors. Table 1 provides a list of DCPS's major initiatives to improve student outcomes, as well as descriptions and the status of these initiatives. Table 1: Status of Major Academic Initiatives during the First 2 Years of DCPS Reform Efforts: Initiative: Reading and math interventions; How it works: Provides supplemental intensive instruction and practice for struggling students; The initiatives target: Reading and math interventions: [Check]; Standardized test scores: [Check]; Student engagement[A]: [Empty]; School year (SY) initiative was (or will be) launched and status: Being revised: [Check]. Initiative: Saturday classes for targeted students; How it works: Extends class time to Saturdays; primarily targeted to students close to meeting academic targets; The initiatives target: Reading and math interventions: [Check]; Standardized test scores: [Check]; Student engagement[A]: [Check]; School year (SY) initiative was (or will be) launched and status: Being revised: [Empty]. Initiative: Targeted instructional practices; How it works: Provides additional practice on answering short answer test questions, using calculators, and playing math games; The initiatives target: Reading and math interventions: [Check]; Standardized test scores: [Check]; Student engagement[A]: [Empty]; School year (SY) initiative was (or will be) launched and status: Being revised: [Empty]. Initiative: Pacing guides; How it works: Provides guidance to teachers to help focus instruction on what students are expected to know and testing timetable; The initiatives target: Reading and math interventions: [Check]; Standardized test scores: [Check]; Student engagement[A]: [Empty]; School year (SY) initiative was (or will be) launched and status: Being revised: [Check]. Initiative: Capital Gains; How it works: Offers money to students for attendance, behavior, and academic performance; The initiatives target: Reading and math interventions: [Empty]; Standardized test scores: [Empty]; Student engagement[A]: [Check]; School year (SY) initiative was (or will be) launched and status: Being revised: [Empty]. Initiative: Staffing model; How it works: Provides access to art, music, and physical education as well as supports for all students; The initiatives target: Reading and math interventions: [Empty]; Standardized test scores: [Empty]; Student engagement[A]: [Check]; School year (SY) initiative was (or will be) launched and status: Being revised: [Check]. Initiative: Teaching and learning framework; (planned); How it works: Provides guidance to teachers on how to plan, deliver, and evaluate instruction; The initiatives target: Reading and math interventions: [Check]; Standardized test scores: [Check]; Student engagement[A]: [Check]; School year (SY) initiative was (or will be) launched and status: Being revised: [Empty]. Source: GAO analysis of DCPS documents. [A] Engaging students in their schooling involves focusing on ways to increase attendance, classroom participation, and other positive behaviors. Such activities may increase student motivation, and eventually may even increase academic achievement. Recently, the Chancellor acknowledged that DCPS, in its effort to remedy the range of issues that plagued the District's public schools, may have launched too many initiatives at once. The Chancellor noted that some schools may have lacked the capacity to implement so many programs effectively. In particular, some schools were undergoing significant organizational changes that may have affected their ability to implement these new academic initiatives. To support such schools, DCPS is considering offering a choice of programs for schools and allowing the principals to determine which programs best suit their schools' needs and capacity. DCPS does not yet know how successful these programs have been in improving student achievement. While DCPS students achieved gains on the 2008 state-wide test, increasing between 8 and 11 percentage points in math and reading for both elementary and secondary levels, it is unclear whether these gains can be attributed to the current reform efforts or to prior efforts.[Footnote 17] While DCPS officials told us that it is generally difficult to isolate and quantify the impact of any single program on student achievement, they were able to review an analysis of reading scores conducted by the vendor of one of its early reading programs. The vendor's analysis showed that on some tests DCPS students who participated in the reading program generally scored higher than those who did not.[Footnote 18] Further, DCPS officials told us they plan to analyze, in late summer of 2009, student outcomes, including state-wide test scores, to assess the effectiveness of various interventions. In addition, DCPS officials told us the success of the math and reading initiatives depended in part on how well teachers implemented them in the classroom. They also noted that there were varying levels of teacher quality and knowledge of effective teaching practices, and that it was difficult to ensure the extent to which teachers implemented the programs effectively. While DCPS had not defined "effective" teaching prior to the rollout of the above initiatives, officials told us that moving forward, they will focus on practicing effective teaching, as opposed to implementing various disparate programs. DCPS is developing a framework that is intended to help teachers understand the priorities moving forward, including understanding what students are expected to learn for each subject, how to prepare lessons, and effective teaching methods to be used. According to DCPS officials, this framework will be aligned to teacher evaluations. DCPS plans to implement this framework by the beginning of the 2009-2010 school year. DCPS Encountered Challenges Allocating Teachers across Schools and Is Revising Its Approach for the 2009-2010 School Year: In an effort to ensure that all students would have access to certain subjects and supports, DCPS changed the way it allocated teachers across its schools for the 2008-2009 school year. This new staffing model was intended to provide all schools with a core of teachers including art, music, and physical education, as well as social workers. It also was intended to provide all schools with reading coaches who work with teachers to improve reading instruction. Prior to this change, DCPS allocated funding to schools using a weighted student formula, which distributed funds to schools on a per pupil basis, so that the greater the enrollment of a school, the greater the amount allocated to that school.[Footnote 19] Principals then chose how to staff the school based on the amount of funding available, staffing requirements, and their perception of the school's needs. Consequently, some schools--especially smaller schools--did not have the student enrollment to support programs,[Footnote 20] such as music and art, and other schools that had the funds to support those programs opted not to do so. While the new staffing model ensures a core staff at all schools regardless of enrollment,[Footnote 21] DCPS allowed principals to request changes based on their school's needs.[Footnote 22] However, DCPS lacked a transparent process for making changes to the staffing allocation. In particular, DCPS did not establish or communicate clear guidance or criteria on how such requests would be treated. Further, DCPS granted or denied requests for changes to the original staffing allocation on a school-by-school basis, and it is unclear whether similar requests were treated in a consistent manner. A more transparent process, one that made public their rationale for decisions, would have helped assure stakeholders, including the D.C. Council, that changes to staffing allocations were made consistently and fairly. The D.C. Council and several community groups have criticized the process for its lack of transparency and questioned the fairness of the decisions made. For example, one independent analysis concluded that some schools received less per pupil funding than others with similar student populations.[Footnote 23] In addition, DCPS officials told us that in some cases, the changes to the original staffing model resulted in schools being granted allocations beyond their budgeted amounts. DCPS revamped its approach for the staffing model for the 2009-2010 school year to address some of these challenges. For example, it established guidance about what changes it will allow principals to make to the staffing model and disseminated this guidance to school leaders at the beginning of the budgeting process. According to DCPS, the new guidance is expected to reduce the number of changes that principals request later in the process. DCPS Closed 23 Schools Primarily Due to Low Enrollment, Restructured 22 Other Schools as Required by NCLBA, and Is Changing Its Process for Selecting Restructuring Options: During the summer of 2008, DCPS closed 23 schools primarily due to low student enrollment. Students from the closed schools, about 5,000 students according to DCPS, enrolled in 1 of 26 schools, referred to as receiving schools. DCPS updated facilities at these receiving schools to accommodate the influx of students from the newly closed schools. In addition, to assist these students and schools with the transition that this reorganization created, DCPS offered a more comprehensive version of its staffing model. In addition to the core staff of the standard staffing model, DCPS allocated additional staff, such as school psychologists and math coaches to the receiving schools. During the consolidation effort, DCPS also created several prekindergarten through grade 8 schools in some cases where elementary schools were underenrolled. In addition, according to DCPS, these prekindergarten through grade 8 schools were intended to create a smoother transition to middle school and reduce the number of elementary schools with different grade levels preparing students for the same middle or junior high school. By closing the 23 underenrolled schools, DCPS estimates it was able to redirect $15 million from administrative and facility costs to support these additional staff. The eight principals we interviewed at receiving schools provided mixed reports about the adequacy of their staffing allocations. On the one hand, three principals reported having adequate staff, and two others cited minor issues[Footnote 24]. The remaining three principals cited issues such as teacher skill levels, teacher vacancies, and inadequate training to accommodate an influx of special education students. In addition, as required by NCLBA, DCPS restructured 22 of its lowest performing schools for the 2008-2009 school year[Footnote 25] after the schools failed to meet academic targets for 6 consecutive years. NCLBA specifies five options for restructuring schools, including replacing selected staff or contracting with another organization or company to run the school (table 2 lists the various NCLBA options and the options DCPS selected for the 2008-2009 school year). At 18 of the 22 schools in restructuring, DCPS replaced the school staff--principals, teachers, and/or administrative support staff--who were deemed relevant to the failure to meet academic targets. For the remaining schools in restructuring, DCPS elected to contract with other organizations or undertake other actions, such as adding more intensive school-level services to support students and families. Table 2: Options DCPS Selected for Schools Implementing Restructuring, SY 2008-2009: Type of school: Elementary; Option 1: Reopen school as a charter school: 0; Option 2: Replace the staff (which could include the principal) relevant to school not meeting academic targets: 3; Option 3: Contract with another organization or company to operate school: 0; Option 4: Turn operation of school over to the state: 0; Option 5: Implement any other major restructuring of school's governance[A]: 1. Type of school: Prekindergarten through grade 8; Option 1: Reopen school as a charter school: 0; Option 2: Replace the staff (which could include the principal) relevant to school not meeting academic targets: 2; Option 3: Contract with another organization or company to operate school: 0; Option 4: Turn operation of school over to the state: 0; Option 5: Implement any other major restructuring of school's governance[A]: 1. Type of school: Middle school/junior high; Option 1: Reopen school as a charter school: 0; Option 2: Replace the staff (which could include the principal) relevant to school not meeting academic targets: 7; Option 3: Contract with another organization or company to operate school: 1; Option 4: Turn operation of school over to the state: 0; Option 5: Implement any other major restructuring of school's governance[A]: 4. Type of school: High school; Option 1: Reopen school as a charter school: 0; Option 2: Replace the staff (which could include the principal) relevant to school not meeting academic targets: 6; Option 3: Contract with another organization or company to operate school: 4; Option 4: Turn operation of school over to the state: 0; Option 5: Implement any other major restructuring of school's governance[A]: 2. Type of school: Total[B]; Option 1: Reopen school as a charter school: 0; Option 2: Replace the staff (which could include the principal) relevant to school not meeting academic targets: 18; Option 3: Contract with another organization or company to operate school: 5; Option 4: Turn operation of school over to the state: 0; Option 5: Implement any other major restructuring of school's governance[A]: 8. Source: GAO analysis based on DCPS data. [A] These include actions such as adding more intensive school-level services to support students and families. [B] DCPS selected more than one option for some schools. [End of table] Restructuring underperforming schools will likely be an ongoing initiative for DCPS, as 89 of its 118 schools[Footnote 26] are in some form of school improvement status. (See figure 2 for more details on DCPS's school improvement status.) Figure 2: Federally Mandated School Improvement Status for DCPS, SY 2008-2009: [Refer to PDF for image: two pie-charts] Total number of schools = 118; Schools not meeting academic targets: 89; Schools meeting academic targets: 29. Total number of schools not meeting academic targets = 89: Schools in needs improvement (first year of improvement)[A]: 14; Schools in needs improvement (second year of improvement)[B]: 23; Schools in corrective action (third year of improvement): 17; Schools in restructuring planning (fourth year of improvement)[D]: 13; Schools in restructuring implementation (fifth year of improvement)[E]: 22. Source: GAO analysis based on DCPS data. [End of figure] [A] Schools in Needs Improvement Status (First Year of Improvement) have missed academic targets for 2 consecutive years. The school district must offer the students in these schools the opportunity to transfer to a higher-performing public school in the district (public school choice). Schools that miss academic targets for the first year are not placed in school improvement status and are not required to undergo any NCLBA interventions. [B] Schools in Needs Improvement Status (Second Year of Improvement) have missed academic targets for 3 consecutive years. The school district must offer students public school choice or supplemental education services (SES), such as tutoring. [C] Schools in Corrective Action Status (Third Year of Improvement) have missed academic targets for 4 consecutive years. The school district must implement at least one of six activities such as replacing selected staff or implementing a new curriculum. The district must also offer students public school choice or SES. [D] Schools in Restructuring Planning (Fourth Year of Improvement) have missed academic targets for 5 consecutive years. The school district is required to plan for a change in governance, such as replacing selected staff or contracting with another organization or company to run the school. The district must also offer students public school choice or SES. [E] Schools in Restructuring Implementation (Fifth Year of Improvement) have missed academic targets for 6 consecutive years. The school district is required to implement a change in governance. The district must also offer students public school choice or SES. DCPS revamped its process for determining the most appropriate restructuring option for the 13 schools that will be restructured in the 2009-2010 school year. Prior to implementing the first round of restructuring (i.e., for the 2008-2009 school year), DCPS officials told us there were insufficient school visits and inadequate training and guidance for teams assigned to evaluate which restructuring option was best suited for a given school. For example, the initial process called for review teams to visit each school once, which according to DCPS officials, did not allow the teams to obtain sufficient evidence to evaluate the schools' condition. DCPS has addressed these issues by requiring two visits to each school, offering more training, and revising the form used to evaluate each school's condition for the next round of restructuring. In addition, DCPS officials told us they cannot continue to rely on replacing teachers and principals as the primary restructuring option because DCPS cannot terminate the teachers,[Footnote 27] and moving these teachers to other schools may undermine the District's reform efforts. DCPS did not assess its capacity for replacing staff at schools restructured in the 2008-2009 school year. According to DCPS, nearly half of the 160 teachers that were removed from these schools had to be placed at 38 other DCPS schools.[Footnote 28] For the 2009- 2010 school year, DCPS has decided to replace select staff at 6 of the 13 schools that will be restructured. (For more details, see the section on teacher and principal quality later in this report.) DCPS and the State Superintendent's Office Are Working to Enhance and Create Data Systems to Monitor Student and School Performance: DCPS reported it has ongoing and planned initiatives to expand data access to principals and teachers, in part to monitor student and school performance. In particular, DCPS reported it made improvements to its primary student data system[Footnote 29] so central office users can better monitor school performance. For example, DCPS officials reported that they consolidated several student data systems by February 2009, including the system containing standardized test scores, into the primary student data system with the intent to improve data accuracy and consistency. They also told us they added software to the primary student data system that enabled central office employees to develop monthly reports of schools' performance data, such as attendance and test scores. DCPS plans to eventually use these monthly reports to enable school leaders to better monitor student progress, and plans to develop an internal Web site that compiles various student and school information in one place for key stakeholders including central office staff and principals. However, DCPS officials told us they have delayed some of these efforts while they attempt to improve coordination among the various departments that were developing and disseminating information to school leaders. DCPS has not yet announced when the project will be completed. See table 3 for more details about key DCPS data initiatives and their status. Table 3: Status of Key DCPS Data Initiatives: Initiative: Upgrade software and hardware on primary student data system; Status: Completed summer 2008. Initiative: Consolidate several data systems into primary student data system; Status: Completed by February 2009. Initiative: Release school performance data in monthly reports to key stakeholders; Status: Being revised; timeline has not been established. Initiative: Implement a Web-accessible school-level data system to generate reports on school performance and demographics; Status: Completed October 2008. Initiative: Launch internal Web site that links to various student and school information for key stakeholders; Status: Expected to be launched late summer 2009. Source: GAO analysis based on DCPS documents and interviews. [End of table] The state superintendent's office also is developing a longitudinal database, called the Statewide Longitudinal Education Data Warehouse (SLED) that is intended to allow DCPS and other stakeholders to access a broad array of information, including standardized test scores of students and information on teachers[Footnote 30]. SLED is intended to allow the District to track student registration and movement among DCPS's schools and the public charter schools more accurately, as well as expand the District's ability to monitor student achievement and growth over time. According to officials in the state superintendent's office, they revised the project schedule to allow more time to assist the charter schools with updating their data systems. In February 2009, the initial release of student data provided a student identification number and information on student eligibility for free or reduced-price lunches and other student demographics for all students attending DCPS's schools and the public charter schools. The state superintendent's office plans for SLED to enable DCPS to link student and teacher data by February 2010. (See table 4 for more details about the status of key SLED deliverables.) This link is to provide DCPS with data on the classes students enrolled in, the teachers that taught the classes, any academic interventions students received, students' grades and test scores, and student demographics. Table 4: Status of Key SLED Deliverables: SLED deliverable: Unique student identifier (student ID); Planned completion date: February 2009; Completed: [Check]. SLED deliverable: Free and reduced-price lunch indicator for students; Planned completion date: February 2009; Completed: [Check]. SLED deliverable: Historical state test scores from 2006-2008 linked to student ID; Planned completion date: June 2009; Completed: [Empty]. SLED deliverable: State test scores from 2009 linked to student ID; Planned completion date: July 2009; Completed: [Empty]. SLED deliverable: Comprehensive student data, including enrollment, grades, and demographics; Planned completion date: October 2009; Completed: [Empty]. SLED deliverable: Link student data to comprehensive teacher data, including unique teacher ID, classes and subjects taught, and certification; Planned completion date: February 2010; Completed: [Empty]. Source: GAO analysis based on Office of the State Superintendent documents. [End of table] DCPS Replaced Teachers and Principals and Introduced Professional Development Initiatives, but Encountered Challenges in Implementation: DCPS is attempting to improve the quality of its teacher and principal workforce by hiring new teachers and principals and by providing professional development. After the 2007-2008 school year, about one- fifth of the teachers and one-third of the principals resigned, retired, or were terminated from DCPS. However, DCPS officials told us that the 2007-2008 and 2008-2009 teacher evaluation process did not allow them to assess whether the teacher workforce improved between these 2 school years. In addition, DCPS introduced professional development initiatives for teachers and principals, but late decisions about the program for teachers led to inconsistent implementation. DCPS Focused on a Workforce Replacement Strategy to Strengthen the Quality of Teachers and Principals, but Is Unsure New Staff Are an Improvement: DCPS focused on a workforce replacement strategy to strengthen teacher and principal quality. DCPS maintains that the quality of teachers is the single greatest determinant to improving student achievement, and a growing body of research has shown that teacher quality is a significant factor in improving student academic performance.[Footnote 31] Yet it is often difficult to remove teachers for performance issues beyond their initial, or probationary, years in a given school system. For example, in the 2006-2007 school year, only 1 teacher was removed from DCPS for poor performance out of more than 4,000 teachers. Representatives from the Washington Teachers' Union agreed that there were several poor performing teachers in DCPS, but stated that the 2- year probationary period is the appropriate time to identify and dismiss poor teachers at will. DCPS began implementing its teacher replacement strategy near the end of the 2007-2008 school year. Specifically, about one-fifth of the teachers and one-third of the principals resigned, retired, or were terminated from the school system at the end of the 2007-2008 school year.[Footnote 32] DCPS terminated about 350 teachers, approximately 100 of whom were released for underperformance at the end of their probationary period, when tenure decisions were made. The remaining 250 teachers were terminated because they did not meet specified time frames to become highly qualified under NCLBA. An additional 400 teachers accepted financial incentives offered by DCPS to resign or retire in the spring of 2008. A DCPS official told us there is anecdotal evidence suggesting DCPS lost some quality teachers through the contract buyouts, but officials noted that DCPS did not have measures in place to deter effective teachers from accepting the buyouts. In addition, DCPS did not renew the contracts of 42 principals, citing their failure to improve student achievement on standardized tests and to adequately implement school-wide programs. To replace the teachers and principals who left the system, DCPS launched a nationwide recruitment effort for the 2008-2009 school year. DCPS hired 566 teachers and 46 principals for the 2008-2009 school year.[Footnote 33] Of the 566 teachers, 395 were hired from traditional backgrounds or other school systems and 171 came from nontraditional paths such as the D.C. Teaching Fellows program and Teach for America.[Footnote 34] (See figure 3 for more details about the flow of teachers into and out of DCPS between the 2007-2008 and 2008-2009 school years.) Figure 3: The Flow of Teachers into and out of DCPS between the 2007- 2008 and 2008-2009 school years: [Refer to PDF for image: illustration] 566 total teachers coming into DCPS School year 2008-2009: * 395 teachers from traditional backgrounds or other school systems; * 123 teachers from the D.C. Teaching Fellows program; * 48 teachers from Teach for America. 817 total teachers leaving DCPS After school year 2007-2008: * 400 teachers who accepted financial incentives from DCPS to resign or retire; * 250 teachers who were let go because they were past the 2-year deadline to become Highly Qualified under NCLBA; * 100 probationary teachers who were let go because they were deemed under performers; * 67 teachers who resigned or retired without financial incentives. Sources: GAO analysis; images, Art Explosion. [End of figure] However, DCPS did not have a new teacher contract in place due to ongoing negotiations with the Washington Teachers' Union and officials told us this may have hindered their efforts to attract top-quality teachers. The Chancellor has stated that she wants to recruit and retain quality teachers by offering merit pay, which would reward teachers with higher salaries based, in part, on their students' scores on standardized state tests. Under the plan, which has been in negotiation with the Washington Teachers' Union since November 2007, teachers could voluntarily relinquish job protections in exchange for base salaries and bonuses totaling over $100,000 per school year. This plan relies on over $200 million in contributions from private foundations to fund the teacher contract, including salary increases and professional development. According to the Chancellor, private foundations continue to pledge their support, even with the current economic downturn. DCPS officials told us the higher annual salaries and bonuses would be sustainable with public funds if private funding is not available when the 5-year contract expires. [Sidebar: Merit Pay for Teachers across School Systems with Mayoral Governance: In addition to DCPS, three of the four school systems under mayoral governance we visited—Chicago, Cleveland, and New York City—have implemented financial incentives, or merit pay, to reward teachers for student achievement gains. DCPS, the Chicago Public Schools, and the Cleveland Metropolitan School District all receive Teacher Incentive Fund grants from Education to help fund their merit pay programs. DCPS and the Chicago Public Schools use these grants to reward the entire staff of high-performing schools, including the principal, teachers, and administrative staff. The Cleveland Metropolitan School District also uses the grants to reward all school employees for achieving school-wide goals, but in addition rewards individual teachers for taking on extra duties and assignments while delivering student achievement gains. While New York City did not use federal funding for its merit pay initiative, the school system also rewards the entire staff of high-performing schools. End of sidebar] In addition, an official told us DCPS does not have an adequate means to assess whether its teacher workforce improved between the 2007-2008 and 2008-2009 school years because the current teacher evaluation system is not an effective way to assess teacher performance. Under this evaluation system, principals evaluate teachers' subject matter knowledge, classroom management skills, and adherence to academic standards, among other elements. However, this system does not measure teachers' impact on student achievement, which, according to DCPS, is a key factor in evaluating teacher effectiveness. In addition, according to DCPS, teacher evaluations conducted in prior years did not adequately distinguish excellent from poor performance--almost all teachers received satisfactory ratings. As a result, DCPS officials told us they cannot determine the quality of the 566 new teachers relative to the 817 teachers who left the system. The current teacher evaluation system remains the primary mechanism for identifying teachers considered ineffective. During the 2008-2009 school year, principals used the evaluation system to place 147 tenured teachers deemed underperforming on 90-day improvement plans. At the end of 90 school days, principals decide whether to retain or terminate these teachers. In prior years, DCPS did not use the 90-day process to this extent. DCPS plans to revise its teacher evaluation process to more directly link teacher performance to student achievement. The proposed system includes a value-added component that would measure teachers, in part, on their ability to improve students' standardized test scores over the course of a school year. This value-added measure would only apply to about 20 percent of the teacher workforce, since not all grades and subjects are tested. DCPS plans to use a less formal student achievement measure for teachers in nontested grades and subjects in the short term, but is working to increase the number of teachers for whom student achievement growth data are available. In addition, DCPS's proposed evaluation system would add classroom observations by third- party observers, called master teachers, who would be knowledgeable about teaching the relevant subject matter and grade level, to supplement school administrators' observations of teachers. To solicit input on the proposed evaluation system, the Chancellor held a series of sessions in spring 2009 with teachers, teacher coaches, and other school staff, and engaged the Washington Teachers' Union. DCPS officials told us that the feedback was generally positive and that teachers found the proposed evaluation system to be fair, transparent, and an improvement over the current evaluation. However, some teachers were concerned about using students' test scores as part of the evaluation. For the 2007-2008 school year, DCPS revised the principal evaluation system, which holds principals accountable for improvements in students' standardized test scores and achieving other standards. DCPS will be able to use this evaluation system to determine if principals performed better during the 2008-2009 school year than in 2007-2008. DCPS Introduced Teacher Coaches and a Principals' Academy to Provide Professional Development and Improve Skills, but Encountered Challenges in Implementation: In addition to the workforce replacement strategy, DCPS changed the way in which it develops its teacher workforce. DCPS began placing teacher coaches in schools to help teachers increase student achievement at their workplaces. Previously, DCPS's teacher training was not systematic or aligned with the school district's goals. For the 2008- 2009 school year, DCPS hired about 150 teacher coaches to improve teachers' skills in delivering reading and math instruction and boost student test scores.[Footnote 35] DCPS officials told us their decision to implement school-based teacher coaches was based on research demonstrating gains in student achievement as a result of teacher coaches collaborating with teachers to improve instruction. For the 2008-2009 school year, teacher coaches focused on helping new teachers and teachers with students in grades 3 through 10 in reading and math instruction. For example, teacher coaches, at the direction of principals, assisted teachers with interpreting student test scores, planning lessons, and using their classroom time constructively. DCPS is planning for teacher coaches to work with teachers in all grades and subjects for the 2009-2010 school year. Late hiring of teacher coaches, however, affected the implementation of the professional development plan for the 2008-2009 school year. DCPS officials told us they made the decision to hire teacher coaches after their review of school restructuring plans in June 2008. DCPS officials told us that, as a result of this late decision, they were unable to adequately recruit a sufficient number of qualified staff to fill these positions. Specifically, qualified teacher coach applicants had accepted jobs elsewhere, since many school systems recruit staff from February through April. [Sidebar: School-based Teacher Coaches Are a Growing Trend in U.S. Public Education: School-based teacher coach programs are increasingly popular in U.S. school districts. Typically in school-based coaching models, veteran teachers are assigned to provide continuous guidance and advice to teachers to help them improve their instruction. During our Boston site visit, officials told us that Boston Public Schools had partnered with a nonprofit organization to introduce a reading coach program, called Collaborative Coaching and Learning. This program, which was implemented district-wide in 2003, provides in-school, in-classroom support for teachers from coaches skilled in content areas, along with time for teachers to collaborate with one another and the coaches to analyze student data, observe model lessons, try out the model lessons, and reflect on their practices together. According to the Boston Plan for Excellence, it is difficult to analyze exactly how changed teacher practices results in increased student learning. A 2003 study concluded that 2 years after piloting the reading coach program, Boston teachers were reflecting more on their own and each other’s work. However, the study agreed that measuring the coaching program’s contribution to increased student learning is difficult. End of sidebar] DCPS intended to staff about 170 teacher coaching positions, however, as DCPS began the 2008-2009 school year, about 20 percent of the coaching positions remained open (19 reading coach vacancies and 16 math coach vacancies). As of late January 2009, there were 157 teacher coaches working on-site in the District's public schools, with 14 total vacancies. Each vacancy represents a school without the full support (either a reading coach or both a reading coach and a math coach) that DCPS wanted to provide. As a result, the ratio of teachers to coaches was higher than it would have been had the positions been filled. In addition, according to DCPS officials and Washington Teachers' Union officials we interviewed, teacher coaches were often unclear on their responsibilities and how to work with teachers, and received some conflicting guidance from principals. For example, these officials told us that some principals did not assign teacher coaches to their intended position. At the beginning of the school year, some principals assigned coaches to cover classes for absent teachers or to evaluate teachers--a practice not allowed under union rules--meaning the coaches were not able to work with teachers. DCPS is also seeking to improve the quality of principals through the Principals Academy developed for the 2008-2009 school year. Consistent with DCPS's belief that principals should be their schools' instructional leaders, the academy's goals include improving principals' leadership skills, helping them interpret student test scores, and providing advice on how to use this information to improve their schools. The Principals Academy convenes monthly and also includes differentiated professional development workshops based on principals' individual needs. The State Superintendent's Office and DCPS Have Developed and Begun Implementing Strategic Plans; However, DCPS Has Not Always Involved Relevant Stakeholders in Planning and Implementing Key Initiatives: The state superintendent plan is a "state-level" strategic plan that covers the District's public schools (and public charter schools). This plan and DCPS's strategic plan each contain elements GAO has identified as key to an effective plan,[Footnote 36] such as aligning short-term objectives to long-term goals in order to delineate how to attain those goals. While DCPS has recently increased efforts to involve stakeholders such as parents and the D.C. Council in key initiatives, past stakeholder involvement was inconsistent. DCPS has not yet developed a method for ensuring more consistent stakeholder involvement. Both the State-Level and DCPS Strategic Plan Were Developed with Stakeholder Involvement and Contain Many Elements of Effective Plans: The state superintendent's office and the State Board of Education collaboratively developed the District's state-level, 5-year strategic plan, and released it in October 2008. This state-level plan spans early childhood and kindergarten through grade 12 education (including public charter schools).[Footnote 37] The plan was developed with stakeholder involvement throughout the process. Officials from the state superintendent's office told us they involved District officials, and stakeholders representing early childhood education, business, and higher education communities, as well as other stakeholders while drafting the plan. In particular, they told us they involved DCPS and the D.C. Deputy Mayor of Education's Office in discussions of the plan. In addition, in September 2008, the state superintendent's office held one public forum to solicit stakeholder input on the draft of the document, and accepted comments on the draft on their Web site. The office released a revised version of the plan within a month of the public forum. Stakeholder involvement in formulating strategic plans allows relevant stakeholders to share their views and concerns. In addition, it affords stakeholders a way to understand the rationale for certain decisions. Ultimately, stakeholder involvement can result in increasing stakeholder support, or ownership, of the strategic plan. [Footnote 38] The state-level plan details the state-level strategy for improving education in the District and delineates accountability measures for DCPS and the public charter schools. In addition, the state-level plan states the mission, vision, and goals of the agency. It includes three broad, long-term goals: to have all children ready for school, all schools ready to prepare students for success, and all District residents ready to be successful in the 21st century economy. Overall, the plan includes many key elements of an effective strategic plan such as the inclusion of objectives that delineate how the state superintendent's office intends to attain each of its goals. The short- term objectives are supported by various strategies, objective measures, and performance targets. For example, one objective under the goal of having the District's schools ready to prepare students for success is to ensure that all students receive rigorous instruction. This objective is broken down into objective measures, such as the percentage of elementary students scoring proficient or above on the state test. Further, the plan specifies annual performance targets for this objective for the years 2008 to 2013. See table 5 for more details on the elements of the state-level strategic plan. DCPS released the draft of its 5-year strategic plan in late October 2008. In contrast to the state-level plan which includes the public charter schools, the DCPS plan is specific to prekindergarten through grade 12 education at its 128 schools. DCPS officials told us they based the draft on the Master Education Plan,[Footnote 39] which the prior DCPS administration developed with stakeholder involvement, and that they sought additional stakeholder input through a series of town hall meetings. After releasing the draft, DCPS held three public forums in the following 3 weeks where attendees provided DCPS officials with feedback on the draft strategic plan. In May 2009, DCPS released the revised draft, which incorporated stakeholder feedback. The DCPS 5-year strategic plan outlines the organization's vision and goals, and includes many elements of an effective strategic plan. For example the plan explains how DCPS's six broad goals are interrelated and how they support the vision. (Table 5 lists the six DCPS goals). Table 5: DCPS's Goals, as Outlined in Its Strategic Plan: 1. Compelling Schools: Create schools that provide a consistent foundation in academics, strong support for social and emotional needs, and a variety of challenging themes and programs. 2. Great People: Develop and retain the most highly effective educators in the country, and recognize and reward them. 3. Aligned Curriculum: Implement a rigorous, relevant, college preparatory curriculum that gives all students meaningful options for life. 4. Data Driven Decisions: Support decision making with accurate information about how students are performing and how the District as a whole is performing. 5. Effective Central Office: Provide schools with support they need to operate effectively. 6. Engaged Community: Partner with families and community members who demand better schools. Source: Making Student Achievement the Focus: A Five-year Action Plan for District of Columbia Public Schools, April 2009. [End of table] In addition, the DCPS plan describes the condition of DCPS prior to the reform effort, the progress made to date, and the steps needed to achieve the long-term goals. However, the DCPS plan does not systematically delineate measurable outcomes with clear time frames and does not always identify key external factors that could increase the risk that an initiative may fail. For example, several objectives are aimed at improving teacher quality; however, the plan lacks specific targets for measuring the expected magnitude of such an improvement. Without such targets, it will be difficult for the public to evaluate DCPS's progress toward improving its teacher workforce.[Footnote 40] In addition, while the strategic plan discusses increased performance- based pay for teachers, it does not specify the cost or explicitly mention the reliance on outside funding streams to achieve the increases.[Footnote 41] Yet, the reliance on outside funding for the initial 5 years is a risk that is not within DCPS's control. Table 6 contains some key elements of the state-level and DCPS's strategic plans. Table 6: Key Elements of Strategic Plans and Their Inclusion in the State Superintendent's Office and DCPS Plans: Element: Mission statement; State Superintendent's Office: Yes; DCPS: Yes. Element: Long-term goals and objectives; State Superintendent's Office: Yes; DCPS: Yes. Element: Approaches to achieve goals and objectives; State Superintendent's Office: Yes; DCPS: Yes. Element: Description of relationship between long-term goals and annual goals; State Superintendent's Office: Yes; DCPS: Partial[A]. Element: Identification of key external factors that could affect achievement of strategic goals; State Superintendent's Office: Yes[B]; DCPS: Partial[B]. Element: Description of how program evaluations were used or will be used to define or revisit strategic goals; State Superintendent's Office: Yes; DCPS: Yes. Element: Description of stakeholder involvement; State Superintendent's Office: Involved stakeholders throughout plan development; DCPS: Plan based on prior administration's plan and stakeholder input incorporated in revision. Source: GAO analysis of strategic plans. Note: See GAO/GGD-10.1.16. [A] DCPS's draft strategic plan delineates goals and explains, with varying degrees of specificity, how it will achieve these goals. These descriptions do not always include specific measures or specific actions. [B] While the state-level plan includes external factors that could affect its achievement of its strategic goals, GAO did not analyze whether the state-level plan exhaustively lists such factors. However, the DCPS's draft strategic plan discusses increasing teacher compensation and performance-based pay without elaborating on how such increases will be funded, or any conditions of funding. [End of table] Officials from the D.C. Deputy Mayor of Education's office told us that as part of their office's coordinating role, it ensured that DCPS and the state-level strategic plans were aligned. However, the office had no documentation showing its efforts to coordinate these plans, such as an alignment study. We found that the two plans were aligned in terms of long-term goals. For example, DCPS's goals could support the state- level goal of having all schools ready. However, we could not evaluate whether more detailed, objective measures and performance targets were aligned because the DCPS strategic plan did not always include specific objective measures and performance targets. DCPS Has Recently Increased Its Efforts to Involve Stakeholders in Various Initiatives, However It Has Not Systematically Included Stakeholders: DCPS officials have several planned and ongoing efforts to involve stakeholders in planning, implementing, and evaluating various initiatives. Stakeholder involvement can be instrumental in these areas because stakeholders can bring different knowledge, points of view, and experiences to planning and implementing reform efforts.[Footnote 42] DCPS officials told us they have a variety of approaches to involve stakeholders, including parents, students, and community groups, as well as institutional stakeholders such as the D.C. Council. For example, DCPS officials told us they reach out to parents, students, and the public by holding monthly community forums, meeting with a group of high school student leaders and a parent advisory group, responding to e-mail, and conducting annual parent and student surveys to gauge the school system's performance. DCPS introduced monthly community forums in July 2008. These forums were generally informational sessions on topics chosen by DCPS officials, and were followed by questions from the audience. In some cases, such as the three forums focused on the strategic plan, DCPS officials facilitated discussions to elicit feedback. DCPS officials told us their efforts to involve students in reform efforts included a student leadership group that met regarding student concerns, and which was credited by DCPS officials for changes in the school lunch program as well as substantial changes to the discipline policy. DCPS also involved other stakeholders, such as parent organizations and the Washington Teachers' Union in its process of changing the discipline policy. In addition, DCPS officials cited the Chancellor's response to e-mail communications as a form of stakeholder involvement. While such communications may have provided stakeholders with a means of connecting to the Chancellor, e-mail communications are generally not public and do not lead to public debate or discourse.[Footnote 43] In spring 2008, DCPS also conducted parent and student surveys to assess stakeholder satisfaction with DCPS schools.[Footnote 44] While DCPS officials told us they have completed the analysis of the parent survey, they have not yet released the results. Further, DCPS did not receive the student survey data until February 2009 due to complications with a vendor who was paid to collect these data. [Footnote 45] As a result of the delays, DCPS officials told us they have been unable to use student survey responses to inform decisions relevant to the 2008-2009 school year. However, officials said they will be able to use the information as a baseline for future surveys. However, such activities do not ensure systematic stakeholder input in planning, implementing, and monitoring key initiatives. During our review, DCPS officials told us that stakeholder involvement was important to their reform efforts and that DCPS was taking steps to increase stakeholder involvement. However in some cases, according to two DCPS officials, DCPS did not have a planning process in place to ensure systematic stakeholder involvement, and we found that DCPS implemented some key initiatives with limited stakeholder involvement.[Footnote 46] For example, key stakeholders, including D.C. Council members and parent groups, told us they were not given the opportunity to provide input to inform DCPS's initial proposals regarding school closures and consolidations, although DCPS did hold numerous meetings after the initial proposal, before finalizing decisions. Similarly, stakeholders told us DCPS did not include them in deliberations and decisions about the establishment of prekindergarten to grade 8 models at some schools. Representatives from one community organization told us that some parents had concerns about the structure and academic setting at the prekindergarten to grade 8 schools, but did not have a venue to express those concerns before decisions about grade configurations were made. In addition, DCPS did not seek input from key stakeholders during the planning and early implementation of the new staffing model that placed art, music, and physical education teachers at schools and which fundamentally changed the way funding is allocated throughout DCPS. DCPS officials told us that they had not planned for the number of changes that were requested by principals. In particular, they told us that the vast majority of school principals requested changes to their initial staffing allocations. Stakeholders did not have a timely opportunity to raise concerns on the potential risks in implementing the staffing model, such as the uneven distribution of resources across schools and overspending at some schools. Stakeholders also said they were not given sufficient time to review the budget for the 2008-2009 school year or to understand the changes in the budget made after the school year began. DCPS officials told us the budget planning process for the 2010-2011 school year involved stakeholders extensively. In particular, DCPS invited the public to a preliminary budget meeting and also provided training on the budget process to some key stakeholders, such as school principals and community members. Lack of stakeholder involvement in such key decisions led stakeholders, including the D.C. Council and parents groups, to voice concerns that DCPS was not operating in a transparent manner or obtaining input from stakeholders with experience relevant to the District's education system. Further, these stakeholders have questioned whether the impact of reform efforts will be compromised because of restricted stakeholder involvement. Stakeholders from other urban school districts we visited told us a lack of stakeholder involvement leads to less transparency as key decisions are made without public knowledge or discourse. In addition, the lack of stakeholder involvement can result in an erosion of support for ongoing reform efforts and poor decisions. For example, officials in Chicago and Boston said public stakeholder involvement was critical to community support for various initiatives, such as decisions on which schools to close. Officials and stakeholders in New York cited lack of stakeholder involvement in decisions that were eventually reversed or revised. For example, changes made to school bus routes without consulting parents meant several route changes were later reversed because they proved to be unworkable. DCPS and the State Superintendent's Office Have Taken Steps to Improve Accountability and Performance, and DCPS Has Yet to Align Key Aspects of Its Performance Management System to Organizational Goals: DCPS and the state superintendent's office have taken steps to improve accountability and performance of their offices. For example, both offices have started implementation of new individual employee performance management systems. While DCPS has taken steps to improve accountability and link its individual performance management system to organizational goals, it has not completed this process or used the results of surveys to improve central office operations. Both DCPS and the State Superintendent's Office Developed and Implemented a Performance Management System, and DCPS Has Not Yet Aligned Key Aspects of the System to Organizational Goals: To increase accountability of its central office, DCPS developed an accountability system and an individual performance management system for central office departments and employees. The central office, which is responsible for providing academic and nonacademic supports[Footnote 47] to DCPS, had operated without such accountability systems prior to the recent reform efforts. For example, previously, performance evaluations were not conducted for most DCPS staff. As a result, central office employees were not held accountable for the quality of services they provided to support schools. To improve accountability for central office departments, DCPS developed departmental scorecards, as a part of its performance management system, to identify and assess performance expectations for each department.[Footnote 48] For example, the scorecard for the Office of Data and Accountability includes measures such as the number of users of the primary student data system. According to a DCPS official, these scorecards are discussed at weekly accountability meetings[Footnote 49] with the Chancellor to hold senior-level managers accountable for meeting performance expectations. For example, at the accountability meeting we attended, DCPS officials from the Office of Data and Accountability used scorecards to discuss their progress with collecting attendance data and setting up processes to strengthen the collection of these data. According to DCPS officials, some departmental leaders have established similar accountability meetings with their staff, although these are not required. In January 2008, DCPS implemented a new performance management system for employees. Performance management systems for employees are generally used to set individual expectations, rate and reward individual performance, and plan work.[Footnote 50] DCPS developed its new performance management system in an effort to improve support services to the schools by improving the accountability and performance of central office employees. In particular, in past school years, teachers complained about not getting paid on time and beginning the school year with inadequate supplies. DCPS's performance management system was put in place, in part, to improve these functions in the central office. While DCPS developed and instituted a new performance management system, it did not fully align individual performance expectations and evaluations to organizational goals, which GAO has identified as a key practice of effective individual performance management systems. [Footnote 51] For example, while DCPS took important steps in developing and implementing its system, such as training department managers to set expectations and give feedback to employees, DCPS has not yet established a uniform policy for setting expectations. Further, DCPS has not yet instituted a system to track how and when such expectations are set. Instead, individual managers established processes specific to their office or department and, as a result, DCPS could not ensure that individual performance expectations were aligned to organizational goals as outlined in the DCPS 5-year strategic plan or in its annual performance plans.[Footnote 52] Without such alignment, employees may not be familiar with the overall organizational goals and their daily activities may not reflect these goals. An explicit alignment of daily activities with broader desired results helps individuals connect their daily activities and organizational goals and encourages individuals to focus on their roles and responsibilities to help achieve the broader goals.[Footnote 53] In addition, as we previously reported,[Footnote 54] DCPS developed individual performance evaluations in December 2007 as a part of its performance management system in order to assess central office employees' performance. Such individual performance evaluations are used to rate central office employees on several core competencies twice a year. For example, employees are rated on how well they demonstrate a commitment to providing high-quality and timely customer service to both external and internal customers of District schools. Prior to our March 2008 testimony, DCPS officials told us that they intended to align the performance management system with organizational goals by January 2009, and DCPS has taken some steps to improve alignment. For example, DCPS officials told us they had better aligned their departmental scorecards to their 2009 annual performance plan. However, DCPS has not yet explicitly linked employee performance evaluations to the agency's overall goals. DCPS officials told us they plan to link the individual performance evaluations with organizational goals in the summer of 2009 to ensure greater accountability in supporting schools. The state superintendent's office also implemented a new performance management system, effective October 2008, to hold its employees accountable and improve the office's performance. The office is converting to a single electronic management system to track and evaluate employee performance. This new system, scheduled to be fully operational by December 2009, will replace the two separate systems that had operated on different cycles. According to an official from this office, the new system is uniform, user friendly, and allows for an easier transfer of performance information from manager to employee. In addition, this system links individual employee evaluations to overall performance goals and the office's strategic plan. Under this new evaluation system, each employee is given a position description, which includes responsibilities and duties linked to the overall goals, mission, and vision of the state superintendent's office. Individual and agency expectations are defined in an annual performance meeting with the employee. The office is currently training supervisory employees on how to use the system before its full implementation in December 2009. DCPS Surveyed Staff at Schools to Measure Satisfaction with Central Office Services; However, DCPS Has Not Yet Used Survey Results to Improve Operations: In November 2007, DCPS conducted a survey of employees within District schools, including teachers and principals, to gauge satisfaction with District services, including central office services during the 2007- 2008 school year. Personnel at the schools are key stakeholders in improving central office functions, and their feedback is important to help DCPS ensure resources are targeted to the highest priorities. [Footnote 55] The American Institutes for Research partnered with DCPS to administer the online survey of teachers, principals, aides, clerks, counselors, project directors/coordinators, related service providers, and other staff. They were asked to provide feedback on numerous topics, including the work environment, facilities and maintenance, professional development, and leadership, as well as central office services. With regard to central office services, the survey's questions were focused on personnel services, budget and procurement services, district departments and support services, food and nutrition services, and technology and data. Of those staff that completed the survey,[Footnote 56] more were satisfied with their schools, such as their work environment and fellow staff members, than with the support system provided by the central office. For example, they were least satisfied with the central office's ability to provide goods and services in a timely manner, compute paychecks accurately, and allot budgeted funds when needed. In addition, staff who completed the survey were least satisfied with facilities office's responsiveness to requests for school repairs, saying they were not completed in a timely manner. DCPS officials told us the results of the survey were shared internally with different central office departments in 2008, and focus groups were formed within a month of the release of the survey results to develop specific action plans to address identified issues. However, DCPS officials were unable to provide us with specific examples of improvements made in central office operations as a result of the survey. Three of the eight principals we met with regarding the school consolidation process stated that they could not always access budgeted funds when needed. In addition, four of the eight principals noted that school repairs were not made in a timely manner.[Footnote 57] One principal told us his payroll was often inaccurate, and some teachers were not always paid on time. DCPS officials told us another staff survey will be administered in spring 2009. Conclusions: The challenge of reforming DCPS is daunting. NCLBA requires 100 percent proficiency by 2014 and the District's students scored significantly lower than the District's own proficiency targets for 2008 and below students in most other urban districts. In the past, support for reform efforts has waned as student achievement did not improve, as buildings deteriorated, and as new superintendents were ushered in every few years to address these problems. The need for rapid reform and results is acute and the District's Mayor and his education team have taken bold steps--such as implementing various classroom-based initiatives, reorganizing schools, and replacing teachers and principals--to improve the learning environment of the District's students and ultimately increase student achievement. However, DCPS lacks certain planning processes, such as communicating information to stakeholders in a timely manner and incorporating stakeholder feedback at key junctures, which would allow for a more transparent process. In addition, DCPS did not gauge its internal capacity prior to implementing certain key initiatives, which, if addressed in the future, could help ensure the sustainability of initiatives. Without these planning processes, an organization risks having to revamp initiatives, leading to delays and compromising the implementation of timely, critical work. While having these planning processes in place will not eliminate all implementation issues, it will help to identify and mitigate risks associated with implementing bold initiatives and identify needed changes in the early stages of the initiative. Furthermore, a lack of these planning processes can result in decisions that are made on an ad hoc basis with resources unevenly distributed as was the case with the District's new staffing model. Ultimately, the lack of such processes while planning and implementing initiatives has impeded the success of some of DCPS's initiatives and could impede the District's continued success and progress in reforming its school system. Stakeholder consultation in planning and implementation efforts can help create a basic understanding of the competing demands that confront most agencies and the limited resources available to them. Stakeholders can then share their expertise and experience, and views on how these demands and resources can be balanced. Continuing to operate without a more formal mechanism--other than community forums or e-mails--for stakeholder involvement could diminish support for the reform efforts, undermine their sustainability, and ultimately compromise the potential gains in student achievement. As more initiatives are developed, the need to balance the expediency of the reform efforts with measures to increase sustainability, such as stakeholder involvement, is critical. In addition, since the Reform Act, the District has taken several steps to improve central office operations, such as providing more accountability at the departmental level and implementing a new individual performance management system. However, DCPS has not taken steps to align its performance management system, including its individual performance evaluations, to its organizational goals, which could result in a disparity between employees' daily activities and services needed to support schools. By ensuring that employees are familiar with the organizational goals and that their daily activities reflect these goals, DCPS could improve central office accountability and support to schools. Recommendations to the Mayor of the District of Columbia: To help ensure the transparency, success, and sustainability of the District's transformation of its public school system, we recommend that the Mayor direct DCPS to establish planning processes that include mechanisms to evaluate its internal capacity and communicate information to stakeholders and, when appropriate, incorporate their views. To strengthen the new individual performance management system and ensure greater accountability of central office employees in their role supporting schools, we recommend that the Mayor direct DCPS to link individual performance evaluations to the agency's overall goals. Agency Comments and Our Evaluation: We provided a draft of this report to DCPS, the Deputy Mayor of Education, and to the Office of the State Superintendent of Education for review and comment. These offices provided written comments on a draft of this report, which are reproduced in appendix I. They also provided technical comments, which we incorporated when appropriate. All three entities concurred with our recommendations. However, they expressed concern with the way in which we evaluated their reform efforts and the overall tone of the draft report. Specifically, District officials stated that we did not measure DCPS's progress in terms of the condition of the school system prior to the reform efforts, but instead measured progress in terms of whether the ultimate goals of the reform efforts had been met. We disagree. We did not measure DCPS's progress against "ultimate goals." As is now reflected in the paragraph describing our approach to this study, we measured the progress of ongoing reform efforts by comparing DCPS's progress to its own time frames for implementing various initiatives. In conducting our review, we spoke with numerous DCPS officials and repeatedly asked for documents and time frames in order to objectively gauge the District's progress. In some cases, DCPS officials did not provide us with such documentation; however, we made a concerted effort to accurately identify current initiatives and related time frames. In addition, we measured completed initiatives against recognized standards. For example, we determined whether or not the DCPS and the state-level strategic plans contained elements that GAO has identified as key to an effective plan. In addition, we described the conditions that existed prior to the reform efforts in order to provide context to the steps DCPS has taken. For example, we noted that prior to the reform efforts, DCPS's teacher training was not systematic or aligned with the school district's goals and that DCPS is now offering on-site professional development to improve teacher skills. We also cited the lack of individual performance evaluations for central office employees prior to the reform efforts that DCPS has made to improve in this area. Furthermore, we made every effort to provide balance and objectivity in our findings. For example, some stakeholders, such as parents groups, union representatives, and the D.C. Council, told us that DCPS made key decisions without their involvement. We revisited this issue with DCPS officials and described several of their efforts to improve stakeholder involvement in the initial draft of our report. We visited four urban school districts with mayoral governance and conducted in-depth interviews to help us better understand the magnitude of the challenges that officials encountered while trying to reform their school systems. We also spoke with superintendents and officials from mayors' offices in these districts about the key lessons they learned as they reformed their school systems, including the risks associated with not having systematic stakeholder involvement. Finally, the District's education offices stated in their response that we characterized the state superintendent's efforts as positive and those of DCPS more negatively. While drafting this report, we intentionally avoided any comparison between DCPS and the state superintendent's office, as their tasks and challenges are dissimilar. After reviewing our draft, DCPS provided us with more information and documentation regarding efforts to involve stakeholders in the development of the October 2008 draft of the DCPS strategic plan and steps taken to introduce alignment of accountability measures to organizational goals. We made changes to our report to reflect the updated information. As agreed with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after its issue date. At that time, we will send copies of this report to the D.C. Mayor's Office, relevant congressional committees, and other interested parties. Copies will also be made available upon request. In addition, the report will be available at no charge on GAO's Web site [hyperlink, http://www.gao.gov]. If you or your staff members have any questions about this report, please contact me at (202) 512-7215 or ashbyc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix II. Signed by: Cornelia M. Ashby, Director: Education, Workforce and Income Security Issues: [End of section] Appendix I: Comments from the D.C. Mayor's Office and District Education Offices: Government Of The District Of Columbia: June 11, 2009: Cornelia M. Ashby: Director, Education, Workforce, and Income Security Issues: United States Government Accountability Office: 441 G Street, N.W., room 5940: Washington. DC 20548 Subject: "District of Columbia Public Schools" June 2009 Report to the Subcommittee on Oversight of Government Management, the Federal Workforce, and the District of Columbia, Committee on Homeland Security and Governmental Affairs, U.S. Senate. Dear Ms. Ashby, We write in response to the draft June 2009 GAO report on the progress of the District of Columbia Public Schools reform efforts. Thank you for the opportunity to review the draft report and provide feedback to you and your team. We believe we have made substantial progress with our reform efforts, but agree that there is still much work to be done. We have significant concerns. however, with the tone of the draft report and the approach used to reach the conclusions presented. and hope that these are addressed through a revised final report. The report represents the conclusion of nearly two years of ongoing evaluation and analysis by the GAO of District education reform efforts since the District established a new school governance model that placed control of the school system under the authority of the Mayor. Overall, we agree with the substance of the two major recommendations from the report: 1) that DCPS establish planning processes that include evaluating internal capacity and incorporating stakeholders at key junctures and 2) that DCPS link individual performance evaluations to the agency's goals to strengthen accountability for DCPS' central office. As we have shared with the GAO team, the Mayor has made constant evaluation, stakeholder input, and alignment of evaluations with objectives central components of ensuring accountability under the governance structure. We disagree strongly, however. with the overall tone of the draft report and the conclusions derived within specific sections. The report falls short of objectively conveying the context for the DCPS initiatives underway and of adequately capturing all of the progress that has been made to date. The findings presented attempt to measure DCPS against a fully implemented reform standard, even though the reform has been underway for only two years. In order to "evaluate the District's reform efforts". as the report describes GAO's charge from Congress, we believe it is more appropriate and accurate to measure DCPS in terms of what has been accomplished since the passage of the Public Education Reform Act, rather than measuring the work in terns of whether all of the ultimate goals have been fully accomplished to date. The draft report does not adequately present the whole picture, and throughout the draft report, the section headings appear to he designed to include a negative finding as a requirement, even when the text describes areas of significant overall achievement. Moreover, in areas where both DCPS and the Office of the State Superintendent arc similarly situated, the draft report describes the progress made at OSSE in more balanced way by focusing on improvements planned in the future. rather than gaps in initial efforts. At DCPS, the discussion centers instead on what DCPS has not yet done. In finalizing the June 2009 report, we ask that the GAO attempt to address the issues outlined above in order to present a clearer, more balanced assessment of where we are in our reforms. We appreciate the opportunity to provide this feedback, and have provided your staff with a list of technical corrections and improvements. We strongly believe in evaluation and assessment, and we want to be held accountable to the public for our work. We hope that the GAO report can he used as a tool not only for Congress. but the public at large, to measure our success. Sincerely, Signed by: Victor Reinoso: Deputy Mayor for Education: Signed by: Michelle Rhee: Chancellor, DCPS: Signed by: Kerri Briggs: State Superintendent: [End of section] Appendix II: GAO Contact and Staff Acknowledgments: GAO Contact: Cornelia Ashby at (202) 512-7215 or ashbyc@gao.gov: Acknowledgments: In addition to the contact named above, Elizabeth Morrison, Assistant Director; Nagla'a El-Hodiri, Analyst-in-Charge; Sheranda Campbell; Jeff Miller; and Vernette Shaw made significant contributions to this report in all aspects of the work. Susan Aschoff, Mark Bird, Timothy Case, Bryon Gordon, Jeffrey Heit, Janice Latimer, Jean McSween, Sandy Silzer, and Sarah Veale provided analytical assistance. Doreen Feldman and Sheila McCoy provided legal support and Lise Levie and Kimberly Siegal verified our findings. [End of section] Related GAO Products: District of Columbia Public Schools: While Early Reform Efforts Tackle Critical Management Issues, a District-wide Strategic Education Plan Would Help Guide Long-Term Efforts. [hyperlink, http://www.gao.gov/products/GAO-08-549T]. Washington, D.C.: March 14, 2008. District of Columbia Opportunity Scholarship Program: Additional Policies and Procedures Would Improve Internal Controls and Program Operations. [hyperlink, http://www.gao.gov/products/GAO-08-9]. Washington, D.C.: November 1, 2007. No Child Left Behind Act: Education Should Clarify Guidance and Address Potential Compliance Issues for Schools in Corrective Action and Restructuring Status. [hyperlink, http://www.gao.gov/products/GAO-07-1035]. Washington, D.C.: September 5, 2007: Charter Schools: Oversight Practices in the District of Columbia. [hyperlink, http://www.gao.gov/products/GAO-05-490]. Washington, D.C.: May 19, 2005. Results-Oriented Cultures: Implementing Steps to Assist Mergers and Organizational Transformation. [hyperlink, http://www.gao.gov/products/GAO-03-669]. Washington, D.C.: July 2, 2003. Results-Oriented Cultures: Creating a Clear Linkage between Individual Performance and Organizational Success. [hyperlink, http://www.gao.gov/products/GAO-03-488]. Washington, D.C.: March 14, 2003. Agencies' Strategic Plans Under GPRA: Key Questions to Facilitate Congressional Review (Version 1). [hyperlink, http://www.gao.gov/products/GAO/GGD-10.1.16]. Washington, D.C.: May 1997. Executive Guide: Effectively Implementing the Government Performance and Results Act. [hyperlink, http://www.gao.gov/products/GAO/GGD-96-118]. Washington, D.C.: June 1996. [End of section] Footnotes: [1] In this report, when we refer to D.C. public schools, we are not including the D.C. public charter schools. Charter schools are public schools that are exempt from certain regulations in exchange for increased accountability for improving student achievement. This report does not include a discussion of charter schools, which are governed in the District by the Public Charter School Board. [2] U.S. Census Bureau data show that the District's school-age population (ages 5-19) declined by about 5 percent from 2000 to 2007. [3] The D.C. Council approved the Reform Act on April 19, 2007. District of Columbia Laws, Act 17-38 (2007). The bill was then passed by Congress and signed into law by President Bush on June 1, 2007. Pub. L. No. 110-33. The Reform Act pertains primarily to the D.C. public school district, but also contains legislation relevant to the District's charter schools. [4] GAO, District of Columbia Public Schools: While Early Reform Efforts Tackle Critical Management Issues, a District-wide Strategic Education Plan Would Help Guide Long-Term Efforts, [hyperlink, http://www.gao.gov/products/GAO-08-549T] (Washington, D.C.: Mar. 14, 2008). [5] The District's education offices include the District of Columbia Public Schools, the Office of the State Superintendent of Education, and the District of Columbia's Department of Education. [6] There were 26 newly consolidated schools. We interviewed principals from the 8 that had received the greatest influx of students from the recently closed schools. [7] DC VOICE is a community organization whose mission is to hold both the public schools and the community accountable for providing high- quality teaching and learning for all. The Council of the Great City Schools is an organization that represents 67 of the largest urban school systems in the United States, including DCPS. Parents United for the D.C. Public Schools is a parent advocacy organization. [8] P.L. 111-5. [9] As of March 2009, the District had 59 public charter schools with enrollment for the 2008-2009 school year around 26,000. [10] The D.C. Committee on Public Education, Our Children, Our Future: Revitalizing The District of Columbia Public Schools (Washington, D.C., June 1989). The D.C. Committee on Public Education was formed in 1988 to develop a long-range plan to improve the quality of education provided in the District. The Committee was composed of 64 individuals from the business and professional community, churches, universities, parents, and education experts who worked closely with the Superintendent, Mayor, Board of Education, and D.C. Council in developing the plan. [11] Council of the Great City Schools, Restoring Excellence to the District of Columbia Public Schools (Washington, D.C., January 2004). [12] The Parthenon Group, Fact-Base for DCPS Reform (Boston, London, San Francisco, December 2006). [13] The District of Columbia has 60 school districts. DCPS is the District's largest school district. The 59 other school districts are public charter schools, and often consist of just 1 school (some charters have multiple campuses). [14] In the spring, DCPS students in grades 3 through 8 and grade 10 take the annual end-of-year state test, known as the District of Columbia Comprehensive Assessment System. In addition, DCPS administers the benchmark test, referred to as the D.C. Benchmark Assessment System, district-wide in grades 3 through 10. This test provides DCPS with information on how students are progressing in reading and math in preparation for the end-of-year state test. The District of Columbia Comprehensive Assessment System was first administered in 2006, prior to the Reform Act. [15] For a discussion of ways in which states can exercise flexibility, see Department of Education Fact Sheet: New No Child Left Behind Flexibility: Highly Qualified Teachers, March 2004. [16] For example, LeapFrog and Read 180 are both reading instruction programs and targeted interventions used to improve basic reading skills for struggling students in the elementary and secondary grade levels. [17] The 2009 state-wide test was administered to students in April 2009 and the test results are expected to be available mid-summer 2009. [18] After reviewing our draft, DCPS provided us with the results of the vendor's analysis, but we did not independently evaluate the methodology or the results. [19] In addition to a standard funding amount, students with certain characteristics are funded at greater levels to account for the increased cost of educating them. For example, schools with students who are English language learners and students with disabilities are allocated additional funds. [20] This issue may have been more acute prior to closing 23 schools due to declining enrollment. [21] This core can be supplemented with other positions based on enrollment or to comply with laws and regulations applicable to certain student populations, such as students with disabilities and English language learners. [22] Under the new staffing model, a school may choose to trade a position offered by the new staffing model for another position based on its needs. For example, a school may employ an art teacher funded by a private entity. A principal at such a school could then trade the art position assigned by the staffing model for a regular classroom teacher. [23] Mary Levy, An Analysis of DCPS General Education Resources in Local School Budgets for FY 2009 (Washington Lawyers Committee for Civil Rights and Urban Affairs, December 2008). DCPS officials told us that they conducted their own analysis in an effort to minimize such differences in the future. GAO did not conduct an independent analysis of the per pupil allocations across schools. [24] In these cases, one principal cited the lack of certification for middle school grades and training on the student data system as problematic for his/her teachers. The other cited a lack of "exemplary" applicants for math and special education. Neither of these two principals reported having vacancies or poor performing teachers. [25] DCPS reported that in the 2007-2008 school year there were 14,257 students attending schools in restructuring planning status. [26] DCPS has a total of 128 schools, but only 118 are required to meet federal accountability standards because these schools do not have students in grades tested under NCLBA, do not have enough students, or have a transient student population. [27] DCPS could not terminate these teachers due to contractual agreements with the teachers' union. Removal would have required a formal process including teacher evaluation and additional assistance for underperformers. See discussion on the teacher evaluation process later in the report. [28] The other half of the 160 teachers resigned from the school system or retired. [29] DCPS's primary student data system is called STARS (Student Tracking and Reporting System). It performs such functions as creating student report cards and tracking student attendance. [30] SLED will serve as a unified repository of school system data needed to improve management, reporting, instruction, trend analysis, and program evaluation for the District. SLED is intended to enable the sharing of critical information spanning a student's lifelong public education experience in the District from early childhood to college and other postsecondary education. SLED is funded in part by the U.S. Department of Education's Statewide Longitudinal Data System Grants Program. Education's Institute of Education Sciences provides monitoring and technical assistance for the project. [31] Goe, Laura. The Link Between Teacher Quality and Student Outcomes: A Research Synthesis, National Comprehensive Center for Teacher Quality, 2007. Despite research consensus that teacher quality impacts student achievement, there is not a universal definition of what teacher quality is. [32] DCPS had about 4,200 teachers for the 2007-2008 school year and about 3,900 teachers for the 2008-2009 school year. [33] DCPS did not need to hire the same number of teachers as the number who left the school system after the 2007-2008 school year because 23 schools closed and district-wide enrollment had again declined by the beginning of the 2008-2009 school year. [34] Founded in 1990, Teach for America is a national program that recruits top college graduates and professionals of all academic majors and career interests to commit to teaching for 2 years in urban and rural public schools. Established in 2001, the D.C. Teaching Fellows recruits a range of successful professionals from noneducation fields to teach in DCPS. [35] Qualifications to become a teacher coach include having a valid teaching license, at least 3 years successful teaching experience (5 years preferred), and being deemed highly qualified under NCLBA. In addition, teacher coaches are expected to have successful experience in providing staff development and exemplary presentation, communication, and organizational skills. [36] GAO, Agencies' Strategic Plans Under GPRA: Key Questions to Facilitate Congressional Review (Version 1), [hyperlink, http://www.gao.gov/products/GAO-GGD-10.1.16] (Washington, D.C.: May 1997). [37] The state superintendent's office serves as a state education agency for DCPS and 59 public charter schools, as of March 2009. [38] GAO, 21st Century Challenges: Transforming Government to Meet Current and Emerging Challenges, [hyperlink, http://www.gao.gov/products/GAO-05-830T] (Washington, D.C.: July 13, 2005). [39] The Master Education Plan dated February 2006 was developed and released by the Superintendent of D.C. schools and the D.C. Board of Education. According to the plan, there was a high degree of stakeholder involvement in developing the 122-page plan, including five community forums, three forums sponsored by the Washington Teachers' Union, and over 15,000 parents participating through phone surveys. [40] The DCPS strategic plan refers the reader to its annual performance plan to see certain performance targets. For example, the fiscal year 2009 performance plan includes projections for student achievement metrics--such as percentage of students who are proficient in math and reading for 2009, 2010, and 2011. Neither the strategic plan nor the annual performance plan has objective measures or performance targets for increasing teacher quality. [41] The Chancellor has said there is $200 million in private funding to pay for substantial increases in teacher salaries and professional development. [42] [hyperlink, http://www.gao.gov/products/GAO-05-830T]. [43] In addition, DCPS officials told us they plan to establish the opportunity for a small group of parents to meet with DCPS officials, including the Chancellor, on an ongoing basis. [44] DCPS conducted a telephone survey and polled 500 parents to assess their satisfaction with their school and the school district as a whole. Parents were asked for feedback on such issues as school safety, quality of instruction, communication, and the level of parental engagement in the decision-making process. The student survey was a voluntary, written survey to assess student views about school safety, services, leadership (principals, teachers, and staff), and instructional practices. [45] DCPS officials told us they experienced problems with the vendor, such as missed deadlines and incomplete and incorrect data. They also told us several times during the course of our work that they were expecting the data; however, each time the vendor failed to deliver. [46] After reading the draft of our report, DCPS officials identified four steps they said DCPS takes to involve stakeholders in key decisions. We requested documentation showing that these steps had been in place during the 2008-2009 school year; however, DCPS did not provide such documentation. [47] Some central office employees provide academic services, such as planning and monitoring academic initiatives, while others work on nonacademic functions, such as purchasing school supplies and managing teacher payrolls. [48] Central office departments include the following departments or offices: Chief Academic Officer, Family and Community Engagement, Office of the Chief Financial Officer, Office of Data and Accountability, Office of Human Capital, Office of Special Education, Operations, and Transformation Management Office. [49] The Chancellor holds weekly School Stat meetings to hold managers accountable for their offices' performance. School Stat is one of DCPS's accountability programs to track key initiatives and develop strategies to promote continuous improvement. [50] Effective performance management systems can be used strategically to drive internal change, achieve desired results, and provide continuity during transitions. GAO, Results-Oriented Cultures: Creating a Clear Linkage between Individual Performance and Organizational Success, [hyperlink, http://www.gao.gov/products/GAO-03-488] (Washington, D.C: Mar. 14, 2003). [51] [hyperlink, http://www.gao.gov/products/GAO-03-488]. GAO identified other key practices for effective individual performance management systems. However, we focused on the two practices that link employee performance to the broader organizational goals. [52] The Mayor's office requires education offices to develop and follow annual performance plans as another component of the accountability process. These performance plans include broad objectives, such as ensuring that schools provide a consistent foundation in academics, retaining the most highly effective and compensated educators, and partnering with families and the community. [53] [hyperlink, http://www.gao.gov/products/GAO-03-488]. [54] In our March 2008 testimony, we reported that DCPS officials told us that employee evaluations do not yet link to their offices' performance goals because they had limited time to implement the new performance system. However, officials stated that they planned to develop the linkages over the next year. GAO-08-549T. [55] GAO, Executive Guide: Effectively Implementing the Government Performance and Results Act, [hyperlink, http://www.gao.gov/products/GAO/GGD-96-118] (Washington, D.C.: June 1996). [56] A total of 3,285 staff completed the survey, yielding an overall response rate of 55 percent. [57] We met with eight principals during our review that received students from closing schools to discuss DCPS's consolidation process. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: