Erin Roth: More accurate DC School Report Card starts with ditching reliance on proficiency rates
In recent weeks a unanimous recommendation by the District’s State Board of Education (SBOE) and new research from the Massachusetts Institute of Technology have reinforced the Office of the DC Auditor’s finding a year ago that school proficiency rates (in math and English language arts) used to rank individual public schools are biased and misleading. The open question — coming out of a rare moment of consensus in the complicated world of school ratings — is this: What will District of Columbia policymakers do with the common but increasingly discredited proficiency rankings?
There are alternatives to be considered that would enable us to judge schools for their accomplishments. But first, let’s take a closer look at the three pieces of evidence that show bias in DC’s current method for ranking our schools
In January, a team of MIT researchers — including one Nobel Prize winner — released new research showing that commonly used school ratings that seek to identify school quality are biased and unfairly penalize schools serving higher percentages of historically underserved groups of students. The study uses data from New York City and Denver to show that many schools’ higher ratings are simply due to serving students with higher test scores, reflecting selection bias rather than school quality. They then offer a model to determine school quality using student growth — adjusted for student background and preparation level — that is less impacted by this type of bias.
On Jan. 19, the SBOE voted to discontinue the District’s School Transparency and Reporting (STAR) Framework summative rating system of schools for the same reason — bias in the ratings disproportionately borne by certain schools. The board recommended that the District’s Office of the State Superintendent for Education (OSSE) replace the summative school ratings with a dashboard-style school report card that would include a variety of metrics, encompassing school climate, student growth, facilities, staff diversity, retention and more.
These outcomes bolster a conclusion in the education data report the Office of the DC Auditor (ODCA) released in March 2021 that the proficiency rates driving our school rating system are biased and misleading. Together, this body of evidence, all pointing in the same direction, provides solid footing for DC to move away from the current rating system driven by proficiency rates.
The education data audit found that students with higher prior achievement are more likely to be grouped in some schools and students with lower prior achievement in others, and that these patterns continue across school transitions — for example, from fifth to sixth grade. This means that proficiency rates are telling families more about entering student test scores than how much students learn each year in any given school. The misleading nature of proficiency-driven school rating systems means that families relying on ratings to inform school choice are likely to incorrectly identify which schools are actually contributing the most to student achievement.
To be clear, research shows that the bias and inaccuracy identified in ratings means that schools serving more Black and brown students, students in poverty, students receiving special education services, and students who are English learners are inaccurately and disproportionately assigned lower ratings. It is then no surprise that research shows individual family decision-making based on biased information has contributed to school segregation, a long-standing issue in the District.
Parents aren’t the only actors who may be making decisions based on erroneous conclusions about proficiency. ODCA found that important trends in student achievement and growth can be lost or misinterpreted when education leaders rely on proficiency. In the report, we recommended moving away from proficiency and instead using student-level data longitudinally. By doing so, OSSE — as the District’s state education agency — could measure student growth and produce accurate and actionable results.
Determining how to move beyond proficiency-driven rating systems does involve more work by OSSE and the SBOE. However, new models can build on the information from the ODCA report that found student test score growth is not related to measures of neighborhood socioeconomic status. A new school rating system can elevate better measures of student growth and use more inclusive metrics of school climate and culture. Instituting such a system would respond to community feedback and reduce the obvious bias in current school quality ratings.
Taken together, the SBOE action, MIT research and ODCA data audit demonstrate that District schools, students, teachers and families are not served by and should not be forced to rely on counterproductive school ratings driven by proficiency rates. OSSE has an opportunity to develop a more equitable and accurate system informed by the evidence.
Erin Roth is director of education research at the Office of the DC Auditor.
About commentaries
The DC Line welcomes commentaries representing various viewpoints on local issues of concern, but the opinions expressed do not represent those of The DC Line. Submissions of up to 850 words may be sent to editor Chris Kain at chriskain@thedcline.org.
So I guess journalistic integrity is not a requirement for commentaries here. This author referred to her own reports in the third person as if someone else wrote them and she is just referencing them.