Study: DC Public School’s Teacher Evaluation System Continues to Improve Teacher Workforce

In 2009, the District of Columbia Public Schools (DCPS) began implementation of IMPACT, an innovative and controversial teacher-evaluation system. As IMPACT enters its second decade, two new studies from researchers at the University of Virginia, Brown University and Stanford University provide evidence that IMPACT continues to support meaningful improvement in the effectiveness of DCPS teachers.

These new studies build on two prior studies that examined the effects of IMPACT in its early years. A 2015 study found that IMPACT induced some less effective teachers to voluntarily leave DCPS and incentivized others to improve their teaching skills. A second, 2017, study employing data from the first years of IMPACT found that the turnover of teachers identified by IMPACT as low performing resulted in DCPS hiring more-effective teachers who substantially improved students’ academic achievement.

However, much has changed since the early years of IMPACT. DCPS has had three new chancellors. Teacher evaluation has come under increased scrutiny nationally, as research situated in other school districts has shown the logistical and political challenges of implementing and sustaining meaningful teacher evaluation. IMPACT itself has undergone substantial modifications in several of its key design elements, including a reduced emphasis on test-based measures of teacher performance. These changes raise the question whether IMPACT continues to improve teaching and learning in DCPS.

Today, a research team from the University of Virginia, Brown University, and Stanford University make public results from two new studies based on data from recent years (2013-2017). In the first study, Thomas Dee, professor at Stanford University, Jessalynn James, a post-doctoral fellow at Brown University, and Jim Wyckoff, professor at the University of Virginia, find that the incentives created by IMPACT continue to encourage some low-performing teachers to voluntarily exit and to instigate improvement among those who remain. In a second paper, James and Wyckoff find that what might first appear to be relatively high turnover among DCPS teachers is largely driven by the turnover of low-performing teachers, whose exit improves student achievement.

“Overall, these findings indicate that rigorous teacher evaluation can make an important difference for both teaching quality and, most importantly, for student achievement.” said James, formerly a PhD student at the University of Virginia.

In their article forthcoming in Education Finance & Policy, James, Dee, and Wyckoff found that immediate feedback accompanied by a performance-based threat of dismissal both increases the voluntary attrition of lower-performing teachers and increases the performance of lower-performing teachers who returned.

Specifically, teachers identified as Minimally Effective on the teacher evaluation scale, meaning they will be fired if they don’t improve to at least the Developing rating category, are approximately 11 percentage points less likely to return the following year compared to an otherwise similar teacher not identified as Minimally Effective, an increase in attrition of approximately 40 percent.

“Not only do we see teachers identified as Minimally Effective choose to leave DC public schools, we also find the Minimally Effective teachers who remain meaningfully improve,” Dee said.

In the second paper, James and Wyckoff examine the effect of teacher turnover on student achievement and find that, when a Highly Effective teacher leaves DCPS, teaching quality suffers and student achievement declines in reading, but not in math. They also find that turnover among teachers rated Highly Effective was relatively low during the 2013-2017 period, at 10 percent, while turnover during this period among teachers identified as Minimally Effective was 55 percent.

“We were quite uncertain what we would find. During this period, IMPACT’s effectiveness could have weakened with transitions in leadership, meaningful design modifications, implementation fatigue, competing priorities, and pressure from stakeholders,” Wyckoff said. “The large effects we identify here, however, suggest that rigorous teacher evaluation can be sustained at least over an eight-year period.”

EdPolicyWorks is a joint collaboration between the Curry School of Education and Human Development and the Frank Batten School of Leadership and Public Policy. EdPolicyWorks brings together researchers from across the University of Virginia and the State to focus on important questions of educational policy and implications for the workforce.