EdLab Podcast: How Ed Policy Gets Implemented

Across the country, educators, researchers, and policymakers are hard at work generating and testing new ideas to improve outcomes for students. Sometimes those ideas work: Whether school-based interventions or larger policy initiatives, they result in improved teaching, learning and development.

But then what happens? 

In this episode of the EdLab podcast, Associate Professor Beth Schueler talks about her work to better understand what happens after we have evidence that an intervention or policy benefits schools. How do leaders and policymakers implement policies that improve schools and school systems?

“How does a policy get on the agenda? How does it get adopted? How does it get passed, implemented and sustained over time?” Schueler said.

The daughter of an educator, Schueler grew up having plenty of conversations about schools around the dinner table. It didn’t take her long to find her way to a career in education policy. Schueler now conducts research at EdPolicyWorks, a research center in partnership with the University of Virginia School of Education and Human Development and the Batten School of Leadership and Public Policy.

You can listen to the conversation with Schueler wherever you listen to your favorite podcasts, including Apple, Amazon, or Spotify.

EdLab

Transcript

Edited for clarity.

Audrey Breen, host: Welcome back to EdLab. In today's episode, we're tackling the question what happens when evidence suggests an intervention makes a positive impact for students? So, we have evidence that something works. Then what? Who decides when and where it gets implemented and for how long?

EdLab is a podcast of the University of Virginia School of Education and Human Development, where we talk to the people behind the school's transformative research, innovative teaching, and thriving partnerships. In each episode, we aim to demystify some of the big questions being asked in education, policy, health and human services. I'm your host, Audrey Breen.

In today's episode, Education Policy Associate Professor Beth Schuler will help us explore the intersection of education policy and politics. As the daughter of a special education teacher, Beth was a part of conversations about schools and education around the dinner table from as early as she can remember. She followed her interest in policy through college and graduate school all the way to her work here at the University of Virginia.

Welcome, Beth. 

Beth Schueler: Thanks. Thanks so much for having me. Excited to be here.

Breen: Wonderful. Thank you. Can you just give us a little bit about your story? So, we are sitting here now at the UVA School of Education and Human Development. What led you up to this point in your career? And then we'll go forward from here.

Schueler: Okay. Well, I'll start at the beginning. I grew up in the San Francisco Bay area in California, and my dad was a high school special education teacher in public schools. And so, I grew up with a lot of dinner table conversations about schools, about inequality. Many of the schools my dad worked in, he was serving kind of a student population that looked quite different and was sort of more disadvantaged, I think, in general than the population of students that I was attending schools with across on the other side of the bay.

But I would go to school with him when he had…when I had pd days or whatever, when I had days off of school. And so, I just started to notice some of those inequities from an early age and got interested in them. But my dad working in this career for many years always gave me the message like, you could be anything you want, just don't become a teacher.

He sort of thought it was you know not as respected of a profession as I think we all would hope, it to be at the School of Education and Human Development. Yeah. And so I was getting that message as a high school student, my nerdy admission is I got interested in policy debate.

So, I was doing policy debate all throughout high school and college and ended up teaching policy debate. And so, despite his warning really fell in love with the teaching aspect but also was interested in it because of the policy kind of component. Pursued a masters in the politics of education at Teachers College in New York City. 

I worked in New York for a city council member who was involved in a lot of education policy issues. There were just like all these folks who felt very passionately about these policy issues. So I was asking myself, well, but do we know that what we're advocating for works and, I kind of ended up going down the research kind of rabbit hole, right?

And getting interested in trying to understand, you know, how we can kind of learn how to improve these systems and improve equity in these systems. So pursued a doctoral degree at the Harvard Graduate School of Education and then a postdoc at the Kennedy School. So just kind of kept down this, this research path and ended up, here at UVA, as a faculty member.

Breen: Well, we are glad you found your way to us. Education policy is a really large umbrella. And perhaps by talking a little bit more in specific detail about what questions you are asking, can you help define what that means?

Schueler: Yeah, that's a great point. So, I should say that I have mostly focused on, in my own work, on education policy, with respect to kind of K-12 education system. I have awesome colleagues who are doing work on early childhood policy and higher ed, but I've mostly focused on K-12 systems and mostly in the United States, although I've done some international work.

And I think I'm really interested in trying to understand how school systems and policy makers, whether it be at federal, state or local levels, can implement policies that are going to help school systems do a better job of addressing broader forms of social inequality, whether it be on the basis of race or social class.

And so, I have been interested in trying to understand, okay, how do we in particular improve the schools and school systems that are serving large concentrations of low-income students of color because, with the idea being that if you could improve those systems, you could kind of put a dent in broader forms of social inequality.

Through that process, I've gotten more and more interested in kind of the political aspect. So, I think of policy and politics as two related but kind of distinct concepts. So, you know, I think the education policy world has gotten a lot better since I've been a part of it at sort of disentangling and trying to understand what works--like, what are the most effective policies, how can we estimate the impact of different programs and policies on student outcomes and on equity of outcomes.

But I think we've paid a little bit less attention to the political dynamics that get us the policies that we end up with or that prevent us from adopting and implementing policies that might promote equity or improve student achievement overall. And so that's another aspect of what I'm really interested in, is trying to understand the political process.

So how does, how does a policy get on the agenda? How does it get adopted? How does it get passed, implemented, sustained over time? Can we identify policies that are not only effective but are also have the ability--they're politically viable, right? That can have the potential to get adopted and sort of sustained over time because it's only useful to identify the effective policies if they're they can actually happen. Right.

Breen: They are used. Yeah. And implemented. You mentioned debate in particular as being a part of your story. I know some of the recent work you've been doing is around debate. Can you speak a little bit of your research on the topic and then explain a little bit about how that could be related or is related to policy?

Schueler: Sure. Yeah. It's interesting. A lot of my work has focused on system level reforms. But this one's been a little bit different and I think is connected to broader interests in not just how politics affects the policy we end up with, but how does the policy we end up with, like, how do schools prepare students to be engaged in the political process? Right.

Breen: Interesting.

Schueler: So, you know, how do schools develop students' broader what I think of as civic competencies: so critical thinking skills, argumentation skills, you know, media literacy, and so forth, the ability to kind of analyze policies and that kind of thing. And so those are some of the skills that these policy debate leagues are trying to actively trying to kind of promote and develop in the students who are participating in their programs now. Debate programs have historically been found in kind of private schools or the more well-resourced suburban public schools. But there's a series of these leagues where the goal is really to expand access to large urban school districts that typically haven't had debate programs. And so, I have been involved in recent years in kind of evaluating the impact of participation in these programs on, students’ academic outcomes, on their critical thinking skills, and also, on their post-secondary attainment and success.

To me, I think there's a few intersections with, with policy. One is, as I mentioned, debate as an example of how schools can prepare students as citizens to be involved in the policy process. 

Breen: Yeah. 

Schueler:  I don't think we have a lot of research on policy education itself, like, how to students learn about public policy.

This work that I mentioned was conducted... was focused on the Boston Public School System. And we just learned that the state legislature in Massachusetts actually funded the Boston debate program for the next fiscal year.

And so, I think there are policy moves that can help to try to scale some of these, effective programs more widely. So, there's sort of policy levers that can help to support those kinds of programs.

Breen: In the beginning of the conversation, you talked about the value of the research element of really working to uncover which of these policy proposals can be most effective. Can you speak a little bit to what you're finding in these debate programs? What outcomes they might bring for students?

Schueler: Yeah, sure. I sort of skipped over that part that didn’t I? Important detail. So yeah, we look at impacts of participation on student academic achievement as measured by standardized test-based measures. But one thing that I thought was particularly unique and interesting about this study was we were able to look at student sub skills on the assessments.

And so what we showed was that the gains on English language arts tests were really concentrated in the standards that are more focused on like analytical thinking skills, critical thinking skills, argumentation skills, as opposed to the standards that assess important but more rote skills, like, can you memorize the conventions of language and apply them, that kind of thing.

Breen: Got it.

So, we're sort of making the argument that this is evidence that these programs really seem to improve critical thinking abilities. And then we also looked at the impact on college going outcomes and found big impacts there. So, these programs seemed to increase the likelihood that students enter higher education.

And I should say, the other thing that is, I think, notable about the study is these are programs for middle and high school students. And we really don't have, as a field, a ton of programs we can point to that really make a meaningful impact in reading and English language arts achievement for older kids.

Breen: Interesting.

There's a lot of work in the, you know, elementary, age, which makes a lot of sense as students are learning to read at that age. But I think for policymakers who are thinking, well, how can I...I think there's sometimes this misperception that, oh, it's too late in high school to make a difference. 

Breen: Right.

Schueler: And so if you're a policymaker listening, you're like, how can I boost my high school students' reading achievement scores. This is a really unique and promising approach.

Breen: You mentioned that this project was primarily based in Boston. Can you speak a little bit to how you connect with partners?

Schueler: Yeah, yeah, that's a great question. Thanks for the opportunity because yeah, it's definitely not me. In addition to just collaborators at other universities and doc students and postdocs and other faculty members, most of the work is, as you sort of suggested, in partnership with other kind of organizations or educational agencies oftentimes.

So the debate work required both data and partnership from the Boston Debate League, that was providing information about who had participated in debate over time, but then also data from the Boston Public Schools who, was able to provide information on both the demographic characteristics of these participants, but also the achievement outcomes. And so that required a lot of coordination between these two partners to make that happen. I'm very grateful for their partnership. 

In other work I've had the opportunity to partner quite a bit with state agencies of education. So, a lot of my work has been in connection with the Massachusetts Department of Elementary and Secondary Education. I have studied a number of district-wide improvement initiatives in Massachusetts over the years, as well as the Virginia Department of Education.

So Covid was unfolding, and I ended up working with my colleague here, Luke Miller, another faculty member at UVA, to launch a project in partnership with the VDOE, where we're looking at how the equity of both student and teacher outcomes have changed in the post-Covid period.

And so, there's a whole team of folks, postdocs and research analysts that are working on that project. And the goal is really to not only produce academic papers, but also policy briefs that are really speaking to the questions that are of interest to the department. And we can be providing research that's more useful for them.

So, we're meeting with folks at the department regularly and trying to put our research skills to good use.

Breen: Can you speak a little bit to the way the pandemic, Covid has shifted the work around education policy?

Schueler: Yeah. It's a big question. I there's a lot of ways to answer it. I'll pick one. I mean, I think, you know, for folks in my field who are very concerned pre-pandemic about educational inequality on kind of racial dimensions, on economic dimensions, I think that was motivated by this belief that there were sort of unacceptably high levels of inequality.

But the truth is that pre-pandemic, we actually had as a country made some progress on narrowing some of those inequities. And what happened with the pandemic I think there's pretty compelling research to show it really just exacerbated and widened inequity of achievement on a number of dimensions.

Policy researchers and policy makers are pretty well aware of this rise in inequity. But like public opinion polling, for example, shows that parents are not quite as worried about this rise in inequity and these kind of academic challenges that I think will continue to affect students in the long run who were impacted during their school years or even before they got to school.

And so I think that's part of why it's so interesting to understand the policy piece, but also the public perception and political piece. Because if we have this kind of disconnect between what policymakers think is going on and what they're prioritizing and what parents care about, it's going to be very hard to have sufficient public will to really address those problems.

Breen: Interesting.

Schueler: Yeah, yeah. Some people are calling it like an urgency gap.

Breen: Interesting. Yeah. Oh, that's an interesting phrase. Well, one final question for you that might help with sort of looking into the future: If you imagined us having a conversation ten years from now, what would you hope to know?

Schueler: Yeah. It's a tough one. I think for me, as I sort of mentioned, I think as a field, we've gotten a lot better at, in a compelling way, identifying what the impact of particular policies and programs is. But we know less about, okay, well, if we know this is effective and we know this reduces inequity, how come it's not...how come we can't make this policy a reality? How come it's hard to scale and implement on a wider basis? 

And so I think what I hope we could know more about is, how do we identify those politically viable policies? How do we get these kind of effective policies adopted and implemented on a wide enough scale to really make a dent in those inequities?

I mean, it's a hard question to answer because I think, in ten years, I couldn't have predicted that the pandemic was going to so dramatically shift the research questions that I was answering and addressing. And I think we try to be nimble and responsive to the needs of the field. And so it's a little bit, we don't know what we are going to need to know in ten years necessarily. 

Breen: Right. 

Schueler: So, there's some enduring questions. And I think the one I mentioned is one of them. But it could change, and new things will definitely pop up. And that's I think what keeps this job interesting is that there's always new problems and new questions. And you're going to have to find new routes to getting at these problems.

Breen: It sounds like there is hope in the sense that we have evidence that there are tools that work, and I appreciate you elevating that that doesn't automatically mean that the solution is moving forward. You know, that there are lots of these other forces and components and parts of this whole system of education.

Well, thank you, Beth, so much for spending some time with us today. It was, as always, a pleasure to hear about your work and what you're doing. And, how about we do this again? 

Schueler: I love it. I’m in.

Breen: And we’ll figure out what questions you're asking and how close we've gotten to some more of those answers. Thank you so much.

Schueler: That's great. Thanks for the opportunity. Take care. Bye.