Education Policy Seminar Series - Spring 2018
For the current Education Policy Seminar Series, please visit our website.
Meta-analysis: Promise and Pitfalls
Austin Nichols, Principal Scientist at Abt Associate’s Social and Economic Policy Division (SEP)
Monday February 19th 2018
Bio: Dr. Austin Nichols researches educational interventions, labor market policy, disability, tax and transfer policy, poverty and income mobility and volatility, health care, nonprofits and industrial organization, international development, and research methods. Dr. Nichols is a Principal Scientist at Abt’s Social and Economic Policy Division (SEP), and serves as project director, director of design and analysis, and internal reviewer on Abt evaluations. He is working on survey design and analysis tasks for an Early Learning Study in Massachusetts, and studies of Broncos FIRST at Western Michigan University, and Success Boston which promotes college persistence among predominantly low-income and first-generation college-goers. He is also conducting analysis on an IES-sponsored evaluation of Academic Language Intervention, and on an evaluation of Preschool Expansion Grants in Massachusetts. He also conducts analysis for the Benefit Offset National Demonstration, whose nearly 1 million participants are SSDI beneficiaries. His other ongoing disability work includes the Stay-at-Work/Return-to-Work project for DOL, and an Earnings Loss Study for VA. His research in the past 2 years has been published in Review of Industrial Organization, the Journal of Policy Analysis and Management, Social Security Bulletin, Economics of Means-Tested Transfer Programs in the United States, World Bank Economic Review, and on the websites of the Urban Institute, Brookings, and Abt Associates.
Abstract: How should we synthesize across studies measuring the impact of some “treatment” or intervention X on some outcome Y? The use of meta-analysis, or its generalization meta-regression (meta-analytic regression), has been promoted as the most efficient means to synthesize this information, and shows tremendous promise in precisely estimating average impacts from a large number of small studies (even when each has insufficient power to detect impacts). However, the method has several potential pitfalls related to the selection of studies and estimates from those studies. The most serious of these concerns the inclusion of estimates constructed using quasi-experimental methods. This presentation will outline this fundamental problem and suggest some feasible solutions, using Bayesian meta-analysis methods.
It’s About Time: Evidence on Time Use in Full-Versus Half-Day Preschool Contexts
Allison Atteberry, Assistant Professor, University of Colorado-Boulder
Monday April 2nd 2018
Bio: Allison Atteberry is an assistant professor in the Research and Evaluation Methodology (REM) program, within the CU-Boulder School of Education. She received her Ph.D. in 2011 from the Stanford School of Education in educational policy analysis, with a minor in statistics. Dr. Atteberry conducts research on teacher- and school-level interventions designed to improve the quality of instruction experienced by historically underserved students. As a field, we are increasingly aware of how difficult it is to determine whether policies, practices, and interventions have the intended impacts, and so Dr. Atteberry approaches her work with a strong interest in what constitutes compelling evidence of causal effects in quantitative research.
Abstract: High quality early childhood education (ECE) programs can alter children’s life trajectories and yield substantial social returns. One promising approach to realizing social benefits from ECE investments is through improvements to their quality and intensity. The current research project adds to the small existing literature by providing new experimental evidence about the impacts of full-day preschool, on a host of immediate- and medium-term outcomes in a small, predominantly low-income, non-White, and ELL district near Denver. Prior to 2016, this district provided only half-day preschool. In 2016-2017, the district created nine new full-day classrooms as part of a Full-Day Pre-K Pilot Program, and since then we have randomized two cohorts of applicants to these full-day slots or the business-as-usual half-day slots. Cohort 1 exhibited large effects on academic and socio-emotional outcomes by the end of the preschool year. Assuming that we continue to find evidence of positive effects of full-day preschool offers, the next policy-relevant question is about how the additional time in school is used, relative to how it would have been used in the absence of the full-day option. In the current paper, we present evidence about time-usage both inside and outside of formal pre-k time from teacher and parent surveys as well as classroom observational data. For districts that may consider adding full-day classrooms to their half-day options, these findings will be essential for thinking about how WPS made use of the extra time.
Rethinking the Problem of Teacher Quality
Susan Moore Johnson, Jerome T. Murphy Research Professor, Harvard Graduate School of Education
Friday April 27th 2018
Bio: Susan Moore Johnson studies, teaches, and consults about teacher policy, organizational change, and administrative practice. A former high school teacher and administrator, Johnson has a continuing interest in the work of teachers and the reform of schools. She has studied the leadership of superintendents and organization of school districts; the effects of collective bargaining on schools; the priorities of local teacher union leaders; teacher evaluation; the use of incentive pay plans for teachers; and the school as a context for adult work. Currently, Johnson directs the Project on the Next Generation of Teachers, which examines how best to recruit, develop, and retain a strong teaching force. She is the author or co-author of six books and many articles. She served as academic dean of the Ed School from 1993 to 1999. Between 2007 and 2015, Johnson was co-chair of the Public Education Leadership Project (PELP) a collaboration between Harvard’s Education and Business Schools.
Abstract: Based on research documenting wide variation in teachers’ performance, policymakers began in 2000 to explain the shortcomings of US schools by pointing to problems of “teacher quality,”. Relying on strategies that would improve schools by increasing human capital one teacher at a time, they adopted new rules and regulations that reduced barriers to teaching for the “best and brightest;” ranked and rewarded teachers based on their students’ standardized test performance; and accelerated dismissal for sub-par teachers. The logic was that schools staffed entirely with effective teachers would succeed. However, these policies did not deliver the results many had hoped, especially in schools serving high-poverty communities. In this talk, I will recommend a different perspective and response, arguing that schools cannot increase their capacity and success by focusing solely on the quality of individual teachers, but must simultaneously develop the school organizations in which those teachers work. Based on school-based studies conducted with my colleagues at the Project on the Next Generation of Teachers over the past 20 years, I will explain how successful schools approach several key practices—hiring, teacher collaboration, and teacher evaluation—as organizational processes, which can contribute steadily to the school’s capacity and all teachers’ effectiveness.