Inequality

Recent work highlights the challenge of scaling evidence-based strategies to achieve social policy objectives. We evaluate, through a randomized control trial, a national financial incentive program designed to increase student engagement with college advising and completion of college and financial aid milestones that prior experimental studies demonstrate contribute to increased college enrollment and success. We find substantial positive effects of the incentive program on each of the incented behaviors: Treated students were more likely to engage regularly with a college advisor; apply to well-matched colleges and universities; and meet with an advisor to review their financial aid awards and discuss college costs. Yet students randomly offered the incentives were no more likely to enroll at higher-quality colleges and universities, despite being high in the distribution of college entrance exam scores and from a socioeconomic background that many institutions indicate is central to their diversity goals. Student responses to a survey administered the summer and fall after high school suggest that lack of admission to the most selective institutions, lack of affordability at selective institutions to which students were admitted, and student preferences to attend institutions closer to home explain the lack of enrollment effects.

More →

The improvement of low-performing school systems is one potential strategy for mitigating educational inequality. Some evidence suggests districtwide reform may be more effective than school-level change, but limited research examines district-level turnaround. There is also little scholarship examining the effects of turnaround reforms on outcomes beyond the first few years of implementation, on outcomes beyond test scores, or on the effectiveness of efforts to replicate district improvement successes beyond an initial reform context. We study these topics in Massachusetts, home to the Lawrence district representing a rare case of demonstrated improvements in the early years of state takeover and turnaround and where state leaders have since intervened in three other contexts as a result. We use statewide student-level administrative data (2006-07 to 2018-19) and event study methods to estimate medium-term reform impacts on test and non-test outcomes across four Massachusetts-based contexts: Lawrence, Holyoke, Springfield, and Southbridge. We find substantial district improvement was possible although sustaining the rate of gains was more complicated. Replicating gains in new contexts was also possible but not guaranteed. 

More →

Although existing research suggests that students benefit on a range of outcomes when they enroll in early algebra classes, policy efforts that accelerate algebra enrollment for large numbers of students often have negative effects. Explanations for this apparent contradiction often emphasize the potential role of teacher and peer effects, which could create positive effects for individual students placed into early algebra that would not translate to larger-scale policies. We use detailed data from Oregon that contain information on the teachers and peers to whom students are exposed in order to investigate these explanations. Our regression discontinuity analyses replicate key findings from prior studies, indicating that placement in eighth-grade algebra boosts student achievement in math and English language arts. We then demonstrate that eighth-grade algebra placement positively affects the achievement level of students’ classmates, as well as the years of experience and value added of students’ math teachers. The effects on peer composition that we observe are large enough to plausibly explain the majority of the effects of eighth-grade algebra on student test scores.

More →

This study reports the findings from a year-long randomized evaluation assessing the impact of assigning 62 classrooms in Nigeria to receive either blocked or interleaved math problem sets. Blocked practice sessions focused on a single skill at a time. Interleaved problem sets alternated between different skills within a practice session. On tests of short-term retention, interleaved practice increased test scores by 0.29 standard deviations. In contrast, we find no evidence that interleaving improves average performance on a cumulative assessment measuring retention of material over the academic year. We find some evidence of large impacts on the cumulative assessment at the bottom of the distribution, but these impacts appear to be offset by negative impacts at the top.

More →

Longitudinal studies can produce biased estimates of learning if children miss tests. In an application to summer learning, we illustrate how missing test scores can create an illusion of large summer learning gaps when true gaps are close to zero. We demonstrate two methods that reduce bias by exploiting the correlations between missing and observed scores on tests taken by the same child at different times. One method, multiple imputation, uses those correlations to fill in missing scores with plausible imputed scores. The other method models the correlations implicitly, using child-level random effects. Widespread adoption of these methods would improve the validity of summer learning studies and other longitudinal research in education.

More →

In this paper, we examine the fundamental and complex role that time plays in the learning process. We begin by developing a conceptual framework to elucidate the multiple obstacles schools face in converting allocated time into learning time. We then synthesize the causal research and document a clear positive effect of time on student achievement of small to medium magnitude, but also with likely diminishing marginal returns. Further descriptive analyses reveal how large differences in the length of the school day and year across public schools are an underappreciated dimension of educational inequality in the United States. Finally, our case study of time loss in one urban district demonstrates the potential to substantially increase learning time within existing constraints.

More →

The Every Student Succeeds Act (ESSA) began a new wave of school accountability under which states draw on multiple measures to assess school quality. States have options in terms of how to weight components in their school quality indices and how many years of data to use to determine school ratings. In this study, we simulate school ratings using eight years of administrative data in North Carolina to demonstrate how state decisions about school ratings and identification influence school ratings and the list of schools identified for improvement. We then evaluate these decisions against a framework that considers the validity, stability, and equity of the ratings, underscoring the inherent tradeoffs that come with each. We show that while a system that weights proficiency more heavily than growth produces more stable school ratings, identifying schools based on multiple years of performance data instead of one more than offsets the loss of stability in shifting to a growth measure. We conclude with recommendations for state accountability systems under ESSA and for federal policymaking moving forward.

More →

School districts across the U.S. have adopted funding policies designed to distribute resources more equitably across schools. However, schools are also increasing external fundraising efforts to supplement district budget allocations. We document the interaction between funding policies and fundraising efforts in Chicago Public Schools (CPS). We find that adoption of a weighted-student funding policy successfully reallocated more dollars to schools with high shares of students eligible for free/reduced-price (FRL) lunch, creating a policy-induced per-pupil expenditure gap. Further, almost all schools raised external funds over the study period with most dollars raised concentrated in schools serving relatively affluent populations. We estimate that external fundraising offset the policy-induced per- pupil expenditure gap between schools enrolling the lowest and highest shares of FRL-eligible students by 26-39 percent. Other districts have attempted to reallocate fundraised dollars to all schools; such a policy in CPS would have little impact on most schools’ budgets.

More →

School districts historically approached conflict-resolution from a zero-sum perspective: suspend students seen as disruptive and potentially harm them, or avoid suspensions and harm their classmates. Restorative practices (RP) -- focused on reparation and shared ownership of disciplinary justice -- are designed to avoid this trade-off by addressing undesirable behavior without imparting harm. This study examines Chicago Public Schools' adoption of RP. We identify decreased suspensions, improved school climate, and find no evidence of increased classroom disruption. We estimate a 19% decrease in arrests, including for violent offenses, with reduced arrests outside of school, providing evidence that RP substantively changed behavior.

More →

Despite documented benefits to college completion, more than a third of students who initially enroll in college do not ultimately earn a credential. Completing college requires students to navigate both institutional administrative tasks (e.g., registering for classes) and academic tasks within courses (e.g., completing homework). In postsecondary education, several promising interventions have shown that text-based outreach and communication can be a low-cost, easy to implement, and effective strategy for supporting administrative task navigation. In this paper, we report on two randomized controlled trials testing the effect of a text-based chatbot with artificial intelligence (AI) capability on students' academic task navigation in introductory courses (political science and economics). We find the academic chatbot significantly shifted students’ final grades, increasing the likelihood students received a course grade of B or higher by 5-6 percentage points and reduced the likelihood students dropped the course.

More →