Documenting COVID’s Schooling Setbacks

Since its founding four years ago, an interdisciplinary team at the Metro Atlanta Policy Lab for Education (MAPLE), part of the Georgia Policy Labs in the Andrew Young School of Policy Studies, has been helping Atlanta-area public school districts improve learning outcomes for an estimated 30 percent of Georgia public school students. Never has their work been more crucial than during the pandemic.

“One of the strengths of the MAPLE team is they’re about student achievement and trying to help the districts,” says Rebecca Braaten, director of research, data and evaluation for the DeKalb County Public Schools. For Braaten, MAPLE has served as an invaluable resource to support the district’s more than 93,000 students.

In mid-March 2020, as a result of the COVID-19 pandemic, Georgia public schools were forced to shut down for the remainder of the school year and pivot to virtual classes. Statewide assessments, comparing student performance to state benchmarks, which districts rely on to track student learning, were cancelled.

To help educators and school leaders better understand the impact on learning, MAPLE began a study to assess how the shutdown was affecting student achievement growth.

“We looked at how students were doing relative to what we would’ve predicted based on their past progress,” says Tim Sass, Distinguished University Professor of Economics and the faculty director of MAPLE.

Their first report, published in May 2021, showed that the pandemic took a heavy toll on learning gains. Students were doing much worse on average than would be expected based on their pre-COVID trajectories. These impacts on student achievement growth varied considerably by subject, grade level, and school district.

The team also broke down the data by race and ethnicity, English learner status, and qualification for free and reduced-price lunch, which Sass describes as “a very crude measure of poverty.”

The news wasn’t all bad. Problems related to technology access — issues that disproportionally affected lower-income students across the nation, especially early on in the pandemic — were generally minimal or even absent in MAPLE’s partner districts, says Jennifer Darling-Aduana, assistant professor in the Department of Learning Sciences in the College of Education & Human Development and a member of the MAPLE research team.

However, students eligible for free or reduced meals generally experienced lower achievement growth (relative to similar students prior to the pandemic) than did students who were not eligible for subsidized meals. On average, historically marginalized groups, such as Black students, Hispanic students and English learners, also experienced greater reductions in achievement growth.

“Generally, students from what we call ‘historically advantaged groups’ ended up doing as well in virtual learning as we would expect in face-to-face instruction,” says Darling-Aduana. “But other students struggled more, in part because they were less likely to have a quiet distraction-free environment with parental support in monitoring.”

The researchers also looked at how the pandemic affected learners with disabilities, although because disabilities are so wide-ranging, they are continuing to further disaggregate data.

After offering in-depth analysis of how the pandemic differentially affected student achievement, MAPLE has provided educators with concrete ways to use federal dollars, offering evidence-based strategies to boost student success. Knowing where the lags in achievement are allows a district to target federal funds to serve the academically neediest students.

“One of the advantages of MAPLE is that you’re able to work with real-time data and then your results are able to inform what the districts are doing in real time,” says Darling-Aduana.

In DeKalb County, for example, MAPLE researchers took a detailed look at tens of thousands of students’ formative or benchmark assessments and were able to track learning loss or gains as the pandemic went on.

“It was important to look at learning loss and then look at next steps and what the recommendations are based on the patterns seen across the district,” says Braaten, who notes that the partnership has been extremely valuable to the district. “Their research helps us engage in the deeper work that needs to be done to move the district forward after this unprecedented disruption.”