How do you measure remote learning engagement?

For the last two months, my colleague Ben Pacht and I have been writing about the work we are doing at United Schools Network (USN) in response to the COVID-19 pandemic and the subsequent school closure order in Ohio (see here, here, and here). In this fourth post in the series, we’ll summarize some key points we’ve made before, plus offer a few ideas specific to measuring and analyzing remote learning engagement.

Getting Better: What is student engagement, anyway?

As we are all aware, the COVID-19 pandemic has fundamentally altered how our society has functioned over the past few months. The education sector is no exception. As science teachers can attest, one of the unfortunate, enduring realities for life on this planet is that a minuscule virus, just microns in diameter, can leave an outsized impact on life as we know it. Yet a second enduring reality is that the form of life known as Homo sapiens is awfully resilient, and educators across the country are thinking creatively to ensure continuity of education for our students in a remote setting.

Successful remote learning requires a whole new system

The transition to remote learning due to the coronavirus pandemic has significantly shifted how United Schools Network (USN) plans and delivers educational experiences to its students. We've outlined our remote learning system—and it is helpful to think of it as a system—in our Education Plan. As soon as that document was created, and the new system was outlined, we immediately started thinking about how to improve it.

Getting Better: Improvement Science during COVID-19

As everybody in the education sector is painfully aware, the COVID-19 outbreak has drastically changed the way we think about and execute our jobs for the foreseeable future. Normal life has been put on hold while we determine how to navigate the uncertain months ahead. Questions materialize much faster than answers, and it is difficult to keep up with the near constant updates coming from federal, state, and local officials. I am hard-pressed to identify a moment in my life that has been as impactful, rapidly-evolving, and confusing. I don’t believe it’s hyperbolic to say that this is a truly unprecedented time for those of us in education.

Seven Early Lessons about Shifting Education during a Crisis

Once we at the United Schools Network learned about Governor DeWine’s school closure order on the afternoon of March 12, the pivot to remote learning began immediately. The first thing we did was create a COVID-19 Task Force in order to start planning for the initial three-week closure. While those plans were quickly implemented, we’ve now transitioned to creating longer term plans that address learning needs as Ohio schools will remain physically closed to May 1 and possibly beyond.

This limited (hopefully) blog series is our attempt to share what we’re learning during the pandemic. We’ve outlined seven early lessons in this post, which are focused on setting a network or district up for success throughout the closure

Improvement Science Tool: Fishbone Diagram

Why are we getting the outcomes that we currently do? In my last post, I implored organizational leaders to slow down and take the time to deeply understand this question before moving ahead with solutions.

The only way to understand current outcomes is to step back and see the system - the people, the policies, the attitudes, and the physical environment - in which a problem resides.

Slow Down to Speed Up

The methods of improvement science are often counterintuitive at first glance (which is probably why I love the approach!). Take the title of this blog post for instance - Slow Down to Speed Up. When we launch an improvement project with a school, very early on in the project I display this quote from Albert Einstein on a slide -"If I only had one hour to save the world, I would spend 55 minutes defining the problem and only five minutes finding the solution." - and ask participants what they think it means. To me, what Einstein is getting at is that it is well worth the time investment to deeply understand a problem before diving headlong into solutions.

Getting Better: Can we nudge our students toward better habits?

Many years ago, I found myself trapped in an interesting cycle with my dental hygienist. A few times each year, I would stretch out on her chair and sit patiently as she picked at, polished, and flossed my teeth. While the overall health of my mouth held up under her close scrutiny, she always mentioned one habit I couldn’t seem to shake.

Probably Wrong, Definitely Incomplete

On its face, probably wrong, definitely incomplete doesn’t inspire confidence. However, we put this phrase in the footer of almost all of the documents we create at School Performance Institute (SPI) to remind us of its value as a core ethos of our improvement work. When people first see this phrase they typically have a reaction something like, “Wow, that doesn’t inspire confidence!”

We disagree.

Launching Improvement Work

We’ve spent the last year learning how to use improvement science to increase the high school readiness rates of our 8th graders. Currently, we're gearing up to spread this improvement work from one middle school improvement team to a second middle school team at United Schools Network. During our upcoming launch day, we'll spend time creating team norms, studying improvement science, and digging into each school's on-track indicator system data. Thoughtful planning and a strong start to this work makes it much more likely that we’ll achieve our ambitious goals this school year.

Lessons in Improvement

Earlier this year, I kicked off our Learning to Improve blog series by discussing seven early lessons from our improvement science work at United Schools Network. The good thing is that I think that all of those lessons were on point. But, we’ve also learned a great deal about using improvement science methodology through our 8th Grade On-Track project this school year. In this post, I will expand on those early lessons and present some of our new learning.

Learning to Improve at United Schools Network

James’ reading grade dropped from a B in 7th grade to a D during the first trimester of his 8th grade year. The rest of his grades were a C or higher, his attendance rate was above 96%, and he had never been in serious trouble. Most people would look at James’ academic, attendance, and behavior stats and not see a student that is in need of intervention. We disagree. At the very moment that his reading grade dropped, James was in need of extra support.