Note: Here’s part nine in our Learning to Improve series which is the primary focus of the School Performance Institute (SPI) blog this year. In it, we spotlight issues related to building the capacity to do improvement science in schools while working on an important problem of practice.

Improvement work in schools is incredibly challenging and while individual success stories are out there, by and large the track record of these efforts have been less-than-stellar.  

This is especially true in schools serving a high-concentration of low-income students. The normal work of running a school has to happen while at the same time that you’re adding in a new set of initiatives aimed at improving some aspect of your organization. This is incredibly hard and complex work that requires time, resources, and effort- it’s not for the faint of heart. In an earlier post, I likened these efforts to changing a tire while driving down the highway.

Improvement Partner

School Performance Institute (SPI), the learning and improvement arm of United Schools Network, serves as both the improvement advisor and project manager for this work using improvement science methodology. At our core is a commitment to continuous learning and improvement and a focus on using data to predict students’ learning, progress, and ultimately their success. This school year, we’ve served as a partner with Columbus Collegiate Academy (CCA) to work on a tough-to-solve problem. SPI fills a number of critical roles including project management, strategic communication, human capital management, improvement research & advising, and data analytics.  

CCA is one of four schools that make up United Schools Network (USN). Nearly 90% of its students are Black or Latino, and 100% of the students are low-income, yet CCA has a history of high 8th grade state test scores. However, by utilizing emerging on-track indicator research from the University of Chicago  and Core Districts, we discovered that more of our 8th graders were off-track for high school readiness than we previously knew. Tapping into the aforementioned on-track indicator research came as a direct result of applying for a Gates Foundation grant last year.

While SPI didn’t receive Gates funding, we initiated an internal 8th Grade On-Track project in partnership with CCA this year. There was urgency to do so because off-track 8th graders become off-track 9th graders, and this is problematic because researchers have shown that freshman year is the make-or-break year for high school graduation. Because of the negative outcomes associated with dropping out, it isn’t an exaggeration to say that the 8th and 9th grades have life-altering implications.

8th Grade On-Track Project

There are three primary goals of our 8th Grade On-Track project. First, the team started by defining what it means to be on-track for high school readiness in 8th grade. For this purpose, we’ve adopted the on-track indicator system that was created by CORE Districts. For example in Oakland, the system has been shown to have 96% accuracy in predicting the 8th graders that go on to graduate from high school. In our role as a data analytics partner, we’ve built a data dashboard that tracks each indicator within the on-track system.

SPI Resource: Data Dashboard

It was critical for us to have a strong definition of what it means to be on-track. Because our project is focused on 8th graders, we knew that the most important next step in their educational journey is high school graduation. While that certainly isn’t the end goal for our students, it’s a necessary next step to having college and career options. In order to be considered on-track, students need to meet all of the following indicators:

Second, we’ve been working to build the improvement science capacity of our project team. Team members are learning to diagnose the root causes of student off-track status and designing interventions as countermeasures for those root causes. Improvement science is critical because early on in the project we realized that our teacher teams didn’t have a system in place for diagnosing why students were off-track and designing interventions tagged to those root causes. Prior to this discovery, it was an invisible problem within our system. We have strong data-driven instructional methods in place, but we’ve found that those methods only take you so far. As a result, we’re working to build the capacity of our team to use improvement science tools for this purpose. What we’ve developed is a simple but profound innovation by which we use a Five Whys tool and Empathy Interviews to figure out why students are off-track. From there, we take that root cause and use a PDSA tool for the iterative testing of interventions. Each PDSA cycle is a mini-experiment in which observed outcomes are compared to predictions and the differences between the two become the learning that drives decisions about next steps with the intervention.

SPI Resource: Five Whys + PDSA Cycles

The on-track indicator system serves as the common goal around which the team coalesces and improvement science is the methodology through which our team works to increase the rate of on-track students.  During the first half of the year, the team spent time researching and choosing an on-track system, assembling the master data sheet, and building improvement science capacity. In the last couple of months, we’ve been using the formula for improvement to work on the third goal of this year’s project, which is to increase on-track rates. Early data from the use of the improvement formula are extremely encouraging and will be used for the rest of the school year to intervene with off-track 8th graders.

More on How We’re Learning to Improve

With funding support from the Martha Holden Jennings Foundation, School Performance Institute is partnering with Columbus Collegiate Academy to improve its high school readiness rates using improvement science methodology. SPI serves as both the improvement advisor and project manager of this work. We are also working to extend this work by building an improvement network focused on increasing 8th grade on-track in five high poverty middle schools.  If you are interested in learning more about the improvement network we are assembling, please email us at spi@unitedschoolsnetwork.org.

We’re also opening our doors to share our improvement practices at our final Study the Network workshop of the year on May 16th.

John A. Dues is the Director of School Performance Institute and the Chief Learning Officer for United Schools Network. The School Performance Institute is the learning and improvement arm of United Schools Network, an education nonprofit in Columbus, Ohio. Send feedback to jdues@unitedschoolsnetwork.org.