Note: Here’s part eight in our Learning to Improve series which is the primary focus of the School Performance Institute (SPI) blog this year. In it, we spotlight issues related to building the capacity to do improvement science in schools while working on an important problem of practice.

In my last post, I provided an analysis for the educational funding strategy shift at the Gates Foundation.

The gist of the shift is that Gates is funding organizations that are marrying on-track indicator systems with improvement science in order to improve important student outcomes. I know this because I went after a Gates Grant last year and didn’t get it. But, I’m glad I did it because of the learning that came out of the process. In fact, it led to the current 8th Grade On-Track project I’m running at Columbus Collegiate Academy this school year.

8th Grade On-Track

Columbus Collegiate Academy (CCA) is a highly disadvantaged middle school in the Near East Side neighborhood with a history of high 8th grade state test scores. However, by utilizing emerging on-track indicator research from Chicago and California, we discovered that more of our 8th graders were off-track for high school readiness than we previously knew. Off-track 8th graders become off-track 9th graders, and this is especially problematic because researchers have shown that freshman year is the make-or-break year for high school graduation. Because of the negative outcomes associated with dropping out, it isn’t an exaggeration to say that the 8th and 9th grades have life-altering implications.   

There are three primary goals of our 8th Grade On-Track project. First, we started by defining what it means to be on-track for high school readiness in 8th grade. For this purpose, we’ve adopted an on-track indicator system that was created in Oakland by CORE Districts. In Oakland, the system has been shown to have 96% accuracy in predicting 8th graders that go on to graduate from high school. Second, we’ve been working to build the improvement science capacity of our project team. Team members are learning to diagnose the various root causes of student off-track status and designing interventions as countermeasures for those root causes. Third, we are working intently to increase the percentage of 8th graders that are on-track for high school readiness.

On-Track Indicator System

It was critical for us to have a strong definition of what it means to be on-track. Because our project is focused on 8th graders, we knew that the most important next step in their educational journey is high school graduation. While that certainly isn’t the end goal for our students, it’s a necessary next step to having college and career options. In order to be considered on-track, students need to meet all of the indicators outlined in the chart below.

The on-track indicator system serves as the common goal that the team coalesces around and improvement science is the methodology through which we work to increase the rates of on-track students.  

Improvement Science

Improvement science is critical because early on in the project we realized that our team didn’t have a strong system in place for diagnosing why students were off-track and designing interventions tagged to those root causes. Previous to uncovering this, it was an invisible problem within our system. We have strong data-driven instructional methods in place in our schools, but I’ve found that those methods only take you so far. As a result, we’re working to build the capacity of our team to use improvement science tools for this purpose. What we’ve developed is a simple but profound innovation by which we use a tool called the Five Whys to figure out why students are off-track. From there, we take that root cause and use a PDSA tool (Plan-Do-Study-Act) for the iterative testing of interventions. Each PDSA cycle is a mini-experiment in which observed outcomes are compared to predictions and the differences between the two become the learning that drives decisions about next steps with the intervention.

I’ve linked examples of both tools that we’ve used in our project this year.

[SPI Resource: Five Whys Tool]

[SPI Resource: PDSA Cycle Tool]

Formula for Improvement

The formula for improvement is simple in its elegance. First, we’ve created a common goal that everyone on our improvement team is working towards. This goal is predictive of the most important next step in our students’ educational journey- high school graduation. Second, we’ve identified improvement science as our methodology through which we work to increase on-track rates. Third, we use the Five Whys and PDSA tools to diagnose why individual students are off-track and then design and test interventions aimed at getting students back on-track. Then we rinse and repeat, meaning we follow that root cause analysis and intervention design and testing cycle over and over for all the students that are off-track.

More on How We’re Learning to Improve

With funding support from the Martha Holden Jennings Foundation, School Performance Institute is partnering with Columbus Collegiate Academy to improve its high school readiness rates using improvement science methodology. SPI serves as both the improvement advisor and project manager of this work. We are also working to build an improvement network focused on increasing 8th grade on-track in high poverty middle schools that will launch this summer. If you are interested in learning more about this work, please email us at spi@unitedschoolsnetwork.org.

We’re also opening our doors to share our improvement practices at our two upcoming Study the Network workshops on April 12th (elementary school) and May 16th (middle school).

John A. Dues is the Director of School Performance Institute and the Chief Learning Officer for United Schools Network. The School Performance Institute is the learning and improvement arm of United Schools Network, an education nonprofit in Columbus, Ohio. Send feedback to jdues@unitedschoolsnetwork.org.