The Power of PDSA
Note: This is Part I of a two-part series on the Plan-Do-Study-Act (PDSA) Cycle developed by W. Edwards Deming.
The Plan-Do-Study-Act Cycle, in the words of W. Edwards Deming, “is a flow diagram for learning, and for improvement of a product or of a process.” Edward Baker picked up on this description and introduced the PDSA this way in The Symphony of Profound Knowledge:
Deming’s Plan, Do, Study, Act (PDSA) cycle is a process to gain empirical knowledge by applying and evaluating theories, formal or informal, in order to learn, to improve, and maybe by serendipity to innovate. Innovation can occur in the cycle, but as Deming often said, we can’t plan to make a discovery. Two components work together in the cycle: the invisible world of theory and observation of the visible world.
Deming’s Plan-Do-Act-Study Cycle
One of the most powerful tools that sits at the heart of Deming’s Theory of Knowledge is in fact the Plan-Do-Act-Study (PDSA) cycle. PDSA cycles are experiments during which you gather evidence to test your theories. Observed outcomes are compared to predictions and the differences between the two become the learning that drives decisions about next steps with your theory. The know-how generated through each successive PDSA cycle ultimately becomes the practice-based evidence that demonstrates that some process, tool, or modified staff role or relationship works effectively under a variety of conditions and that quality outcomes will reliably ensue within your organization.
This is a key differentiator of PDSAs as a learning process as compared to using ideas generated through traditional research methods, even gold standard methods such as randomized controlled trials. By their very design, studies that result in evidence-based practices discount externalities instead of solving for them. This is not helpful for educators working in real schools and classrooms. The idea that many interventions are effective in some places but almost none of them work everywhere is such a common idea in the education research sector that this phenomenon has its own name- effects heterogeneity.
Beyond the concern of effects heterogeneity, there are a number of other reasons the PDSA is an effective tool. People learn better when they make predictions as a part of the process because making a prediction during the planning phase of the PDSA forces us to think ahead about the outcomes. In my experience, we are often overly optimistic in terms of both the speed and magnitude at which improvement will occur. But because making a prediction causes us to examine more deeply the system, question, or theory we have in mind as we develop change ideas, we both get to see the thinking process of our team members as well as improve at making predictions over time. Learning about your ability to predict in addition to your organization’s ability to predict is a key part of the Theory of Knowledge.
It will be helpful here to turn to the template I use in my improvement work at United Schools Network. Snapshots of each section of the template along with a brief description of each of the five sections follow. In next month’s post I will do the same thing with an actual PDSA Cycle that I designed and implemented through an improvement project in Spring 2019.
PDSA Template
I first came across this PDSA template at the annual improvement summit at the Carnegie Foundation for the Advancement of Teaching, and they in turn, had gotten the template through their relationship with the Institute for Healthcare Improvement (IHI). Over the years, I’ve made minor modifications to the template, such as including a space for operational definitions, but this by and large is the IHI version. The section for operational definitions appears in the blank template, but this addition came after we had run a number of PDSAs. As a result, this section will not appear in the example PDSA next month.
PDSA Template: Header & Title
One of the innovations that I have made to the Carnegie and IHI version of the PDSA template is the header that appears at the top. This small tweak has been helpful for me, so I’m going to start here in terms of the template snapshots and descriptions. For the five PDSA template sections, I’ll start with a brief overview of the step in the process, and then I’ll drop in the snapshot of the form followed by its description.
In the header of the document there are three components to the template. The top left corner includes the title of the improvement project as well as the PDSA cycle start and end dates. These are the dates for this particular testing cycle rather than the start and end dates for the entire project. An improvement project can have many PDSA testing cycles; I’ve found testing cycles that are 2-4 weeks long to be a good rule of thumb for length. This length strikes a balance between having enough time to gather useful data with a short enough cycle that it can be useful in practical terms. The top right corner includes the label PDSA Tracker and is hyperlinked to a spreadsheet in which I track all of the testing cycles for a particular improvement project. The middle of the header includes a logo for our school network, which is helpful in case this document is shared with an external school or district or perhaps through a conference presentation.
Below the actual header, there is a place to record the title of the change idea that is being tested as well as the cycle number. For example, during the pandemic I ran an improvement project called Distant but Engaged with the aim of improving student engagement rates during remote learning. We tested the idea of offering synchronous intervention sessions in reading to 7th graders. The title of the third testing cycle for this change idea read “PDSA 3: How do you like me live?”
PDSA Template: Plan
Planning is prediction, but the further you look out into the future the more that you have to rely on theory to support your ideas. Dr. Deming suggested that a theory be tested first on a limited basis in a practical way in case the theory does not produce the type of improvement that was hypothesized at the outset. It is important here to remember Deming’s admonishment that “experience teaches nothing; experience is not knowledge.” Experience can only teach us something useful if it is framed in the testing of a theory.
The Plan section of the PDSA cycle includes both the details of the test as well as the plan for collecting data during the test cycle. I’ve broken the plan for the test and the plan for collecting data into four sections. In the first box, you list the questions you are trying to answer during the testing cycle as well as the explicit predictions of each improvement team member. My preference for the prediction is to have each individual make their own prediction independently, list those predictions separately in a bulleted list, and then discuss how each person came up with their prediction. Doing so allows for you to get a sense for how team members are thinking about the problem as well as their confidence in the change idea. In the next section, the Who, What, Where, and When of the Plan is outlined in detail. This box should include the logistics of the test, so it is very clear who is doing what as well as where and when those tasks are going to take place. It is critical to be specific so everyone knows their job in the testing cycle, and there should be enough detail so that others could replicate the test in the future if so desired.
The importance of Operational Definitions is paramount; the core concepts under study should be defined in a way that allows everyone to understand how measurement will occur. The definitions listed should include the criterion (What do you want to improve?), the test procedure (What methodology will be used for improvement?), and the decision rule (How will you know when things have improved?).
Similar to the Who, What, Where, and When section, the Plan for collecting data should be detailed and specific. Name the person(s) that will be responsible for data collection and include how often and where this data will be stored. My preference has been to add a table for storing the data right under the Plan for collecting data section, so that everything is in one easy-to-access place.
PDSA Template: Do
The Do phase is pretty straightforward. The team runs the test on a small scale or in other words, the Plan is implemented.
In this section of the template, the team records what actually happened, the data that was collected, and the observations that were made. At first, I was a bit confused with how the information in this section was different from what is captured in the Study section. The important distinction here is that in the Do section you simply record information about the implementation of the Plan without analysis or interpretation.
PDSA Template: Study
A key revelation of the PDSA process occurred when I realized that the power of the testing cycle lies in the comparison of the prediction to the outcomes. Each PDSA cycle is a mini-experiment designed to test a change idea. Observed outcomes are compared to predictions and the differences between the two become the learning that drives decisions about next steps with the change idea.
Again, the purpose of the Study section is to analyze the results of the test and compare them to your predictions. In the Study phase you are not only summarizing the results you are interpreting those results and reflecting on what you learned. The learning gleaned from the experiment will serve as the basis for how the team decides to Act in the final stage of the PDSA process.
PDSA Template: Act
After the test has been completed and analyzed, the final step in the PDSA cycle is the Act stage. In this stage the team has to decide how to react to the test results. No matter what happens, even if the test results were completely different from what was expected, learning occurred. Instead of fretting about a “failed” test, the team should instead focus on what was learned. Additionally, if the “start small, learn fast” approach was held to in the PDSA cycle, then the team can also be thankful that the learning occurred prior to a significant investment of time, effort, and resources.
In the Act step, the team uses the learning from the Do and Study steps to make plans for what to do next. The immediate decision falls into one of three buckets, or what I call the 3 A’s - abandon, adapt, or adopt. The team might choose to abandon a change idea if the results didn’t indicate that the idea was effective in bringing about the intended outcome. This typically won’t happen after only one testing cycle, but if the change idea has been run through a number of cycles without success, then this might be a prudent choice. A second option is to adapt a change idea based on the learning from the previous cycle. It may be that the test didn’t go perfectly as planned but some component may have worked to an extent that the team chooses to build on it in subsequent iterations. The final choice is to adopt and standardize the change idea in your organization. This typically would occur after the idea has been tested under various conditions through multiple testing cycles.
PDSA Template: Footer
It’s worth mentioning that the PDSA template, like all of the documents I design for my improvement work, have the same phrase in the footer. The footer always reads “Probably wrong, definitely incomplete”, an idea I picked up from a Carnegie improvement conference. I thought the idea was so important, that I not only include it in the footer, but also inculcate improvement team members to that way of thinking.
On its face, “Probably wrong, definitely incomplete” doesn’t inspire confidence, however, it is a core ethos of both improvement work and the PDSA cycle. It’s a foundational idea for the culture of improvement that is absolutely critical when trying to improve complex problems in schools. It’s not that we don’t have confidence in our ability to improve really tough-to-solve problems. It’s just that if you don’t humble yourself at the beginning of these types of projects, I believe you are setting yourself up for failure and that you likely know far less about the road to improvement than you think at the onset. If you charge ahead without the needed know-how, your improvement project is doomed from the start.
This post was focused on taking you through the four steps of the PDSA cycle and a template that makes running PDSAs more straightforward. Next month, I’ll provide an example of the PDSA in action from an improvement project that we ran at United Schools Network. With the template and example in hand, you’ll be able to harness the power of the PDSA in your own school improvement work.
***
John A. Dues is the Chief Learning Officer for United Schools Network, a nonprofit charter-management organization that supports four public charter schools in Columbus, Ohio. Send feedback to jdues@unitedschoolsnetwork.org.