When we as educational leaders are confronted with a problem, we typically assemble a school or district team to attempt to improve it. The team relies on the expertise they’ve acquired across their careers as classroom teachers, building administrators, and district level leaders. Let’s call this subject matter-knowledge. It includes skills like data-driven instruction, curriculum planning, and leading professional development, among many other activities. Subject-matter knowledge is critical for developing changes that result in improvement. While an obvious necessity, this type of knowledge alone is insufficient.
Read MoreFor the past four months, I’ve been writing about organizational goal-setting. In Part I of the series, I proposed four conditions that organizations should understand prior to setting a goal. In Part II, I introduced the idea of “arbitrary and capricious” education goals as well as the first five of my 10 Key Lessons for Data Analysis. In Part III I rounded out the lessons with an introduction to lessons 6-10. In this installment, we’ll take a look at an applied example of the lessons in action.
Read MoreFor the past three months, I’ve been writing about organizational goal-setting. In Part I of the series, I proposed four conditions that organizations should understand prior to setting a goal. In Part II, I introduced the idea of “arbitrary and capricious” education goals and key data analysis lessons 1-5 . In this installment, I’ll outline key lessons 6-10 and then tie up the series in Part IV with an applied example from United Schools Network.
Read MoreJanuary is a popular month to set new goals, so I decided to kick-off this year with a four-part series on this very topic. In Part I of the series, I proposed four conditions that organizations should understand prior to setting a goal.
Organizations should understand the capability of the system or process under study.
Organizations should understand variation within the system or process under study.
Organizations should understand if the system or process under study is stable.
Organizations should have a logical answer to the question, “By what method?”
Absent an understanding of these conditions, goals are too often “arbitrary and capricious.”
At a recent district leadership team meeting, I put the following quote up on a slide: “Goal setting is often an act of desperation.”1 We are in the midst of updating our strategic plan at United Schools Network, so the purpose of the quote was to start a discussion on healthy goal-setting and to provide a framework for any goal-setting the team would do as a part of this process. I think the typical reaction to the quote is something like the following: “But I thought goal-setting was something highly effective people and organizations do?” I would argue however, that this is rarely the case, be it in organizations or accountability systems, and only can be true if a number of conditions are met during the process.
Read MoreA critical component of the Planning phase of the cycle is the idea of operational definitions. The concept of operational definitions is straightforward. The idea is that language must be made operational in order to perform the basic functions in an organization. To put it another way, an operational definition puts communicable meaning into a concept. Concepts that are important to schools such as attendance, engagement, and learning have no communicable value until they are expressed in operational terms.
Read MoreNow that I’ve outlined the basic idea of the PDSA cycle, it will be helpful to turn to a real PDSA that I used in my work at United Schools Network. This in fact was the first PDSA I ever designed, so it by no means is being held up as an exemplar. However, I think it is useful as an introductory point to the concept because this particular example is so simple. I’m also happy to report that for a first attempt, this PDSA cycle was fairly successful.
Read MoreOne of the most powerful tools that sits at the heart of Deming’s Theory of Knowledge is in fact the Plan-Do-Act-Study (PDSA) cycle. PDSA cycles are experiments during which you gather evidence to test your theories. Observed outcomes are compared to predictions and the differences between the two become the learning that drives decisions about next steps with your theory. The know-how generated through each successive PDSA cycle ultimately becomes the practice-based evidence that demonstrates that some process, tool, or modified staff role or relationship works effectively under a variety of conditions and that quality outcomes will reliably ensue within your organization.
Read MoreLast month, I discussed a powerful tool, the process behavior chart, that can be used to filter the noise out of our data. The whole point of this series has been to think through how to properly interpret and react to data, which includes the filtering process. Unfortunately, much of what happens on the data analysis front in the education sector is akin to writing fiction. Writing fiction will be the main topic of this post.
Read More
Last month, I discussed the difference between information and knowledge by analogizing the two concepts to data ponds (information) and data streams (knowledge). A key idea in the transformation of information to knowledge is adding the element of time and visualizing the data in a tool called a process behavior chart. Part of the power of the process behavior chart (PBC) is its ability to filter out the noise in our data; the idea of filtering out data “noise” is the focus of this post.
Read MoreLast month, I outlined why data has no meaning apart from their context. The discussion centered on some key ideas for presenting data in context as well as a logical definition of improvement. I also introduced an example of how data is often misinterpreted in the education sector. In this post, I’ll begin to lay the foundation for understanding variation in quality improvement work; this will be a precursor to comprehending why so much of the data analysis that is done in organizations is akin to writing fiction.
Read More
In the K-12 education sector, one of the primary uses of data is in state accountability systems. Many states now issue district and school report cards typically based on various performance metrics such as proficiency rates on standardized tests, absenteeism rates, and college and career readiness indicators. Unfortunately though, as James Leonard stated so eloquently in The New Philosophy for K-12 Education:
Absent an understanding of the type of variation present, any discussion of accountability is a burlesque!