Getting Better: Illusory Correlations, Oh My!
Do most people only use 10% of their brain power? Are some people left-brained and others right-brained? Does playing Mozart’s music to infants boost their intelligence? Is the defining feature of dyslexia reversing letters? Do students learn best when teaching styles are matched to their learning styles? Are we in the middle of a massive epidemic of infantile autism? Contrary to popular opinion...no, no, no, no, no, and no.
It’d be a huge waste of brain if we only used 10% of it, and studies have shown that losing far less than 90% of your brain can be catastrophic. Furthermore, while the hemispheres of our brain are responsible, at times, for different functions, they work in an integrated manner to execute tasks. There’s no evidence that Mozart’s music, while beautiful, adds points to your young one’s IQ score (if only it were that easy…). While it’d be nice if dyslexia was just about reversing letters, making it easy to target intervention, dyslexia is a deficit in processing written language that can manifest in a number of different ways. As pervasive as the idea of learning styles is in the field of education, there’s currently no evidence that giving “visual” work to a “visual” learner and “auditory” work to an “auditory” learner increases student achievement. And no, we’re not experiencing an epidemic of autism...it’s just a lot easier to qualify as autistic now. Back in the 1980s, the American Psychiatric Association diagnostic manual (DSM-III) required individuals to meet six out of six criteria to be diagnosed with autism, but that changed by the mid-1990s to eight out of sixteen criteria. In addition, the manual expanded the number of diagnoses in the autism family, like Asperger’s syndrome, which also increased the number of students considered autistic.
Scott Lilienfeld, Steven Jay Lynn, John Ruscio, and Barry Beyerstein explore these myths and others in their deeply engaging work 50 Great Myths of Popular Psychology. It covers a wide range of myths from the informative (i.e. learning styles, left-brained / right-brained, etc.) to the tragic (i.e. the myth that criminal profiling is helpful in solving cases...in reality, profilers are statistically no better than you or me at crime-busting). What is striking, however, upon examining a number of the myths that relate more directly to those of us in education, is how much time, money, and energy can be wasted pursuing initiatives or solutions that are simply not grounded in reality. To see why, we turn to a term Lilienfeld et al. bring up frequently: illusory correlation.
How many times have you heard a variation of the phrase, “Well, it’s Friday, so the kids are going to be wild today!” or “There was a full moon last night, so today should be interesting!”? If you’re in education, chances are you’ve heard something along those lines quite frequently. My wife, who is a nurse, reports that the full moon also serves as an ominous warning to nurses on the night shift...get ready for an interesting night! Certainly, some educators and nurses are just having fun with the phrase and making conversation, but a surprising percentage of folks, when you ask them directly, think a full moon produces a behavioral difference in others (not themselves, of course...). However, as Lilienfeld et al. conclude through an examination of the data, there is no correlation between a full moon and, in their specific study, an increased number of psychiatric admits at hospitals. This is an example of an illusory correlation, when we think that two unrelated events are related.
Lilienfeld and his co-authors call the above the “Great Four-Fold Table of Life,” and apply it to many situations to demonstrate the power, and danger, of illusory correlations. Many of us tend to focus too much on the “A” and discount the frequency of B, C, and D. Remember that one night when that patient ran around the unit naked and knocked everything over? That was crazy...definitely a full moon that night! Sure, it probably was a pretty crazy night, but 1) it’s highly likely that that is an uncommon occurrence , 2) it had nothing to do with the full moon, and 3) the person may not even be remembering right in the first place (our memories are tragically fallible). Was there really a full moon that night? As we’ll see, illusory correlations can penetrate deeply into the educational vernacular as well, with potentially harmful consequences.
Our first example is ripped straight from the headlines, as the conversation around phonics instruction has become mainstream, even gracing the front page of the New York Times. The table above illustrates an illusory correlation many in the education world make that can have a detrimental effect on struggling students. While a chunk of students (some say ~40%) may learn how to decode in the absence of explicit phonics instruction, the majority of children need systematic phonics instruction in order to be able to decode effectively. The research on the benefit of explicit phonics instruction is well established (check out this Willingham book for a great overview), and overgeneralizing based on the “As” (a minority of students) prevents many “Bs” from getting what they need and deserve.
A second illusory correlation that plagues the education sector is the project-based learning trend that many propagate. I’ve noticed that many in the education sector have been overly influenced by the “As” above, concluding that because a number of students are able to communicate conceptual understanding following a project, creating lessons around projects is the way to go. For some students, who possess extensive prior knowledge and vocabulary, projects can be an effective way to express that knowledge in an interesting way. However, I fear that we significantly underestimate the frequency of “Bs”, whether they are under-served and/or academically struggling students who don’t have the knowledge necessary to engage in the activity, or they are the more timid and/or less confident students who piggy-back off of the others in the group during project-based lessons. Often, educators use projects as the means, not the end, and if students lack the requisite knowledge to engage in the activity, they flounder. Projects are best used at the culmination of a particular unit of study, after all students have had the opportunity to learn core knowledge and vocabulary from the teacher. Furthermore, our ideal picture of project-based learning often does not reflect the reality: students engaging in off-topic conversation, focusing on the Powerpoint font instead of the content, and unequal distribution of work in the group are just a few examples. In many cases, the project-based lesson is likely not the reason the “As” have success; we may have a directionality problem, in that those are the students who demonstrate success regardless of the lesson format.
Our final illusory correlation likely has a significant impact on who receives support in secondary schools, and, perhaps more importantly, who does not. Often, we assume that the students with high standardized test scores are also the students on track for high school readiness. As Emily Krone Phillips discusses in The Make or Break Year, and as we have found internally during our 8th Grade On-Track project, that is often not the case. At our east-side middle school, there are 13 students who scored “Advanced” or “Accelerated” on the spring 2019 Ohio ELA state test but are “off-track” for high school readiness (you can see our on-track indicator system in the previous link). At our west-side middle school, there are 9 such students. It would be easy to look at the academic record of those students and think, they’ll be okay, they have the ability, they’ll kick it into gear in high school. As Phillips points out, it often doesn’t work out that way. When thinking about who drops out, many people conjure up an image of the “pregnant teenager” or “wrong crowd kid”, but many dropouts are actually young people who don’t possess strong work habits and/or slowly disidentify with high school because they lack a strong attachment to the school (relationships). Many have demonstrated strong academic ability in the past, but it’s not enough to keep them afloat in a high school environment that expects you to be quite independent. While standardized assessments are still a strong predictor of success in college and beyond, GPA has been shown to be a stronger predictor of high school readiness and success. Obviously, we need to help the “Cs” and the “Ds”, but we can’t forget the “Bs” in the table above! They are a bigger group than we think, and could benefit greatly from increased support as they make the transition from 8th grade to 9th grade.
Back in 1998, Georgia governor Zell Miller inserted $105,000 into the state budget to fund a Baby Einstein cassette (if you’re scratching your head, click here) for every new baby in Georgia. The Florida state senate followed suit, requiring daycares that receive state funding to play classical music daily. However, there was one big problem. Playing classical music to infants doesn’t boost their intelligence. The Mozart Effect is a myth. According to Lilienfeld et al., the myth, which spread like wildfire, was based on one study that tracked how college students (not infants) did on a paper cutting/folding task immediately after listening to Mozart. The study focused on short-term effects (not long-term), was unable to be replicated, and researchers concluded the finding came down to short-term arousal (i.e. the students could’ve had a glass of lemonade and it would’ve had the same positive effect on their paper cutting/folding ability). While it was a small amount of money compared to the overall state budget, Georgia essentially spent $105K on a myth.
I can’t help but wonder if stories like this are more common than we think in education. Illusory correlations rob us of our time, energy, and, sometimes, money, that we could spend on initiatives or programs grounded in reality. As educators, we should all be skeptics. There are few evidence-based solutions (RCTs, replicated studies, etc.), many unproven solutions (worked in one specific context, not replicated, etc.), a bunch of myths, and no magic wands in education. As educators, we must maintain a delicate balance of being incredibly optimistic about helping our students, but relentlessly skeptical and discerning when determining how to spend our time, energy, and money in doing just that. Only one thing is for certain...we can go ahead and let those Baby Einstein cassettes collect dust in the corner.
Ben Pacht is the Director of Improvement of the School Performance Institute in Columbus, Ohio. The School Performance Institute is the learning and improvement arm of the United Schools Network. Send feedback to bpacht@unitedschoolsnetwork.org.