Do we really need to transform our education system?

There are frequent calls for transformation and reform in the education sector going back decades. As a starting point, I’m going to examine two reports -- one well-known (“A Nation at Risk”) and the other lesser so (the “Sandia Report”) -- that attempt to answer the question posed in the title. Over the next several posts, I will offer my own thoughts as to the need for educational transformation (spoiler alert, I think it is necessary). In those future articles, I’ll provide a definition for transformation as well as a description for what exactly we would be transforming from and what we would be transforming to. But first, let’s dive into those two reports.

A Nation at Risk?

“A Nation at Risk” was published by the National Commission on Excellence in Education in April 1983. The report cited statistics such as: "The SAT demonstrates a virtually unbroken decline from 1963 to 1980. Average verbal scores fell over 50 points and average mathematics scores dropped nearly 40 points." The report did not mince words and, in addition to the alarming statistics, it included a number of memorable lines including the following:

We report to the American people that while we can take justifiable pride in what our schools and colleges have historically accomplished and contributed to the United States and the well-being of its people, the educational foundations of our society are presently being eroded by a rising tide of mediocrity that threatens our very future as a Nation and a people.[1]

The purpose of the report was “to help define the problems afflicting American education and to provide solutions…”[2] In other words, it was meant to awaken the American public to the problems afflicting its schools. Awaken it did; the report has been referenced in numerous books and articles in the proceeding four decades and served as the foundation for many of the education reforms that have been undertaken since the report was published including Goals 2000, No Child Left Behind, and Race to the Top.

Less than a decade after “A Nation at Risk” was published, a far lesser-known follow up report was commissioned by the Department of Energy. Prior to my research for a book that I am writing, I in fact, had never heard of the “Sandia Report”, named for the engineers from the Sandia National Laboratories who authored it. They had set out to create economic forecasts and were not centrally concerned with education test scores. However, they were quite surprised by what they found when they completed their own analysis of the education data from “A Nation at Risk”. On nearly every measure of achievement, the Sandia analysts found steady or slightly improving trends in the test data.[3]

While it was true that SAT scores had declined from the early 1960s until the early 1980s as “A Nation at Risk” asserted, this data lacked important context. It is this context that led Sandia’s authors to the conclusion that the decline in average scores did not mean that more recent high school students weren’t as capable as their peers in the 1960’s.

How could this be?

When the Sandia researchers broke the test scores out by subgroups of students (e.g., race, socioeconomic status, class rank, etc.), they found steady or improving scores for all groups. This seemingly confounding finding can be chalked up to a statistical phenomenon called Simpson’s Paradox which occurs when trends that appear in an aggregated data set (as in “A Nation at Risk”) reverse when the data are separated into subgroups (as in the “Sandia Report”). In other words, the declining SAT results underscore a more diverse mix of students taking the test over time rather than decreasing student performance. Similarly, the Sandia researchers found steady or slightly improving trends on nearly every measure they investigated including dropout statistics, standardized tests, postsecondary studies, educational funding, and international assessment comparisons.

Counterintuitive Ideas

There are two seemingly counterintuitive ideas here. On one hand, I don’t believe that there is evidence that schools have been on a steady decline for the last 50-60 years. This was the thesis of “A Nation at Risk”; one that many others have picked up on since then. On the other hand, I believe that to achieve equitable outcomes for all students, schools must undergo a transformation on an order of magnitude seldom seen in the history of organizations. On their face, these two ideas may seem to be mutually exclusive, but the opposite is true. Test scores are a narrow definition of success, and while they could potentially provide useful information to educators and policy makers, they are more of an inspection and sorting mechanism than an improvement tool.

The Sandia authors drew a similar conclusion in 1992 despite the steady or improving trends they noted in their analysis. From their report:

First, it is not clear to us that all the measures analyzed by us and others are appropriate barometers of performance for the education system (recall that our selection criterion was that a measure be popular, not necessarily appropriate). Thus, the trend data on some of these measures, positive or negative, may be irrelevant. Second, even if a particular measure is appropriate, steady or slightly improving performance may not be adequate to meet future societal requirements in an increasingly competitive world. Finally, in some appropriate measures, the performance of the U.S. education system is clearly deficient.[4]

For most people, what you see in the educational data from the last half century or so depends in large part on what you thought about our country’s schools before you looked. The authors of “A Nation at Risk” started with the premise that our schools were on the decline, and that’s what they found. That’s what makes the “Sandia Report” so unique. The authors saw themselves as outsiders expecting to verify the findings of the commission. Instead, they came to very different conclusions. And yet, it is the central premise of “A Nation at Risk” that has come to dominate the narrative around our schools.

Blog Series: Transformation from Mythology to the New Philosophy 

I chose to introduce this blog series with the juxtaposition of these two reports to illustrate a key point about the education reform battles. There are so many agendas in these battles that it is hard to get to the truth about how our schools are performing. “A Nation at Risk” drew the conclusion that our schools were in decline, but this conclusion was based on oversimplified and cherry-picked data. A better, more nuanced analysis came along in the “Sandia Report”, but it has garnered significantly less attention and has had much less influence on the educational policy front. Test scores and other similar measures of performance change over time, sometimes improving and sometimes declining. The “Sandia Report” provided a number of useful insights that could potentially facilitate sound data-based decision-making. I’m certainly not arguing to abandon this type of approach if it is as free as possible of preconceived notions and political agendas. But, and this is a big but, to achieve this, the decision-making must sit on top of a solid philosophical foundation. This foundation will be the subject of next month’s post.

***

John A. Dues is the Chief Learning Officer for United Schools Network, a nonprofit charter management organization that supports four public charter schools in Columbus, Ohio. Send feedback to jdues@unitedschoolsnetwork.org. 

Notes

[1]. U.S. National Commission on Excellence in Education, A Nation at Risk: the Imperative for Educational Reform: a Report to the Nation and the Secretary of Education, United States Department of Education, Washington, D.C.: Superintendent of Documents, U.S. Government Printing Office distributor, 1983.

[2]. This quote comes from the introductory letter included in “A Nation at Risk” written by David Pierpont Gardner, the chairman of the commission, to Secretary of Education T.H. Bell.

[3]. Various media articles I read referred to the report as the “Sandia Report.” The research article I read for this report has the following citation: C.C. Carson, R.M. Huelskamp, and T.D. Woodall, “Perspectives on Education in America: An Annotated Briefing,” The Journal of Educational Research 86, no. 5 (May-June 1993): 259-265, 267-291, 293-297, 299-307, 309-310.

[4]. Ibid., 259.