Educational statistics and rankings: what they do, and do not, represent

Dr. John Stockwell: Executive Director, Spartanburg Academic Movement and Member, SC Education Oversight Committee

Dr. John Stockwell: Executive Director, Spartanburg Academic Movement and Member, SC Education Oversight Committee

US News and South Carolina’s Education Ranking

US News and World Report caught the bull by the tail three decades ago when it launched its annual “college rankings” issue. Though the print magazine is defunct, the rankings live on, now including healthcare, crime, infrastructure, opportunity, economy, and pre-K through 12 education. 

Their popularity resides in the reduction of complicated comparisons to simple digits. Few of us have the inclination to dig into their methodology, but we should.

Earlier this month US News issued state-by-state rankings of pre-K – 12 education. South Carolina ranked 48th.  To argue with the ranking sounds defensive.  On the other hand, letting the bull rampage the china shop is irresponsible.

There is no single ranking that can represent with validity the complexity of education. For example, comparing “states” with one another masks the incredible disparities that exist “within states.” In one of South Carolina’s counties, the 4-year graduation rate is 59%.  Spartanburg County’s is 87.2%. It is meaningless to declare that the graduation rate is 73.1%.  Yet, this is the simplistic analytical model upon which much of the rankings are based.

US News relies on proxies to calculate rankings: ACT scores, graduation rates, 8th grade math and reading, pre-K enrollments; and it selects arbitrarily how it will weight these proxies to arrive at rankings.

US News uses the ACT as the chief indicator of college readiness. It fails to mention who takes and doesn’t take the ACT from state to state. Among the five states ranked highest (MA, NJ, NH, CT, and VT), less than one-third of high-schoolers take the ACT, and they self-select to do so. 

South Carolina requires 100% of 11th graders to take the ACT. Why? Because we want a realistic look at post-secondary readiness among all students, not a self-selected few. We want the capacity to advise all students where to turn for post-secondary certification. We want to create the possibility that a surprisingly good score will persuade an uncertain student to go to college.   

Of course, requiring the ACT of all 11th graders “hurts” our statewide average, but it helps us immeasurably learn more about our graduates. Were we to limit testing to a self-selected few, we too could trade away that knowledge for higher rankings. But which is the better practice?

South Carolina requires 24 credits for graduation. New Jersey requires 22 and other states among the so-called top five require 20 or fewer. Their graduation rates are slightly higher than ours, and that simple fact – absent the hours required to graduate – yields higher ratings. Which is the better practice?

Research demonstrates that the return on investment in early childhood education is substantially greater than any other stage of learning. South Carolina’s pre-K quality rating by the National Institute for Early Education Research is 4th in the nation. Is the credit in the rankings equal to the ROI?

State Superintendent Molly Spearman points out that we’re leading the country in our apprenticeship program. Not credited in the rankings.

The “Profile of the South Carolina Graduate” expects schools to address not only world-class knowledge but also world-class skills and personal characteristics. The ambition of the “Profile” is a national best practice. Not in the rankings. 

Finally, what fails completely to get attention in the rankings is the profound impact of household poverty on academic achievement. It is no accident that the top five ranked states have incomes substantially higher than South Carolina’s. Yet Spartanburg County is addressing poverty and academic achievement head-on, witnessing some of our greatest gains among our most disadvantaged populations. Not in the rankings.

Consider interventions across our county impacting academic achievement from cradle to career: declining teenage births, increasing birth weights, Quality Counts child care centers, expanding 3K and 4K in schools, developmental delay interventions, programs to address summer slide, individual graduation planning starting in 8th grade, rich array of AP courses, dual credit opportunities, expanding apprenticeships, increasing graduation rate seven years running. None credited in the rankings.

Why does US News “rank” states in the first place? In the somewhat patronizing words of assistant managing editor Mark Silva, “as many balances of power shift from Washington, D.C., to the states, it’s essential to understand which states are doing best at what matters most to Americans.” Sounds good. Sells well. But it masks some really sloppy social science, and thereby damages reputations.

South Carolina’s teachers, schools, Department of Education, Education Oversight Committee, and we here in Spartanburg County are undertaking the more rigorous science of examining what really works in teaching and learning. And we are looking the difficult challenges of poverty straight in the eye, addressing learning disparities directly.

If we’re going to look the bull straight in the eye, it’s wiser to take it by the horns than by the tail.

John C. Stockwell, Ph.D.

Executive Director, Spartanburg Academic Movement

Member, SC Education Oversight Committee

March 17, 2017