What’s in a number? If you’re talking about educational choice, numbers like the ones coming out of Arrupe Jesuit High School can be beacons of hope for tough populations of kids. In other cases, numbers can convey more concerning trends. And in still other cases, numbers can be rigged in such a way that they don’t tell the full story.

I was reminded of the importance of questioning numbers in the education debate yesterday as I read a great Denver Post piece from the Education Commission of the States’ Peter Huidekoper. The piece points out a disconcerting disconnect (alliteration!) between higher graduation numbers, low ACT scores, and ugly remediation rates in some Colorado schools.

For those who don’t know, “remediation rate” refers to the percentage of students who require remedial courses in college before they begin full-fledged coursework. If you’re feeling a bit nerdy and want to dive deeper, the Colorado Department of Higher Education just released the 2014 Legislative Report on Remedial Education, which somewhat paradoxically covers 2013 high school graduates.

(For the record, I believe there are significant limitations when it comes to how Colorado calculates its remediation rate. But that’s a discussion for another time…)

Huidekoper starts out by noting that higher ACT scores tend to correspond with higher graduation rates and lower remediation rates. Makes sense to me. But then he points out something odd: There are cases in which graduation rates are soaring despite low ACT scores. Many of those low-scoring graduates—graduates who are expected to be college and career ready when they receive their diplomas—find themselves in need of remediation when they get to college. 

… it is curious to see Westminster High and Alameda International with ACT scores of 16.3 (or 3.7 points below the state average) and yet, a year later, the graduation rates neared or exceeded the state average of 76.9 percent. No surprise, really, to see that most of those graduates who went on to college that fall required remedial classes.

Strange, too, to see Bruce Randolph with an ACT average of only 16.0 for its juniors and yet, a year later, 91.4 percent of that class earned a diploma. Of the 27 Bruce Randolph graduates enrolled in higher education the next fall, 16 of them (59.3 percent) needed remedial classes.

Also puzzling: South High, with an ACT score of 16.1, had a graduation rate exceeding the state average: 78.1 percent. The remediation rate for that class at South — 64.5 percent — is also telling.

Huidekoper finds the same basic pattern when he examines data from the class of 2014, though remediation data for those students won’t be available until next year.

All of this is problematic. Why? Because it calls into question the value of a diploma in Colorado, as well as the extent to which we should get excited about graduation rate increases alone. At the end of the day, simply increasing the number of kids who receive diplomas doesn’t mean much if those kids aren’t ready to move on to college or a career.

To this little observer, these data also underscore the uneven footing on which many kids find themselves in Colorado schools, with some schools sending kids off to their futures ready and prepared for success, and others simply shuffling students through while falling distressingly short of that mark. I’ll refrain from making (another) argument for educational choice here, but I think you see where I’m going.

Certainly, we want to see higher graduation rates. But as a recent National Public Radio report and related presentation illustrate, there is are right ways to push that rate up, and there are wrong ways to push that rate up. We ought to be wary of simply lowering the bar for students or creating incentives for folks to game their numbers by moving struggling kids off the books. Such approaches may lead to short-term gains (and associated celebrations) on paper, but they can only hurt the affected students in the long run.

So by all means, let’s celebrate the success of our students. But let’s also make sure we’re placing the emphasis on real success instead of numbers that may not tell us the whole story.