What exactly should we expect of Colorado’s school district leaders? With a title like SUPERintendent, are we expecting too much of what they can accomplish? What difference does it make for what students in a district learn to have an experienced superintendent as opposed to someone new at the helm?

A brand-new Brookings study strongly suggests that it doesn’t make much difference at all. The academic heavyweight team of Russ Whitehurst, Matt Chingos, and Katharine Lindquist surveyed 10 years of data in school districts across Florida and North Carolina, and found that superintendents account for a mere 0.3 percent of differences in student academic achievement.

So are they saying that it makes no difference who serves in a school district’s top position, reporting directly to the locally elected board of education? Are we to believe that it didn’t matter having my one-time educrush Michelle Rhee running D.C. public schools rather than her predecessors? That Mike Miles left no meaningful mark in Harrison? That a cage-busting leader like Dougco’s Liz Fagen is interchangeable with the average large school district superintendent?

Writing at Jay Greene’s blog, Matt Ladner succinctly clarifies what the Brookings report says:

The authors are careful to explain that their research does not suggest that there are no dud or superstar superintendents. It’s just that, on average, superintendents don’t make much of a difference. They liken this to the effect of money managers who on average add no value, although it is possible that some of them are great and some awful. Of course, much or all of that difference between great and awful could be random chance.

According to Brookings, other school district factors are five times more significant than who serves as superintendent. The school has 10 times more effect, and the individual teacher over 13 times as much. It shouldn’t be a great surprise to anyone that factors closest to the student are more meaningful than those further removed. I’m sure you could find even less effect on student achievement from who is Colorado governor or President of the U.S.

Could someone say this report vindicates the decentralized, innovative (and productive) approach in Falcon 49, which made the superintendent position obsolete?

Whitehurst, Chingos, and Lindquist offer in their conclusion:

Superintendents may well have impacts on factors we have not addressed in our study, such as the financial health of the district, parent and student satisfaction, and how efficiently tax dollars are spent. And to be certain, they occupy one of the American school system’s most complex and demanding positions. But our results make clear that, in general, school district superintendents have very little influence on student achievement in the districts in which they serve.

The importance of these other factors cannot be disregarded, and as usual, makes the case for further research. But to the extent that a leader like Jeffco’s Dan McMinimee is implementing policy goals and changes championed by an active, reform-minded board, does that not account for something, too? If nothing else, count this post as an excuse for me to urge you to read the two-part interview series (here and here) McMinimee recently did with Colorado Public Radio.

Maybe the answer is that great school district leadership is just plain hard to quantify. While it’s hard to draw a straight line between the person in the superintendent’s office and how well a 4th-grade student learns, we shouldn’t throw up our hands and say it doesn’t matter at all. But maybe it should give Colorado pause to rethink what K-12 education governance looks like.