Scott Alexander’s extensive review of Freddie DeBoer’s book The Cult of Smart includes this discussion of DeBoer’s belief that American charter schools are fraudulent and only manage their headline-worthy educational outcomes by “cooking the books”:

I think DeBoer would argue he’s not against improving schools. He just thinks all attempts to do it so far have been crooks and liars pillaging the commons, so much so that we need a moratorium on this kind of thing until we can figure out what’s going on. But I’m worried that his arguments against existing school reform are in some cases kind of weak.

DeBoer does make things hard for himself by focusing on two of the most successful charter school experiments. If he’d been a little less honest, he could have passed over these and instead mentioned the many charter schools that fail, or just sort of plod onward doing about as well as public schools do. I think the closest thing to a consensus right now is that most charter schools do about the same as public schools for white/advantaged students, and slightly better than public schools for minority/disadvantaged students. But DeBoer very virtuously thinks it’s important to confront his opponents’ strongest cases, so these are the ones I’ll focus on here.

Success Academy is a chain of New York charter schools with superficially amazing results. They take the worst-off students — “76% of students are less advantaged and 94% are minorities” — and achieve results better than the ritziest schools in the best neighborhoods — it ranked “in the top 1% of New York state schools in math, and in the top 3% for reading” — while spending “as much as $3000 to $4000 less per child per year than their public school counterparts.” Its supporters credit it with showing “what you can accomplish when you are free from the regulations and mindsets that have taken over education, and do things in a different way.”

DeBoer will have none of it. He thinks they’re cooking the books by kicking out lower-performing students in a way public schools can’t do, leaving them with a student body heavily-selected for intelligence. Any remaining advantage is due to “teacher tourism”, where ultra-bright Ivy League grads who want a “taste of the real world” go to teach at private schools for a year or two before going into their permanent career as consultants or something. This would work — many studies show that smarter teachers make students learn more (though this specifically means high-IQ teachers; making teachers get more credentials has no effect). But it doesn’t scale (there are only so many Ivy League grads willing to accept low salaries for a year or two in order to have a fun time teaching children), and it only works in places like New York (Ivy League grads would not go to North Dakota no matter how fun a time they were promised).

I’m not sure I share this perspective. Success Academy isn’t just cooking the books — you would test for that using a randomized trial with intention-to-treat analysis. The one that I found is small-n, short timescale, and a little ambiguous, but I think basically supports the contention that there’s something there beyond selection bias. Teacher tourism might be a factor, but hardly justifies DeBoer’s “charter schools are frauds, shut them down” perspective. Even if Success Academy’s results are 100% because of teacher tourism, they found a way to educate thousands of extremely disadvantaged minority kids to a very high standard at low cost, a way public schools had previously failed to exploit. That’s not “cheating”, it’s something exciting that we should celebrate. If it doesn’t scale, it doesn’t scale, but maybe the same search process that found this particular way can also find other ways? Surely it doesn’t seem like the obvious next step is to ban anyone else from even trying?

And we only have DeBoer’s assumption that all of this is teacher tourism. Success Academy itself claims that they have lots of innovative teaching methods and a different administrative culture. If this explains even 10% of their results, spreading it to other schools would be enough to make the US rocket up the PISA rankings and become an unparalleled educational powerhouse. I’m not claiming to know for sure that this is true, but not even being curious about this seems sort of weird; wanting to ban stuff like Success Academy so nobody can ever study it again doubly so.

DeBoer’s second tough example is New Orleans. Hurricane Katrina destroyed most of their schools, forcing the city to redesign their education system from the ground up. They decided to go a 100% charter school route, and it seemed to be very successful. Unlike Success Academy, this can’t be selection bias (it was every student in the city), and you can’t argue it doesn’t scale (it scaled to an entire city!). But DeBoer writes:

After Hurricane Katrina, the neoliberal powers that be took advantage of a crisis (as they always do) to enforce their agenda. The schools in New Orleans were transformed into a 100% charter system, and reformers were quick to crow about improved test scores, the only metric for success they recognize. Whether these gains stand up to scrutiny is debatable. But even if these results hold, the notion of using New Orleans as a model for other school districts is absurd on its face. When we make policy decisions, we want to isolate variables and compare like with like, to whatever degree possible. The story of New Orleans makes this impossible. Katrina changed everything in the city, where 100,000 of the city’s poorest residents were permanently displaced. The civic architecture of the city was entirely rebuilt. Billions of dollars of public and private money poured in. An army of do-gooders arrived to try to save the city, willing to work for lower wages than they would ordinarily accept. How could these massive overall social changes possibly be replicated elsewhere? And how could we have any faith that adopting the New Orleans schooling system — without the massive civic overhaul — would replicate the supposed advantages?

These are good points, and I would accept them from anyone other than DeBoer, who will go on to say in a few chapters that the solution to our education issues is a Marxist revolution that overthrows capitalism and dispenses with the very concept of economic value. If he’s willing to accept a massive overhaul of everything, that’s failed every time it’s tried, why not accept a much smaller overhaul-of-everything, that’s succeeded at least once? There are plenty of billionaires willing to pour fortunes into reforming various cities — DeBoer will go on to criticize them as deluded do-gooders a few chapters later. If billions of dollars plus a serious commitment to ground-up reform are what we need, let’s just spend billions of dollars and have a serious commitment to ground-up reform! If more hurricanes is what it takes to fix education, I’m willing to do my part by leaving my air conditioner on “high” all the time.