Your description of the rise and fall of progressive education reminds of what I had often thought about the 19th century German education system.
Germans organized their education system along two tracks: the “progressive” education for the best students; the “practical” education for the masses. This is, in a sense, how they wound up producing the top scientists, engineers, and military officers, etc. in the early 20th century, among the few chosen “elites” who got the chance to think for themselves, while the rest were, effectively, taught to follow formulas/orders.
This is, in a sense, fundamentally wrong: it is inherently undemocratic, exclusivist, and elitist. It rests on dividing a priori the students into those who “deserve” to learn properly and those who are deserving only to learn by rote. But, at the same time, I’ve come to suspect that most people don’t really care to be vertical learners, ever curious about everything and yearning to learn more. Heck, I used to be one, but somewhere in the past few years, after being burnt out (not a grammatical error) in academic life, I’ve begun to appreciate the social value of being a follower who does not step out of bounds and started wondering if it is necessarily the best idea to get every student to be vertical learners.
That brings us back to the old German education system, and older than that, Plato’s thinking about how societies should be organized, with a few elites empowered to think “creatively,” while the rest are required to follow, with varying degrees. We, as society (in United States, at least), couldn’t really accept the idea of multi-tiered education that segregated between the elites and the plebes from the beginning, I wonder. So we jumped back and forth between the two tiers that made up the German education system: trying the “progressive education” for all exposed how many students can’t (and don’t care to) benefit much from its features, while trying the formulaic education for all exposed how it held back many bright students.
I keep wondering if there is a good way to strike the balance: create a multi-tiered approach where students slot into the track, or combinations of tracks, that fits their needs and wants best, but without the baggage of elitism or exclusivity. This seems doable, in principle, but can it be implemented practically?