| 0:37 | Intro. [Recording date: July 21st, 2026.] Russ Roberts: Today is July 21st, 2026, and my guest is Daisy Christodoulou. Daisy is the Director of Education at No More Marking, a provider of evaluation software for schools. She was last here in March of 2025, talking about education and feedback. Daisy, welcome back to EconTalk. Daisy Christodoulou: Great to be here, Russ. |
| 0:58 | Russ Roberts: Now, our topic for today is the intersection between meritocracy, education, governance, but we're going to go back in time and look at some crazy historical moments in civil service history, and we're going to start with a failed Chinese civil servant exam taker, Hong Xiuquan. Why is he interesting? Daisy Christodoulou: So, Hong Xiuquan is really interesting because he's probably the instigator of the worst civil war--maybe the worst war--in history. It's not very well known in the West. And in fact, one of the leading American historians of this war, this civil war, has said that he didn't even read about it till he was an undergraduate. So, it's remarkably kind of not very well known in the world, given how significant it is and how major a factor it is in Chinese history. And, Hong Xiuquan, he's basically a failed civil service candidate. And, this all takes place in the mid-19th century in China. And at that time, China has these very rigorous exams to become a civil servant. So, if you want to become a civil servant in 19th century China, you sit for these very, very tough exams. They have multiple stages. So, you will sit some that are just close to your village; you will sit some, you'll go to the provincial level. And, by the mid-19th century, these exams have been around for centuries. And, China's population has been increasing over that time, but the number of openings--number of job openings available for the exam takers--hasn't gone up. So, it's very intense competition, and you have all of these incredibly bright young Chinese men studying really hard to try and pass these exams. And, they're tough exams, and you have to go away for the provincial ones, and they're three days, and you're sitting in cells working away at these exams. And, if you fail them, you can take them again. And, they don't happen every year; they happen every couple of years. And, what you have is--you might see some parallels with today--you have students who are maybe the brightest student in their village, maybe even the brightest student in their province, but they get to the stage where they're competing against all the brightest students from all over China, not very many jobs to have. So, you can be very bright and have spent years studying and still fail, and not get the job. And, you have people who take them, carry on taking them into their 40s, 50s, 60s--keep going back and taking them. And, Hong Xiuquan takes them quite a few times, these exams, and he fails them. The fourth time he fails them, he's absolutely devastated. And, he has what we call now a kind of mental breakdown, and he goes back home to his village, and he's seeing visions. And, in the previous times when he's taken these exams, he's also come into contact with some Christian missionaries, and he's picked up a few of their tracts. And so, after this final failure, he's got these tracts on his mind, and he sees these visions that tell him he's the younger brother of Christ. And he's inspired to start some kind of Millenarian kind of revolution. And, his first few followers are people like him--his cousin and some friends--people who have also taken these exams and not succeeded. And, he basically moves across the countryside kind of picking up an army, and the army grows and grows. And, as I say, he becomes the leader of what is known now as the Taiping Rebellion, which, as I say, is one of the worst civil wars in history. Maybe around 30 million people die. Eventually, it's put down and he's killed, but not before enormous kind of death, destruction, suffering. And, I think a proximate cause of this terrible war is the frustrations, the pent-up frustrations of these young men who are trying to take these exams and failing, and not getting the jobs that they hope. But, I also think you can look, in a way, at what's the ultimate cause of this revolution? Because the exams that Hong Xiuquan is taking, the exams he's taking, they are full of the ancient Chinese classics. And, by the time he comes to take them, they're quite stereotyped, and they're quite rigid. And, there's a particular format called the eight-legged essay, where you have to write an essay that is set out in a really precise and specific way. There's nothing on them about the kind of modern Western knowledge of scientific and maths that's transforming the rest of the world. So, you can make another argument that certainly the students who failed these exams, it's not great for them, but for the students who passed them and become the bureaucrats who are running China, they're really lacking in the kind of knowledge that is transforming the rest of the world. So, you can make a case that these exams play a very important proximate and ultimate role in one of the worst civil wars in history. |
| 6:12 | Russ Roberts: So, we could spend an hour on the 19th century Chinese civil service and how the entry to it was controlled, but we are not going to. We're going to divide it between the Chinese civil service in the 19th century and the British civil service in the--no, actually, we are going to touch on the latter, but we'll move on to other things. But, it's a fascinating story. It raises a question which I don't think you talked about in a preliminary essay you shared with me, which is: Why were people so eager to be a civil servant in China in 1850, say? And, it raises a general question, which we're going to spend some time on, which is: The prestige and pay and honor that's associated with the civil service differs wildly by culture and country, and the significance of the civil service, the bureaucracy, I think, varies quite a bit. So, let's start with China: Why were people so eager to have these jobs? Daisy Christodoulou: So look, I don't want to over-claim and pretend to be an expert in China. My entry into this has been through exams, and my specialism is assessment and exams. But, I think, obviously, when you just see the sheer numbers of young boys--and it is boys who are studying for these exams--you can read some of the literature that the failed candidates leave behind. For whatever reason, it is hugely prestigious. I think it's partly there aren't many other jobs, or outlets, for if you have academic tendencies and you enjoy reading and you enjoy studying, so there's not much else that, necessarily, you can do that will give you a living. And, in fact, when these candidates fail, a lot of the time the only option for them is to go back to their village and become a teacher and to try and teach the next students who are coming through, and see if that can get them to succeed. So, there aren't necessarily many other things you can do. And, I think, yeah, the prestige and the power--the civil service in any society is powerful. That they do have powers. And, for young men from lower social backgrounds, again, in any society, you would say it's one of the few ways of advancement and the few ways to develop yourself. |
| 8:46 | Russ Roberts: You said something kind of shocking to me a minute ago. You said something about how the bureaucracy runs the country. Now, the bureaucracy often thinks they run the country, and people are often under the illusion that even a dictator runs the country, when a dictator can't run the country either. But certainly, in modern democratic states, the bureaucracy doesn't run the country, although they may run parts of it in a more serious way than the legislature runs the country, or the so-called president, or whoever is the leader, the prime minister. But, I want to go back a sec and just talk about your specialty, which is exams. Because, on the surface, if you think it's important that the bureaucrats who are, quote, "running the country" are skilled and capable, an exam has an appeal--because otherwise, it's who you know, and so on. And, what's remarkable about the Chinese exam--and will prove remarkable about the British case when we turn to it--is that the content of these exams is not what you or I would design to make sure that the best ended up on top. So, talk about what happened in England around this time and the reforms that were made, and how peculiar they were, and strange. Daisy Christodoulou: Absolutely. So, by really a kind of historical coincidence, a historical irony, as this terrible civil war is breaking out in China, and almost literally hours of news of it is making its way back to Britain, Britain is engaged in a big national debate about how it should recruit and select its own civil servants. And obviously, Britain, at this time, has an empire; so, in particular, it has India. Now, India, at this point, is in flux as well, but essentially, Britain is either directly or indirectly ruling vast swaths of the Indian subcontinent. And, also, you have a lot of changes going on at home, and you have a lot of growth in the home civil service, as you might call it. So, there's an increasing debate about what's the best way to pick all of these administrators and to recruit them. And as you say, one problem you get when it comes to recruiting bureaucrats and recruiting administrators who are going to run the show, whether you think they're really running the show or not, I think that it's fair to say they have at least some opportunities for power and influence, even if they're not necessarily always running things in the way they might think. The risk is that you get people just appointing on for patronage, basis of nepotism, people giving jobs to their mates' nephew. So that's a risk. And, if you want people who are competent to do the job, as you say, one thing you can do is say, 'We'll have an exam.' But, it really matters what is on the exam. And, on the Chinese civil service exam at that time, as I say, is really a lot of ancient classical Chinese knowledge and languages, and this eight-legged essay. And Britain are kicking around ideas and saying, 'Well, what goes on? If we're going to have an exam, what should go on the exam?' And, there have been a few kind of experiments prior to this time, and there are a few places where exams are starting to be used. But there is this major point in 1854 where a major Report is written that proposes there should be one central exam for applicants to the civil service--the British Civil Service--and that this exam basically should consist of a lot of knowledge of the classical languages of Greek and Latin. And, these are very prestigious at the time in Britain. They're taught in a lot of the public schools, the great public schools. They're taught at the famous ancient universities, Oxford and Cambridge. And, you have two administrators of the time--a guy called Stafford Northcote, another man called Charles Trevelyan--who write a Report that essentially proposes that these curriculums, with a heavy emphasis on ancient Greek and Latin, should become the basis for the exam that selects for civil servants. Russ Roberts: And, the Report basically gets implemented, and this is the-- Daisy Christodoulou: Yeah, pretty much. Yeah. So, the Report is commissioned by William Gladstone, who is obviously a quite famous Prime Minister eventually. So, it's commissioned by Gladstone. It's written in 1854. It is just a Report, so it isn't immediately an Act of Parliament. But gradually, over the next decade or so, most of its recommendations do become law and are implemented. And, to the extent today that it is now regarded, even in Britain today, as the founding document of the British Civil Service. So, a very senior official currently in the British Civil Service--actually, I think he lost his job a couple of months ago--but a very senior figure, Ollie Robbins, has said very recently: 'Northcote-Trevelyan,' the Northcote-Trevelyan Report, as it's known, 'is the founding document of the Modern Civil Service.' So, it's been hugely influential, and it does come in for its fair share of discussion. And, it's become something that's been discussed a little bit more lately, as I think people wonder about modern Britain, and modern Britain civil service, and whether modern Britain civil service is as effective as it should be. And, if some of the problems of that do go back all the way to this Northcote-Trevelyan Report. |
| 14:38 | Russ Roberts: So, I want to set this next part up a little bit with some background and some things to think about for listeners, and for you and me, Daisy. Because, you could take this conversation so far to be about some obscure exam in 19th-century England, and who cares about Northcote-Trevelyan? But, here's what's important, and I want to bring in a couple other things. America is pretty disdainful of its bureaucratic system. Americans do not look at, quote, "bureaucrats"--the polite term is 'administrators'; the more negative term is 'bureaucrats'--and, say, Health and Human Services [HHS] or various--Department of Education--they don't have a lot of prestige. And a lot of people would say they don't do a very good job. At the other end, for both the United Kingdom and the United States, you have the military, and the standards and excellence is totally different. One might suppose that's because one is a matter of life and death and one isn't. So, indulging in hiring students who are good at Latin and Greek, you could say, 'Well, who cares if the Treasury Department has a lot of those? It's not so important.' But, when it comes to military failures, you want them to be able to know what they're doing, so you'd want a different test for the Royal Navy or the U.S. Air Force, or the Marines, or the U.S. Army. Now, before we get to--I just want to put that on the table in terms of the importance of this, but I want to also talk about this reform. Because, as you point out--and this is extremely interesting to me--basically, they took this prestigious gate of getting into the civil service, and the gatekeeper became, 'Are you really good at what we teach at the two most prestigious universities in the land?' And, it's hard not to view this as a public choice story--meaning the field in economics that assumes policy decisions are made often driven by self-interest and not the national good, or the public good. So, basically, they basically circled the wagons--Northcote and Trevelyan. Forget what they said about it: what they actually did in that report was encourage the British Civil Service to keep hiring people like themselves, people who've gone to Oxford and Cambridge, who were really good at Latin and Greek. And, as you point out in your essay--which is hilarious--people at the time said, 'You know, it really doesn't matter what they study, or what the criteria are, or what the subject matter of the test is. It just has to be hard.' And that will get the people who are good at hard things, and they'll be skilled. They'll be the most skilled people because they could do the hardest things. Even if it's nonsense. Thomas Macaulay, the great historian, says, 'Yeah, if they studied astrology, or alchemy, or the Ptolemaic system of astronomy, of the solar system instead of Newton, they're hard to master, and that's what counts.' So, I've thrown a lot on the table here. Take a crack at reacting to that. Daisy Christodoulou: Yeah, for sure. So, I basically agree with what you've said, your view of Northcote-Trevelyan. But what I will point out to you--so, what you've said, this is a public choice theory. This is basically a way of getting people--almost like a self-perpetuating oligarchy--of getting people who look like you, and sound like you, and are like you, into positions of power and influence, and keeping that ticking over. So, I would agree with you on that. And, as you say, I've got this essay where I kind of make that argument. However, what I will say is that is not the dominant historical opinion of Northcote-Trevelyan. So, as I say, it does have a little bit of fame here in the United Kingdom. It's something people kind of think about and talk about. The dominant, I would say, take on Northcote-Trevelyan is that it's meritocratic and it's a good thing. Russ Roberts: Right. Daisy Christodoulou: And, I would say it's only in the last decade or so that there's been a little bit more revisionist scholarship on that. And, the book that I draw on, that I think makes the revisionist argument really well, is a book by a guy called Michael Coolican, called No Tradesmen and No Women. And it's a history of the British Civil Service. And, he takes this quite revisionist stance, where he says, 'Everyone thinks Northcote-Trevelyan is about meritocracy because it introduces exams.' He said, 'But, it isn't about meritocracy.' He has a line where he says, 'Okay, it probably did reduce the opportunities for patronage of the really explicit corrupt type, where you pay someone to give your underperforming nephew a job.' So, it probably does reduce that; but he says it eliminated patronage, but it also eliminated professionalism. Because, actually, prior to Northcote-Trevelyan, the system is starting to evolve in a lot of important ways to be using exams already, but to be using exams that are relevant to the job you'll be doing. So, I would say when I was growing up, the dominant take on Northcote-Trevelyan was, 'Oh, before Northcote-Trevelyan, everything was super-corrupt. People paid for jobs. Northcote-Trevelyan came in and professionalized everything.' What I think this guy Michael Coolican is arguing, and what I would agree with, and what I think you were saying too, is actually what was happening is before Northcote-Trevelyan, there definitely was some corruption, but there were also a lot of moves to modernization and professionalism. There were civil service departments that were using exams that were useful for the specific job that the candidate would be doing. And actually, Northcote-Trevelyan came in and said, 'We're going to replace all that with one central exam that's going to be based on topics that really have no relevance to the job in hand.' So, it is a revisionist take. It is a controversial one. And there are people, I think, who have grown up with that standard take on Northcote-Trevelyan who don't like that revisionism, so it is controversial. But, I think this book by Michael Coolican does a fantastic job of drawing out a lot of the debates. And in particular, he draws out a lot of the importance of a guy who is the Master of Balliol College at this time, a very important figure behind the scenes. He's called Benjamin Jowett, and he is incredibly keen to be getting his classics graduates from Balliol College and Oxford into positions of influence all over the British establishment. And, he's very successful, and he is kind of a major figure in the background of Northcote-Trevelyan and subsequent developments. So, I would agree with you, with how you put it there, but as I say, I would say this is a relatively controversial take, even for him. |
| 21:29 | Russ Roberts: So, talk about the military. And, again, I think we can think about bureaucrats who run, say, a welfare program, or a transfer program that distributes money in certain ways, and it could be run better or worse, obviously. But, a lot of times what's at stake in these conversations--and you and I recently had a chance to talk about this a little bit in England--is basic, fundamental infrastructure. And, so, for example, when I was in England recently, a couple trains I was scheduled to take were just canceled out of the blue, with no real--a half an hour before I get on, just canceled, signal issues, I don't know. I don't know if that's a fair criticism that means the system is not run so well. You evoked pride for the Elizabeth Line, which I took without any problems while I was in London, and I do like it a lot. And again, we might think about the military, where, 'Oh, well, that plane didn't work,' or 'This aircraft carrier sunk.' It would be unacceptable. No one would take it for a minute, and the people who failed there would be fired. That seems to never happen with transportation--infrastructure of that kind. There seem to be very little consequences. Which makes you wonder whether the people who are in the jobs have the skills necessary to do it. So, talk a little bit about your take on the military and transportation. And, then you can bring in Samuel Pepys, because if we're going to delve into strange moments in British history, we can mention him, although we will not discuss his X-rated diary, but those who are interested can pick it up on their Kindle. Daisy Christodoulou: So, Russ, I think at the heart of this is a classic EconTalk theme, which we chatted about when I was on an episode before, of feedback loops and price mechanisms. So, the major problem I think you have with bureaucracies is often there isn't a price mechanism; you don't have those feedback loops. And actually, when it comes to the military, I know what you're saying--that obviously, the military, in lots of ways, there's more competence there. But I think you've also got to be aware throughout history, and in every single country you can think of, there's a history of military mess-ups, too. Russ Roberts: Oh, sure. Daisy Christodoulou: Like, military bureaucrats don't always kind of get it right. And I think what you have in bureaucracies is you don't have a price mechanism. Often, there's a reason for that. So, again, I don't want to get too off-topic going into this, but obviously, with the military, you understand why the state has to have control of that, and you don't necessarily want to be outsourcing everything to countries that are your enemy, whatever. So, in a lot of proxies[?processes?], you don't have that price signal, you don't have the feedback loops, and things can go awry. And I would say--actually, it's a theme of British military history--that often, after a long period of peace, the first year of wars go really badly and a lot of things go wrong. And then you have a different feedback loop which asserts itself, which is lots of people dying on the battlefield. And, that is almost a replacement for the price mechanism. Russ Roberts: Absolutely. Daisy Christodoulou: And, in a lot of ways, it is a more brutal master than the price mechanism-- Russ Roberts: Absolutely-- Daisy Christodoulou: So, the first year of World War I, not just Britain, but every participant country, just have all kinds of crises, where the peacetime military bureaucracy and the civilian bureaucracy was not up to the task of total war. And so, in Britain, we have a famous thing: we have a munitions crisis. There just aren't enough shells. And, this is something that you have to get a mix, let's say, of civilian bureaucrats, military bureaucrats. You have a change in the top of government to handle this issue. So, I think you have issues across the board, all kinds of bureaucracies, that when there isn't a feedback mechanism, things can go wrong, and then you have feedback mechanisms assert themselves, and people have to scramble to keep up and fix things. And so, if we bring this back to the kind of issue of then how do you select, recruit, and not just actually even select and recruit, but what do you do once you've recruited your bureaucrats? How do you check they're doing the right thing and are keeping the military ready for the eventual war, or keeping whatever system they have got running? Obviously, you have to be thinking about--actually, exams do form a major part of this. And, again, it might seem like exams are a really modern technology, but they're not. So, as I say, in China, they go back centuries. And, the reason why exams matter is because if you do want a way of selecting for competent people who are going to be able to run these organizations well, an exam is not a bad option. And again, we tend to think of exams--in the United Kingdom, people talk about Northcote-Trevelyan in 1854 as being a landmark, and people also talk about how exams for the army don't come in until the late-19th century. So, there's a set of reforms in the late-19th century, after Northcote-Trevelyan, called the Wolseley reforms, which bring in more meritocratic examinations. And I don't know, I feel like for those of you who are keen on Gilbert and Sullivan--I think Gilbert and Sullivan are quite popular in the United States. Russ Roberts: I'm a big fan-- Daisy Christodoulou: There's that song, 'He is the very model of a modern major general'-- Russ Roberts: Right-- Daisy Christodoulou: that lists all the things that a modern major general has to know. And, that's a satire--so it's late-19th century Gilbert and Sullivan--that's a satire on the Wolseley reforms. So everyone--they're, again, quite famous, and people will look at those and go, 'Oh, so the army bring in kind of exams in the late-19th century.' And, the Napoleonic Era--so the era before that--officers would buy commissions. So, if you wanted to rise in the army, you actually had to be quite wealthy, and you purchased commissions. And that persisted in the army until the late-19th century. So, people often say, 'Oh, look, the military has this very kind of un-meritocratic and patronage-based system right up to the 19th century.' But, that is not the whole story. The army, for sure, has quite a patronage-based system up to the late 19th century. But, the major service in Britain is not the army. The major branch of the military in Britain at this time is the Navy. It's the Royal Navy. France is the great land power. Britain, England, is the great sea power. And actually, for that really important service, which lays the foundation of so much of British power, they had exams for much, much longer. And, the guy who introduced exams into the Navy way back in 1677--so almost, I think, over 200 years before they introduced the army--and the man who introduces them is a bureaucrat, and he's the famous--he's more famous now for his X-rated diaries--Samuel Pepys. So, Samuel Pepys, very famous now, not just for slightly naughty diaries, but also diaries that he wrote at the time of some great upheaval in London, so he gives us these really vivid firsthand accounts of the Great Fire of London. And, I grew up a 10-minute walk from where Pepys lived and worked, and I actually have an office next door to where his offices were, so I'm very interested in Pepys. He's a completely fascinating figure. And, as I say, he is better known now for these really compelling and vivid firsthand accounts of the Great Fire of London, and for, as I say, some quite X-rated bits in his diaries. But, as a figure at that time, he is incredibly important. And in 1677, after a series of quite bad naval defeats to the Dutch, Pepys introduces oral exams for Royal Navy lieutenants because he is concerned about the fact that lieutenants are often well-bred gentlemen with good connections, but they don't really know much about running a ship. And actually running a ship is really quite hard. And, at that point, there's a lot of naval innovation going on, and it's not something you can really just bluff your way through. So, he introduces their oral examinations, and it's a panel of three senior officers who will question the candidate very closely on the practicalities of running a ship. And then, what you also get come in later, over the next few decades, century, whatever, is you also get quite a lot of written examinations come in. So, if you look at some of the most famous characters, not actually just in British naval history, but arguably in all British history, people like Horatio Nelson, James Cook, people like Flinders, Vancouver, Bligh, people who go around the world and chart maps of so many parts of the world, a lot of them have taken exams that were either the ones that, really, Pepys kind of designed or the ones that built on that foundation. So, that kind of exam--that really practical examination in cartography, in trigonometry, which you do need to be able to navigate in charting and map making--those really practical examinations are there from really quite an early stage in Britain, and play a really central part in the development of one of its most powerful and competent institutions, the Royal Navy. |
| 30:45 | Russ Roberts: Let's go back to the general idea of an exam, and in the case of the Royal Navy, navigation, certain basic math skills. Of course, a lot of it's going to be how to deal with people, which is very hard to have a test for. You can get recommendations from people who have seen you in action doing similar processes. But, one of the big questions, I think, in education today is entry exams into college. A lot of American colleges dropped the SAT [Scholastic Aptitude Test] exam as a gatekeeper, feeling it was either unfair, racist, or ineffective. A lot of people disagreed with that. A lot of schools have brought that back. But, I want you to talk about this general question of transferable skills. And, I made fun of Macaulay earlier, but I want to defend him for a minute and get your reaction. So, Macaulay argues that it doesn't matter what you put on the civil service exam: it just has to be hard. And, of course, now we would call this a signaling technique. 'We're going to force people to jump through a bunch of hoops, and it almost doesn't matter,' says Macaulay, 'what the hoops are because what we really want are people are good at jumping.' And, since it's prestigious, it's going to attract the smart people, and the smartest ones will figure it out. The content is not so important. What's important is making sure the cream rises to the top. The best people do the jumping, and that's the case. And, you make the point undeniable that, well, wouldn't you want them to learn something that was true? And, you can elaborate on that, but I want to defend Macaulay for a minute, suggesting that--I'm not totally comfortable with it, but I'll try--that if you give a really hard exam about something that's really obscure--Sanskrit grammar or Thucydides, which Northcote himself said was part of the reason he was successful--he had read Thucydides in Greek, and that helped him build up his brain, and he got smarter. It turns out Thucydides, by the way, a lot of what Thucydides wrote probably didn't happen. He exaggerated; he wasn't exactly accurate, 100%. So, that makes the argument: Through this difficult parsing of this hard text in a difficult language, you find out who the smartest people are, and that's all you really need. What's wrong with that argument? Daisy Christodoulou: So, we can view this as a debate for British history between Pepys and Macaulay: that Pepys said, 'We need an exam, and the exam needs to be in stuff that's useful.' And Macaulay said, 'We need an exam, but it doesn't matter what the exam's on; we just want to find the super smart people, and then they can turn their hand to anything.' And, I actually just really want to actually read out what Macaulay says, because people might think we're creating a straw man here-- Russ Roberts: Fair enough-- Daisy Christodoulou: when we're saying this. And, actually, I just want to point out that Macaulay made--he, himself, made--something that sounds like a really extreme straw man argument. He made that argument, and every time I read it, I am just astonished. I cannot believe it. Macaulay is a big deal, and he's a big guy, and he's very famous. Russ Roberts: Great historian. Daisy Christodoulou: Yeah. And, he's an MP [Member of Parliament]. He's giving this speech that I'm about to quote from in Parliament. After he gives this speech, another MP jumps up and says, 'This speech has never been exceeded within these walls for the development of statesman-like policy and practical good sense.' And, actually, the nice thing about this is, people are a bit down on our politicians now. I read this speech and the responses it gets, and I think, 'Oh, people must have been crazy,' because this speech has no good sense to my mind. This speech is mad. Russ Roberts: Before you read it, I want to let listeners know--I think I know what you're going to read, which is: there's a term in here called a 'wrangler.' And, a wrangler is a person who is good at wrangling, meaning not animals, which is the American use of the term, but debating back and forth some topic. And, there's also a term in here, the 'wooden spoon.' And, the wooden spoon, according to Claude--I looked it up and you can verify it--is an award given to the worst person who passes. So, you know, the joke is: What do you call the person who finishes last in their medical school class? And the answer is: Doctor. But, it turned out in England, you got the wooden spoon. It means you just got across the line. Daisy Christodoulou: And, we still use wooden spoon in sports now, people finishing bottom of something. So, it's your term in English, idiom. Okay, so here we go. This is Macaulay. This is Macaulay's take on exams. This is 1833: Whatever be the languages, whatever be the sciences, which it is, in any age or country, the fashion to teach, the persons who become the greatest proficients in those languages and those sciences will generally be the flower of the youth, the most acute, the most industrious, the most ambitious of honourable distinctions. If the Ptolemaic system were taught at Cambridge instead of the Newtonian, the senior wrangler would nevertheless be in general a superior man to the wooden spoon. If, instead of learning Greek, we learned the Cherokee, the man who understood the Cherokee best, who made the most correct and melodious Cherokee verses, who comprehended most accurately the effect of the Cherokee particles, would generally be a superior man to him who was destitute of these accomplishments. If astrology were taught at our Universities, the young man who cast nativities best would generally turn out a superior man. If alchymy were taught, the young man who showed most activity in the pursuit of the philosopher's stone would generally turn out a superior man. Okay, that's Macaulay done. Now we're back to me. This is just Macaulay getting carried away with his own eloquence. He is literally saying, 'I would rather people study things that are completely wrong, that we know to be completely wrong, like the philosopher's stone,' and that's fine, because if they study those and they do well at it, and they cast nativities the best, then they'll be the smartest guys. They'll be the hardest working, most industrious, so that's fine. Those are the people we want. What he seems to be completely missing here is that the stuff you study really matters. And, that's why I think--go back to where we opened when we're talking about China in the 19th century. If you take the most industrious and hardest working, and most ambitious youths of your country, and in the most productive years of their life, teach them things that are wrong, or just not that useful, what is going to happen is there will be other countries who take the same tranche of their students and teach them things that are true and useful. And then where are you going to be? Which country do you think is going to come out best? So, I really just find it astonishing for Macaulay. I think it's just unbelievably misguided. |
| 37:47 | Russ Roberts: I want to defend him. Daisy Christodoulou: Go on. Russ Roberts: And, I can't resist the urge--well, I am going to resist that urge. I'll tell you later off air. But, I'm going to defend Macaulay. Daisy Christodoulou: Go ahead. Go on. Russ Roberts: Because behind Macaulay's view--so, you're making one point, which is that if you want to incentivize people to master something, you should incentivize them to master something valuable, especially if it might take years, because they could then take that true knowledge, rather than just having jumped through the hoops, and signal that they're the smartest: they actually would have added--besides their raw intelligence--they would have added content that would make them more effective in their job. So, I agree with you 100%. The real question for me--and this is a modern education question, which I know you have views on: Isn't the real question, as you describe it, I think, in your essay, turning the brain into this razor-sharp tool? And, it doesn't matter whether you study, to take a famous example in Jewish life, the Talmud, which is notoriously difficult. So, you study the Talmud, and your brain gets more skilled. Okay, you may not agree with what's in the Talmud; you may think it's nonsense. It's just a myth or superstition. But the act of mastering it makes you--many people would say in Jewish life--it'll make you a better lawyer. Because you've studied all these debates back and forth and these cases. And although the cases may not be relevant, because they're talking about something that doesn't exist anymore in Jewish life, all that really matters is that you've honed your reasoning ability. And that's what education is about. It's about turning your brain into a powerful tool that you could then--or a search light, you could go across the landscape and turn it on to any topic and illuminate it--because you've got this great brain. What's wrong with that argument? Daisy Christodoulou: Yeah. So, what we are kicking around now, this argument we're having, people obviously were having the argument in the 19th century. So, Macaulay did not have it all his own way in the 19th century. People were saying things like I'm saying, at the time. And, by the time you get to the late-19th, early-20th century, the people arguing back against the kind of 'just study hard stuff and it will make your brain sharp,' there are people who are arguing back against this. And one of the things that--and so, then they developed, basically, some empirical evidence on this question, which we can take a look at now. And, actually, one of the most important aspects of this debate is that in the late-19th, early-20th century, medicine is really developing as a science. And, a lot of people are starting--medicine is becoming more of a career and more of a career pathway. And yet, a lot of top medical schools in the United States and the United Kingdom, and this is very much--at this point, a big U.S. debate, too--a lot of people who want to become doctors are saying, 'Well, actually, why do I have to qualify in Greek to get into medical school?' Or, 'Why do I have to qualify in Greek to become a doctor?' And actually, 'Is this not a bit of a waste of my time, given that medical knowledge is expanding at such a rate that maybe my time would be better off spent doing that instead?' And, so, I think one of the best statements of the anti-Macaulay argument comes from an American medical educator, who was extraordinarily influential in early-20th century America. His name's Abraham Flexner. And, he writes a fantastic article in The Atlantic magazine in 1917, and he takes on a lot of these arguments from the point of view of medical training. And, one of the things he says, which I think really counters that Macaulay argument, is he has this great line where he says, 'I can compare the champions of mental discipline'--so that's the Macaulay type people who think that it doesn't matter what you study, it just matters that you build discipline. He says, 'I compare the champions of mental discipline to Egyptian priests who planted rows of dead sticks, which for disciplinary purposes, they watered regularly. Had they planted corn, they would have got the same discipline and something more, the corn, for example.' So, what he's saying is you've got all these aspiring medics in early-20th century America, and you're saying to lots of them, 'You need to be studying Greek and Latin before you can look at a medical textbook.' And, he's saying medicine is hard enough. Medicine is tough enough in itself. Instead of getting our kids to study Greek and Latin because that is hard stuff that trains their mind, we could just get them to study the hard stuff of medicine that is useful. So, he is a really important figure. And, actually, I would say in practical terms, kind of carries the day in terms of being very influential in the establishment of medical schools in America. And the other crucial figure who lays down a lot of empirical evidence on this point is another American, E.L. Thorndike, who is a pioneer of experimental psychology. And, again, Thorndike is listening to these arguments and sort of thinking, 'Well, can we bring any data to bear? Can we do some research?' And, he lays the foundation for a lot of later experimental psychology, and he does these very pioneering studies on whether skills do transfer across domains. So essentially, the Macaulay argument, if you like: it doesn't matter what you study; you just need to hone your brain to a razor-point sharpness. And, in Macaulay's idea, the hardest thing you can study is ancient Greek. So, if you can translate the Melian Dialogue, you can basically do anything. It's not just ancient Greek. The hardest bit of ancient Greek is Thucydides. So people are like, 'Once you can translate Thucydides, you can do anything.' And, what Thorndike is saying is: 'Well, let's test this.' He comes up with a couple of ways of testing it. So, one of the things is he looks at the kind of students who are doing the classical languages at school, and he says, 'Well, do they do better on all their other subjects?' So, that's just a kind of more of a statistical survey. And, he basically finds, once you control for prior attainment, they don't. The second type of experiments we[?] do, which again have been very influential, is he trains people up in doing one skill really well, and then he gives them another task to do that's similar but different. And he sees, 'Well, does it transfer across?' So, again, a simple example, but he gives the experimental subjects practice in estimating the shapes of rectangles, and they get a lot of practice at that, and they get better at that. And then, he says, 'Does that make you better at estimating the shapes of triangles?' Russ Roberts: You mean the area? Daisy Christodoulou: YeH, I think the area, estimating the area. So, he gives experimental subjects practice estimating the area of rectangles. And they get better at that, and they have the practice, and they improve. And then, he says, 'Will that transfer over to improvements in estimating the area of triangles?' And, he finds, no, it doesn't. And, even on that very simple, quite close allied skill, it doesn't. And again, I think if we go back to Flexner, are we really saying that if you can translate the Melian Dialogue, that is going to make you a really fantastic surgeon? Like, really? So, where your Talmud point--where you say, 'Oh, doing the Talmud makes you a better lawyer'--this is where you get some interesting things about transfer, which in the modern literature will be called far transfer and near transfer. So, near transfer would be when you're transferring--and Thorndike comes up with a term--he talks about where there are identical elements. So, where there are identical elements in the skill you've studied and the thing you are transferring to, then you can have some transfer take place. But, when there is not really that much in common--and that's in the modern literature you would say far transfer--it just kind of breaks down. So, I can see that, obviously, the Talmud, and indeed ancient Greek, there are obviously some near-transfer things that you're going to take from that that would be useful in everyday life. But are you going to be able to far-transfer to lots of other things? I don't think so. So, if it's really important to you to have trained doctors and trained medics, and people who understand medical science, and you want to train and design a training program for them, or in Pepys's example, if it's really important for you to have trained navigators on board a ship, then you should be coming up with a training program that is quite closely allied to the final skill that it is you want to see, and you should be trying to make sure there are ways that there is that identical elements, and that where there is transfer needed, it's a near transfer, and you can test that, and you can see that it's happening. |
| 46:33 | Russ Roberts: So, deep down, I agree with a lot of that, but I want to push back on it a little bit. Which is: I've been fortunate to meet some very high-IQ people in my life, and some of them are only good at one thing, but a lot of them are good at a lot of things. Now, I think the relevant point would be: Well, they're not just good at those things; they know a lot about them; and those are two very different things. They have a base knowledge of context and raw fact and information that's crucial for allowing that well-honed, razor-sharp brain to do something. And, I think when we think about--now, let's move away from civil service and Royal Navy. When we think about what is the purpose of education--let's start with K-through-12 or college--I like to believe that what I do here at Shalem College is, quote, "Help my students learn how to think." Now, you could make fun of that and say, 'Well, but that's an amorphous skill that doesn't transfer to lots of things.' And yet, I think when our students deal with, say, the Iliad or the Odyssey, or Plato or Aristotle, they don't read them in the original languages. They usually read them in translation, but they've learned some, actually, some of the Greek. I think they get good at understanding things, and they get good at reading, and they get good at raising questions, interpreting text. And those are useful skills, and they do transfer to other things. Maybe they're limited. But, I'm curious, given your Thorndike-ian pessimism about transferability, if that's the case, should there be anything other than trade schools that teach people expertise in narrow areas where expertise is crucial? Daisy Christodoulou: No. So, look, I definitely don't want to say that. I'm definitely not making the argument [inaudible 00:48:40]. Russ Roberts: I know you don't. I'm just pushing you into a corner there. Daisy Christodoulou: You're right, you're right. And, I would agree, I think, with a lot of what you're doing at Shalem College. I think the point I would make is that if you are setting out with a goal to get students to think well, I think that's a legitimate goal, but you have to acknowledge that being able to think well depends on domain knowledge. It depends on specific knowledge of a particular domain. So, for example, literature, I do think literature is a completely legitimate topic of study. And, what I would say is if you are studying ancient literature, either in translation or in the original language, if you read a lot of literary texts, you will come to one that you have never read before, that's unseen, and you can have a really good go at understanding it even if you haven't seen it before, because you've got the kind of near transfer from all of the other knowledge of literary texts. And I think that kind of thing is important. So, what I would say is, is that whatever it is we are trying to teach--whether it is literature, whether it is navigation, whether it is good administration, whether it is how to build munition shells, whatever the thing is we want people to have skills in--what we have to understand is those skills are domain-specific. And I think what has been neglected in a lot of modern education is the domain specificity of skills. And in modern education--so if we complete the kind of historical tour of where this is going, in the late-19th, early-20th century, the argument for knowledge doesn't matter, and it's just to kind of create a razor-sharp brain. That school of thought is known as the faculty psychology, or mental discipline. And, it's known as that in both the United States and the United Kingdom, and it's influential. And then you have people, as I say, like Thorndike and Flexner come along and say, 'Hang on a minute.' And, I think what you've had since then, in the sort of postwar era and coming up to today, is a lot more research on what experts, what kind of training they do and what kind of skills they have. And, again, you've had a lot more research on how even moving slightly outside your domain can really challenge you, and your skills can break down. And some of the most famous research was done with chess players, by Herbert Simon, who is, again, another American, Nobel laureate in economics. He did some really famous research on chess players, where he showed that if you show them a chess board with a bunch of pieces for five seconds, take it away, if those pieces are drawn from an actual game--like a genuine game scenario--their memory is insanely good. They look at it a couple of seconds--bang--they've got it. They can reproduce it almost perfectly-- Russ Roberts: Chess players. Serious chess players-- Daisy Christodoulou: Yeah, really good chess players. Chess Grandmasters. If you give them a chess board with 25 pieces, you give them a couple of seconds to look at it, ask them to reproduce it, they're insanely good. But, if you give them the exact same task with one small difference--so the pieces are not taken from a chess game, they just put at random on the board--they cannot do it. So, again, this is disproving that general mental tool idea of mental skill. Because, why is it they can be so good at 25 pieces from an actual chess game but terrible at 25 pieces placed at random? They don't have a generally good memory skill. What they have stored in long-term memory is a schema of thousands of chess games that they've seen, where they know the kind of typical patterns that you would get. And, that's what makes it so easy for them to remember and to perform these apparently superhuman feats of memory. And again, that's a really seminal study. Simon and Chase's "Skill in Chess" has been replicated with all kinds of other experts. And, the thing I always say is: people talk about critical thinking as being this all-purpose transferable skill. I mean, the thing I always like to joke about is you can think of a physics professor who is really good at critical thinking within the context of solving a physics problem and really poor at critical thinking in the context of solving a problem in his marriage. We all know that person, right? So, the idea that these skills--you can have this very generic skill, like problem solving or critical thinking--it's just not there. One other paper I'll put in, that a lot of Americans have been putting in here, there's a lot of really good American researchers here. One of my favorite ones on creativity has a little American sporting reference for you in the title. It's by a leading researcher in creativity, James Kaufman, and it's called "Could Steven Spielberg Have Managed the Yankees?" Russ Roberts: That's a good question. Daisy Christodoulou: It's a very good question, and he concludes no, because he concludes that, again, Steven Spielberg's creativity is built up within a specific domain of filmmaking. And when you jump to a very different domain that would require far transfer. Like, to managing the Yankees, sure, you need creativity to manage the Yankees, but it's just really different knowledge. The creativity is within a completely different domain. And the other one, last name check, is the guy who I think has done so much work on this, is completely brilliant, is Professor Daniel Willingham at University of Virginia. Russ Roberts: He's been on the program talking about some of these issues. We'll link to that episode. Daisy Christodoulou: Absolutely brilliant. Enormously influential for a lot of people here in the United Kingdom. And, he just has brilliant, brilliant, very teacher-friendly articles about how all of this research should manifest itself in terms of how you design a curriculum and assessment. So, I would say the empirical argument, we're arguing back with Macaulay, arguing with him in the grave, the empirical argument since Macaulay is just completely, I would say, a slam dunk against his argument: Skills are domain-specific. It matters what you teach. Russ Roberts: I'm glad you got 'slam dunk' in there for the Americans. Daisy Christodoulou: I love it. Russ Roberts: I think a simpler test--I haven't read the Kaufman article, but it's famously the case that successful coaches in college basketball do not become successful pro coaches, and vice versa. Successful pro coaches often struggle in college--although there are obviously exceptions; and they're in other sports as well besides basketball. Like: How far is that? That seems pretty close, and yet there are some specific skills in there they just don't have or have never honed, and the general skill actually is not so helpful. |
| 55:21 | Russ Roberts: I want to close with a different--what some might think of as different, but I actually think of as quite related--and I heard you talk about a little bit recently. We were in England at this conference together, and it fascinates me. Let me set it up this way. One could argue--I don't believe this, but many people think this is the case--that the goal of education is to encourage skepticism. And, you could argue that a lot of EconTalk is about skepticism. Someone says: studies show about chess players, but hey, there's publication bias and replicability problems, and maybe that Herbert Simon thing isn't true, and so I'm going to be skeptical about it until I get to look closely and see if it was well done. And, so, that honing of the brain to be skeptical, one might argue, is a very important part of education. Don't just believe everything you hear. And, you could argue that there's not many things more important in education than being skeptical of what you're told, and to make sure that the things that you're told or that you come to believe have real foundations and evidence behind them, and so on. And I struggle with the fact--so I have a part of me that's very much in that direction, in that domain. And yet at the same time, pushed too far, that kind of skepticism can lead to what you would call cynicism: Nothing is true. Everything is relative. There are no ultimate values. There's no ultimate truth. You can never know. And therefore you should just be cynical about everything, not just skeptical. And, in my experience, that leads to nihilism, which is not a healthy thing in my view. But, I heard you say at this conference we were at that very few great teachers--and this is just a casual, empirical observation on your part--are cynics. Did I understand you correctly? And then, react to what I said. Daisy Christodoulou: Yeah. So, what I said was that every great head teacher I've met--and I work in education, I've met quite a lot of head teachers--I would say, again, completely anecdotal-- Russ Roberts: When you say head teachers, do you mean 'principals' in American parlance? Daisy Christodoulou: Yes, principals. Yeah. Sorry, principals. So, I would say every great principal I've met--and again, this is totally anecdotal, no data behind this. Maybe I'm doing a Macaulay and going ahead of myself and getting convinced by my own eloquence. But, I would say every great head teacher I've met doesn't really have a sense of irony. They can have a sense of humor. They can be funny and be witty, but they don't have that sense of irony. And, what I mean by irony is that sense of undercutting, of that sense of seeing something: yeah, maybe it is related to skepticism, but kind of seeing-- Russ Roberts: It's cynicism, I think. Daisy Christodoulou: It's related to skepticism and cynicism. So I think irony goes too far, and it becomes cynicism. And, I think the reason for this is a head teacher is setting rules, and they can't be undercutting the rules. They have to believe in the rules. And, I think I said to you, too: when I started out as a teacher, I did an English literature degree, so I need to put my cards on the table. I've spent this whole podcast talking to you about how the study of ancient literature is a terrible idea, and it's basically been my entire academic field of study. So, yeah: make of that what you will. But, a big part of the study of English literature is, irony is a huge part of it. And a lot of the English authors, I would say, we most admire are ones who are masters of irony. I mean, Jane Austen leaps to mind, but there are plenty of others. And, I remember--I came out of education, I came straight out of my degree and graduated in the summer, and by the autumn I was teaching. And, it's really hard to sustain a sense of irony in a classroom with teenagers, because you have to kind of enforce these rules and believe in them. And, if you don't, and if you're the one who is constantly undercutting, it just gets really difficult. And so, I think you can probably, as a teacher, just about sustain it, but I think it gets harder if, as I say, you're a principal and you're at the top of the tree, and you are responsible for upholding these rules. And so, I think there is a kind of an issue that a lot of the qualities, and in lots of ways, I do admire these great masters of irony, like Jane Austen, but there are a lot of ways in which it's kind of hard to really construct a coherent worldview on that. Because, as you say, one of the great issues with skepticism is it undercuts itself. And, I think the great tension here is that skepticism and rationality have historically given us enormous benefits. So, modern science is incredible when it's, to a large part, built on not trusting and verifying. Russ Roberts: Prove it, yeah. Daisy Christodoulou: But, on the other hand, if you just literally believe in nothing and you are trying to prove everything, you will just end up in a reductio ad absurdum, and you will end up really in something--and this, again, we're talking about Pepys, but you can go right back to the 17th century here--you will end up in a kind of antinomian state of mind, where you just trust nobody and nothing. And, I think this goes to the Royal Society, the Royal Society's motto, and Pepys was involved, I think, in setting up the Royal Society. The Royal Society's motto is 'Nullius in verba': Take no one's word for it. Russ Roberts: Yeah. Daisy Christodoulou: And, on the one hand, yeah, fantastic. That's amazing. And, the way the Royal Society did that, did result in some of the most amazing advances that the entire world have benefited from. On the other hand, actually, it just doesn't work. To absolutely, definitively prove even one of the most basic facts that you take for granted in your life could take a lifetime. So, ultimately, we are all taking something on someone else's word. You can't not. But, this is the tension. You go too far one way, and you're credulous, and you just become gullible, and you fall for scams and conservatives. But you go too far the other way, and you just undercut yourself and, as I say, become antinomian, and the ground beneath you falls away. So, there has to be some kind of place that you can find where you do find some secure ground to put your foot on. But, when you find that, I think you do really kind of have to defend it a bit. It's a tricky one. |
| 1:01:32 | Russ Roberts: Yeah. I think I first read it in David Epstein's book, where he talks about NASA [National Aeronautics and Space Administration], which, by the way, is another government agency that very much prioritized skill, particular skills of astrophysics and mathematics, wisely, to keep people alive; but fails from time to time, as we know. And David writes--I think it's where I saw it--on the wall it says, 'In God we trust. Everyone else, bring data.' And, it's a beautiful idea. And, in a certain sense, you want to live your life that way, requiring take no one's word, prove it. Where's the data? Where's the evidence? And yet, there's a dance, as a human being, where, not just the practicality part, that you can't prove everything and you have to accept some things on faith. But I think there's something more important, which is some things aren't amenable to proof. Some of those are in marriage, that you were talking about earlier--how you deal with people, whether it's your spouse or friend, or a colleague at work. And, this is related for me. I don't know if it makes sense, but I think about my youth. I was brought up on certain types of humor, and they were all ironic. I was, I think, 12 years old in the heyday of Marvel Comics, when Spider-Man and the Fantastic Four, and others, were coming out of the pen and mind of Stan Lee and Jack Kirby, and others. And they were always ironic. Their sense of humor was always about--they were always wise-cracking. They were always putting things down and being hip. And, I think that's a huge part of American culture. I was part of that as a kid. The humor--I was a big fan of the Firesign Theatre, which was this set of crazy recordings when I was in high school, of super-ironic stuff. And, I think, I don't know if it's healthy. I mean, I love my sense of humor. I'm totally happy with it, and I passed, I hope, some of it onto my children through--but part of what they watched was The Andy Griffith Show, which I'll just mention as incredibly funny, but also incredibly charming, and there's zero irony in it. It's good-hearted in its nature. It's virtuous in its nature. There's no attempt to show that you're cooler than the people who keep the rules. But, all that other stuff is about breaking the rules. And it's a fascinating issue. I think you said it well about you need a place to stand, and if you were only going to stand on places where you know it's safe, you're in trouble. So, I don't know what else to say about it. You can take us home if you want to add anything. Daisy Christodoulou: Yeah, I think I agree with that. And, I think the other thing we were chatting about, when we spoke in person, was to what extent may be--I get your point, there's an ironic strain in American culture. America is large, it contains multitudes. But, it does feel like maybe there's a strain of earnestness in American culture that often maybe British people maybe sometimes like to mock. And, I've probably been guilty of that. But then, you look at it, and you think: that earnestness does have its strengths. And, as you say, there is a point at which the irony can shade into cynicism, and the cynicism can be something that can really stop you getting things done if you just are always seeing the bad side of things. So, there has to become some kind of value to earnestness as a way of--if you want to achieve things and you want to build things, you can't always be knocking things down. You have to want to try to build. And, I wonder if some element of earnestness is necessary for being a builder and not a destroyer. Russ Roberts: My guest today has been Daisy Christodoulou. Daisy, thanks for being part of EconTalk. Daisy Christodoulou: Fantastic. Thank you, Russ. |
What does a failed 19th-century Chinese civil exam and one of the deadliest wars in history have to do with how we hire, educate, and test people today? In this EconTalk conversation, Russ Roberts and education expert Daisy Christodoulou trace a surprising thread from the Taiping Rebellion through Britain's Northcote-Trevelyan reforms, Samuel Pepys's naval exams, and Macaulay's belief that a hard exam on anything--even alchemy or the Cherokee language--reveals the best minds. Christodoulou dismantles that idea with evidence that skills are stubbornly domain-specific: chess masters, surgeons, even Steven Spielberg couldn't simply transfer their brilliance elsewhere. Along the way they wrestle with meritocracy, why bureaucracies fail without feedback, and whether great teachers must resist irony. A rich, wide-ranging discussion that will change how you think about testing and expertise.
READER COMMENTS
Eric
Sep 7 2026 at 12:19pm
I believe there is a stronger steel man case that could be made in support of a particular application for “Macaulay’s belief that a hard exam on anything–even alchemy or the Cherokee language–reveals the best minds”.
Most of the empirical evidence mentioned in the episode showed that the skills and knowledge that one learned by studying some-other-hard-subject X did not tend to transfer to other tasks. That’s fine. Let’s take as a given the even stronger assumption that there will be zero transfer of acquired skills or knowledge from hard-subject X to the field of actual interest. Take as a given that the skills and knowledge from studying Greek or Latin or alchemy or whatever does not mean you are now well equipped to be a doctor or a leader in the royal navy.
Consider instead a different question. Suppose that instead of choosing who will now be a doctor or who will now command a ship in the navy, you are trying to decide which candidates will be chosen for the limited number of openings to enter medical school or enter the naval academy.
It is pointless to test for proficiency in the subjects they will be taught. Assume the candidates do not yet have those skills or that knowledge. That will be the point of the instruction they would receive, if admitted.
If you are offering training in some highly complex and difficult domain, what you may want to know is which of these candidates already shows demonstrated aptitude for mastering some other highly complex and difficult domain — of any kind.
You don’t care whether their acquired skills and knowledge from that other domain transfer or not. Assume the skills and knowledge don’t transfer at all. What you are assessing is the best available indication of whether they will be able to master the new domain of this training program, based on previously mastering some other difficult domain. Which potential students are most likely to swim rather than drown, if they take this on?
There is a difference between looking for the knowledge and skills provided by mastering some-hard-subject X (which probably do not transfer), and instead looking for the ability to learn and master a difficult domain of expertise when given the challenging training for that new domain.
It is not “What have you learned?”, but rather “Who are the strongest proven learners?”
The proper use for this kind of examination would be for admission to the training program for the desired difficult domain.
DG
Sep 10 2026 at 11:35pm
To add to this point, in addition to selecting for very smart, very hard working people it sounds like the exams tested some abilities that were genuinely relevant to service in the British colonies: (A) facility with languages, and (B) the ability to understand people and cultures very different from one’s own. Additionally, thinking about Ancient Greece and Rome a lot also seems to encourage reflection on the impermanence of empires and the ways ideas and cultures persist over very long periods of time. I’m sure the system was far from perfect, but I’m not convinced you could have done a lot better.
Preston
Sep 14 2026 at 10:39am
Eric, this is simply the signaling model which Russ mentions in passing. I would liked to have seen this explored more fully in the episode. If you think transfer is near-zero, then education becomes all signaling (Bryan Caplan’s position).
Doug Iliff, MD
Sep 10 2026 at 12:35pm
Wholeheartedly agree with Eric, especially in fields where information and skills are always changing— military and medicine are good examples, but there are plenty more.
Every physician can look back on the extensive runup to medical school and question the requirements. Have I ever, in 50+ years of practice, given a single thought to calculus or organic chemistry? I knew I was training to be a doctor, not a chemist or engineer; so the requirements could certainly be trimmed— although, as Eric points out, running a gauntlet of prerequisites does require a modicum of intelligence and a whole lot of persistence. What medical educators have always wanted, however, was a way to track emotional intelligence. Still looking for that program.
On a related subject, Mortimer Adler once gave an insightful lecture titled “Everybody’s Business.” As the founder of a classical K-12 school, I was persuaded that there was a way to orient the teaching of many subjects toward generalists, rather than specialists. It is, I think, useful for students to read Shakespeare, but not to debate who wrote Shakespeare. It wouldn’t hurt every high school senior to understand the Krebs Cycle without diving deeply into biochemical formulas. Comprehending the wonders of a geometrical proof wouldn’t hurt, either.
E.D. Hirsch, in his book Cultural Literacy, also made a good case for the accumulation of background knowledge in the education of a generalist. I have absolutely no such knowledge about hip-hop, its origins, or its practitioners— and no desire or need to gain that knowledge. But we are presently experiencing the problems which arise when college graduates have insufficient background in history or economics.
There is no doubt that educational pedagogy has run off the rails over the past century. Whether the democratic experiment survives may depend upon adopting various forms of classical education from the bottom up through college.
Robert Ferrell
Sep 11 2026 at 7:16pm
This episode, especially the last 15 minutes, made me think of the Turgenev novel “Fathers and Sons”. It’s a quick read. If you haven’t read it you might enjoy it. I like the Constance Garnett translation.
As always, thank you for such an interesting podcast. I’ve been listening to EconTalk ever since my phone had a podcast player. ~15? years?
Among the happy outcomes for me, you introduced me to Smith’s Theory of Moral Sentiments. I spoke about it at home so much my 18 yo daughter requested we read it together this summer. As a dad, it was a joy to be asked:) In her case, she prefers to get to the primary source when possible, so ideas and concepts I shared from Smith she needed to read firsthand rather than accept my (biased) interpretation. (Succinctly, she wanted to fact check me.) She’s now off to one of those old New England colleges which, of late, has a poor reputation for open conversations and free expression. I hope her penchant to ask uncomfortable questions doesn’t cause her too much discomfort, although I’d rather that than have that curiosity squelched.
Relating to this episode, I wonder why students don’t push back on the college faculty, administration and fellow students who advocate for censorship; self-censorship; shouting down of “unpopular” speakers; and even accept violence. (See the recent FIRE.org releases about free speech on campus.) Aren’t students supposed to rebel against the status quo? Are many students in fancy-pants schools graduating with neither critical thinking habits nor marketable skills?
In any case, thank you again for so many great conversations. I have learned so much. And I have enjoyed so much. You have enriched my life and also enriched my family.
Kevin Ryan
Sep 12 2026 at 2:00pm
I am very much interested in the Taiping rebellion and, for podcast listeners, I would strongly recommend the current series on it in Chris Stewart’s History of China podcast.
Have to say I found the reference to it in this interview a red herring. Hong Xiuquan did not fail the exams and have a breakdown because they were testing the wrong subjects. The reality is that there were few jobs and Hong was just not good enough in comparison with other candidates to get one.
More generally I agree with Eric’s comments above.
What was being sought in both the Chinese and British systems was administrators able to turn their hand to a large variety of tasks and duties for which it was hard to prepare in advance and technical knowledge had only limited benefit. They were not selecting amongst medical students.
Now, as an MBA myself, I have to believe it was possible to improve on learning the classics as the basis for selection. That said, I spent a number of years working for a prestigious British organisation whose management echelons were populated by people who had been through the Oxbridge university system in a variety of subjects, and with technical knowledge I considered inferior to mine in the subjects on which I was interacting with them. But I was nearly always impressed by the ease at which they were able to grasp the key issues in these technical areas. I was in no doubt that, even if they were not perfect, they were better men than me. (Well the majority were men, but my conclusion also applied to women in these positions).
Have to say I did not find this interview very convincing. A bit of confirmation bias, I feared. But I would admit to being a cynic.
Tom Murin
Sep 16 2026 at 11:53am
I agree with Eric (and Macaulay).
I was an officer in the US Navy. My degree is in History. You had to obtain a passing score on the Officer Aptitude Test (really an IQ test) to determine if you were trainable.
I learned naval engineering, navigation, weapons systems, military law, communications, and more – before I was assigned to my first ship (a guided missile frigate). I received excellent initial training for my assignment, but obviously had a lot more to learn when. When we qualified for a watch-standing position on the ship – we had to pass an oral board. They’re called “murder boards” for a reason. You had to know a tremendous amount about the position you were trying to qualify for – and there was no way to BS through it.
Andrew
Sep 30 2026 at 1:19pm
Good discussion as always with Daisy.
Just had to pop up to see if anyone had picked Russ (and Claude) up on misidentifing “Wrangler”, which I believe was a reference to this:
https://en.wikipedia.org/wiki/Wrangler_(University_of_Cambridge)
A classic example of some domain-specific knowledge. I’m sure Daisy knew this but was just too polite…
Comments are closed.