Adam Mastroianni on Learning and Mostly Forgetting
Oct 9 2023

brain-vobes-300x300.jpgHow much do we remember of what we learn in school or from conversation? Psychologist Adam Mastroianni says: from little to nothing much. What do our brains retain? Mastroianni argues that often it's a mix of emotions, meanings, and values that end up shaping who we are, what Mastroianni calls "vibes." Listen as he and EconTalk's Russ Roberts discuss the role of vibes in knowledge acquisition and the implications for how we teach, learn, and speak to those around us.

Andy Matuschak on Books and Learning
Software Engineer Andy Matuschak talks about his essay "Why Books Don't Work" with EconTalk host Russ Roberts. Matuschak argues that most books rely on transmissionism, the idea that an author can share an idea in print and the reader will...
Bryan Caplan on the Case Against Education
Bryan Caplan of George Mason University and the author of The Case Against Education talks about the book with EconTalk host Russ Roberts. Caplan argues that very little learning takes place in formal education and that very little of the...
Explore audio transcript, further reading that will help you delve deeper into this week’s episode, and vigorous conversations in the form of our comments section below.


Ben Service
Oct 9 2023 at 5:18pm

The minimum wage example was an excellent example of how to deliver a message, I am sure the picnic blanket person could have got a different (maybe better depending on the intent) outcome if they had thought about the point of the comment.  Do you want to give facts, give opinions, open a conversation, hear someone else’s point of view, change someone’s mind or shock, horror, change your own mind? They all take a different style.  Many engineers (of which I am one) and economists could take a leaf out of the books of other disciplines when they are talking to others and ask those questions before starting an interaction, I think other disciplines maybe do it more innately from their training and don’t have to explicitly think about it.

Russ of course has taken many leaves out of many books and is pretty good at this stuff, us mere mortals can but try to be better an more thoughtful.

Chris Alexander
Oct 10 2023 at 4:06am

They had the quote below on the wall at my school. Having googled the author, I see that he had some “problematic” characteristics but I think this writing captures a lot. I especially like the “shadow of lost knowledge at least protects you from many illusions”. I like to believe that’s right

“At school you are engaged not so much in acquiring knowledge as in making mental efforts under criticism. A certain amount of knowledge you can indeed with average faculties acquire so as to retain; nor need you regret the hours you spent on much that is forgotten, for the shadow of lost knowledge at least protects you from many illusions. But you go to a great school not so much for knowledge as for arts and habits; for the habit of attention, for the art of expression, for the art of assuming at a moment’s notice a new intellectual position, for the art of entering quickly into another person’s thoughts, for the habit of submitting to censure and refutation, for the art of indicating assent or dissent in graduated terms, for the habit of regarding minute points of accuracy, for the art of working out what is possible in a given time, for taste, for discrimination, for mental courage, and for mental soberness.”


Anthony Guzzaldo
Oct 10 2023 at 10:22am

I’d like to make a suggestion for a guest that could present another (I think more correct) view on knowledge. Daniel Willingham is a cognitive psychologist who has made a career, among other things, of championing knowledge in education. I think he’s also making the rounds on podcasts right now to promote a recent book.

Nick Ronalds
Oct 12 2023 at 9:59am

I was struck by Russ’s insight that we learn dramatically better through stories and humor. I was hoping to find in “delve deeper” some recommendations on exactly that. What are some good sources for memorable stories illustrating important economic principles?

Russ’s other great point is that communication shuts down if our counterparty gets of whiff of being lectured. Is the best way to converse productively about difficult topics, especially when you think your counterparty is wrong, with questions? In the spirit of “maybe we can get to the bottom of this puzzle together”? What works when talking about difficult and controversial topics?

Ronald Menich
Oct 13 2023 at 5:55pm

Recall is vastly more difficult than recognition.

Mastroianni might only recall 19 out of the 32 classes he took in college.   But imagine that he were presented with a list of 100 courses and asked to recognize which 32 he took in college: I imaging that he would recognize almost all of them.

When working, we have access to much auxiliary documentation and are not in a pure recall situation.  Recognition may be a more relevant measure.


David M
Oct 15 2023 at 4:59pm

I would guess Adam’s main point (i.e., students mostly absorb knowledge at a “vibes” level) is true for many students in many disciplines. It hasn’t been true of my experience in mathematics, physics, or computer science though. These fields have the property that concepts learned in upper-level courses truly rely on concepts learned in lower-level courses. For example, a student learning about differential equations will definitely rely on knowledge from calculus (which relies on knowledge from algebra, etc.).

As a result, I don’t think this “vibes” view of education is generally true. It may be more true in social sciences, which have a less mature body of theory built up around them.

As an aside: Adam’s experience (and the experience of his students) with the R programming language is at least partially due to R being a terrible language with an illogical design. Many computer scientists like to make fun of it. For better or for worse, statisticians have built their computing ecosystem around R.

Oct 24 2023 at 8:45pm

No sense in listening as I guess I won’t learn anything 😊

Michelle W
Nov 8 2023 at 2:43pm

I really enjoyed this podcast.  I agree with David M above.  I have an engineering degree, worked as a programmer for many years and now teach an engineering curriculum in a public high school.  I’m constantly trying to find a “handle” for students.  Something / anything that might be useful to them in their future lives.

Skills that are practiced and used at a subsequent level are much more likely to stick, whether math, programming, music or sports.  However, my favorite class in college was my elective class in Art History.  The visuals, and the ability of the professor to tell a story about each artist and artwork was so compelling, and it gave me a lifelong love of art.  Great example of “vibe”.

Thank you!

Norma Frank
Nov 12 2023 at 2:32pm

In re Russ Robert’s’ conversation with Mastrionni (sp?) about remembering what we learned in school, I am 82 and I am amazed at how much I remember. Things come up in conversation and I remember things I learned in the past. Now a fair amount of that is from my own reading but I can distinctly remember things I know I learned in class. I am not an unusual person. I’m smart enough but not that smart. I’ve known a lot of smarter people, and I admire that trait, but I’m just not unusual. So there must be a lot of people who remember what they learned in class.
And it’s not just paying attention. I learned a lot in English History, in college, where I was constantly distracted by what I already knew about the people/places we were covering.

Comments are closed.


Watch this podcast episode on YouTube:

This week's guest:

This week's focus:

Additional ideas and people mentioned in this podcast episode:

A few more readings and background resources:

A few more EconTalk podcast episodes:

More related EconTalk podcast episodes, by Category:

* As an Amazon Associate, Econlib earns from qualifying purchases.

TimePodcast Episode Highlights

Intro. [Recording date: September 13, 2023.]

Russ Roberts: Today is September 13th, 2023, and my guest is psychologist Adam Mastroianni. His substack is Experimental History. I encourage you to read it. It is phenomenal. This is his third appearance on EconTalk. He was last here recently, in August of 2023, talking about how you can't reach the brain through the ears. That is: how hard it is to tell someone something and then for them to remember it, absorb it, and apply it.

Today's conversation is a sequel and maybe also a prequel. We're going to continue to talk about learning and the acquisition of wisdom and understanding based on some other writing you've done, Adam, and if--whatever we get to we'll link to--but we're going to start with an essay you wrote called "You'll Forget Most of What You Learn. What Should You Do About That?"

Adam, welcome back to EconTalk.

Adam Mastroianni: Hey, thanks for having me back. It's good to see you again.

Russ Roberts: Good to see you.


Russ Roberts: Let's talk about formal education, what we call the classroom. As you start in your essay, most of us spend years there. Do we really forget most of what we heard? Most? In fact, almost all, maybe all?

Adam Mastroianni: Yeah. I mean, you could answer this question a few different ways. Right? So, people have done some studies on this where they try to do the formal thing and track the things that people are supposed to learn in their classes and then follow up with them years later. I mean, you can get pretty much every number you want, depending on how long you wait, how you do the test, but it's somewhere between a lot of what people learn they lose and pretty much everything.

I mean, you can run this test on yourself, right? I graduated from college in 2014 and as just a little test, I tried to list every class that I took. I knew that there were 32 because I took four every semester and I got to 19 out of the 32. And so, if I can't even name a good chunk of the classes that I took less than 10 years ago--I mean maybe that knowledge is accessible in a different way, but clearly it's not very accessible.

The other example I have of this was just, happening upon an episode of Are you Smarter Than a Fifth Grader, where a woman--to the dismay of all the fifth graders on stage--is screaming over and over, 'There are 352 feet in a yard.'

So, it's bleak: whatever the number is, it's not a great one.

Russ Roberts: I want to answer to that, and the answer I used to give--it's interesting; I think as an economist with a free market bent most of my life, I had trouble--I still have a little bit of trouble, but I certainly had trouble when I was younger--accepting that really horribly stupid, irrational things would persist. I figured people would usually figure things out, make them better. So, the idea that we would spend 12 to 16 years in a classroom and get, quote, "almost nothing out of it" seemed improbable to me. It still does a little bit. So, I'm going to defend it a little bit, but I'm very sympathetic to your point now that I'm older.

One argument would be: Well, okay, you don't remember facts, right? I don't remember atomic weight. I don't remember exactly what the red shift is. I know it has something to do with speed of things and their frequencies changing. But, that's not really what--those are just facts. They're not important. I can Google those or look them up in a book. What's important are modes of thinking; and that's what I learned in school. I learned how to think, or I learned frameworks for thinking; and that's what I learned. That's more important than facts.

Adam Mastroianni: Yeah. Well, I think two problems with that. One is it does really seem that what we're doing a lot of the time in the classroom is trying to transmit the facts. That, if you went into a classroom and they were like, 'Look, we know you're not going to remember any of the facts here and you can look them up anyway. What we're really going to do is teach how to think,' I'd go, like, 'Okay.' But there doesn't seem to be a point at which the teaching-to-think actually happens.

I know that this is the conceit behind higher education. The example I use in the piece is: If you want to learn how to learn, we do actually know something about how to learn. It is nothing like we get people to learn in undergrad.

So, for instance, all of the things that people naturally do to try to remember things better--highlight things in their textbooks, take bullet pointed notes, rereading things--we don't have any--there's pretty good evidence that doesn't help you remember those things. Certainly, cramming doesn't help you remember most of the incentives that we create for people: encourage them to cram for the test, take the test, and then don't remember anything afterward. So, if that is what we're doing--which I agree we're doing some of--but it doesn't seem to really be what we're trying to do, and it doesn't seem to be what we mainly do.

Russ Roberts: So, here at Shalem College, we do--for the first year or so of the students' experience--read Great Books in small seminars and explore them. And I do think, besides finding out what happened in the Odyssey, I do think students acquire insights into themselves and learn how to read from that experience, which is much more important than knowing what happened to Odysseus when he encountered the Sirens.

So, I think there's some of it; but I certainly agree with you that--let's call it the lecturing format. I was a very interactive lecturer, but still I lectured. I think the lecturing format is pretty dismal in being transformative. And I think back to my college astronomy class: I don't remember one thing from it. It doesn't mean I got nothing out of it, but I don't remember one thing. That's really weird because I love the nighttime sky.

Now, one of the things I've learned since I took my astronomy class--unless I learned it and forgot it--is that when you look up at the nighttime sky in a decent place, meaning not too much light pollution, and you see thousands of stars, virtually every, and in most cases, every star that you see is in the Milky Way. That's extraordinary. When I learned that fact--and I learned it about six years ago from a friend of my nephew who was staying with us for a night in a place with really good stars--he just mentioned that in an offhand way. I thought, 'Well, that can't be true. I know that.' But, I didn't know it. I think it is true. And the only exception is that maybe--I think it's Andromeda, you can kind of see as a sort of smudge with the naked eye--but every single star you see is from the Milky Way, which means that you are seeing 1/300 billionth, at most, of the stars in the universe.

To me, that's a mind-boggling thing. And I hope maybe for some of our listeners that would be the only thing they take away from our conversation, Adam. Probably not. But is that not extraordinary? And, why would I not want to know that in college unless I knew it, was told it, forgot it, and when I got older and heard it, I was in a better place. I was more interested. Very possible; but it is remarkable that I literally cannot name one thing I learned.

Adam Mastroianni: Yeah. And I think the important part of that fact isn't necessarily, like, that particular ratio, or even necessarily that every star you see is in the Milky Way. I think the important thing to take away from it is you are very, very, very small and the things that you can see are a very, very tiny sliver of all the things that exist. And putting the numbers and the facts to it help access that experience. But, I think it is that thing that is quite important.

And it probably helps if you receive that knowledge as you are looking at the stars. Right? There's a reason why that feeling tends to arise when we are able to do that, and why--I feel like I can access a little bit of what you're talking about, this feeling of being very small, when you just describe it to me. But right now I feel kind of big. I'm in a room where I own all the stuff in it; and I feel like I'm the center of the universe. I live in a city, and so I don't feel all that small at all.

And so, it is not even really, like--the fact is a handle that's bolted onto the thing that we really want to get, which is this knowledge of being very small. Maybe knowledge isn't even the right word for it.

Russ Roberts: Or a feeling of awe, a feeling of wonder.

Adam Mastroianni: Yeah, yeah.


Russ Roberts: I'm going to give one more example, though, and I want you to react to it because I think it's sort of interesting. I think it was third grade--might have been fifth; somewhere in between third and fifth grade--we had to do an assignment on Greece, the country, mostly Ancient Greece. So, some people did the culture, one person did--I don't know what--different history, different pieces of it.

And one of my classmates, whose name, of course, I've forgotten; but I do remember his report vividly. He was assigned Greek military history. And his entire report was one sentence: 'Greece,' well, maybe two, 'Greece has been in many wars, too many to talk about right now.' He sat down. It's a great speech, it's a great presentation, and remarkably memorable.

Now, I want you to respond to that because it's the only thing I remember pretty much from that class or maybe the whole year. And, why do I remember that? It's funny; I've told it dozens of times, I've repeated it. So, jokes and stories we do often remember--not deep understanding, wisdom, and so on. And I think that's important.

Adam Mastroianni: Yeah. Yeah. I mean, and there's probably so much to that story that I'm assuming that you know that I don't know. That kid could have been the class clown. He could have been self-aware about this. So, it's not clear to me from the story whether he was self-aware or whether he was sort of a savant, like a child beyond his years. Or, whether, also, I think you said--I was just assuming it was a boy, that--

Russ Roberts: It was, in my memory.

Adam Mastroianni: or whether he was just really just phoning it in and didn't even realize that that was a very silly thing to say.

I have a similar story from Social Studies class in ninth grade or something. We were talking about the Treaty of Versailles and we were reading out of the textbook--I think we were going through the room and each of us was reading a paragraph. And this kid, Isaac, had to read the first time we encountered the word Versailles, and he said 'Ver-sal'-lees.' Which--I grew up in Ohio; there is a town where people, they don't say 'Ver-sal'-lees,' but they do say 'Ver-sales.' And, like, we stopped and everybody laughed at him. And I remember years later people would talk about 'Versallies.' And so I can't even--I'm embarrassed to say this, I can't even tell you which war it was that the Treaty of Versailles ended--but I do know what it was like for this guy to be ridiculed in front of his classmates for mispronouncing it, which I don't think was the thing that our teacher hoped that we kept with us 20 years later.


Russ Roberts: Let's then go to the next part of your insight. So, we have a reality--I think it's a reality--there have been a number of tests in economics of this. I used to be skeptical of them and I'm not anymore, how little people remember of microeconomic theory, say, after the course is over. I think Robert Frank on this program made the observation that months--if it's far enough after the course is over, people who haven't taken the class, do as well as people who took the class. Which should be a very good measure of how bad learning is. Let's assume it's true. Despite that, you have something positive to say about the learning that we experience. What is it?

Adam Mastroianni: So, it isn't the case that we lose everything. And, I think what remains is really interesting; and it seems to remain for a long time even without really using it, doing all the things that we normally do to keep memories strong. I mean, we just told two stories from our childhoods that--you said you've told yours before. I don't know if I've ever told that story before, but it was very accessible to me.

And so, what was it, in the stories, and what is it in the memories that remain? What are those? And what I call them in the piece is vibes, for lack of a better word--which is: somewhere between implicit and explicit memory, it is some mixture of feelings. It is also, I think, sort of propositional knowledge that can't [?trust, remember?] easily.

So, what was the vibes of, like, seeing that kid mispronounce 'Versailles' and be ridiculed? Well, I learned things like you better be really careful in front of your peers and don't do anything stupid because it will follow you for a long time. I learned that can happen at any time. I learned that something totally arbitrary could, like, lose you esteem among your peers. None of this had anything to do with what we were supposed to be learning at the time.

Another thing that comes to mind: When you mentioned remembering microeconomics, I took a macro class in my freshman year of college. I don't really remember anything about the class except for there was a day at the end when she showed us slides from her trip to China and, like, students started walking out; and she said, she was, like, 'Don't. I'm going to test you on this.'

And so, I learned something important about what it takes to, like, retain people's knowledge and, like, build a moment inside a classroom. I learned something about the arbitrariness of knowledge. Right? The thing that she had to resort to was sort of this cudgel of: you have to sit and listen to basically me telling you about my vacation because I can test you on it.

None of these things have anything to do with macroeconomics. I can't tell you anything about it, but I think those things are best described as vibes.

Russ Roberts: Fortunately, as the host of the show, I don't have to reveal what I remember about macroeconomics. I think it's greater than your knowledge, but I don't want to go any further than that.

This is awkward. I want to disagree with you a little bit. And it's your essay, so you can--maybe I misunderstand it. I think the things that you're talking about that remain, I think you're trying to salvage the classroom by saying you did learn something. Maybe it's enough to say you felt something. And I want to suggest the possibility that much of life--our experiences--of course, they have lessons. Many of our experiences produce lessons: Don't mispronounce words, don't say words that you're not 100% sure of.

But I think it's more than that, a lot more than that. It's about that much of life isn't the things we learn, but it's the things we feel.

Now, as academics, that's very disturbing. We don't like it. You refer to that at times in your piece--in a different essay we might get to today. But, isn't the lesson here that much of life is how we experience things rather than the content, and that that--maybe that's okay, but it's definitely not the alleged purpose of formal classroom learning?

Adam Mastroianni: Yeah, no; I think we're in agreement on this. I think because the word 'feel' does a lot of work: that, usually, I think we assume when we're talking about, 'I felt something,' we assume that means an emotion. And obviously emotion is part of this, but it isn't just that I felt sad at this time. When we say that word 'feel,' what we really mean is, 'I had some kind of experience that is difficult to articulate.' And so, emotions are part of that, but they're not all of that.

Like, when I saw that professor losing the classroom, I did feel something like secondhand anxiety. I felt sympathy. But, I think though--what those feelings were, my brain's way of telling me, like, 'Pay attention to this moment. Something important is happening, and that's why we're going to encode it.' And so, those emotions are almost like tags in a filing cabinet that allow you to find it again.

So, yeah, I agree: and, like, I don't have any hope of salvaging the classroom in particular, but I do think these things can happen there and I think they can happen more if you appreciate that those are the things that are much more likely to make it beyond the few months after your class--that, like, I effortlessly remembered that experience years later without ever having being tested on it. Without ever--no one ever told me I was going to be tested on it. Something about it made it stick, and I do think it was a feeling.


Russ Roberts: So, you give an example of what you call bad vibes and good vibes from your psychology education, which is both amusing and, I thought, informative. So, give us the example of the bad vibe, the opening moments of one psychology class, and a different one--a good one.

Adam Mastroianni: Yeah. The bad vibes were in a cognitive psychology class I took. I remember vividly in the first lecture, the professor said, 'Cognitive psychology is pretty boring.' The bottom fell out of my stomach, because I went, 'Oh, no. I have to take this class.' And, I mean, it turned out to be pretty boring because he felt that way about it. I do remember that.

But, as I point out the essay, there is something I really remember well from that class, which is the concept of greebels, which are like weird little alien formations that were used to settle or try to settle a debate in cognitive psychology and neuroscience about, like, what is this patch of brain that we call the fusiform face area that seems to light up in response to faces? Or is it specific to faces or is it actually for processing fine configural details? So, they made these little alien creatures they called greebels; they trained people to distinguish between them; and then found that the fusiform face area also responds to greebels as well as faces after you acquainted[?equated?] yourself with them. Obviously, the problem here being, like, well, the greebles kind of have faces.

Russ Roberts: And that's worth at least some healthy fraction of $60,000 for the tuition you paid for that class.

The tragedy of that story is that you remember two things. One, a really inexcusable, unprofessional opening remark by a faculty member; and two, a phenomenon that is really not that interesting is not a transformative experience. But you have a good one.

Adam Mastroianni: Yeah, yeah. The good vibes were a different psychology class I took: It was Psych 101, and the professor--his name was Danny Oppenheimer--he stood up at the front of the class and he had this whole routine that was all based around a bag of M&Ms. And I don't remember the whole routine, but I remember the fact of the routine, parts of which included getting somebody to stand up and he is, like, 'Okay, I'm going to toss you the bag of M&Ms,' and tosses it to the student; the student catches it. And he's explaining that what the brain has to do in order to coordinate the body in order to capture the M&Ms, it's pretty amazing. You have to be able to tell the difference between where the bag of the M&Ms is and the rest of the world around it. This is not a trivial computational problem to solve. You have to coordinate your limbs to put your hands in the right position to catch it. I was like, 'Oh, this is all pretty cool. This guy's throwing M&Ms at students. That's kind of neat.'

He tells the student to toss the bag back to him. He catches it, he goes, 'Obedience. That's also something that we will catch in this class.'

Now, the student is a little embarrassed. Everybody's laughing. I'm getting goosebumps even just thinking about it. I was like, 'Oh, this is kind of fun.'

He tosses the M&Ms back to the student, and he says, 'Okay, now toss it back to me.' And the student doesn't do anything. And he goes, 'Learning, we'll also talk about that.' And I go, 'Oh, he predicted what the student would do.' And it did make me feel a little bit like, maybe there is something to this psychology thing, if you could do this with it.

And the rest of the semester felt that way. I remember weeping at the end of the semester just at the beauty and the amount of knowledge that I felt like I had encountered. Not that I could remember all of it afterward; but there's a reason why I became a psychologist--that I kept chasing that feeling that I had: that there were interesting things to discover here and we could do cool things with them.

So, those were some good vibes.

Russ Roberts: And you mention in your essay that even though he had this delightful interaction--experiential moment with the student in the opening lecture--the rest of the class was just him, at 2X, telling you stuff.

Adam Mastroianni: Yes.

Russ Roberts: And, it was sufficiently interesting that it was that powerful that it made you weep it when it was over. Do you really think that's all you got out of it was that feeling, though?

Adam Mastroianni: I mean, I do think that because--I mean, it was my major and I kept, I stayed in the field--there's probably more that I remember from that class because I kept using it. So, it's not zero. It's definitely not zero. I don't know if it's a very high percentage. And it's not--sorry, go ahead.

Russ Roberts: No, but my point is the fact that you can't remember it is not--

Adam Mastroianni: Yes--

Russ Roberts: that decisive.

Adam Mastroianni: Yes.

Russ Roberts: Maybe you used it in other ways and your brain retained certain patterns that matter. Maybe.

Adam Mastroianni: Yeah, but for the amount of time and attention that I paid to that class at the time, it doesn't feel like a great ratio. And I mean, I do think this ultimately makes sense. There is something there, right? I'm glad that I took that class. It changed my life.

But, at the time when you're in it, it feels like what you're doing is assembling all this house-of-cards of facts and knowledge that then topples over afterward, rather than trying to optimize for the things that stick around much later. Or at least teaching in such a way where you realize that, like, the point of this cannot be that at the end of this we administer to you a multiple choice test and we hope you get like 95% of the questions right, and then we hope that 10 years later, you would still get most of the questions right. That: if you're doing that, I think you're wasting your time.


Russ Roberts: So, let me try a different take on this. I lived with my parents for 18 years and then some summers. The 18 years was full-time. There were some times where I was asleep. There were times when my dad was on a trip. There was times my mom was on a trip. But, we spent a lot of hours together. I don't remember very many of them. I really don't. I remember more than I can say: every once in a while a memory will arise that I had forgotten--whatever that means--that I couldn't consciously bring to mind. But, when we think of the quantity of hours--let's just talk about the four hours at night for 18 years. It's thousands of hours where my brain has very little direct recall, and yet I'm pretty confident they shaped me. I want to say two things actually. One is most of what I do remember is vibe. I remember that they cared for me probably more than I remember the content of what they wanted me to absorb.

But, my dad was very much a teacher--he was an amateur teacher--but he loved teaching me things. And I don't think I remember very many of them, but I do remember that vibe. And I wouldn't want to say, I don't think, that even though I can't remember them, they probably didn't--I don't want to say they didn't have much impact.

So, I'm torn. I find your argument profound--and there's more to say: we're going to get to it in a minute--but I wonder if it's, if you protest too much.

Adam Mastroianni: Yeah. I think it comes down to what is it that we hope to do and are we doing that or not? That, if what your father hoped and if what my Psych 101 professor had hoped, is that we really thought the multiple choice question, knowledge at the end of it: I think their hopes were misplaced. But that might not have been what they actually hoped.

And in fact, I sent my professor this essay and he largely agreed that the point of that class wasn't that, at the end of it, I can tell you what the fundamental attribution error is, or I'd know what the cerebrum does. It was to understand at a deeper level what psychology is about: that it is possible to ask and answer questions about the human mind, and in some sense of how that's done. And that does touch plenty of explicit knowledge. Right? I should be able to tell you some things; and I am able to tell you some things. But what I really should be able to do is--well, I don't know--what I'm able to do, what really should happen to me is that I come away going, 'There's something interesting and important here. Some of my theories of the world are constrained. I know some things not to be true. I know a few things to be true, and I have a better sense of how one might go about doing the business of psychology.' I mean, that is what I ended up doing the rest of my life.

And so, I think maybe the hope is similar for a parent: that, yes, maybe when they teach you how to change a tire, they really do hope that you remember how to change the tire the next time you have a flat tire.

But, I think any parent's hope is exactly what you described, which is you knew that you were cared for. And beyond that, like, whether you know the proper way to descale a bathtub or the correct amount of water to give a certain kind of plant, like, these things are so much further down the priorities list that in the moment we hope that people will remember these, but we hope that they remember the lesson behind the lesson, which is that there is a certain way of living that is good and there's a certain way that this person feels about you, which is also good.

I'm sure I'm going to fall into the same trap when I'm a father. I really hope that they remember all the specific things, but obviously I hope they remember the thing behind the thing.


Russ Roberts: Yeah. Yeah. I read a quote, a beautiful quote, from the piece where you tried to describe what you're trying to describe. Adam, I think you're really interested in that challenge, how hard it is to describe the things you want to describe. You said, quote:

So far, I've used the word "feelings" to describe these indelible, ineffable memories. That implies that these memories are all about emotions, and no doubt emotions are part of them. But there's even more than that. It's hard to describe, but: the transition of winter into spring, seeing the Statue of Liberty in person for the first time, the phase shift that happens when you enter a dance party, the last day of camp, being at a wedding that probably shouldn't happen--I'm sure these all evoke images and emotions, but there's something underneath it all, binding it all together. That's what I mean by a feeling. A combination of emotions, aesthetics, meaning, and values. And when you layer feelings on top of each other, you get a vibe. (It's a silly word, but it's the only one that fits.)

End of quote.

You want to say some more things about that or try to? I know it's hard to put into words. You did a lovely job trying there.

Adam Mastroianni: Yeah. I don't know if I could do any better than that.

I think there is a thing there: that when I say something like the transition of summer into fall, I think it evokes--I mean for me, anyway--it evokes something. It evokes, like: school is coming, change is happening. I used to talk about this with my sister when we were kids that, like, fall felt a certain way. And, when we say 'felt,' it wasn't exactly emotions. Right? Like, there's emotions in there. Some of it is trepidation and excitement for school. But it really was more like: there's a way that is fall, and when you're not in it, you don't feel it. Man, it is really hard to describe this thing. But I certainly hope that people listening, like, also, they go, like--I mean, I guess you only wouldn't get it if you didn't live in a place where you didn't have fall.

Russ Roberts: And, it's got a smell. I can smell fall. If you say, 'Does fall have a smell?' All of a sudden without conscious effort, I could smell burning leaves. I can smell a pile of leaves. The air feels different.

So, it's not--actually, I think the philosophical term for this is qualia--I mean, which is a phrase I often struggle with, what its actual meaning is. It's the full range of how we are conscious of things.

The other thing I would just mention is what I was trying to get at earlier--I didn't say it very well--about experience. Most of the great conversations I've had in my life were not about what I learned from them. There are many conversations I've learned interesting things. This is one of them. I'm sure that I will learn something from talking to you for an hour. But, most of what I love and remember about conversations is how they make me feel. And most of them don't make me feel anything--which is a tragedy.

But, there's a handful that were profound; and it's not because I learned something really important. It's because I connected to another human being, and deeply.

That is a huge part of life that just--it's outside of learning. And I think a lot of what you're capturing about what we actually remember from the classroom is that level of--it's an intimacy. It's not with another human being. It's with a field, a discipline, a subject matter. That you could feel your brain working, that you felt alive, intellectually. Those are profound, but they're not content.

Adam Mastroianni: Yeah. Yeah. Yeah. Like, psychology is a way, and economics is a way. Sometimes interacting with economists--it is hard to articulate--but I can feel it is a different way than psychology is a way. That, there are some differences in values I think we could articulate: That, if you give a talk in front of economists, you're much more likely to be interrupted and you're more likely to get criticized. Whereas psychologists, we sit patiently until the end of the talk and then we say, 'Thank you so much for your terrific talk.' And then maybe we ask some challenging questions.

But, I think that small cultural difference is the mere, like, filing cabinet tag on top of, like, many deep differences. Like, that's not a superficial difference. I think that is connected somehow to the differences in the way that we go about doing what we do in a way that--I don't know enough about economics to know how it's different there. But, yeah.

Russ Roberts: Yeah; I think that's 100% true.


Russ Roberts: And I think--one of my children says that what you study in college is your favorite way of looking at the world. You study science: you look at the world a particular way. You study psychology: you look at a particular way. Economics--and they're all different. You tend to organize the chaos of the facts and causal mechanisms that you encounter through that framework.

And, if you do it long enough, you forget that it's not the only one. And you just become--and that's very powerful because it opens many doors you would not be able to otherwise open, but it closes off other doors that you will not be able to open because you have such a narrow tool.

Adam Mastroianni: Yeah. To what you said earlier about the way things make you feel--I thought about this in the moment and forgot it. One of the quotes I have in the piece is this quote that I think is very powerful and good, which is:

"People may not always remember what you say, but they'll always remember the way they make you feel."

And what I think is so funny about that quote is no one remembers who said it. It is sometimes attributed to Maya Angelou. But, I looked at some of the etymology of this quote, and no one really knows where it comes from, it comes up.

But I think the quote itself is the thing that it is describing, right? No one remembers who says this thing, but you do remember that quote because it does make you feel a certain way--because you know it to be true, that there are ways that people made you feel. There's a way that a field made you feel.

And the reason I remained a psychologist was because it made me feel a certain way: that the affordances that it presented to me were ones that I knew how to grasp. And they felt good in my hands when I grasped them; and it felt like I could move around reality with the handles that it gave me. And that was a feeling.

I think you're totally right, that you get very used to those handles and you stop seeing other handles that didn't fit your hands as well. And so, now I can only think in the kinds of experiments that you would run to answer the questions psychologically. And I don't think in terms of anything else that you could do--that I've been given 32 different types of hammer and no screwdrivers. Which is okay. I think there are plenty of nails in the world to hit, and other people will have their screwdrivers and they'll drive--you don't have to have all the tools yourself, but that is a thing that goes away.

Russ Roberts: I want to come back to that quote, that unattributed, unknown quote: 'People won't remember what you said...'. If that statement is true--let's pretend it's literally true: People don't remember your words, but they remember how you made them feel. It should change how you conduct a conversation, shouldn't it?

Adam Mastroianni: Yeah. Yeah. Yeah. I think because what happens is, the person comes away with experiencing the emergent property of 'you.' The property is, of course, made out of some of the things that you say. And so, it does make sense to try to say things well. But it's not--just like the thing that you take from the class is not going to be the answers to the multiple choice questions. The thing that the person takes from interacting with you is not going to be the transcript of your conversation. That will be part of it. That's in, like, the collection that remains in their head afterward. But, it's not all of it.

It is also the vibe that they got. And I can remember situations where the vibe that I got was: I'm not really welcome in this conversation. Even if the words don't say that. Or where the vibe I got was, like, the world is full of possibilities and, like, we need to chase after them and grasp them. I have a friend of mine that, like, that is the way that I feel when I talk to him, and that's why I love talking to him.

Yeah. I don't know if we think about this all that much. Think about, like--

Russ Roberts: I don't think we do--

Adam Mastroianni: Yeah.

Russ Roberts: I think this is fine because I'm going to say: it behooves us to think about it--a word I rarely get to use and often use as an example of a word that's dying out without anyone's control; but it's hanging on because it does have the occasional use.


Russ Roberts: I may have told this story before--I apologize to listeners; maybe you've heard it--but a friend of mine has told me that he was in graduate school in economics. He was at a picnic with a bunch of people, and the minimum wage came up. And he's an economist, other people were not economists. He said, 'Minimum wage, it's not really a good thing because it reduces employment opportunities for low-skilled workers.' He told me later--I said, 'How'd that go over?' He said, 'They edged away from me on the blanket.'

And, you know, as a so-called rational economist, I always found that offensive. Like, 'Come on, that's not nice. Why wouldn't you engage? Why wouldn't you ever say in response to that,'--and I've had many moments like that in my life; I've talked about it some on the program--but why wouldn't the other person say, 'Oh, that's interesting. I've never thought of that.'

That's not what they say. Almost no one says that. In fact, almost no one ever says, 'Oh, I always thought the minimum wage was good, but you're an economist, so you should know more about it than I do, so I guess I've been wrong all my life.' That never happens. They never even say, 'Wow, how would that work? Is that possible?' Or, 'Is there evidence for that?' They edge away from you on the blanket, folks, and that's because they're not hearing the words. To the extent they're hearing them, they're hearing a threat. Not just a threat, 'I don't agree with you'. A threat that says, 'I'm not a nice person.' That's what they heard. They heard an announcement. He should have put a card around his neck, 'Stay away. I am not a nice person. I don't have a caring heart.' That's what they heard. And I think--well, anyway, your turn, Adam.

Adam Mastroianni: Yeah. Yeah. I mean, the vibe of that sentence, I think to most people is, 'I don't care about poor people.' Like, 'Let them starve.'

Russ Roberts: Yep.

Adam Mastroianni: And, which is why if you want to have a discussion about that, you have to begin by giving off good vibes and going, like, 'Look, we all agree that what we would like is for people who work, including low-skilled, undesirable jobs, that they can make a decent living that allows them to live a good life. Now, we might think that the best way to do that is to make it illegal to pay them below a certain amount of money.' That makes total intuitive sense.

But, I have some evidence or there's a lot of evidence that that doesn't actually get us the end that we want. This is actually--I've run some studies on people being able to convince one another, and one of the issues that we work on is minimum wage. And, we ask people, 'Is there anything that could change your mind about these issues?'

So, some of them are, like, abortion or gun control, and often people say, 'No, there's nothing.'

But, minimum wage is one where people say, 'If you could show me that it doesn't actually get the thing that I want to get, I would change my mind on it.'

Now, whether that's actually true or not is a separate issue, but I think this is one where, if you can peel--if you can get the vibes right, I think people might actually be able to receive the data itself.

This is tough for me, too, because it feels so intuitively right: we should just make it illegal to do the thing that we don't want. And the idea that we're going to allow people to do whatever they want feels like, 'But no, that doesn't give us what we want. You have to make it mandatory.'


Russ Roberts: One of the great advantages of a decent economics class is that even if you don't remember anything in particular, you might remember that there is an occasional unintended consequence of a well-intentioned regulation or law, piece of legislation. That's a powerful thing. That's a great thing to have in your mental toolbox.

So, again, I'm a little bit skeptical of the fact that we learn--that it's mostly vibes. I think there are some mental lessons we learn, even if we can't articulate them well.

But I want to come back--let me make that point in a different way. So, a famous example of unintended consequences: The city has too many snakes, so they put a bounty on snakes, and if you bring a snake to the snake office, you get paid. You get a reward. So, the idea is, 'Oh, people will go out and trap snakes and we'll have fewer of them.'

Of course, if you pick the wrong amount, people will raise snakes. And that would be defeating the purpose of the legislation.

And: that's a story. And, I am--as I get older and read pieces like yours--I am overwhelmed by how much people remember lessons that are delivered by stories, fables, parables, jokes; things that rhyme; poems, songs, etc. And they're underutilized. They're considered cheesy, a little bit. It's, like, bad form to, when you can give a formal lecture in mathematics, say, versus a story--that snake story I just told. I suspect a year from now, there might be some listeners who actually remember that, and there might even be some listeners who will see a different example and realize the snake story is the same set of unintended incentives. So, I think stories, the jokes are undervalued.

Adam Mastroianni: Yeah. I mean, I think what that story does--what it does for me when I first heard it--is that it evokes this feeling of, 'Ohhhh!' The way I first heard the story, there's two turns in it. The first turn is: People start raising snakes. I go, 'Oh yeah, people would do that.' I hadn't really thought through the implications of this, but yeah, if I was in this situation, it makes total sense to raise the snakes.

And then, the second turn in the story is, 'Oh no, we're paying people to raise snakes. We're not going to pay bounties on snakes anymore.'

And then I'd go, 'Ohhh. People would release the snakes into the wild. Then I'd go, 'Ohhh, yeah. Of course they would do that. Now if they're not going to pay for my snakes anymore, I'm just going to let them go again.' I think that 'Ohhh,' that feeling, that is to me what the vibe is.

And I think the explicit knowledge of this is, like, a checklist thing where you say, like, 'Make sure for every policy you check for unintended consequences.'

I think that's actually much less effective than this feeling of: I feel a little bit suspicious when we do things, that there might be something weird that happens.

I think that's actually the important thing: that when you hear about policies that people want to put in place, you just get a little, like, 'Euhh, I don't know,' and that is what encourages you to go looking for, like, 'Well, but that might cause this thing and this thing and this thing.'

And I think that's the vibe at work, rather than, like, the explicit knowledge of the story. I don't think people even need to refer back to, like, 'Well, isn't this like the snakes and paying for them?' I think it is internalizing this feeling of skepticism and almost queasiness that the world is very simple and you always get exactly the things that you are trying to pay for. That's the vibe to me.

Russ Roberts: Yeah, and it's an 'Aha.' It's an aha moment. It's the, 'And then what?' that economists often want to get people to imagine and start thinking about.


Russ Roberts: I'm going to tell you another story if I might. And I'm probably going to get it wrong because I can't remember the details, but it doesn't matter. It's about a prisoner of war camp, or some kind of setting where there's guards and there's workers. And every night or once a week, maybe, one of the workers comes to the front gate. So, he's going back to his outside-the-camp life; and he's got a wheelbarrow with a tarp over it. And the guard is suspicious, 'What's this guy stealing?' And he looks under the tarp and he can't find anything.

And they play this game for a couple of years, till finally the guard is reassigned to another camp. And before he leaves, he goes to the prisoner or the worker in the prisoner work camp, and he says, 'You know, I just had a suspicion, you know, that you're smuggling something out of the camp and I've never been able to find it. Am I right?'

He says, 'Yeah, I was smuggling something.' He goes, 'What was it?' He goes, 'Wheelbarrows.'

So, that to me is a metaphor for what we're talking about in the classroom. And of course it applies to EconTalk--this weird experience of doing 900-plus hours of conversation with people. I think I'm smuggling out knowledge, but it turns out I'm smuggling wheelbarrows. People are learning other things from this than the content of the conversations that take place. Which is very flattering, and I've embraced it. I think it's a beautiful thing.

But, what is the implication of that for being a faculty member in a normal economics or psychology department? If you're listening to this and you recognize this as a reality and a truth, how should it change the way you run your class?

And, one way is you could say you should tell more stories, you should be more--but, let's suppose that I really care about the vibe part of it.

I don't want to react to your insight and say, 'I guess I better ask them to repeat the equations more to their friends outside of the classroom.' Right? Which is a classic memory trick: You should repeat things you want to retain. It works a little bit, maybe. I don't think it's a bad idea. Actually, you have to do it three times. If it didn't work, it's because you didn't do it enough. So, I don't know if any of that's true.

But, let's suppose you're faced with this reality. You're going to spend hours with this group of students. Should it change what you do or should you just say, 'Well, nothing I can do about it.'

Adam Mastroianni: I think it should change what you do. So, how it changes for me is--I can best tell this through the example of how I tried to teach students the R programming language, which is, like--I don't know any other programming languages, so maybe this is an especially counterintuitive one, but there's a real hump that you have to get over at the beginning where you start to think like the programming language wants you to think, and students find this very difficult. In a class that is meant to teach students the R programming language, obviously you have to spend some time doing the things--that sort of multiple choice knowledge. It's not that we want anyone to, like, memorize the commands and we test them on them later, but you do need some of that. So, we do work on some of that.

But, the most important thing that I can get students to understand is that it's okay that they are going to feel like it's stupid and pointless--and, like, byzantine and Kafkaesque--how they interact with this programming language. And that's fine. Because, most students feel like something's gone wrong when they hit that wall: that they, like, do something and it doesn't work. They go, 'Well, I'm not doing it right.' Like, 'Why is it so dumb?'

If I can get them to understand the vibe of, 'It's okay to feel dumb,' I still--I've worked with this language for 10 years and I still feel dumb when I interact with it--that, like, in fact, you need to accept feeling dumb in order to interact with this language. Well, if I can show them that through the way that I interact with it, that I type stuff in and fail, and then I go, 'Oh, I've just got to Google that,' and to show that I'm not losing passion for doing this as I'm interacting with it. And the students who pick up on that vibe, who come along with it, who go, like, 'Oh, this is a different way of learning for me than I'm used to,' they're the ones who continue using the language afterward. Like, they're the ones who actually learn it.

The ones who try to master it in the way that they would master multiplication tables of just memorization, they don't get anywhere. It doesn't submit to that kind of learning.

So, that's part of the difference: That I project to them the vibe of, like, 'I'm pretty stupid at this, and that's fine.' Like, I get by doing this thing. I spend a lot of time on Stack Overflow. This might all be out of date now that you can just ask ChatGPT how to do your coding for you. But, even there, it's like: Look, you may get to the point one day where you speak this fluently. That takes a long time, and to get there, you really have to use it all the time. You're probably not going to do that. What I would rather convey to you is this vibe of: It's all right not to know stuff. It's okay to look stuff up all the time. And only if you get that vibe will you actually get out of this language what it has to give you. So, that is I think the difference for going for a VOS[?]-based approach.


Russ Roberts: I think there's a profound implication of that, both for parents and teachers, which is--I think we already know this, but it's really hard to actually behave accordingly--which is that when your students ask you a question or a child asks you a question, a lot of times the right answer is not to tell them the right answer. And you write about this in the essay. They'll ask a question, 'How do I do that? I'm really frustrated.' And you don't tell them. And you don't not tell them by saying, 'Well, you're going to have to figure it out for yourself. That's part of the class.' You just say, 'I don't remember that either. I usually just Google it.' And there are probably times when you do remember it, but you want them to go through that.

And, so often, because we do not appreciate the lesson that you're conveying in this essay, we think the right way to educate is to make sure they know the answer. So we tell them the answer. And we write the essay for our kid because they'll get a good--not that they'll get a good grade--so they'll know what the right way is to write a good essay. That's worse than a waste of time. Not only did they not learn something, they learn something that's not helpful, which is to give up or to seek a different source.

And I think the implication, also, is very important in the era of ChatGPT. As I think many colleges are, we're trying to figure out here at Shalem College: I don't want rules on ChatGPT for our students because I don't want enforce them. I don't even try to figure out how I could enforce them. But, I do want them to believe that if they ask ChatGPT to figure out what this poem means or this passage, they won't experience the full effect because the goal is to figure out what the passage means. Part of the goal is to experience the struggle and the knowledge that comes from the nth time. And that's hard. We don't live in an era where that kind of discipline and perseverance is generally rewarded. We make life easy for everybody as much as possible. So, it's going to be an interesting time as we deal with this.

Adam Mastroianni: Yeah. I think part of what implies to students that it's not okay to struggle is that we evaluate them. This is another essay I wrote about wanting to be a teacher, but that[?] may be a cop[?], where, if I try to tell students, they're,[?] like, 'It's okay to be stupid about this and to not know how it works and to Google stuff,' and then six weeks later I go, 'Now is test day and laptops down. Just tell me all the things that you memorized,'--I was lying to them. Right? I'm not conveying the vibe that I actually wanted to convey.

If it is true that you can struggle and fail and the thing that you're supposed to figure out is how to figure out things, then I should create an environment where you are allowed to figure out things. And if I'm going to evaluate you, I have to figure out a way of evaluating you that evaluates your ability to figure out things. I think that's sort of at odds with the thing that it's trying to assess. So--

Russ Roberts: And that's--

Adam Mastroianni: Yeah, so go ahead.

Russ Roberts: No, you go ahead.

Adam Mastroianni: So, if you want people to figure stuff out, I don't think you give them a multiple choice question test at the end. I think that heavily implies to them that, like, the vibe here is that knowledge is arbitrary, and the point of learning is to satisfy my whims and to divine my intentions: to know the things that I am going to ask you later. And I think that is exactly the wrong vibe that we want to give people.

Russ Roberts: Yeah, I used to do that pretty ruthlessly and relentlessly, actually. I used to tell my students, 'Your job in this course is to learn how to think like I do. At the end of it, you might decide it's a bad way to think, but that's how you're going to be evaluated. And I'm not going to tell you what the assumptions you should make to help you answer a question. My questions are a little bit open-ended and the best answers are the ones I deem interesting. And you have to learn what that is.' And that's really cruel, actually. It was hard and a lot of people didn't like it. And I now, older, understand that.

Having said that, students who managed to get over that hump got a tool that many of them, I think, still use--which is deeply gratifying.


Russ Roberts: Your remark about tricking them or seemingly misleading them reminds me of something else you wrote recently--about science, and wouldn't it be nice to live, not in a classroom where I tell you what I know about the subject, but where we explore a subject together. You called it Science House.

So, you have a bunch of really smart people who hang out together and actually through conversation rather than lecturing learn to--as the novices learn, get initiated into the experience of science--and the so-called experts convey that, but in a different way than they do in a classroom. It's not dissimilar from the stories that Ed Leamer told of--and we'll put a link up to this episode; it's one of my all-time favorites--of how he helped a student understand something rather than telling it to her.

So, that's a lovely idea. It has no viability in today's world because students are buying a credential in most situations and we are providing that as educators and charging them to provide the credential, which is going to be a multiple choice exam that you cleared a hurdle of 95 or 90 or whatever it is, or 80, depending on the test.

As opposed to--again, I like to think what we try to do here at Shalem College, we're definitely swimming against the tide. We're basically saying to people, 'Come explore with us for a while and you'll grow from it and you'll end up making your country better because you'll be a better human being. You won't know necessarily as much as somebody who takes a multiple choice test that day of the exam, but 10 years from now, you'll be a different person.' I think that's what education should be. It's clear that the model in most societies, most systems, is credentialing. And I think that is--I don't know if it's tragic or not, but it's not education. It's something else.

Adam Mastroianni: Yeah. Something that I've realized about the world is everybody is looking for someone else to do their homework for them. Like, that is the point of a credential, is: I can't tell how good you are, or I don't want to do the work that it would take to know if you have the skills necessary to do the thing that I want you to do, or I don't even understand the skills well enough to do this. Or, I know if you are trying to get a programming job, they'll ask you to program, but if you're trying to get a managing job, they don't ask you to manage, right? Because they probably don't even know what it takes to manage well enough. So, what they're looking for is some signal that you know that. And so they look, 'Well, do you have a degree and where is it from and how good are the grades that you got when you went there?' All of these things that are, I think, really orthogonally related to whether you actually possess the traits that they want you to possess.

And that actually assessing people takes a lot of work, work that no one is really willing to do. And so, instead, we all want to outsource it to other people.

And so, you go pay $60,000 a year to get assessed, and then come to me and show me what your assessment says.

But the people doing the assessment, they are not doing the thing that you might hope that they would be doing. They're doing their own thing--that their incentives lead them to do it, in their little world.

When I am teaching a class, I don't care about later telling McKinsey or Goldman Sachs whether this person would be a good employee for them. I'm trying to teach them psychology.

And so all of us are hoping that someone else in the chain is going to save us time, but no one really seems to do it. Which is why I dream of this place where we come and work together and we have this freedom. You eventually can just do things that make it clear to people the skills that you have. Like, you can't fake asking a question about the world, collecting data on it, and writing it up in an accessible way.

That, like, if you can produce an interesting blog post about a study that you ran that anyone could read and understand, that even someone who is not putting that much work into assessing you, can tell that you know what you're doing.

Whereas, if what you do is produce journal articles that obey all the norms of the field and they try to be inaccessible to outsiders, unless you are an expert in that field, I don't know if you wrote a good economics paper or not.

And so, I'm just looking at, like, 'Well, I do know which journals are good. Is this in a good journal? Okay, then you must be a good practitioner of this.' Which is why I have this hope that, like, if you actually conveyed the vibes well--if you actually taught people well--they would be able to do things that make assessing them very easy.

And the point wasn't ever to make them very accessible. The point was to do the things that we all wanted to do in the first place.

That's my hope, anyway.

Russ Roberts: Yeah, and I think part of that, of course, is that the skills that we want in an employee or a colleague are I often difficult to measure.

Just to take an example, I want a thoughtful person who can ask good questions. That's worth a lot to me--as a friend; forget as a scientific colleague. A lot of--it covers a lot of ground for me. How do you know who that person is?

Well, in real life, what you do is you have a lot of conversations with them, and you figure out who is interesting and who isn't; and you spend your time accordingly.

But, it's very hard to assess managerial ability. It's very hard to assess integrity. It's very hard to assess a bunch of things. So, we use screening and other types of forms of activity that--you're right, we outsource it; and we know it's crude and it might be better than we're able to do.


Russ Roberts: I think there's something else--I don't know if we'll find it today--but: I think I've learned something profound from your essay about thinking about conversation.

I'm not sure I've gotten to the sweet spot for thinking about education. And I think it's a lot more than simply how overcoming, say, an intellectual challenge makes you feel or the confidence you gain from that. Those things are not unimportant.

But, for something that we spend billions of dollars on publicly and privately education, it's interesting how little we've thought about these questions. At one point you say, 'I'd like to go to the literature.' There isn't any on this question of vibe, feeling, emotion. The fact that those are not just accessible, but they're the first thing that comes to mind--just without thinking about it, one of my favorite classes in college--I'm not going to say what the subject was, but I remember two things about it: It made sense. I loved that. It fit together.

And the second thing was is that, in office hours one time, the professor told me to get an article published, how excited he was. He lectured for hours. I don't remember any of his lectures, but I do remember that he was really proud of it. And then he confessed that after the third or fourth one, it's not that exciting, and, kind of the thrill is gone. Gosh.

Adam Mastroianni: Wow. Liberal Arts universities will tell you that what you are doing there is learning how to learn. Which sounds good. But then, I don't think there's actually a strong theory about how the learning-how-to-learn happens. That there's this trust that: Yeah, you take a philosophy class and you take a math class and you take a biology class, and the emergent property of taking all those classes is you learn how to learn.

But, none of those individuals I think are on mission for teaching people how to learn. They're there to teach people biology--like, that's what's on the syllabus.

I think, in fact, the incentives are structured such that you often learn the opposite. What you learn is how to get good grades in these classes, because that's what you're assessed on. So why would you learn anything else? That's the way that you make yourself legible to the university--and to people afterward who are looking at your transcript.

So, what would it really take to teach people how to learn? I think it might involve letting go of assessment. And I think a lot of it would involve not just trusting that there's some emergent property that you get from taking many different classes--that, you need to approach it in some different way.


Russ Roberts: No, I think that's very important. I think that's a statement about the nature of the Ph.D. experience, and how--you become a teacher, you tend to teach what you were taught. If it's at the undergraduate level, you dumb down what you were taught at the graduate level. You don't think about that many of these students will never go on to graduate school. It may be the only economics or psychology class they're going to take, and therefore maybe you should teach it differently.

The idea that we might structure economics and biology and psychology and math as a regime of practice rather than filling up the bucket with facts and regularities is alien to everything that our graduate education system does to the people we end up hiring as teachers. And, that's a whole 'nother conversation.

But, you're right. What teachers do is they play in their silo, and the best ones, like your M&M-thrower, inspire students. And I'm sure many of the students in that class loved that class, even if they didn't become professional psychologists as you chose to do, Ph.D.s, but it requires a different kind of education.

We have a member of our faculty who told me his job in his class is to help our students become the best people they can be. Well, I didn't get trained in that at Chicago in my Ph.D. program, but I don't think it should be alien to how I teach an undergraduate who is never going to be a Ph.D. in economics, and yet I don't have any--never thought about it until recently.

Adam Mastroianni: Yeah. The class I taught most recently is Negotiation, and this is a class where--there's so much that we convey to students I think in the parts of the class that we don't think that much about. What students do a lot is they do mock negotiations, and I think we transmit the vibe in what those negotiations are about. Really, we think of those negotiations--it's just like it's, like, really just a cover story for creating a situation where people can work on the concepts that we want them to work on. But, the fact that we are doing a negotiation about buying a vineyard--actually that means something: like, we are conveying to students that is the kind of negotiation that we expect them to encounter in the world. I think there's a reason why it's negotiation for a vineyard rather than a negotiation for a parking lot; and those things mean different things. We are telling these students that we expect them to be upper-class people who would buy vineyards, which in the case of Columbia Business School, I think maybe isn't inaccurate, but is a weird value to be conveying at the university level.

And I think no one has ever really thought about what it meant that the negotiation is about a vineyard. They care a lot about which concepts we hit, and those do matter. But, if we're really trying to make people into good people, we should care a lot about what the cover story is or what the situation is that we are putting people in.

Russ Roberts: And you're newly married, and--

Adam Mastroianni: Yes--

Russ Roberts: much of marriage is about negotiation, and it's negotiation that has no price tags, usually. There are some economists, maybe; but most human beings aren't negotiating with their spouses over--they're not haggling over price. They're haggling over responsibility; they're haggling over a variety of other things. Doing that as a full human being, it seems like a very good skill that's never taught as far as I know--maybe by some marriage counselors--but it's not in that class. And it belongs in there, it seems to me.

Adam Mastroianni: Yeah. I think if we took seriously the mission of making people better people, I would have taught that class differently: that, the guy who trained me how to teach negotiation did describe it as a process of making jerks into less jerks--which I do think is true--but I never felt like I was actually given free-enough reign to do that. And maybe that's beyond the scope of what Columbia University thinks it's able to do--that it requires staking a claim in some values. You can't just go, like, 'Well, some of you are going to go cut down rainforests. Some of you are going to go run hospitals; and these are the same thing, and that's all good as long as you give money to us later.' I think you have to be willing to say, like, 'It is possible for you to go do jobs that we think are bad, and we would really prefer that you don't do them.'

I don't know--that obviously opens a dangerous can of worms. But I do want to be able to talk to my students that, like: There are bad actors in the world, and some of you could become them. Right? Like, these people come from somewhere; they go get MBAs [Masters of Business Administration]; I want to prevent you from becoming a bad actor. That's part of what could happen in this negotiation class: that, I want you to use these skills for good. And so, I want to put you in situations where perhaps you might be tempted to use them for bad. And the lesson is how to use them for good.

It doesn't really happen. I try to tell them not to lie, and I try to put them in situations where lying actually sometimes ends up with worse outcomes, but it's difficult; and I didn't feel like I actually had legitimacy or the institutional mandate and support to do that.

Russ Roberts: Yeah but you're a psychologist, so you could also teach them how to overcome their natural inclination to be a bad actor. In theory, you could--I don't know if you actually could--but that would be kind of cool. If they wanted. You could give them the tools, if they wanted. They didn't want to be a bad actor, and maybe you're going to give them--you're going to teach them R so they can use data to be an even worse actor. R is pretty value-neutral. In theory, negotiation is, too; but of course it's not. It's part of a bigger picture.

Adam Mastroianni: Yeah. I mean, the other lever that I had to pull--right?--is in the way that I organized my class. And this is a way that I tried to convey the vibes that I thought would make people better people: which was, for one thing, trusting them and showing that I trusted them. That, so many classes have these baroque policies for what happens if you don't come to class, and sometimes they're punitive--you know: You have to come or I'll take points away, or you have to get a doctor's note.

And so, I just said, 'It is important to come to class. It's a very experiential class, and other people benefit from you being here, too. So, please do come to class. It is part of your grade. If you're not going to be here, all you have to do is fill out this form that says you're not going to be here, and you can click this button that says, like, 'I have a good reason not to be here,' and I will trust you. And if you use this for nefarious purposes, all you're doing is stealing Monopoly Money [play-money from the board game Monopoly--Econlib Ed]. These points are made up. I really hope that you don't do it. Like, I would be disappointed if you were, but I'm not going to police you because I think this is the way that humans should relate to one another. So, that's how I choose to treat you, because I would like to be treated like this as well.'

And I hope that that did a little bit to, like, produce this vibe of, like, 'Oh, isn't it good when we trust one another?' And yeah, actually people can abuse systems, but wouldn't you rather be in a system that treated you well, that could be abused rather than you get policed just as much as the person who is trying to do the bad thing, and it turns out that the person stealing the point, it's like they don't really do anything to you. Most of you are just going to get As. In fact, I'm going to break the curve a little bit because I think the curve is dumb. That's what I try to do.

Russ Roberts: My guest today has been Adam Mastroianni. Adam, thanks for being part of EconTalk.

Adam Mastroianni: Thanks for having me.

More EconTalk Episodes