Steven Pinker: AI in the Age of Reason

00:00:00.000 | You've studied the human mind, cognition, language, vision, evolution, psychology, from

00:00:05.600 | child to adult, from the level of individual to the level of our entire civilization, so I feel

00:00:12.240 | like I can start with a simple multiple choice question. What is the meaning of life? Is it

00:00:19.440 | A) to attain knowledge, as Plato said? B) to attain power, as Nietzsche said? C) to escape

00:00:26.560 | death, as Ernest Becker said? D) to propagate our genes, as Darwin and others have said? E)

00:00:33.760 | there is no meaning, as the nihilists have said? F) knowing the meaning of life is beyond our

00:00:40.080 | cognitive capabilities, as Stephen Pinker said, based on my interpretation 20 years ago? And

00:00:46.160 | G) none of the above? I'd say A comes closest, but I would amend that to attaining not only

00:00:52.960 | knowledge, but fulfillment more generally. That is, life, health, stimulation, access to the

00:01:02.080 | living cultural and social world. Now this is our meaning of life, it's not the meaning of life,

00:01:09.520 | if you were to ask our genes. Their meaning is to propagate copies of themselves,

00:01:16.160 | but that is distinct from the meaning that the brain that they lead to sets for itself.

00:01:22.400 | So to you, knowledge is a small subset or a large subset?

00:01:27.840 | It's a large subset, but it's not the entirety of human striving, because we also want to

00:01:34.000 | interact with people, we want to experience beauty, we want to experience the richness of

00:01:39.680 | the natural world. But understanding what makes the universe tick is way up there.

00:01:47.840 | For some of us more than others, certainly for me that's one of the top five.

00:01:54.000 | So is that a fundamental aspect? Are you just describing your own preference,

00:01:59.520 | or is this a fundamental aspect of human nature, is to seek knowledge? In your latest book,

00:02:05.280 | you talk about the power, the usefulness of rationality and reason and so on. Is that a

00:02:10.880 | fundamental nature of human beings, or is it something we should just strive for?

00:02:16.960 | It's both. We're capable of striving for it, because it is one of the things that

00:02:21.840 | make us what we are — Homo sapiens, wise men. We are unusual among animals in the degree to which

00:02:31.600 | we acquire knowledge and use it to survive. We make tools, we strike agreements via language,

00:02:39.760 | we extract poisons, we predict the behavior of animals, we try to get at the workings of plants.

00:02:47.760 | And when I say "we," I don't just mean "we" in the modern West, but "we" as a species everywhere,

00:02:52.640 | which is how we've managed to occupy every niche on the planet, how we've managed to drive other

00:02:58.160 | animals to extinction. And the refinement of reason in pursuit of human well-being, of health,

00:03:06.480 | happiness, social richness, cultural richness, is our main challenge in the present. That is,

00:03:13.680 | using our intellect, using our knowledge to figure out how the world works, how we work,

00:03:19.280 | in order to make discoveries and strike agreements that make us all better off in the long run.

00:03:25.200 | Right. And you do that almost undeniably and in a data-driven way in your recent book. But I'd like

00:03:32.320 | to focus on the artificial intelligence aspect of things, and not just artificial intelligence,

00:03:37.360 | but natural intelligence too. So, 20 years ago, in the book you've written on how the mind works,

00:03:42.880 | you conjecture, again, am I right to interpret things? You can correct me if I'm wrong, but

00:03:50.480 | you conjecture that human thought in the brain may be a result of a massive network of highly

00:03:56.000 | interconnected neurons. So, from this interconnectivity emerges thought. Compared to artificial

00:04:02.480 | neural networks, which we use for machine learning today, is there something fundamentally more

00:04:08.640 | complex, mysterious, even magical about the biological neural networks versus the ones

00:04:14.240 | we've been starting to use over the past 60 years and have become to success in the past 10?

00:04:22.240 | There is something a little bit mysterious about the human neural networks, which is that

00:04:28.640 | each one of us who is a neural network knows that we ourselves are conscious.

00:04:32.800 | Conscious not in the sense of registering our surroundings or even registering our internal

00:04:38.080 | state, but in having subjective first-person present-tense experience. That is, when I see

00:04:43.680 | red, it's not just different from green, but there's a redness to it that I feel.

00:04:50.400 | Whether an artificial system would experience that or not, I don't know, and I don't think I

00:04:54.720 | can know. That's why it's mysterious. If we had a perfectly lifelike robot that was behaviorally

00:05:00.480 | indistinguishable from a human, would we attribute consciousness to it, or ought we to attribute

00:05:06.800 | consciousness to it? That's something that it's very hard to know. But putting that aside,

00:05:12.160 | putting aside that largely philosophical question, the question is, is there some difference between

00:05:18.960 | the human neural network and the ones that we're building in artificial intelligence will mean that

00:05:23.840 | we're, on the current trajectory, not going to reach the point where we've got a lifelike robot

00:05:30.400 | indistinguishable from a human, because the way their so-called neural networks are organized are

00:05:35.600 | different from the way ours are organized? I think there's overlap, but I think there are

00:05:39.760 | some big differences that the current neural networks, current so-called deep learning systems

00:05:48.560 | are in reality not all that deep. That is, they are very good at extracting high-order

00:05:53.280 | statistical regularities, but most of the systems don't have a semantic level, a level of actual

00:06:00.000 | understanding of who did what to whom, why, where, how things work, what causes what else.

00:06:06.400 | - Do you think that kind of thing can emerge as it does? So artificial neural networks are much

00:06:10.960 | smaller, the number of connections and so on, than the current human biological networks,

00:06:16.080 | but do you think, sort of, to go to consciousness or to go to this higher level semantic reasoning

00:06:22.160 | about things, do you think that can emerge with just a larger network, with a more richly,

00:06:26.960 | weirdly interconnected network? - Separate, again, consciousness,

00:06:31.360 | because consciousness isn't even a matter of complexity. - It's a really weird one.

00:06:34.080 | - Yeah, you could sensibly ask the question of whether shrimp are conscious, for example. They're

00:06:38.560 | not terribly complex, but maybe they feel pain. So let's just put that part of it aside. But

00:06:45.360 | I think sheer size of a neural network is not enough to give it structure and knowledge,

00:06:51.760 | but if it's suitably engineered, then why not? That is, we're neural networks, natural selection

00:06:58.640 | did a kind of equivalent of engineering of our brains, so I don't think there's anything

00:07:03.280 | mysterious in the sense that no system made out of silicon could ever do what a human brain can

00:07:10.560 | do. I think it's possible in principle. Whether it'll ever happen depends not only on how clever

00:07:16.160 | we are in engineering these systems, but whether we even want to, whether that's even a sensible

00:07:21.200 | goal. That is, you can ask the question, is there any locomotion system that is as good as a human?

00:07:29.520 | Well, we kind of want to do better than a human, ultimately, in terms of legged locomotion.

00:07:33.520 | There's no reason that humans should be our benchmark. They're tools that might be better

00:07:39.360 | in some ways. It may be that we can't duplicate a natural system because at some point it's so

00:07:49.280 | much cheaper to use a natural system that we're not going to invest more brainpower and resources.

00:07:54.240 | So for example, we don't really have an exact substitute for wood. We still build houses out

00:08:00.080 | of wood, we still build furniture out of wood. We like the look, we like the feel. Wood has

00:08:04.320 | certain properties that synthetics don't. It's not that there's anything magical or mysterious

00:08:09.920 | about wood. It's just that the extra steps of duplicating everything about wood is something

00:08:16.960 | we just haven't bothered because we have wood. Like when I say cotton, I mean, I'm wearing

00:08:20.160 | cotton clothing now. It feels much better than polyester. It's not that cotton has something

00:08:26.240 | magic in it. And it's not that we couldn't ever synthesize something exactly like cotton.

00:08:33.120 | But at some point it's just not worth it. We've got cotton. And likewise, in the case of human

00:08:37.760 | intelligence, the goal of making an artificial system that is exactly like the human brain

00:08:43.520 | is a goal that we probably no one is going to pursue to the bitter end, I suspect. Because

00:08:49.440 | if you want tools that do things better than humans, you're not going to care whether it

00:08:53.600 | does something like humans. So for example, diagnosing cancer or predicting the weather,

00:08:58.720 | why set humans as your benchmark? - But in general, I suspect you also believe

00:09:05.280 | that even if the human should not be a benchmark and we don't want to imitate humans in their

00:09:10.480 | system, there's a lot to be learned about how to create an artificial intelligence system by

00:09:15.520 | studying the human. - Yeah, I think that's right. In the same way that to build flying machines,

00:09:22.400 | we want to understand the laws of aerodynamics, including birds, but not mimic the birds.

00:09:27.440 | - Right, exactly. - But they're the same laws.

00:09:28.880 | - You have a view on AI, artificial intelligence and safety, that from my perspective is

00:09:39.600 | refreshingly rational, or perhaps more importantly, has elements of positivity to it,

00:09:49.360 | which I think can be inspiring and empowering as opposed to paralyzing. For many people,

00:09:55.440 | including AI researchers, the eventual existential threat of AI is obvious, not only possible,

00:10:02.080 | but obvious. And for many others, including AI researchers, the threat is not obvious.

00:10:07.680 | So Elon Musk is famously in the highly concerned about AI camp, saying things like AI is far more

00:10:16.640 | dangerous than nuclear weapons, and that AI will likely destroy human civilization. So in February,

00:10:24.080 | he said that if Elon was really serious about AI, the threat of AI, he would stop building

00:10:31.920 | self-driving cars that he's doing very successfully as part of Tesla. Then he said,

00:10:37.040 | wow, if even Pinker doesn't understand the difference between narrow AI, like a car,

00:10:41.520 | and general AI, when the latter literally has a million times more compute power and an open-ended

00:10:48.400 | utility function, humanity is in deep trouble. So first, what did you mean by the statement about

00:10:55.120 | Elon Musk should stop building self-driving cars if he's deeply concerned?

00:10:59.200 | - Not the last time that Elon Musk has fired off an intemperate tweet.

00:11:03.520 | - Yeah, well, we live in a world where Twitter has power.

00:11:07.440 | - Yes. Yeah, I think that there are two kinds of existential threat that have been discussed

00:11:16.640 | in connection with artificial intelligence, and I think that they're both incoherent.

00:11:19.760 | One of them is a vague fear of AI takeover, that just as we subjugated animals and less

00:11:28.800 | technologically advanced peoples, so if we build something that's more advanced than us,

00:11:33.360 | it will inevitably turn us into pets or slaves or domesticated animal equivalents.

00:11:40.160 | I think this confuses intelligence with a will to power, that it so happens that in the intelligence

00:11:47.200 | system we are most familiar with, namely Homo sapiens, we are products of natural selection,

00:11:52.640 | which is a competitive process, and so bundled together with our problem-solving capacity

00:11:57.760 | are a number of nasty traits like dominance and exploitation and maximization of power and glory

00:12:06.080 | and resources and influence. There's no reason to think that sheer problem-solving capability

00:12:12.080 | will set that as one of its goals. Its goals will be whatever we set its goals as, and as long as

00:12:17.680 | someone isn't building a megalomaniacal artificial intelligence, then there's no reason to think that

00:12:23.680 | it would naturally evolve in that direction. Now, you might say, "Well, what if we gave it the goal

00:12:27.120 | of maximizing its own power source?" Well, that's a pretty stupid goal to give an autonomous system.

00:12:34.400 | You don't give it that goal. I mean, that's just self-evidently idiotic.

00:12:38.560 | - So if you look at the history of the world, there's been a lot of opportunities where

00:12:42.800 | engineers could instill in a system destructive power, and they choose not to because that's the

00:12:47.680 | natural process of engineering. - Well, except for weapons. I mean,

00:12:50.800 | if you're building a weapon, its goal is to destroy people, and so I think there are good

00:12:55.200 | reasons to not build certain kinds of weapons. I think building nuclear weapons was a massive mistake.

00:13:00.320 | - You do. You think... So maybe pause on that because that is one of the serious threats.

00:13:07.760 | Do you think that it was a mistake in a sense that it should have been stopped early on,

00:13:14.080 | or do you think it's just an unfortunate event of invention that this was invented?

00:13:19.840 | Do you think it's possible to stop, I guess, is the question I'm asking.

00:13:22.320 | - Yeah, it's hard to rewind the clock because, of course, it was invented in the context of

00:13:25.760 | World War II and the fear that the Nazis might develop one first. Then once it was initiated

00:13:32.480 | for that reason, it was hard to turn off, especially since winning the war against the

00:13:38.880 | Japanese and the Nazis was such an overwhelming goal of every responsible person that there was

00:13:45.280 | just nothing that people wouldn't have done then to ensure victory. It's quite possible if World

00:13:50.320 | War II hadn't happened that nuclear weapons wouldn't have been invented. We can't know,

00:13:55.200 | but I don't think it was by any means a necessity any more than some of the other

00:13:59.200 | weapon systems that were envisioned but never implemented, like planes that would disperse

00:14:05.440 | poison gas over cities like crop dusters, or systems to try to create earthquakes and tsunamis

00:14:13.040 | in enemy countries, to weaponize the weather, weaponize solar flares, all kinds of crazy schemes

00:14:19.040 | that we thought the better of. I think analogies between nuclear weapons and artificial intelligence

00:14:24.720 | are fundamentally misguided because the whole point of nuclear weapons is to destroy things.

00:14:29.920 | The point of artificial intelligence is not to destroy things. The analogy is misleading.

00:14:36.080 | >>Corey: There's two artificial intelligence you mentioned. The first one that gets highly

00:14:39.920 | intelligent or power hungry.

00:14:41.920 | >>Kaiser: Yeah, in a system that we design ourselves, where we give it the goals. Goals

00:14:45.200 | are external to the means to attain the goals. If we don't design an artificially intelligent

00:14:54.240 | system to maximize dominance, then it won't maximize dominance. It's just that we're so

00:15:00.320 | familiar with Homo sapiens, where these two traits come bundled together, particularly in men,

00:15:06.320 | that we are apt to confuse high intelligence with a will to power, but that's just an error.

00:15:14.480 | The other fear is that will be collateral damage, that will give artificial intelligence a goal,

00:15:21.440 | like make paperclips, and it will pursue that goal so brilliantly that before we can stop it,

00:15:27.440 | it turns us into paperclips. We'll give it the goal of curing cancer, and it will turn us into

00:15:32.800 | guinea pigs for lethal experiments, or give it the goal of world peace, and its conception of

00:15:38.480 | world peace is no people, therefore no fighting, and so it will kill us all. Now, I think these

00:15:42.880 | are utterly fanciful. In fact, I think they're actually self-defeating. They, first of all,

00:15:48.240 | assume that we're going to be so brilliant that we can design an artificial intelligence that can

00:15:52.480 | cure cancer, but so stupid that we don't specify what we mean by curing cancer in enough detail

00:15:58.880 | that it won't kill us in the process, and it assumes that the system will be so smart that

00:16:04.800 | it can cure cancer, but so idiotic that it can't figure out that what we mean by curing cancer is

00:16:10.800 | not killing everyone. So I think that the collateral damage scenario, the value alignment problem,

00:16:16.880 | is also based on a misconception. - So one of the challenges, of course,

00:16:20.960 | we don't know how to build either system currently, or are we even close to knowing? Of course,

00:16:26.080 | those things can change overnight, but at this time, theorizing about it is very challenging

00:16:30.880 | in either direction, so that's probably at the core of the problem, is without that ability to

00:16:37.600 | reason about the real engineering things here at hand, is your imagination runs away with things.

00:16:43.360 | - Exactly. - But let me sort of ask,

00:16:45.120 | what do you think was the motivation and the thought process of Elon Musk? I build autonomous

00:16:51.840 | vehicles, I study autonomous vehicles, I study Tesla autopilot, I think it is one of the greatest

00:16:58.000 | currently large-scale applications of artificial intelligence in the world. It has potentially a

00:17:04.320 | very positive impact on society. So how does a person who's creating this very good, quote-unquote,

00:17:10.880 | narrow AI system also seem to be so concerned about this other general AI? What do you think

00:17:19.280 | is the motivation there? What do you think is the thinking process?

00:17:21.040 | - Well, you probably have to ask him, and he is notoriously flamboyant, impulsive,

00:17:29.600 | as we have just seen, to the detriment of his own goals of the health of a company.

00:17:35.040 | So I don't know what's going on in his mind, you probably have to ask him.

00:17:40.080 | But I don't think the distinction between special-purpose AI and so-called general AI is

00:17:47.360 | relevant, that in the same way that special-purpose AI is not going to do anything conceivable in

00:17:54.000 | order to attain a goal. All engineering systems are designed to trade off across multiple goals.

00:18:00.560 | When we built cars in the first place, we didn't forget to install brakes, because the goal of a

00:18:05.840 | car is to go fast. It occurred to people, yes, you want it to go fast, but not always, so you

00:18:12.080 | build in brakes too. Likewise, if a car is going to be autonomous, and program it to take the

00:18:18.960 | shortest route to the airport, it's not going to take the diagonal and mow down people and trees

00:18:23.360 | and fences, because that's the shortest route. That's not what we mean by the shortest route

00:18:27.440 | when we program it, and that's just what an intelligence system is by definition. It takes

00:18:34.000 | into account multiple constraints. The same is true, in fact, even more true of so-called

00:18:39.200 | general intelligence. That is, if it's genuinely intelligent, it's not going to pursue some goal

00:18:45.760 | single-mindedly, omitting every other consideration and collateral effect. That's not

00:18:52.720 | artificial and general intelligence, that's artificial stupidity. I agree with you, by the way,

00:18:58.560 | on the promise of autonomous vehicles for improving human welfare. I think it's spectacular,

00:19:03.200 | and I'm surprised at how little press coverage notes that in the United States alone, something

00:19:08.320 | like 40,000 people die every year on the highways, vastly more than are killed by terrorists. We

00:19:14.240 | spend a trillion dollars on a war to combat deaths by terrorism, about half a dozen a year,

00:19:20.560 | whereas year in, year out, 40,000 people are massacred on the highways, which could be brought

00:19:26.000 | down to very close to zero. I'm with you on the humanitarian benefit. Let me just mention that

00:19:32.960 | it's, as a person who's building these cars, it is a little bit offensive to me to say that

00:19:36.880 | engineers would be clueless enough not to engineer safety into systems. I often stay up at night

00:19:42.720 | thinking about those 40,000 people that are dying, and everything I try to engineer is to save those

00:19:48.480 | people's lives. So every new invention that I'm super excited about, every new, and in all the

00:19:55.600 | deep learning literature and CVPR conferences and NIPS, everything I'm super excited about

00:20:01.120 | is all grounded in making it safe and help people. So I just don't see how that trajectory can all

00:20:09.280 | of a sudden slip into a situation where intelligence will be highly negative.

00:20:13.600 | - You and I certainly agree on that, and I think that's only the beginning of the

00:20:17.920 | potential humanitarian benefits of artificial intelligence. There's been enormous attention to

00:20:24.320 | what are we gonna do with the people whose jobs are made obsolete by artificial intelligence,

00:20:28.720 | but very little attention given to the fact that the jobs that are gonna be made obsolete are

00:20:32.480 | horrible jobs. The fact that people aren't gonna be picking crops and making beds and driving trucks

00:20:39.760 | and mining coal, these are soul-deadening jobs, and we have a whole literature

00:20:44.720 | sympathizing with the people stuck in these menial, mind-deadening, dangerous jobs.

00:20:51.280 | If we can eliminate them, this is a fantastic boon to humanity. Now granted, you solve one

00:20:57.520 | problem and there's another one, namely how do we get these people a decent income, but if we're

00:21:03.520 | smart enough to invent machines that can make beds and put away dishes and handle hospital patients,

00:21:10.640 | I think we're smart enough to figure out how to redistribute income to apportion some of the

00:21:16.320 | vast economic savings to the human beings who will no longer be needed to make beds.

00:21:21.520 | - Okay, Sam Harris says that it's obvious that eventually AI will be an existential risk.

00:21:28.400 | He's one of the people who says it's obvious. We don't know when the claim goes, but eventually

00:21:36.240 | it's obvious, and because we don't know when, we should worry about it now. It's a very interesting

00:21:41.760 | argument in my eyes. So how do we think about timescale? How do we think about existential

00:21:49.120 | threats when we don't really, we know so little about the threat, unlike nuclear weapons perhaps,

00:21:55.040 | about this particular threat, that it could happen tomorrow, right? But very likely it won't.

00:22:02.320 | - Yeah, definitely.

00:22:02.960 | - Very likely it'd be 100 years away. So how do we ignore it? How do we talk about it?

00:22:08.320 | Do we worry about it? How do we think about those?

00:22:11.440 | - What is it?

00:22:13.920 | - A threat that we can imagine. It's within the limits of our imagination,

00:22:18.560 | but not within our limits of understanding to accurately predict it.

00:22:23.360 | - But what is the it that we're afraid of?

00:22:26.320 | - Oh, AI, sorry, AI being the existential threat. AI can always--

00:22:31.200 | - But how? Like enslaving us or turning us into paperclips?

00:22:34.400 | - I think the most compelling from the Sam Harris perspective would be the paperclip situation.

00:22:38.720 | - Yeah, I just think it's totally fanciful. I mean, don't build a system. First of all,

00:22:45.600 | the code of engineering is you don't implement a system with massive control before testing it.

00:22:52.000 | Now, perhaps the culture of engineering will radically change, then I would worry,

00:22:56.080 | but I don't see any signs that engineers will suddenly do idiotic things, like put an

00:23:01.200 | electrical power plant in control of a system that they haven't tested

00:23:06.000 | first. Or all of these scenarios not only imagine almost a magically powered intelligence,

00:23:14.720 | including things like cure cancer, which is probably an incoherent goal because there's

00:23:20.160 | so many different kinds of cancer, or bring about world peace. I mean, how do you even

00:23:24.880 | specify that as a goal? But the scenarios also imagine some degree of control of every molecule

00:23:31.040 | in the universe, which not only is itself unlikely, but we would not start to connect

00:23:37.440 | these systems to infrastructure without testing as we would any kind of engineering system.

00:23:45.680 | Now, maybe some engineers will be irresponsible, and we need legal and regulatory and legal

00:23:53.840 | responsibility implemented so that engineers don't do things that are stupid by their own standards.

00:24:00.640 | But I've never seen enough of a plausible scenario of existential threat to devote large

00:24:08.560 | amounts of brain power to forestall it. >>Corey: So you believe in the power en masse

00:24:14.640 | of the engineering of reason, as you argue in your latest book of Reason and Science, to be

00:24:20.640 | the very thing that guides the development of new technologies so it's safe and also keeps us safe.

00:24:26.400 | >>Kaiser: Yeah, if the same — granted, the same culture of safety that currently is part of the

00:24:32.480 | engineering mindset for airplanes, for example. So yeah, I don't think that that should be thrown

00:24:39.360 | out the window and that untested, all-powerful systems should be suddenly implemented. But

00:24:44.880 | there's no reason to think they are. And in fact, if you look at the progress of artificial

00:24:49.120 | intelligence, it's been impressive, especially in the last 10 years or so. But the idea that

00:24:53.840 | suddenly there'll be a step function, that all of a sudden, before we know it, it will be

00:24:58.400 | all-powerful, that there'll be some kind of recursive self-improvement, some kind of

00:25:03.520 | fume, is also fanciful. Certainly by the technology that now impresses us, such as

00:25:12.560 | deep learning, where you train something on hundreds of thousands or millions of examples,

00:25:18.320 | they're not hundreds of thousands of problems of which curing cancer is a typical example.

00:25:25.200 | And so the kind of techniques that have allowed AI to increase in the last five years are not

00:25:31.440 | the kind that are going to lead to this fantasy of exponential, sudden self-improvement.

00:25:38.320 | >>Zubin: So —

00:25:39.200 | >>Kaiser: I think it's kind of a magical thinking. It's not based on

00:25:42.880 | our understanding of how AI actually works.

00:25:45.120 | >>Zubin: Now, give me a chance here. So you said fanciful, magical thinking.

00:25:49.280 | In his TED Talk, Sam Harris says that thinking about AI killing all human civilization is somehow

00:25:55.360 | fun, intellectually. Now, I have to say, as a scientist and engineer, I don't find it fun.

00:26:00.480 | But when I'm having beer with my non-AI friends, there is indeed something

00:26:06.400 | fun and appealing about it. Like talking about an episode of Black Mirror, considering

00:26:12.080 | if a large meteor is headed towards Earth — we were just told a large meteor is headed towards

00:26:17.440 | Earth, something like this. And can you relate to this sense of fun? And do you

00:26:23.040 | understand the psychology of it?

00:26:24.480 | >>Kaiser: Yes, right. Good question. I personally don't find it fun. I find it

00:26:30.400 | kind of actually a waste of time, because there are genuine threats that we ought to be thinking

00:26:36.720 | about, like pandemics, like cybersecurity vulnerabilities, like the possibility of

00:26:44.160 | nuclear war, and certainly climate change. This is enough to fill many conversations.

00:26:51.200 | And I think Sam did put his finger on something, namely that there is a community,

00:26:57.040 | sometimes called the rationality community, that delights in using its brainpower to come up with

00:27:05.360 | scenarios that would not occur to mere mortals, to less cerebral people. So there is a kind of

00:27:12.640 | intellectual thrill in finding new things to worry about that no one has worried about yet.

00:27:17.440 | I actually think, though, that it's — not only is it a kind of fun that doesn't give me

00:27:22.160 | particular pleasure, but I think there can be a pernicious side to it, namely that you

00:27:27.840 | overcome people with such dread, such fatalism, that there's so many ways to die to annihilate

00:27:35.840 | our civilization, that we may as well enjoy life while we can. There's nothing we can do about it.

00:27:40.720 | If climate change doesn't do us in, then runaway robots will. So let's enjoy ourselves now.

00:27:46.880 | We've got to prioritize. We have to look at threats that are close to certainty,

00:27:54.320 | such as climate change, and distinguish those from ones that are merely imaginable,

00:27:58.960 | but with infinitesimal probabilities. And we have to take into account people's worry budget. You

00:28:05.920 | can't worry about everything. And if you sow dread and fear and terror and fatalism, it can lead to

00:28:13.200 | a kind of numbness. Well, these problems are overwhelming, and the engineers are just going

00:28:17.520 | to kill us all. So let's either destroy the entire infrastructure of science, technology,

00:28:25.760 | or let's just enjoy life while we can. - So there's a certain line of worry,

00:28:31.600 | which I'm worried about a lot of things in engineering. There's a certain line of worry

00:28:34.560 | when you cross, you're allowed to cross, that it becomes paralyzing fear as opposed to

00:28:40.960 | productive fear. And that's kind of what you're highlighting. - Exactly right. And we've seen

00:28:47.200 | some, we know that human effort is not well calibrated against risk in that, because a basic

00:28:55.840 | tenet of cognitive psychology is that perception of risk and hence perception of fear is driven by

00:29:02.720 | imaginability, not by data. And so we misallocate vast amounts of resources to avoiding terrorism,

00:29:11.200 | which kills on average about six Americans a year, with one exception of 9/11. We invade countries,

00:29:16.880 | we invent entire new departments of government with massive, massive expenditure of resources

00:29:24.480 | and lives to defend ourselves against a trivial risk. Whereas guaranteed risks, you mentioned,

00:29:31.360 | as one of them is, you mentioned traffic fatalities, and even risks that are

00:29:39.280 | not here, but are plausible enough to worry about, like pandemics, like nuclear war,

00:29:46.240 | receive far too little attention. In presidential debates, there's no discussion of

00:29:51.760 | how to minimize the risk of nuclear war. Lots of discussion of terrorism, for example. And so we,

00:29:58.880 | I think it's essential to calibrate our budget of fear, worry, concern planning to the actual

00:30:07.760 | probability of harm. - Yep. So let me ask this, then this question. So speaking of imaginability,

00:30:15.760 | you said that it's important to think about reason. And one of my favorite people who likes

00:30:21.120 | to dip into the outskirts of reason through fascinating exploration of his imagination

00:30:27.200 | is Joe Rogan. - Oh, yes. - So who has, through reason, used to believe a lot of conspiracies,

00:30:34.880 | and through reason has stripped away a lot of his beliefs in that way. So it's fascinating,

00:30:39.760 | actually, to watch him, through rationality, kind of throw away the ideas of Bigfoot and 9/11. I'm

00:30:47.520 | not sure exactly. - Kim Trails. I don't know what he believes in. Yes, okay. - But he no longer-

00:30:51.360 | - Believed in, no, that's right. - Believed in, that's right. - No, he's become a real force for

00:30:54.400 | good. - Yep. So you were on the Joe Rogan podcast in February and had a fascinating conversation,

00:31:00.240 | but as far as I remember, didn't talk much about artificial intelligence. I will be on his podcast

00:31:05.920 | in a couple weeks. Joe is very much concerned about existential threat of AI. I'm not sure if

00:31:11.520 | you're, which is why I was hoping that you would get into that topic. And in this way, he represents

00:31:17.920 | quite a lot of people who look at the topic of AI from 10,000 foot level. So as an exercise of

00:31:25.600 | communication, you said it's important to be rational and reason about these things. Let me

00:31:30.400 | ask, if you were to coach me as an AI researcher about how to speak to Joe and the general public

00:31:36.160 | about AI, what would you advise? - Well, the short answer would be to read the sections that I wrote

00:31:41.680 | in Enlightenment, you know, about AI. But a longer reason would be, I think, to emphasize, and I

00:31:46.720 | think you're very well positioned as an engineer to remind people about the culture of engineering,

00:31:51.840 | that it really is safety oriented. Another discussion in Enlightenment now, I plot rates

00:32:00.560 | of accidental death from various causes, plane crashes, car crashes, occupational accidents,

00:32:07.280 | even death by lightning strikes. And they all plummet because the culture of engineering is,

00:32:14.320 | how do you squeeze out the lethal risks? Death by fire, death by drowning, death by asphyxiation,

00:32:21.760 | all of them drastically declined because of advances in engineering, that I gotta say,

00:32:25.680 | I did not appreciate until I saw those graphs. And it is because, exactly, people like you who

00:32:32.480 | stamp it and I think, oh my God, is what I'm inventing likely to hurt people? And to deploy

00:32:40.400 | ingenuity to prevent that from happening. Now, I'm not an engineer, although I spent 22 years at MIT,

00:32:45.920 | so I know something about the culture of engineering. My understanding is that this

00:32:49.120 | is the way you think if you're an engineer. And it's essential that that culture not be suddenly

00:32:55.600 | switched off when it comes to artificial intelligence. So, I mean, that could be a

00:33:00.400 | problem, but is there any reason to think it would be switched off? - I don't think so. And

00:33:03.840 | one, there's not enough engineers speaking up for this way, for the excitement, for the positive

00:33:11.040 | view of human nature, what you're trying to create is the positivity. Like everything we try to

00:33:15.520 | invent is trying to do good for the world. But let me ask you about the psychology of negativity.

00:33:20.880 | It seems just objectively, not considering the topic, it seems that being negative about the

00:33:26.640 | future makes you sound smarter than being positive about the future, irregardless of topic. Am I

00:33:32.080 | correct in this observation? And if so, why do you think that is? - Yeah, I think there is that

00:33:37.760 | phenomenon that, as Tom Lehrer, the satirist said, "Always predict the worst and you'll be

00:33:43.440 | hailed as a prophet." It may be part of our overall negativity bias. We are, as a species,

00:33:50.880 | more attuned to the negative than the positive. We dread losses more than we enjoy gains. And

00:33:57.280 | that might open up a space for prophets to remind us of harms and risks and losses that we may have

00:34:06.080 | overlooked. So I think there is that asymmetry. - So you've written some of my favorite books

00:34:15.120 | all over the place. So starting from Enlightenment Now to The Better Angels of Our Nature,

00:34:21.680 | Blank Slate, How the Mind Works, the one about language, Language Instinct. Bill Gates,

00:34:29.200 | big fan too, said of your most recent book that it's my new favorite book of all time.

00:34:35.680 | So for you as an author, what was the book early on in your life that had a profound impact on the

00:34:43.840 | way you saw the world? - Certainly this book, Enlightenment Now, is influenced by David Deutsch's

00:34:49.520 | The Beginning of Infinity. A rather deep reflection on knowledge and the power of knowledge to

00:34:57.280 | improve the human condition. And with bits of wisdom such as that problems are inevitable,

00:35:03.040 | but problems are solvable given the right knowledge, and that solutions create new

00:35:07.440 | problems that have to be solved in their turn. That's, I think, a kind of wisdom about the human

00:35:11.920 | condition that influenced the writing of this book. There's some books that are excellent but

00:35:16.240 | obscure, some of which I have on a page of my website. I read a book called The History of

00:35:21.760 | Force, self-published by a political scientist named James Paine on the historical decline of

00:35:27.440 | violence, and that was one of the inspirations for The Better Angels of Our Nature. - What about

00:35:34.000 | early on? If you look back when you were maybe a teenager, is there some-- - I loved a book called

00:35:39.200 | One, Two, Three, Infinity. When I was a young adult, I read that book by George Gamow, the

00:35:44.240 | physicist. He had very accessible and humorous explanations of relativity, of number theory, of

00:35:52.320 | dimensionality, high multiple dimensional spaces, in a way that I think is still delightful 70 years

00:36:01.360 | after it was published. I like the Time-Life Science series. These are books that arrive

00:36:07.360 | every month that my mother subscribed to, each one on a different topic. One would be on electricity,

00:36:14.480 | one would be on forests, one would be on evolution, and then one was on the mind.

00:36:18.320 | I was just intrigued that there could be a science of mind, and that book I would cite as an

00:36:25.280 | influence as well. Then later on-- - That's when you fell in love with the idea of studying the

00:36:29.360 | mind? - That's one of the-- - Was that the thing that grabbed you? - It was one of the things,

00:36:32.480 | I would say. I read as a college student the book Reflections on Language by Noam Chomsky,

00:36:40.160 | who spent most of his career here at MIT. Richard Dawkins, two books, The Blind Watchmaker and The

00:36:46.640 | Selfish Gene, were enormously influential, mainly for the content, but also for the writing style,

00:36:54.000 | the ability to explain abstract concepts in lively prose. Stephen Jay Gould's first collection,

00:37:02.480 | Ever Since Darwin, also an excellent example of lively writing. George Miller, a psychologist

00:37:10.000 | that most psychologists are familiar with, came up with the idea that human memory has a capacity of

00:37:15.600 | seven plus or minus two chunks. That's probably his biggest claim to fame. But he wrote a couple

00:37:20.640 | of books on language and communication that I read as an undergraduate. Again, beautifully written

00:37:25.840 | and intellectually deep. - Wonderful. Stephen, thank you so much for taking the time today.

00:37:31.680 | - My pleasure. Thanks a lot, Lex.

00:37:33.280 | - Thanks.

00:37:34.820 | [END]

00:37:35.860 | [END]

00:37:36.740 | [END]

00:37:36.820 | [END]

00:37:36.900 | [BLANK_AUDIO]

00:37:45.200 | [BLANK_AUDIO]

Steven Pinker: AI in the Age of Reason | Lex Fridman Podcast #3

Chapters