Melanie Mitchell: Concepts, Analogies, Common Sense & Future of AI

00:00:00.000 | The following is a conversation with Melanie Mitchell.

00:00:03.180 | She's a professor of computer science

00:00:04.860 | at Portland State University

00:00:06.720 | and an external professor at Santa Fe Institute.

00:00:10.020 | She has worked on and written about artificial intelligence

00:00:13.000 | from fascinating perspectives,

00:00:14.940 | including adaptive complex systems, genetic algorithms,

00:00:18.420 | and the copycat cognitive architecture,

00:00:20.980 | which places the process of analogy making

00:00:23.340 | at the core of human cognition.

00:00:26.300 | From her doctoral work with her advisors,

00:00:28.500 | Douglas Hofstadter and John Holland,

00:00:30.980 | to today, she has contributed a lot of important ideas

00:00:34.180 | to the field of AI, including her recent book,

00:00:36.940 | simply called "Artificial Intelligence,

00:00:39.860 | "A Guide for Thinking Humans."

00:00:42.780 | This is the Artificial Intelligence Podcast.

00:00:45.860 | If you enjoy it, subscribe on YouTube,

00:00:48.260 | give it five stars on Apple Podcast,

00:00:50.260 | support it on Patreon,

00:00:51.700 | or simply connect with me on Twitter

00:00:53.780 | at Lex Friedman, spelled F-R-I-D-M-A-N.

00:00:58.100 | I recently started doing ads at the end of the introduction.

00:01:01.580 | I'll do one or two minutes after introducing the episode

00:01:04.340 | and never any ads in the middle

00:01:05.900 | that can break the flow of the conversation.

00:01:08.340 | I hope that works for you

00:01:09.580 | and doesn't hurt the listening experience.

00:01:12.140 | I provide timestamps for the start of the conversation,

00:01:14.900 | but it helps if you listen to the ad

00:01:17.060 | and support this podcast

00:01:18.500 | by trying out the product or service being advertised.

00:01:21.260 | This show is presented by Cash App,

00:01:24.940 | the number one finance app in the App Store.

00:01:27.500 | I personally use Cash App to send money to friends,

00:01:30.260 | but you can also use it to buy, sell,

00:01:32.580 | and deposit Bitcoin in just seconds.

00:01:35.020 | Cash App also has a new investing feature.

00:01:38.180 | You can buy fractions of a stock, say $1 worth,

00:01:41.140 | no matter what the stock price is.

00:01:43.380 | Brokerage services are provided by Cash App Investing,

00:01:46.180 | a subsidiary of Square and a member of SIPC.

00:01:49.940 | I'm excited to be working with Cash App

00:01:51.860 | to support one of my favorite organizations called FIRST,

00:01:55.260 | best known for their FIRST Robotics and LEGO competitions.

00:01:58.900 | They educate and inspire hundreds of thousands of students

00:02:02.620 | in over 110 countries

00:02:04.420 | and have a perfect rating on Charity Navigator,

00:02:06.940 | which means that donated money is used

00:02:09.020 | to maximum effectiveness.

00:02:11.660 | When you get Cash App from the App Store or Google Play

00:02:14.900 | and use code LEXPODCAST, you'll get $10,

00:02:18.540 | and Cash App will also donate $10 to FIRST,

00:02:21.340 | which again is an organization

00:02:23.220 | that I've personally seen inspire girls and boys

00:02:26.100 | to dream of engineering a better world.

00:02:28.980 | And now, here's my conversation with Melanie Mitchell.

00:02:32.820 | The name of your new book is "Artificial Intelligence,"

00:02:36.860 | subtitle, "A Guide for Thinking Humans."

00:02:39.700 | The name of this podcast is "Artificial Intelligence."

00:02:42.960 | So let me take a step back

00:02:44.100 | and ask the old Shakespeare question about roses.

00:02:46.940 | And what do you think of the term artificial intelligence

00:02:51.100 | for our big and complicated and interesting field?

00:02:55.500 | - I'm not crazy about the term. (laughs)

00:02:57.900 | I think it has a few problems

00:02:59.980 | because it means so many different things

00:03:04.380 | to different people.

00:03:05.620 | And intelligence is one of those words

00:03:07.460 | that isn't very clearly defined either.

00:03:10.060 | There's so many different kinds of intelligence,

00:03:14.420 | degrees of intelligence, approaches to intelligence.

00:03:18.900 | John McCarthy was the one who came up

00:03:21.220 | with the term artificial intelligence.

00:03:23.220 | And from what I read,

00:03:24.380 | he called it that to differentiate it from cybernetics,

00:03:28.780 | which was another related movement at the time.

00:03:33.700 | And he later regretted calling it artificial intelligence.

00:03:38.340 | Herbert Simon was pushing

00:03:40.720 | for calling it complex information processing,

00:03:43.860 | which got nixed,

00:03:47.080 | but probably is equally vague, I guess.

00:03:52.080 | - Is it the intelligence or the artificial

00:03:55.360 | in terms of words that's most problematic, would you say?

00:03:58.720 | - Yeah, I think it's a little of both.

00:04:01.040 | But it has some good sides

00:04:02.960 | because I personally was attracted to the field

00:04:07.060 | because I was interested in phenomenon of intelligence.

00:04:11.280 | And if it was called complex information processing,

00:04:13.620 | maybe I'd be doing something wholly different now.

00:04:16.200 | - What do you think of,

00:04:17.200 | I've heard that term used cognitive systems, for example.

00:04:20.320 | So using cognitive.

00:04:22.760 | - Yeah, I mean, cognitive has certain associations with it.

00:04:27.760 | And people like to separate things

00:04:29.920 | like cognition and perception,

00:04:32.120 | which I don't actually think are separate.

00:04:33.940 | But often people talk about cognition

00:04:36.800 | as being different from other aspects of intelligence.

00:04:41.400 | It's sort of higher level.

00:04:42.720 | - So to you, cognition is this broad,

00:04:44.640 | beautiful mess of things that encompasses the whole thing.

00:04:47.920 | Memory, perception.

00:04:48.760 | - Yeah, I think it's hard to draw lines like that.

00:04:53.040 | When I was coming out of grad school in 1990,

00:04:56.640 | which is when I graduated,

00:04:58.360 | that was during one of the AI winters.

00:05:00.680 | And I was advised to not put AI,

00:05:05.160 | artificial intelligence on my CV,

00:05:06.780 | but instead call it intelligence systems.

00:05:09.220 | So that was kind of a euphemism, I guess.

00:05:13.900 | - What about, to stick briefly on terms and words,

00:05:18.900 | the idea of artificial general intelligence,

00:05:24.100 | or like Jan LeCun prefers human level intelligence.

00:05:29.100 | Sort of starting to talk about ideas

00:05:32.460 | that achieve higher and higher levels of intelligence.

00:05:37.740 | And somehow artificial intelligence seems to be

00:05:40.860 | a term used more for the narrow,

00:05:43.180 | very specific applications of AI.

00:05:45.340 | And sort of, what set of terms appeal to you

00:05:50.340 | to describe the thing that perhaps we strive to create?

00:05:56.020 | - People have been struggling with this

00:05:57.420 | for the whole history of the field.

00:05:59.220 | And defining exactly what it is that we're talking about.

00:06:03.420 | You know, John Searle had this distinction

00:06:05.620 | between strong AI and weak AI.

00:06:08.500 | And weak AI could be general AI,

00:06:10.460 | but his idea was strong AI was the view

00:06:14.580 | that a machine is actually thinking.

00:06:17.180 | That as opposed to simulating thinking

00:06:22.580 | or carrying out processes that we would call intelligent.

00:06:27.580 | - At a high level, if you look at the founding

00:06:34.460 | of the field of McCarthy and Searle and so on,

00:06:38.940 | are we closer to having a better sense of that line

00:06:43.940 | between narrow, weak AI and strong AI?

00:06:49.380 | - Yes, I think we're closer to having a better idea

00:06:55.420 | of what that line is.

00:06:57.140 | Early on, for example, a lot of people thought

00:07:01.660 | that playing chess would be,

00:07:04.460 | you couldn't play chess if you didn't have

00:07:07.980 | sort of general human level intelligence.

00:07:11.140 | And of course, once computers were able

00:07:13.340 | to play chess better than humans, that revised that view.

00:07:18.340 | And people said, okay, well, maybe now we have

00:07:21.500 | to revise what we think of intelligence as.

00:07:25.260 | And so that's kind of been a theme

00:07:28.740 | throughout the history of the field is that

00:07:30.940 | once a machine can do some task,

00:07:33.500 | we then have to look back and say,

00:07:36.420 | oh, well, that changes my understanding

00:07:38.740 | of what intelligence is because I don't think

00:07:40.540 | that machine is intelligent.

00:07:43.020 | At least that's not what I want to call intelligence.

00:07:45.580 | - Do you think that line moves forever?

00:07:47.620 | Or will we eventually really feel as a civilization

00:07:51.220 | like we've crossed the line if it's possible?

00:07:54.020 | - It's hard to predict, but I don't see any reason

00:07:56.460 | why we couldn't, in principle,

00:07:58.640 | create something that we would consider intelligent.

00:08:03.120 | I don't know how we will know for sure.

00:08:06.320 | Maybe our own view of what intelligence is

00:08:10.480 | will be refined more and more until we finally figure out

00:08:13.920 | what we mean when we talk about it.

00:08:15.680 | But I think eventually we will create machines

00:08:22.120 | in a sense that have intelligence.

00:08:24.400 | They may not be the kinds of machines we have now.

00:08:28.040 | And one of the things that that's going to produce

00:08:31.800 | is making us sort of understand

00:08:34.880 | our own machine-like qualities

00:08:38.520 | that we, in a sense, are mechanical

00:08:43.040 | in the sense that like cells,

00:08:45.880 | cells are kind of mechanical.

00:08:47.680 | They have algorithms, they process information by,

00:08:52.680 | and somehow out of this mass of cells,

00:08:57.040 | we get this emergent property that we call intelligence.

00:09:01.240 | But underlying it is really just cellular processing

00:09:06.240 | and lots and lots and lots of it.

00:09:10.440 | - Do you think we'll be able to,

00:09:12.220 | do you think it's possible to create intelligence

00:09:14.400 | without understanding our own mind?

00:09:16.440 | You said sort of in that process

00:09:18.200 | we'll understand more and more,

00:09:19.440 | but do you think it's possible to sort of create

00:09:22.960 | without really fully understanding

00:09:24.920 | from a mechanistic perspective,

00:09:27.560 | sort of from a functional perspective,

00:09:29.120 | how our mysterious mind works?

00:09:31.880 | - If I had to bet on it, I would say no.

00:09:36.680 | We do have to understand our own minds,

00:09:39.440 | at least to some significant extent.

00:09:42.840 | But I think that's a really big open question.

00:09:47.120 | I've been very surprised at how far

00:09:49.280 | kind of brute force approaches based on, say,

00:09:52.840 | big data and huge networks can take us.

00:09:57.360 | I wouldn't have expected that.

00:09:59.080 | And they have nothing to do with the way our minds work.

00:10:03.080 | So that's been surprising to me, so it could be wrong.

00:10:06.800 | - To explore the psychological and the philosophical,

00:10:09.560 | do you think we're okay as a species

00:10:11.760 | with something that's more intelligent than us?

00:10:15.960 | Do you think perhaps the reason we're pushing that line

00:10:19.720 | further and further is we're afraid of acknowledging

00:10:23.280 | that there's something stronger, better,

00:10:25.800 | smarter than us humans?

00:10:29.000 | - Well, I'm not sure we can define intelligence that way

00:10:31.600 | because smarter than is with respect to what,

00:10:36.600 | computers are already smarter than us in some areas.

00:10:42.880 | They can multiply much better than we can.

00:10:45.600 | They can figure out driving routes to take

00:10:50.240 | much faster and better than we can.

00:10:51.840 | They have a lot more information to draw on.

00:10:54.400 | They know about traffic conditions and all that stuff.

00:10:57.400 | So for any given particular task,

00:11:02.240 | sometimes computers are much better than we are

00:11:04.680 | and we're totally happy with that, right?

00:11:07.080 | I'm totally happy with that.

00:11:08.520 | It doesn't bother me at all.

00:11:10.560 | I guess the question is,

00:11:12.200 | which things about our intelligence would we feel

00:11:18.280 | very sad or upset that machines had been able to recreate?

00:11:24.440 | So in the book, I talk about my former PhD advisor,

00:11:27.400 | Douglas Hofstadter, who encountered

00:11:30.440 | a music generation program.

00:11:32.920 | And that was really the line for him,

00:11:36.760 | that if a machine could create beautiful music,

00:11:40.080 | that would be terrifying for him

00:11:44.080 | because that is something he feels is really at the core

00:11:48.240 | of what it is to be human,

00:11:50.160 | creating beautiful music, art, literature.

00:11:53.360 | I don't think, he doesn't like the fact that

00:11:57.920 | machines can recognize spoken language really well.

00:12:03.840 | He personally doesn't like using speech recognition,

00:12:09.520 | but I don't think it bothers him to his core

00:12:11.600 | 'cause it's like, okay, that's not at the core of humanity.

00:12:15.760 | But it may be different for every person,

00:12:17.920 | what really they feel would usurp their rights

00:12:22.920 | usurp their humanity.

00:12:25.240 | And I think maybe it's a generational thing also.

00:12:27.440 | Maybe our children or our children's children

00:12:30.720 | will be adapted, they'll adapt to these new devices

00:12:35.720 | that can do all these tasks and say,

00:12:38.680 | yes, this thing is smarter than me in all these areas,

00:12:41.560 | but that's great 'cause it helps me.

00:12:44.880 | - Looking at the broad history of our species,

00:12:50.520 | why do you think so many humans have dreamed

00:12:52.720 | of creating artificial life and artificial intelligence

00:12:55.360 | throughout the history of our civilization?

00:12:57.360 | So not just this century or the 20th century,

00:13:00.680 | but really many, throughout many centuries

00:13:03.840 | that preceded it.

00:13:04.740 | - That's a really good question.

00:13:07.800 | And I have wondered about that.

00:13:09.360 | 'Cause I myself was driven by curiosity

00:13:15.320 | about my own thought processes

00:13:18.720 | and thought it would be fantastic

00:13:20.800 | to be able to get a computer

00:13:22.120 | to mimic some of my thought processes.

00:13:24.920 | And I'm not sure why we're so driven.

00:13:28.960 | I think we want to understand ourselves better.

00:13:33.960 | And we also want machines to do things for us.

00:13:43.240 | But I don't know, there's something more to it

00:13:46.240 | because it's so deep in the kind of mythology

00:13:49.560 | or the ethos of our species.

00:13:53.360 | And I don't think other species have this drive.

00:13:56.640 | So I don't know.

00:13:57.600 | - If you were to sort of psychoanalyze yourself

00:13:59.960 | in your own interest in AI,

00:14:02.260 | what excites you about creating intelligence?

00:14:07.520 | You said understanding our own selves?

00:14:09.800 | - Yeah, I think that's what drives me particularly.

00:14:13.820 | I'm really interested in human intelligence.

00:14:18.820 | But I'm also interested in the sort of the phenomenon

00:14:25.820 | of intelligence more generally.

00:14:28.340 | And I don't think humans are the only thing

00:14:29.780 | with intelligence, or even animals.

00:14:34.260 | But I think intelligence is a concept

00:14:39.660 | that encompasses a lot of complex systems.

00:14:43.780 | And if you think of things like insect colonies

00:14:47.740 | or cellular processes or the immune system

00:14:52.000 | or all kinds of different biological

00:14:54.200 | or even societal processes have as an emergent property,

00:14:59.200 | some aspects of what we would call intelligence.

00:15:02.460 | You know, they have memory, they do process information,

00:15:05.120 | they have goals, they accomplish their goals, et cetera.

00:15:08.500 | And to me, the question of what is this thing

00:15:12.700 | we're talking about here was really fascinating to me.

00:15:17.700 | And exploring it using computers seemed to be

00:15:21.820 | a good way to approach the question.

00:15:23.980 | - So do you think kind of of intelligence,

00:15:26.140 | do you think of our universe as a kind of hierarchy

00:15:28.620 | of complex systems and then intelligence

00:15:31.020 | is just the property of any, you can look at any level

00:15:35.560 | and every level has some aspect of intelligence.

00:15:39.300 | So we're just like one little speck

00:15:40.940 | in that giant hierarchy of complex systems.

00:15:43.580 | - I don't know if I would say any system

00:15:47.620 | like that has intelligence.

00:15:49.800 | But I guess what I wanna, I don't have a good enough

00:15:53.820 | definition of intelligence to say that.

00:15:56.780 | - So let me do sort of a multiple choice, I guess.

00:15:59.380 | So you said ant colonies.

00:16:02.540 | So are ant colonies intelligent?

00:16:04.540 | Are the bacteria in our body intelligent?

00:16:09.460 | And then going to the physics world,

00:16:12.740 | molecules and the behavior at the quantum level

00:16:16.260 | of electrons and so on, are those kinds of systems,

00:16:21.260 | do they possess intelligence?

00:16:22.940 | Like where's the line that feels compelling to you?

00:16:27.720 | - I don't know, I mean, I think intelligence is a continuum.

00:16:30.560 | And I think that the ability to, in some sense,

00:16:35.200 | have intention, have a goal,

00:16:37.480 | have some kind of self-awareness is part of it.

00:16:45.280 | So I'm not sure if, it's hard to know

00:16:48.520 | where to draw that line.

00:16:50.380 | I think that's kind of a mystery.

00:16:52.420 | But I wouldn't say that say that,

00:16:56.400 | the planets orbiting the sun is an intelligent system.

00:17:00.760 | I mean, I would find that maybe not the right term

00:17:05.080 | to describe that.

00:17:06.320 | And this is, there's all this debate in the field

00:17:09.160 | of like, what's the right way to define intelligence?

00:17:12.600 | What's the right way to model intelligence?

00:17:15.320 | Should we think about computation?

00:17:16.800 | Should we think about dynamics?

00:17:18.160 | And should we think about free energy

00:17:21.740 | and all of that stuff?

00:17:23.560 | And I think that it's a fantastic time to be in the field

00:17:28.320 | because there's so many questions

00:17:30.400 | and so much we don't understand.

00:17:32.080 | There's so much work to do.

00:17:33.880 | - So are we the most special kind of intelligence

00:17:38.360 | in this kind of, you said there's a bunch

00:17:41.600 | of different elements and characteristics

00:17:43.920 | of intelligence systems and colonies.

00:17:47.200 | Is human intelligence the thing in our brain?

00:17:53.080 | Is that the most interesting kind of intelligence

00:17:55.400 | in this continuum?

00:17:57.100 | - Well, it's interesting to us 'cause it is us.

00:18:01.480 | I mean, interesting to me, yes.

00:18:03.400 | And because I'm part of the human--

00:18:06.720 | - But to understanding the fundamentals of intelligence,

00:18:08.800 | what I'm getting at, is studying the human,

00:18:11.040 | is sort of, everything we've talked about,

00:18:13.200 | what you talk about in your book,

00:18:14.400 | what just the AI field, this notion,

00:18:18.640 | yes, it's hard to define, but it's usually talking

00:18:21.560 | about something that's very akin to human intelligence.

00:18:24.480 | - Yeah, to me it is the most interesting

00:18:26.840 | because it's the most complex, I think.

00:18:29.960 | It's the most self-aware.

00:18:32.120 | It's the only system, at least that I know of,

00:18:34.960 | that reflects on its own intelligence.

00:18:37.780 | - And you talk about the history of AI

00:18:41.040 | and us, in terms of creating artificial intelligence,

00:18:45.000 | being terrible at predicting the future

00:18:48.480 | with AI or with tech in general.

00:18:50.880 | So why do you think we're so bad at predicting the future?

00:18:55.880 | Are we hopelessly bad?

00:18:59.080 | So no matter what, whether it's this decade

00:19:01.960 | or the next few decades, every time we make a prediction,

00:19:04.880 | there's just no way of doing it well?

00:19:06.920 | Or as the field matures, we'll be better and better at it?

00:19:10.880 | - I believe as the field matures, we will be better.

00:19:13.720 | And I think the reason that we've had so much trouble

00:19:16.040 | is that we have so little understanding

00:19:18.400 | of our own intelligence.

00:19:20.320 | So there's the famous story about Marvin Minsky

00:19:25.320 | assigning computer vision as a summer project

00:19:32.600 | to his undergrad students.

00:19:34.640 | And I believe that's actually a true story.

00:19:36.640 | - Yeah, no, there's a write-up on it

00:19:39.080 | that everyone should read.

00:19:41.000 | I think it's like a proposal

00:19:42.480 | that describes everything that should be done

00:19:46.000 | in that project.

00:19:46.840 | And it's hilarious because it, I mean, you can explain it,

00:19:49.920 | but from my recollection, it describes basically

00:19:52.600 | all the fundamental problems of computer vision,

00:19:55.000 | many of which still haven't been solved.

00:19:57.680 | - Yeah, and I don't know how far

00:19:59.560 | they really expect it to get.

00:20:01.400 | But I think that, and they're really,

00:20:04.320 | Marvin Minsky is a super smart guy

00:20:06.120 | and very sophisticated thinker.

00:20:08.400 | But I think that no one really understands or understood,

00:20:13.880 | still doesn't understand how complicated,

00:20:17.640 | how complex the things that we do are

00:20:22.160 | because they're so invisible to us.

00:20:24.640 | To us, vision, being able to look out at the world

00:20:27.640 | and describe what we see, that's just immediate.

00:20:31.680 | It feels like it's no work at all.

00:20:33.380 | So it didn't seem like it would be that hard,

00:20:35.920 | but there's so much going on unconsciously,

00:20:39.320 | sort of invisible to us that I think we overestimate

00:20:44.480 | how easy it will be to get computers to do it.

00:20:49.480 | - And sort of for me to ask an unfair question,

00:20:53.880 | you've done research, you've thought about

00:20:56.520 | many different branches of AI through this book,

00:20:59.920 | widespread looking at where AI has been, where it is today.

00:21:04.060 | If you were to make a prediction,

00:21:08.800 | how many years from now would we as a society

00:21:12.080 | create something that you would say

00:21:15.760 | achieved human level intelligence

00:21:18.280 | or superhuman level intelligence?

00:21:21.720 | - That is an unfair question.

00:21:25.080 | - A prediction that will most likely be wrong.

00:21:28.480 | But it's just your notion because--

00:21:29.960 | - Okay, I'll say more than 100 years.

00:21:34.280 | - More than 100 years.

00:21:35.320 | - And I quoted somebody in my book who said that

00:21:38.480 | human level intelligence is 100 Nobel prizes away.

00:21:42.560 | (laughing)

00:21:44.640 | Which I like 'cause it's a nice way to sort of,

00:21:48.000 | it's a nice unit for prediction.

00:21:49.720 | And it's like that many fantastic discoveries

00:21:55.640 | have to be made.

00:21:56.560 | And of course there's no Nobel Prize in AI.

00:22:00.680 | - Right.

00:22:01.520 | - Not yet at least.

00:22:03.080 | - If we look at that 100 years,

00:22:05.260 | your sense is really the journey to intelligence

00:22:10.200 | has to go through something more complicated

00:22:15.200 | that's akin to our own cognitive systems.

00:22:19.400 | Understanding them, being able to create them

00:22:21.600 | in the artificial systems as opposed to sort of

00:22:25.400 | taking the machine learning approaches of today

00:22:28.860 | and really scaling them and scaling them exponentially

00:22:33.520 | with both compute and hardware and data.

00:22:37.880 | - That would be my guess.

00:22:40.580 | I think that in the sort of going along in the narrow AI

00:22:47.160 | that these current approaches will get better.

00:22:53.520 | I think there's some fundamental limits

00:22:56.800 | to how far they're gonna get.

00:22:59.320 | I might be wrong, but that's what I think.

00:23:01.760 | And there's some fundamental weaknesses that they have

00:23:06.680 | that I talk about in the book

00:23:09.960 | that just comes from this approach of supervised learning,

00:23:14.960 | requiring sort of feed forward networks and so on.

00:23:23.640 | I don't think it's a sustainable approach

00:23:31.240 | to understanding the world.

00:23:34.200 | - Yeah, I'm personally torn on it.

00:23:36.480 | Sort of I've, everything you read about in the book

00:23:39.480 | and sort of we're talking about now, I agree with you,

00:23:43.760 | but I'm more and more, depending on the day,

00:23:47.400 | first of all, I'm deeply surprised by the success

00:23:50.080 | of machine learning and deep learning in general.

00:23:52.720 | From the very beginning,

00:23:53.840 | when I was, it's really been my main focus of work.

00:23:57.280 | I'm just surprised how far it gets.

00:23:59.400 | And I'm also think we're really early on

00:24:03.560 | in these efforts of these narrow AI.

00:24:07.080 | So I think there'll be a lot of surprise of how far it gets.

00:24:10.880 | I think we'll be extremely impressed.

00:24:14.360 | Like my sense is everything I've seen so far,

00:24:17.120 | and we'll talk about autonomous driving and so on.

00:24:19.480 | I think we can get really far,

00:24:21.760 | but I also have a sense that we will discover

00:24:24.720 | just like you said, is that even though we'll get really far

00:24:29.240 | in order to create something like our own intelligence

00:24:31.720 | is actually much farther than we realize.

00:24:34.120 | - Right.

00:24:34.960 | - I think these methods are a lot more powerful

00:24:37.160 | than people give them credit for actually.

00:24:39.120 | So then of course there's the media hype,

00:24:41.160 | but I think there's a lot of researchers in the community,

00:24:43.720 | especially like not undergrads, right?

00:24:46.680 | But like people who've been in AI,

00:24:48.840 | they're skeptical about how far deep learning can get.

00:24:50.960 | And I'm more and more thinking that it can actually get

00:24:54.840 | farther than we realize.

00:24:56.960 | - It's certainly possible.

00:24:58.480 | One thing that surprised me when I was writing the book

00:25:00.840 | is how far apart different people in the field are

00:25:03.800 | on their opinion of how far the field has come

00:25:08.440 | and what is accomplished and what's gonna happen next.

00:25:11.560 | - What's your sense of the different,

00:25:13.800 | who are the different people, groups, mindsets,

00:25:17.560 | thoughts in the community about where AI is today?

00:25:21.820 | - Yeah, they're all over the place.

00:25:24.120 | So there's kind of the singularity transhumanism group.

00:25:29.120 | I don't know exactly how to characterize that approach.

00:25:33.200 | - I agree, first of all.

00:25:34.320 | - Yeah, the sort of exponential progress.

00:25:37.920 | We're on the sort of almost at the hugely accelerating

00:25:42.920 | part of the exponential.

00:25:45.760 | And by in the next 30 years,

00:25:49.720 | we're going to see super intelligent AI and all that,

00:25:54.120 | and we'll be able to upload our brains and that.

00:25:57.400 | So there's that kind of extreme view that most,

00:26:00.520 | I think most people who work in AI don't have.

00:26:04.640 | They disagree with that.

00:26:06.080 | But there are people who are maybe aren't singularity people,

00:26:11.080 | but they do think that the current approach

00:26:16.880 | of deep learning is going to scale

00:26:20.040 | and is going to kind of go all the way basically

00:26:23.840 | and take us to true AI or human level AI

00:26:26.760 | or whatever you wanna call it.

00:26:29.120 | And there's quite a few of them.

00:26:30.880 | And a lot of them, like a lot of the people I've met

00:26:34.800 | who work at big tech companies in AI groups

00:26:39.800 | kind of have this view that we're really not that far.

00:26:46.200 | - Just to linger on that point,

00:26:47.440 | sort of if I can take as an example, like Jan Lekun,

00:26:50.960 | I don't know if you know about his work

00:26:52.640 | and so his viewpoints on this.

00:26:54.480 | - I do.

00:26:55.320 | - He believes that there's a bunch of breakthroughs,

00:26:57.840 | like fundamental, like Nobel prizes that are needed still.

00:27:01.080 | But I think he thinks those breakthroughs

00:27:03.600 | will be built on top of deep learning.

00:27:06.600 | And then there's some people who think we need to kind of

00:27:09.480 | put deep learning to the side a little bit

00:27:12.720 | as just one module that's helpful

00:27:15.480 | in the bigger cognitive framework.

00:27:17.800 | - Right, so I think, from what I understand,

00:27:21.320 | Jan Lekun is rightly saying supervised learning

00:27:26.320 | is not sustainable.

00:27:28.000 | We have to figure out how to do unsupervised learning,

00:27:31.120 | that that's gonna be the key.

00:27:33.640 | And I think that's probably true.

00:27:38.320 | I think unsupervised learning

00:27:40.760 | is gonna be harder than people.

00:27:42.920 | - Think. (laughs)

00:27:44.560 | I mean, the way that we humans do it.

00:27:47.080 | Then there's the opposing view,

00:27:50.440 | you know, there's the Gary Marcus kind of hybrid view

00:27:55.440 | where deep learning is one part,

00:27:58.160 | but we need to bring back kind of these symbolic approaches

00:28:02.240 | and combine them.

00:28:03.440 | Of course, no one knows how to do that very well.

00:28:06.680 | - Which is the more important part.

00:28:08.480 | - Right.

00:28:09.320 | - To emphasize and how do they, yeah,

00:28:11.200 | how do they fit together?

00:28:12.080 | What's the foundation?

00:28:13.800 | What's the thing that's on top?

00:28:15.400 | What's the cake?

00:28:16.240 | What's the icing?

00:28:17.080 | - Right. (laughs)

00:28:18.640 | Then there's people pushing different things.

00:28:22.720 | There's the people, the causality people

00:28:24.840 | who say, you know, deep learning as it's formulated today

00:28:28.720 | completely lacks any notion of causality

00:28:32.080 | and that dooms it.

00:28:35.160 | And therefore, we have to somehow give it

00:28:37.720 | some kind of notion of causality.

00:28:41.340 | There's a lot of push from the more cognitive science crowd

00:28:46.340 | saying we have to look at developmental learning.

00:28:54.160 | We have to look at how babies learn.

00:28:56.740 | We have to look at intuitive physics,

00:29:01.000 | all these things we know about physics.

00:29:03.040 | And as somebody kind of quipped,

00:29:05.320 | we also have to teach machines intuitive metaphysics,

00:29:08.840 | which means like objects exist.

00:29:12.280 | (both laugh)

00:29:14.560 | Causality exists.

00:29:16.140 | You know, these things that maybe we're born with.

00:29:19.280 | I don't know.

00:29:20.200 | That they don't have,

00:29:21.400 | the machines don't have any of that.

00:29:23.560 | You know, they look at a group of pixels

00:29:26.600 | and maybe they get 10 million examples,

00:29:31.400 | but they can't necessarily learn

00:29:34.360 | that there are objects in the world.

00:29:38.160 | So there's just a lot of pieces of the puzzle

00:29:41.160 | that people are promoting

00:29:43.080 | and with different opinions of like how important they are

00:29:47.640 | and how close we are to being able to put them all together

00:29:52.000 | to create general intelligence.

00:29:54.080 | - Looking at this broad field,

00:29:56.580 | what do you take away from it?

00:29:57.800 | Who's the most impressive?

00:29:59.580 | Is it the cognitive folks, the Gary Marcus camp,

00:30:03.320 | the Yon camp, unsupervised and self-supervised?

00:30:07.000 | There's the supervisors

00:30:08.600 | and then there's the engineers

00:30:09.640 | who are actually building systems.

00:30:11.560 | You have sort of the Andrej Karpathy at Tesla

00:30:14.720 | building actual, you know, it's not philosophy,

00:30:17.960 | it's real systems that operate in the real world.

00:30:21.040 | What do you take away from all this beautiful variety?

00:30:23.880 | - I don't know if, you know,

00:30:25.600 | these different views are not necessarily

00:30:27.520 | mutually exclusive.

00:30:29.660 | And I think people like Jan LeCun

00:30:34.760 | agrees with the developmental psychology,

00:30:37.160 | causality, intuitive, physics, et cetera.

00:30:43.200 | But he still thinks that it's learning,

00:30:46.000 | like end-to-end learning is the way to go.

00:30:48.320 | - Will take us perhaps all the way.

00:30:50.120 | - Yeah, and that we don't need,

00:30:51.120 | there's no sort of innate stuff that has to get built in.

00:30:55.800 | This is, you know, it's because it's a hard problem.

00:31:02.260 | I personally, you know,

00:31:04.300 | I'm very sympathetic to the cognitive science side

00:31:07.260 | 'cause that's kind of where I came in to the field.

00:31:10.500 | I've become more and more sort of an embodiment adherent

00:31:15.500 | saying that, you know, without having a body,

00:31:18.580 | it's gonna be very hard to learn

00:31:20.880 | what we need to learn about the world.

00:31:22.780 | - That's definitely something I'd love to talk about

00:31:26.900 | in a little bit, to step into the cognitive world.

00:31:31.580 | And if you don't mind,

00:31:32.780 | 'cause you've done so many interesting things.

00:31:34.300 | If we look to CopyCat,

00:31:36.980 | taking a couple of decades step back,

00:31:40.300 | you, Douglas Hostetter, and others

00:31:43.380 | have created and developed CopyCat more than 30 years ago.

00:31:48.380 | - That's painful to hear.

00:31:49.980 | - What is it?

00:31:51.940 | What is CopyCat?

00:31:54.340 | - It's a program that makes analogies in an idealized domain,

00:32:00.740 | idealized world of letter strings.

00:32:03.620 | So as you say, 30 years ago, wow.

00:32:05.560 | So I started working on it

00:32:07.900 | when I started grad school in 1984.

00:32:12.620 | Wow.

00:32:14.380 | (both laughing)

00:32:15.420 | Dates me.

00:32:16.260 | And it's based on Doug Hostetter's ideas

00:32:21.460 | that about that analogy

00:32:24.660 | is really a core aspect of thinking.

00:32:30.280 | I remember he has a really nice quote

00:32:32.420 | in the book by himself and Immanuel Sander

00:32:36.940 | called "Surfaces and Essences."

00:32:38.820 | I don't know if you've seen that book,

00:32:39.820 | but it's about analogy.

00:32:42.040 | He says, "Without concepts, there can be no thought,

00:32:46.780 | "and without analogies, there can be no concepts."

00:32:51.180 | So the view is that analogy

00:32:52.620 | is not just this kind of reasoning technique

00:32:55.060 | where we go, shoe is to foot,

00:32:59.460 | glove is to what,

00:33:01.020 | these kinds of things that we have on IQ tests or whatever,

00:33:05.440 | but that it's much deeper,

00:33:06.540 | it's much more pervasive in everything we do,

00:33:10.940 | in our language, our thinking, our perception.

00:33:14.900 | So he had a view that was a very active perception idea.

00:33:20.900 | So the idea was that instead of having

00:33:25.100 | kind of a passive network in which you have input

00:33:30.100 | that's being processed through these feed-forward layers,

00:33:35.480 | and then there's an output at the end,

00:33:37.060 | that perception is really a dynamic process

00:33:41.460 | where our eyes are moving around

00:33:43.380 | and they're getting information,

00:33:44.780 | and that information is feeding back

00:33:47.020 | to what we look at next, influences what we look at next

00:33:51.540 | and how we look at it.

00:33:53.220 | And so CopyCat was trying to do that,

00:33:56.060 | kind of simulate that kind of idea

00:33:57.720 | where you have these agents,

00:34:02.640 | it's kind of an agent-based system,

00:34:04.100 | and you have these agents that are picking things to look at

00:34:08.040 | and deciding whether they were interesting or not,

00:34:11.180 | and whether they should be looked at more,

00:34:13.580 | and that would influence other agents.

00:34:15.900 | - How did they interact?

00:34:17.580 | - So they interacted through this global,

00:34:19.700 | kind of what we call the workspace.

00:34:22.180 | So it was actually inspired by the old blackboard systems

00:34:25.620 | where you would have agents that post information

00:34:28.920 | on a blackboard, a common blackboard.

00:34:30.860 | This is like very old-fashioned AI.

00:34:32.940 | - Is that, are we talking about like in physical space?

00:34:36.300 | Is this a computer program?

00:34:37.380 | - It's a computer program.

00:34:38.300 | - So agents posting concepts on a blackboard kind of thing?

00:34:41.940 | - Yeah, we called it a workspace.

00:34:43.900 | And the workspace is a data structure.

00:34:48.420 | The agents are little pieces of code

00:34:50.720 | that you could think of them as little detectors

00:34:54.100 | or little filters that say,

00:34:55.940 | I'm gonna pick this place to look,

00:34:57.460 | and I'm gonna look for a certain thing,

00:34:59.060 | and is this the thing I think is important?

00:35:01.820 | Is it there?

00:35:03.020 | So it's almost like a convolution in a way,

00:35:06.940 | except a little bit more general,

00:35:10.020 | and then highlighting it in the workspace.

00:35:13.740 | - Once it's in the workspace,

00:35:16.260 | how do the things that are highlighted relate to each other?

00:35:18.820 | Like what's, is it--

00:35:19.660 | - So there's different kinds of agents

00:35:21.520 | that can build connections between different things.

00:35:23.840 | So just to give you a concrete example,

00:35:25.560 | what CopyCat did was it made analogies

00:35:28.360 | between strings of letters.

00:35:30.320 | So here's an example.

00:35:31.960 | ABC changes to ABD.

00:35:35.360 | What does IJK change to?

00:35:38.160 | And the program had some prior knowledge about the alphabet.

00:35:42.440 | It knew the sequence of the alphabet.

00:35:44.280 | It had a concept of letter, of successor of letter.

00:35:49.300 | It had concepts of sameness.

00:35:50.960 | So it had some innate things programmed in.

00:35:54.000 | But then it could do things like, say,

00:35:58.320 | discover that ABC is a group of letters in succession.

00:36:03.320 | And then an agent can mark that.

00:36:10.120 | - So the idea that there could be a sequence of letters,

00:36:16.020 | is that a new concept that's formed,

00:36:18.160 | or that's a concept that's innate?

00:36:19.400 | - That's a concept that's innate.

00:36:21.500 | - Sort of, can you form new concepts,

00:36:23.700 | or are all concepts innate?

00:36:25.060 | - So in this program,

00:36:27.500 | all the concepts of the program were innate.

00:36:30.220 | So, 'cause we weren't, I mean,

00:36:32.220 | obviously that limits it quite a bit.

00:36:35.600 | But what we were trying to do is say,

00:36:37.180 | suppose you have some innate concepts.

00:36:39.460 | How do you flexibly apply them to new situations?

00:36:45.140 | And how do you make analogies?

00:36:47.820 | - Let's step back for a second.

00:36:49.020 | So I really like that quote,

00:36:50.900 | that you say, "Without concepts, there can be no thought,

00:36:53.700 | "and without analogies, there can be no concepts."

00:36:56.580 | In a Santa Fe presentation,

00:36:58.460 | you said that it should be one of the mantras of AI.

00:37:00.900 | - Yes.

00:37:01.900 | - And that you also yourself said,

00:37:04.260 | "How to form and fluidly use concept

00:37:06.560 | "is the most important open problem in AI."

00:37:09.860 | - Yes.

00:37:11.260 | - How to form and fluidly use concepts

00:37:14.500 | is the most important open problem in AI.

00:37:16.980 | So let's, what is a concept and what is an analogy?

00:37:21.880 | - A concept is in some sense a fundamental unit of thought.

00:37:27.120 | So say we have a concept of a dog, okay?

00:37:33.260 | And a concept is embedded in a whole space of concepts

00:37:45.180 | so that there's certain concepts that are closer to it

00:37:48.740 | or farther away from it.

00:37:50.260 | - Are these concepts, are they really like fundamental,

00:37:53.140 | like we mentioned innate, almost like axiomatic,

00:37:55.620 | like very basic, and then there's other stuff

00:37:57.980 | built on top of it?

00:37:58.900 | - Yeah. - Or does this include

00:37:59.740 | everything?

00:38:01.100 | Are they complicated?

00:38:02.500 | - You can certainly form new concepts.

00:38:07.020 | - Right, I guess that's the question I'm asking.

00:38:08.380 | - Yeah.

00:38:09.220 | - Can you form new concepts that are

00:38:10.820 | complex combinations of other concepts?

00:38:14.420 | - Yes, absolutely.

00:38:15.980 | And that's kind of what we do in learning.

00:38:20.060 | - And then what's the role of analogies in that?

00:38:23.020 | - So analogy is when you recognize that one situation

00:38:28.020 | is essentially the same as another situation.

00:38:34.700 | And essentially is kind of the key word there

00:38:38.780 | 'cause it's not the same.

00:38:40.020 | So if I say, last week I did a podcast interview

00:38:45.020 | actually like three days ago in Washington, DC.

00:38:52.980 | And that situation was very similar to this situation,

00:38:56.580 | although it wasn't exactly the same.

00:38:58.740 | It was a different person sitting across from me.

00:39:00.780 | We had different kinds of microphones.

00:39:03.380 | The questions were different.

00:39:04.740 | The building was different.

00:39:06.140 | There's all kinds of different things,

00:39:07.140 | but really it was analogous.

00:39:09.980 | Or I can say, so doing a podcast interview,

00:39:14.460 | that's kind of a concept.

00:39:15.900 | It's a new concept.

00:39:17.100 | You know, I never had that concept before.

00:39:19.740 | (both laughing)

00:39:21.060 | I don't know if it's here, essentially.

00:39:22.780 | I mean, and I can make an analogy with it

00:39:26.980 | like being interviewed for a news article in a newspaper.

00:39:31.140 | And I can say, well, you kind of play the same role

00:39:35.420 | that the newspaper reporter played.

00:39:39.860 | It's not exactly the same

00:39:41.780 | 'cause maybe they actually emailed me

00:39:43.660 | some written questions rather than talking.

00:39:45.900 | And the writing, the written questions play,

00:39:50.100 | are analogous to your spoken questions.

00:39:53.020 | You know, there's just all kinds of similarities.

00:39:54.700 | - And this somehow probably connects to conversations

00:39:57.180 | you have over Thanksgiving dinner,

00:39:58.580 | just general conversations.

00:40:00.860 | There's like a thread you can probably take

00:40:03.540 | that just stretches out in all aspects of life

00:40:06.700 | that connect to this podcast.

00:40:08.420 | I mean, conversations between humans.

00:40:11.420 | - Sure, and if I go and tell a friend of mine

00:40:16.420 | about this podcast interview,

00:40:19.180 | my friend might say, oh, the same thing happened to me.

00:40:22.660 | You know, let's say, you know,

00:40:23.820 | you ask me some really hard question

00:40:26.020 | and I have trouble answering it.

00:40:29.260 | My friend could say, the same thing happened to me,

00:40:31.620 | but it was like, it wasn't a podcast interview.

00:40:34.060 | It wasn't, it was a completely different situation.

00:40:39.060 | And yet my friend is seeing essentially the same thing.

00:40:44.220 | You know, we say that very fluidly,

00:40:45.780 | the same thing happened to me.

00:40:48.380 | - Essentially the same thing, right.

00:40:49.940 | - But we don't even say that, right?

00:40:51.100 | We just say the same thing. - Right, you imply it, yes.

00:40:52.900 | - Yeah, and the view that kind of went into, say,

00:40:56.900 | a coffee cat, that whole thing is that,

00:40:58.980 | that act of saying the same thing happened to me

00:41:03.620 | is making an analogy.

00:41:05.540 | And in some sense, that's what underlies all of our concepts.

00:41:10.540 | - Why do you think analogy making that you're describing

00:41:14.060 | is so fundamental to cognition?

00:41:17.020 | Like, it seems like it's the main element action

00:41:20.020 | of what we think of as cognition.

00:41:22.500 | - Yeah, so it can be argued that

00:41:25.940 | all of this generalization we do of concepts

00:41:30.940 | and recognizing concepts in different situations

00:41:37.540 | is done by analogy.

00:41:42.660 | That that's, every time I'm recognizing that, say,

00:41:47.660 | you're a person, that's by analogy

00:41:53.780 | 'cause I have this concept of what person is

00:41:55.700 | and I'm applying it to you.

00:41:57.380 | And every time I recognize a new situation,

00:42:02.380 | like one of the things I talked about in the book

00:42:06.540 | was the concept of walking a dog,

00:42:09.700 | that that's actually making an analogy

00:42:11.780 | because all of that, the details are very different.

00:42:15.460 | - So reasoning could be reduced down

00:42:19.420 | to essentially analogy making.

00:42:21.780 | So all the things we think of as like,

00:42:25.220 | yeah, like you said, perception.

00:42:26.820 | So what's perception is taking raw sensory input

00:42:29.660 | and it's somehow integrating into our understanding

00:42:33.020 | of the world, updating the understanding.

00:42:34.740 | And all of that has just this giant mess of analogies

00:42:39.180 | that are being made.

00:42:40.180 | - I think so, yeah.

00:42:41.260 | - If you could just linger on it a little bit,

00:42:44.260 | like what do you think it takes

00:42:46.220 | to engineer a process like that

00:42:49.100 | for us in our artificial systems?

00:42:52.180 | - We need to understand better, I think,

00:42:56.940 | how we do it, how humans do it.

00:43:00.940 | And it comes down to internal models, I think.

00:43:07.740 | People talk a lot about mental models,

00:43:11.180 | that concepts are mental models,

00:43:13.340 | that I can, in my head, I can do a simulation

00:43:19.540 | of a situation like walking a dog.

00:43:22.540 | And that there's some work in psychology

00:43:25.620 | that promotes this idea that all of concepts

00:43:29.460 | are really mental simulations.

00:43:31.860 | That whenever you encounter a concept

00:43:35.140 | or situation in the world, or you read about it

00:43:37.220 | or whatever, you do some kind of mental simulation

00:43:40.740 | that allows you to predict what's gonna happen,

00:43:43.780 | to develop expectations of what's gonna happen.

00:43:48.020 | So that's the kind of structure I think we need,

00:43:51.620 | is that kind of mental model that,

00:43:55.620 | and in our brains, somehow these mental models

00:43:58.100 | are very much interconnected.

00:44:00.420 | - Again, so a lot of stuff we're talking about

00:44:03.740 | are essentially open problems, right?

00:44:05.980 | So if I ask a question, I don't mean

00:44:08.700 | that you would know the answer,

00:44:10.020 | I'm really just hypothesizing.

00:44:11.380 | But how big do you think is the network

00:44:17.380 | graph data structure of concepts that's in our head?

00:44:22.380 | Like if we're trying to build that ourselves,

00:44:26.500 | like we take it, that's one of the things

00:44:28.780 | we take for granted, we think, I mean,

00:44:30.420 | that's why we take common sense for granted,

00:44:32.580 | we think common sense is trivial.

00:44:34.740 | But how big of a thing of concepts is

00:44:39.740 | that underlies what we think of as common sense,

00:44:42.420 | for example?

00:44:43.260 | - Yeah, I don't know.

00:44:45.460 | And I don't even know what units to measure it in.

00:44:48.100 | You say how big is it?

00:44:50.380 | - That's beautifully put, right?

00:44:51.980 | What?

00:44:53.100 | - But we have, it's really hard to know.

00:44:55.660 | We have, what, 100 billion neurons or something,

00:45:00.660 | I don't know, and they're connected

00:45:04.300 | via trillions of synapses,

00:45:07.860 | and there's all this chemical processing going on.

00:45:10.540 | There's just a lot of capacity for stuff.

00:45:13.740 | And their information's encoded

00:45:15.860 | in different ways in the brain.

00:45:17.180 | It's encoded in chemical interactions,

00:45:19.900 | it's encoded in electric firing and firing rates.

00:45:24.220 | And nobody really knows how it's encoded,

00:45:25.780 | but it just seems like there's a huge amount of capacity.

00:45:29.020 | So I think it's huge, it's just enormous.

00:45:32.460 | And it's amazing how much stuff we know.

00:45:36.740 | - Yeah.

00:45:37.580 | But we know, and not just know like facts,

00:45:42.780 | but it's all integrated into this thing

00:45:44.820 | that we can make analogies with.

00:45:46.500 | - Yes.

00:45:47.340 | - There's a dream of semantic web,

00:45:49.300 | and there's a lot of dreams from expert systems

00:45:52.980 | of building giant knowledge bases.

00:45:55.420 | Do you see a hope for these kinds of approaches

00:45:58.940 | of converting Wikipedia into something

00:46:01.860 | that could be used in analogy making?

00:46:05.100 | - Sure.

00:46:07.220 | And I think people have made some progress

00:46:09.540 | along those lines.

00:46:10.700 | People have been working on this for a long time.

00:46:13.300 | But the problem is, and this I think

00:46:15.940 | is the problem of common sense.

00:46:18.100 | People have been trying to get these common sense networks.

00:46:20.980 | Here at MIT, there's this concept net project.

00:46:24.100 | But the problem is that, as I said,

00:46:27.460 | most of the knowledge that we have is invisible to us.

00:46:31.860 | It's not in Wikipedia.

00:46:33.180 | (both laughing)

00:46:34.860 | It's very basic things about

00:46:39.540 | intuitive physics, intuitive psychology,

00:46:44.100 | intuitive metaphysics, all that stuff.

00:46:47.260 | - If you were to create a website

00:46:49.220 | that described intuitive physics, intuitive psychology,

00:46:53.500 | would it be bigger or smaller than Wikipedia?

00:46:56.500 | What do you think?

00:46:57.380 | - I guess described to whom?

00:47:00.740 | (both laughing)

00:47:02.660 | I'm sorry, but--

00:47:04.140 | - No, that's really good.

00:47:05.380 | I think it's exactly right, yeah.

00:47:07.060 | That's a hard question because, you know,

00:47:09.060 | how do you represent that knowledge is the question, right?

00:47:12.100 | I can certainly write down F equals MA

00:47:15.780 | and Newton's laws and a lot of physics

00:47:19.700 | can be deduced from that.

00:47:21.180 | But that's probably not the best representation

00:47:27.060 | of that knowledge for doing the kinds of reasoning

00:47:32.060 | we want a machine to do.

00:47:36.140 | - So, I don't know, it's impossible to say now.

00:47:40.340 | (both laughing)

00:47:41.380 | And people, you know, the projects like,

00:47:43.340 | there's a famous psych project, right,

00:47:46.460 | that Douglas Linat did that was trying--

00:47:50.020 | - I think still going.

00:47:51.020 | - I think it's still going.

00:47:52.020 | And the idea was to try and encode

00:47:54.780 | all of common sense knowledge,

00:47:56.220 | including all this invisible knowledge

00:47:58.460 | in some kind of logical representation.

00:48:03.420 | And it just never, I think, could do any of the things

00:48:08.420 | that he was hoping it could do,

00:48:10.940 | because that's just the wrong approach.

00:48:12.940 | - Of course, that's what they always say, you know,

00:48:16.740 | and then the history books will say,

00:48:18.860 | well, the psych project finally found a breakthrough

00:48:21.860 | in 2058 or something.

00:48:24.420 | You know, so much progress has been made

00:48:27.380 | in just a few decades that--

00:48:28.700 | - Yeah, it could be.

00:48:29.540 | - Who knows what the next breakthroughs will be.

00:48:31.940 | - It could be.

00:48:32.780 | It's certainly a compelling notion,

00:48:34.700 | what the psych project stands for.

00:48:36.460 | - I think Linat was one of the earliest people

00:48:39.900 | to say common sense is what we need.

00:48:43.100 | - Important.

00:48:43.940 | - That's what we need.

00:48:44.780 | All this like expert system stuff,

00:48:46.980 | that is not gonna get you to AI.

00:48:49.140 | You need common sense.

00:48:50.420 | And he basically gave up his whole academic career

00:48:55.420 | to go pursue that.

00:48:57.620 | And I totally admire that,

00:48:59.380 | but I think that the approach itself will not,

00:49:03.540 | in 2020 or 2040 or wherever, be successful.

00:49:08.980 | - What do you think is wrong with the approach?

00:49:10.300 | What kind of approach might be successful?

00:49:13.100 | - Well, if I knew that.

00:49:15.460 | - Again, nobody knows the answer, right?

00:49:16.940 | - If I knew that, you know, one of my talks,

00:49:19.940 | one of the people in the audience,

00:49:21.180 | this was a public lecture,

00:49:22.220 | one of the people in the audience said,

00:49:24.220 | what AI companies are you investing in?

00:49:27.020 | (both laughing)

00:49:28.300 | - Investment advice, okay.

00:49:29.900 | - I'm a college professor for one thing,

00:49:31.860 | so I don't have a lot of extra funds to invest,

00:49:34.740 | but also, no one knows what's gonna work in AI, right?

00:49:39.300 | That's the problem.

00:49:40.340 | - Let me ask another impossible question

00:49:43.100 | in case you have a sense.

00:49:44.780 | In terms of data structures

00:49:46.460 | that will store this kind of information,

00:49:49.540 | do you think they've been invented yet,

00:49:51.900 | both in hardware and software?

00:49:53.620 | Or is something else needs to be,

00:49:57.020 | are we totally, you know?

00:49:58.260 | - I think something else has to be invented.

00:50:00.460 | That's my guess.

00:50:03.540 | - Is the breakthroughs that's most promising,

00:50:06.420 | would that be in hardware or in software?

00:50:08.740 | Do you think we can get far with the current computers?

00:50:12.660 | Or do we need to do something?

00:50:14.220 | - I see what you're saying.

00:50:16.380 | I don't know if Turing computation is gonna be sufficient.

00:50:19.860 | Probably, I would guess it will.

00:50:22.020 | I don't see any reason why we need anything else.

00:50:25.980 | So in that sense, we have invented the hardware we need,

00:50:28.980 | but we just need to make it faster and bigger.

00:50:31.860 | And we need to figure out the right algorithms

00:50:34.260 | and the right sort of architecture.

00:50:38.540 | - Turing, that's a very mathematical notion.

00:50:43.020 | When we have to build intelligence,

00:50:44.900 | it's now an engineering notion

00:50:46.740 | where you throw all that stuff.

00:50:48.260 | - Well, I guess it is a question.

00:50:51.060 | People have brought up this question,

00:50:56.220 | and when you asked about, is our current hardware,

00:50:58.980 | will our current hardware work?

00:51:02.220 | Well, Turing computation says that our current hardware

00:51:06.980 | is in principle a Turing machine, right?

00:51:13.300 | So all we have to do is make it faster and bigger.

00:51:16.500 | But there have been people like Roger Penrose,

00:51:20.220 | if you might remember that he said,

00:51:22.500 | "Turing machines cannot produce intelligence

00:51:26.420 | "because intelligence requires continuous valued numbers."

00:51:30.540 | I mean, that was sort of my reading of his argument

00:51:34.860 | and quantum mechanics and what else, whatever.

00:51:38.500 | But I don't see any evidence for that,

00:51:41.700 | that we need new computation paradigms.

00:51:46.580 | But I don't know if we're,

00:51:50.460 | I don't think we're gonna be able to scale up

00:51:53.900 | our current approaches to programming these computers.

00:51:58.420 | - What is your hope for approaches like CopyCat

00:52:00.780 | or other cognitive architectures?

00:52:02.700 | I've talked to the creator of Soar, for example.

00:52:04.660 | I've used Actar myself.

00:52:06.020 | I don't know if you're familiar with it.

00:52:07.020 | - Yeah, I am.

00:52:07.860 | - What's your hope of approaches like that

00:52:12.060 | in helping develop systems of greater and greater

00:52:16.300 | intelligence in the coming decades?

00:52:19.980 | - Well, that's what I'm working on now

00:52:22.180 | is trying to take some of those ideas and extending it.

00:52:26.100 | So I think there's some really promising approaches

00:52:30.140 | that are going on now that have to do with

00:52:34.140 | more active generative models.

00:52:39.140 | So this is the idea of this simulation

00:52:41.980 | in your head of a concept.

00:52:43.620 | When you're perceiving a new situation,

00:52:49.860 | you have some simulations in your head.

00:52:51.300 | Those are generative models.

00:52:52.540 | They're generating your expectations.

00:52:54.580 | They're generating predictions.

00:52:55.940 | - So that's part of a perception.

00:52:57.220 | You have a meta model that generates a prediction

00:53:00.660 | and you compare it with, and then the difference

00:53:03.580 | somehow forms. - And you also,

00:53:05.180 | that generative model is telling you where to look

00:53:08.380 | and what to look at and what to pay attention to.

00:53:11.620 | And I think it affects your perception.

00:53:14.060 | It's not that just you compare it with your perception.

00:53:18.140 | It becomes your perception in a way.

00:53:21.900 | It's kind of a mixture of the bottom-up information

00:53:28.300 | coming from the world and your top-down model

00:53:31.860 | being imposed on the world is what becomes your perception.

00:53:36.140 | - So your hope is something like that

00:53:37.380 | can improve perception systems

00:53:39.620 | and that they can understand things better.

00:53:41.780 | - Yes. - They understand things.

00:53:42.900 | - Yes.

00:53:44.180 | - What's the step, what's the analogy-making step there?

00:53:49.180 | - Well, there, the idea is that you have

00:53:52.860 | this pretty complicated conceptual space.

00:53:57.100 | You can talk about a semantic network or something like that

00:54:00.420 | with these different kinds of concept models in your brain

00:54:05.420 | that are connected.

00:54:07.260 | So let's take the example of walking a dog

00:54:10.860 | 'cause we were talking about that.

00:54:12.380 | Okay, let's say I see someone out on the street

00:54:15.220 | walking a cat.

00:54:16.620 | Some people walk their cats, I guess.

00:54:18.540 | Seems like a bad idea, but.

00:54:19.860 | - Yeah. (laughs)

00:54:21.740 | - So my model, there's connections

00:54:24.220 | between my model of a dog and model of a cat.

00:54:28.860 | And I can immediately see the analogy

00:54:32.460 | that those are analogous situations.

00:54:38.700 | But I can also see the differences

00:54:40.820 | and that tells me what to expect.

00:54:43.260 | So also, I have a new situation.

00:54:48.260 | So another example with the walking the dog thing

00:54:51.260 | is sometimes people, I see people riding their bikes

00:54:54.540 | with a leash, holding a leash,

00:54:55.740 | and the dog's running alongside.

00:54:57.620 | Okay, so I know that the, I recognize that

00:55:01.100 | as kind of a dog walking situation

00:55:03.900 | even though the person's not walking, right?

00:55:06.780 | And the dog's not walking.

00:55:08.460 | Because I have these models that say,

00:55:12.780 | okay, riding a bike is sort of similar to walking

00:55:16.580 | or it's connected, it's a means of transportation.

00:55:20.180 | But I, because they have their dog there,

00:55:22.860 | I assume they're not going to work,

00:55:24.420 | but they're going out for exercise.

00:55:26.380 | And these analogies help me to figure out

00:55:30.260 | kind of what's going on, what's likely.

00:55:33.180 | - But sort of these analogies are very human interpretable.

00:55:37.300 | So that's that kind of space.

00:55:39.020 | And then you look at something like

00:55:41.460 | the current deep learning approaches,

00:55:43.460 | they kind of help you to take raw sensory information

00:55:46.700 | and to sort of automatically build up hierarchies

00:55:49.460 | of what you can even call them concepts.

00:55:53.020 | They're just not human interpretable concepts.

00:55:55.620 | What's your, what's the link here?

00:55:58.660 | Do you hope, it's sort of the hybrid system question.

00:56:05.780 | How do you think the two can start to meet each other?

00:56:08.220 | What's the value of learning in this systems of forming,

00:56:13.220 | of analogy making?

00:56:16.020 | - The goal of, you know, the original goal of deep learning

00:56:20.580 | in at least visual perception was that

00:56:24.260 | you would get the system to learn to extract features

00:56:27.300 | that at these different levels of complexities.

00:56:30.340 | It may be edge detection, and that would lead into

00:56:33.340 | learning simple combinations of edges,

00:56:36.660 | and then more complex shapes,

00:56:38.100 | and then whole objects or faces.

00:56:41.540 | And this was based on the ideas of the neuroscientists,

00:56:47.740 | Hubel and Wiesel, who had seen,

00:56:52.860 | laid out this kind of structure in brain.

00:56:55.300 | And I think that's right to some extent.

00:57:02.020 | Of course, people have found that the whole story

00:57:05.860 | is a little more complex than that,

00:57:07.340 | and the brain, of course, always is,

00:57:09.140 | and there's a lot of feedback.

00:57:10.660 | So I see that as absolutely a good brain-inspired approach

00:57:17.020 | to some aspects of perception.

00:57:25.660 | But one thing that it's lacking, for example,

00:57:29.460 | is all of that feedback, which is extremely important.

00:57:33.300 | - The interactive element that you mentioned.

00:57:35.500 | - The expectation, right, the conceptual level.

00:57:39.020 | - Going back and forth with the expectation

00:57:42.180 | and the perception, and just going back and forth.

00:57:44.220 | - So, right, so that is extremely important.

00:57:47.980 | And, you know, one thing about deep neural networks

00:57:52.220 | is that in a given situation, like, you know,

00:57:55.380 | they're trained, right, they get these weights

00:57:57.820 | and everything, but then now I give them a new image,

00:58:01.220 | let's say.

00:58:02.420 | They treat every part of the image in the same way.

00:58:07.420 | You know, they apply the same filters at each layer

00:58:13.540 | to all parts of the image.

00:58:15.900 | There's no feedback to say, like,

00:58:17.580 | oh, this part of the image is irrelevant.

00:58:19.940 | I shouldn't care about this part of the image.

00:58:23.060 | Or this part of the image is the most important part.

00:58:27.020 | And that's kind of what we humans are able to do

00:58:30.140 | because we have these conceptual expectations.

00:58:33.420 | - There's, by the way, a little bit of work in that.

00:58:35.580 | There's certainly a lot more in what's under the--

00:58:38.940 | - Attention. - Called attention

00:58:40.260 | in natural language processing now.

00:58:42.060 | That's exceptionally powerful, and it's a very,

00:58:48.500 | just as you say, it's a really powerful idea.

00:58:50.660 | But again, in sort of machine learning,

00:58:53.380 | it all kind of operates in an automated way

00:58:55.740 | that's not human--

00:58:56.580 | - It's not, okay, so, right.

00:58:59.380 | It's not dynamic, I mean, in the sense that

00:59:01.380 | as a perception of a new example is being processed,

00:59:06.380 | those attention's weights don't change.

00:59:10.900 | - Right, so, I mean, there's a kind of notion

00:59:17.540 | that there's not a memory.

00:59:20.380 | So you're not aggregating.

00:59:22.860 | The idea of this mental model.

00:59:25.060 | - Yes. - Yeah.

00:59:26.580 | I mean, that seems to be a fundamental idea.

00:59:28.620 | There's not a really powerful,

00:59:30.980 | I mean, there's some stuff with memory,

00:59:32.380 | but there's not a powerful way to represent the world

00:59:37.380 | in some sort of way that's deeper than,

00:59:42.340 | I mean, it's so difficult because neural networks

00:59:46.180 | do represent the world.

00:59:47.620 | They do have a mental model, right?

00:59:50.900 | But it just seems to be shallow.

00:59:55.180 | It's hard to criticize them at the fundamental level,

00:59:59.500 | to me at least.

01:00:01.660 | It's easy to criticize them,

01:00:05.180 | well, look, like exactly what you're saying,

01:00:07.180 | mental models sort of almost put a psychology hat on,

01:00:11.660 | say, look, these networks are clearly not able to achieve

01:00:15.860 | what we humans do with forming mental models,

01:00:18.060 | the analogy making, so on.

01:00:20.060 | But that doesn't mean that they fundamentally

01:00:22.020 | cannot do that.

01:00:24.260 | It's very difficult to say that, I mean, at least to me.

01:00:26.580 | Do you have a notion that the learning approaches really,

01:00:29.860 | I mean, they're going to, not only are they limited today,

01:00:33.980 | but they will forever be limited in being able

01:00:38.100 | to construct such mental models.

01:00:41.460 | - I think the idea of the dynamic perception is key here.

01:00:47.380 | The idea that moving your eyes around

01:00:53.800 | and getting feedback, and that's something that,

01:00:57.900 | there's been some models like that.

01:01:00.340 | There's certainly recurrent neural networks

01:01:02.620 | that operate over several time steps.

01:01:04.760 | But the problem is that the actual, the recurrence is,

01:01:10.880 | basically the feedback is, at the next time step,

01:01:18.420 | is the entire hidden state of the network,

01:01:23.500 | which is, and it turns out that that doesn't work very well.

01:01:28.500 | - But see, the thing I'm saying is, mathematically speaking,

01:01:34.300 | it has the information in that recurrence

01:01:38.700 | to capture everything.

01:01:39.860 | It just doesn't seem to work.

01:01:41.380 | - Yeah, right.

01:01:42.540 | - So, it's like, it's the same Turing machine question,

01:01:47.440 | right, yeah, maybe theoretically,

01:01:53.140 | computers, anything that's Turing,

01:01:56.980 | a universal Turing machine can be intelligent,

01:01:59.740 | but practically, the architecture might be

01:02:03.220 | a very specific kind of architecture

01:02:04.740 | to be able to create it.

01:02:05.940 | So, it's just, I guess it sort of asks

01:02:08.700 | almost the same question again,

01:02:10.180 | is how big of a role do you think deep learning

01:02:14.380 | needs, will play, or needs to play in this, in perception?

01:02:21.140 | - I think that deep learning as it currently exists,

01:02:26.140 | that kind of thing will play some role,

01:02:30.340 | but I think that there's a lot more going on in perception.

01:02:36.820 | But who knows, the definition of deep learning,

01:02:40.100 | I mean, it's pretty broad.

01:02:42.020 | It's kind of an umbrella for a lot of different things.

01:02:43.700 | - So, what I mean is purely sort of neural networks.

01:02:46.420 | - Yeah, and a feed-forward neural networks.

01:02:48.580 | - Essentially, or there could be recurrence,

01:02:50.780 | but sometimes it feels like,

01:02:54.300 | for instance, I talked to Gary Marcus,

01:02:56.260 | it feels like the criticism of deep learning

01:02:58.940 | is kind of like us birds criticizing airplanes

01:03:03.020 | for not flying well, or that they're not really flying.

01:03:07.740 | Do you think deep learning,

01:03:10.660 | do you think it could go all the way,

01:03:12.860 | like John Licklund thinks?

01:03:15.140 | Do you think that, yeah, the brute force learning approach

01:03:20.500 | can go all the way?

01:03:21.860 | - I don't think so, no.

01:03:23.100 | I mean, I think it's an open question,

01:03:25.460 | but I tend to be on the innateness side

01:03:30.060 | that there's some things that we've been evolved

01:03:35.060 | to be able to learn,

01:03:39.660 | and that learning just can't happen without them.

01:03:44.580 | So, one example, here's an example I had in the book

01:03:48.100 | that I think is useful to me, at least,

01:03:50.620 | in thinking about this.

01:03:51.460 | So, this has to do with the DeepMind's

01:03:56.180 | Atari game-playing program, okay?

01:03:59.340 | And it learned to play these Atari video games

01:04:02.780 | just by getting input from the pixels of the screen,

01:04:07.780 | and it learned to play the game Breakout

01:04:15.780 | 1,000% better than humans, okay?

01:04:18.220 | That was one of their results, and it was great.

01:04:20.460 | And it learned this thing where it tunneled

01:04:22.980 | through the side of the bricks in the Breakout game,

01:04:26.460 | and the ball could bounce off the ceiling,

01:04:28.700 | and then just wipe out bricks.

01:04:30.740 | Okay, so there was a group who did an experiment

01:04:35.740 | where they took the paddle that you move with the joystick

01:04:41.540 | and moved it up two pixels or something like that.

01:04:45.660 | And then they looked at a deep Q-learning system

01:04:50.500 | that had been trained on Breakout and said,

01:04:52.140 | "Could it now transfer its learning

01:04:53.900 | to this new version of the game?"

01:04:55.860 | Of course, a human could, but, and it couldn't.

01:04:59.660 | Maybe that's not surprising, but I guess the point is

01:05:01.500 | it hadn't learned the concept of a paddle.

01:05:05.220 | It hadn't learned the concept of a ball

01:05:08.140 | or the concept of tunneling.

01:05:09.460 | It was learning something, you know,

01:05:11.860 | we, looking at it, kind of anthropomorphized it and said,

01:05:16.660 | "Oh, here's what it's doing in the way we describe it."

01:05:19.220 | But it actually didn't learn those concepts.

01:05:21.500 | And so because it didn't learn those concepts,

01:05:24.020 | it couldn't make this transfer.

01:05:26.940 | - Yeah, so that's a beautiful statement,

01:05:28.980 | but at the same time, by moving the paddle,

01:05:31.640 | we also anthropomorphize flaws to inject into the system

01:05:36.460 | that will then flip how impressed we are by it.

01:05:40.020 | What I mean by that is, to me, the Atari games

01:05:43.740 | were to me deeply impressive that that was possible at all.

01:05:48.740 | So like I have to first pause on that

01:05:50.780 | and people should look at that, just like the game of Go,

01:05:53.780 | which is fundamentally different to me

01:05:56.060 | than what Deep Blue did.

01:05:59.640 | Even though there's still Monte Carlo,

01:06:01.740 | there's still Tree Search.

01:06:03.300 | It's just everything that DeepMind has done

01:06:07.260 | in terms of learning, however limited it is,

01:06:10.500 | is still deeply surprising to me.

01:06:12.100 | - Yeah, I'm not trying to say

01:06:14.540 | that what they did wasn't impressive.

01:06:16.540 | I think it was incredibly impressive.

01:06:18.180 | - To me, it's interesting.

01:06:19.940 | Is moving the board just another thing

01:06:24.100 | that needs to be learned?

01:06:25.100 | So like we've been able to, maybe, maybe,

01:06:28.340 | been able to, through the current neural networks,

01:06:30.180 | learn very basic concepts that are not enough

01:06:32.860 | to do this general reasoning.

01:06:34.740 | And maybe with more data, I mean the data,

01:06:39.740 | the interesting thing about the examples

01:06:42.380 | that you talk about beautifully

01:06:45.060 | is it's often flaws of the data.

01:06:49.020 | - Well, that's the question.

01:06:49.980 | I mean, I think that is the key question,

01:06:52.220 | whether it's a flaw of the data or not.

01:06:53.860 | - Or the mechanics.

01:06:54.700 | - Because the reason I brought up this example

01:06:57.100 | was 'cause you were asking, do I think that

01:06:59.300 | learning from data could go all the way?

01:07:02.580 | And this was why I brought up the example,

01:07:04.900 | because I think, and this is not at all

01:07:08.260 | to take away from the impressive work that they did,

01:07:12.260 | but it's to say that when we look

01:07:14.340 | at what these systems learn,

01:07:16.100 | do they learn the human, the things that we humans

01:07:23.460 | consider to be the relevant concepts?

01:07:25.740 | And in that example, it didn't.

01:07:29.780 | Sure, if you train it on moving,

01:07:33.620 | the paddle being in different places,

01:07:36.300 | maybe it could deal with, maybe it would learn that concept.

01:07:41.180 | I'm not totally sure, but the question is,

01:07:44.460 | scaling that up to more complicated worlds,

01:07:47.460 | to what extent could a machine

01:07:51.820 | that only gets this very raw data

01:07:54.820 | learn to divide up the world into relevant concepts?

01:07:59.540 | And I don't know the answer, but I would bet

01:08:04.020 | that without some innate notion, that it can't do it.

01:08:09.020 | - Yeah, 10 years ago, I 100% agree with you

01:08:13.420 | as the most experts in AI system,

01:08:16.100 | but now I have a glimmer of hope.

01:08:20.140 | - Okay, that's fair enough.

01:08:21.820 | - And I think that's what deep learning did

01:08:23.700 | in the community, is, no, no, no,

01:08:25.300 | I still, if I had to bet all my money,

01:08:27.140 | 100% deep learning will not take us all the way,

01:08:30.060 | but there's still, I was so personally sort of surprised

01:08:35.060 | by the Atari games, by Go, by the power of self-play,

01:08:39.900 | of just game playing against each other,

01:08:42.180 | that I was, like many other times,

01:08:44.980 | just humbled of how little I know

01:08:46.860 | about what's possible in this way of approaching it.

01:08:50.060 | - Yeah, I think, fair enough.

01:08:51.740 | Self-play is amazingly powerful.

01:08:54.260 | And that goes way back to Arthur Samuel, right,

01:08:58.700 | with his checker playing program,

01:09:01.140 | and that which was brilliant,

01:09:03.540 | and surprising that it did so well.

01:09:05.940 | - So just for fun, let me ask you

01:09:09.700 | on the topic of autonomous vehicles.

01:09:11.340 | It's the area that I work, at least these days,

01:09:15.340 | most closely on, and it's also area

01:09:18.100 | that I think is a good example that you use

01:09:20.780 | as sort of an example of things we, as humans,

01:09:25.340 | don't always realize how hard it is to do.

01:09:28.660 | It's like the constant trend in AI,

01:09:30.900 | or the different problems that we think are easy

01:09:32.900 | when we first try them, and then realize how hard it is.

01:09:36.460 | Okay, so why, you've talked about this,

01:09:41.460 | autonomous driving being a difficult problem,

01:09:43.540 | more difficult than we realize, humans give it credit for.

01:09:46.860 | Why is it so difficult?

01:09:48.140 | What are the most difficult parts, in your view?

01:09:50.540 | - I think it's difficult because of the world

01:09:55.780 | is so open-ended as to what kinds of things can happen.

01:10:00.180 | So you have sort of what normally happens,

01:10:05.180 | which is just you drive along,

01:10:07.380 | and nothing surprising happens,

01:10:10.580 | and autonomous vehicles can do,

01:10:13.420 | the ones we have now, evidently,

01:10:15.500 | can do really well on most normal situations,

01:10:19.540 | as long as the weather is reasonably good and everything.

01:10:23.340 | But if some, we have this notion of edge case,

01:10:28.900 | or things in the tail of the distribution,

01:10:32.700 | people call it the long tail problem,

01:10:34.740 | which says that there's so many possible things

01:10:37.900 | that can happen that was not in the training data

01:10:42.100 | of the machine that it won't be able to handle it

01:10:47.100 | because it doesn't have common sense.

01:10:50.900 | - Right, it's the old, the paddle moved problem.

01:10:54.700 | - Yeah, it's the paddle moved problem, right.

01:10:57.860 | And so my understanding,

01:10:59.180 | and you probably are more of an expert than I am on this,

01:11:02.140 | is that current self-driving car vision systems

01:11:07.140 | have problems with obstacles,

01:11:10.460 | meaning that they don't know which obstacles,

01:11:13.900 | which quote-unquote obstacles they should stop for

01:11:16.620 | and which ones they shouldn't stop for.

01:11:18.580 | And so a lot of times I read that they tend to slam

01:11:22.260 | on the brakes quite a bit,

01:11:23.900 | and the most common accidents with self-driving cars

01:11:28.140 | are people rear-ending them,

01:11:30.500 | 'cause they were surprised,

01:11:32.180 | they weren't expecting the car to stop.

01:11:35.740 | - Yeah, so there's a lot of interesting questions there,

01:11:39.100 | whether, 'cause you mentioned kind of two things.

01:11:42.900 | So one is the problem of perception,

01:11:45.100 | of understanding, of interpreting the objects

01:11:49.500 | that are detected correctly.

01:11:51.540 | And the other one is more like the policy,

01:11:54.380 | the action that you take, how you respond to it.

01:11:57.740 | So a lot of the cars braking is a kind of notion of,

01:12:02.460 | to clarify it, there's a lot of different kind of things

01:12:06.380 | that are people calling autonomous vehicles,

01:12:07.940 | but the L4 vehicles with a safety driver

01:12:11.780 | are the ones like Waymo and Cruise and those companies,

01:12:15.780 | they tend to be very conservative and cautious.

01:12:18.660 | So they tend to be very, very afraid

01:12:21.260 | of hurting anything or anyone

01:12:22.980 | and getting in any kind of accidents.

01:12:24.940 | So their policy is very kind of,

01:12:27.900 | that results in being exceptionally responsive

01:12:31.100 | to anything that could possibly be an obstacle, right?

01:12:33.620 | - Right, which the human drivers around it,

01:12:37.260 | it's unpredictable, it behaves unpredictably.

01:12:41.660 | - Yeah, that's not a very human thing to do, caution.

01:12:44.100 | That's not the thing we're good at, especially in driving.

01:12:46.580 | We're in a hurry, often angry and et cetera,

01:12:49.780 | especially in Boston.

01:12:50.900 | So, and then there's sort of another,

01:12:53.900 | and a lot of times that's,

01:12:55.940 | machine learning is not a huge part of that.

01:12:58.260 | It's becoming more and more unclear to me

01:13:00.540 | how much, sort of speaking to public information,

01:13:05.540 | because a lot of companies say they're doing deep learning

01:13:09.220 | and machine learning just to attract good candidates.

01:13:12.580 | The reality is in many cases,

01:13:14.780 | it's still not a huge part of the perception.

01:13:18.540 | There's LIDAR and there's other sensors

01:13:20.460 | that are much more reliable for obstacle detection.

01:13:23.900 | And then there's Tesla approach, which is vision only.

01:13:27.940 | And there's, I think a few companies doing that,

01:13:30.860 | but Tesla most sort of famously pushing that forward.

01:13:33.420 | - And that's because the LIDAR is too expensive, right?

01:13:36.140 | - Well, I mean, yes, but I would say

01:13:41.140 | if you were to for free give to every Tesla vehicle,

01:13:45.260 | I mean, Elon Musk fundamentally believes

01:13:47.660 | that LIDAR is a crutch, right?

01:13:49.260 | Fantasy said that.

01:13:50.820 | That if you want to solve the problem with machine learning,

01:13:55.620 | LIDAR should not be the primary sensor is the belief.

01:14:00.620 | The camera contains a lot more information.

01:14:04.220 | So if you want to learn, you want that information.

01:14:08.480 | But if you want to not to hit obstacles, you want LIDAR.

01:14:13.700 | Right, it's sort of, it's this weird trade-off

01:14:16.340 | because yeah, so what Tesla vehicles have a lot of,

01:14:21.700 | which is really the thing, the fallback,

01:14:26.700 | the primary fallback sensor is radar,

01:14:29.740 | which is a very crude version of LIDAR.

01:14:32.420 | It's a good detector of obstacles,

01:14:35.060 | except when those things are standing, right?

01:14:38.140 | The stopped vehicle.

01:14:39.980 | - Right, that's why it had problems

01:14:41.380 | with crashing into stopped fire trucks.

01:14:43.620 | - Stopped fire trucks, right?

01:14:44.900 | So the hope there is that the vision sensor

01:14:47.860 | would somehow catch that and infer.

01:14:50.420 | So there's a lot of problems with perception.

01:14:52.660 | They are doing actually some incredible stuff in the,

01:14:58.500 | almost like an active learning space

01:15:02.500 | where it's constantly taking edge cases and pulling back in.

01:15:06.940 | There's this data pipeline.

01:15:08.780 | Another aspect that is really important

01:15:13.060 | that people are studying now is called multitask learning,

01:15:15.860 | which is sort of breaking apart this problem,

01:15:18.420 | whatever the problem is, in this case, driving,

01:15:20.620 | into dozens or hundreds of little problems

01:15:24.460 | that you can turn into learning problems.

01:15:26.260 | So this giant pipeline, it's kind of interesting.

01:15:30.300 | I've been skeptical from the very beginning,

01:15:33.340 | but become less and less skeptical over time

01:15:35.540 | how much of driving can be learned.

01:15:37.580 | I still think it's much farther than the CEO

01:15:42.060 | of that particular company thinks it will be,

01:15:44.700 | but it's constantly surprising

01:15:48.140 | that through good engineering and data collection

01:15:51.740 | and active selection of data,

01:15:54.180 | how you can attack that long tail.

01:15:56.660 | And it's an interesting open question

01:15:58.940 | that you're absolutely right.

01:16:00.140 | There's a much longer tail and all these edge cases

01:16:03.060 | that we don't think about,

01:16:04.620 | but it's a fascinating question

01:16:06.500 | that applies to natural language and all spaces.

01:16:09.340 | How big is that long tail?

01:16:13.060 | And I mean, not to linger on the point,

01:16:16.940 | but what's your sense in driving

01:16:19.220 | in these practical problems of the human experience?

01:16:23.780 | Can it be learned?

01:16:26.780 | So the current, what are your thoughts

01:16:28.180 | of sort of Elon Musk thought,

01:16:30.620 | let's forget the thing that he says

01:16:32.060 | it'd be solved in a year,

01:16:33.620 | but can it be solved in a reasonable timeline

01:16:38.620 | or do fundamentally other methods need to be invented?

01:16:42.020 | - So I don't, I think that ultimately driving,

01:16:47.020 | so it's a trade-off in a way,

01:16:49.980 | being able to drive and deal with any situation

01:16:55.220 | that comes up does require kind of full human intelligence

01:17:00.220 | and even in humans aren't intelligent enough to do it

01:17:03.300 | 'cause humans, I mean, most human accidents

01:17:06.220 | are because the human wasn't paying attention

01:17:10.020 | or the humans drunk or whatever.

01:17:12.420 | - And not because they weren't intelligent enough.

01:17:14.180 | - And not because they weren't intelligent enough, right.

01:17:16.900 | Whereas the accidents with autonomous vehicles

01:17:23.340 | is because they weren't intelligent enough.

01:17:25.740 | - They're always paying attention.

01:17:26.580 | - Yeah, they're always paying attention.

01:17:27.620 | So it's a trade-off, you know,

01:17:29.540 | and I think that it's a very fair thing to say

01:17:32.660 | that autonomous vehicles will be ultimately safer

01:17:36.300 | than humans 'cause humans are very unsafe.

01:17:39.620 | It's kind of a low bar.

01:17:42.340 | - But just like you said,

01:17:45.540 | I think humans got a better rap, right?

01:17:49.100 | 'Cause we're really good at the common sense thing.

01:17:50.900 | - Yeah, we're great at the common sense thing.

01:17:52.420 | We're bad at the paying attention thing.

01:17:54.020 | - Paying attention thing, right.

01:17:55.100 | - Especially when we're, you know, driving's kind of boring

01:17:57.180 | and we have these phones to play with and everything.

01:17:59.900 | But I think what's gonna happen is that

01:18:05.100 | for many reasons, not just AI reasons,

01:18:08.460 | but also like legal and other reasons,

01:18:11.540 | that the definition of self-driving is gonna change

01:18:16.540 | or autonomous is gonna change.

01:18:18.380 | It's not gonna be just,

01:18:21.380 | I'm gonna go to sleep in the back

01:18:23.820 | and you just drive me anywhere.

01:18:25.380 | It's gonna be more,

01:18:27.740 | certain areas are going to be instrumented

01:18:33.180 | to have the sensors and the mapping

01:18:36.580 | and all of the stuff you need

01:18:37.900 | that the autonomous cars won't have

01:18:40.140 | to have full common sense.

01:18:42.420 | And they'll do just fine in those areas

01:18:45.620 | as long as pedestrians don't mess with them too much.

01:18:48.900 | That's another question.

01:18:50.060 | (laughs)

01:18:50.900 | - That's right.

01:18:51.740 | That's the human.

01:18:52.580 | - But I don't think we will have fully autonomous

01:18:57.580 | self-driving in the way that like most,

01:19:00.540 | the average person thinks of it.

01:19:02.260 | The person thinks of it for a very long time.

01:19:05.260 | - And just to reiterate,

01:19:07.580 | this is the interesting open question

01:19:10.020 | that I think I agree with you on

01:19:12.180 | is to solve fully autonomous driving,

01:19:14.860 | you have to be able to engineer in common sense.

01:19:17.820 | - Yes.

01:19:18.660 | - I think it's an important thing to hear and think about.

01:19:23.780 | I hope that's wrong,

01:19:25.060 | but I currently agree with you

01:19:28.060 | that unfortunately you do have to have,

01:19:31.580 | to be more specific,

01:19:32.780 | sort of these deep understandings of physics

01:19:35.300 | and of the way this world works.

01:19:38.580 | And also the human dynamics.

01:19:39.900 | Like you mentioned, pedestrians and cyclists,

01:19:41.540 | actually that's whatever that nonverbal communication

01:19:45.460 | as some people call it,

01:19:47.220 | there's that dynamic that is also part of this common sense.

01:19:51.380 | - Right.

01:19:52.220 | And we humans are pretty good at predicting

01:19:55.860 | what other humans are gonna do.

01:19:57.980 | - And how our actions impact the behaviors

01:20:01.380 | so there's this weird game theoretic dance

01:20:03.780 | that we're good at somehow.

01:20:05.660 | And the funny thing is,

01:20:07.860 | 'cause I've watched countless hours of pedestrian video

01:20:12.060 | and talked to people,

01:20:13.140 | we humans are also really bad at articulating

01:20:15.900 | the knowledge we have.

01:20:17.460 | - Right.

01:20:18.300 | - Which has been a huge challenge.

01:20:20.260 | - Yes.

01:20:21.100 | - So you've mentioned embodied intelligence.

01:20:24.220 | What do you think it takes to build a system

01:20:26.380 | of human level intelligence?

01:20:27.660 | Does it need to have a body?

01:20:30.460 | - I'm not sure, but I'm coming around to that more and more.

01:20:35.460 | - And what does it mean to be,

01:20:37.620 | I don't mean to keep bringing up Yalun Kun.

01:20:40.500 | - He looms very large.

01:20:42.780 | - Well, he certainly has a large personality, yes.

01:20:46.860 | He thinks that the system needs to be grounded,

01:20:50.300 | meaning it needs to sort of be able to interact

01:20:53.020 | with reality,

01:20:54.340 | but doesn't think it necessarily needs to have a body.

01:20:56.940 | So when you think of--

01:20:57.780 | - So what's the difference?

01:20:59.540 | I guess I wanna ask, when you mean body,

01:21:02.580 | do you mean you have to be able to play with the world?

01:21:05.300 | Or do you also mean like there's a body

01:21:07.300 | that you have to preserve?

01:21:09.700 | - Oh, that's a good question.

01:21:12.660 | I haven't really thought about that,

01:21:13.820 | but I think both, I would guess.

01:21:16.660 | Because I think you,

01:21:20.300 | I think intelligence,

01:21:24.100 | it's so hard to separate it from our self,

01:21:28.580 | our desire for self-preservation,

01:21:32.020 | our emotions,

01:21:34.700 | our all that non-rational stuff

01:21:37.620 | that kind of gets in the way of logical thinking.

01:21:42.620 | Because the way,

01:21:45.820 | we're talking about human intelligence

01:21:48.700 | or human level intelligence, whatever that means,

01:21:51.380 | a huge part of it is social.

01:21:56.820 | We were evolved to be social

01:21:58.900 | and to deal with other people.

01:22:01.620 | And that's just so ingrained in us

01:22:04.860 | that it's hard to separate intelligence from that.

01:22:08.740 | I think AI for the last 70 years

01:22:14.700 | or however long it's been around,

01:22:17.100 | it has largely been separated.

01:22:18.700 | There's this idea that there's like,

01:22:20.860 | it's kind of very Cartesian.

01:22:23.780 | There's this thinking thing

01:22:26.580 | that we're trying to create,

01:22:27.820 | but we don't care about all this other stuff.

01:22:30.900 | And I think the other stuff is very fundamental.

01:22:35.260 | - So there's the idea that things like emotion

01:22:37.820 | get in the way of intelligence.

01:22:40.220 | - As opposed to being an integral part of it.

01:22:42.700 | - Integral part of it.

01:22:43.540 | So, I mean, I'm Russian,

01:22:45.460 | so romanticize the notions of emotion and suffering

01:22:48.460 | and all that kind of fear of mortality,

01:22:51.180 | those kinds of things.

01:22:52.020 | So in AI, especially,

01:22:55.540 | so I've--

01:22:56.660 | - By the way, did you see that?

01:22:57.860 | There was this recent thing going around the internet.

01:23:00.340 | This, some, I think he's a Russian or some Slavic

01:23:03.860 | had written this thing,

01:23:05.220 | sort of anti the idea of super intelligence.

01:23:08.740 | I forgot, maybe he's Polish.

01:23:10.460 | Anyway, so he had all these arguments

01:23:12.460 | and one was the argument from Slavic pessimism.

01:23:15.620 | (both laughing)

01:23:18.260 | My favorite.

01:23:19.100 | - Do you remember what the argument is?

01:23:22.020 | - It's like nothing ever works.

01:23:24.020 | - Yeah. - Everything sucks.

01:23:26.500 | - So what do you think is the role,

01:23:30.100 | like that's such a fascinating idea

01:23:32.020 | that what we perceive as sort of the limits of human,

01:23:36.380 | of the human mind, which is emotion and fear

01:23:41.500 | and all those kinds of things are integral to intelligence.

01:23:45.080 | Could you elaborate on that?

01:23:48.220 | Like what, why is that important, do you think?

01:23:54.020 | For human level intelligence?

01:23:57.460 | - At least for the way that humans work,

01:24:00.780 | it's a big part of how it affects how we perceive the world.

01:24:04.860 | It affects how we make decisions about the world.

01:24:07.780 | It affects how we interact with other people.

01:24:10.020 | It affects our understanding of other people.

01:24:13.180 | For me to understand your,

01:24:16.780 | what you're likely to do,

01:24:21.540 | I need to have kind of a theory of mind

01:24:23.180 | and that's very much a theory of emotion

01:24:26.660 | and motivations and goals.

01:24:29.860 | And to understand that,

01:24:34.340 | we have this whole system of mirror neurons.

01:24:41.060 | I sort of understand your motivations

01:24:45.500 | through sort of simulating it myself.

01:24:49.380 | So, it's not something that I can prove that's necessary,

01:24:54.380 | but it seems very likely.

01:24:58.180 | - So, okay.

01:25:01.740 | You've written the op-ed in the New York Times

01:25:04.100 | titled "We Shouldn't Be Scared by Superintelligent AI"

01:25:07.740 | and it criticized a little bit Stuart Russell

01:25:10.940 | and Nick Bostrom.

01:25:11.900 | Can you try to summarize that article's key ideas?

01:25:18.260 | - So, it was spurred by an earlier New York Times op-ed

01:25:22.820 | by Stuart Russell, which was summarizing his book

01:25:26.300 | called "Human Compatible."

01:25:28.900 | And the article was saying,

01:25:31.420 | if we have superintelligent AI,

01:25:37.300 | we need to have its values aligned with our values

01:25:40.900 | and it has to learn about what we really want.

01:25:43.940 | And he gave this example.

01:25:45.580 | What if we have a superintelligent AI

01:25:48.900 | and we give it the problem of solving climate change

01:25:52.820 | and it decides that the best way to lower the carbon

01:25:57.100 | in the atmosphere is to kill all the humans?

01:25:59.900 | Okay.

01:26:00.740 | So, to me, that just made no sense at all

01:26:02.620 | because a superintelligent AI,

01:26:06.340 | first of all, trying to figure out

01:26:10.860 | what superintelligence means.

01:26:14.180 | And it seems that something that's superintelligent

01:26:19.180 | can't just be intelligent along this one dimension of,

01:26:25.180 | okay, I'm gonna figure out all the steps,

01:26:27.180 | the best optimal path to solving climate change

01:26:30.620 | and not be intelligent enough to figure out

01:26:33.020 | that humans don't wanna be killed,

01:26:35.820 | that you could get to one without having the other.

01:26:39.940 | And Bostrom, in his book,

01:26:43.780 | talks about the orthogonality hypothesis

01:26:46.620 | where he says he thinks that a system's,

01:26:49.480 | I can't remember exactly what it is,

01:26:52.780 | but a system's goals and its values

01:26:56.980 | don't have to be aligned.

01:26:58.420 | There's some orthogonality there

01:27:00.420 | which didn't make any sense to me.

01:27:02.580 | - So, you're saying in any system

01:27:05.180 | that's sufficiently, not even superintelligent,

01:27:07.700 | but as it approaches greater and greater intelligence,

01:27:10.300 | there's a holistic nature that will sort of,

01:27:12.380 | a tension that will naturally emerge

01:27:15.420 | that prevents it from sort of

01:27:16.580 | any one dimension running away.

01:27:18.420 | - Yeah, yeah, exactly.

01:27:20.100 | So, Bostrom had this example of the superintelligent AI

01:27:25.100 | that turns the world into paperclips

01:27:31.140 | 'cause its job is to make paperclips or something.

01:27:34.300 | And that just, as a thought experiment,

01:27:36.540 | didn't make any sense to me. (laughs)

01:27:38.540 | - Well, as a thought experiment

01:27:40.380 | or as a thing that could possibly be realized?

01:27:43.740 | - Either.

01:27:44.580 | So, I think that what my op-ed was trying to do

01:27:48.180 | was say that intelligence is more complex

01:27:51.380 | than these people are presenting it,

01:27:54.940 | that it's not so separable,

01:27:59.380 | the rationality, the values, the emotions,

01:28:05.060 | all of that, that it's the view

01:28:07.900 | that you could separate all these dimensions

01:28:10.460 | and build a machine that has one of these dimensions

01:28:13.140 | and it's superintelligent in one dimension,

01:28:15.500 | but it doesn't have any of the other dimensions.

01:28:17.860 | That's what I was trying to criticize,

01:28:22.860 | that I don't believe that.

01:28:25.820 | - So, can I read a few sentences from Yoshua Bengio,

01:28:31.220 | who is always super eloquent?

01:28:35.140 | So, he writes, "I have the same impression as Melanie

01:28:41.300 | "that our cognitive biases are linked

01:28:43.180 | "with our ability to learn to solve many problems.

01:28:46.380 | "They may also be a limiting factor for AI.

01:28:49.100 | "However," this is a may in quotes,

01:28:54.200 | "things may also turn out differently

01:28:55.800 | "and there's a lot of uncertainty

01:28:57.020 | "about the capabilities of future machines.

01:29:00.560 | "But more importantly for me,

01:29:02.560 | "the value alignment problem is a problem

01:29:05.440 | "well before we reach some hypothetical superintelligence.

01:29:08.900 | "It is already posing a problem

01:29:10.500 | "in the form of super powerful companies

01:29:12.660 | "whose objective function may not be sufficiently aligned

01:29:17.660 | "with humanity's general well-being,

01:29:19.260 | "creating all kinds of harmful side effects."

01:29:21.860 | So, he goes on to argue that,

01:29:23.820 | the orthogonality and those kinds of things,

01:29:29.500 | the concerns of just aligning values

01:29:33.020 | with the capabilities of the system

01:29:35.200 | is something that might come long

01:29:37.740 | before we reach anything like superintelligence.

01:29:40.780 | So, your criticism is kind of really nice to saying,

01:29:44.500 | this idea of superintelligence systems

01:29:46.980 | seem to be dismissing fundamental parts

01:29:49.100 | of what intelligence would take.

01:29:50.820 | And then Yoshua kind of says, "Yes,

01:29:53.740 | "but if we look at systems that are much less intelligent,

01:29:58.160 | "there might be these same kinds of problems that emerge."

01:30:02.160 | - Sure, but I guess the example that he gives there

01:30:06.740 | of these corporations, that's people, right?

01:30:10.100 | Those are people's values.

01:30:12.140 | I mean, we're talking about people,

01:30:14.420 | the corporations are,

01:30:17.260 | their values are the values of the people

01:30:20.580 | who run those corporations.

01:30:21.700 | - But the idea is the algorithm, that's right.

01:30:24.740 | So, the fundamental person,

01:30:27.160 | the fundamental element of what does the bad thing

01:30:30.600 | is a human being.

01:30:32.280 | But the algorithm kind of controls the behavior

01:30:37.120 | of this mass of human beings.

01:30:39.160 | - Which algorithm?

01:30:40.000 | - For a company that's,

01:30:42.880 | so for example, if it's an advertisement-driven company

01:30:45.320 | that recommends certain things and encourages engagement,

01:30:50.320 | so it gets money by encouraging engagement,

01:30:54.600 | and therefore the company more and more,

01:30:57.140 | it's like the cycle that builds an algorithm

01:31:01.420 | that enforces more engagement

01:31:04.140 | and may perhaps more division in the culture

01:31:06.180 | and so on, so on.

01:31:07.820 | - I guess the question here is sort of who has the agency?

01:31:12.140 | So, you might say, for instance,

01:31:15.020 | we don't want our algorithms to be racist.

01:31:17.500 | And facial recognition,

01:31:21.940 | some people have criticized some facial recognition systems

01:31:24.500 | as being racist 'cause they're not as good

01:31:27.200 | on darker skin than lighter skin.

01:31:30.920 | Okay, but the agency there,

01:31:33.280 | the actual facial recognition algorithm

01:31:37.220 | isn't what has the agency.

01:31:39.000 | It's not the racist thing, right?

01:31:41.440 | It's the, I don't know,

01:31:45.040 | the combination of the training data,

01:31:49.440 | the cameras being used, whatever.

01:31:52.000 | But my understanding of,

01:31:54.160 | and I agree with Benjio there that he,

01:31:57.420 | I think there are these value issues

01:32:00.380 | with our use of algorithms,

01:32:03.140 | but my understanding of what Russell's argument was

01:32:08.140 | is more that the machine itself has the agency now.

01:32:15.020 | It's the thing that's making the decisions,

01:32:17.660 | and it's the thing that has what we would call values.

01:32:20.720 | - Yes.

01:32:23.920 | - Whether that's just a matter of degree,

01:32:26.000 | it's hard to say, right?

01:32:27.940 | But I would say that's sort of qualitatively different

01:32:30.340 | than a face recognition neural network.

01:32:34.180 | - And to broadly linger on that point,

01:32:38.720 | if you look at Elon Musk or Stuart Russell or Bostrom,

01:32:42.920 | people who are worried about existential risks of AI,

01:32:46.300 | however far into the future,

01:32:47.900 | their argument goes is it eventually happens.

01:32:50.720 | We don't know how far, but it eventually happens.

01:32:53.580 | Do you share any of those concerns?

01:32:56.800 | And what kind of concerns in general do you have about AI

01:33:00.120 | that approach anything like existential threat to humanity?

01:33:04.340 | - So I would say, yes, it's possible,

01:33:10.480 | but I think there's a lot more closer-in

01:33:14.520 | existential threats to humanity.

01:33:16.040 | - 'Cause you said like 100 years for, so your time--

01:33:19.120 | - It's more than 100 years.

01:33:20.280 | - More than 100 years, and so that means--

01:33:22.200 | - Maybe even more than 500 years, I don't know.

01:33:25.440 | - So the existential threats are so far out

01:33:28.320 | that the future is, I mean,

01:33:31.040 | there'll be a million different technologies

01:33:33.240 | that we can't even predict now

01:33:34.480 | that will fundamentally change the nature of our behavior,

01:33:37.600 | reality, society, and so on before then.

01:33:39.880 | - I think so, I think so.

01:33:41.680 | And we have so many other pressing existential threats

01:33:45.880 | going on right now.

01:33:46.720 | - Nuclear weapons even.

01:33:48.040 | - Nuclear weapons, climate problems,

01:33:50.720 | poverty, possible pandemics, you can go on and on.

01:33:58.280 | And I think worrying about existential threat from AI

01:34:04.800 | is not the best priority for what we should be worried about.

01:34:14.160 | That's kind of my view, 'cause we're so far away.

01:34:15.800 | But I'm not necessarily criticizing Russell or Bostrom

01:34:20.800 | or whoever for worrying about that.

01:34:26.840 | And I think some people should be worried about it.

01:34:30.000 | It's certainly fine, but I was more sort of getting

01:34:34.040 | at their view of what intelligence is.

01:34:39.040 | So I was more focusing on their view of super intelligence

01:34:43.320 | than just the fact of them worrying.

01:34:48.320 | And the title of the article was written

01:34:52.920 | by the New York Times editors.

01:34:54.960 | I wouldn't have called it that.

01:34:56.520 | - We shouldn't be scared by super intelligence.

01:34:59.520 | - No.

01:35:00.360 | - If you wrote it, it'd be like,

01:35:01.320 | we should redefine what you mean by super intelligence.

01:35:03.440 | - I actually said something like super intelligence

01:35:07.440 | is not a sort of coherent,

01:35:12.440 | sort of coherent idea.

01:35:14.080 | But that's not like something New York Times would put in.

01:35:19.680 | - And the follow-up argument that Yoshua makes also,

01:35:23.440 | not argument, but a statement,

01:35:24.920 | and I've heard him say it before,

01:35:26.520 | and I think I agree.

01:35:28.480 | He kind of has a very friendly way of phrasing it

01:35:30.800 | as it's good for a lot of people

01:35:33.000 | to believe different things.

01:35:34.600 | - He's such a nice guy.

01:35:36.880 | - Yeah, but it's also practically speaking,

01:35:39.600 | like we shouldn't be, like while your article stands,

01:35:44.600 | like Stuart Russell does amazing work,

01:35:47.360 | Bostrom does amazing work, you do amazing work.

01:35:50.120 | And even when you disagree about the definition

01:35:53.200 | of super intelligence or the usefulness of even the term,

01:35:56.840 | it's still useful to have people that like use that term,

01:36:01.720 | right, and then argue.

01:36:03.000 | - Sure, I absolutely agree with Benjo there.

01:36:06.520 | And I think it's great that, you know,

01:36:08.600 | and it's great that New York Times

01:36:09.800 | will publish all this stuff.

01:36:10.880 | - That's right.

01:36:12.560 | It's an exciting time to be here.

01:36:14.520 | What do you think is a good test of intelligence?

01:36:17.320 | Like is natural language ultimately a test

01:36:20.680 | that you find the most compelling, like the original,

01:36:23.720 | or the higher levels of the Turing test kind of, yeah?

01:36:28.720 | - Yeah, I still think the original idea of the Turing test

01:36:34.560 | is a good test for intelligence.

01:36:37.640 | I mean, I can't think of anything better.

01:36:39.960 | You know, the Turing test,

01:36:41.640 | the way that it's been carried out so far

01:36:43.560 | has been very impoverished, if you will.

01:36:48.560 | But I think a real Turing test that really goes into depth,

01:36:52.800 | like the one that I mentioned, I talk about in the book,

01:36:54.720 | I talk about Ray Kurzweil and Mitchell Kapoor

01:36:57.880 | have this bet, right, that in 2029,

01:37:02.880 | I think is the date there,

01:37:05.800 | a machine will pass the Turing test.

01:37:07.440 | And they have a very specific, like how many hours,

01:37:12.440 | expert judges and all of that.

01:37:14.920 | And, you know, Kurzweil says yes, Kapoor says no.

01:37:18.120 | We only have like nine more years to go to see.

01:37:21.000 | But I, you know, if something, a machine could pass that,

01:37:26.960 | I would be willing to call it intelligent.

01:37:30.360 | - Of course, nobody will.

01:37:33.840 | They will say that's just a language model, right?

01:37:37.080 | If it does.

01:37:38.080 | So you would be comfortable, so language,

01:37:40.840 | a long conversation that, well, yeah,

01:37:45.160 | I mean, you're right, because I think probably

01:37:46.880 | to carry out that long conversation,

01:37:48.920 | you would literally need to have

01:37:50.400 | deep common sense understanding of the world.

01:37:52.400 | - I think so, I think so.

01:37:54.600 | - And the conversation is enough to reveal that.

01:37:57.600 | - I think so. - Perhaps it is.

01:37:59.920 | So another super fun topic of complexity

01:38:03.640 | that you have worked on, written about.

01:38:09.640 | Let me ask the basic question, what is complexity?

01:38:13.360 | - So complexity is another one of those terms,

01:38:16.480 | like intelligence, it's perhaps overused.

01:38:19.280 | But my book about complexity was about this wide area

01:38:28.240 | of complex systems, studying different systems in nature,

01:38:33.240 | in technology, in society, in which you have emergence,

01:38:38.240 | kind of like I was talking about with intelligence.

01:38:41.880 | You know, we have the brain, which has billions of neurons,

01:38:46.320 | and each neuron individually could be said

01:38:49.960 | to be not very complex compared to the system as a whole,

01:38:53.720 | but the system, the interactions of those neurons

01:38:58.160 | and the dynamics creates these phenomena

01:39:01.040 | that we call intelligence or consciousness,

01:39:04.340 | that we consider to be very complex.

01:39:09.440 | So the field of complexity is trying to find

01:39:13.800 | general principles that underlie all these systems

01:39:17.080 | that have these kinds of emergent properties.

01:39:20.120 | - And the emergence occurs from,

01:39:23.400 | underlying the complex system is usually simple,

01:39:26.840 | fundamental interactions.

01:39:28.240 | - Yes.

01:39:29.080 | - And the emergence happens when there's just a lot

01:39:33.160 | of these things interacting.

01:39:34.960 | - Yes.

01:39:35.800 | - And then most of science today,

01:39:41.160 | can you talk about what is reductionism?

01:39:43.480 | - Well, reductionism is when you try and take a system

01:39:50.520 | and divide it up into its elements,

01:39:55.240 | whether those be cells or atoms or subatomic particles,

01:40:00.240 | whatever your field is,

01:40:02.880 | and then try and understand those elements

01:40:06.480 | and then try and build up an understanding

01:40:09.480 | of the whole system by looking at sort of the sum

01:40:12.480 | of all the elements.

01:40:14.360 | - Yeah, so what's your sense,

01:40:16.200 | whether we're talking about intelligence

01:40:17.920 | or these kinds of interesting complex systems,

01:40:21.080 | is it possible to understand them in a reductionist way?

01:40:25.160 | Which is probably the approach

01:40:27.000 | of most of science today, right?

01:40:29.240 | - I don't think it's always possible to understand

01:40:33.400 | the things we want to understand the most.

01:40:35.840 | So I don't think it's possible to look at single neurons

01:40:40.000 | and understand what we call intelligence,

01:40:45.880 | to look at sort of summing up.

01:40:48.360 | And the sort of the summing up is the issue here

01:40:53.440 | that we're, you know, one example is that the human genome,

01:40:57.840 | right, so there was a lot of work on excitement

01:41:01.920 | about sequencing the human genome,

01:41:04.000 | because the idea would be that we'd be able to find genes

01:41:08.680 | that underlies diseases.

01:41:11.480 | But it turns out that, and it was a very reductionist idea.

01:41:15.800 | You know, we figure out what all the parts are,

01:41:19.240 | and then we would be able to figure out which parts cause

01:41:21.920 | which things.

01:41:23.080 | But it turns out that the parts don't cause the things

01:41:25.440 | that we're interested in.

01:41:26.280 | It's like the interactions, it's the networks of these parts.

01:41:30.920 | And so that kind of reductionist approach

01:41:34.280 | didn't yield the explanation that we wanted.

01:41:37.240 | - What do you, what to use the most beautiful,

01:41:41.840 | complex system that you've encountered?

01:41:44.280 | - Most beautiful.

01:41:45.880 | - That you've been captivated by?

01:41:48.040 | Is it sort of, I mean, for me,

01:41:52.120 | it's the simplest would be cellular automata.

01:41:55.280 | - Oh, yeah.

01:41:56.120 | So I was very captivated by cellular automata,

01:41:58.680 | and worked on cellular automata for several years.

01:42:01.880 | - Do you find it amazing, or is it surprising

01:42:05.480 | that such simple systems, such simple rules

01:42:08.440 | in cellular automata can create sort of seemingly

01:42:12.080 | unlimited complexity?

01:42:14.760 | - Yeah, that was very surprising to me.

01:42:16.800 | - How do you make sense of it?

01:42:17.760 | How does that make you feel?

01:42:19.160 | Is it just ultimately humbling,

01:42:21.720 | or is there a hope to somehow leverage this

01:42:24.240 | into a deeper understanding,

01:42:27.040 | and even able to engineer things like intelligence?

01:42:30.320 | - It's definitely humbling.

01:42:32.600 | How humbling in that, also kind of awe-inspiring that,

01:42:37.600 | it's that awe-inspiring part of mathematics

01:42:43.000 | that these incredibly simple rules can produce

01:42:46.240 | this very beautiful, complex, hard to understand behavior.

01:42:52.200 | And that's, it's mysterious, you know,

01:42:56.800 | and surprising still, but exciting,

01:43:01.800 | 'cause it does give you kind of the hope

01:43:03.600 | that you might be able to engineer complexity

01:43:06.600 | just from simple rules. - From simple rules

01:43:08.720 | from the beginnings.

01:43:10.000 | Can you briefly say what is the Santa Fe Institute?

01:43:12.320 | Its history, its culture, its ideas, its future.

01:43:14.840 | So I've never, as I mentioned to you, I've never been,

01:43:18.360 | but it's always been this, in my mind,

01:43:20.800 | this mystical place where brilliant people

01:43:23.280 | study the edge of chaos.

01:43:25.400 | - Yeah, exactly.

01:43:26.600 | So the Santa Fe Institute was started in 1984,

01:43:32.760 | and it was created by a group of scientists,

01:43:37.280 | a lot of them from Los Alamos National Lab,

01:43:40.000 | which is about a 40-minute drive from Santa Fe Institute.

01:43:46.920 | They were mostly physicists and chemists,

01:43:49.160 | but they were frustrated in their field

01:43:53.640 | because they felt that their field wasn't approaching

01:43:58.480 | kind of big interdisciplinary questions

01:44:01.680 | like the kinds we've been talking about.

01:44:04.400 | And they wanted to have a place

01:44:06.120 | where people from different disciplines

01:44:08.920 | could work on these big questions

01:44:11.280 | without sort of being siloed into physics,

01:44:14.600 | chemistry, biology, whatever.

01:44:17.840 | So they started this institute,

01:44:20.000 | and this was people like George Cowan,

01:44:24.480 | who was a chemist in the Manhattan Project,

01:44:27.560 | and Nicholas Metropolis, who, a mathematician, physicist,

01:44:32.560 | Marie Gelman, physicist in his own,

01:44:37.440 | so some really big names here,

01:44:39.840 | Ken Arrow, a Nobel Prize-winning economist.

01:44:43.000 | And they started having these workshops.

01:44:46.240 | And this whole enterprise kind of grew

01:44:50.800 | into this research institute

01:44:54.120 | that itself has been kind of on the edge of chaos

01:44:58.440 | its whole life because it doesn't have any,

01:45:01.000 | it doesn't have a significant endowment,

01:45:04.040 | and it's just been kind of living on

01:45:07.600 | whatever funding it can raise through donations

01:45:12.520 | and grants and however it can,

01:45:17.520 | you know, business associates and so on.

01:45:20.840 | But it's a great place.

01:45:23.000 | It's a really fun place to go think about ideas

01:45:25.600 | that you wouldn't normally encounter.

01:45:29.080 | - So Sean Carroll, so physicists.

01:45:33.120 | - Yeah, he's on the external faculty.

01:45:35.080 | - And you mentioned that there's,

01:45:36.240 | so there's some external faculty

01:45:37.640 | and there's people that are--

01:45:38.480 | - A very small group of resident faculty.

01:45:41.600 | - Maybe about 10 who are there on five-year terms

01:45:46.600 | that can sometimes get renewed.

01:45:49.320 | And then they have some postdocs,

01:45:51.280 | and then they have this much larger,

01:45:53.680 | on the order of 100, external faculty

01:45:56.040 | or people like me who come and visit

01:45:58.240 | for various periods of time.

01:45:59.920 | - So what do you think is the future

01:46:01.120 | of the Santa Fe Institute?

01:46:03.160 | And if people are interested,

01:46:04.800 | what's there in terms of the public interaction

01:46:09.360 | or students or so on that could be a possible interaction

01:46:13.360 | with the Santa Fe Institute or its ideas?

01:46:15.680 | - Yeah, so there's a few different things they do.

01:46:18.680 | They have a complex system summer school

01:46:22.040 | for graduate students and postdocs,

01:46:23.720 | and sometimes faculty attend too.

01:46:25.920 | And that's a four-week,

01:46:27.800 | very intensive residential program

01:46:30.040 | where you go and you listen to lectures

01:46:32.600 | and you do projects, and people really like that.

01:46:36.200 | I mean, it's a lot of fun.

01:46:38.200 | They also have some specialty summer schools.

01:46:42.440 | There's one on computational social science.

01:46:45.600 | There's one on climate and sustainability,

01:46:50.600 | I think it's called.

01:46:51.760 | There's a few.

01:46:53.800 | And then they have short courses

01:46:56.200 | where just a few days on different topics.

01:46:58.760 | They also have an online education platform

01:47:03.960 | that offers a lot of different courses

01:47:06.080 | and tutorials from SFI faculty,

01:47:08.320 | including an introduction to complexity course that I taught.

01:47:12.960 | (both laughing)

01:47:14.320 | - Awesome, and there's a bunch of talks too online.

01:47:17.800 | There's guest speakers and so on.

01:47:19.600 | They host a lot of different-

01:47:20.440 | - Yeah, they have sort of technical seminars and colloquia,

01:47:25.440 | and they have a community lecture series,

01:47:28.460 | like public lectures,

01:47:30.240 | and they put everything on their YouTube channel

01:47:32.240 | so you can see it all.

01:47:33.720 | - Watch it.

01:47:34.880 | - Douglas Hostadter, author of "Gertl Escherbach,"

01:47:39.360 | was your PhD advisor.

01:47:40.600 | He mentioned a couple of times, and collaborator.

01:47:43.520 | Do you have any favorite lessons or memories

01:47:45.920 | from your time working with him

01:47:47.960 | that continues to this day, I guess?

01:47:50.080 | But just even looking back

01:47:52.160 | throughout your time working with him.

01:47:54.440 | - So one of the things he taught me

01:47:57.480 | was that when you're looking at a complex problem

01:48:04.120 | to idealize it as much as possible,

01:48:07.840 | to try and figure out what is the essence of this problem.

01:48:12.200 | And this is how the CopyCat program came into being,

01:48:16.560 | was by taking analogy making and saying,

01:48:19.000 | "How can we make this as idealized as possible,

01:48:21.320 | but still retain really the important things

01:48:23.960 | we wanna study?"

01:48:25.680 | And that's really been a core theme of my research, I think.

01:48:34.040 | And I continue to try and do that.

01:48:36.480 | And it's really very much kind of physics-inspired.

01:48:40.200 | Hofstadter was a PhD in physics.

01:48:42.680 | That was his background.

01:48:44.200 | - So like first principles kind of thinking,

01:48:45.840 | like you're reduced to the most fundamental aspect

01:48:48.920 | of the problem so that you can focus

01:48:50.960 | on solving that fundamental aspect.

01:48:52.280 | - Yeah, and in AI, that was,

01:48:54.720 | people used to work in these micro-worlds, right?

01:48:57.740 | Like the blocks world was a very early,

01:49:01.120 | important area in AI.

01:49:03.080 | And then that got criticized because they said,

01:49:06.040 | "Oh, you can't scale that to the real world."

01:49:08.820 | And so people started working on much more

01:49:12.080 | real-world-like problems.

01:49:14.600 | But now there's been kind of a return,

01:49:17.920 | even to the blocks world itself.

01:49:19.800 | We've seen a lot of people who are trying to work

01:49:22.240 | on more of these very idealized problems

01:49:24.600 | for things like natural language and common sense.

01:49:29.120 | So that's an interesting evolution of those ideas.

01:49:32.320 | - So perhaps the blocks world represents

01:49:34.640 | the fundamental challenges of the problem of intelligence

01:49:37.760 | more than people realize.

01:49:38.960 | - It might, yeah.

01:49:40.140 | - Is there, sort of, when you look back

01:49:43.520 | at your body of work and your life,

01:49:44.960 | you've worked in so many different fields,

01:49:47.040 | is there something that you're just really proud of

01:49:50.340 | in terms of ideas that you've gotten a chance

01:49:52.440 | to explore, create yourself?

01:49:54.340 | - So I am really proud of my work on the Copycat project.

01:49:59.680 | I think it's really different from what

01:50:01.600 | almost everyone has done in AI.

01:50:04.960 | I think there's a lot of ideas there to be explored.

01:50:08.960 | And I guess one of the happiest days of my life,

01:50:12.980 | aside from the births of my children,

01:50:17.460 | was the birth of Copycat,

01:50:20.040 | when it actually started to be able to make

01:50:22.720 | really interesting analogies.

01:50:25.520 | And I remember that very clearly.

01:50:27.920 | - That was a very exciting time.

01:50:30.180 | - Where you kind of gave life to an artificial system.

01:50:34.360 | - That's right.

01:50:35.200 | - What, in terms of what people can interact,

01:50:37.240 | I saw there's like a, I think it's called MetaCopycat,

01:50:40.520 | or is it-- - MetaCat.

01:50:41.640 | - MetaCat.

01:50:42.620 | And there's a Python 3 implementation.

01:50:45.600 | If people actually wanted to play around with it

01:50:47.440 | and actually get into it and study it

01:50:49.000 | and maybe integrate into, whether it's with deep learning

01:50:52.400 | or any other kind of work they're doing,

01:50:55.800 | what would you suggest they do to learn more about it

01:50:58.160 | and to take it forward in different kinds of directions?

01:51:01.360 | - Yeah, so there's Douglas Hofstadter's book

01:51:04.920 | called "Fluid Concepts and Creative Analogies"

01:51:07.640 | talks in great detail about Copycat.

01:51:10.120 | I have a book called "Analogy Making as Perception,"

01:51:13.840 | which is a version of my PhD thesis on it.

01:51:16.960 | There's also code that's available

01:51:19.240 | that you can get it to run.

01:51:21.200 | I have some links on my webpage

01:51:23.480 | to where people can get the code for it.

01:51:25.480 | And I think that would really be the best way

01:51:28.640 | to get into it. - Just dive in.

01:51:29.480 | - Dive in, yeah. - And play with it.

01:51:31.160 | Well, Melanie, it was an honor talking to you.

01:51:33.920 | I really enjoyed it.

01:51:34.760 | Thank you so much for your time today.

01:51:36.120 | - Thanks, it's been really great.

01:51:37.720 | - Thanks for listening to this conversation

01:51:40.400 | with Melanie Mitchell.

01:51:41.720 | And thank you to our presenting sponsor, Cash App.

01:51:45.000 | Download it, use code LEXPODCAST.

01:51:47.840 | You'll get $10, and $10 will go to FIRST,

01:51:50.800 | a STEM education nonprofit

01:51:52.560 | that inspires hundreds of thousands of young minds

01:51:55.200 | to learn and to dream of engineering our future.

01:51:58.880 | If you enjoy this podcast, subscribe on YouTube,

01:52:01.280 | give it five stars on Apple Podcast,

01:52:03.200 | support it on Patreon, or connect with me on Twitter.

01:52:06.580 | And now, let me leave you with some words of wisdom

01:52:09.440 | from Douglas Hufstadter and Melanie Mitchell.

01:52:12.520 | "Without concepts, there can be no thought.

01:52:15.320 | "And without analogies, there can be no concepts."

01:52:18.600 | And Melanie adds,

01:52:20.460 | "How to form and fluidly use concepts

01:52:23.560 | "is the most important open problem in AI."

01:52:26.900 | Thank you for listening, and hope to see you next time.

01:52:31.000 | (upbeat music)

01:52:33.580 | (upbeat music)

01:52:36.160 | [BLANK_AUDIO]

Melanie Mitchell: Concepts, Analogies, Common Sense & Future of AI | Lex Fridman Podcast #61

Chapters