Jeff Hawkins: Thousand Brains Theory of Intelligence

00:00:00.000 | The following is a conversation with Jeff Hawkins.

00:00:02.400 | He's the founder of the Redwood Center

00:00:04.160 | for Theoretical Neuroscience in 2002 and NuMenta in 2005.

00:00:09.000 | In his 2004 book titled "On Intelligence"

00:00:11.940 | and in the research "Before and After,"

00:00:13.880 | he and his team have worked to reverse engineer

00:00:16.200 | the neurocortex and propose artificial intelligence

00:00:19.160 | architectures, approaches, and ideas

00:00:21.360 | that are inspired by the human brain.

00:00:23.660 | These ideas include hierarchical temporal memory, HTM,

00:00:26.640 | from 2004, and new work, "The Thousands Brains Theory

00:00:30.040 | of Intelligence" from 2017, '18, and '19.

00:00:33.760 | Jeff's ideas have been an inspiration

00:00:36.080 | to many who have looked for progress

00:00:38.200 | beyond the current machine learning approaches,

00:00:40.440 | but they have also received criticism

00:00:42.720 | for lacking a body of empirical evidence

00:00:44.640 | supporting the models.

00:00:46.240 | This is always a challenge when seeking

00:00:48.080 | more than small incremental steps forward in AI.

00:00:51.440 | Jeff is a brilliant mind, and many of the ideas

00:00:54.120 | he has developed and aggregated from neuroscience

00:00:56.480 | are worth understanding and thinking about.

00:00:59.120 | There are limits to deep learning

00:01:00.920 | as it is currently defined.

00:01:02.960 | Forward progress in AI is shrouded in mystery.

00:01:05.760 | My hope is that conversations like this

00:01:07.760 | can help provide an inspiring spark for new ideas.

00:01:11.440 | This is the Artificial Intelligence Podcast.

00:01:14.040 | If you enjoy it, subscribe on YouTube, iTunes,

00:01:16.720 | or simply connect with me on Twitter

00:01:18.640 | at Lex Friedman, spelled F-R-I-D.

00:01:21.520 | And now, here's my conversation with Jeff Hawkins.

00:01:26.320 | Are you more interested in understanding the human brain

00:01:29.840 | or in creating artificial systems

00:01:32.000 | that have many of the same qualities

00:01:34.640 | but don't necessarily require that you

00:01:37.640 | actually understand the underpinning workings of our mind?

00:01:41.480 | - So, there's a clear answer to that question.

00:01:43.960 | My primary interest is understanding the human brain.

00:01:46.760 | No question about it.

00:01:47.680 | But, I also firmly believe that we will not be able

00:01:52.680 | to create fully intelligent machines

00:01:55.040 | until we understand how the human brain works.

00:01:57.280 | So, I don't see those as separate problems.

00:02:00.120 | I think there's limits to what can be done

00:02:01.720 | with machine intelligence if you don't understand

00:02:03.520 | the principles by which the brain works.

00:02:05.680 | And so, I actually believe that studying the brain

00:02:07.920 | is actually the fastest way to get to machine intelligence.

00:02:12.000 | - And within that, let me ask the impossible question.

00:02:14.640 | How do you, not define, but at least think about

00:02:17.520 | what it means to be intelligent?

00:02:19.400 | - So, I didn't try to answer that question first.

00:02:22.240 | We said, let's just talk about how the brain works

00:02:24.480 | and let's figure out how certain parts of the brain,

00:02:26.680 | mostly the neocortex, but some other parts too,

00:02:29.880 | the parts of the brain most associated with intelligence,

00:02:32.320 | and let's discover the principles by how they work.

00:02:35.800 | 'Cause intelligence isn't just like some mechanism

00:02:39.320 | and it's not just some capabilities.

00:02:40.640 | It's like, okay, we don't even know

00:02:42.520 | where to begin on this stuff.

00:02:44.040 | And so, now that we've made a lot of progress on this,

00:02:49.040 | after we've made a lot of progress

00:02:50.440 | on how the neocortex works and we can talk about that,

00:02:53.160 | I now have a very good idea what's gonna be required

00:02:55.840 | to make intelligent machines.

00:02:57.160 | I can tell you today, some of the things

00:02:59.560 | are gonna be necessary, I believe,

00:03:02.120 | to create intelligent machines.

00:03:03.440 | - Well, so we'll get there.

00:03:04.560 | We'll get to the neocortex and some of the theories

00:03:07.400 | of how the whole thing works.

00:03:09.160 | And you're saying, as we understand more and more

00:03:11.720 | about the neocortex, about our own human mind,

00:03:14.740 | we'll be able to start to more specifically define

00:03:17.680 | what it means to be intelligent.

00:03:18.640 | It's not useful to really talk about that until--

00:03:21.840 | - I don't know if it's not useful.

00:03:23.560 | Look, there's a long history of AI, as you know.

00:03:26.160 | And there's been different approaches taken to it.

00:03:28.880 | And who knows, maybe they're all useful.

00:03:31.280 | So, the good old-fashioned AI, the expert systems,

00:03:37.240 | the current convolutional neural networks,

00:03:38.920 | they all have their utility.

00:03:40.360 | They all have a value in the world.

00:03:43.760 | But I would think almost everyone would agree

00:03:45.200 | that none of them are really intelligent

00:03:46.600 | in a sort of a deep way that humans are.

00:03:49.880 | And so, it's just the question of how do you get

00:03:53.640 | from where those systems were or are today

00:03:56.440 | to where a lot of people think we're gonna go.

00:03:59.240 | And there's a big, big gap there, a huge gap.

00:04:02.320 | And I think the quickest way of bridging that gap

00:04:06.220 | is to figure out how the brain does that.

00:04:08.800 | And then we can sit back and look and say,

00:04:10.200 | oh, which of these principles that the brain works on

00:04:12.960 | are necessary and which ones are not?

00:04:15.160 | Clearly, we don't have to build this

00:04:16.600 | and intelligent machines aren't gonna be built

00:04:18.480 | out of organic living cells.

00:04:22.760 | But there's a lot of stuff that goes on in the brain

00:04:24.720 | that's gonna be necessary.

00:04:25.940 | - So, let me ask maybe, before we get into the fun details,

00:04:30.320 | let me ask maybe a depressing or difficult question.

00:04:33.280 | Do you think it's possible that we will never

00:04:36.240 | be able to understand how our brain works?

00:04:38.080 | That maybe there's aspects to the human mind,

00:04:41.880 | like we ourselves cannot introspectively get to the core,

00:04:46.160 | that there's a wall you eventually hit.

00:04:48.160 | - Yeah, I don't believe that's the case.

00:04:50.220 | I have never believed that's the case.

00:04:53.240 | There's not been a single thing humans

00:04:56.120 | have ever put their minds to.

00:04:57.360 | We've said, oh, we reached the wall,

00:04:58.640 | we can't go any further.

00:05:00.320 | People keep saying that.

00:05:01.680 | People used to believe that about life.

00:05:03.440 | Alain Vital, right?

00:05:04.480 | There's like, what's the difference between living matter

00:05:06.400 | and non-living matter?

00:05:07.280 | Something special that you never understand.

00:05:09.120 | We no longer think that.

00:05:10.640 | So, there's no historical evidence

00:05:13.640 | that suggests this is the case.

00:05:14.720 | And I just never even consider that's a possibility.

00:05:17.620 | I would also say, today, we understand so much

00:05:21.860 | about the neocortex.

00:05:22.820 | We've made tremendous progress in the last few years

00:05:25.480 | that I no longer think of it as an open question.

00:05:29.160 | The answers are very clear to me,

00:05:31.880 | and the pieces we don't know are clear to me,

00:05:34.800 | but the framework is all there,

00:05:36.120 | and it's like, oh, okay, we're gonna be able to do this.

00:05:38.600 | This is not a problem anymore.

00:05:39.960 | Just takes time and effort.

00:05:41.080 | But there's no mystery, big mystery anymore.

00:05:44.040 | - So, then let's get into it for people like myself

00:05:47.780 | who are not very well versed in the human brain,

00:05:52.780 | except my own.

00:05:53.820 | Can you describe to me, at the highest level,

00:05:57.300 | what are the different parts of the human brain,

00:05:59.120 | and then zooming in on the neocortex,

00:06:02.060 | the parts of the neocortex, and so on, a quick overview?

00:06:05.500 | - Yeah, sure.

00:06:06.620 | The human brain, we can divide it roughly into two parts.

00:06:10.800 | There's the old parts, lots of pieces,

00:06:14.200 | and then there's the new part.

00:06:15.680 | The new part is the neocortex.

00:06:18.040 | It's new because it didn't exist before mammals.

00:06:20.440 | The only mammals have a neocortex,

00:06:22.200 | and in humans, in primates, it's very large.

00:06:24.760 | In the human brain, the neocortex occupies

00:06:26.900 | about 70 to 75% of the volume of the brain.

00:06:30.640 | It's huge.

00:06:32.080 | And the old parts of the brain are,

00:06:34.840 | there's lots of pieces there.

00:06:36.000 | There's the spinal cord, and there's the brainstem,

00:06:38.720 | and the cerebellum, and the different parts

00:06:40.240 | of the basal ganglia, and so on.

00:06:42.020 | In the old parts of the brain,

00:06:42.960 | you have the autonomic regulation,

00:06:44.800 | like breathing and heart rate.

00:06:46.280 | You have basic behaviors, so like walking and running

00:06:49.460 | are controlled by the old parts of the brain.

00:06:51.380 | All the emotional centers of the brain

00:06:53.080 | are in the old part of the brain,

00:06:53.940 | so when you feel anger or hungry,

00:06:55.060 | lost, or things like that,

00:06:56.100 | those are all in the old parts of the brain.

00:06:57.940 | And we associate with the neocortex

00:07:02.180 | all the things we think about

00:07:03.340 | as sort of high-level perception and cognitive functions,

00:07:08.100 | anything from seeing and hearing and touching things

00:07:12.220 | to language to mathematics and engineering

00:07:15.140 | and science and so on.

00:07:16.940 | Those are all associated with the neocortex,

00:07:19.760 | and they're certainly correlated.

00:07:21.740 | Our abilities in those regards are correlated

00:07:23.980 | with the relative size of our neocortex

00:07:25.800 | compared to other mammals.

00:07:27.940 | So that's like the rough division,

00:07:30.500 | and you obviously can't understand the neocortex

00:07:33.420 | completely isolated, but you can understand a lot of it

00:07:37.020 | with just a few interfaces to the old parts of the brain.

00:07:40.340 | And so it gives you a system to study.

00:07:44.980 | The other remarkable thing about the neocortex

00:07:48.020 | compared to the old parts of the brain

00:07:49.880 | is the neocortex is extremely uniform.

00:07:52.880 | It's not visibly or anatomically,

00:07:55.740 | or it's very, it's like a,

00:07:58.780 | I always like to say it's like the size of a dinner napkin,

00:08:01.300 | about two and a half millimeters thick,

00:08:03.740 | and it looks remarkably the same everywhere.

00:08:05.980 | Everywhere you look in that two and a half millimeters

00:08:07.940 | is this detailed architecture,

00:08:10.100 | and it looks remarkably the same everywhere.

00:08:11.620 | And that's across species,

00:08:12.660 | a mouse versus a cat and a dog and a human.

00:08:15.420 | Where if you look at the old parts of the brain,

00:08:17.100 | there's lots of little pieces that do specific things.

00:08:19.660 | So it's like the old parts of our brain evolved,

00:08:22.100 | like this is the part that controls heart rate,

00:08:23.700 | and this is the part that controls this,

00:08:24.900 | and this is this kind of thing,

00:08:25.820 | and that's this kind of thing.

00:08:27.220 | And these evolved for eons, a long, long time,

00:08:30.100 | and they have their specific functions.

00:08:31.620 | And all of a sudden mammals come along,

00:08:33.260 | and they got this thing called the neocortex,

00:08:35.220 | and it got large by just replicating the same thing

00:08:38.180 | over and over and over again.

00:08:39.420 | This is like, wow, this is incredible.

00:08:42.660 | So all the evidence we have,

00:08:45.380 | and this is an idea that was first articulated

00:08:50.020 | in a very cogent and beautiful argument

00:08:52.020 | by a guy named Vernon Malkasal in 1978, I think it was,

00:08:55.660 | that the neocortex all works on the same principle.

00:09:01.580 | So language, hearing, touch, vision, engineering,

00:09:05.260 | all these things are basically underlying,

00:09:06.980 | are all built in the same computational substrate.

00:09:10.340 | They're really all the same problem.

00:09:11.820 | - So at the low level, the building blocks all look similar.

00:09:14.860 | - Yeah, and they're not even that low level.

00:09:16.300 | We're not talking about like neurons.

00:09:17.900 | We're talking about this very complex circuit

00:09:19.940 | that exists throughout the neocortex is remarkably similar.

00:09:23.500 | It's like, yes, you see variations of it here and there,

00:09:26.580 | more of the cell, less and less, and so on.

00:09:29.620 | But what Malkasal argued was, he says,

00:09:32.940 | if you take a section of neocortex,

00:09:35.580 | why is one a visual area and one is a auditory area?

00:09:38.620 | And his answer was, it's because one is connected to eyes

00:09:43.180 | and one is connected to ears.

00:09:45.420 | - Literally, you mean just it's most closest

00:09:47.820 | in terms of number of connections to the sensor.

00:09:50.420 | - Literally, if you took the optic nerve

00:09:52.900 | and attached it to a different part of the neocortex,

00:09:55.300 | that part would become a visual region.

00:09:57.940 | This experiment was actually done by Morganka Sur

00:10:01.420 | - Oh boy.

00:10:02.260 | - In developing, I think it was lemurs,

00:10:04.980 | I can't remember what it was, some animal.

00:10:06.700 | And there's a lot of evidence to this.

00:10:08.940 | If you take a blind person,

00:10:09.940 | a person who's born blind at birth,

00:10:12.180 | they're born with a visual neocortex.

00:10:15.420 | It may not get any input from the eyes

00:10:18.260 | because of some congenital defect or something.

00:10:21.220 | And that region does something else.

00:10:24.660 | It picks up another task.

00:10:26.980 | So it's this very complex thing.

00:10:31.980 | It's not like, oh, they're all built on neurons.

00:10:33.700 | No, they're all built in this very complex circuit

00:10:36.420 | and somehow that circuit underlies everything.

00:10:40.260 | And so this is the,

00:10:42.140 | it's called the common cortical algorithm, if you will.

00:10:45.860 | Some scientists just find it hard to believe

00:10:47.940 | and they just say, I can't believe that's true,

00:10:50.020 | but the evidence is overwhelming in this case.

00:10:52.060 | And so a large part of what it means

00:10:54.300 | to figure out how the brain creates intelligence

00:10:56.380 | and what is intelligence in the brain

00:10:59.860 | is to understand what that circuit does.

00:11:02.020 | If you can figure out what that circuit does,

00:11:05.020 | as amazing as it is, then you understand

00:11:08.140 | what all these other cognitive functions are.

00:11:10.500 | - So if you were to sort of put neocortex

00:11:13.300 | outside of your book on intelligence,

00:11:15.140 | you look, if you wrote a giant tome,

00:11:17.500 | a textbook on the neocortex,

00:11:19.780 | and you look maybe a couple centuries from now,

00:11:23.760 | how much of what we know now

00:11:25.480 | would still be accurate two centuries from now?

00:11:27.720 | So how close are we in terms of understanding?

00:11:30.840 | - I have to speak from my own particular experience here.

00:11:33.000 | So I run a small research lab here.

00:11:35.880 | It's like any other research lab.

00:11:38.040 | I'm sort of the principal investigator.

00:11:39.440 | There's actually two of us

00:11:40.280 | and there's a bunch of other people.

00:11:42.560 | And this is what we do.

00:11:43.840 | We study the neocortex and we publish our results and so on.

00:11:46.960 | So about three years ago,

00:11:48.520 | we had a real breakthrough in this field.

00:11:52.480 | Just tremendous breakthrough.

00:11:53.320 | We've now published, I think, three papers on it.

00:11:56.480 | And so I have a pretty good understanding

00:12:00.200 | of all the pieces and what we're missing.

00:12:02.320 | I would say that almost all the empirical data

00:12:06.240 | we've collected about the brain, which is enormous.

00:12:08.520 | If you don't know the neuroscience literature,

00:12:10.320 | it's just incredibly big.

00:12:13.960 | And it's, for the most part, all correct.

00:12:16.840 | It's facts and experimental results

00:12:20.240 | and measurements and all kinds of stuff.

00:12:22.960 | But none of that has been really assimilated

00:12:25.800 | into a theoretical framework.

00:12:27.840 | It's data without, in the language of Thomas Kuhn,

00:12:32.240 | the historian, it would be sort of a pre-paradigm science.

00:12:35.280 | Lots of data, but no way to fit it in together.

00:12:38.160 | I think almost all of that's correct.

00:12:39.520 | There's gonna be some mistakes in there.

00:12:42.160 | And for the most part,

00:12:43.240 | there aren't really good cogent theories

00:12:45.480 | about how to put it together.

00:12:47.240 | It's not like we have two or three competing good theories,

00:12:50.040 | which ones are right and which ones are wrong.

00:12:51.520 | It's like, nah, people just like scratching their heads,

00:12:53.720 | throwing things, you know.

00:12:54.760 | Some people have given up on trying to like figure out

00:12:56.480 | what the whole thing does.

00:12:57.600 | In fact, there's very, very few labs that we do

00:13:01.000 | that focus really on theory

00:13:03.320 | and all this unassimilated data and trying to explain it.

00:13:06.800 | So it's not like we've got it wrong.

00:13:08.920 | It's just that we haven't got it at all.

00:13:11.160 | - So it's really, I would say, pretty early days

00:13:15.080 | in terms of understanding the fundamental theories,

00:13:18.400 | forces of the way our mind works.

00:13:20.280 | - I don't think so.

00:13:21.120 | I would have said that's true five years ago.

00:13:23.800 | So as I said,

00:13:27.040 | we had some really big breakthroughs on this recently

00:13:29.360 | and we started publishing papers on this.

00:13:30.840 | So you could-- - So we'll get to that.

00:13:34.240 | - But so I don't think it's,

00:13:36.000 | I'm an optimist and from where I sit today,

00:13:38.280 | most people would disagree with this,

00:13:39.480 | but from where I sit today, from what I know,

00:13:41.720 | it's not super early days anymore.

00:13:45.080 | The way these things go is it's not a linear path, right?

00:13:48.240 | You don't just start accumulating

00:13:49.880 | and get better and better and better.

00:13:50.840 | No, you've got all this stuff you've collected.

00:13:52.960 | None of it makes sense.

00:13:53.800 | All these different things are just sort of around.

00:13:55.680 | And then you're gonna have some breaking points

00:13:57.160 | where all of a sudden, oh my God, now we got it right.

00:13:59.440 | That's how it goes in science.

00:14:01.160 | And I personally feel like we passed that little thing

00:14:04.520 | about a couple of years ago,

00:14:06.360 | all that big thing a couple of years ago.

00:14:07.640 | So we can talk about that.

00:14:09.680 | Time will tell if I'm right,

00:14:11.040 | but I feel very confident about it.

00:14:12.680 | That's why I'm willing to say it on tape like this.

00:14:15.280 | (laughing)

00:14:17.080 | - At least very optimistic.

00:14:18.080 | So let's, before those few years ago,

00:14:20.240 | let's take a step back to HTM,

00:14:23.240 | the hierarchical temporal memory theory,

00:14:26.000 | which you first proposed on intelligence

00:14:27.560 | and went through a few different generations.

00:14:29.320 | Can you describe what it is,

00:14:31.280 | how it evolved through the three generations

00:14:33.640 | since you first put it on paper?

00:14:35.440 | - Yeah, so one of the things that neuroscientists

00:14:39.320 | just sort of missed for many, many years,

00:14:43.000 | and especially people who are thinking about theory,

00:14:45.800 | was the nature of time in the brain.

00:14:49.120 | Brains process information through time.

00:14:51.680 | The information coming into the brain is constantly changing.

00:14:54.760 | The patterns from my speech right now,

00:14:57.600 | if you're listening to it at normal speed,

00:15:00.120 | would be changing on your ears

00:15:01.480 | about every 10 milliseconds or so, you'd have it change.

00:15:04.040 | This constant flow.

00:15:05.720 | When you look at the world,

00:15:06.720 | your eyes are moving constantly,

00:15:08.160 | three to five times a second,

00:15:09.680 | and the inputs completely.

00:15:11.360 | If I were to touch something like a coffee cup,

00:15:13.440 | as I move my fingers, the input changes.

00:15:15.160 | So this idea that the brain works on time-changing patterns

00:15:19.480 | is almost completely, or was almost completely missing

00:15:22.320 | from a lot of the basic theories,

00:15:23.600 | like fears of vision and so on.

00:15:25.000 | It's like, oh no, we're gonna put this image

00:15:26.840 | in front of you and flash it and say, what is it?

00:15:29.560 | Convolutional neural networks work that way today, right?

00:15:32.160 | Classify this picture.

00:15:34.240 | But that's not what vision is like.

00:15:35.960 | Vision is this sort of crazy time-based pattern

00:15:38.760 | that's going all over the place,

00:15:40.040 | and so is touch and so is hearing.

00:15:41.840 | So the first part of hierarchical temporal memory

00:15:43.760 | was the temporal part.

00:15:45.160 | It's to say, you won't understand the brain,

00:15:48.280 | nor will you understand intelligent machines

00:15:50.040 | unless you're dealing with time-based patterns.

00:15:52.480 | The second thing was, the memory component of it was,

00:15:55.440 | is to say that we aren't just processing input.

00:16:00.280 | We learn a model of the world.

00:16:02.800 | And the memory stands for that model.

00:16:04.640 | We have to, the point of the brain,

00:16:06.360 | the part of the neocortex, it learns a model of the world.

00:16:08.440 | We have to store things, our experiences,

00:16:11.520 | in a form that leads to a model of the world

00:16:14.200 | so we can move around the world,

00:16:15.640 | we can pick things up and do things,

00:16:16.880 | navigate and know how it's going on.

00:16:18.160 | So that's what the memory referred to.

00:16:19.920 | And many people just, they were thinking about

00:16:22.080 | like certain processes without memory at all.

00:16:25.120 | They're just like processing things.

00:16:26.720 | And then finally, the hierarchical component

00:16:28.960 | was a reflection to that the neocortex,

00:16:32.280 | although it's this uniform sheet of cells,

00:16:34.440 | different parts of it project to other parts,

00:16:37.600 | which project to other parts.

00:16:39.360 | And there is a sort of a rough hierarchy in terms of that.

00:16:43.080 | So the hierarchical temporal memory is just saying,

00:16:46.000 | look, we should be thinking about the brain

00:16:47.720 | as time-based, model memory-based

00:16:52.040 | and hierarchical processing.

00:16:54.000 | And that was a placeholder for a bunch of components

00:16:58.200 | that we would then plug into that.

00:17:00.840 | We still believe all those things I just said,

00:17:02.640 | but we now know so much more that I'm stopping to use

00:17:06.960 | the word hierarchical temporal memory yet

00:17:08.200 | because it's insufficient to capture the stuff we know.

00:17:11.320 | So again, it's not incorrect, but it's, I now know more

00:17:14.320 | and I would rather describe it more accurately.

00:17:16.840 | - Yeah, so you're basically, we can think of HTM

00:17:20.360 | as emphasizing that there's three aspects of intelligence

00:17:24.800 | that are important to think about

00:17:25.920 | whatever the eventual theory converges to.

00:17:28.880 | So in terms of time, how do you think of nature of time

00:17:32.480 | across different timescales?

00:17:33.840 | So you mentioned things changing,

00:17:36.800 | sensory inputs changing every 10, 20 minutes.

00:17:39.200 | What about every few minutes, every few months and years?

00:17:42.160 | - Well, if you think about a neuroscience problem,

00:17:44.880 | the brain problem, neurons themselves can stay active

00:17:49.680 | for certain periods of time.

00:17:51.600 | They can go to parts of the brain

00:17:52.840 | where they stay active for minutes.

00:17:54.280 | So you could hold a certain perception or an activity

00:17:58.080 | for a certain period of time,

00:18:01.360 | but most of them don't last that long.

00:18:04.520 | And so if you think about your thoughts

00:18:07.240 | or the activity neurons,

00:18:09.120 | if you're gonna wanna involve something

00:18:10.680 | that happened a long time ago,

00:18:12.000 | even just this morning, for example,

00:18:14.400 | the neurons haven't been active throughout that time.

00:18:16.400 | So you have to store that.

00:18:17.800 | So if I ask you, what did you have for breakfast today?

00:18:20.720 | That is memory.

00:18:22.000 | That is, you've built it into your model of the world now.

00:18:24.160 | You remember that.

00:18:25.000 | And that memory is in the synapses,

00:18:27.880 | is basically in the formation of synapses.

00:18:30.080 | And so you're sliding into what,

00:18:35.080 | you said different timescales.

00:18:36.720 | There's timescales of which we are like

00:18:38.640 | understanding my language and moving about

00:18:40.440 | and seeing things rapidly over time.

00:18:41.840 | That's the timescales of activities of neurons.

00:18:44.280 | But if you wanna get longer timescales,

00:18:46.200 | then it's more memory.

00:18:47.160 | And we have to invoke those memories to say,

00:18:49.560 | oh yes, well now I can remember what I had for breakfast

00:18:51.800 | because I stored that someplace.

00:18:54.240 | I may forget it tomorrow, but I'd store it for now.

00:18:58.200 | - So does memory also need to have,

00:19:01.560 | so the hierarchical aspect of reality

00:19:06.240 | is not just about concepts, it's also about time?

00:19:08.800 | Do you think of it that way?

00:19:10.320 | - Yeah, time is infused in everything.

00:19:12.880 | It's like you really can't separate it out.

00:19:15.560 | If I ask you, how does the brain learn a model

00:19:20.120 | of this coffee cup here?

00:19:21.400 | I have a coffee cup, and I'm at the coffee cup.

00:19:23.240 | I say, well time is not an inherent property

00:19:26.000 | of the model I have of this cup,

00:19:28.480 | whether it's a visual model or tactile model.

00:19:31.400 | I can sense it through time,

00:19:32.560 | but the model itself doesn't really have much time.

00:19:34.880 | If I asked you, if I said,

00:19:36.360 | well what is the model of my cell phone?

00:19:38.960 | My brain has learned a model of the cell phone.

00:19:40.760 | So if you have a smartphone like this.

00:19:43.320 | And I said, well this has time aspects to it.

00:19:45.640 | I have expectations when I turn it on,

00:19:48.000 | what's gonna happen, what or how long it's gonna take

00:19:50.440 | to do certain things.

00:19:51.920 | If I bring up an app, what sequences.

00:19:54.000 | And so I have, it's like melodies in the world.

00:19:57.240 | Melody has a sense of time.

00:19:58.520 | So many things in the world move and act,

00:20:01.200 | and there's a sense of time related to them.

00:20:03.720 | Some don't, but most things do actually.

00:20:08.240 | So it's sort of infused throughout the models of the world.

00:20:12.080 | You build a model of the world,

00:20:13.680 | you're learning the structure of the objects in the world,

00:20:16.400 | and you're also learning how those things

00:20:18.520 | change through time.

00:20:19.720 | - Okay, so it really is just the fourth dimension

00:20:23.880 | that's infused deeply.

00:20:25.240 | And you have to make sure that your models

00:20:27.440 | of intelligence incorporate it.

00:20:30.040 | So like you mentioned, the state of neuroscience

00:20:34.800 | is deeply empirical.

00:20:36.000 | A lot of data collection, that's where it is.

00:20:41.000 | You mentioned Thomas Kuhn, right?

00:20:43.080 | - Yeah.

00:20:44.520 | - And then you're proposing a theory of intelligence,

00:20:47.960 | and which is really the next step,

00:20:50.480 | the really important step to take.

00:20:52.880 | But why is HTM, or what we'll talk about soon,

00:20:57.880 | the right theory?

00:21:03.680 | So is it backed by intuition?

00:21:07.680 | Is it backed by evidence?

00:21:09.920 | Is it backed by a mixture of both?

00:21:11.960 | Is it kind of closer to where string theory is in physics,

00:21:15.560 | where there's mathematical components which show that,

00:21:19.720 | you know what, it seems that this,

00:21:22.960 | it fits together too well for it not to be true,

00:21:26.280 | which is where string theory is.

00:21:28.120 | Is that where you're kind of saying--

00:21:29.720 | - It's a mixture of all those things,

00:21:30.720 | although definitely where we are right now

00:21:32.760 | is definitely much more on the empirical side

00:21:34.640 | than let's say string theory.

00:21:36.040 | The way this goes about, we're theorists, right?

00:21:39.280 | So we look at all this data,

00:21:40.760 | and we're trying to come up with some sort of model

00:21:42.440 | that explains it, basically.

00:21:44.360 | And unlike string theory,

00:21:46.840 | there's vast more amounts of empirical data here

00:21:50.240 | that I think than most physicists deal with.

00:21:53.360 | And so our challenge is to sort through that

00:21:57.560 | and figure out what kind of constructs would explain this.

00:22:02.040 | And when we have an idea,

00:22:04.960 | you come up with a theory of some sort,

00:22:06.400 | you have lots of ways of testing it.

00:22:08.720 | First of all, there are 100 years

00:22:13.400 | of assimilated, unassimilated,

00:22:15.360 | empirical data from neuroscience.

00:22:16.600 | So we go back and read papers,

00:22:18.160 | and we say, "Oh, did someone find this already?"

00:22:20.680 | We can predict X, Y, and Z,

00:22:23.280 | and maybe no one's even talked about it

00:22:25.240 | since 1972 or something,

00:22:27.080 | but we go back and find that,

00:22:27.920 | and we say, "Oh, either it can support the theory,

00:22:31.080 | "or it can invalidate the theory."

00:22:33.400 | And then we say, "Okay, we have to start over again.

00:22:34.800 | "Oh, no, it's supported.

00:22:35.800 | "Let's keep going with that one."

00:22:38.120 | So the way I kind of view it,

00:22:40.480 | when we do our work, we come up,

00:22:43.280 | we look at all this empirical data,

00:22:45.440 | and it's what I call is a set of constraints.

00:22:47.680 | We're not interested in something

00:22:48.680 | that's biologically inspired.

00:22:49.880 | We're trying to figure out how the actual brain works.

00:22:52.120 | So every piece of empirical data

00:22:53.640 | is a constraint on a theory.

00:22:55.520 | In theory, if you have the correct theory,

00:22:57.000 | it needs to explain every pin, right?

00:22:59.400 | So we have this huge number of constraints on the problem,

00:23:03.160 | which initially makes it very, very difficult.

00:23:05.960 | If you don't have many constraints,

00:23:07.240 | you can make up stuff all the day.

00:23:08.480 | You can say, "Oh, here's an answer,

00:23:09.480 | "and how you can do this, you can do that, you can do this."

00:23:11.360 | But if you consider all biology as a set of constraints,

00:23:13.760 | all neuroscience is a set of constraints,

00:23:15.560 | and even if you're working on one little part

00:23:17.240 | of the neocortex, for example,

00:23:18.400 | there are hundreds and hundreds of constraints.

00:23:20.640 | These are empirical constraints

00:23:22.480 | that it's very, very difficult initially

00:23:24.840 | to come up with a theoretical framework for that.

00:23:27.280 | But when you do, and it solves all those constraints at once,

00:23:31.360 | you have a high confidence

00:23:33.000 | that you got something close to correct.

00:23:35.640 | It's just mathematically almost impossible not to be.

00:23:39.160 | So that's the curse and the advantage of what we have.

00:23:43.000 | The curse is we have to meet all these constraints,

00:23:47.600 | which is really hard.

00:23:48.960 | But when you do meet them,

00:23:50.920 | then you have a great confidence

00:23:53.240 | that you've discovered something.

00:23:54.920 | In addition, then we work with scientific labs.

00:23:58.040 | So we'll say, "Oh, there's something we can't find.

00:23:59.920 | "We can predict something,

00:24:01.160 | "but we can't find it anywhere in the literature."

00:24:04.160 | So we will then, we have people we've collaborated with,

00:24:06.920 | we'll say, sometimes they'll say,

00:24:08.740 | "You know what?

00:24:09.580 | "I have some collected data, which I didn't publish,

00:24:11.760 | "but we can go back and look at it

00:24:13.000 | "and see if we can find that."

00:24:14.800 | Which is much easier than designing a new experiment.

00:24:17.400 | Neuroscience experiments take a long time, years.

00:24:20.360 | So although some people are doing that now too.

00:24:23.160 | But between all of these things,

00:24:26.840 | I think it's a reasonable,

00:24:29.640 | it's actually a very, very good approach.

00:24:31.640 | We are blessed with the fact that we can test our theories

00:24:35.080 | out the yin-yang here

00:24:36.080 | because there's so much unassimilated data.

00:24:37.960 | And we can also falsify our theories very easily,

00:24:40.120 | which we do often.

00:24:41.520 | - It's kind of reminiscent to whenever

00:24:44.000 | that was with Copernicus.

00:24:45.480 | When you figure out that the sun's at the center

00:24:48.600 | of the solar system as opposed to Earth,

00:24:53.240 | the pieces just fall into place.

00:24:54.880 | - Yeah, I think that's the general nature of a hot moment.

00:24:58.980 | And it's Copernicus, it could be,

00:25:02.040 | you could say the same thing about Darwin.

00:25:04.160 | You could say the same thing about the double helix.

00:25:09.720 | That people have been working on a problem for so long

00:25:12.840 | and have all this data and they can't make sense of it,

00:25:14.600 | they can't make sense of it.

00:25:15.840 | But when the answer comes to you

00:25:17.440 | and everything falls into place,

00:25:19.400 | it's like, oh my gosh, that's it.

00:25:21.720 | That's got to be right.

00:25:23.120 | I asked both Jim Watson and Francis Crick about this.

00:25:28.080 | I asked them, when you were working on trying

00:25:31.960 | to discover the structure of the double helix,

00:25:34.480 | and when you came up with the sort of,

00:25:38.880 | the structure that ended up incorrect,

00:25:41.000 | but it was sort of a guess, it wasn't really verified yet.

00:25:45.680 | I said, did you know that it was right?

00:25:48.440 | And they both said, absolutely.

00:25:50.240 | So we absolutely knew it was right.

00:25:51.880 | And it doesn't matter if other people didn't believe it

00:25:54.760 | or not, we knew it was right.

00:25:55.640 | They'd get around to thinking it

00:25:56.680 | and agree with it eventually anyway.

00:25:59.080 | And that's the kind of thing you hear a lot with scientists

00:26:01.320 | who really are studying a difficult problem.

00:26:04.200 | And I feel that way too about our work.

00:26:07.120 | - Have you talked to Crick or Watson

00:26:08.480 | about the problem you're trying to solve,

00:26:11.840 | of finding the DNA of the brain?

00:26:15.920 | - Yeah, in fact, Francis Crick was very interested

00:26:19.080 | in this in the latter part of his life.

00:26:21.560 | And in fact, I got interested in brains

00:26:23.760 | by reading an essay he wrote in 1979

00:26:26.920 | called "Thinking About the Brain."

00:26:28.800 | And that was when I decided I'm going to leave

00:26:31.720 | my profession of computers and engineering

00:26:34.200 | and become a neuroscientist.

00:26:35.600 | I'm just reading that one essay from Francis Crick.

00:26:37.680 | I got to meet him later in life.

00:26:39.880 | I spoke at the Salk Institute and he was in the audience.

00:26:44.680 | And then I had a tea with him afterwards.

00:26:46.720 | He was interested in a different problem.

00:26:50.280 | He was focused on consciousness.

00:26:52.400 | - The easy problem, right?

00:26:54.280 | - Well, I think it's the red herring.

00:26:58.680 | And so we weren't really overlapping a lot there.

00:27:01.320 | Jim Watson, who's still alive,

00:27:05.400 | is also interested in this problem.

00:27:07.960 | When he was director of the Cold Spring Harbor Laboratories,

00:27:11.120 | he was really sort of behind moving

00:27:14.560 | in the direction of neuroscience there.

00:27:16.600 | And so he had a personal interest in this field.

00:27:19.320 | And I have met with him numerous times.

00:27:22.280 | And in fact, the last time was a little bit over a year ago.

00:27:27.680 | I gave a talk at Cold Spring Harbor Labs

00:27:30.360 | about the progress we were making in our work.

00:27:34.640 | And it was a lot of fun because he said,

00:27:39.640 | "Well, you wouldn't be coming here

00:27:41.080 | unless you had something important to say.

00:27:42.400 | So I'm gonna go attend your talk."

00:27:44.760 | So he sat in the very front row.

00:27:46.640 | Next to him was the director of the lab, Bruce Stillman.

00:27:50.160 | So these guys are in the front row of this auditorium.

00:27:52.560 | Nobody else in the auditorium wants to sit in the front row

00:27:54.600 | because there's Jim Watson and there's the director.

00:27:57.000 | And I gave a talk and I had dinner with Jim afterwards.

00:28:02.000 | But there's a great picture of my colleague,

00:28:06.040 | Subhathayam Hantuk, where I'm up there

00:28:08.560 | sort of explaining the basics of this new framework we have.

00:28:11.720 | And Jim Watson's on the edge of his chair.

00:28:13.800 | He's literally on the edge of his chair,

00:28:15.160 | like intently staring up at the screen.

00:28:17.800 | And when he discovered the structure of DNA,

00:28:21.760 | the first public talk he gave was at Cold Spring Harbor Labs.

00:28:25.920 | And there's a picture, there's a famous picture

00:28:27.480 | of Jim Watson standing at the whiteboard

00:28:29.320 | with an overhead thing pointing at something,

00:28:31.560 | pointing at the double helix at this point.

00:28:33.320 | And it actually looks a lot like the picture of me.

00:28:34.960 | So it's sort of funny.

00:28:36.080 | There's Eric up talking about the brain

00:28:37.440 | and there's Jim Watson staring intently at it.

00:28:39.280 | And of course, there was whatever, 60 years earlier,

00:28:41.640 | he was standing, pointing at the double helix.

00:28:44.280 | - It's one of the great discoveries in all of whatever,

00:28:47.720 | biology, science, all science is DNA.

00:28:49.720 | So it's funny that there's echoes of that

00:28:53.320 | in your presentation.

00:28:54.520 | Do you think in terms of evolutionary timeline in history,

00:28:58.360 | the development of the neocortex was a big leap

00:29:01.960 | or is it just a small step?

00:29:06.040 | So like if we ran the whole thing over again,

00:29:09.800 | from the birth of life on earth,

00:29:12.640 | how likely would we develop the mechanism of the neocortex?

00:29:15.280 | - Okay, well, those are two separate questions.

00:29:17.240 | One is, was it a big leap?

00:29:18.640 | And one was how likely it is, okay?

00:29:21.360 | They're not necessarily related.

00:29:22.880 | - Maybe correlated.

00:29:23.720 | - Maybe correlated, maybe not.

00:29:25.080 | And we don't really have enough data

00:29:26.080 | to make a judgment about that.

00:29:28.080 | I would say definitely it was a big leap.

00:29:29.960 | And I can tell you why I don't think

00:29:31.840 | it was just another incremental step.

00:29:34.040 | I'll get to that in a moment.

00:29:35.880 | I don't really have any idea how likely it is.

00:29:38.400 | If we look at evolution, we have one data point,

00:29:40.920 | which is earth, right?

00:29:42.520 | Life formed on earth billions of years ago,

00:29:45.180 | whether it was introduced here or it created here

00:29:48.080 | or someone introduced it, we don't really know,

00:29:49.520 | but it was here early.

00:29:51.160 | It took a long, long time to get to multicellular life.

00:29:55.120 | And then for multicellular life,

00:29:57.160 | it took a long, long time to get the neocortex.

00:30:02.280 | And we've only had the neocortex for a few hundred thousand

00:30:04.520 | years, so that's like nothing, okay?

00:30:07.960 | So is it likely?

00:30:09.560 | Well, it certainly isn't something that happened

00:30:11.120 | right away on earth.

00:30:13.520 | And there were multiple steps to get there.

00:30:15.160 | So I would say it's probably not something

00:30:17.160 | that would happen instantaneously on other planets

00:30:19.000 | that might have life.

00:30:20.560 | It might take several billion years on average.

00:30:23.120 | Is it likely?

00:30:24.320 | I don't know, but you'd have to survive

00:30:25.680 | for several billion years to find out, probably.

00:30:29.300 | Is it a big leap?

00:30:30.220 | Yeah, I think it is a qualitative difference

00:30:35.220 | than all other evolutionary steps.

00:30:37.800 | I can try to describe that if you'd like.

00:30:39.800 | - Sure, in which way?

00:30:41.960 | - Yeah, I can tell you how.

00:30:43.920 | Pretty much, let's start with a little preface.

00:30:47.700 | Many of the things that humans are able to do

00:30:50.460 | do not have obvious survival advantages precedent.

00:30:55.460 | We create music.

00:31:00.200 | Is there a really survival advantage to that?

00:31:02.640 | Maybe, maybe not.

00:31:03.880 | What about mathematics?

00:31:04.840 | Is there a real survival advantage to mathematics?

00:31:06.800 | (mimics music)

00:31:08.680 | You can stretch it.

00:31:09.520 | You can try to figure these things out, right?

00:31:12.200 | But most of evolutionary history,

00:31:14.760 | everything had immediate survival advantages to it.

00:31:18.660 | So I'll tell you a story, which I like, may or may not be true

00:31:23.120 | but the story goes as follows.

00:31:26.440 | Organisms have been evolving

00:31:30.800 | since the beginning of life here on Earth

00:31:32.800 | and adding this sort of complexity onto that

00:31:35.680 | and this sort of complexity onto that.

00:31:36.800 | And the brain itself is evolved this way.

00:31:39.640 | In fact, there's old parts and older parts

00:31:42.400 | and older, older parts of the brain

00:31:43.680 | that kind of just keeps calming on new things

00:31:45.440 | and we keep adding capabilities.

00:31:47.260 | And we got to the neocortex.

00:31:48.700 | Initially it had a very clear survival advantage

00:31:52.500 | in that it produced better vision and better hearing

00:31:55.060 | and better touch and maybe so on.

00:31:57.780 | But what I think happens is that evolution discovered,

00:32:01.100 | it took a mechanism and this is in our recent theory

00:32:05.060 | but it took a mechanism that evolved a long time ago

00:32:08.140 | for navigating in the world, for knowing where you are.

00:32:10.380 | These are the so-called grid cells and place cells

00:32:13.340 | of an old part of the brain.

00:32:15.140 | And it took that mechanism for building maps of the world

00:32:20.140 | and knowing where you are on those maps

00:32:22.600 | and how to navigate those maps

00:32:24.160 | and turns it into a sort of a slimmed down,

00:32:27.080 | idealized version of it.

00:32:28.380 | And that idealized version could now apply

00:32:31.600 | to building maps of other things,

00:32:32.800 | maps of coffee cups and maps of phones,

00:32:35.120 | maps of mathematics. - Concepts almost.

00:32:37.240 | - Concepts, yes, and not just almost, exactly.

00:32:40.280 | And so, and it just started replicating this stuff.

00:32:44.160 | You just think more and more and more

00:32:45.000 | and so we went from being sort of

00:32:47.440 | dedicated purpose neural hardware

00:32:50.080 | to solve certain problems that are important to survival

00:32:53.200 | to a general purpose neural hardware

00:32:55.820 | that could be applied to all problems.

00:32:58.120 | And now it's escaped the orbit of survival.

00:33:01.700 | We are now able to apply it to things

00:33:04.440 | which we find enjoyment,

00:33:06.760 | but aren't really clearly survival characteristics.

00:33:13.860 | And that it seems to only have happened in humans

00:33:16.700 | to the large extent.

00:33:18.200 | And so that's what's going on.

00:33:20.940 | Where we sort of have,

00:33:22.900 | we've sort of escaped the gravity of evolutionary pressure

00:33:26.320 | in some sense in the neocortex.

00:33:28.580 | And it now does things which are not,

00:33:31.540 | that are really interesting,

00:33:32.740 | discovering models of the universe

00:33:34.320 | which may not really help us, doesn't matter.

00:33:37.020 | How does it help us surviving,

00:33:38.560 | knowing that there might be multiverses

00:33:40.200 | or that there might be the age of the universe

00:33:42.900 | or how do various stellar things occur.

00:33:46.060 | It doesn't really help us survive at all.

00:33:47.780 | But we enjoy it and that's what happened.

00:33:50.420 | - Or at least not in the obvious way perhaps.

00:33:53.260 | It is required,

00:33:54.840 | if you look at the entire universe in an evolutionary way,

00:33:58.500 | it's required for us to do interplanetary travel

00:34:00.860 | and therefore survive, pass our own sun.

00:34:03.100 | But you know, let's not get too--

00:34:04.460 | - Yeah, but evolution works at one time frame.

00:34:07.300 | - Yes, yeah.

00:34:08.140 | - It's survival, if you think of survival of the phenotype,

00:34:11.300 | survival of the individual.

00:34:12.740 | - Exactly.

00:34:13.580 | - What you're talking about there

00:34:14.780 | is spans well beyond that.

00:34:16.300 | So there's no genetic,

00:34:18.660 | I'm not transferring any genetic traits to my children

00:34:23.380 | that are gonna help them survive better on Mars.

00:34:25.860 | - Right, totally different mechanism, that's right.

00:34:28.200 | So let's get into the new,

00:34:30.300 | as you've mentioned, this idea,

00:34:32.940 | I don't know if you have a nice name, Thousand--

00:34:35.580 | - We call it the Thousand Brain Theory of Intelligence.

00:34:37.340 | - I like it.

00:34:38.420 | Can you talk about this idea of spatial view of concepts

00:34:43.420 | and so on?

00:34:44.460 | - Yeah, so can I just describe sort of the,

00:34:46.500 | there's an underlying core discovery,

00:34:49.300 | which then everything comes from that.

00:34:51.140 | That's a very simple, this is really what happened.

00:34:54.580 | We were deep into problems about understanding

00:34:58.580 | how we build models of stuff in the world

00:35:00.540 | and how we make predictions about things.

00:35:03.020 | And I was holding a coffee cup just like this in my hand.

00:35:07.220 | And my finger was touching the side, my index finger,

00:35:10.540 | and then I moved it to the top

00:35:12.700 | and I was gonna feel the rim at the top of the cup.

00:35:15.460 | And I asked myself a very simple question.

00:35:18.300 | I said, well, first of all, let's say,

00:35:20.100 | I know that my brain predicts what it's gonna feel

00:35:22.260 | before it touches it.

00:35:23.320 | You can just think about it and imagine it.

00:35:26.060 | And so we know that the brain's making predictions

00:35:27.660 | all the time.

00:35:28.500 | So the question is, what does it take to predict that?

00:35:31.540 | And there's a very interesting answer.

00:35:33.620 | First of all, it says,

00:35:34.600 | the brain has to know it's touching a coffee cup,

00:35:36.500 | it has to have a model of a coffee cup,

00:35:37.980 | and needs to know where the finger currently is

00:35:41.020 | on the cup relative to the cup.

00:35:43.260 | Because when I make a movement,

00:35:44.420 | it needs to know where it's going to be on the cup

00:35:46.340 | after the movement is completed relative to the cup.

00:35:50.380 | And then it can make a prediction

00:35:51.900 | about what it's gonna sense.

00:35:53.340 | So this told me that the inner cortex,

00:35:54.960 | which is making this prediction,

00:35:56.380 | needs to know that it's sensing it's touching a cup,

00:35:59.380 | and it needs to know the location of my finger

00:36:01.420 | relative to that cup in a reference frame of the cup.

00:36:04.380 | It doesn't matter where the cup is relative to my body,

00:36:06.300 | it doesn't matter its orientation,

00:36:08.260 | none of that matters.

00:36:09.160 | It's where my finger is relative to the cup,

00:36:10.940 | which tells me then that the neocortex

00:36:13.540 | has a reference frame that's anchored to the cup.

00:36:17.340 | 'Cause otherwise I wouldn't be able to say the location,

00:36:19.260 | and I wouldn't be able to predict my new location.

00:36:21.500 | And then we quickly, very instantly,

00:36:23.740 | you can say, well, every part of my skin

00:36:25.420 | could touch this cup,

00:36:26.260 | and therefore every part of my skin is making predictions,

00:36:28.100 | and every part of my skin must have a reference frame

00:36:30.940 | that it's using to make predictions.

00:36:33.520 | So the big idea is that throughout the neocortex,

00:36:38.520 | there are, everything is being stored

00:36:44.480 | and referenced in reference frames.

00:36:46.720 | You can think of them like XYZ reference frames,

00:36:48.800 | but they're not like that.

00:36:50.360 | We know a lot about the neural mechanisms for this,

00:36:52.040 | but the brain thinks in reference frames.

00:36:54.860 | And as an engineer, if you're an engineer,

00:36:56.720 | this is not surprising.

00:36:57.720 | You'd say, if I wanted to build a CAD model

00:37:00.320 | of the coffee cup,

00:37:01.160 | well, I would bring it up in some CAD software,

00:37:03.240 | and I would assign some reference frame

00:37:04.760 | and say this features at this locations and so on.

00:37:06.960 | But the fact that this,

00:37:08.520 | the idea that this is occurring

00:37:09.680 | throughout the neocortex everywhere,

00:37:12.280 | it was a novel idea.

00:37:14.320 | And then a zillion things fell into place after that,

00:37:19.040 | a zillion.

00:37:19.900 | So now we think about the neocortex

00:37:21.840 | as processing information quite differently

00:37:23.400 | than we used to do it.

00:37:24.240 | We used to think about the neocortex

00:37:25.480 | as processing sensory data

00:37:27.680 | and extracting features from that sensory data,

00:37:29.800 | and then extracting features from the features,

00:37:31.800 | very much like a deep learning network does today.

00:37:34.880 | But that's not how the brain works at all.

00:37:36.600 | The brain works by assigning everything,

00:37:39.280 | every input, everything to reference frames.

00:37:41.840 | And there are thousands, hundreds of thousands of them

00:37:44.560 | active at once in your neocortex.

00:37:46.400 | It's a surprising thing to think about,

00:37:49.520 | but once you've sort of internalized this,

00:37:51.040 | you understand that it explains almost every,

00:37:53.800 | almost all the mysteries we've had about this structure.

00:37:57.720 | So one of the consequences of that

00:38:00.160 | is that every small part of the neocortex,

00:38:02.600 | say a millimeter square,

00:38:04.320 | and there's 150,000 of those.

00:38:06.320 | So it's about 150,000 square millimeters.

00:38:08.640 | If you take every little square millimeter of the cortex,

00:38:11.400 | it's got some input coming into it,

00:38:13.280 | and it's gonna have reference frames

00:38:14.920 | where it's assigning that input to,

00:38:16.800 | and each square millimeter

00:38:18.960 | can learn complete models of objects.

00:38:21.000 | So what do I mean by that?

00:38:22.000 | If I'm touching the coffee cup,

00:38:23.280 | well, if I just touch it in one place,

00:38:25.580 | I can't learn what this coffee cup is

00:38:27.180 | because I'm just feeling one part.

00:38:29.000 | But if I move it around the cup

00:38:31.080 | and touch it at different areas,

00:38:32.520 | I can build up a complete model of the cup

00:38:34.080 | because I'm now filling in that three-dimensional map,

00:38:36.680 | which is the coffee cup.

00:38:37.520 | I can say, oh, what am I feeling

00:38:38.640 | in all these different locations?

00:38:39.880 | That's the basic idea.

00:38:40.800 | It's more complicated than that.

00:38:43.000 | But so through time,

00:38:44.760 | and we talked about time earlier,

00:38:46.240 | through time, even a single column,

00:38:48.160 | which is only looking at,

00:38:49.040 | or a single part of the cortex

00:38:50.280 | which is only looking at a small part of the world

00:38:52.720 | can build up a complete model of an object.

00:38:55.080 | And so if you think about the part of the brain

00:38:57.120 | which is getting input from all my fingers,

00:38:59.080 | so they're spread across the top of your head here.

00:39:01.640 | This is the somatosensory cortex.

00:39:04.000 | There's columns associated

00:39:05.160 | of all the different areas of my skin.

00:39:07.360 | And what we believe is happening

00:39:10.040 | is that all of them are building models of this cup,

00:39:12.840 | every one of them, or things.

00:39:15.320 | They're not all building,

00:39:16.560 | not every column or every part of the cortex

00:39:18.160 | builds models of everything,

00:39:19.480 | but they're all building models of something.

00:39:21.680 | And so you have,

00:39:23.880 | so when I touch this cup with my hand,

00:39:26.860 | there are multiple models of the cup being invoked.

00:39:29.120 | If I look at it with my eyes,

00:39:30.600 | there are again many models of the cup being invoked

00:39:32.680 | 'cause each part of the visual system,

00:39:34.440 | the brain doesn't process an image.

00:39:35.960 | That's a misleading idea.

00:39:38.920 | It's just like your fingers touching the cup.

00:39:40.600 | So different parts of my retina

00:39:41.520 | are looking at different parts of the cup.

00:39:43.160 | And thousands and thousands of models of the cup

00:39:45.680 | are being invoked at once.

00:39:47.560 | And they're all voting with each other,

00:39:49.060 | trying to figure out what's going on.

00:39:50.280 | So that's why we call it

00:39:51.120 | the thousand brains theory of intelligence

00:39:52.420 | because there isn't one model of a cup.

00:39:54.880 | There are thousands of models of this cup.

00:39:56.440 | There are thousands of models of your cell phone

00:39:58.040 | and about cameras and microphones and so on.

00:40:00.960 | It's a distributed modeling system,

00:40:03.000 | which is very different

00:40:03.840 | than what people have thought about it.

00:40:04.960 | - So that's a really compelling and interesting idea.

00:40:07.440 | I have two first questions.

00:40:08.840 | So one on the ensemble part of everything coming together,

00:40:12.200 | you have these thousand brains.

00:40:13.760 | How do you know which one has done the best job

00:40:18.040 | of forming the--

00:40:18.880 | - Great question.

00:40:19.700 | Let me try to explain.

00:40:20.540 | There's a problem that's known in neuroscience

00:40:23.640 | called the sensor fusion problem.

00:40:25.320 | - Yes.

00:40:26.160 | - And so the idea is there's something like,

00:40:27.760 | oh, the image comes from the eye.

00:40:29.180 | There's a picture on the retina

00:40:30.680 | and it gets projected to the neocortex.

00:40:32.440 | Oh, by now it's all spread out all over the place

00:40:35.160 | and it's kind of squirrelly and distorted

00:40:37.160 | and pieces are all over the,

00:40:39.040 | it doesn't look like a picture anymore.

00:40:40.960 | When does it all come back together again?

00:40:43.680 | Or you might say, well, yes,

00:40:45.440 | but I also have sounds or touches associated with the cup.

00:40:48.660 | So I'm seeing the cup and touching the cup.

00:40:50.720 | How do they get combined together again?

00:40:52.680 | So this is called the sensor fusion problem.

00:40:54.280 | As if all these disparate parts have to be brought together

00:40:56.880 | into one model someplace.

00:40:59.040 | That's the wrong idea.

00:41:01.160 | The right idea is that you got all these guys voting.

00:41:03.540 | There's auditory models of the cup.

00:41:05.440 | There's visual models of the cup.

00:41:06.660 | There's tactile models of the cup.

00:41:08.360 | There are one, in the vision system,

00:41:10.720 | there might be ones that are more focused

00:41:11.920 | on black and white and ones focusing on color.

00:41:13.640 | It doesn't really matter.

00:41:14.480 | There's just thousands and thousands of models of this cup.

00:41:17.040 | And they vote.

00:41:17.920 | They don't actually come together in one spot.

00:41:20.640 | Just literally think of it this way.

00:41:21.920 | Imagine you have these columns are like about the size

00:41:25.160 | of a little piece of spaghetti.

00:41:26.280 | Okay, like two and a half millimeters tall

00:41:28.440 | and about a millimeter in width.

00:41:29.960 | They're not physical, but you can think of them that way.

00:41:33.280 | And each one's trying to guess

00:41:34.600 | what this thing is we're touching.

00:41:35.960 | Now, they can do a pretty good job

00:41:38.040 | if they're allowed to move over touch.

00:41:40.080 | I can reach my hand into a black box

00:41:41.560 | and move my finger around an object.

00:41:43.480 | And if I touch enough spaces, I go, okay,

00:41:45.520 | now I know what it is.

00:41:46.940 | But often we don't do that.

00:41:48.280 | Often I can just reach and grab something

00:41:49.520 | with my hand all at once and I get it.

00:41:51.000 | Or if I had to look through the world through a straw,

00:41:53.720 | so I'm only invoking one little column,

00:41:55.880 | I can only see part of something

00:41:56.720 | 'cause I have to move the straw around.

00:41:58.160 | But if I open my eyes, I see the whole thing at once.

00:42:00.480 | So what we think is going on is all these little pieces

00:42:02.800 | of spaghetti, if you will, all these little columns

00:42:04.280 | in the cortex are all trying to guess

00:42:06.420 | what it is that they're sensing.

00:42:08.600 | They'll do a better guess if they have time

00:42:10.740 | and can move over time.

00:42:11.700 | So if I move my eyes and move my fingers.

00:42:13.600 | But if they don't, they have a poor guess.

00:42:16.560 | It's a probabilistic guess of what they might be touching.

00:42:20.060 | Now imagine they can post their probability

00:42:22.920 | at the top of little pieces of spaghetti.

00:42:24.600 | Each one of them says, I think,

00:42:25.800 | and it's not really a probability distribution.

00:42:27.440 | It's more like a set of possibilities.

00:42:29.480 | In the brain, it doesn't work as a probability distribution.

00:42:32.000 | It works as more like what we call a union.

00:42:34.000 | You could say, and one column says,

00:42:35.880 | I think it could be a coffee cup,

00:42:37.880 | soda can, or a water bottle.

00:42:39.960 | And another column says, I think it could be a coffee cup

00:42:42.320 | or a telephone or a camera or whatever, right?

00:42:46.100 | And all these guys are saying what they think it might be.

00:42:49.960 | And there's these long range connections

00:42:51.640 | in certain layers in the cortex.

00:42:53.480 | So there's some layers in some cells types in each column

00:42:57.680 | send the projections across the brain.

00:43:00.080 | And that's the voting occurs.

00:43:01.760 | And so there's a simple associative memory mechanism.

00:43:04.120 | We've described this in a recent paper

00:43:06.160 | and we've modeled this.

00:43:07.360 | That says they can all quickly settle on the only

00:43:11.920 | or the one best answer for all of them.

00:43:14.920 | If there is a single best answer,

00:43:16.400 | they all vote and say, yep, it's gotta be the coffee cup.

00:43:18.960 | And at that point, they all know it's a coffee cup.

00:43:21.080 | And at that point, everyone acts as if it's a coffee cup.

00:43:23.400 | They're, yep, we know it's a coffee cup,

00:43:24.240 | even though I've only seen one little piece of this world,

00:43:26.400 | I know it's a coffee cup I'm touching

00:43:27.720 | or I'm seeing or whatever.

00:43:29.000 | And so you can think of all these columns

00:43:30.920 | are looking at different parts in different places,

00:43:33.060 | different sensory input, different locations.

00:43:35.240 | They're all different.

00:43:36.220 | But this layer that's doing the voting, it solidifies.

00:43:40.480 | It's just like it crystallizes and says,

00:43:42.280 | oh, we all know what we're doing.

00:43:44.160 | And so you don't bring these models together in one model,

00:43:46.480 | you just vote and there's a crystallization of the vote.

00:43:49.160 | - Great, that's at least a compelling way to think

00:43:52.280 | about the way you form a model of the world.

00:43:57.280 | Now, you talk about a coffee cup.

00:44:00.440 | Do you see this, as far as I understand,

00:44:03.240 | you were proposing this as well,

00:44:04.720 | that this extends to much more than coffee cups?

00:44:06.920 | - Yeah, it does.

00:44:09.560 | - Or at least the physical world,

00:44:10.840 | it expands to the world of concepts.

00:44:14.120 | - Yeah, it does.

00:44:15.040 | And well, the first, the primary face of evidence for that

00:44:18.240 | is that the regions of the neocortex

00:44:20.720 | that are associated with language or high level thought

00:44:23.160 | or mathematics or things like that,

00:44:24.500 | they look like the regions of the neocortex

00:44:26.200 | that process vision, hearing and touch.

00:44:28.320 | They don't look any different.

00:44:29.720 | Or they look only marginally different.

00:44:31.640 | And so one would say, well, if Vernon Mouncastle,

00:44:36.440 | who proposed that all the parts of the neocortex

00:44:38.880 | are doing the same thing, if he's right,

00:44:41.080 | then the parts that are doing language

00:44:42.840 | or mathematics or physics

00:44:44.600 | are working on the same principle.

00:44:45.760 | They must be working on the principle of reference frames.

00:44:48.600 | So that's a little odd thought.

00:44:50.200 | But of course, we had no prior idea how these things happen.

00:44:55.100 | So let's go with that.

00:44:56.520 | And we, in our recent paper,

00:44:59.960 | we talked a little bit about that.

00:45:01.720 | I've been working on it more since.

00:45:02.920 | I have better ideas about it now.

00:45:05.440 | I'm sitting here, I'm very confident

00:45:07.080 | that that's what's happening.

00:45:08.120 | And I can give you some examples

00:45:09.360 | to help you think about that.

00:45:11.320 | It's not we understand it completely,

00:45:12.600 | but I understand it better

00:45:13.640 | than I've described it in any paper so far.

00:45:16.440 | But we did put that idea out there.

00:45:17.800 | It says, okay, it's a good place to start.

00:45:22.720 | And the evidence would suggest it's how it's happening.

00:45:25.120 | And then we can start tackling that problem

00:45:26.760 | one piece at a time.

00:45:27.600 | Like, what does it mean to do high-level thought?

00:45:29.160 | What does it mean to do language?

00:45:30.120 | How would that fit into a reference frame framework?

00:45:34.320 | - Yeah, so there's a,

00:45:36.120 | I don't know if you could tell me if there's a connection,

00:45:37.720 | but there's an app called Anki

00:45:40.280 | that helps you remember different concepts.

00:45:42.520 | And they talk about like a memory palace

00:45:45.200 | that helps you remember completely random concepts

00:45:47.880 | by trying to put them in a physical space in your mind

00:45:51.480 | and putting them next to each other.

00:45:52.320 | - It's called the method of loci.

00:45:53.680 | - Loci, yeah.

00:45:54.760 | For some reason, that seems to work really well.

00:45:57.680 | Now, that's a very narrow kind of application

00:45:59.520 | of just remembering some facts.

00:46:00.680 | - But that's a very, very telling one.

00:46:03.160 | - Yes, exactly.

00:46:04.000 | So this seems like you're describing a mechanism

00:46:06.800 | why this seems to work.

00:46:09.680 | - So basically, the way, what we think is going on

00:46:11.880 | is all things you know, all concepts, all ideas,

00:46:15.120 | words, everything you know are stored in reference frames.

00:46:20.120 | And so if you wanna remember something,

00:46:24.320 | you have to basically navigate through a reference frame

00:46:26.880 | the same way a rat navigates to a maze

00:46:28.640 | and the same way my finger navigates to this coffee cup.

00:46:31.440 | We're moving through some space.

00:46:33.520 | And so if you have a random list of things

00:46:35.920 | you were asked to remember,

00:46:37.480 | by assigning them to a reference frame,

00:46:39.360 | you already know very well to see your house, right?

00:46:42.120 | And the idea of the method of loci is you can say,

00:46:43.640 | "Okay, in my lobby, I'm gonna put this thing.

00:46:45.840 | "And then in the bedroom, I put this one.

00:46:47.680 | "I go down the hall, I put this thing."

00:46:49.000 | And then you wanna recall those facts

00:46:50.920 | or recall those things.

00:46:51.760 | You just walk mentally, you walk through your house.

00:46:54.200 | You're mentally moving through a reference frame

00:46:56.640 | that you already had.

00:46:57.760 | And that tells you,

00:46:59.360 | there's two things that are really important about it.

00:47:00.680 | It tells us the brain prefers to store things

00:47:02.840 | in reference frames.

00:47:04.000 | And that the method of recalling things

00:47:06.920 | or thinking, if you will,

00:47:08.320 | is to move mentally through those reference frames.

00:47:11.600 | You could move physically through some reference frames.

00:47:13.600 | Like I could physically move through the reference frame

00:47:15.280 | of this coffee cup.

00:47:16.360 | I can also mentally move through the reference frame

00:47:18.000 | of the coffee cup, imagining me touching it.

00:47:20.040 | But I can also mentally move my house.

00:47:22.480 | And so now we can ask yourself,

00:47:24.720 | are all concepts stored this way?

00:47:26.800 | There was some recent research using human subjects

00:47:31.440 | in fMRI, and I'm gonna apologize for not knowing the name

00:47:34.000 | of the scientist who did this.

00:47:36.720 | But what they did is they put humans in this fMRI machine,

00:47:41.120 | which is one of these imaging machines.

00:47:42.840 | And they gave the humans tasks to think about birds.

00:47:46.480 | So they had different types of birds,

00:47:47.800 | and birds that look big and small,

00:47:49.680 | and long necks and long legs, things like that.

00:47:52.240 | And what they could tell from the fMRI,

00:47:54.200 | it was a very clever experiment,

00:47:57.600 | you get to tell when humans were thinking about the birds,

00:48:00.800 | that the knowledge of birds was arranged

00:48:04.440 | in a reference frame,

00:48:05.520 | similar to the ones that are used

00:48:07.120 | when you navigate in a room.

00:48:09.000 | These are called grid cells,

00:48:10.360 | and there are grid cell-like patterns of activity

00:48:12.840 | in the neocortex when they do this.

00:48:15.360 | So it's a very clever experiment.

00:48:18.960 | And what it basically says,

00:48:20.160 | that even when you're thinking about something abstract,

00:48:22.160 | and you're not really thinking about it

00:48:23.440 | as a reference frame, it tells us the brain

00:48:25.600 | is actually using a reference frame.

00:48:26.960 | And it's using the same neural mechanisms,

00:48:28.800 | these grid cells are the basic same neural mechanisms

00:48:30.800 | that we propose, the grid cells which exist

00:48:33.560 | in the old part of the brain, the entorhinal cortex,

00:48:36.120 | that that mechanism is now similar mechanism

00:48:38.880 | is used throughout the neocortex.

00:48:40.080 | It's the same nature to preserve this interesting way

00:48:43.160 | of creating reference frames.

00:48:44.560 | And so now they have empirical evidence

00:48:46.960 | that when you think about concepts like birds,

00:48:49.520 | that you're using reference frames

00:48:51.240 | that are built on grid cells.

00:48:53.160 | So that's similar to the method of loci,

00:48:55.160 | but in this case, the birds are related,

00:48:56.840 | so they create their own reference frame,

00:48:58.600 | which is consistent with bird space.

00:49:01.120 | And when you think about something, you go through that,

00:49:03.520 | you can make the same example, let's take a mathematics.

00:49:06.600 | Let's say you wanna prove a conjecture.

00:49:09.200 | What is a conjecture?

00:49:10.040 | Conjecture is a statement you believe to be true,

00:49:13.240 | but you haven't proven it.

00:49:15.080 | And so it might be an equation,

00:49:16.480 | I wanna show that this is equal to that.

00:49:19.200 | And you have some places you start with,

00:49:21.120 | you say, well, I know this is true, and I know this is true.

00:49:23.240 | And I think that maybe to get to the final proof,

00:49:25.880 | I need to go through some intermediate results.

00:49:28.640 | What I believe is happening is literally these equations

00:49:33.120 | or these points are assigned to a reference frame,

00:49:36.360 | a mathematical reference frame.

00:49:38.080 | And when you do mathematical operations,

00:49:39.840 | a simple one might be multiply or divide,

00:49:41.680 | but you might be able to plus transform or something else.

00:49:44.080 | That is like a movement in the reference frame of the math.

00:49:47.520 | And so you're literally trying to discover a path

00:49:50.280 | from one location to another location

00:49:52.680 | in a space of mathematics.

00:49:56.160 | And if you can get to these intermediate results,

00:49:58.240 | then you know your map is pretty good,

00:50:00.440 | and you know you're using the right operations.

00:50:02.960 | Much of what we think about is solving hard problems

00:50:05.960 | is designing the correct reference frame for that problem,

00:50:08.840 | figuring out how to organize the information

00:50:11.120 | and what behaviors I wanna use in that space

00:50:14.320 | to get me there.

00:50:15.240 | - Yeah, so if you dig in an idea of this reference frame,

00:50:19.320 | whether it's the math, you start a set of axioms

00:50:21.760 | to try to get to proving the conjecture.

00:50:24.320 | Can you try to describe, maybe take a step back,

00:50:28.160 | how you think of the reference frame in that context?

00:50:30.680 | Is it the reference frame that the axioms are happy in?

00:50:35.680 | Is it the reference frame that might contain everything?

00:50:38.760 | Is it a changing thing as you--

00:50:41.760 | - You have many, many reference frames.

00:50:43.320 | In fact, the way the theory,

00:50:44.560 | the thousand brain theory of intelligence says

00:50:46.120 | that every single thing in the world

00:50:47.360 | has its own reference frame.

00:50:48.280 | So every word has its own reference frames.

00:50:50.840 | And we can talk about this, the mathematics work out,

00:50:54.440 | this is no problem for neurons to do this.

00:50:55.960 | - But how many reference frames does a coffee cup have?

00:50:58.760 | Like it's on a table.

00:51:00.080 | - Let's say you ask how many reference frames

00:51:03.720 | could the column in my finger

00:51:06.040 | that's touching the coffee cup have?

00:51:07.440 | Because there are many, many models of the coffee cup.

00:51:10.480 | So there is no one model of a coffee cup.

00:51:13.040 | There are many models of a coffee cup.

00:51:14.240 | And you could say, well,

00:51:15.200 | how many different things can my finger learn?

00:51:17.280 | Is this the question you wanna ask?

00:51:19.560 | Imagine, I say every concept, every idea,

00:51:21.760 | everything you've ever know about that you can say,

00:51:23.840 | I know that thing has a reference frame associated with it.

00:51:28.240 | And what we do when we build composite objects,

00:51:30.160 | we assign reference frames to points

00:51:33.120 | and other reference frames.

00:51:33.960 | So my coffee cup has multiple components to it.

00:51:37.040 | It's got a rim, it's got a cylinder, it's got a handle.

00:51:39.800 | And those things have their own reference frames

00:51:42.800 | and they're assigned to a master reference frame,

00:51:45.040 | which is called this cup.

00:51:46.400 | And now I have this Numenta logo on it.

00:51:48.160 | Well, that's something that exists elsewhere in the world.

00:51:50.400 | It's its own thing.

00:51:51.240 | So it has its own reference frame.

00:51:52.280 | So we now have to say,

00:51:53.120 | well, how can I assign the Numenta logo reference frame

00:51:56.760 | onto the cylinder or onto the coffee cup?

00:51:59.240 | So it's all, we talked about this in the paper

00:52:01.520 | that came out in December of this last year.

00:52:05.780 | The idea of how you can assign reference frames

00:52:08.800 | to reference frames, how neurons could do this.

00:52:10.600 | - So, well, my question is,

00:52:12.680 | even though you mentioned reference frames a lot,

00:52:14.800 | I almost feel it's really useful to dig into

00:52:16.960 | how you think of what a reference frame is.

00:52:20.160 | I mean, it was already helpful for me to understand

00:52:22.080 | that you think of reference frames

00:52:23.720 | as something there is a lot of.

00:52:26.360 | - Okay, so let's just say that we're gonna have

00:52:28.800 | some neurons in the brain, not many actually,

00:52:31.080 | 10,000, 20,000 are gonna create

00:52:32.760 | a whole bunch of reference frames.

00:52:34.320 | What does it mean?

00:52:35.400 | Right, what is a reference frame in this case?

00:52:37.320 | First of all, these reference frames are different

00:52:40.080 | than the ones you might be used to.

00:52:42.120 | We know lots of reference frames.

00:52:43.440 | For example, we know the Cartesian coordinates, XYZ,

00:52:46.080 | that's a type of reference frame.

00:52:47.580 | We know longitude and latitude,

00:52:50.280 | that's a different type of reference frame.

00:52:52.800 | If I look at a printed map,

00:52:54.540 | you might have columns A through M

00:52:58.440 | and rows one through 20,

00:53:00.080 | that's a different type of reference frame.

00:53:01.400 | It's a kind of a Cartesian reference frame.

00:53:03.560 | The interesting thing about the reference frames

00:53:06.600 | in the brain, and we know this

00:53:07.880 | because these have been established through neuroscience

00:53:10.840 | studying the entorhinal cortex.

00:53:12.280 | So I'm not speculating here, okay?

00:53:13.560 | This is known neuroscience in an old part of the brain.

00:53:16.780 | The way these cells create reference frames,

00:53:18.840 | they have no origin.

00:53:20.720 | So what it's more like you have a point,

00:53:24.360 | a point in some space,

00:53:26.280 | and you, given a particular movement,

00:53:29.060 | you can then tell what the next point should be.

00:53:31.400 | And you can then tell what the next point would be,

00:53:34.080 | and so on.

00:53:35.460 | You can use this to calculate

00:53:38.700 | how to get from one point to another.

00:53:40.320 | So how do I get from my house to my home?

00:53:43.200 | Or how do I get my finger from the side of my cup

00:53:44.960 | to the top of the cup?

00:53:46.720 | How do I get from the axioms to the conjecture?

00:53:52.680 | So it's a different type of reference frame.

00:53:54.440 | And I can, if you want, I can describe in more detail,

00:53:57.360 | I can paint a picture

00:53:58.520 | how you might want to think about that.

00:53:59.760 | - It's really helpful to think.

00:54:00.600 | It's something you can move through.

00:54:02.120 | - Yeah.

00:54:02.960 | - But is it helpful to think of it as spatial in some sense?

00:54:07.960 | Or is there something that's more--

00:54:09.560 | - No, it's definitely spatial.

00:54:11.160 | It's spatial in a mathematical sense.

00:54:13.840 | - How many dimensions?

00:54:14.800 | Can it be a crazy number of dimensions?

00:54:16.280 | - Well, that's an interesting question.

00:54:17.480 | In the old part of the brain, the entorhinal cortex,

00:54:20.260 | they studied rats.

00:54:21.820 | And initially it looks like,

00:54:23.000 | oh, this is just two dimensional.

00:54:24.280 | It's like the rat is in some box in a maze or whatever,

00:54:27.320 | and they know where the rat is using

00:54:28.880 | these two dimensional reference frames

00:54:30.360 | to know where it is in the maze.

00:54:32.440 | We said, well, okay, but what about bats?

00:54:35.560 | That's a mammal and they fly in three dimensional space.

00:54:38.800 | How do they do that?

00:54:39.620 | They seem to know where they are, right?

00:54:41.760 | So this is a current area of active research.

00:54:44.360 | And it seems like somehow the neurons

00:54:46.440 | in the entorhinal cortex can learn three dimensional space.

00:54:50.320 | We just, two members of our team,

00:54:52.720 | along with Ilif Fet from MIT,

00:54:55.940 | just released a paper literally last week.

00:54:59.560 | It's on bioRxiv, where they show that you can,

00:55:03.600 | if you, the way these things work,

00:55:05.480 | and I won't get, unless you want to,

00:55:06.680 | I won't get into the detail,

00:55:08.120 | but grid cells can represent any n-dimensional space.

00:55:13.120 | It's not inherently limited.

00:55:15.340 | You can think of it this way.

00:55:16.640 | If you had two dimensional,

00:55:18.120 | the way it works is you add a bunch

00:55:19.240 | of two dimensional slices.

00:55:20.760 | That's the way these things work.

00:55:21.920 | There's a whole bunch of two dimensional models.

00:55:24.280 | And you can just, you can slice up any n-dimensional space

00:55:27.520 | with two dimensional projections.

00:55:29.320 | So, and you could have one dimensional model.

00:55:31.640 | So there's nothing inherent about the mathematics

00:55:34.400 | about the way the neurons do this,

00:55:35.800 | which constrain the dimensionality of the space,

00:55:39.480 | which I think was important.

00:55:41.460 | So obviously I have a three dimensional map of this cup.

00:55:44.080 | Maybe it's even more than that, I don't know.

00:55:46.360 | But it's clearly a three dimensional map of the cup.

00:55:48.360 | I don't just have a projection of the cup.

00:55:50.920 | But when I think about birds,

00:55:52.040 | or when I think about mathematics,

00:55:53.240 | perhaps it's more than three dimensions.

00:55:55.280 | Who knows?

00:55:56.300 | - So in terms of each individual column

00:56:00.120 | building up more and more information over time,

00:56:04.060 | do you think that mechanism is well understood?

00:56:06.440 | In your mind, you've proposed a lot of architectures there.

00:56:09.880 | Is that a key piece, or is it,

00:56:11.860 | is the big piece, the thousand brain theory of intelligence,

00:56:16.280 | the ensemble of it all?

00:56:17.520 | - Well, I think they're both big.

00:56:18.480 | I mean, clearly the concept, as a theorist,

00:56:20.960 | the concept is most exciting, right?

00:56:23.120 | We want-- - A high level concept.

00:56:23.960 | - A high level concept.

00:56:24.800 | This is a totally new way of thinking about

00:56:26.360 | how the neural characteristics work.

00:56:27.240 | So that is appealing.

00:56:28.680 | It has all these ramifications.

00:56:30.720 | And with that as a framework for how the brain works,

00:56:33.800 | you can make all kinds of predictions

00:56:35.040 | and solve all kinds of problems.

00:56:36.280 | Now we're trying to work through

00:56:37.300 | many of these details right now.

00:56:38.480 | Okay, how do the neurons actually do this?

00:56:40.560 | Well, it turns out, if you think about grid cells

00:56:42.540 | and place cells in the old parts of the brain,

00:56:44.760 | there's a lot that's known about them,

00:56:46.000 | but there's still some mysteries.

00:56:47.060 | There's a lot of debate about exactly the details,

00:56:49.080 | how these work and what are the signs.

00:56:50.720 | And we have that still, that same level of detail,

00:56:52.800 | that same level of concern.

00:56:54.160 | What we spend here most of our time doing

00:56:56.820 | is trying to make a very good list

00:57:00.080 | of the things we don't understand yet.

00:57:02.640 | That's the key part here.

00:57:04.020 | What are the constraints?

00:57:05.240 | It's not like, oh, this thing seems to work, we're done.

00:57:07.040 | No, it's like, okay, it kind of works,

00:57:08.800 | but these are other things we know it has to do

00:57:10.680 | and it's not doing those yet.

00:57:12.840 | I would say we're well on the way here.

00:57:15.000 | We're not done yet.

00:57:17.060 | There's a lot of trickiness to this system,

00:57:20.000 | but the basic principles about how different layers

00:57:23.140 | in the neocortex are doing much of this, we understand,

00:57:26.440 | but there's some fundamental parts

00:57:28.580 | that we don't understand as well.

00:57:29.980 | - So what would you say is one of the harder open problems

00:57:34.060 | or one of the ones that have been bothering you,

00:57:37.180 | keeping you up at night the most?

00:57:38.420 | - Oh, well, right now, this is a detailed thing

00:57:40.660 | that wouldn't apply to most people, okay?

00:57:42.220 | (laughing)

00:57:43.060 | - Sure.

00:57:43.880 | - But you want me to answer that question?

00:57:44.720 | - Yeah, please.

00:57:46.200 | We've talked about as if, oh, to predict what you're going

00:57:49.620 | to sense on this coffee cup, I need to know

00:57:51.500 | where my finger is gonna be on the coffee cup.

00:57:53.620 | That is true, but it's insufficient.

00:57:55.420 | Think about my finger touches the edge of the coffee cup.

00:57:58.500 | My finger can touch it at different orientations.

00:58:01.680 | I can rotate my finger around here,

00:58:03.580 | and that doesn't change.

00:58:06.340 | I can make that prediction and somehow,

00:58:08.780 | so it's not just the location,

00:58:10.100 | there's an orientation component of this as well.

00:58:13.300 | This is known in the old parts of the brain too,

00:58:15.140 | there's things called head direction cells,

00:58:16.600 | which way the rat is facing.

00:58:18.040 | It's the same kind of basic idea.

00:58:20.440 | So if my finger were a rat, in three dimensions,

00:58:23.620 | I have a three-dimensional orientation,

00:58:25.740 | and I have a three-dimensional location.

00:58:27.220 | If I was a rat, I would have a,

00:58:28.620 | you might think of it as a two-dimensional location,

00:58:30.620 | a two-dimensional, a one-dimensional orientation,

00:58:32.540 | like just which way is it facing?

00:58:35.100 | So how the two components work together,

00:58:38.260 | how does it, I combine orientation,

00:58:41.500 | the orientation of my sensor,

00:58:43.940 | as well as the location, is a tricky problem.

00:58:48.940 | And I think I've made progress on it.

00:58:52.740 | - So at a bigger version of that,

00:58:55.140 | so perspective's super interesting, but super specific.

00:58:58.460 | - Yeah, I warned you.

00:58:59.980 | - No, no, no, it's really good,

00:59:01.260 | but there's a more general version of that.

00:59:03.760 | Do you think context matters,

00:59:06.940 | the fact that we are in a building in North America,

00:59:12.620 | that we, in the day and age where we have mugs,

00:59:15.940 | I mean, there's all this extra information

00:59:19.180 | that you bring to the table about

00:59:21.460 | everything else in the room

00:59:22.540 | that's outside of just the coffee cup.

00:59:24.260 | - Of course it is. - How does it get--

00:59:25.460 | - Yeah, so-- - Connected, do you think?

00:59:27.340 | - Yeah, and that is another really interesting question.

00:59:30.320 | I'm gonna throw that under the rubric,

00:59:32.120 | or the name of attentional problems.

00:59:35.060 | First of all, we have this model,

00:59:36.140 | I have many, many models.

00:59:37.980 | - And also the question, does it matter, because--

00:59:40.100 | - Well, it matters for certain things,

00:59:41.140 | and of course it does.

00:59:42.620 | Maybe what we think of as a coffee cup

00:59:44.980 | in another part of the world

00:59:45.900 | is viewed as something completely different.

00:59:47.660 | Or maybe our logo, which is very benign

00:59:50.420 | in this part of the world,

00:59:51.340 | it means something very different

00:59:52.540 | in another part of the world.

00:59:53.780 | So those things do matter.

00:59:56.500 | I think the way to think about it is the following,

01:00:00.380 | one way to think about it,

01:00:01.740 | is we have all these models of the world, okay?

01:00:04.780 | And we model everything.

01:00:06.180 | And as I said earlier, I kind of snuck it in there,

01:00:08.900 | our models are actually, we build composite structure.

01:00:12.500 | So every object is composed of other objects,

01:00:15.260 | which are composed of other objects,

01:00:16.420 | and they become members of other objects.

01:00:18.700 | So this room has chairs, and a table,

01:00:20.300 | and a room, and walls, and so on.

01:00:21.580 | Now we can just arrange these things in a certain way,

01:00:24.300 | and go, oh, that's in the Nementa Conference Room.

01:00:26.600 | And what we do is when we go around the world,

01:00:31.940 | and we experience the world,

01:00:34.340 | by walking to a room, for example,

01:00:36.420 | the first thing I do is say, oh, I'm in this room.

01:00:38.340 | Do I recognize the room?

01:00:39.340 | Then I can say, oh, look, there's a table here.

01:00:42.540 | And by attending to the table,

01:00:44.120 | I'm then assigning this table in a context of the room.

01:00:46.260 | Then I say, oh, on the table, there's a coffee cup.

01:00:48.740 | Oh, and on the table, there's a logo.

01:00:50.400 | And in the logo, there's the word Nementa.

01:00:51.860 | Oh, and look, in the logo, there's the letter E.

01:00:54.020 | Oh, and look, it has an unusual serif.

01:00:56.360 | And it doesn't actually, but pretend it does.

01:00:58.620 | So the point is, your attention is kind of drilling

01:01:03.860 | deep in and out of these nested structures.

01:01:07.460 | And I can pop back up, and I can pop back down.

01:01:09.340 | I can pop back up, and I can pop back down.

01:01:10.900 | So when I attend to the coffee cup,

01:01:13.220 | I haven't lost the context of everything else,

01:01:15.660 | but it's sort of, there's this sort of nested structure.

01:01:18.900 | - So the attention filters the reference frame formation

01:01:22.120 | for that particular period of time?

01:01:24.380 | - Yes, it basically, moment to moment,

01:01:26.580 | you attend the subcomponents,

01:01:28.380 | and then you can attend the subcomponents

01:01:29.700 | to subcomponents.

01:01:30.540 | - And you can move up and down.

01:01:31.380 | - You can move up and down.

01:01:32.300 | We do that all the time.

01:01:33.140 | You're not even, now that I'm aware of it,

01:01:35.540 | I'm very conscious of it.

01:01:36.660 | But until, but most people don't even think about this.

01:01:39.940 | You just walk in a room, and you don't say,

01:01:41.660 | oh, I looked at the chair, and I looked at the board,

01:01:43.460 | and I looked at that word on the board,

01:01:44.580 | and I looked over here, what's going on?

01:01:46.460 | Right?

01:01:47.300 | - So what percent of your day are you deeply aware of this,

01:01:49.980 | and what part can you actually relax and just be Jeff?

01:01:52.820 | - Me personally, like my personal day?

01:01:54.340 | - Yeah.

01:01:55.180 | - Unfortunately, I'm afflicted with too much of the former.

01:01:57.980 | (laughing)

01:02:00.740 | I think--

01:02:01.580 | - Well, unfortunately or unfortunately, okay.

01:02:02.780 | - Yeah, so I think--

01:02:03.620 | - You don't think it's useful?

01:02:04.540 | - Oh, it is useful, totally useful.

01:02:06.780 | I think about this stuff almost all the time.

01:02:09.180 | And one of my primary ways of thinking

01:02:12.540 | is when I'm asleep at night,

01:02:13.860 | I always wake up in the middle of the night.

01:02:15.820 | And then I stay awake for at least an hour with my eyes shut

01:02:19.340 | in sort of a half-sleep state thinking about these things.

01:02:21.620 | I come up with answers to problems very often

01:02:23.700 | in that sort of half-sleeping state.

01:02:25.620 | I think about it on my bike ride.

01:02:26.700 | I think about it on walks.

01:02:27.540 | I'm just constantly thinking about this.

01:02:28.780 | I have to almost schedule time

01:02:32.420 | to not think about this stuff,

01:02:34.100 | 'cause it's very, it's mentally taxing.

01:02:37.820 | - Are you, when you're thinking about this stuff,

01:02:39.780 | are you thinking introspectively,

01:02:41.180 | like almost taking a step outside of yourself

01:02:43.700 | and trying to figure out what is your mind doing right now?

01:02:45.700 | - I do that all the time, but that's not all I do.

01:02:48.200 | I'm constantly observing myself.

01:02:50.780 | So as soon as I started thinking about grid cells,

01:02:53.060 | for example, and getting into that,

01:02:55.260 | I started saying, oh, well, grid cells

01:02:56.780 | can have my place of sense in the world.

01:02:58.140 | You know, that's where you know where you are.

01:02:59.660 | And it's interesting, you know,

01:03:00.700 | we always have a sense of where we are, unless we're lost.

01:03:03.020 | And so I started at night,

01:03:04.220 | when I got up to go to the bathroom,

01:03:05.940 | I would start trying to do it completely

01:03:07.260 | with my eyes closed all the time.

01:03:08.500 | And I would test my sense of grid cells.

01:03:10.020 | I would walk, you know, five feet and say,

01:03:12.700 | okay, I think I'm here.

01:03:13.700 | Am I really there?

01:03:14.540 | What's my error?

01:03:15.460 | And then I would calculate my error again

01:03:16.780 | and see how the errors could accumulate.

01:03:17.940 | So even something as simple as getting up

01:03:19.420 | in the middle of the night to go to the bathroom,

01:03:20.420 | I'm testing these theories out.

01:03:22.660 | It's kind of fun.

01:03:23.500 | I mean, the coffee cup is an example of that too.

01:03:25.620 | So I think, I find that these sort of everyday

01:03:29.500 | introspections are actually quite helpful.

01:03:32.860 | It doesn't mean you can ignore the science.

01:03:34.860 | I mean, I spend hours every day

01:03:37.060 | reading ridiculously complex papers.

01:03:39.140 | That's not nearly as much fun,

01:03:41.740 | but you have to sort of build up those constraints

01:03:44.140 | and the knowledge about the field and who's doing what

01:03:46.860 | and what exactly they think is happening here.

01:03:48.860 | And then you can sit back and say,

01:03:50.060 | okay, let's try to piece this all together.

01:03:52.460 | Let's come up with some, you know,

01:03:55.140 | I'm very, in this group, hearing people,

01:03:57.700 | they know I do this all the time.

01:03:59.260 | I come in with these introspective ideas and say,

01:04:01.180 | well, have you ever thought about this?

01:04:02.340 | Now watch, well, let's all do this together.

01:04:04.660 | And it's helpful.

01:04:05.940 | It's not, as long as you don't,

01:04:09.580 | all you did was that,

01:04:10.420 | then you're just making up stuff, right?

01:04:12.300 | But if you're constraining it by the reality

01:04:14.780 | of the neuroscience, then it's really helpful.

01:04:17.780 | - So let's talk a little bit about deep learning

01:04:20.180 | and the successes in the applied space of neural networks.

01:04:25.180 | Ideas of training model on data

01:04:29.020 | and these simple computational units,

01:04:31.380 | neuron, artificial neurons that,

01:04:34.540 | with back propagation, statistical ways

01:04:37.460 | of being able to generalize from the training set

01:04:41.700 | onto data that's similar to that training set.

01:04:44.300 | So where do you think are the limitations

01:04:47.420 | of those approaches?

01:04:48.460 | What do you think are its strengths

01:04:50.380 | relative to your major efforts

01:04:52.180 | of constructing a theory of human intelligence?

01:04:55.140 | - Yeah, well, I'm not an expert in this field.

01:04:57.840 | I'm somewhat knowledgeable, so I, but I'm not--

01:04:59.820 | - A little bit is in just your intuition.

01:05:01.700 | What are your--

01:05:02.540 | - Well, I have a little bit more than intuition,

01:05:03.900 | but I just wanna say like, you know,

01:05:06.660 | one of the things that you asked me,

01:05:07.740 | do I spend all my time thinking about neuroscience?

01:05:09.260 | I do, that's to the exclusion of thinking about things

01:05:11.380 | like convolutional neural networks.

01:05:13.700 | But I try to stay current.

01:05:15.300 | So look, I think it's great, the progress they've made.

01:05:17.920 | It's fantastic.

01:05:18.820 | And as I mentioned earlier,

01:05:19.900 | it's very highly useful for many things.

01:05:21.860 | The models that we have today are actually derived

01:05:26.220 | from a lot of neuroscience principles.

01:05:28.300 | They are distributed processing systems

01:05:30.060 | and distributed memory systems,

01:05:31.300 | and that's how the brain works.

01:05:33.300 | They use things that we might call them neurons,

01:05:35.940 | but they're really not neurons at all.

01:05:37.060 | So we can just, they're not really neurons.

01:05:39.260 | So they're distributed processing systems.

01:05:41.360 | And that nature of hierarchy that came also

01:05:45.500 | from neuroscience, and so there's a lot of things,

01:05:48.060 | the learning rules, basically, not back prop,

01:05:50.520 | but other, you know, sort of heavy entitlement.

01:05:52.140 | - I'd be curious to say they're not neurons at all.

01:05:55.100 | Can you describe in which way?

01:05:56.220 | I mean, some of it is obvious,

01:05:57.760 | but I'd be curious if you have specific ways

01:06:00.380 | in which you think are the biggest differences.

01:06:02.860 | - Yeah, we had a paper in 2016 called

01:06:04.980 | Why Neurons Have Thousands of Synapses.

01:06:06.980 | And if you read that paper,

01:06:09.740 | you'll know what I'm talking about here.

01:06:11.420 | A real neuron in the brain is a complex thing.

01:06:14.060 | Let's just start with the synapses on it,

01:06:17.220 | which is a connection between neurons.

01:06:19.060 | Real neurons can have everywhere

01:06:20.740 | from five to 30,000 synapses on them.

01:06:23.500 | The ones near the cell body,

01:06:27.240 | the ones that are close to the soma, the cell body,

01:06:30.460 | those are like the ones that people model

01:06:32.140 | in artificial neurons.

01:06:33.780 | There's a few hundred of those,

01:06:35.100 | maybe they can affect the cell,

01:06:37.140 | they can make the cell become active.

01:06:39.780 | 95% of the synapses can't do that.

01:06:43.580 | They're too far away.

01:06:44.660 | So if you activate one of those synapses,

01:06:46.060 | it just doesn't affect the cell body enough

01:06:47.920 | to make any difference.

01:06:48.940 | - Any one of them individually.

01:06:50.140 | - Any one of them individually,

01:06:50.980 | or even if you do a mass of them.

01:06:55.080 | What real neurons do is the following.

01:06:57.440 | If you activate, or you get 10 to 20 of them

01:07:02.440 | active at the same time,

01:07:04.480 | meaning they're all receiving an input at the same time,

01:07:06.680 | and those 10 to 20 synapses or 40 synapses

01:07:09.080 | within a very short distance on the dendrite,

01:07:11.280 | like 40 microns, a very small area.

01:07:13.280 | So if you activate a bunch of these

01:07:14.560 | right next to each other at some distant place,

01:07:17.560 | what happens is it creates what's called

01:07:19.640 | the dendritic spike.

01:07:21.260 | And dendritic spike travels through the dendrites

01:07:24.500 | and can reach the soma or the cell body.

01:07:26.860 | Now, when it gets there, it changes the voltage,

01:07:31.220 | which is sort of like gonna make the cell fire,

01:07:33.540 | but never enough to make the cell fire.

01:07:36.020 | It's sort of what we call it says we depolarize the cell,

01:07:38.440 | you raise the voltage a little bit,

01:07:39.540 | but not enough to do anything.

01:07:41.580 | It's like, well, good as that.

01:07:42.540 | And then it goes back down again.

01:07:44.420 | So we proposed a theory,

01:07:47.740 | which I'm very confident in basics are,

01:07:50.460 | is that what's happening there is those 95%

01:07:53.680 | of those synapses are recognizing dozens

01:07:56.320 | to hundreds of unique patterns.

01:07:58.420 | They can write about 10, 20 synapses at a time,

01:08:02.040 | and they're acting like predictions.

01:08:04.420 | So the neuron actually is a predictive engine on its own.

01:08:07.600 | It can fire when it gets enough,

01:08:09.660 | what they call proximal input

01:08:10.840 | from those ones near the cell fire,

01:08:11.920 | but it can get ready to fire from dozens

01:08:14.920 | to hundreds of patterns that are recognized

01:08:16.520 | as from the other guys.

01:08:18.080 | And the advantage of this to the neuron

01:08:21.220 | is that when it actually does produce a spike

01:08:23.460 | and action potential,

01:08:24.740 | it does so slightly sooner than it would have otherwise.

01:08:27.660 | And so what good is slightly sooner?

01:08:29.700 | Well, the slightly sooner part is it,

01:08:31.780 | there's all the neurons in the excitatory neurons

01:08:34.040 | in the brain are surrounded by these inhibitory neurons.

01:08:36.660 | And they're very fast, the inhibitory neurons,

01:08:38.940 | these baskets all.

01:08:40.440 | And if I get my spike out a little bit sooner

01:08:43.300 | than someone else, I inhibit all my neighbors around me.

01:08:46.540 | Right?

01:08:47.380 | So what you end up with is a different representation.

01:08:49.740 | You end up with a representation

01:08:50.880 | that matches your prediction.

01:08:52.080 | It's a sparser representation,

01:08:53.800 | meaning there's fewer neurons are active,

01:08:55.740 | but it's much more specific.

01:08:57.880 | And so we showed how networks of these neurons

01:09:00.300 | can do very sophisticated temporal prediction, basically.

01:09:04.180 | So this, to summarize this,

01:09:07.020 | real neurons in the brain are time-based prediction engines.

01:09:10.980 | And there's no concept of this at all

01:09:14.700 | in artificial, what we call point neurons.

01:09:18.100 | I don't think you can build a brain without them.

01:09:20.060 | I don't think you can build intelligence without them,

01:09:21.340 | because it's where a large part of the time comes from.

01:09:26.020 | These are predictive models,

01:09:27.080 | and the time is, there's a prior and a prediction

01:09:31.180 | and an action, and it's inherent

01:09:33.120 | through every neuron in the neocortex.

01:09:34.940 | So I would say that point neurons

01:09:37.180 | sort of model a piece of that,

01:09:39.140 | and not very well at that either.

01:09:40.620 | But like, for example, synapses

01:09:44.180 | are very unreliable,

01:09:46.040 | and you cannot assign any precision to them.

01:09:49.900 | So even one digit of precision is not possible.

01:09:52.480 | So the way real neurons work is they don't add these,

01:09:55.540 | they don't change these weights accurately,

01:09:57.420 | like artificial neural networks do.

01:09:59.380 | They basically form new synapses.

01:10:01.060 | And so what you're trying to always do

01:10:02.540 | is detect the presence of some 10 to 20 active synapses

01:10:07.540 | at the same time, as opposed,

01:10:09.500 | and they're almost binary.

01:10:11.380 | It's like, 'cause you can't really represent

01:10:12.860 | anything much finer than that.

01:10:14.660 | So these are the kind of,

01:10:16.260 | and I think that's actually another essential component,

01:10:18.100 | because the brain works on sparse patterns,

01:10:20.980 | and all that mechanism is based on sparse patterns,

01:10:24.220 | and I don't actually think you could build real brains

01:10:26.660 | or machine intelligence

01:10:28.380 | without incorporating some of those ideas.

01:10:30.740 | - It's hard to even think about the complexity

01:10:32.700 | that emerges from the fact that the timing

01:10:35.180 | of the firing matters in the brain,

01:10:37.180 | the fact that you form new synapses,

01:10:41.020 | and everything you just mentioned in the past couple minutes.

01:10:44.980 | - Trust me, if you spend time on it,

01:10:46.580 | you can get your mind around it.

01:10:47.980 | It's not like, it's no longer a mystery to me.

01:10:49.900 | - No, but sorry, as a function, in a mathematical way,

01:10:53.860 | can you start getting an intuition

01:10:55.980 | about what gets it excited, what not,

01:10:58.580 | and what kind of representation?

01:10:59.420 | - Yeah, it's not as easy as,

01:11:01.100 | there's many other types of neural networks

01:11:04.700 | that are more amenable to pure analysis,

01:11:07.640 | especially very simple networks.

01:11:10.820 | Oh, I have four neurons and they're doing this.

01:11:12.620 | Can we describe them mathematically

01:11:14.540 | what they're doing type of thing?

01:11:16.340 | Even the complexity of convolutional neural networks today,

01:11:19.380 | it's sort of a mystery,

01:11:20.340 | they can't really describe the whole system.

01:11:22.540 | And so it's different.

01:11:24.820 | My colleague Subitain Ahmad, he did a nice paper on this.

01:11:29.820 | You can get all this stuff on our website

01:11:32.780 | if you're interested.

01:11:34.140 | Talking about sort of the mathematical properties

01:11:36.220 | of sparse representations.

01:11:37.700 | And so we can't, what we can do is we can show mathematically

01:11:40.660 | for example, why 10 to 20 synapses to recognize a pattern

01:11:44.980 | is the correct number,

01:11:46.060 | is the right number you'd wanna use.

01:11:47.820 | And by the way, that matches biology.

01:11:50.020 | We can show mathematically some of these concepts

01:11:53.940 | about the show why the brain is so robust

01:11:58.660 | to noise and error and fallout and so on.

01:12:01.060 | We can show that mathematically

01:12:02.300 | as well as empirically in simulations.

01:12:05.060 | But the system can't be analyzed completely.

01:12:07.900 | Any complex system can't.

01:12:09.500 | And so that's out of the realm.

01:12:12.020 | But there is mathematical benefits and intuitions

01:12:17.020 | that can be derived from mathematics.

01:12:19.500 | And we try to do that as well.

01:12:20.620 | Most of our papers have a section about that.

01:12:23.340 | - So I think it's refreshing and useful for me

01:12:25.940 | to be talking to you about deep neural networks.

01:12:29.100 | Because your intuition basically says

01:12:30.940 | that we can't achieve anything like intelligence

01:12:34.580 | with artificial neural networks.

01:12:35.980 | - Well, not in their current form.

01:12:36.980 | - Not in the current form.

01:12:37.820 | - I'm sure we can do it in the ultimate form.

01:12:39.340 | - Sure.

01:12:40.220 | - So let me dig into it

01:12:41.300 | and see what your thoughts are there a little bit.

01:12:43.340 | So I'm not sure if you read this little blog post

01:12:46.020 | called Bitter Lesson by Rich Sutton.

01:12:48.260 | Recently he's a reinforcement learning pioneer.

01:12:51.660 | I'm not sure if you're familiar with him.

01:12:53.260 | His basic idea is that all the stuff we've done in AI

01:12:56.780 | in the past 70 years, he's one of the old school guys.

01:13:00.680 | The biggest lesson learned is that

01:13:05.500 | all the tricky things we've done,

01:13:09.020 | they benefit in the short term,

01:13:10.420 | but in the long term what wins out

01:13:12.100 | is a simple general method that just relies on Moore's Law.

01:13:16.320 | On computation getting faster and faster.

01:13:19.780 | - This is what he's saying,

01:13:21.240 | this is what has worked up to now.

01:13:23.220 | - This is what has worked up to now.

01:13:25.380 | If you're trying to build a system,

01:13:29.020 | if we're talking about,

01:13:30.060 | he's not concerned about intelligence,

01:13:31.420 | he's concerned about a system that works

01:13:34.420 | in terms of making predictions

01:13:36.500 | on applied narrow AI problems.

01:13:38.780 | That's what this discussion is about.

01:13:40.620 | That you just try to go as general as possible

01:13:44.220 | and wait years or decades for the computation

01:13:48.500 | to make it actually--

01:13:50.220 | - Is he saying that as a criticism

01:13:51.700 | or is he saying this is a prescription

01:13:53.300 | of what we ought to be doing?

01:13:54.380 | - Well, it's very difficult.

01:13:55.900 | He's saying this is what has worked,

01:13:58.020 | and yes, a prescription,

01:13:59.260 | but it's a difficult prescription

01:14:00.360 | because it says all the fun things

01:14:02.380 | you guys are trying to do, we are trying to do,

01:14:05.860 | he's part of the community.

01:14:07.380 | He's saying it's only going to be short-term gains.

01:14:10.740 | So this all leads up to a question, I guess,

01:14:13.780 | on artificial neural networks

01:14:15.580 | and maybe our own biological neural networks

01:14:19.060 | is do you think if we just scale things up significantly,

01:14:23.780 | so take these dumb artificial neurons,

01:14:27.140 | the point neurons, I like that term,

01:14:29.000 | if we just have a lot more of them,

01:14:33.260 | do you think some of the elements

01:14:34.540 | that we see in the brain may start emerging?

01:14:38.100 | - No, I don't think so.

01:14:39.580 | We can do bigger problems of the same type.

01:14:43.460 | I mean, it's been pointed out by many people

01:14:45.300 | that today's convolutional neural networks

01:14:46.900 | aren't really much different

01:14:47.860 | than the ones we had quite a while ago.

01:14:50.220 | We just, they're bigger and train more

01:14:51.860 | and we have more labeled data and so on.

01:14:53.940 | But I don't think you can get to the kind of things

01:14:58.620 | I know the brain can do

01:15:00.740 | and that we think of as intelligence by just scaling it up.

01:15:03.740 | So that may be, it's a good description

01:15:06.620 | of what's happened in the past,

01:15:07.700 | what's happened recently with the reemergence

01:15:09.960 | of artificial neural networks.

01:15:12.540 | It may be a good prescription

01:15:14.420 | for what's gonna happen in the short term,

01:15:16.520 | but I don't think that's the path.

01:15:19.200 | I've said that earlier.

01:15:20.900 | There's an alternate path.

01:15:21.740 | I should mention to you, by the way,

01:15:22.940 | that we've made sufficient progress

01:15:25.920 | on the whole cortical theory in the last few years

01:15:28.940 | that last year we decided to start actively pursuing

01:15:33.940 | how do we get these ideas embedded into machine learning?

01:15:39.180 | Well, that's again, being led by my colleague,

01:15:41.860 | Subrata Iman, and he's more of a machine learning guy

01:15:45.140 | and more of a neuroscience guy.

01:15:46.740 | So this is now our new, this is, I wouldn't say our focus,

01:15:51.220 | but it is now an equal focus here

01:15:54.180 | because we need to proselytize what we've learned

01:15:58.260 | and we need to show how it's beneficial

01:16:00.220 | to the machine learning.

01:16:03.740 | So we're putting, we have a plan in place right now.

01:16:05.580 | In fact, we just did our first paper on this.

01:16:07.700 | I can tell you about that.

01:16:09.700 | But one of the reasons I wanna talk to you

01:16:11.340 | is because I'm trying to get more people

01:16:14.100 | in the machine learning community to say,

01:16:15.980 | I need to learn about this stuff

01:16:17.120 | and maybe we should just think about this a bit more

01:16:19.380 | about what we've learned about the brain

01:16:20.860 | and what are those team at Numenta, what have they done?

01:16:23.860 | Is that useful for us?

01:16:25.200 | - Yeah, so is there elements

01:16:26.700 | of all the cortical theory,

01:16:28.740 | things we've been talking about

01:16:29.860 | that may be useful in the short term?

01:16:31.940 | - Yes, in the short term, yes.

01:16:33.460 | - This is the, sorry to interrupt,

01:16:34.780 | but the open question is,

01:16:37.820 | it certainly feels from my perspective

01:16:39.300 | that in the long term,

01:16:41.080 | some of the ideas we've been talking about

01:16:42.860 | will be extremely useful.

01:16:44.300 | The question is whether in the short term.

01:16:46.060 | - Well, this is a, always what I would call

01:16:48.380 | the entrepreneur's dilemma.

01:16:50.660 | So you have this long-term vision,

01:16:53.100 | oh, we're gonna all be driving electric cars

01:16:55.340 | or we're all gonna have computers

01:16:56.820 | or we're all gonna, whatever.

01:16:59.060 | And you're at some point in time

01:17:01.100 | and you say, I can see that long-term vision,

01:17:03.020 | I'm sure it's gonna happen.

01:17:03.840 | How do I get there without killing myself?

01:17:05.820 | Without going out of business, right?

01:17:07.420 | That's the challenge.

01:17:08.780 | That's the dilemma,

01:17:09.620 | that's the really difficult thing to do.

01:17:11.140 | So we're facing that right now.

01:17:13.140 | So ideally what you'd wanna do

01:17:14.660 | is find some steps along the way

01:17:16.100 | that you can get there incrementally.

01:17:17.420 | You don't have to like throw it all out

01:17:19.220 | and start over again.

01:17:20.500 | The first thing that we've done

01:17:22.340 | is we focus on the sparse representations.

01:17:25.380 | So just in case you don't know what that means

01:17:28.420 | or some of the listeners don't know what that means.

01:17:31.260 | In the brain, if I have like 10,000 neurons,

01:17:34.100 | what you would see is maybe 2% of them active at a time.

01:17:36.980 | You don't see 50%, you don't see 30%,

01:17:39.540 | you might see 2%.

01:17:41.220 | And it's always like that.

01:17:42.580 | - For any set of sensory inputs?

01:17:44.420 | - It doesn't matter if anything,

01:17:45.340 | it doesn't matter any part of the brain.

01:17:47.380 | - But which neurons differs?

01:17:51.100 | Which neurons are active?

01:17:52.660 | - Yeah, so let's say I take 10,000 neurons

01:17:55.380 | that are representing something.

01:17:56.220 | They're sitting there in a little block together.

01:17:57.940 | It's a teeny little block of neurons, 10,000 neurons.

01:17:59.940 | And they're representing a location,

01:18:01.620 | they're representing a cup,

01:18:02.500 | they're representing the input from my sensors.

01:18:04.060 | I don't know, it doesn't matter.

01:18:05.400 | It's representing something.

01:18:07.020 | The way the representations occur,

01:18:09.140 | it's always a sparse representation,

01:18:10.620 | meaning it's a population code.

01:18:11.900 | So which 200 cells are active tells me what's going on.

01:18:14.980 | It's not, individual cells aren't that important at all.

01:18:18.080 | It's the population code that matters.

01:18:20.260 | And when you have sparse population codes,

01:18:23.140 | then all kinds of beautiful properties come out of them.

01:18:26.300 | So the brain uses sparse population codes.

01:18:28.060 | And we've written and described these benefits

01:18:30.780 | in some of our papers.

01:18:32.420 | So they give this tremendous robustness to the systems.

01:18:37.420 | Brains are incredibly robust.

01:18:39.160 | Neurons are dying all the time and spasming

01:18:41.140 | and synapses falling apart and all the time,

01:18:43.940 | and it keeps working.

01:18:45.300 | So what Subutai and Louise,

01:18:49.180 | one of our other engineers here have done,

01:18:52.620 | I've shown they're introducing sparseness

01:18:55.680 | into convolutional neural networks.

01:18:56.820 | Now other people are thinking along these lines,

01:18:58.100 | but we're going about it in a more principled way, I think.

01:19:00.940 | And we're showing that if you enforce sparseness

01:19:04.060 | throughout these convolutional neural networks,

01:19:06.380 | in both the, which sort of which neurons are active

01:19:11.300 | and the connections between them,

01:19:13.760 | that you get some very desirable properties.

01:19:15.640 | So one of the current hot topics in deep learning right now

01:19:18.860 | are these adversarial examples.

01:19:20.900 | So you give me any deep learning network

01:19:23.500 | and I can give you a picture that looks perfect

01:19:26.040 | and you're gonna call it,

01:19:27.100 | you're gonna say the monkey is an airplane.

01:19:29.840 | (laughs)

01:19:30.680 | So that's a problem.

01:19:32.520 | And DARPA just announced some big thing.

01:19:34.120 | They're trying to have some contest for this.

01:19:36.580 | But if you enforce sparse representations here,

01:19:40.180 | many of these problems go away.

01:19:41.500 | They're much more robust.

01:19:43.020 | And they're not easy to fool.

01:19:44.940 | So we've already shown some of those results.

01:19:48.340 | Just literally in January or February,

01:19:51.140 | just like last month we did that.

01:19:52.780 | And you can, I think it's on bioRxiv right now or on iCry,

01:19:58.480 | you can read about it.

01:19:59.560 | But so that's like a baby step.

01:20:02.460 | Okay, that's a take something from the brain.

01:20:04.340 | We know about sparseness.

01:20:05.620 | We know why it's important.

01:20:06.540 | We know what it gives the brain.

01:20:08.060 | So let's try to enforce that onto this.

01:20:09.500 | - What's your intuition why sparsity leads to robustness?

01:20:12.380 | 'Cause it feels like it would be less robust.

01:20:15.340 | - Why would you feel the rest robust to you?

01:20:17.380 | (laughs)

01:20:19.060 | - So it just feels like if the fewer neurons are involved,

01:20:24.060 | the more fragile the representation.

01:20:26.620 | - But I didn't say there was lots of few.

01:20:28.180 | I said let's say 200.

01:20:29.820 | That's a lot.

01:20:30.980 | - There's still a lot.

01:20:32.580 | - So here's an intuition for it.

01:20:35.220 | This is a bit technical.

01:20:36.380 | So for engineers, machine learning people, this would be easy

01:20:41.240 | but other listeners, maybe not.

01:20:42.860 | If you're trying to classify something,

01:20:45.700 | you're trying to divide some very high dimensional space

01:20:48.340 | into different pieces, A and B.

01:20:50.340 | And you're trying to create some point where you say,

01:20:52.780 | all these points in this high dimensional space are A

01:20:54.740 | and all these points in this high dimensional space are B.

01:20:57.540 | And if you have points that are close to that line,

01:21:01.940 | it's not very robust.

01:21:02.860 | It works for all the points you know about,

01:21:04.900 | but it's not very robust

01:21:07.060 | 'cause you can just move a little bit

01:21:08.220 | and you've crossed over the line.

01:21:10.260 | When you have sparse representations,

01:21:12.680 | imagine I pick, I'm gonna pick 200 cells active

01:21:16.000 | out of 10,000.

01:21:17.480 | So I have 200 cells active.

01:21:20.320 | Now let's say I pick randomly a different representation, 200.

01:21:24.380 | The overlap between those is gonna be very small,

01:21:26.720 | just a few.

01:21:28.040 | I can pick millions of samples randomly of 200 neurons

01:21:32.800 | and not one of them will overlap more than just a few.

01:21:36.100 | So one way to think about it is

01:21:39.120 | if I wanna fool one of these representations

01:21:41.440 | to look like one of those other representations,

01:21:43.440 | I can't move just one cell or two cells

01:21:45.600 | or three cells or four cells.

01:21:46.760 | I have to move 100 cells.

01:21:49.120 | And that makes them robust.

01:21:52.640 | - In terms of further, so you mentioned sparsity.

01:21:56.120 | - Will it be the next thing?

01:21:57.240 | - Yeah.

01:21:58.080 | - Okay, so we picked one.

01:22:00.400 | We don't know if it's gonna work well yet.

01:22:02.360 | So again, we're trying to come up incremental ways

01:22:04.520 | to moving from brain theory to adding pieces

01:22:07.840 | to machine learning, current machine learning world

01:22:10.120 | and one step at a time.

01:22:12.240 | So the next thing we're gonna try to do

01:22:13.720 | is sort of incorporate some of the ideas

01:22:15.800 | of the thousand brains theory,

01:22:19.060 | that you have many, many models that are voting.

01:22:22.560 | Now that idea is not new.

01:22:23.680 | There's a mixture of models

01:22:24.780 | that's been around for a long time.

01:22:27.120 | But the way the brain does it is a little different.

01:22:29.720 | And the way it votes is different.

01:22:33.600 | And the kind of way it represents uncertainty is different.

01:22:37.120 | So we're just starting this work,

01:22:39.940 | but we're gonna try to see if we can sort of incorporate

01:22:42.240 | some of the principles of voting

01:22:43.720 | or principles of a thousand brain theory,

01:22:45.920 | like lots of simple models that talk to each other

01:22:49.440 | in a very certain way.

01:22:53.000 | And can we build more machines,

01:22:56.280 | the systems that learn faster and also,

01:22:58.940 | well, mostly are multimodal and robust

01:23:03.920 | to multimodal type of issues.

01:23:07.520 | - So one of the challenges there is,

01:23:10.080 | the machine learning, computer vision community

01:23:13.120 | has certain sets of benchmarks.

01:23:15.600 | So it's a test based on which they compete.

01:23:18.160 | And I would argue, especially from your perspective,

01:23:22.080 | that those benchmarks aren't that useful

01:23:24.680 | for testing the aspects that the brain is good at

01:23:28.840 | or intelligent.

01:23:29.920 | They're not really testing intelligence.

01:23:31.280 | It's a very fine, and it's been extremely useful

01:23:34.760 | for developing specific mathematical models,

01:23:37.400 | but it's not useful in the longterm

01:23:40.400 | for creating intelligence.

01:23:41.680 | So you think you also have a role in proposing better tests?

01:23:46.680 | - Yeah, this is a very,

01:23:48.480 | you've identified a very serious problem.

01:23:50.520 | First of all, the tests that they have

01:23:53.320 | are the tests that they want,

01:23:54.600 | not the test of the other things

01:23:55.860 | that we're trying to do, right?

01:23:59.000 | What are the, so on.

01:24:01.720 | The second thing is sometimes these,

01:24:04.240 | to be competitive in these tests,

01:24:06.600 | you have to have huge data sets and huge computing power.

01:24:09.940 | And so, and we don't have that here.

01:24:13.400 | We don't have it as well as other big teams

01:24:15.480 | and big companies do.

01:24:17.600 | So there's numerous issues there.

01:24:21.120 | We come at it, our approach to this is all based on,

01:24:24.280 | in some sense, you might argue elegance.

01:24:26.120 | We're coming at it from like a theoretical base

01:24:27.760 | that we think, oh my God, this is so clearly elegant.

01:24:29.980 | This is how brains work, this is what intelligence is.

01:24:31.880 | But the machine learning world has gotten in this phase

01:24:33.920 | where they think it doesn't matter,

01:24:35.520 | doesn't matter what you think,

01:24:36.600 | as long as you do 0.1% better on this benchmark,

01:24:39.440 | that's all that matters.

01:24:40.800 | And that's a problem.

01:24:42.740 | We have to figure out how to get around that.

01:24:46.080 | That's a challenge for us.

01:24:47.320 | That's one of the challenges that we have to deal with.

01:24:50.520 | So I agree, you've identified a big issue.

01:24:52.840 | It's difficult for those reasons.

01:24:55.920 | But part of the reasons I'm talking to you here today

01:24:59.960 | is I hope I'm going to get some machine learning people

01:25:01.680 | to say, I'm going to read those papers.

01:25:03.320 | Those might be some interesting ideas.

01:25:05.200 | I'm tired of doing this 0.1% improvement stuff.

01:25:07.720 | - Well, that's why I'm here as well,

01:25:10.440 | because I think machine learning now as a community

01:25:13.080 | is at a place where the next step

01:25:15.520 | needs to be orthogonal to what has received success

01:25:20.840 | in the past.

01:25:21.680 | - You see other leaders saying this,

01:25:23.080 | machine learning leaders, you know.

01:25:25.480 | Jeff Hinton with his capsules idea.

01:25:27.920 | Many people have gotten up to say,

01:25:29.280 | we're going to hit road,

01:25:31.040 | maybe we should look at the brain, things like that.

01:25:33.440 | So hopefully that thinking will occur organically.

01:25:38.080 | And then we're in a nice position for people to come

01:25:40.760 | and look at our work and say,

01:25:41.720 | well, what can we learn from these guys?

01:25:43.160 | - Yeah, MIT is launching a billion dollar computing college

01:25:47.480 | that's centered around this idea.

01:25:49.440 | - On this idea of what?

01:25:51.000 | - Well, the idea that the humanities, psychology,

01:25:54.200 | neuroscience have to work all together

01:25:56.320 | to get to build the S.

01:25:58.920 | - Yeah, I mean, Stanford just did this human centered AI.

01:26:02.560 | I'm a little disappointed in these initiatives because,

01:26:05.360 | you know, they're focusing on sort of the human side of it

01:26:09.960 | and it could very easily slip into

01:26:12.200 | how humans interact with intelligent machines,

01:26:16.080 | which is nothing wrong with that,

01:26:17.640 | but that is orthogonal to what we're trying to do.

01:26:20.400 | We're trying to say like,

01:26:21.400 | what is the essence of intelligence?

01:26:23.720 | In fact, I want to build intelligent machines

01:26:25.480 | that aren't emotional, that don't smile at you,

01:26:28.120 | that aren't trying to tuck you in at night.

01:26:31.800 | - Yeah, there is that pattern that when you talk about

01:26:35.680 | understanding humans is important

01:26:37.000 | for understanding intelligence,

01:26:38.360 | that you start slipping into topics of ethics or,

01:26:41.860 | yeah, like you said, the interactive elements

01:26:44.800 | as opposed to, no, no, no,

01:26:46.240 | we have to zoom in on the brain,

01:26:47.360 | study what the human brain, the baby.

01:26:51.400 | Let's study what a brain does.

01:26:53.200 | And then we can decide which parts of that

01:26:54.720 | we want to recreate in some system.

01:26:57.720 | But until you have that theory about what the brain does,

01:26:59.840 | what's the point?

01:27:00.680 | You know, it's just, you're going to be wasting time.

01:27:03.200 | - Just to break it down on the artificial neural network side,

01:27:05.560 | maybe you can speak to this on the biological neural network

01:27:08.840 | side, the process of learning

01:27:10.120 | versus the process of inference.

01:27:11.800 | Maybe you can explain to me,

01:27:15.600 | is there a difference between,

01:27:18.440 | you know, in artificial neural networks,

01:27:19.840 | there's a difference between the learning stage

01:27:21.480 | and the inference stage.

01:27:22.960 | Do you see the brain as something different?

01:27:25.000 | One of the big distinctions that people often say,

01:27:29.000 | I don't know how correct it is,

01:27:30.640 | is artificial neural networks need a lot of data.

01:27:32.960 | They're very inefficient learning.

01:27:34.800 | Do you see that as a correct distinction

01:27:37.320 | from the biology of the human brain,

01:27:40.320 | that the human brain is very efficient

01:27:42.000 | or is that just something we deceive ourselves with?

01:27:44.240 | - No, it is efficient, obviously.

01:27:45.400 | We can learn new things almost instantly.

01:27:47.560 | - And so what elements do you think--

01:27:50.040 | - Yeah, I can talk about that.

01:27:50.880 | You brought up two issues there.

01:27:52.320 | So remember I talked early about the constraints.

01:27:54.840 | We always feel, well, one of those constraints

01:27:57.280 | is the fact that brains are continually learning.

01:28:00.960 | That's not something we said, oh, we can add that later.

01:28:03.800 | That's something that was upfront,

01:28:05.840 | had to be there from the start,

01:28:07.760 | made our problems harder.

01:28:11.280 | But we showed, going back to the 2016 paper

01:28:14.440 | on sequence memory, we showed how that happens.

01:28:16.760 | How do brains infer and learn at the same time?

01:28:19.920 | And our models do that.

01:28:22.240 | They're not two separate phases

01:28:24.040 | or two separate sets of time.

01:28:26.320 | I think that's a big, big problem in AI,

01:28:29.760 | at least for many applications, not for all.

01:28:32.520 | So I can talk about that.

01:28:34.480 | There are some, it gets detailed.

01:28:37.160 | There are some parts of the neocortex in the brain

01:28:39.640 | where actually what's going on,

01:28:41.720 | there's these cycles,

01:28:44.240 | they're like cycles of activity in the brain.

01:28:46.840 | And there's very strong evidence

01:28:49.240 | that you're doing more of inference on one part of the phase

01:28:52.280 | and more of learning on the other part of the phase.

01:28:54.120 | So the brain can actually sort of separate

01:28:55.480 | different populations of cells

01:28:56.640 | or going back and forth like this.

01:28:58.320 | But in general, I would say that's an important problem.

01:29:01.560 | We have all of our networks that we've come up with do both.

01:29:05.600 | And they're learning, continuous learning networks.

01:29:08.200 | And you mentioned benchmarks earlier.

01:29:10.960 | Well, there are no benchmarks about that.

01:29:12.520 | - Exactly.

01:29:13.640 | - So we have to like, we get in our little soapbox

01:29:17.200 | and hey, by the way, this is important,

01:29:19.240 | and here's a mechanism for doing that.

01:29:20.560 | But until you can prove it to someone

01:29:23.920 | in some commercial system or something, it's a little harder.

01:29:26.680 | - So yeah, one of the things I had to linger on that is,

01:29:30.080 | in some ways to learn the concept of a coffee cup,

01:29:33.760 | you only need this one coffee cup

01:29:35.880 | and maybe some time alone in a room with it.

01:29:38.000 | - Well, the first thing is,

01:29:39.920 | imagine I reach my hand into a black box

01:29:41.760 | and I'm reaching, I'm trying to touch something.

01:29:43.640 | I don't know up front if it's something I already know

01:29:46.160 | or if it's a new thing.

01:29:47.800 | And I have to, I'm doing both at the same time.

01:29:50.440 | I don't say, oh, let's see if it's a new thing.

01:29:53.240 | Oh, let's see if it's an old thing.

01:29:54.680 | I don't do that.

01:29:55.520 | As I go, my brain says, oh, it's new or it's not new.

01:29:59.360 | And if it's new, I start learning what it is.

01:30:02.240 | So, and by the way, it starts learning from the get go,

01:30:04.800 | even if it's gonna recognize it.

01:30:05.960 | So they're not separate problems.

01:30:08.880 | And so that's the thing there.

01:30:10.080 | The other thing you mentioned was the fast learning.

01:30:12.640 | So I was just talking about continuous learning,

01:30:15.600 | but there's also fast learning.

01:30:16.680 | Literally, I can show you this coffee cup

01:30:18.800 | and I say, here's a new coffee cup.

01:30:20.080 | It's got the logo on it.

01:30:21.360 | Take a look at it, done, you're done.

01:30:23.880 | You can predict what it's gonna look like,

01:30:25.400 | you know, in different positions.

01:30:27.480 | So I can talk about that too.

01:30:29.560 | In the brain, the way learning occurs,

01:30:34.280 | I mentioned this earlier, but I'll mention it again.

01:30:35.720 | The way learning occurs,

01:30:36.880 | imagine I am a section of a dendrite of a neuron.

01:30:39.280 | And I wanna learn, I'm gonna learn something new.

01:30:43.520 | I'm just, doesn't matter what it is.

01:30:44.640 | I'm just gonna learn something new.

01:30:46.240 | I need to recognize a new pattern.

01:30:49.000 | So what I'm gonna do is I'm gonna form new synapses.

01:30:51.760 | New synapses, we're gonna rewire the brain

01:30:55.240 | onto that section of the dendrite.

01:30:58.000 | Once I've done that,

01:30:59.400 | everything else that neuron has learned

01:31:01.120 | is not affected by it.

01:31:02.680 | That's because it's isolated

01:31:04.440 | to that small section of the dendrite.

01:31:06.480 | They're not all being added together like a point neuron.

01:31:09.640 | So if I learn something new on this segment here,

01:31:11.800 | it doesn't change any of the learning

01:31:13.240 | that occur anywhere else in that neuron.

01:31:14.920 | So I can add something without affecting previous learning.

01:31:18.480 | And I can do it quickly.

01:31:19.840 | Now let's talk, we can talk about the quickness,

01:31:22.360 | how it's done in real neurons.

01:31:24.080 | You might say, well,

01:31:24.920 | doesn't it take time to form synapses?

01:31:26.800 | Yes, it can take maybe an hour to form a new synapse.

01:31:30.080 | We can form memories quicker than that.

01:31:32.600 | And I can explain that in how it's tuned if you want.

01:31:35.920 | But it's getting a bit neuroscience-y.

01:31:38.360 | - That's great, but is there an understanding

01:31:41.440 | of these mechanisms at every level?

01:31:43.360 | So from the short-term memories

01:31:44.920 | and the forming of new connections?

01:31:48.720 | - So this idea of synaptogenesis,

01:31:50.520 | the growth of new synapses,

01:31:51.640 | that's well described, it's well understood.

01:31:54.160 | - And that's an essential part of learning.

01:31:55.920 | - That is learning.

01:31:56.880 | - That is learning. - It's learning, okay?

01:31:59.040 | Going back many, many years,

01:32:03.920 | people, what's his name,

01:32:06.360 | the psychologist who proposed, Heb, Donald Heb,

01:32:09.640 | he proposed that learning was the modification

01:32:12.040 | of the strength of a connection between two neurons.

01:32:15.440 | People interpreted that as the modification

01:32:18.200 | of the strength of a synapse.

01:32:19.680 | He didn't say that.

01:32:20.960 | He just said there's a modification

01:32:22.360 | between the effect of one neuron and another.

01:32:24.560 | So synaptogenesis is totally consistent

01:32:26.520 | with what Donald Heb said.

01:32:28.160 | But anyway, there's these mechanisms,

01:32:29.880 | the growth of new synapse.

01:32:30.880 | You can go online, you can watch a video

01:32:32.240 | of a synapse growing in real time.

01:32:33.920 | It's literally, you can see this little thing going,

01:32:35.480 | boop, right?

01:32:37.120 | It's pretty impressive.

01:32:38.440 | So those mechanisms are known.

01:32:39.760 | Now, there's another thing that we've speculated

01:32:42.360 | and we've written about,

01:32:43.520 | which is consistent with known neuroscience,

01:32:45.760 | but it's less proven.

01:32:48.360 | And this is the idea,

01:32:49.200 | how do I form a memory really, really quickly?

01:32:51.600 | Like instantaneous.

01:32:52.800 | If it takes an hour to grow a synapse,

01:32:54.840 | that's not instantaneous.

01:32:56.840 | So there are types of synapses called silent synapses.

01:33:01.720 | They look like a synapse, but they don't do anything.

01:33:04.080 | They're just sitting there.

01:33:04.920 | It's like, if an action potential comes in,

01:33:07.880 | it doesn't release any neurotransmitter.

01:33:10.120 | Some parts of the brain have more of these than others.

01:33:12.480 | For example, the hippocampus has a lot of them,

01:33:14.040 | which is where we associate most short-term memory with.

01:33:17.040 | So what we speculated, again, in that 2016 paper,

01:33:22.080 | we proposed that the way we form very quick memories,

01:33:26.400 | very short-term memories, or quick memories,

01:33:28.920 | is that we convert silent synapses into active synapses.

01:33:33.920 | It's like saying a synapse has a zero weight and a one weight

01:33:37.840 | but the long-term memory has to be formed by synaptogenesis.

01:33:41.440 | So you can remember something really quickly

01:33:43.280 | by just flipping a bunch of these guys

01:33:44.640 | from silent to active.

01:33:46.160 | It's not from 0.1 to 0.15.

01:33:49.080 | It's like, doesn't do anything till it releases transmitter.

01:33:52.200 | And if I do that over a bunch of these,

01:33:53.480 | I've got a very quick short-term memory.

01:33:55.760 | So I guess the lesson behind this

01:33:58.480 | is that most neural networks today are fully connected.

01:34:01.840 | Every neuron connects every other neuron

01:34:03.360 | from layer to layer.

01:34:04.560 | That's not correct in the brain.

01:34:06.040 | We don't want that.

01:34:06.960 | We actually don't want that.

01:34:08.320 | It's bad.

01:34:09.240 | You want a very sparse connectivity

01:34:10.680 | so that any neuron connects to some subset of the neurons

01:34:14.480 | in the other layer,

01:34:15.320 | and it does so on a dendrite by dendrite segment basis.

01:34:18.960 | So it's a very parcelated out type of thing.

01:34:21.560 | And that then learning is not adjusting all these weights,

01:34:25.360 | but learning is just saying,

01:34:26.320 | okay, connect to these 10 cells here right now.

01:34:30.120 | - In that process, with artificial neural networks,

01:34:32.920 | it's a very simple process of back propagation

01:34:36.000 | that adjusts the weights.

01:34:37.120 | The process of synaptogenesis.

01:34:40.120 | - Synaptogenesis.

01:34:40.960 | - Synaptogenesis.

01:34:42.240 | - It's even easier.

01:34:43.080 | - It's even easier. - It's even easier.

01:34:44.240 | Back propagation requires something

01:34:46.400 | that really can't happen in brains.

01:34:48.680 | This back propagation of this error signal.

01:34:51.160 | That really can't happen.

01:34:52.000 | People are trying to make it happen in brains,

01:34:53.440 | but it doesn't happen in brains.

01:34:54.720 | This is pure Hebbian learning.

01:34:56.760 | What synaptogenesis is pure Hebbian learning.

01:34:58.640 | It's basically saying,

01:35:00.120 | there's a population of cells over here

01:35:01.520 | that are active right now.

01:35:02.960 | And there's a population of cells over here

01:35:04.280 | that are active right now.

01:35:05.320 | How do I form connections between those active cells?

01:35:07.960 | And it's literally saying, this guy became active,

01:35:11.480 | these 100 neurons here became active

01:35:13.200 | before this neuron became active.

01:35:15.040 | So form connections to those ones.

01:35:17.080 | That's it.

01:35:17.920 | There's no propagation of error, nothing.

01:35:19.880 | All the networks we do, all the models we have

01:35:22.200 | work on, almost completely on Hebbian learning,

01:35:26.400 | but in dendritic segments and multiple synapses

01:35:31.400 | at the same time.

01:35:33.040 | - So now let's return the question

01:35:34.520 | that you already answered,

01:35:35.840 | and maybe you can answer it again.

01:35:37.780 | If you look at the history of artificial intelligence,

01:35:41.280 | where do you think we stand?

01:35:43.560 | How far are we from solving intelligence?

01:35:45.800 | You said you were very optimistic.

01:35:47.720 | Can you elaborate on that?

01:35:48.880 | - Yeah, it's always the crazy question

01:35:52.080 | to ask, 'cause no one can predict the future.

01:35:55.080 | - Absolutely.

01:35:55.920 | - So I'll tell you a story.

01:35:58.160 | I used to run a different neuroscience institute

01:36:01.360 | called the Redbird Neuroscience Institute,

01:36:02.560 | and we would hold these symposiums,

01:36:04.680 | and we'd get like 35 scientists

01:36:06.360 | from around the world to come together.

01:36:08.000 | And I used to ask them all the same question.

01:36:10.360 | I would say, well, how long do you think it'll be

01:36:11.680 | before we understand how the neocortex works?

01:36:14.480 | And everyone went around the room,

01:36:15.480 | and they had introduced a name,

01:36:16.520 | and they'd have to answer that question.

01:36:18.200 | So I got, the typical answer was 50 to 100 years.

01:36:22.920 | Some people would say 500 years.

01:36:24.720 | Some people said never.

01:36:25.840 | I said, why are you a neuroscientist?

01:36:27.640 | (laughing)

01:36:29.600 | It's good pay.

01:36:30.440 | (laughing)

01:36:32.760 | It's interesting.

01:36:33.600 | But it doesn't work like that.

01:36:36.280 | As I mentioned earlier, these are step functions.

01:36:39.600 | Things happen, and then bingo, they happen.

01:36:41.760 | You can't predict that.

01:36:43.600 | I feel I've already passed a step function.

01:36:45.600 | So if I can do my job correctly over the next five years,

01:36:49.080 | then meaning I can proselytize these ideas,

01:36:53.520 | I can convince other people they're right,

01:36:56.160 | we can show that other people,

01:36:58.760 | machine learning people should pay attention to these ideas,

01:37:01.440 | then we're definitely in an under 20 year timeframe.

01:37:04.600 | If I can do those things, if I'm not successful in that,

01:37:07.800 | and this is the last time anyone talks to me,

01:37:09.800 | and no one reads our papers,

01:37:11.320 | and I'm wrong or something like that,

01:37:14.000 | then I don't know.

01:37:15.960 | But it's not 50 years.

01:37:17.840 | The same thing about electric cars,

01:37:22.960 | how quickly are they gonna populate the world?

01:37:25.040 | It probably takes about a 20 year span.

01:37:27.040 | It'll be something like that,

01:37:28.840 | but I think if I can do what I said, we're starting it.

01:37:31.760 | - And of course, there could be other,

01:37:34.200 | you said step functions, it could be

01:37:36.160 | everybody gives up on your ideas for 20 years,

01:37:40.080 | and then all of a sudden, somebody picks it up again.

01:37:42.160 | Wait, that guy was onto something.

01:37:43.600 | - Yeah, so that would be a failure on my part.

01:37:46.240 | Think about Charles Babbage.

01:37:49.800 | Charles Babbage, he's the guy who invented the computer

01:37:52.200 | back in the 1800s.

01:37:53.360 | And everyone forgot about it until 100 years later.

01:37:59.480 | It's, hey, this guy figured this stuff out a long time ago.

01:38:02.400 | But he was ahead of his time.

01:38:03.960 | I don't think, as I said,

01:38:06.480 | I recognize this is part of any entrepreneur's challenge.

01:38:09.560 | I use entrepreneur broadly in this case.

01:38:11.480 | I'm not meaning like I'm building a business

01:38:12.960 | trying to sell something.

01:38:13.800 | I mean, I'm trying to sell ideas.

01:38:15.920 | And this is the challenge as to how you get people

01:38:19.400 | to pay attention to you,

01:38:20.640 | how do you get them to give you

01:38:22.600 | positive or negative feedback,

01:38:24.760 | how do you get the people

01:38:25.600 | to act differently based on your ideas.

01:38:27.280 | So we'll see what we do on that.

01:38:30.200 | - So you know that there's a lot of hype

01:38:32.320 | behind artificial intelligence currently.

01:38:34.480 | As you look to spread the ideas

01:38:39.560 | that are in your cortical theory,

01:38:41.840 | the things you're working on,

01:38:43.320 | do you think there's some possibility

01:38:45.120 | we'll hit an AI winter once again?

01:38:47.320 | - Yeah, it's certainly a possibility.

01:38:49.000 | No question about it. - That's something

01:38:49.840 | you worry about?

01:38:50.680 | - Yeah, well, I guess, do I worry about it?

01:38:52.760 | I haven't decided yet if that's good or bad for my mission.

01:38:57.400 | - That's true.

01:38:59.280 | That's very true.

01:39:00.120 | Because it's almost like you need the winter

01:39:03.000 | to refresh the palette.

01:39:04.320 | - Yeah, it's like, I want,

01:39:06.480 | here's what you wanna have it is.

01:39:07.920 | You want, like, to the extent that everyone

01:39:10.800 | is so thrilled about the current state

01:39:13.040 | of machine learning and AI,

01:39:15.440 | and they don't imagine they need anything else,

01:39:18.120 | it makes my job harder.

01:39:19.760 | If everything crashed completely,

01:39:22.560 | and every student left the field,

01:39:24.280 | and there was no money for anybody to do anything,

01:39:26.200 | and it became an embarrassment

01:39:27.480 | to talk about machine intelligence and AI,

01:39:29.040 | that wouldn't be good for us either.

01:39:30.760 | You want sort of the soft landing approach, right?

01:39:33.400 | You want enough people, the senior people in AI

01:39:36.640 | and machine learning to say,

01:39:37.480 | "You know, we need other approaches.

01:39:38.840 | "We really need other approaches.

01:39:40.360 | "Damn, we need other approaches.

01:39:41.920 | "Maybe we should look to the brain.

01:39:43.000 | "Okay, let's look to the brain.

01:39:44.120 | "Who's got some brain ideas?

01:39:45.280 | "Okay, let's start a little project on the side here,

01:39:47.800 | "trying to do brain idea related stuff."

01:39:49.680 | That's the ideal outcome we would want.

01:39:51.800 | So I don't want a total winter,

01:39:53.960 | and yet I don't want it to be sunny all the time either.

01:39:56.680 | (laughing)

01:39:57.680 | - So what do you think it takes

01:39:59.120 | to build a system with human level intelligence

01:40:03.000 | where once demonstrated, you would be very impressed?

01:40:06.800 | So does it have to have a body?

01:40:08.720 | Does it have to have the C word we used before,

01:40:12.760 | consciousness, as an entirety, in a holistic sense?

01:40:17.760 | - First of all, I don't think the goal

01:40:20.480 | is to create a machine that is human level intelligence.

01:40:23.740 | I think it's a false goal.

01:40:24.960 | Back to Turing, I think it was a false statement.

01:40:27.360 | We want to understand what intelligence is,

01:40:29.040 | and then we can build intelligent machines

01:40:30.760 | of all different scales, all different capabilities.

01:40:33.680 | You know, a dog is intelligent.

01:40:35.280 | I don't need, you know,

01:40:36.120 | that'd be pretty good to have a dog, you know?

01:40:38.480 | What about something that doesn't look like an animal

01:40:40.040 | at all, in different spaces?

01:40:41.600 | So my thinking about this is that we want to define

01:40:44.960 | what intelligence is, agree upon

01:40:46.800 | what makes an intelligent system.

01:40:48.760 | We can then say, okay, we're now gonna build systems

01:40:51.040 | that work on those principles, or some subset of them,

01:40:54.280 | and we can apply them to all different types of problems.

01:40:57.280 | And the kind, the idea, it's like computing.

01:41:00.800 | We don't ask, if I take a little, you know,

01:41:03.920 | little one chip computer, I don't say,

01:41:05.720 | well, that's not a computer,

01:41:06.640 | because it's not as powerful as this,

01:41:08.120 | you know, big server over here.

01:41:09.640 | No, no, because we know that what the principles

01:41:11.280 | of computing are, and I can apply those principles

01:41:12.920 | to a small problem or into a big problem.

01:41:14.880 | And same, intelligence just needs to get there.

01:41:16.520 | We have to say, these are the principles.

01:41:17.640 | I can make a small one, a big one,

01:41:19.000 | I can make them distributed,

01:41:19.960 | I can put them on different sensors.

01:41:21.600 | They don't have to be human-like at all.

01:41:23.200 | Now, you did bring up a very interesting question

01:41:24.720 | about embodiment.

01:41:25.640 | Does it have to have a body?

01:41:27.480 | It has to have some concept of movement.

01:41:30.640 | It has to be able to move through these reference frames

01:41:33.240 | I talked about earlier.

01:41:34.440 | Whether it's physically moving, like I need,

01:41:36.360 | if I'm gonna have an AI that understands coffee cups,

01:41:38.800 | it's gonna have to pick up the coffee cup

01:41:40.520 | and touch it and look at it with its eyes and hands,

01:41:43.200 | or something equivalent to that.

01:41:45.440 | If I have a mathematical AI,

01:41:48.200 | maybe it needs to move through mathematical spaces.

01:41:51.360 | I could have a virtual AI that lives in the internet,

01:41:55.280 | and its movements are traversing links,

01:41:59.040 | and digging into files, but it's got a location

01:42:01.560 | that it's traveling through some space.

01:42:05.080 | You can't have an AI that just takes some flash thing input,

01:42:09.080 | we call it flash inference.

01:42:10.680 | Here's a pattern, done.

01:42:12.880 | No, it's movement, time, movement pattern,

01:42:15.280 | movement pattern, movement pattern, attention,

01:42:16.960 | digging, building structure,

01:42:18.600 | just figuring out the model of the world.

01:42:20.440 | So some sort of embodiment,

01:42:22.720 | whether it's physical or not, has to be part of it.

01:42:25.760 | - So self-awareness in the way to be able to answer

01:42:28.000 | where am I?

01:42:28.840 | - Well, you're bringing up self-awareness,

01:42:29.680 | it's a different topic, self-awareness.

01:42:31.440 | - No, no, the very narrow definition of self,

01:42:33.680 | meaning knowing a sense of self enough to know

01:42:37.720 | where am I in the space where it's interesting.

01:42:40.360 | - Basically, the system needs to know its location,

01:42:43.520 | or each component of the system needs to know

01:42:46.000 | where it is in the world at that point in time.

01:42:48.600 | - So self-awareness and consciousness.

01:42:51.680 | Do you think, one, from the perspective of neuroscience

01:42:55.640 | and neocortex, these are interesting topics,

01:42:58.200 | solvable topics, do you have any ideas

01:43:00.840 | of why the heck it is that we have

01:43:02.680 | a subjective experience at all?

01:43:04.440 | - Yeah, I have a lot of questions.

01:43:05.280 | - And is it useful, or is it just a side effect of us?

01:43:08.480 | - It's interesting to think about.

01:43:10.160 | I don't think it's useful as a means

01:43:12.840 | to figure out how to build intelligent machines.

01:43:15.200 | It's something that systems do,

01:43:20.200 | and we can talk about what it is,

01:43:21.880 | that are like, well, if I build a system like this,

01:43:24.040 | then it would be self-aware,

01:43:25.320 | or if I build it like this, it wouldn't be self-aware.

01:43:28.360 | So that's a choice I can have.

01:43:30.080 | It's not like, oh my God, it's self-aware!

01:43:32.320 | I can't turn, I heard an interview recently

01:43:35.840 | with this philosopher from Yale,

01:43:37.160 | I can't remember his name, I apologize for that.

01:43:39.080 | But he was talking about, well,

01:43:40.000 | if these computers are self-aware,

01:43:41.480 | then it would be a crime to unplug 'em.

01:43:42.960 | And I'm like, oh, come on, that's not,

01:43:45.120 | I unplug myself every night, I go to sleep.

01:43:46.680 | What, is that a crime?

01:43:48.280 | I plug myself in again in the morning, and there I am.

01:43:52.000 | People get kind of bent out of shape about this.

01:43:56.040 | I have very definite, very detailed understanding,

01:43:59.520 | or opinions about what it means to be conscious,

01:44:02.280 | and what it means to be self-aware.

01:44:04.320 | I don't think it's that interesting a problem.

01:44:06.760 | You've talked to Christoph Koch,

01:44:08.720 | he thinks that's the only problem.

01:44:10.920 | I didn't actually listen to your interview with him,

01:44:12.360 | but I know him, and I know that's the thing he cares about.

01:44:15.800 | - He also thinks intelligence and consciousness

01:44:17.640 | are disjoint, so I mean, it's not,

01:44:19.320 | you don't have to have one or the other.

01:44:21.000 | So he is-- - I disagree with that.

01:44:22.760 | I just totally disagree with that.

01:44:24.600 | - So where's your thoughts on consciousness,

01:44:26.320 | where does it emerge from?

01:44:27.680 | 'Cause it is-- - So then we have to

01:44:29.200 | break it down to the two parts, okay?

01:44:30.880 | 'Cause consciousness isn't one thing,

01:44:32.160 | that's part of the problem with that term,

01:44:33.720 | is it means different things to different people,

01:44:35.520 | and there's different components of it.

01:44:37.640 | There is a concept of self-awareness, okay?

01:44:40.840 | That can be very easily explained.

01:44:43.120 | You have a model of your own body,

01:44:46.080 | the neocortex models things in the world,

01:44:48.120 | and it also models your own body.

01:44:50.520 | And then it has a memory.

01:44:53.360 | It can remember what you've done, okay?

01:44:55.840 | So it can remember what you did this morning,

01:44:57.520 | can remember what you had for breakfast, and so on.

01:44:59.600 | And so I can say to you, "Okay, Lex,

01:45:01.800 | "were you conscious this morning

01:45:04.400 | "when you had your bagel?"

01:45:06.880 | And you'd say, "Yes, I was conscious."

01:45:08.760 | Now what if I could take your brain

01:45:10.240 | and revert all the synapses back

01:45:12.000 | to the state they were this morning?

01:45:14.160 | And then I said to you, "Lex, were you conscious

01:45:16.400 | "when you ate the bagel?"

01:45:17.240 | And you said, "No, I wasn't conscious."

01:45:18.480 | I said, "Here's a video of you eating the bagel."

01:45:19.720 | And you say, "I wasn't there, I have no,

01:45:22.280 | "that's not possible 'cause I must've been unconscious

01:45:24.640 | "at that time."

01:45:25.600 | So we can just make this one-to-one correlation

01:45:27.400 | between memory of your body's trajectory through the world

01:45:30.920 | over some period of time,

01:45:32.000 | a memory of it, and the ability to recall that memory

01:45:34.160 | is what you would call conscious.

01:45:35.800 | I was conscious of that, it's a self-awareness.

01:45:38.840 | And any system that can recall,

01:45:41.240 | memorize what it's done recently,

01:45:43.440 | and bring that back and invoke it again,

01:45:46.280 | would say, "Yeah, I'm aware.

01:45:48.080 | "I remember what I did, all right, I got it."

01:45:51.240 | That's an easy one,

01:45:52.320 | although some people think that's a hard one.

01:45:54.360 | The more challenging part of consciousness

01:45:57.280 | is one that's sometimes used, going by the word, equalia,

01:46:00.320 | which is, why does an object seem red?

01:46:04.760 | Or what is pain?

01:46:06.760 | And why does pain feel like something?

01:46:08.640 | Why do I feel redness?

01:46:10.280 | Or why do I feel pain-ness?

01:46:11.680 | And then I could say, well, why does sight

01:46:14.200 | seem different than hearing?

01:46:15.560 | That's the same problem, it's really,

01:46:17.480 | these are all just neurons, and so how is it that,

01:46:20.240 | why does looking at you feel different than hearing you?

01:46:24.080 | It feels different, but this is neurons in my head,

01:46:26.040 | they're all doing the same thing.

01:46:27.800 | So that's an interesting question.

01:46:29.760 | The best treatise I've read about this

01:46:31.480 | is by a guy named O'Regan,

01:46:33.560 | he wrote a book called, "Why Red Doesn't Sound Like a Bell."

01:46:37.480 | It's a little, it's not a trade book, easy to read,

01:46:42.120 | and it's an interesting question.

01:46:46.000 | Take something like color.

01:46:47.840 | Color really doesn't exist in the world.

01:46:49.320 | It's not a property of the world.

01:46:51.120 | Property of the world that exists is light frequency,

01:46:54.200 | and that gets turned into, we have certain cells

01:46:57.440 | in the retina that respond to different frequencies

01:46:59.280 | different than others, and so when they enter the brain,

01:47:01.400 | you just have a bunch of axons

01:47:02.400 | that are firing at different rates,

01:47:04.480 | and from that, we perceive color.

01:47:06.800 | But there is no color in the brain.

01:47:07.920 | I mean, there's no color coming in on those synapses,

01:47:10.800 | it's just a correlation between some axons

01:47:14.320 | and some property of frequency.

01:47:16.360 | And that isn't even color itself.

01:47:18.840 | Frequency doesn't have a color.

01:47:20.160 | It's just what it is.

01:47:22.960 | So then the question is, well,

01:47:24.400 | why does it even appear to have a color at all?

01:47:26.840 | - Just as you're describing it,

01:47:29.080 | there seems to be a connection

01:47:30.280 | to those ideas of reference frames.

01:47:32.560 | I mean, it just feels like consciousness,

01:47:37.040 | having the subject, assigning the feeling of red

01:47:40.960 | to the actual color or to the wavelength

01:47:45.000 | is useful for intelligence.

01:47:47.960 | - Yeah, I think that's a good way of putting it.

01:47:49.640 | It's useful as a predictive mechanism

01:47:51.640 | or useful as a generalization idea.

01:47:53.880 | It's a way of grouping things together

01:47:55.360 | to say it's useful to have a model like this.

01:47:57.560 | - Yes.

01:47:58.920 | - Think about the well-known syndrome

01:48:02.680 | that people who've lost a limb experience

01:48:04.880 | called phantom limbs.

01:48:07.040 | And what they claim is they can have their arm is removed,

01:48:12.040 | but they feel their arm.

01:48:13.360 | They not only feel it, they know it's there.

01:48:16.000 | It's there, I know it's there.

01:48:17.800 | They'll swear to you that it's there.

01:48:19.040 | And then they can feel pain in their arm

01:48:20.360 | and they'll feel pain in their finger.

01:48:21.880 | And if they move their non-existent arm behind their back,

01:48:25.320 | then they feel the pain behind their back.

01:48:27.360 | So this whole idea that your arm exists

01:48:30.160 | is a model of your brain.

01:48:31.400 | It may or may not really exist.

01:48:33.520 | And just like, but it's useful to have a model of something

01:48:38.560 | that sort of correlates to things in the world

01:48:40.400 | so you can make predictions about what would happen

01:48:42.000 | when those things occur.

01:48:43.560 | It's a little bit of a fuzzy,

01:48:44.680 | but I think you're getting right towards the answer there.

01:48:46.520 | It's useful for the model of to express things certain ways

01:48:51.360 | that we can then map them into these reference frames

01:48:53.720 | and make predictions about them.

01:48:55.880 | I need to spend more time on this topic.

01:48:57.760 | It doesn't bother me.

01:48:58.960 | - Do you really need to spend more time?

01:49:00.400 | - Yeah, I--

01:49:01.880 | - It does feel special that we have subjective experience,

01:49:04.780 | but I'm yet to know why.

01:49:07.400 | - I'm just personally curious.

01:49:09.120 | It's not necessary for the work we're doing here.

01:49:11.480 | I don't think I need to solve that problem

01:49:13.160 | to build intelligent machines at all, not at all.

01:49:15.640 | - But there is sort of the silly notion

01:49:17.840 | that you described briefly

01:49:19.520 | that doesn't seem so silly to us humans is,

01:49:23.080 | you know, if you're successful building

01:49:24.400 | intelligent machines, it feels wrong to then turn them off.

01:49:29.400 | Because if you're able to build a lot of them,

01:49:33.240 | it feels wrong to then be able to, you know,

01:49:36.960 | to turn off the--

01:49:38.720 | - Well, why?

01:49:39.560 | But just, let's break that down a bit.

01:49:41.840 | As humans, why do we fear death?

01:49:43.920 | There's two reasons we fear death.

01:49:45.720 | Well, first of all, I'll say when you're dead,

01:49:48.240 | it doesn't matter at all, who cares, you're dead.

01:49:50.360 | So why do we fear death?

01:49:51.800 | We fear death for two reasons.

01:49:53.440 | One is because we are programmed genetically to fear death.

01:49:57.720 | That's a survival and propagating the genes thing.

01:50:01.800 | And we also are programmed to feel sad

01:50:05.080 | when people we know die.

01:50:06.840 | We don't feel sad for someone we don't know dies.

01:50:08.520 | There's people dying right now,

01:50:09.560 | they don't understand, I don't feel bad about them

01:50:11.040 | 'cause I don't know them.

01:50:11.880 | But if I knew them, I'd feel really bad.

01:50:13.400 | So again, these are old brain,

01:50:16.840 | genetically embedded things that we fear death.

01:50:20.480 | Outside of those uncomfortable feelings,

01:50:24.320 | there's nothing else to worry about.

01:50:25.880 | - Well, wait, hold on a second.

01:50:27.400 | Do you know the Denial of Death by Becker?

01:50:30.360 | - No.

01:50:31.200 | - You know, there's a thought that death is,

01:50:34.460 | you know, our whole conception of our world model

01:50:41.280 | kind of assumes immortality.

01:50:43.800 | And that death is this terror that underlies it all.

01:50:47.040 | So like--

01:50:47.880 | - Well, some people's world model, not mine.

01:50:50.400 | - But, okay, so what Becker would say

01:50:52.760 | is that you're just living in an illusion.

01:50:54.520 | You've constructed an illusion for yourself

01:50:56.200 | because it's such a terrible terror,

01:50:59.000 | the fact that--

01:51:00.160 | - What's the illusion?

01:51:01.160 | - The illusion that death doesn't matter.

01:51:02.640 | You're still not coming to grips with--

01:51:04.800 | - The illusion of what?

01:51:05.620 | That death is--

01:51:07.120 | - Going to happen.

01:51:08.680 | - Oh, like it's not gonna happen?

01:51:10.440 | You're actually operating.

01:51:11.920 | You haven't, even though you said you've accepted it,

01:51:14.320 | you haven't really accepted the notion you're gonna die

01:51:16.160 | is what he was saying.

01:51:17.000 | So it sounds like,

01:51:18.000 | (laughing)

01:51:19.720 | it sounds like you disagree with that notion.

01:51:21.480 | - Yeah, yeah, totally.

01:51:22.640 | (laughing)

01:51:23.680 | Like, literally, every night I go to bed, it's like dying.

01:51:28.080 | - Little deaths.

01:51:28.920 | - It's little deaths.

01:51:29.760 | And if I didn't wake up, it wouldn't matter to me.

01:51:33.000 | Only if I knew that was gonna happen would it be bothering me.

01:51:35.240 | If I didn't know it was gonna happen, how would I know?

01:51:37.640 | Then I would worry about my wife.

01:51:39.560 | So imagine I was a loner and I lived in Alaska.

01:51:43.080 | And I lived out there and there was no animals,

01:51:45.480 | nobody knew I existed.

01:51:46.520 | I was just eating these roots all the time.

01:51:48.800 | And nobody knew I was there.

01:51:51.160 | And one day I didn't wake up.

01:51:53.360 | What pain in the world would there exist?

01:51:57.080 | - Well, so most people that think about this problem

01:51:59.840 | would say that you're just deeply enlightened

01:52:02.000 | or are completely delusional.

01:52:04.200 | (laughing)

01:52:05.960 | But I would say that's a very enlightened way

01:52:10.760 | to see the world.

01:52:11.760 | That's the rational one as well.

01:52:14.000 | - Well, I think it's rational, that's right.

01:52:15.840 | - But the fact is we don't,

01:52:17.920 | I mean, we really don't have an understanding

01:52:22.440 | of why the heck it is we're born and why we die

01:52:25.000 | and what happens after we die.

01:52:26.120 | - Well, maybe there isn't a reason, maybe there is.

01:52:27.960 | So I'm interested in those big problems too, right?

01:52:30.200 | You know, you interviewed Max Tegmark,

01:52:32.360 | you know, and there's people like that, right?

01:52:33.680 | I'm interested in those big problems as well.

01:52:35.320 | And in fact, when I was young,

01:52:38.360 | I made a list of the biggest problems I could think of.

01:52:41.320 | First, why does anything exist?

01:52:43.480 | Second, why did we have the laws of physics that we have?

01:52:46.720 | Third, is life inevitable and why is it here?

01:52:50.200 | Fourth, is intelligence inevitable and why is it here?

01:52:53.200 | I stopped there because I figured

01:52:55.080 | if you can make a truly intelligent system,

01:52:57.480 | that would be the quickest way

01:52:59.320 | to answer the first three questions.

01:53:01.120 | (laughing)

01:53:03.320 | I'm serious.

01:53:04.520 | And so I said, my mission, you know, you asked me earlier,

01:53:08.000 | my first mission is to understand the brain,

01:53:09.480 | but I felt that is the shortest way

01:53:10.800 | to get to true machine intelligence.

01:53:12.280 | And I wanna get to true machine intelligence

01:53:13.720 | because even if it doesn't occur in my lifetime,

01:53:15.960 | other people will benefit from it

01:53:17.540 | because I think it'll occur in my lifetime,

01:53:19.260 | but you know, 20 years, you never know.

01:53:21.260 | And, but that will be the quickest way for us to,

01:53:25.920 | you know, we can make super mathematicians,

01:53:27.880 | we can make super space explorers,

01:53:29.560 | we can make super physicists brains that do these things.

01:53:34.280 | And that can run experiments that we can't run,

01:53:37.520 | we don't have the abilities to manipulate things and so on,

01:53:40.400 | but we can build intelligent machines

01:53:41.680 | that do all those things and with the ultimate goal

01:53:44.880 | of finding out the answers to the other questions.

01:53:47.700 | - Let me ask you another depressing and difficult question,

01:53:51.560 | which is once we achieve that goal,

01:53:55.200 | do you, of creating, no, of understanding intelligence,

01:54:01.080 | do you think we would be happier,

01:54:03.020 | more fulfilled as a species?

01:54:04.840 | - The understanding intelligence

01:54:05.800 | or understanding the answers to the big questions?

01:54:07.960 | - Understanding intelligence.

01:54:08.960 | - Oh, totally, totally.

01:54:11.800 | It'd be far more fun place to live.

01:54:14.000 | - You think so?

01:54:14.840 | - Oh yeah, why not?

01:54:15.720 | I mean, you know, just put aside this, you know,

01:54:18.120 | Terminator nonsense and just think about,

01:54:22.800 | you can think about, we can talk about the risk of AI

01:54:25.760 | if you want.

01:54:26.600 | - I'd love to, so let's talk about the Terminator.

01:54:28.280 | - But I think the world would be far better knowing things.

01:54:30.680 | We're always better than know things.

01:54:32.120 | Do you think it's better, is it a better place to live in

01:54:35.120 | that I know that our planet is one of many

01:54:37.480 | in the solar system and the solar system

01:54:38.800 | is one of many in the galaxies?

01:54:40.000 | I think it's a more, I dread, I sometimes think like,

01:54:43.400 | God, what it would be like to live 300 years ago?

01:54:45.360 | I'd be looking up at the sky, I can't understand anything.

01:54:47.440 | Oh my God, I'd be like going to bed every night going,

01:54:49.240 | what's going on here?

01:54:50.160 | - Well, I mean, in some sense I agree with you,

01:54:52.480 | but I'm not exactly sure.

01:54:54.760 | So I'm also a scientist, so I share your views,

01:54:57.220 | but I'm not, we're like rolling down the hill together.

01:55:01.720 | - What's down the hill?

01:55:03.440 | I feel like we're climbing a hill.

01:55:05.280 | - Whatever--

01:55:06.120 | - We're getting closer to enlightenment

01:55:07.640 | and you're getting down the hill.

01:55:10.200 | - We're climbing, we're getting pulled up a hill

01:55:12.240 | by our curiosity.

01:55:13.840 | - Our poliocity is, we're pulling ourselves up the hill

01:55:16.120 | by our curiosity.

01:55:16.960 | - Yeah, assistive is doing the same thing with the rock.

01:55:19.160 | - Yeah, yeah, yeah.

01:55:20.840 | - But okay, our happiness aside,

01:55:23.240 | do you have concerns about, you know,

01:55:25.920 | you talk about Sam Harris, Elon Musk,

01:55:29.040 | of existential threats of intelligence systems?

01:55:31.920 | - No, I'm not worried about existential threats at all.

01:55:33.760 | There are some things we really do need to worry about.

01:55:36.380 | Even today's AI, we have things we have to worry about.

01:55:38.400 | We have to worry about privacy

01:55:39.520 | and about how it impacts false beliefs in the world.

01:55:42.740 | And we have real problems that,

01:55:45.960 | and things to worry about with today's AI.

01:55:48.240 | And that will continue as we create

01:55:50.160 | more intelligent systems.

01:55:51.440 | There's no question, you know, the whole issue about,

01:55:53.840 | you know, making intelligent armament and weapons

01:55:57.060 | is something that really we have to think about

01:55:58.840 | carefully.

01:55:59.920 | I don't think of those as existential threats.

01:56:01.880 | I think those are the kind of threats we always face,

01:56:04.320 | and we'll have to face them here

01:56:05.840 | and we'll have to deal with them.

01:56:08.540 | We could talk about what people think

01:56:12.040 | are the existential threats,

01:56:13.880 | but when I hear people talking about them,

01:56:16.220 | they all sound hollow to me.

01:56:17.760 | They're based on ideas,

01:56:19.140 | they're based on people who really have no idea

01:56:20.880 | what intelligence is.

01:56:22.160 | And if they knew what intelligence was,

01:56:24.920 | they wouldn't say those things.

01:56:26.640 | So those are not experts in the field.

01:56:28.760 | - No.

01:56:29.600 | - So there's two, right?

01:56:32.040 | So one is like super intelligence.

01:56:33.720 | So a system that becomes far, far superior

01:56:37.680 | in reasoning ability than us humans.

01:56:42.680 | - How is that an existential threat?

01:56:44.960 | - Then, so there's a lot of ways in which it could be.

01:56:49.120 | One way is us humans are actually irrational, inefficient,

01:56:54.000 | and get in the way of,

01:56:57.320 | of not happiness, but whatever the objective function is

01:57:02.120 | of maximizing that objective function.

01:57:04.080 | - Yeah, yeah.

01:57:04.920 | - And super intelligent.

01:57:05.760 | - The paperclip problem and things like that.

01:57:06.580 | - But so the paperclip problem,

01:57:07.840 | but with a super intelligent.

01:57:09.440 | - Yeah, yeah, yeah, yeah.

01:57:10.480 | So we already faced this threat in some sense.

01:57:14.200 | They're called bacteria.

01:57:17.320 | These are organisms in the world

01:57:18.960 | that would like to turn everything into bacteria.

01:57:21.320 | And they're constantly morphing,

01:57:23.040 | they're constantly changing to evade our protections.

01:57:26.360 | And in the past, they have killed huge swaths

01:57:30.680 | of populations of humans on this planet.

01:57:33.400 | So if you wanna worry about something

01:57:34.560 | that's gonna multiply endlessly, we have it.

01:57:38.360 | And I'm far more worried in that regard.

01:57:40.600 | I'm far more worried that some scientists

01:57:42.640 | in a laboratory will create a super virus

01:57:44.580 | or a super bacteria that we cannot control.

01:57:47.120 | That is a more of an existential threat.

01:57:49.640 | Putting an intelligence thing on top of it

01:57:52.160 | actually seems to make it less existential to me.

01:57:54.200 | It's like, it limits its power.

01:57:56.600 | It limits where it can go.

01:57:57.680 | It limits the number of things it can do in many ways.

01:57:59.760 | A bacteria is something you can't even see.

01:58:02.480 | So that's only one of those problems.

01:58:04.200 | - Yes, exactly.

01:58:05.040 | So the other one, just in your intuition about intelligence,

01:58:09.520 | when you think about the intelligence of us humans,

01:58:12.440 | do you think of that as something,

01:58:14.960 | if you look at intelligence on a spectrum

01:58:16.920 | from zero to us humans,

01:58:18.880 | do you think you can scale that

01:58:21.000 | to something far superior?

01:58:22.840 | - Yeah. - All the mechanisms

01:58:23.840 | we've been talking about.

01:58:24.680 | - I wanna make another point here, Lex, before I get there.

01:58:26.760 | - Sure.

01:58:28.320 | - Intelligence is the neocortex.

01:58:30.880 | It is not the entire brain.

01:58:32.340 | The goal is not to make a human.

01:58:36.160 | The goal is not to make an emotional system.

01:58:38.360 | The goal is not to make a system

01:58:39.520 | that wants to have sex and reproduce.

01:58:41.400 | Why would I build that?

01:58:42.840 | If I wanna have a system that wants to reproduce

01:58:44.520 | and have sex, make bacteria, make computer viruses.

01:58:47.240 | Those are bad things, don't do that.

01:58:49.760 | Those are really bad, don't do those things.

01:58:52.340 | Regulate those.

01:58:53.520 | But if I just say I wanna have an intelligence system,

01:58:56.080 | why does it have to have any of the human-like emotions?

01:58:58.520 | Why does it even care if it lives?

01:59:00.380 | Why does it even care if it has food?

01:59:02.520 | It doesn't care about those things.

01:59:03.800 | It's just in a trance thinking about mathematics

01:59:07.620 | or it's out there just trying to build

01:59:09.080 | the space fort on Mars.

01:59:12.260 | That's a choice we make.

01:59:15.280 | Don't make human-like things.

01:59:17.120 | Don't make replicating things.

01:59:18.440 | Don't make things that have emotions.

01:59:19.800 | Just stick to the neocortex.

01:59:20.920 | - So that's a view actually that I share

01:59:23.040 | but not everybody shares in the sense that

01:59:25.340 | you have faith and optimism about us

01:59:27.920 | as engineers of systems, humans as builders of systems

01:59:31.320 | to not put in stupid things.

01:59:34.720 | - So this is why I mentioned the bacteria one.

01:59:37.560 | Because you might say, well, some person's gonna do that.

01:59:40.680 | Well, some person today could create a bacteria

01:59:42.840 | that's resistant to all the known antibacterial agents.

01:59:46.840 | So we already have that threat.

01:59:49.120 | We already know this is going on.

01:59:51.200 | It's not a new threat.

01:59:52.680 | So just accept that and then we have to deal with it.

01:59:56.160 | Yeah, so my point has nothing to do with intelligence.

01:59:59.960 | Intelligence is a separate component

02:00:01.840 | that you might apply to a system

02:00:03.480 | that wants to reproduce and do stupid things.

02:00:05.960 | Let's not do that.

02:00:07.160 | - Yeah, in fact, it is a mystery

02:00:08.280 | why people haven't done that yet.

02:00:10.440 | My dad is a physicist, believes that the reason,

02:00:14.800 | for example, nuclear weapons haven't proliferated

02:00:18.080 | amongst evil people.

02:00:19.040 | So one belief that I share is that

02:00:21.680 | there's not that many evil people in the world

02:00:24.520 | that would use, whether it's bacteria or nuclear weapons,

02:00:31.280 | or maybe the future AI systems, to do bad.

02:00:35.040 | So the fraction is small.

02:00:36.200 | And the second is that it's actually really hard,

02:00:38.400 | technically.

02:00:39.880 | So the intersection between evil and competent is small

02:00:44.240 | in terms of--

02:00:45.080 | - And by the way, to really annihilate humanity,

02:00:46.960 | you'd have to have sort of the nuclear winter phenomenon,

02:00:50.760 | which is not one person shooting or even 10 bombs.

02:00:54.040 | You'd have to have some automated system

02:00:56.400 | that detonates a million bombs,

02:00:58.480 | or whatever many thousands we have.

02:01:00.400 | - So it's extreme evil combined with extreme competence.

02:01:03.040 | - And just by building some stupid system

02:01:05.040 | that would automatically, you know,

02:01:06.400 | Dr. Strangelove type of thing,

02:01:08.040 | detonate all of it.

02:01:10.400 | I mean, look, we could have some nuclear bomb go off

02:01:13.080 | in some major city in the world.

02:01:14.480 | I think that's actually quite likely, even in my lifetime.

02:01:17.080 | I don't think that's an unlikely thing.

02:01:18.440 | And it'll be a tragedy.

02:01:19.600 | But it won't be an existential threat.

02:01:23.120 | And it's the same as, you know, the virus of 1917,

02:01:26.520 | or whatever it was, you know, the influenza.

02:01:28.920 | These bad things can happen, and the plague, and so on.

02:01:33.800 | We can't always prevent them.

02:01:35.320 | We always try, but we can't.

02:01:37.000 | But they're not existential threats

02:01:38.240 | until we combine all those crazy things together

02:01:40.360 | in one form.

02:01:41.200 | - So on the spectrum of intelligence from zero to human,

02:01:45.440 | do you have a sense of whether it's possible

02:01:47.960 | to create several orders of magnitude,

02:01:51.560 | or at least double that of human intelligence?

02:01:54.680 | Talking about neuro-cortex--

02:01:55.920 | - I think it's the wrong thing to say,

02:01:57.720 | double the intelligence.

02:01:59.000 | Break it down into different components.

02:02:01.600 | Can I make something that's a million times faster

02:02:03.680 | than a human brain?

02:02:04.520 | Yes, I can do that.

02:02:06.280 | Could I make something that has a lot more storage

02:02:10.280 | than a human brain?

02:02:11.120 | Yes, I can do that.

02:02:11.960 | More copies to come.

02:02:13.600 | Can I make something that attaches to different sensors

02:02:15.640 | than a human brain?

02:02:16.480 | Yes, I can do that.

02:02:17.320 | Could I make something that's distributed?

02:02:19.320 | So, we talked earlier about the parts

02:02:21.720 | in the neocortex voting.

02:02:23.160 | They don't have to be co-located.

02:02:24.280 | Like, you know, they could be all around the place.

02:02:25.720 | I could do that, too.

02:02:26.760 | Those are the levers I have, but is it more intelligent?

02:02:32.440 | Well, it depends what I train it on.

02:02:33.800 | What is it doing?

02:02:35.280 | - Well, so here's the thing.

02:02:36.720 | So let's say larger neocortex,

02:02:39.440 | and or whatever size that allows

02:02:43.080 | for higher and higher hierarchies to form,

02:02:47.920 | we're talking about reference frames and context.

02:02:50.160 | - Could I have something that's a super physicist

02:02:51.920 | or a super mathematician?

02:02:52.920 | Yes.

02:02:53.760 | - And the question is, once you have a super physicist,

02:02:56.680 | will they be able to understand something?

02:02:58.780 | Do you have a sense that it will be orders of magnitude,

02:03:02.200 | like us compared to ants?

02:03:03.040 | - Could we ever understand it?

02:03:04.560 | - Yeah.

02:03:06.080 | - Most people cannot understand

02:03:09.440 | general relativity.

02:03:11.920 | It's a really hard thing to get.

02:03:13.280 | I mean, you know, you can paint it in a fuzzy picture,

02:03:15.800 | stretchy space, you know.

02:03:17.560 | But the field equations to do that,

02:03:19.920 | and the deep intuitions are really, really hard.

02:03:23.080 | And I've tried, I'm unable to do it.

02:03:26.240 | It's easy to get special relativity,

02:03:28.800 | but general relativity, man, that's too much.

02:03:31.000 | And so we already live with this to some extent.

02:03:34.960 | The vast majority of people can't understand,

02:03:36.880 | actually, what the vast majority of other people

02:03:38.640 | actually know.

02:03:39.480 | We're just, either we don't have the effort to,

02:03:41.960 | or we can't, or we don't have time,

02:03:43.280 | or just not smart enough, whatever.

02:03:45.400 | So, but we have ways of communicating.

02:03:48.560 | Einstein has spoken in a way that I can understand.

02:03:51.620 | He's given me analogies that are useful.

02:03:54.600 | I can use those analogies for my own work

02:03:56.880 | and think about, you know, concepts that are similar.

02:03:59.900 | It's not stupid.

02:04:02.200 | It's not like he's existed in some other plane

02:04:04.040 | and there's no connection to my plane in the world here.

02:04:06.680 | So that will occur.

02:04:07.840 | It already has occurred.

02:04:09.280 | That's what my point of this story is.

02:04:10.760 | It already has occurred.

02:04:11.720 | We live with it every day.

02:04:13.160 | One could argue that with,

02:04:15.720 | when we create machine intelligence

02:04:17.040 | that think a million times faster than us,

02:04:18.720 | that it'll be so far, we can't make the connections.

02:04:20.920 | But, you know, at the moment,

02:04:22.560 | everything that seems really, really hard

02:04:25.660 | to figure out in the world,

02:04:26.700 | when you actually figure it out, it's not that hard.

02:04:29.000 | You know, if we can,

02:04:29.840 | almost everyone can understand the multiverses.

02:04:32.160 | Almost everyone can understand quantum physics.

02:04:34.040 | Almost everyone can understand these basic things,

02:04:36.120 | even though hardly any people could figure those things out.

02:04:39.000 | - Yeah, but really understand.

02:04:41.320 | - But you don't need to really,

02:04:42.360 | only a few people really need to understand.

02:04:43.840 | - You need to only understand the projections,

02:04:47.920 | the sprinkles of the useful insights from that.

02:04:49.760 | - That was my example of Einstein, right?

02:04:51.800 | His general theory of relativity is one thing

02:04:53.820 | that very, very, very few people can get.

02:04:56.240 | And what if we just said those other few people

02:04:58.280 | are also artificial intelligences?

02:05:00.600 | How bad is that?

02:05:01.440 | - In some sense, they are, right?

02:05:02.760 | - Yeah, right.

02:05:03.600 | They say already.

02:05:04.640 | Einstein wasn't a very normal person.

02:05:06.320 | He had a lot of weird quirks.

02:05:07.600 | And so did the other people who worked with him.

02:05:09.480 | So, you know, maybe they already were

02:05:11.120 | sort of this astral plane of intelligence

02:05:13.440 | that we live with it already.

02:05:15.280 | It's not a problem.

02:05:17.040 | It's still useful.

02:05:18.440 | - So do you think we are the only intelligent life

02:05:23.000 | out there in the universe?

02:05:24.920 | - I would say that intelligent life has

02:05:28.320 | and will exist elsewhere in the universe.

02:05:29.780 | I'll say that.

02:05:30.620 | There is a question about contemporaneous intelligence life,

02:05:34.040 | which is hard to even answer when we think about relativity

02:05:36.760 | and the nature of space time.

02:05:38.620 | Can't say what exactly is this time

02:05:41.080 | someplace else in the world.

02:05:43.120 | But I think it's, you know,

02:05:44.560 | I do worry a lot about the filter idea,

02:05:48.400 | which is that perhaps intelligent species

02:05:52.220 | don't last very long.

02:05:54.000 | And so we haven't been around very long.

02:05:55.680 | And as a technological species,

02:05:57.160 | we've been around for almost nothing, right?

02:05:59.280 | You know, what, 200 years, something like that.

02:06:02.680 | And we don't have any data, a good data point

02:06:04.920 | on whether it's likely that we'll survive or not.

02:06:07.420 | So do I think that there have been intelligent life

02:06:10.980 | elsewhere in the universe?

02:06:11.820 | Almost certain, of course.

02:06:13.440 | In the past, in the future, yes.

02:06:15.280 | Does it survive for a long time?

02:06:17.860 | I don't know.

02:06:18.840 | This is another reason I'm excited about our work,

02:06:21.120 | is our work meaning the general world of AI.

02:06:24.200 | I think we can build intelligent machines that outlast us.

02:06:32.080 | You know, they don't have to be tied to earth.

02:06:34.080 | They don't have to, you know,

02:06:35.840 | I'm not saying they're recreating, you know, aliens.

02:06:39.080 | I'm just saying, if I asked myself,

02:06:41.960 | and this might be a good point to end on here.

02:06:44.320 | If I asked myself, you know,

02:06:45.140 | what's special about our species?

02:06:47.280 | We're not particularly interesting physically.

02:06:49.080 | We're not, we don't fly.

02:06:50.660 | We're not good swimmers.

02:06:51.500 | We're not very fast.

02:06:52.440 | We're not very strong, you know.

02:06:54.000 | It's our brain.

02:06:54.840 | That's the only thing.

02:06:55.660 | And we are the only species on this planet

02:06:57.460 | that's built the model of the world

02:06:58.800 | that extends beyond what we can actually sense.

02:07:01.200 | We're the only people who know about the far side of the moon

02:07:03.920 | and the other universes,

02:07:05.160 | and I mean, other galaxies and other stars,

02:07:07.280 | and about what happens in the atom.

02:07:09.520 | There's no, that knowledge doesn't exist anywhere else.

02:07:12.440 | It's only in our heads.

02:07:13.800 | Cats don't do it.

02:07:14.640 | Dogs don't do it.

02:07:15.460 | Monkeys don't do it.

02:07:16.300 | It's just not.

02:07:17.140 | And that is what we've created that's unique.

02:07:18.320 | Not our genes, it's knowledge.

02:07:20.400 | And if I asked me, what is the legacy of humanity?

02:07:23.160 | What should our legacy be?

02:07:25.080 | It should be knowledge.

02:07:25.920 | We should preserve our knowledge

02:07:27.540 | in a way that it can exist beyond us.

02:07:30.060 | And I think the best way of doing that,

02:07:32.000 | in fact, you have to do it,

02:07:33.040 | is it has to go along with intelligent machines

02:07:34.840 | that understand that knowledge.

02:07:36.400 | It's a very broad idea,

02:07:39.880 | but we should be thinking,

02:07:41.880 | I call it estate planning for humanity.

02:07:43.760 | We should be thinking about what we wanna leave behind

02:07:46.520 | when as a species we're no longer here.

02:07:49.280 | And that'll happen sometime.

02:07:51.040 | Sooner or later, it's gonna happen.

02:07:52.440 | - And understanding intelligence and creating intelligence

02:07:56.040 | gives us a better chance to prolong--

02:07:58.360 | - It does give us a better chance to prolong life, yes.

02:08:01.100 | It gives us a chance to live on other planets.

02:08:03.200 | But even beyond that,

02:08:04.980 | I mean, our solar system will disappear one day,

02:08:07.700 | just given enough time.

02:08:08.700 | So I don't know, I doubt we'll ever be able to travel

02:08:11.780 | to other things.

02:08:14.020 | But we could tell the stars,

02:08:15.500 | but we could send intelligent machines to do that.

02:08:17.780 | - So you have an optimistic, a hopeful view

02:08:21.820 | of our knowledge of the echoes of human civilization

02:08:26.100 | living through the intelligence systems we create?

02:08:29.280 | - Oh, totally.

02:08:30.120 | Well, I think the intelligence systems are great,

02:08:31.440 | or in some sense, the vessel for bringing them beyond Earth,

02:08:35.120 | or making them last beyond humans themselves.

02:08:39.200 | So--

02:08:40.020 | - How do you feel about that?

02:08:41.280 | That they won't be human, quote unquote.

02:08:43.720 | - Okay, it's not, but human, what is human?

02:08:46.160 | Our species are changing all the time.

02:08:48.680 | Human today is not the same as human just 50 years ago.

02:08:51.720 | What is human?

02:08:53.480 | Do we care about our genetics?

02:08:54.560 | Why is that important?

02:08:56.180 | As I point out, our genetics are no more interesting

02:08:58.340 | than a bacterium's genetics,

02:08:59.500 | no more interesting than a monkey's genetics.

02:09:01.780 | What we have, what's unique and what's valuable

02:09:04.620 | is our knowledge, what we've learned about the world.

02:09:07.460 | And that is the rare thing,

02:09:09.700 | that's the thing we want to preserve.

02:09:11.500 | It's, who cares about our genes?

02:09:13.620 | (laughing)

02:09:15.500 | - It's the knowledge.

02:09:16.340 | - It's the knowledge.

02:09:17.160 | - That's a really good place to end.

02:09:19.100 | Thank you so much for talking to me.

02:09:20.180 | - Oh, it was fun.

02:09:21.220 | (laughing)

02:09:23.460 | (upbeat music)

02:09:26.040 | (upbeat music)

02:09:28.620 | (upbeat music)

02:09:31.200 | (upbeat music)

02:09:33.780 | (upbeat music)

02:09:36.360 | (upbeat music)

02:09:38.940 | (Session concluded at 4pm)

Jeff Hawkins: Thousand Brains Theory of Intelligence | Lex Fridman Podcast #25

Chapters