Gary Marcus: Nature vs Nurture is a False Dichotomy

00:00:00.000 | You've talked about this, you've written about it, you've thought about it, nature versus

00:00:04.880 | nurture.

00:00:07.280 | So what innate knowledge do you think we're born with and what do we learn along the way

00:00:13.220 | in those early months and years?

00:00:16.200 | - Can I just say how much I like that question?

00:00:19.760 | You phrased it just right and almost nobody ever does, which is what is the innate knowledge

00:00:24.960 | and what's learned along the way?

00:00:27.240 | So many people dichotomize it and they think it's nature versus nurture when it is obviously

00:00:32.720 | has to be nature and nurture, they have to work together.

00:00:36.480 | You can't learn the stuff along the way unless you have some innate stuff.

00:00:40.280 | But just because you have the innate stuff doesn't mean you don't learn anything.

00:00:43.960 | And so many people get that wrong, including in the field.

00:00:47.000 | Like people think if I work in machine learning, the learning side, I must not be allowed to

00:00:52.400 | work on the innate side or that will be cheating.

00:00:55.800 | People have said that to me and it's just absurd.

00:00:59.520 | So thank you.

00:01:01.360 | - But you could break that apart more.

00:01:03.200 | I've talked to folks who studied the development of the brain and the growth of the brain in

00:01:08.560 | the first few days, in the first few months in the womb, all of that, is that innate?

00:01:17.400 | So that process of development from a stem cell to the growth of the central nervous

00:01:22.040 | system and so on to the information that's encoded through the long arc of evolution.

00:01:30.200 | So all of that comes into play and it's unclear.

00:01:33.240 | It's not just whether it's a dichotomy or not, it's where most or where the knowledge

00:01:39.160 | is encoded.

00:01:40.160 | So what's your intuition about the innate knowledge, the power of it, what's contained

00:01:47.120 | in it, what can we learn from it?

00:01:49.280 | - One of my earlier books was actually trying to understand the biology of this.

00:01:51.840 | The book was called "The Birth of the Mind."

00:01:53.920 | How is it the genes even build innate knowledge?

00:01:56.960 | And from the perspective of the conversation we're having today, there's actually two questions.

00:02:01.760 | One is what innate knowledge or mechanisms or what have you, people or other animals

00:02:07.400 | might be endowed with.

00:02:08.680 | I always like showing this video of a baby ibex climbing down a mountain.

00:02:12.560 | That baby ibex, a few hours after its birth, knows how to climb down a mountain.

00:02:16.320 | That means that it knows not consciously something about its own body and physics and 3D geometry

00:02:22.960 | and all of this kind of stuff.

00:02:25.440 | So there's one question about what does biology give its creatures and what is evolved in

00:02:30.040 | our brains?

00:02:31.200 | How is that represented in our brains?

00:02:32.800 | The question I thought about in the book "The Birth of the Mind."

00:02:35.560 | And then there's a question of what AI should have.

00:02:37.200 | And they don't have to be the same.

00:02:39.720 | But I would say that it's a pretty interesting set of things that we are equipped with that

00:02:46.560 | allows us to do a lot of interesting things.

00:02:48.360 | So I would argue or guess based on my reading of the developmental psychology literature,

00:02:53.080 | which I've also participated in, that children are born with a notion of space, time, other

00:03:00.680 | agents, places, and also this kind of mental algebra that I was describing before.

00:03:08.120 | No certain causation if I didn't just say that.

00:03:10.960 | So at least those kinds of things.

00:03:13.480 | They're like frameworks for learning the other things.

00:03:16.560 | - Are they disjoint in your view or is it just somehow all connected?

00:03:20.720 | You've talked a lot about language.

00:03:23.080 | Is it all kind of connected in some mesh that's language-like of understanding concepts altogether?

00:03:30.400 | - I don't think we know for people how they're represented and machines just don't really

00:03:34.480 | do this yet.

00:03:35.480 | So I think it's an interesting open question both for science and for engineering.

00:03:41.480 | Some of it has to be at least interrelated in the way that the interfaces of a software

00:03:47.600 | package have to be able to talk to one another.

00:03:50.040 | So the systems that represent space and time can't be totally disjoint because a lot of

00:03:57.120 | the things that we reason about are the relations between space and time and cause.

00:04:00.800 | So I put this on and I have expectations about what's going to happen with the bottle cap

00:04:05.640 | on top of the bottle and those span space and time.

00:04:10.840 | If the cap is over here, I get a different outcome.

00:04:13.800 | If the timing is different, if I put this here, after I move that, then I get a different

00:04:18.520 | outcome that relates to causality.

00:04:20.840 | So obviously these mechanisms, whatever they are, can certainly communicate with each other.

00:04:27.920 | - So I think evolution had a significant role to play in the development of this whole collage,

00:04:33.800 | right?

00:04:34.960 | How efficient do you think is evolution?

00:04:36.600 | - Oh, it's terribly inefficient except that--

00:04:39.320 | - Okay, well, can we do better?

00:04:41.040 | - Well, let's come to that in a second.

00:04:43.640 | It's inefficient except that once it gets a good idea, it runs with it.

00:04:48.760 | So it took, I guess, a billion years, roughly a billion years to evolve to a vertebrate

00:05:00.640 | brain plan.

00:05:01.640 | And once that vertebrate brain plan evolved, it spread everywhere.

00:05:06.320 | So fish have it and dogs have it and we have it.

00:05:09.480 | We have adaptations of it and specializations of it.

00:05:11.960 | But and the same thing with a primate brain plan.

00:05:14.960 | So monkeys have it and apes have it and we have it.

00:05:18.960 | So there are additional innovations like color vision and those spread really rapidly.

00:05:23.720 | So it takes evolution a long time to get a good idea, but being anthropomorphic and not

00:05:29.440 | literal here.

00:05:31.160 | But once it has that idea, so to speak, which caches out into one set of genes or in the

00:05:35.480 | genome, those genes spread very rapidly.

00:05:38.460 | And they're like subroutines or libraries, I guess the word people might use nowadays

00:05:42.240 | or be more familiar with.

00:05:43.520 | They're libraries that get used over and over again.

00:05:46.640 | So once you have the library for building something with multiple digits, you can use

00:05:50.920 | it for a hand, but you can also use it for a foot.

00:05:53.400 | You just kind of reuse the library with slightly different parameters.

00:05:57.320 | Evolution does a lot of that, which means that the speed over time picks up.

00:06:01.320 | So evolution can happen faster because you have bigger and bigger libraries.

00:06:06.300 | And what I think has happened in attempts at evolutionary computation is that people

00:06:13.020 | start with libraries that are very, very minimal, like almost nothing.

00:06:19.480 | And then progress is slow and it's hard for someone to get a good PhD thesis out of it

00:06:24.660 | and they give up.

00:06:26.120 | If we had richer libraries to begin with, if you were evolving from systems that had

00:06:30.640 | an originate structure to begin with, then things might speed up.

00:06:34.660 | - More and more PhD students, if the evolutionary process is indeed in a meta way, runs away

00:06:41.240 | with good ideas, you need to have a lot of ideas, pool of ideas in order for it to discover

00:06:46.220 | one that you can run away with.

00:06:48.180 | And PhD students representing individual ideas as well.

00:06:50.940 | - Yeah, I mean, you could throw a billion PhD students at it.

00:06:54.020 | - Yeah, the monkeys are typewriters with Shakespeare, yeah.

00:06:57.180 | - Well, I mean, those aren't cumulative, right?

00:06:59.940 | That's just random.

00:07:01.420 | And part of the point that I'm making is that evolution is cumulative.

00:07:04.640 | So if you have a billion monkeys independently, you don't really get anywhere.

00:07:10.360 | But if you have a billion monkeys, and I think Dawkins made this point originally, or probably

00:07:13.780 | other people, Dawkins made it very nice, either a selfish gene or blind watchmaker.

00:07:19.340 | If there's some sort of fitness function that can drive you towards something, I guess that's

00:07:24.020 | Dawkins point.

00:07:25.020 | And my point, which is a variation on that, is that if the evolution is cumulative, the

00:07:30.140 | related points, then you can start going faster.

00:07:33.460 | - Do you think something like the process of evolution is required to build intelligent

00:07:37.100 | systems?

00:07:38.100 | So if we-- - Not logically.

00:07:39.440 | So all the stuff that evolution did, a good engineer might be able to do.

00:07:44.920 | So for example, evolution made quadrupeds, which distribute the load across a horizontal

00:07:51.540 | surface.

00:07:52.540 | A good engineer could come up with that idea.

00:07:54.020 | I mean, sometimes good engineers come up with ideas by looking at biology.

00:07:57.340 | There's lots of ways to get your ideas.

00:08:00.460 | Probably what I'm suggesting is we should look at biology a lot more.

00:08:03.780 | We should look at the biology of thought and understanding, and the biology by which creatures

00:08:10.160 | intuitively reason about physics or other agents, or how do dogs reason about people?

00:08:15.900 | They're actually pretty good at it.

00:08:17.180 | If we could understand, at my college we joked dognition, if we could understand dognition

00:08:23.100 | well and how it was implemented, that might help us with our AI.

00:08:27.620 | - So do you think it's possible that the kind of timescale that evolution took is the kind

00:08:35.400 | of timescale that will be needed to build intelligent systems, or can we significantly

00:08:39.300 | accelerate that process inside a computer?

00:08:43.180 | - I mean, I think the way that we accelerate that process is we borrow from biology.

00:08:48.540 | Not slavishly, but I think we look at how biology has solved problems, and we say, does

00:08:53.540 | that inspire any engineering solutions here?

00:08:56.740 | Try to mimic biological systems, and then therefore have a shortcut.

00:08:59.780 | - Yeah, I mean, there's a field called biomimicry, and people do that for material science all

00:09:05.220 | the time.

00:09:06.860 | We should be doing the analog of that for AI, and the analog for that for AI is to look

00:09:12.540 | at cognitive science, or the cognitive sciences, which is psychology, maybe neuroscience, linguistics,

00:09:18.260 | and so forth.

00:09:19.420 | Look to those for insight.

00:09:20.540 | - Thank you.

00:09:21.540 | - Thank you.

00:09:26.540 | - Thank you.

00:09:31.540 | - Thank you.

00:09:36.540 | [BLANK_AUDIO]

Gary Marcus: Nature vs Nurture is a False Dichotomy | AI Podcast Clips

Chapters