Stephen Wolfram: ChatGPT and the Nature of Truth, Reality & Computation

00:00:00.000 | I can tell chat GPT, create a piece of code,

00:00:03.200 | and then just run it on my computer.

00:00:05.160 | And I'm like, that sort of personalizes for me

00:00:08.840 | the what could possibly go wrong, so to speak.

00:00:11.480 | - Was that exciting or scary, that possibility?

00:00:14.580 | - It was a little bit scary actually,

00:00:15.880 | because it's kind of like, if you do that,

00:00:18.480 | what is the sandboxing that you should have?

00:00:20.560 | And that's a version of that question for the world.

00:00:24.800 | That is, as soon as you put the AIs in charge of things,

00:00:27.960 | how many constraints should there be on these systems

00:00:31.600 | before you put the AIs in charge of all the weapons

00:00:34.360 | and all these different kinds of systems?

00:00:36.720 | - Well, here's the fun part about sandboxes,

00:00:39.240 | is the AI knows about them.

00:00:41.540 | It has the tools to crack them.

00:00:44.560 | The following is a conversation with Stephen Wolfram,

00:00:49.440 | his fourth time on this podcast.

00:00:51.400 | He's a computer scientist, mathematician,

00:00:53.640 | theoretical physicist,

00:00:55.120 | and the founder of Wolfram Research,

00:00:57.480 | a company behind Mathematica, Wolfram Alpha,

00:01:00.240 | Wolfram Language,

00:01:01.320 | and the Wolfram Physics and Metamathematics projects.

00:01:04.880 | He has been a pioneer

00:01:07.520 | in exploring the computational nature of reality.

00:01:10.080 | And so he's the perfect person to explore with together

00:01:14.320 | the new, quickly evolving landscape

00:01:16.600 | of large language models

00:01:18.280 | as human civilization journeys towards

00:01:20.080 | building super intelligent AGI.

00:01:23.720 | This is the Lex Friedman Podcast.

00:01:25.900 | To support it,

00:01:26.740 | please check out our sponsors in the description.

00:01:28.840 | And now, dear friends, here's Stephen Wolfram.

00:01:32.400 | You announced the integration of Chad GPT

00:01:36.200 | and Wolfram Alpha and Wolfram Language.

00:01:38.760 | So let's talk about that integration.

00:01:40.960 | What are the key differences

00:01:42.820 | from the high philosophical level,

00:01:45.160 | maybe the technical level,

00:01:46.320 | between the capabilities of,

00:01:49.840 | broadly speaking, the two kinds of systems,

00:01:51.620 | large language models,

00:01:53.040 | and this computational,

00:01:54.280 | gigantic computational system infrastructure

00:01:56.520 | that is Wolfram Alpha?

00:01:57.440 | - Yeah, so what does something like Chad GPT do?

00:02:00.640 | It's mostly focused on make language

00:02:04.520 | like the language that humans have made

00:02:07.160 | and put on the web and so on.

00:02:08.880 | So, you know, it's primary,

00:02:11.760 | sort of underlying technical thing is

00:02:14.440 | you've given a prompt,

00:02:15.880 | it's trying to continue that prompt

00:02:17.600 | in a way that's somehow typical

00:02:20.160 | of what it's seen based on a trillion words of text

00:02:23.340 | that humans have written on the web.

00:02:25.120 | And the way it's doing that is

00:02:28.140 | with something which is probably quite similar

00:02:30.700 | to the way we humans do the first stages of that,

00:02:33.920 | using a neural net and so on,

00:02:35.400 | and just saying,

00:02:36.400 | given this piece of text,

00:02:39.760 | let's ripple through the neural net

00:02:41.660 | one word and get one word at a time of output.

00:02:45.340 | And it's kind of a shallow computation

00:02:48.480 | on a large amount of kind of training data

00:02:51.520 | that is what we humans have put on the web.

00:02:54.040 | That's a different thing

00:02:55.540 | from sort of the computational stack

00:02:58.000 | that I spent the last, I don't know,

00:02:59.420 | 40 years or so building,

00:03:00.920 | which has to do with what can you compute many steps,

00:03:04.700 | potentially a very deep computation.

00:03:07.000 | It's not sort of taking the statistics

00:03:09.900 | of what we humans have produced

00:03:12.220 | and trying to continue things based on that statistics.

00:03:15.320 | Instead, it's trying to take kind of the formal structure

00:03:19.520 | that we've created in our civilization,

00:03:21.400 | whether it's from mathematics

00:03:23.180 | or whether it's from kind of systematic knowledge

00:03:25.720 | of all kinds,

00:03:26.880 | and use that to do arbitrarily deep computations

00:03:31.120 | to figure out things that aren't just,

00:03:34.120 | let's match what's already been kind of said on the web,

00:03:37.760 | but let's potentially be able to compute something new

00:03:40.400 | and different that's never been computed before.

00:03:42.680 | So as a practical matter,

00:03:44.640 | you know, what we're,

00:03:47.280 | our goal is to have made

00:03:49.280 | as much as possible of the world computable

00:03:51.700 | in the sense that if there's a question

00:03:53.960 | that in principle is answerable

00:03:55.560 | from some sort of expert knowledge that's been accumulated,

00:03:58.300 | we can compute the answer to that question,

00:04:01.760 | and we can do it in a sort of reliable way

00:04:04.480 | that's the best one can do

00:04:06.360 | given what the expertise

00:04:08.260 | that our civilization has accumulated.

00:04:10.280 | It's a very, it's a much more sort of labor intensive

00:04:14.080 | on the side of kind of being creating

00:04:17.160 | kind of the computational system to do that.

00:04:20.280 | Obviously the, in the kind of the chat GPT world,

00:04:24.480 | it's like take things which were produced

00:04:27.200 | for quite other purposes,

00:04:28.360 | namely all the things we've written out on the web and so on

00:04:31.960 | and sort of forage from that things

00:04:34.960 | which are like what's been written on the web.

00:04:37.520 | So I think, you know, as a practical point of view,

00:04:39.640 | I view sort of the chat GPT thing as being wide and shallow

00:04:44.600 | and what we're trying to do

00:04:46.140 | with sort of building out computation

00:04:48.880 | as being this sort of deep, also broad,

00:04:51.800 | but most importantly kind of deep type of thing.

00:04:56.240 | I think another way to think about this is

00:04:58.320 | you go back in human history, you know, I don't know,

00:05:01.160 | thousand years or something,

00:05:02.760 | and you say, what can the typical person,

00:05:05.480 | what's the typical person going to figure out?

00:05:08.040 | Well, the answer is there's certain kinds of things

00:05:09.560 | that we humans can quickly figure out.

00:05:11.600 | That's sort of what our, you know,

00:05:14.680 | at our neural architecture

00:05:16.360 | and the kinds of things we learn in our lives let us do.

00:05:19.880 | But then there's this whole layer of kind of formalization

00:05:23.060 | that got developed in which is, you know,

00:05:25.520 | the kind of whole sort of story of intellectual history

00:05:29.080 | and the whole kind of depth of learning,

00:05:31.380 | that formalization turned into things like logic,

00:05:33.660 | mathematics, science, and so on.

00:05:36.600 | And that's the kind of thing that allows one

00:05:38.640 | to kind of build these towers of,

00:05:40.560 | of sort of towers of things you work out.

00:05:45.200 | It's not just, I can immediately figure this out.

00:05:47.640 | It's no, I can use this kind of formalism

00:05:50.400 | to go step by step and work out something

00:05:52.980 | which was not immediately obvious to me.

00:05:55.320 | And that's kind of the story

00:05:56.400 | of what we're trying to do computationally

00:05:58.200 | is to be able to build those kind of tall towers

00:06:01.560 | of what implies what implies what and so on.

00:06:04.800 | And as opposed to kind of the,

00:06:07.240 | yes, I can immediately figure it out.

00:06:08.580 | It's just like what I saw somewhere else

00:06:11.040 | in something that I heard or remembered

00:06:13.080 | or something like this.

00:06:14.120 | - What can you say about the kind of formal structure,

00:06:17.640 | the kind of formal foundation you can build

00:06:20.280 | such a formal structure on,

00:06:22.380 | about the kinds of things you would start on

00:06:24.720 | in order to build this kind of

00:06:27.240 | deep computable knowledge trees?

00:06:30.080 | - So the question is sort of,

00:06:31.040 | how do you think about computation?

00:06:33.920 | And there's a couple of points here.

00:06:36.200 | One is what computation intrinsically is like.

00:06:40.280 | And the other is what aspects of computation

00:06:43.560 | we humans with our minds

00:06:45.220 | and with the kinds of things we've learned

00:06:47.460 | can sort of relate to in that computational universe.

00:06:51.260 | So if we start on the kind of what can computation be like,

00:06:55.020 | it's something I've spent some big chunk of my life studying

00:06:58.600 | is imagine that you're,

00:07:00.460 | we usually write programs where we kind of know

00:07:02.980 | what we want the program to do.

00:07:04.820 | And we carefully write many lines of code

00:07:07.300 | and we hope that the program does what we intended it to do.

00:07:10.620 | But the thing I've been interested in is

00:07:13.020 | if you just look at the kind of natural science of programs,

00:07:15.980 | so you just say, I'm gonna make this program,

00:07:17.740 | it's a really tiny program.

00:07:19.300 | Maybe I even pick the pieces of the program at random,

00:07:21.540 | but it's really tiny.

00:07:22.360 | And by really tiny, I mean,

00:07:24.100 | less than a line of code type thing.

00:07:26.380 | You say, what does this program do?

00:07:28.420 | And you run it.

00:07:29.500 | And big discovery that I made in the early 80s

00:07:32.280 | is that even extremely simple programs,

00:07:35.440 | when you run them can do really complicated things.

00:07:38.140 | Really surprised me.

00:07:38.980 | It took me several years to kind of realize

00:07:41.160 | that that was a thing, so to speak.

00:07:43.500 | But that realization that even very simple programs

00:07:46.860 | can do incredibly complicated things

00:07:48.380 | that we very much don't expect.

00:07:50.720 | That discovery, I mean, I realized that that's very much,

00:07:53.420 | I think how nature works.

00:07:55.380 | That is nature has simple rules,

00:07:57.140 | but yet does all sorts of complicated things

00:07:59.660 | that we might not expect.

00:08:01.240 | You know, a big thing of the last few years

00:08:03.540 | has been understanding that that's how the whole universe

00:08:05.780 | and physics works, but that's a quite separate topic.

00:08:09.380 | But so there's this whole world of programs

00:08:12.380 | and what they do and very rich, sophisticated things

00:08:15.620 | that these programs can do.

00:08:17.180 | But when we look at many of these programs,

00:08:19.140 | we look at them and say, well, that's kind of,

00:08:20.620 | I don't really know what that's doing.

00:08:22.300 | It's not a very human kind of thing.

00:08:24.680 | So on the one hand, we have sort of what's possible

00:08:26.900 | in the computational universe.

00:08:28.580 | On the other hand, we have the kinds of things

00:08:30.180 | that we humans think about, the kinds of things

00:08:32.500 | that are developed in kind of our intellectual history.

00:08:35.300 | And that's really the challenge

00:08:39.260 | to sort of making things computational

00:08:41.520 | is to connect what's computationally possible

00:08:44.060 | out in the computational universe

00:08:46.000 | with the things that we humans

00:08:47.860 | sort of typically think about with our minds.

00:08:50.620 | Now, that's a complicated kind of moving target

00:08:53.840 | because the things that we think about change over time.

00:08:57.320 | We've learned more stuff.

00:08:58.320 | We've invented mathematics.

00:08:59.700 | We've invented various kinds of ideas

00:09:02.560 | and structures and so on.

00:09:03.840 | So it's gradually expanding.

00:09:05.380 | We're kind of gradually colonizing more and more

00:09:08.100 | of this kind of intellectual space of possibilities.

00:09:10.820 | But the real thing, the real challenge is

00:09:13.540 | how do you take what is computationally possible?

00:09:16.800 | How do you take, how do you encapsulate

00:09:18.940 | the kinds of things that we think about

00:09:21.140 | in a way that kind of plugs in

00:09:22.620 | to what's computationally possible?

00:09:24.640 | And actually the big sort of idea there

00:09:28.880 | is this idea of kind of symbolic programming,

00:09:31.420 | symbolic representations of things.

00:09:33.900 | And so the question is,

00:09:35.560 | when you look at sort of everything in the world

00:09:37.660 | and you kind of take some visual scene

00:09:39.580 | or something you're looking at,

00:09:40.980 | and you say, well, how do I turn that into something

00:09:43.740 | that I can kind of stuff into my mind?

00:09:45.900 | You know, there are lots of pixels in my visual scene,

00:09:48.720 | but the things that I remembered from that visual scene are,

00:09:52.020 | you know, there's a chair in this place.

00:09:54.600 | It's a kind of a symbolic representation

00:09:57.440 | of the visual scene.

00:09:58.280 | There are two chairs and a table or something,

00:10:00.240 | rather than there are all these pixels

00:10:01.680 | arranged in all these detailed ways.

00:10:03.540 | And so the question then is how do you take

00:10:05.820 | sort of all the things in the world

00:10:08.280 | and make some kind of representation

00:10:10.380 | that corresponds to the types of ways

00:10:12.860 | that we think about things?

00:10:13.940 | And human language is sort of one form of representation

00:10:18.460 | that we have.

00:10:19.280 | We talk about chairs,

00:10:20.120 | that's a word in human language and so on.

00:10:22.020 | How do we take, but human language is not in and of itself

00:10:26.440 | something that plugs in very well to sort of computation.

00:10:31.360 | It's not something from which you can immediately

00:10:33.120 | compute consequences and so on.

00:10:35.720 | And so you have to kind of find a way to take

00:10:38.920 | sort of the stuff we understand from human language

00:10:42.920 | and make it more precise.

00:10:44.840 | And that's really the story of symbolic programming.

00:10:48.300 | And you know, what that turns into is something

00:10:51.260 | which I didn't know at the time

00:10:52.960 | it was going to work as well as it has.

00:10:54.600 | But back in the 1979 or so,

00:10:56.920 | I was trying to build my first big computer system

00:10:59.360 | and trying to figure out, you know,

00:11:00.820 | how should I represent computations at a high level?

00:11:03.920 | And I kind of invented this idea of using

00:11:07.520 | kind of symbolic expressions, you know,

00:11:09.960 | structured as it's kind of like a function

00:11:12.800 | and a bunch of arguments,

00:11:14.420 | but that function doesn't necessarily evaluate to anything.

00:11:17.960 | It's just a thing that sits there representing a structure.

00:11:22.240 | And so building up that structure,

00:11:24.660 | and it's turned out that structure has been extremely,

00:11:28.460 | it's a good match for the way that we humans,

00:11:31.280 | it seems to be a good match for the way that we humans

00:11:34.280 | kind of conceptualize higher level things.

00:11:36.920 | And it's been for the last, I don't know,

00:11:38.480 | 45 years or something, it's served me remarkably well.

00:11:42.480 | - So building up that structure

00:11:44.200 | using this kind of symbolic representation.

00:11:46.600 | But what can you say about abstractions here?

00:11:49.880 | Because you could just start with your physics project,

00:11:52.520 | you could start at a hypergraph at a very, very low level

00:11:55.440 | and build up everything from there, but you don't.

00:11:58.040 | You take shortcuts. - Right.

00:12:00.160 | - You take the highest level of abstraction,

00:12:03.360 | convert that, the kind of abstraction that's convertible

00:12:07.080 | to something computable using symbolic representation.

00:12:11.120 | And then that's your new foundation

00:12:13.580 | for that little piece of knowledge.

00:12:15.080 | Somehow all of that is integrated.

00:12:17.240 | - Right, so the sort of a very important phenomenon

00:12:20.520 | that is kind of a thing that I've sort of realized

00:12:23.920 | is just, it's one of these things that sort of

00:12:26.200 | in the future of kind of everything

00:12:29.200 | is going to become more and more important

00:12:30.800 | as this phenomenon of computational irreducibility.

00:12:33.640 | And the question is, if you know the rules for something,

00:12:37.200 | you have a program, you're gonna run it,

00:12:39.280 | you might say, I know the rules, great.

00:12:40.920 | I know everything about what's gonna happen.

00:12:43.040 | Well, in principle you do,

00:12:44.640 | because you can just run those rules out

00:12:47.520 | and just see what they do.

00:12:48.400 | You might run them a million steps,

00:12:50.040 | you see what happens, et cetera.

00:12:52.000 | The question is, can you like immediately jump ahead

00:12:55.600 | and say, I know what's gonna happen after a million steps

00:12:58.080 | and the answer is 13 or something.

00:13:00.440 | And one of the very critical things to realize is

00:13:04.440 | if you could reduce that computation,

00:13:07.400 | there is in a sense, no point in doing the computation.

00:13:10.280 | The place where you really get value

00:13:11.840 | out of doing the computation is when

00:13:14.240 | you had to do the computation to find out the answer.

00:13:17.000 | But this phenomenon that you have to do the computation

00:13:19.160 | to find out the answer,

00:13:20.040 | this phenomenon of computational irreducibility

00:13:22.440 | seems to be tremendously important

00:13:24.120 | for thinking about lots of kinds of things.

00:13:26.040 | So one of the things that happens is,

00:13:28.080 | okay, you've got a model of the universe at the low level

00:13:30.960 | in terms of atoms of space and hypergraphs

00:13:33.040 | and rewriting hypergraphs and so on.

00:13:35.120 | And it's happening 10 to the 100 times every second,

00:13:38.120 | let's say, well, you say, great, then we've nailed it.

00:13:41.520 | We know how the universe works.

00:13:43.840 | Well, the problem is the universe can figure out

00:13:46.280 | what it's gonna do.

00:13:47.400 | It does those 10 to the 100 steps.

00:13:50.360 | But for us to work out what it's gonna do,

00:13:52.680 | we have no way to reduce that computation.

00:13:55.240 | The only way to do the computation,

00:13:56.920 | to see the result of the computation is to do it.

00:13:59.880 | And if we're operating within the universe,

00:14:01.960 | we're kind of, there's no opportunity to do that

00:14:04.480 | 'cause the universe is doing it

00:14:06.080 | as fast as the universe can do it.

00:14:07.960 | And that's what's happening.

00:14:09.760 | So what we're trying to do,

00:14:11.400 | and a lot of the story of science

00:14:13.320 | and a lot of other kinds of things

00:14:14.920 | is finding pockets of reducibility.

00:14:17.760 | That is, you could have a situation

00:14:19.720 | where everything in the world

00:14:20.800 | is full of computational irreducibility.

00:14:22.840 | We never know what's gonna happen next.

00:14:24.800 | The only way we can figure out what's gonna happen next

00:14:26.880 | is just let the system run and see what happens.

00:14:29.680 | So in a sense, the story of most kinds of science,

00:14:33.120 | inventions, a lot of kinds of things

00:14:35.760 | is the story of finding these places

00:14:37.720 | where we can locally jump ahead.

00:14:40.000 | And one of the features of computational irreducibility

00:14:42.040 | is there are always pockets of reducibility.

00:14:45.280 | There are always places,

00:14:46.360 | there are always an infinite number of places

00:14:47.920 | where you can jump ahead.

00:14:49.200 | There's no way where you can jump completely ahead,

00:14:52.280 | but there are little patches,

00:14:53.960 | little places where you can jump ahead a bit.

00:14:56.440 | And I think, we can talk about physics project and so on,

00:14:59.320 | but I think the thing we realize is we kind of exist

00:15:03.080 | in a slice of all the possible

00:15:04.840 | computational irreducibility in the universe.

00:15:06.920 | We exist in a slice

00:15:08.600 | where there's a reasonable amount of predictability.

00:15:11.040 | And in a sense, as we try and construct

00:15:13.720 | these kind of higher levels of abstraction,

00:15:17.360 | symbolic representations and so on,

00:15:19.040 | what we're doing is we're finding these lumps

00:15:21.320 | of reducibility that we can kind of attach ourselves to

00:15:25.080 | and about which we can kind of have

00:15:27.200 | fairly simple narrative things to say.

00:15:29.560 | Because in principle, I say,

00:15:31.000 | what's gonna happen in the next few seconds?

00:15:33.920 | Oh, there are these molecules moving around

00:15:35.560 | in the air in this room,

00:15:36.960 | and oh gosh, it's an incredibly complicated story.

00:15:40.320 | And that's a whole computationally irreducible thing,

00:15:42.960 | most of which I don't care about.

00:15:45.080 | And most of it is, well, the air is still gonna be here

00:15:48.680 | and nothing much is going to be different about it.

00:15:51.080 | And that's a kind of reducible fact

00:15:53.760 | about what is ultimately at an underlying level

00:15:56.400 | of computationally irreducible process.

00:15:58.360 | - And life would not be possible

00:16:01.960 | if we didn't have a large number of such reducible pockets.

00:16:06.960 | - Yes.

00:16:07.840 | - And that's amenable to reduction into something symbolic.

00:16:11.360 | - Yes, I think so.

00:16:12.200 | I mean, life in the way that we experience it,

00:16:16.160 | that, I mean, one might,

00:16:20.080 | depending on what we mean by life, so to speak,

00:16:22.780 | the experience that we have

00:16:25.440 | of sort of consistent things happening in the world,

00:16:28.280 | the idea of space, for example, where there's,

00:16:31.280 | we can just say, you're here, you move there.

00:16:34.360 | It's kind of the same thing.

00:16:35.880 | It's still you in that different place,

00:16:38.040 | even though you're made of different atoms of space

00:16:40.040 | and so on.

00:16:40.920 | This idea that there's sort of this level of predictability

00:16:45.920 | of what's going on,

00:16:47.440 | that's us finding a slice of reducibility

00:16:50.720 | in what is underneath

00:16:52.120 | this computationally irreducible kind of system.

00:16:55.240 | And I think that's sort of the thing,

00:16:57.360 | which is actually my favorite discovery

00:17:00.280 | of the last few years,

00:17:01.740 | is the realization that it is sort of the interaction

00:17:05.380 | between the sort of underlying computational irreducibility

00:17:08.800 | and our nature as kind of observers

00:17:12.780 | who sort of have to key into computational reducibility.

00:17:16.280 | That fact leads to the main laws of physics

00:17:20.160 | that we discovered in the 20th century.

00:17:22.400 | So this is, we talk about this in more detail,

00:17:25.080 | but this is, to me, it's kind of our nature as observers,

00:17:30.080 | the fact that we are computationally bounded observers,

00:17:34.300 | we don't get to follow all those little pieces

00:17:36.380 | of computational irreducibility.

00:17:38.080 | To stuff what is out there in the world into our minds

00:17:42.380 | requires that we are looking at things that are reducible,

00:17:45.540 | we are compressing,

00:17:47.020 | kind of we're extracting just some essence,

00:17:49.260 | some kind of symbolic essence

00:17:51.200 | of what's the detail of what's going on in the world.

00:17:54.140 | That together with one other condition

00:17:56.620 | that at first seems sort of trivial, but isn't,

00:17:59.900 | which is that we believe we are persistent in time.

00:18:02.860 | That is, you know--

00:18:05.340 | - So that's the causality.

00:18:07.380 | - Here's the thing, at every moment,

00:18:09.900 | according to our theory,

00:18:11.220 | we're made of different atoms of space.

00:18:13.740 | At every moment, sort of the microscopic detail

00:18:16.620 | of what the universe is made of is being rewritten.

00:18:20.300 | And that's, and in fact, the very fact

00:18:21.940 | that there's coherence between different parts of space

00:18:24.540 | is a consequence of the fact

00:18:25.620 | that there are all these little processes going on

00:18:27.460 | that kind of knit together the structure of space.

00:18:29.160 | It's kind of like if you wanted to have a fluid

00:18:31.100 | with a bunch of molecules in it,

00:18:32.520 | if those molecules weren't interacting,

00:18:34.580 | you wouldn't have this fluid that would pour

00:18:36.420 | and do all these kinds of things.

00:18:37.740 | It would just be sort of a free-floating collection

00:18:39.580 | of molecules.

00:18:40.620 | So similarly it is with space,

00:18:42.460 | that the fact that space is kind of knitted together

00:18:44.780 | is a consequence of all this activity in space.

00:18:47.300 | And the fact that kind of what we consist of

00:18:50.940 | sort of this series of,

00:18:53.740 | you know, we're continually being rewritten.

00:18:55.980 | And the question is, why is it the case

00:18:58.020 | that we think of ourselves

00:18:59.420 | as being the same us through time?

00:19:02.700 | That's kind of a key assumption.

00:19:04.480 | I think it's a key aspect of what we see

00:19:06.900 | as sort of our consciousness, so to speak,

00:19:09.200 | is that we have this kind of

00:19:10.440 | consistent thread of experience.

00:19:12.280 | - Well, isn't that just another limitation of our mind

00:19:17.280 | that we want to reduce reality into some,

00:19:22.320 | that kind of temporal consistency

00:19:25.680 | is just a nice narrative to tell ourselves.

00:19:28.340 | - Well, the fact is, I think it's critical

00:19:30.780 | to the way we humans typically operate

00:19:33.060 | is that we have a single thread of experience.

00:19:35.620 | You know, if you imagine sort of a mind

00:19:39.000 | where you have, you know, maybe that's what's happening

00:19:41.400 | in various kinds of minds

00:19:42.560 | that aren't working the same way other minds work,

00:19:45.280 | is that you're splitting

00:19:46.120 | into multiple threads of experience.

00:19:48.160 | It's also something where, you know,

00:19:50.580 | when you look at, I don't know,

00:19:51.620 | quantum mechanics, for example,

00:19:53.600 | in the insides of quantum mechanics,

00:19:55.880 | it's splitting into many threads of experience.

00:19:58.380 | But in order for us humans to interact with it,

00:20:01.120 | you kind of have to knit

00:20:02.900 | all those different threads together

00:20:04.800 | so that we say, oh yeah, a definite thing happened,

00:20:07.020 | and now the next definite thing happens, and so on.

00:20:09.680 | And I think, you know, sort of inside,

00:20:11.940 | it's sort of interesting to try and imagine

00:20:16.480 | what's it like to have kind of these

00:20:19.760 | fundamentally multiple threads of experience going on.

00:20:22.340 | I mean, right now, different human minds

00:20:25.160 | have different threads of experience.

00:20:26.840 | We just have a bunch of minds

00:20:28.040 | that are interacting with each other,

00:20:29.560 | but we don't have a, you know,

00:20:31.000 | within each mind, there's a single thread.

00:20:33.900 | And that is indeed a simplification.

00:20:36.320 | I think it's a thing, you know,

00:20:37.960 | the general computational system

00:20:39.600 | does not have that simplification.

00:20:41.720 | And it's one of the things, you know,

00:20:44.320 | people often seem to think that, you know,

00:20:46.000 | consciousness is the highest level

00:20:48.120 | of kind of things that can happen in the universe,

00:20:50.160 | so to speak, but I think that's not true.

00:20:51.880 | I think it's actually a specialization

00:20:55.480 | in which, among other things,

00:20:57.080 | you have this idea of a single thread of experience,

00:20:59.160 | which is not a general feature of anything

00:21:01.960 | that could kind of computationally happen in the universe.

00:21:04.080 | - So it's a feature of a computationally limited system

00:21:07.060 | that's only able to observe reducible pockets.

00:21:12.060 | So, I mean, this word observer,

00:21:16.680 | it means something in quantum mechanics.

00:21:18.880 | It means something in a lot of places.

00:21:22.340 | It means something to us humans as conscious beings.

00:21:25.680 | So what's the importance of the observer?

00:21:29.640 | What is the observer and what's the importance

00:21:31.280 | of the observer in the computational universe?

00:21:33.740 | - So this question of what is an observer,

00:21:36.080 | what's the general idea of an observer,

00:21:37.960 | is actually one of my next projects,

00:21:39.360 | which got somewhat derailed

00:21:40.520 | by the current sort of AI mania, but--

00:21:43.080 | - Is there a connection there,

00:21:44.040 | or is that, do you think the observer

00:21:46.360 | is primarily a physics phenomenon?

00:21:48.160 | Is it related to the whole AI thing?

00:21:50.080 | - Yes. - Yes, it is related.

00:21:51.640 | So one question is what is a general observer?

00:21:54.480 | So, we know, we have an idea

00:21:56.320 | what is a general computational system.

00:21:58.320 | We think about Turing machines,

00:21:59.680 | we think about other models of computation.

00:22:01.880 | There's a question, what is a general model of an observer?

00:22:04.800 | And there's kind of observers like us,

00:22:08.440 | which is kind of the observers we're interested in.

00:22:10.720 | We could imagine an alien observer

00:22:12.500 | that deals with computational irreducibility,

00:22:14.320 | and it has a mind that's utterly different from ours

00:22:16.640 | and completely incoherent with what we're like.

00:22:20.080 | But the fact is that if we are talking

00:22:23.080 | about observers like us,

00:22:24.920 | that one of the key things is this idea

00:22:27.320 | of kind of taking all the detail of the world

00:22:30.320 | and being able to stuff it into a mind,

00:22:32.480 | being able to take all the detail

00:22:34.520 | and kind of extract out of it

00:22:37.200 | a smaller set of kind of degrees of freedom,

00:22:40.440 | a smaller number of elements

00:22:42.560 | that will sort of fit in our minds.

00:22:44.880 | And I think this question,

00:22:46.680 | so I've been interested in trying to characterize

00:22:49.480 | what is the general observer?

00:22:51.200 | And the general observer is, I think, in part,

00:22:54.920 | there are many, let me give an example of it.

00:22:57.000 | You know, you have a gas,

00:22:57.900 | it's got a bunch of molecules bouncing around,

00:23:00.240 | and the thing you're measuring about the gas is its pressure.

00:23:04.160 | And the only thing you as an observer care about is pressure.

00:23:07.040 | And that means you have a piston on the side of this box,

00:23:10.000 | and the piston is being pushed by the gas.

00:23:12.400 | And there are many, many different ways

00:23:13.960 | that molecules can hit that piston.

00:23:15.960 | But all that matters is the kind of aggregate

00:23:19.120 | of all those molecular impacts,

00:23:21.160 | 'cause that's what determines pressure.

00:23:22.760 | So there's a huge number of different configurations

00:23:24.840 | of the gas, which are all equivalent.

00:23:27.040 | So I think one key aspect of observers

00:23:29.080 | is this equivalency of many different configurations

00:23:31.940 | of a system saying,

00:23:33.120 | "All I care about is this aggregate feature.

00:23:35.440 | All I care about is this overall thing."

00:23:38.380 | And that's sort of one aspect.

00:23:40.640 | And we see that in lots of different,

00:23:42.720 | again, it's the same story over and over again,

00:23:44.800 | that there's a lot of detail in the world,

00:23:47.320 | but what we are extracting from it is something,

00:23:49.960 | a sort of a thin summary of that detail.

00:23:53.520 | - Is that thin summary nevertheless true?

00:23:56.960 | Can it be a crappy approximation?

00:24:01.440 | - Sure. - That on average is correct?

00:24:03.560 | I mean, if we look at the observer that's the human mind,

00:24:05.840 | it seems like there's a lot of very,

00:24:07.640 | as represented by natural language, for example,

00:24:10.520 | there's a lot of really crappy approximation.

00:24:12.880 | - Sure. - And that could be

00:24:14.000 | maybe a feature of it.

00:24:15.640 | - Well, yes. - But there's ambiguity.

00:24:17.260 | - Right, right.

00:24:18.180 | You don't know, it could be the case.

00:24:20.560 | You're just measuring the aggregate impacts

00:24:22.560 | of these molecules, but there is some tiny,

00:24:24.800 | tiny probability that molecules will arrange themselves

00:24:27.640 | in some really funky way.

00:24:29.280 | And that just measuring that average

00:24:31.920 | isn't going to be the main point.

00:24:33.680 | By the way, an awful lot of science

00:24:35.840 | is very confused about this.

00:24:37.320 | Because you look at papers

00:24:39.640 | and people are really keen, they draw this curve

00:24:41.920 | and they have these bars on the curve and things.

00:24:45.000 | It's just this curve.

00:24:46.200 | And it's this one thing.

00:24:47.580 | And it's supposed to represent some system

00:24:50.080 | that has all kinds of details in it.

00:24:52.480 | And this is a way that lots of science has gotten wrong.

00:24:55.080 | Because people say, I remember years ago

00:24:56.960 | I was studying snowflake growth.

00:24:58.600 | You have a snowflake and it's growing,

00:25:00.920 | it has all these arms, it's doing complicated things.

00:25:03.720 | But there was a literature on this stuff

00:25:05.720 | and it talked about what's the rate of snowflake growth.

00:25:08.440 | And it got pretty good answers

00:25:10.520 | for the rate of the growth of the snowflake.

00:25:12.560 | And I looked at it more carefully

00:25:14.040 | and they had these nice curves

00:25:15.400 | of snowflake growth rates and so on.

00:25:17.280 | I looked at it more carefully and I realized,

00:25:19.200 | according to their models, the snowflake will be spherical.

00:25:22.160 | And so they got the growth rate right,

00:25:25.500 | but the detail was just utterly wrong.

00:25:28.320 | And not only the detail, the whole thing was not capturing,

00:25:33.320 | it was capturing this aspect of the system

00:25:36.080 | that was in a sense missing the main point

00:25:38.320 | of what was going on.

00:25:39.400 | - What is the geometric shape of a snowflake?

00:25:43.480 | - Snowflakes start in the phase of water

00:25:46.760 | that's relevant to formation of snowflakes.

00:25:49.120 | It's a phase of ice which starts

00:25:51.240 | with a hexagonal arrangement of water molecules.

00:25:54.240 | And so it starts off growing as a hexagonal plate.

00:25:56.960 | And then what happens is--

00:25:58.480 | - It's a plate, oh, oh, versus sphere, sphere versus plate.

00:26:00.880 | - Well, no, no, but it's much more than that.

00:26:02.320 | I mean, snowflakes are fluffy.

00:26:04.280 | Typical snowflakes have little dendritic arms.

00:26:07.640 | And what actually happens is, it's kind of cool

00:26:10.120 | because you can make these very simple discrete models

00:26:13.080 | with cellular automata and things that figure this out.

00:26:16.540 | You start off with this hexagonal thing,

00:26:19.380 | and then the places, it starts to grow little arms.

00:26:23.040 | And every time a little piece of ice adds itself

00:26:26.400 | to the snowflake, the fact that that ice condensed

00:26:29.520 | from the water vapor heats the snowflake up locally.

00:26:33.160 | And so it makes it less likely for another piece of ice

00:26:37.680 | to accumulate right nearby.

00:26:40.320 | So this leads to a kind of growth inhibition.

00:26:41.800 | So you grow an arm and it is a separated arm

00:26:46.040 | because right around the arm, it got a little bit hot

00:26:49.420 | and it didn't add more ice there.

00:26:51.520 | So what happens is it grows, you have a hexagon,

00:26:53.560 | it grows out arms, the arms grow arms,

00:26:56.640 | and then the arms grow arms, grow arms.

00:26:58.560 | And eventually, actually, it's kind of cool

00:26:59.880 | because it actually fills in another hexagon,

00:27:01.840 | a bigger hexagon.

00:27:03.240 | And when I first looked at this,

00:27:04.560 | I had a very simple model for this.

00:27:06.240 | I realized when it fills in that hexagon,

00:27:08.160 | it actually leaves some holes behind.

00:27:10.120 | So I thought, well, is that really right?

00:27:12.240 | So I look at these pictures of snowflakes

00:27:13.760 | and sure enough, they have these little holes in them

00:27:15.920 | that are kind of scars of the way that these arms grow out.

00:27:19.280 | - So you can't fill in backfill holes.

00:27:23.800 | So it just keeps going up. - They don't backfill, yeah.

00:27:24.960 | They don't backfill.

00:27:25.880 | - And presumably, there's a limitation of how big,

00:27:28.240 | like you can't arbitrarily grow.

00:27:31.220 | - I'm not sure.

00:27:32.060 | I mean, the thing falls through the,

00:27:33.440 | I mean, I think it hits the ground at some point.

00:27:36.880 | I think you can grow, I think you can grow in the lab.

00:27:39.800 | I think you can grow pretty big ones.

00:27:41.360 | I think you can grow many, many iterations

00:27:43.540 | of this kind of, goes from hexagon, it grows out arms,

00:27:46.480 | it turns back, it fills back into a hexagon,

00:27:48.440 | it grows more arms again.

00:27:49.720 | - In 3D. - No, it's flat, usually.

00:27:51.520 | - Why is it flat?

00:27:53.360 | Why doesn't it span out?

00:27:55.760 | Okay, okay, okay, wait a minute.

00:27:56.600 | You said it's fluffy,

00:27:57.520 | and fluffy is a three-dimensional property, no?

00:27:59.960 | - No, it's fluffy.

00:28:01.420 | Snow is, okay, so what makes, we're really in a detail.

00:28:06.420 | - I like this, let's go there.

00:28:08.340 | There's multiple snowflakes become fluffy.

00:28:10.260 | Well, a single snowflake is not fluffy.

00:28:12.220 | - No, no, a single snowflake is fluffy.

00:28:14.540 | And what happens is, if you have snow

00:28:18.180 | that is just pure hexagons,

00:28:20.100 | they fit together pretty well.

00:28:23.100 | It doesn't have a lot of air in it.

00:28:26.140 | And they can also slide against each other pretty easily.

00:28:28.540 | And so the snow can be pretty,

00:28:30.540 | I think avalanches happen sometimes

00:28:32.700 | when the things tend to be these hexagonal plates,

00:28:36.480 | and it kind of slides.

00:28:37.880 | But then when the thing has all these arms

00:28:40.000 | that have grown out, they don't fit together very well.

00:28:43.080 | And that's why the snow has lots of air in it.

00:28:45.720 | And if you look at one of these snowflakes,

00:28:47.000 | and if you catch one, you'll see it has these little arms.

00:28:49.960 | And people, actually people often say,

00:28:52.520 | no two snowflakes are alike.

00:28:55.020 | That's mostly because as a snowflake grows,

00:28:58.240 | they do grow pretty consistently

00:28:59.680 | with these different arms and so on.

00:29:01.320 | But you capture them at different times.

00:29:03.600 | As they fell through the air in a different way,

00:29:07.140 | you'll catch this one at this stage.

00:29:09.440 | And as it goes through different stages,

00:29:10.880 | they look really different.

00:29:12.120 | And so that's why it kind of looks like

00:29:14.040 | no two snowflakes are alike,

00:29:15.600 | because you caught them at different times.

00:29:17.960 | - So the rules under which they grow are the same.

00:29:20.480 | It's just the timing is.

00:29:22.020 | - Yes.

00:29:22.860 | - Okay, so the point is, science is not able

00:29:25.020 | to describe the full complexity of snowflake growth.

00:29:29.480 | - Well, science, if you do what people might often do,

00:29:33.560 | which is say, okay, let's make it scientific.

00:29:36.280 | Let's turn it into one number.

00:29:38.560 | And that one number is kind of the growth rate of the arms

00:29:40.640 | or some such other thing.

00:29:42.040 | That fails to capture sort of the detail

00:29:44.480 | of what's going on inside the system.

00:29:46.280 | And that's in a sense a big challenge for science,

00:29:48.700 | is how do you extract from the natural world, for example,

00:29:53.320 | those aspects of it that you are interested

00:29:56.580 | in talking about?

00:29:57.420 | Now, you might just say, I don't really care

00:29:58.940 | about the fluffiness of the snowflakes.

00:30:00.780 | All I care about is the growth rate of the arms,

00:30:03.140 | in which case, you can have a good model

00:30:06.140 | without knowing anything about the fluffiness.

00:30:08.980 | But the fact is, as a practical,

00:30:11.420 | if you say, what is the most obvious feature of a snowflake?

00:30:15.220 | Oh, that it has this complicated shape.

00:30:17.260 | Well, then you've got a different story

00:30:19.260 | about what you model.

00:30:20.260 | I mean, this is one of the features

00:30:21.700 | of sort of modeling in science.

00:30:23.840 | That what is a model?

00:30:24.820 | A model is some way of reducing the actuality of the world

00:30:28.880 | to something where you can readily sort of give a narrative

00:30:31.840 | for what's happening, where you can basically

00:30:34.240 | make some kind of abstraction of what's happening

00:30:36.880 | and answer questions that you care about answering.

00:30:39.860 | If you wanted to answer all possible questions

00:30:41.840 | about the system, you'd have to have the whole system,

00:30:44.400 | 'cause you might care about this particular molecule.

00:30:46.200 | Where did it go?

00:30:47.280 | And your model, which is some big abstraction of that,

00:30:50.380 | has nothing to say about that.

00:30:52.300 | So, one of the things that's often confusing in science

00:30:55.520 | is people will have, I've got a model, somebody says.

00:30:58.000 | Somebody else will say, I don't believe in your model

00:30:59.680 | because it doesn't capture the feature of the system

00:31:01.640 | that I care about.

00:31:03.200 | There's always this controversy about,

00:31:05.460 | is it a correct model?

00:31:07.240 | Well, no model is, except for the actual system itself,

00:31:10.600 | is a correct model in the sense that it captures everything.

00:31:13.760 | Question is, does it capture what you care about capturing?

00:31:16.440 | Sometimes that's ultimately defined

00:31:18.340 | by what you're going to build technology out of,

00:31:20.360 | things like this.

00:31:21.640 | The one counterexample to this is,

00:31:23.840 | if you think you're modeling the whole universe

00:31:25.680 | all the way down, then there is a notion of a correct model.

00:31:30.320 | But even that is more complicated

00:31:31.840 | because it depends on kind of how observers sample things

00:31:35.200 | and so on, that's a separate story.

00:31:36.940 | But at least at the first level, to say,

00:31:39.720 | this thing about, oh, it's an approximation,

00:31:41.580 | you're capturing one aspect,

00:31:42.760 | you're not capturing other aspects.

00:31:44.680 | When you really think you have a complete model

00:31:46.840 | for the whole universe, you better be capturing

00:31:49.060 | ultimately everything, even though to actually run

00:31:52.700 | that model is impossible

00:31:54.400 | because of computational irreducibility.

00:31:56.400 | The only thing that successfully runs that model

00:31:59.380 | is the actual running of the universe.

00:32:01.140 | - Is the universe itself.

00:32:02.180 | But okay, so what you care about is an interesting concept.

00:32:06.380 | So that's a human concept.

00:32:08.740 | So that's what you're doing with Wolfram Alpha

00:32:11.340 | and Wolfram Language, is you're trying to come up

00:32:14.740 | with symbolic representations as simple as possible.

00:32:19.400 | So a model that's as simple as possible

00:32:24.400 | that fully captures stuff we care about.

00:32:26.560 | - Yes, so I mean, for example, we could,

00:32:30.380 | we'll have a thing about data about movies, let's say.

00:32:34.020 | We could be describing every individual pixel

00:32:36.620 | in every movie and so on,

00:32:37.640 | but that's not the level that people care about.

00:32:40.380 | And it's, yes, this is a, I mean,

00:32:42.360 | and that level that people care about is somewhat related

00:32:47.000 | to what's described in natural language.

00:32:49.320 | But what we're trying to do is to find a way

00:32:52.640 | to sort of represent precisely so you can compute things.

00:32:55.480 | See, one thing we say,

00:32:57.340 | you give a piece of natural language,

00:32:58.840 | question is you feed it to a computer.

00:33:00.900 | You say, does the computer understand this natural language?

00:33:04.480 | Well, the computer processes it in some way, it does this,

00:33:07.400 | maybe it can make a continuation of the natural language.

00:33:09.920 | Maybe it can go on from the prompt

00:33:11.680 | and say what it's gonna say.

00:33:13.200 | You say, does it really understand it?

00:33:15.700 | Hard to know, but for in this kind of computational world,

00:33:20.700 | there is a very definite definition of does it understand,

00:33:24.480 | which is, could it be turned

00:33:26.060 | into this symbolic computational thing

00:33:28.860 | from which you can compute all kinds of consequences?

00:33:31.740 | And that's the sense in which one has sort of a target

00:33:34.840 | for the understanding of natural language.

00:33:37.000 | And that's kind of our goal is to have as much as possible

00:33:40.980 | about the world that can be computed in a reasonable way,

00:33:45.160 | so to speak, be able to be sort of captured

00:33:48.600 | by this kind of computational language.

00:33:50.280 | That's kind of the goal.

00:33:51.560 | And I think for us humans,

00:33:53.440 | the main thing that's important is

00:33:55.680 | as we formalize what we're talking about,

00:33:58.520 | it gives us a way of kind of building a structure

00:34:01.680 | where we can sort of build this tower

00:34:03.400 | of consequences of things.

00:34:05.040 | So if we're just saying,

00:34:06.220 | well, let's talk about it in natural language,

00:34:08.360 | it doesn't really give us some hard foundation

00:34:10.880 | that lets us build step by step to work something out.

00:34:14.000 | I mean, it's kind of like what happens in math.

00:34:15.920 | If we were just sort of vaguely talking about math,

00:34:19.160 | but didn't have the kind of full structure of math

00:34:21.680 | and all that kind of thing,

00:34:22.940 | we wouldn't be able to build

00:34:23.920 | this kind of big tower of consequences.

00:34:26.160 | And so, in a sense, what we're trying to do

00:34:28.160 | with the whole computational language effort

00:34:31.440 | is to make a formalism for describing the world

00:34:34.440 | that makes it possible to kind of build

00:34:36.280 | this tower of consequences.

00:34:38.360 | - Well, can you talk about this dance

00:34:40.560 | between natural language and Wolfram language?

00:34:44.560 | So there's this gigantic thing called the internet

00:34:47.040 | where people post memes and diary type thoughts

00:34:52.040 | and very important sounding articles and all of that

00:34:56.240 | that makes up the training data set for GPT.

00:34:59.840 | And then there's Wolfram language.

00:35:02.400 | How can you map from the natural language of the internet

00:35:06.460 | to the Wolfram language?

00:35:08.520 | Is there a manual, is there an automated way of doing that

00:35:13.520 | as we look into the future?

00:35:15.800 | - Well, so Wolfram Alpha, what it does,

00:35:18.960 | its kind of front end is turning natural language

00:35:22.560 | into computational language.

00:35:24.400 | - What you mean by that is there's a prompt,

00:35:26.720 | you ask a question, what is the capital of some country?

00:35:30.320 | - And it turns into, what's the distance

00:35:32.500 | between Chicago and London or something?

00:35:35.460 | And that will turn into geo distance of entity, city,

00:35:40.460 | et cetera, et cetera, et cetera.

00:35:41.820 | Each one of those things is very well defined.

00:35:44.340 | We know, given that it's the entity, city, Chicago,

00:35:47.500 | et cetera, et cetera, et cetera, Illinois, United States,

00:35:51.180 | we know the geo location of that, we know its population,

00:35:54.560 | we know all kinds of things about it,

00:35:56.220 | which we have curated that data to be able to know that

00:36:01.060 | with some degree of certainty, so to speak.

00:36:03.220 | And then we can compute things from this.

00:36:06.900 | And that's kind of the, yeah, that's the idea.

00:36:11.220 | - But then something like GPT, large language models,

00:36:15.060 | do they allow you to make that conversion

00:36:18.680 | much more powerful?

00:36:19.840 | - Okay, so that's an interesting thing,

00:36:21.040 | which we still don't know everything about, okay?

00:36:23.680 | I mean, this question of going from natural language

00:36:27.700 | to computational language, in Wolfram Alpha,

00:36:30.580 | we've now, Wolfram Alpha's been out and about

00:36:32.980 | for what, 13 and a half years now.

00:36:35.020 | And we've achieved, I don't know what it is,

00:36:37.980 | 98%, 99% success on queries that get put into it.

00:36:42.980 | Now, obviously, there's a sort of feedback loop

00:36:44.900 | 'cause the things that work are things people go on

00:36:46.860 | putting into it.

00:36:47.700 | So that, but we've got to a very high success rate

00:36:52.700 | of the little fragments of natural language

00:36:55.340 | that people put in, questions, math calculations,

00:36:58.740 | chemistry calculations, whatever it is.

00:37:02.780 | We do very well at that,

00:37:03.820 | turning those things into computational language.

00:37:06.620 | Now, from the very beginning of Wolfram Alpha,

00:37:09.300 | I thought about, for example,

00:37:11.540 | writing code with natural language.

00:37:14.020 | In fact, I had, I was just looking at this recently,

00:37:16.360 | I had a post that I wrote in 2010, 2011,

00:37:19.420 | called something like programming with natural language

00:37:21.540 | is actually going to work, okay?

00:37:23.900 | And so, we had done a bunch of experiments

00:37:26.900 | using methods that were a little bit,

00:37:29.900 | some of them a little bit machine learning-like,

00:37:32.480 | but certainly not the same kind of idea

00:37:35.820 | of vast training data and so on

00:37:37.860 | that is the story of large language models.

00:37:40.100 | Actually, I know that post, a piece of utter trivia,

00:37:43.080 | but that post, Steve Jobs forwarded that post

00:37:46.720 | around to all kinds of people at Apple.

00:37:48.620 | And he, you know, that was,

00:37:49.960 | 'cause he never really liked programming languages.

00:37:51.900 | So he was very happy to see the idea

00:37:54.540 | that you could get rid of this kind of layer

00:37:57.860 | of kind of engineering-like structure.

00:38:00.500 | He would have liked, you know, I think what's happening now

00:38:02.680 | because it really is the case that you can, you know,

00:38:05.460 | this idea that you have to kind of learn

00:38:07.440 | how the computer works to use a programming language

00:38:10.800 | is something that is, I think, a thing that, you know,

00:38:14.220 | just like you had to learn the details of the opcodes

00:38:16.360 | to know how assembly language worked and so on.

00:38:18.360 | It's kind of a thing that's a limited time horizon,

00:38:22.160 | but kind of the, you know, so this idea

00:38:26.800 | of how elaborate can you make kind of the prompt,

00:38:30.640 | how elaborate can you make the natural language

00:38:32.840 | and abstract from it computational language?

00:38:35.940 | It's a very interesting question.

00:38:37.600 | And, you know, what chat-GBT, you know,

00:38:41.660 | GBT-4 and so on can do is pretty good.

00:38:45.060 | It isn't, it's a very interesting process.

00:38:48.320 | I mean, I'm still trying to understand this workflow.

00:38:49.960 | We've been working out a lot of tooling

00:38:52.060 | around this workflow.

00:38:53.120 | - The natural language to computational language.

00:38:56.160 | - Right.

00:38:57.000 | - And the process, especially if it's conversation,

00:38:59.120 | like dialogue, it's like multiple queries kind of thing.

00:39:02.200 | - Yeah, right.

00:39:03.040 | There's so many things that are really interesting

00:39:04.960 | that work and so on.

00:39:06.840 | So first thing is, can you just walk up to the computer

00:39:09.520 | and expect to sort of specify a computation?

00:39:12.560 | What one realizes is humans have to have some idea

00:39:16.760 | of kind of this way of thinking

00:39:17.940 | about things computationally.

00:39:19.440 | Without that, you're kind of out of luck

00:39:21.080 | because you just have no idea

00:39:22.160 | what you're going to walk up to a computer.

00:39:24.240 | - I remember when I should tell a silly story about myself.

00:39:26.960 | The very first computer I saw,

00:39:28.720 | which is when I was 10 years old,

00:39:30.400 | and it was a big mainframe computer and so on.

00:39:32.400 | And I didn't really understand what computers did.

00:39:34.880 | And it's like, somebody was showing me this computer

00:39:36.800 | and it's like, you know,

00:39:38.680 | can the computer work out the weight of a dinosaur?

00:39:41.520 | It's like, that isn't a sensible thing to ask.

00:39:43.880 | That's kind of, you know, you have to give it,

00:39:45.800 | that's not what computers do.

00:39:47.700 | I mean, in Wolfram Alpha, for example,

00:39:49.440 | you could say, what's the typical weight of a stegosaurus?

00:39:51.560 | And it will give you some answer,

00:39:52.720 | but that's a very different kind of thing

00:39:54.480 | from what one thinks of computers as doing.

00:39:56.960 | And so the kind of the question of, you know,

00:40:00.760 | first thing is people have to have an idea

00:40:03.200 | of what computation is about.

00:40:06.440 | And, you know, I think it's a very, you know,

00:40:08.160 | for education, that is the key thing.

00:40:10.920 | It's kind of this notion, not computer science,

00:40:14.960 | not sort of the details of programming,

00:40:17.000 | but just this idea of how do you think

00:40:18.640 | about the world computationally.

00:40:20.360 | Computation, thinking about the world computationally

00:40:23.720 | is kind of this formal way of thinking about the world.

00:40:26.720 | We've had other ones like logic was a formal way,

00:40:29.440 | you know, as a way of sort of abstracting

00:40:31.120 | and formalizing some aspects of the world.

00:40:33.160 | Mathematics is another one.

00:40:34.720 | Computation is this very broad way of sort of formalizing

00:40:37.680 | the way we think about the world.

00:40:39.360 | And the thing that's cool about computation

00:40:42.040 | is if we can successfully formalize things

00:40:45.040 | in terms of computation, computers can help us figure out

00:40:47.960 | what the consequences are.

00:40:49.320 | It's not like you formalized it with math.

00:40:51.080 | Well, that's nice, but now you have to,

00:40:53.120 | if you're not using a computer to do the math,

00:40:55.960 | you have to go work out a bunch of stuff yourself.

00:40:58.720 | So I think, but this idea, let's see, I mean,

00:41:01.880 | that we're trying to take kind of the,

00:41:05.800 | we're talking about sort of natural language

00:41:07.560 | and its relationship to computational language.

00:41:09.760 | The thing, the sort of the typical workflow,

00:41:12.560 | I think is first human has to have some kind of idea

00:41:16.200 | of what they're trying to do.

00:41:17.520 | That if it's something that they want to sort of build

00:41:20.360 | a tower of capabilities on,

00:41:22.760 | something that they want to sort of formalize

00:41:24.520 | and make computational.

00:41:26.080 | So then human can type something into, you know,

00:41:30.040 | some LLM system and sort of say vaguely what they want

00:41:35.040 | in sort of computational terms.

00:41:38.040 | Then it does pretty well at synthesizing

00:41:40.600 | well from language code.

00:41:42.080 | And it'll probably do better in the future

00:41:44.000 | 'cause we've got a huge number of examples

00:41:46.280 | of natural language input together

00:41:49.000 | with the Wolfram language translation of that.

00:41:51.200 | So it's kind of a, you know, that's a thing

00:41:54.880 | where you can kind of extrapolating from all those examples

00:41:59.640 | makes it easier to do that task.

00:42:01.960 | - Is the prompter task could also kind of debugging

00:42:04.480 | the Wolfram language code?

00:42:06.400 | Or is your hope to not do that debugging?

00:42:08.920 | - Oh, no, no, no.

00:42:09.760 | I mean, so there are many steps here.

00:42:11.280 | Okay, so first, the first thing is you type natural language.

00:42:14.760 | It generates Wolfram language code.

00:42:15.600 | - Do you have examples by the way?

00:42:16.960 | Do you have an example that is the dinosaur example?

00:42:20.320 | Do you have an example that jumps to mind

00:42:22.080 | that we should be thinking about some dumb example?

00:42:24.280 | - It's like take my heart rate data

00:42:27.200 | and, you know, figure out whether I, you know,

00:42:32.200 | make a moving average every seven days or something

00:42:35.920 | and work out what the, and make a plot of the results.

00:42:39.840 | Okay, so that's a thing which is, you know,

00:42:42.000 | about two thirds of a line of Wolfram language code.

00:42:45.320 | I mean, it's, you know, list plot of moving average

00:42:48.480 | of some data bin or something of the data

00:42:51.640 | and then you'll get the result.

00:42:53.680 | And, you know, the vague thing that I was just saying

00:42:56.080 | in natural language could, would almost certainly

00:43:00.240 | correctly turn into that very simple piece

00:43:02.360 | of Wolfram language code.

00:43:03.760 | - So you start mumbling about heart rate.

00:43:06.760 | - Yeah.

00:43:07.600 | - And then kind of, you know,

00:43:09.160 | you arrive at the moving average kind of idea.

00:43:12.080 | - Right, you say average over seven days,

00:43:13.840 | maybe it'll figure out that that's a moving, you know,

00:43:15.680 | that that can be encapsulated as this moving average idea.

00:43:18.560 | I'm not sure.

00:43:19.560 | But then the typical workflow that I'm seeing is

00:43:24.040 | you generate this piece of Wolfram language code.

00:43:26.040 | It's pretty small usually.

00:43:27.480 | It's, and if it isn't small, it probably isn't right.

00:43:31.360 | But, you know, if it's, it's pretty small

00:43:34.520 | and, you know, Wolfram language is,

00:43:36.680 | one of the ideas of Wolfram language

00:43:38.200 | is it's a language that humans can read.

00:43:40.280 | It's not a language which, you know,

00:43:41.920 | programming languages tend to be this one way story

00:43:44.600 | of humans write them and computers execute from them.

00:43:48.880 | Wolfram language is intended to be something

00:43:50.480 | which is sort of like math notation,

00:43:52.760 | something where, you know, humans write it

00:43:55.120 | and humans are supposed to read it as well.

00:43:57.240 | And so kind of the workflow that's emerging

00:43:59.880 | is kind of this, this human mumbles some things,

00:44:03.640 | you know, large language model produces a fragment

00:44:07.360 | of Wolfram language code.

00:44:08.960 | Then you look at that, you say,

00:44:11.200 | yeah, that looks, well, typically you just run it first.

00:44:13.760 | You see, does it produce the right thing?

00:44:15.560 | You look at what it produces.

00:44:16.720 | You might say, that's obviously crazy.

00:44:18.520 | You look at the code, you see, I see why it's crazy.

00:44:21.640 | You fix it.

00:44:22.560 | If you really care about the result

00:44:24.000 | and you really want to make sure it's right,

00:44:25.080 | you better look at that code and understand it

00:44:27.400 | because that's the way you have the sort of checkpoint

00:44:29.960 | of did it really do what I expected it to do?

00:44:32.320 | Now you go beyond that.

00:44:34.280 | I mean, it's, it's, it's, you know,

00:44:35.960 | what we find is, for example,

00:44:37.480 | let's say the code does the wrong thing.

00:44:39.640 | Then you can often say to the large language model,

00:44:42.400 | can you adjust this to do this?

00:44:44.720 | And it's pretty good at doing that.

00:44:46.680 | - Interesting.

00:44:47.520 | So you're using the output of the code

00:44:51.200 | to give you hints about the,

00:44:55.160 | the function of the code.

00:44:57.680 | So you're debugging based on the output of the code,

00:45:00.720 | not the code itself.

00:45:01.600 | - Right.

00:45:02.440 | The plugin that we have, you know, for chat GPT,

00:45:05.480 | it does that routinely.

00:45:07.000 | You know, it will send the thing in,

00:45:09.560 | it will get a result.

00:45:10.920 | It will discover, the LLM will discover itself

00:45:13.240 | that the result is not plausible.

00:45:14.800 | And it will go back and say, oh, I'm sorry.

00:45:16.440 | It's very polite.

00:45:17.280 | And it, you know, it goes back and says,

00:45:19.560 | I'll rewrite that piece of code

00:45:20.840 | and then it will try it again and get the result.

00:45:24.120 | The other thing that's pretty interesting is

00:45:25.960 | when you're just running.

00:45:26.800 | So one of the new concepts that we have,

00:45:28.600 | we invented this whole idea of notebooks

00:45:30.640 | back 36 years ago now.

00:45:33.280 | And so now there's the question of sort of

00:45:34.800 | how do you combine this idea of notebooks

00:45:37.520 | where you have, you know, text and code and output?

00:45:41.080 | How do you combine that with the notion of chat and so on?

00:45:44.280 | And there's some really interesting things there.

00:45:45.960 | Like for example, a very typical thing now

00:45:48.560 | is we have these notebooks where as soon as the,

00:45:52.400 | if the thing produces errors,

00:45:55.720 | if the, you know, run this code

00:45:57.160 | and it produces messages and so on,

00:45:59.400 | the LLM automatically not only looks at those messages,

00:46:03.400 | it can also see all kinds of internal information

00:46:05.840 | about stack traces and things like this.

00:46:07.960 | And it can then, it does a remarkably good job

00:46:11.080 | of guessing what's wrong and telling you.

00:46:13.960 | So in other words, it's looking at things,

00:46:15.600 | sort of interesting.

00:46:16.440 | It's kind of a typical sort of AI-ish thing

00:46:19.360 | that it's able to have more sensory data

00:46:21.600 | than we humans are able to have.

00:46:23.040 | 'Cause it's able to look at a bunch of stuff

00:46:24.520 | that we humans would kind of glaze over looking at.

00:46:27.680 | And it's able to then come up with,

00:46:29.840 | oh, this is the explanation of what's happening.

00:46:32.080 | - And what is the data, the stack trace,

00:46:34.240 | the code you've written previously,

00:46:35.760 | the natural language you've written?

00:46:36.840 | - Yeah, it's also what's happening is,

00:46:38.560 | one of the things that's, is for example,

00:46:40.880 | when there's these messages,

00:46:41.960 | there's documentation about these messages.

00:46:43.760 | There's examples of where the messages have occurred.

00:46:45.360 | Otherwise, all these kinds of things.

00:46:47.440 | The other thing that's really amusing with this

00:46:49.680 | is when it makes a mistake,

00:46:51.360 | one of the things that's in our prompt

00:46:53.080 | when the code doesn't work is read the documentation.

00:46:56.120 | (laughing)

00:46:57.120 | And we have another piece of the plugin

00:46:59.960 | that lets it read documentation.

00:47:01.600 | And that again, is very, very useful.

00:47:03.960 | 'Cause it will figure out, sometimes it'll get,

00:47:07.440 | it'll make up the name of some option for some function

00:47:10.560 | that doesn't really exist, read the documentation.

00:47:12.920 | It'll have some wrong structure for the function and so on.

00:47:16.640 | It's some, that's a powerful thing.

00:47:18.040 | I mean, the thing that I've realized is,

00:47:21.200 | we built this language over the course of all these years

00:47:23.560 | to be nice and coherent and consistent and so on.

00:47:25.720 | So it's easy for humans to understand.

00:47:28.040 | Turns out there was a side effect that I didn't anticipate,

00:47:31.160 | which is it makes it easier for AIs to understand.

00:47:33.960 | - So it's almost like another natural language.

00:47:36.440 | But so Wolfram language is a kind of foreign language.

00:47:40.840 | - Yes, yes.

00:47:42.040 | - You have a lineup, English, French, Japanese,

00:47:45.440 | Wolfram language, and then, I don't know, Spanish.

00:47:49.920 | And then the system is not gonna notice, hopefully.

00:47:52.240 | - Well, yes.

00:47:53.720 | I mean, maybe.

00:47:55.040 | You know, that's an interesting question.

00:47:56.560 | 'Cause it really depends on what I see as being

00:47:59.920 | an important piece of fundamental science

00:48:02.400 | that basically just jumped out at us with Chachapiti.

00:48:05.940 | 'Cause I think, you know, the real question is,

00:48:09.600 | why does Chachapiti work?

00:48:11.360 | How is it possible to encapsulate, you know,

00:48:14.280 | to successfully reproduce all these kinds of things

00:48:17.260 | in natural language, you know, with a, you know,

00:48:20.880 | a comparatively small, he says, you know,

00:48:23.320 | couple of hundred billion, you know,

00:48:25.040 | weights of neural net and so on.

00:48:27.160 | And I think that, you know, that relates to kind of

00:48:29.960 | a fundamental fact about language, which, you know,

00:48:33.760 | the main thing is that I think there's a structure

00:48:36.920 | to language that we haven't kind of really explored

00:48:40.160 | very well, it's kind of the semantic grammar

00:48:42.440 | I'm talking about language.

00:48:46.040 | I mean, we kind of know that when we set up human language,

00:48:50.460 | we know that it has certain regularities.

00:48:52.440 | We know that it has a certain grammatical structure,

00:48:55.400 | you know, noun followed by verb, followed by noun,

00:48:58.600 | adjectives, et cetera, et cetera, et cetera.

00:49:00.760 | That's its kind of grammatical structure.

00:49:03.200 | But I think the thing that Chachapiti is showing us

00:49:06.040 | is that there's an additional kind of regularity

00:49:09.040 | to language, which has to do with the meaning

00:49:11.260 | of the language beyond just this pure, you know,

00:49:13.800 | part of speech combination type of thing.

00:49:16.360 | And I think the kind of the one example of that

00:49:20.240 | that we've had in the past is logic.

00:49:22.860 | And, you know, I think my sort of kind of picture

00:49:27.860 | of how was logic invented, how was logic discovered?

00:49:31.920 | It really was the thing that was discovered

00:49:33.760 | in its original conception.

00:49:35.600 | It was discovered presumably by Aristotle,

00:49:38.140 | who kind of listened to a bunch of people,

00:49:40.280 | orators, you know, giving speeches,

00:49:42.360 | and this one made sense, that one doesn't make sense,

00:49:46.000 | this one, and, you know, you see these patterns of,

00:49:49.480 | you know, if the, you know, I don't know what,

00:49:52.760 | you know, if the Persians do this,

00:49:55.300 | then the this does that, et cetera, et cetera, et cetera.

00:49:58.680 | And what Aristotle realized is there's a structure

00:50:02.240 | to those sentences, there's a structure to that rhetoric

00:50:05.140 | that doesn't matter whether it's the Persians and the Greeks

00:50:07.920 | or whether it's the cats and the dogs.

00:50:10.080 | It's just, you know, P and Q, you can abstract

00:50:13.120 | from the details of these particular sentences,

00:50:16.420 | you can lift out this kind of formal structure,

00:50:19.220 | and that's what logic is.

00:50:20.880 | - That's a heck of a discovery, by the way,

00:50:22.700 | logic, you're making me realize now.

00:50:25.460 | - Yeah. - It's not obvious.

00:50:27.200 | - The fact that there is an abstraction

00:50:29.400 | from natural language that has,

00:50:32.360 | where you can fill in any word you want.

00:50:34.600 | - Yeah.

00:50:35.440 | - Is a very interesting discovery.

00:50:36.740 | Now, it took a long time to mature.

00:50:38.860 | I mean, Aristotle had this idea of syllogistic logic,

00:50:42.320 | where there were these particular patterns

00:50:44.540 | of how you could argue things, so to speak.

00:50:47.260 | And, you know, in the Middle Ages,

00:50:48.920 | part of education was you memorized the syllogisms,

00:50:51.760 | I forget how many there were, but 15 of them or something.

00:50:54.680 | And they all had names, they all had mnemonics,

00:50:56.600 | like I think Barbara and Sellerant

00:50:58.400 | were two of the mnemonics for the syllogisms.

00:51:01.240 | And people would kind of, this is a valid argument

00:51:03.520 | 'cause it follows the Barbara syllogism, so to speak.

00:51:06.680 | And it took until 1830, you know, with George Boole

00:51:11.360 | to kind of get beyond that and kind of see that

00:51:14.320 | there was a level of abstraction

00:51:16.840 | that was beyond this particular template

00:51:19.320 | of a sentence, so to speak.

00:51:21.040 | And that's, you know, what's interesting there is,

00:51:23.720 | in a sense, you know, chat-GBT is operating

00:51:27.760 | at the Aristotelian level.

00:51:29.400 | It's essentially dealing with templates of sentences.

00:51:32.500 | By the time you get to Boole and Boolean algebra

00:51:35.200 | and this idea of, you know, you can have arbitrary depth

00:51:37.760 | nested collections of ands and ors and nots,

00:51:40.600 | and you can resolve what they mean,

00:51:43.120 | that's the kind of thing, that's a computation story.

00:51:45.760 | That's, you know, you've gone beyond the pure sort of

00:51:48.240 | templates of natural language to something

00:51:50.560 | which is an arbitrarily deep computation.

00:51:53.120 | But the thing that I think we realized from chat-GBT

00:51:56.400 | is, you know, Aristotle stopped too quickly.

00:51:59.440 | And there was more that you could have lifted

00:52:01.580 | out of language as formal structures.

00:52:04.080 | And I think there's, you know, in a sense,

00:52:06.040 | we've captured some of that in, you know,

00:52:08.320 | some of what is in language,

00:52:10.720 | there's a lot of kind of little calculi,

00:52:14.680 | little algebras of what you can say,

00:52:17.400 | what language talks about.

00:52:18.920 | I mean, whether it's, I don't know, if you say,

00:52:21.280 | I go from place A to place B, place B to place C,

00:52:26.600 | then I know I've gone from place A to place C.

00:52:29.520 | If A is a friend of B and B is a friend of C,

00:52:32.360 | it doesn't necessarily follow that A is a friend of C.

00:52:35.280 | These are things that are, you know, that there are,

00:52:39.040 | if you go from place A to place B, place B to place C,

00:52:42.760 | it doesn't matter how you went.

00:52:44.620 | Like logic, it doesn't matter whether you flew there,

00:52:47.480 | walked there, swam there, whatever.

00:52:50.220 | You still, this transitivity of where you go

00:52:53.800 | is still valid.

00:52:54.980 | And there are many kinds of kind of features, I think,

00:52:58.120 | of the way the world works that are captured

00:53:02.240 | in these aspects of language, so to speak.

00:53:04.880 | And I think what "Chat GPT" effectively has found,

00:53:07.880 | just like it discovered logic, you know,

00:53:09.720 | people are really surprised it can do

00:53:11.160 | these logical inferences.

00:53:13.160 | It discovered logic the same way Aristotle

00:53:15.040 | discovered logic, by looking at a lot of sentences,

00:53:17.680 | effectively, and noticing the patterns in those sentences.

00:53:20.760 | - But it feels like it's discovering something

00:53:22.360 | much more complicated than logic.

00:53:24.920 | So this kind of semantic grammar,

00:53:26.920 | I think you wrote about this,

00:53:28.480 | maybe we can call it the laws of language,

00:53:32.480 | I believe you call, or which I like,

00:53:34.400 | the laws of thought.

00:53:36.000 | - Yes, that was the title that George Boole had

00:53:38.340 | for his Boolean algebra back in 1830, but yes.

00:53:41.200 | - Laws of thought?

00:53:42.280 | - Yes, that was what he said.

00:53:43.680 | - Ooh.

00:53:45.040 | - All right.

00:53:45.880 | - So he thought he nailed it with Boolean algebra.

00:53:48.840 | - Yeah. - There's more to it.

00:53:50.520 | - And it's a good question, how much more is there to it?

00:53:53.960 | And it seems like one of the reasons,

00:53:56.300 | as you imply that the reason "Chat GPT" works

00:54:01.200 | is that there's a finite number of things to it.

00:54:06.200 | - Yeah, I mean, it's--

00:54:07.360 | - It's discovering the laws, in some sense,

00:54:09.880 | GPT's discovering the laws of semantic grammar

00:54:14.000 | that underlies language.

00:54:15.480 | - Yes, and what's sort of interesting is,

00:54:18.200 | in the computational universe,

00:54:19.720 | there's a lot of other kinds of computation

00:54:21.460 | that you could do.

00:54:22.700 | They're just not ones that we humans have cared about

00:54:26.600 | and operate with, and that's probably because our brains

00:54:30.280 | are built in a certain way, and the neural nets

00:54:33.400 | of our brains are not that different, in some sense,

00:54:35.700 | from the neural nets of a large language model,

00:54:39.220 | and that's kind of, and so when we think about,

00:54:41.720 | and maybe you can talk about this some more,

00:54:43.680 | but when we think about sort of what will AIs ultimately do,

00:54:47.080 | the answer is, insofar as AIs are just doing computation,

00:54:50.860 | they can run off and do all these kinds

00:54:52.840 | of crazy computations, but the ones that we sort of have,

00:54:57.760 | have decided we care about is this kind of very limited set.

00:55:01.900 | - That's where the reinforcement learning

00:55:06.080 | with human feedback seems to come in.

00:55:08.220 | The more the AIs say the stuff that kind of interests us,

00:55:11.280 | the more we're impressed by it.

00:55:13.540 | So it can do a lot of interesting, intelligent things,

00:55:15.620 | but we're only interested in the AI systems

00:55:19.240 | when they communicate in a human-like way

00:55:23.160 | about human-like topics.

00:55:25.020 | - Yes, well, it's like technology.

00:55:27.600 | I mean, in a sense, the physical world

00:55:30.340 | provides all kinds of things.

00:55:31.760 | You know, there are all kinds of processes

00:55:33.240 | going on in physics.

00:55:34.680 | Only a limited set of those are ones that we capture

00:55:38.200 | and use for technology, 'cause they're only a limited set

00:55:40.960 | where we say, you know, this is a thing

00:55:43.640 | that we can sort of apply to the human purposes

00:55:46.600 | we currently care about.

00:55:47.440 | I mean, you might've said, okay,

00:55:48.800 | you pick up a piece of rock.

00:55:50.980 | You say, okay, this is a nice silicate.

00:55:52.480 | It contains all kinds of silicon.

00:55:54.100 | I don't care.

00:55:55.340 | Then you realize, oh, we could actually turn this

00:55:57.160 | into a semiconductor wafer

00:55:59.800 | and make a microprocessor out of it.

00:56:01.760 | And then we care a lot about it.

00:56:03.320 | - Yes.

00:56:04.160 | - And it's, you know, it's this thing about what do we,

00:56:06.820 | you know, in the evolution of our civilization,

00:56:09.780 | what things do we identify as being things we care about?

00:56:12.600 | I mean, it's like, you know,

00:56:13.880 | when there was a little announcement recently

00:56:16.280 | of the possibility of a high temperature superconductor

00:56:18.720 | that involved, you know, the element lutetium,

00:56:21.160 | which, you know, generally nobody has cared about.

00:56:23.920 | (Lex laughing)

00:56:24.760 | And, you know, it's kind of,

00:56:26.880 | but suddenly if there's this application

00:56:28.960 | that relates to kind of human purposes,

00:56:31.240 | we start to care a lot.

00:56:32.760 | - So given your thinking that GPT may have discovered

00:56:37.760 | inklings of laws of thought,

00:56:40.460 | do you think such laws exist?

00:56:41.980 | Can we linger on that?

00:56:43.220 | - Yeah.

00:56:44.060 | - What's your intuition here?

00:56:45.700 | - Oh, definitely.

00:56:46.540 | I mean, the fact is, look, the logic is but the first step.

00:56:51.540 | There are many other kinds of calculi about things

00:56:55.300 | that we consider, you know,

00:56:58.580 | about sort of things that happen in the world

00:57:01.020 | or things that are meaningful.

00:57:02.780 | - Well, how do you know logic is not the last step?

00:57:04.860 | You know what I mean?

00:57:05.820 | So like what-

00:57:06.660 | - Well, because we can plainly see that that thing,

00:57:08.840 | I mean, if you say,

00:57:10.620 | here's a sentence that is syntactically correct, okay?

00:57:14.680 | You look at it and you're like, you know,

00:57:16.740 | the happy electron, you know, ate, I don't know what,

00:57:21.740 | some something that it just,

00:57:24.220 | you look at it and it's like, this is meaningless.

00:57:26.220 | It's just a bunch of words.

00:57:27.820 | It's syntactically correct.

00:57:29.140 | The nouns and the verbs are in the right place,

00:57:31.220 | but it just doesn't mean anything.

00:57:32.920 | And so there clearly is some rule

00:57:37.020 | that there are rules that determine

00:57:38.860 | when a sentence has the potential to be meaningful

00:57:41.980 | that go beyond the pure parts of speech syntax.

00:57:45.620 | And so the question is, what are those rules?

00:57:47.820 | And are there a fairly finite set of those rules?

00:57:50.700 | My guess is that there's a fairly finite set of those rules.

00:57:53.940 | And they, you know, once you have those rules,

00:57:56.540 | you have a kind of a construction kit,

00:57:58.500 | just like the rules of syntactic grammar

00:58:02.100 | give you a construction kit

00:58:03.380 | for making syntactically correct sentences.

00:58:06.060 | So you can also have a construction kit

00:58:08.220 | for making semantically correct sentences.

00:58:10.980 | Those sentences may not be realized in the world.

00:58:13.540 | I mean, I think, you know, the elephant flew to the moon.

00:58:16.380 | - Yeah.

00:58:17.220 | - A syntactically, a semantically, you know,

00:58:20.020 | we know, we have an idea.

00:58:21.660 | If I say that to you, you kind of know what that means.

00:58:24.820 | But the fact is it hasn't been realized

00:58:26.780 | in the world, so to speak.

00:58:27.940 | - So semantically correct, perhaps,

00:58:30.220 | is things that can be imagined with the human mind.

00:58:32.660 | No.

00:58:34.220 | Things that are consistent with both our imagination

00:58:39.220 | and our understanding of physical reality.

00:58:42.140 | I don't know.

00:58:43.220 | - Yeah, good question.

00:58:44.620 | I mean, it's a good question.

00:58:46.140 | It's a good question.

00:58:47.780 | I mean, I think it is,

00:58:50.100 | given the way we have constructed language,

00:58:52.940 | it is things which fit

00:58:56.140 | with the things we're describing in language.

00:58:57.740 | It's a bit circular in the end,

00:58:59.740 | because, you know, you can,

00:59:01.620 | and the sort of boundaries

00:59:04.380 | of what is physically realizable.

00:59:07.140 | Okay, let's take the example of motion, okay?

00:59:10.180 | Motion is a complicated concept.

00:59:11.860 | It might seem like it's a concept

00:59:13.260 | that should have been figured out by the Greeks,

00:59:15.180 | you know, long ago,

00:59:16.460 | but it's actually a really pretty complicated concept,

00:59:18.580 | 'cause what is motion?

00:59:19.900 | Motion is you can go from place A to place B,

00:59:23.220 | and it's still you when you get to the other end, right?

00:59:26.620 | You take an object, you move it,

00:59:29.340 | and it's still the same object,

00:59:30.620 | but it's in a different place.

00:59:32.300 | Now, even in ordinary physics,

00:59:34.500 | that doesn't always work that way.

00:59:36.140 | If you're near a space-time singularity

00:59:37.900 | in a black hole, for example,

00:59:39.580 | and you take your teapot or something,

00:59:41.820 | you don't have much of a teapot

00:59:43.300 | by the time it's near the space-time singularity.

00:59:45.260 | It's been completely, you know,

00:59:46.940 | deformed beyond recognition.

00:59:49.420 | But, so that's a case where pure motion doesn't really work.

00:59:52.780 | You can't have a thing stay the same.

00:59:55.180 | But, so this idea of motion is something

00:59:58.180 | that sort of is a slightly complicated idea,

01:00:01.060 | but once you have the idea of motion,

01:00:03.820 | you can start, once you have the idea

01:00:05.300 | that you're gonna describe things

01:00:06.980 | as being the same thing, but in a different place,

01:00:10.300 | that sort of abstracted idea then has, you know,

01:00:14.900 | that has all sorts of consequences,

01:00:16.260 | like this transitivity of motion,

01:00:17.700 | go from A to B, B to C, you've gone from A to C.

01:00:20.860 | And that's, so that level of description,

01:00:23.700 | you can have what are sort of inevitable consequences.

01:00:28.140 | There are inevitable features

01:00:29.220 | of the way you've sort of set things up.

01:00:31.220 | And that's, I think, what this sort of semantic grammar

01:00:33.660 | is capturing, is things like that.

01:00:36.780 | And, you know, I think that it's a question

01:00:38.900 | of what does the word mean?

01:00:40.060 | When you say, I go from, I move from here to there,

01:00:42.700 | well, it's complicated to say what that means.

01:00:44.500 | This is this whole issue of, you know,

01:00:46.100 | is pure motion possible, et cetera, et cetera, et cetera.

01:00:48.600 | But once you have kind of got an idea of what that means,

01:00:51.780 | then there are inevitable consequences of that idea.

01:00:55.820 | - But the very idea of meaning,

01:00:57.940 | it seems like there's some words that become,

01:01:00.220 | it's like there's a latent ambiguity to them.

01:01:06.620 | I mean, it's the word like emotionally loaded words,

01:01:10.100 | like hate and love.

01:01:11.460 | - Right. - It's like, what are they?

01:01:14.300 | What do they mean exactly?

01:01:15.580 | So especially when you have relationships

01:01:19.340 | between complicated objects,

01:01:21.140 | we seem to take this kind of shortcut,

01:01:23.500 | descriptive shortcut to describe like,

01:01:26.340 | object A hates object B.

01:01:28.420 | What's that really mean?

01:01:29.860 | - Right, well, words are defined

01:01:33.260 | by kind of our social use of them.

01:01:35.820 | I mean, it's not, you know, a word,

01:01:38.480 | in computational language, for example,

01:01:40.940 | when we say we have a construct there,

01:01:44.780 | we expect that that construct is a building block

01:01:48.240 | from which we can construct an arbitrarily tall tower.

01:01:50.980 | So we have to have a very solid building block.

01:01:53.460 | And, you know, we have to, it turns into a piece of code,

01:01:56.100 | it has documentation, it's a whole thing.

01:01:59.780 | But the word hate, you know,

01:02:01.900 | the documentation for that word,

01:02:04.440 | well, there isn't a standard documentation for that word,

01:02:07.100 | so to speak.

01:02:08.060 | It's a complicated thing defined by kind of how we use it.

01:02:12.220 | When, you know, if it wasn't for the fact

01:02:14.260 | that we were using language, I mean,

01:02:16.100 | so what is language at some level?

01:02:17.620 | Language is a way of packaging thoughts

01:02:20.140 | so that we can communicate them to another mind.

01:02:23.180 | - Can these complicated words be converted

01:02:26.220 | into something that a computation engine can use?

01:02:30.060 | - Right, so I think the answer to that is

01:02:32.620 | that what one can do in computational language

01:02:36.420 | is make a specific definition.

01:02:40.620 | And if you have a complicated word,

01:02:42.620 | like let's say the word eat, okay?

01:02:45.380 | You'd think, oh, it's a simple word,

01:02:46.740 | it's, you know, animals eat things, whatever else.

01:02:48.940 | But, you know, you do programming,

01:02:50.380 | you say this function eats arguments,

01:02:53.820 | which is sort of poetically similar

01:02:56.580 | to the animal eating things.

01:02:58.180 | But if you start to say, well, what are the implications

01:03:00.520 | of, you know, the function eating something?

01:03:04.060 | You know, can the function be poisoned?

01:03:05.980 | Well, maybe it can't actually,

01:03:07.020 | but, you know, if there's a type mismatch

01:03:09.580 | or something in some language.

01:03:11.400 | But, you know, in what, how far does that analogy go?

01:03:15.340 | And it's just an analogy.

01:03:17.340 | Whereas if you use the word eat

01:03:20.660 | in a computational language level,

01:03:22.780 | you would define there isn't a thing

01:03:25.040 | which you anchor to the kind of natural language concept eat,

01:03:29.100 | but it is now some precise definition of that,

01:03:31.820 | that then you can compute things from.

01:03:33.420 | - But don't you think the analogy is also precise?

01:03:35.300 | Software eats the world.

01:03:37.140 | Don't you think there's something concrete

01:03:42.140 | in terms of meaning about analogies?

01:03:44.940 | - Sure.

01:03:45.820 | But the thing that sort of is the first target

01:03:48.580 | for computational language is to take sort of

01:03:52.300 | the ordinary meaning of things and try and make it precise,

01:03:56.020 | make it sufficiently precise,

01:03:57.280 | you can build these towers of computation on top of it.

01:03:59.780 | So it's kind of like, if you start with a piece of poetry

01:04:02.540 | and you say, I'm going to define my program

01:04:05.060 | with this piece of poetry,

01:04:06.900 | it's kind of like, that's a difficult thing.

01:04:10.420 | It's better to say, I'm gonna just have

01:04:12.080 | this boring piece of prose

01:04:13.980 | and it's using words in the ordinary way.

01:04:16.280 | And that's how I'm communicating with my computer.

01:04:19.460 | And that's how I'm going to build the solid building block

01:04:22.260 | from which I can construct

01:04:23.500 | this whole kind of computational tower.

01:04:25.220 | - So there's some sense where if you take a poem

01:04:28.380 | and reduce it to something computable,

01:04:29.900 | you're gonna have very few things left.

01:04:31.780 | So maybe there's a bunch of human interaction

01:04:34.760 | that's just poetic, aimless nonsense.

01:04:39.540 | That's just like recreational hamstring a wheel.

01:04:44.100 | It's not actually producing anything.

01:04:45.860 | - Well, I think that that's a complicated thing

01:04:48.580 | because in a sense, human linguistic communication

01:04:51.300 | is there's one mind, it's producing language,

01:04:55.300 | that language is having an effect on another mind.

01:04:58.500 | And the question of the sort of a type of effect

01:05:02.560 | that is well-defined, let's say,

01:05:04.700 | where for example, it's very independent of the two minds.

01:05:08.300 | It doesn't, you know, there's communication

01:05:10.460 | where it can matter a lot,

01:05:12.020 | sort of what the experience of one mind is

01:05:16.140 | versus another one and so on.

01:05:17.680 | - Yeah, but what is the purpose

01:05:23.720 | of natural language communication?

01:05:25.420 | - Well, I think the--

01:05:28.020 | - Versus, so computational language somehow

01:05:31.620 | feels more amenable to the definition of purpose.

01:05:34.560 | It's like, yeah, you're given two clean representations

01:05:39.560 | of a concept and you can build a tower based on that.

01:05:42.760 | Is natural language the same thing but more fuzzy?

01:05:46.160 | - Well, I think the story of natural language,

01:05:49.480 | that's the great invention of our species.

01:05:51.760 | We don't know whether it exists in other species,

01:05:54.160 | but we know it exists in our species.

01:05:56.320 | It's the thing that allows you to sort of communicate

01:06:00.280 | abstractly from like one generation

01:06:02.960 | of the species to another.

01:06:04.400 | You can, you know, there is an abstract version

01:06:06.480 | of knowledge that can be passed down.

01:06:08.480 | It doesn't have to be, you know, genetics.

01:06:10.760 | It doesn't have to be, you know,

01:06:12.200 | you don't have to apprentice the next species,

01:06:14.240 | you know, the next generation of birds to the previous one

01:06:16.820 | to show them how something works.

01:06:18.800 | There is this abstracted version of knowledge

01:06:21.640 | that can be kind of passed down.

01:06:23.500 | Now that, you know, it relies on,

01:06:26.120 | it still tends to rely because language is fuzzy.

01:06:29.280 | It does tend to rely on the fact that, you know,

01:06:31.360 | if we look at the, you know, some ancient language

01:06:34.820 | that where we don't have a chain of translations from it

01:06:37.400 | until what we have today,

01:06:39.080 | we may not understand that ancient language.

01:06:41.880 | And we may not understand, you know,

01:06:43.200 | its concepts may be different

01:06:44.600 | from the ones that we have today.

01:06:46.240 | We still have to have something of a chain,

01:06:48.240 | but it is something where we can realistically expect

01:06:51.060 | to communicate abstract ideas.

01:06:53.320 | And that's, you know, that's one of the big,

01:06:55.160 | big roles of a language.

01:06:57.000 | I think, you know, in,

01:07:00.840 | it's, you know, that's been this ability

01:07:04.360 | to sort of concretify abstract things

01:07:09.160 | is what language has provided.

01:07:11.360 | - Do you see natural language and thought as the same?

01:07:14.280 | The stuff that's going inside your mind?

01:07:16.240 | - Well, that's been a long debate in philosophy.

01:07:20.440 | - It seems to be become more important now

01:07:22.440 | when we think about how intelligent GPT is.

01:07:27.440 | - Whatever that means.

01:07:28.880 | - Whatever that means,

01:07:29.720 | but it seems like the stuff that's going on

01:07:31.240 | in the human mind seems something like intelligence.

01:07:34.280 | - Yeah. - And it's language.

01:07:35.120 | - But we call it intelligence.

01:07:36.440 | - Yeah, we call it, well, yes.

01:07:38.360 | And so you start to think of, okay,

01:07:40.040 | what's the relationship between thought,

01:07:43.440 | the language of thought, the laws of thought,

01:07:45.440 | the laws of, then the words like reasoning,

01:07:49.120 | and the laws of language,

01:07:51.200 | and how that has to do with computation,

01:07:53.480 | which seems like a more rigorous,

01:07:55.080 | precise ways of reasoning.

01:07:57.520 | - Right, which are beyond human.

01:07:59.520 | I mean, much of what computers do, humans do not do.

01:08:03.400 | I mean, you might say--

01:08:04.520 | - Humans are a subset, presumably.

01:08:07.200 | - Yes. - Hopefully.

01:08:08.040 | - Yes, yes, right.

01:08:10.680 | You know, you might say,

01:08:12.480 | who needs computation when we have large language models?

01:08:15.440 | Large language models can just,

01:08:17.000 | eventually you'll have a big enough neural net,

01:08:18.400 | it can do anything.

01:08:19.560 | But they're really doing the kinds of things

01:08:22.000 | that humans quickly do.

01:08:23.600 | And there are plenty of sort of formal things

01:08:25.480 | that humans never quickly do.

01:08:27.320 | For example, I don't know, you know,

01:08:29.720 | you can, some people can do mental arithmetic,

01:08:31.840 | they can do a certain amount of math in their minds.

01:08:34.560 | I don't think many people can run a program in their minds

01:08:37.760 | of any sophistication.

01:08:39.680 | It's just not something people do,

01:08:41.160 | it's not something people have even thought of doing,

01:08:43.560 | 'cause it's just, it's kind of not, you know,

01:08:46.680 | you can easily run it on a computer.

01:08:48.640 | - An arbitrary program.

01:08:50.120 | - Yeah. - Aren't we running

01:08:51.840 | specialized programs? - Yeah, yeah, yeah.

01:08:53.240 | But if I say to you, here's a Turing machine,

01:08:56.320 | you know, tell me what it does after 50 steps.

01:08:58.960 | And you're like, trying to think about that in your mind.

01:09:01.320 | That's really hard to do.

01:09:03.000 | It's not what people do.

01:09:04.400 | I mean-- - Well, in some sense,

01:09:06.000 | people program, they build a computer,

01:09:09.200 | they program it, just to answer your question

01:09:11.880 | about what the system does after 50 steps.

01:09:14.480 | I mean, humans build computers.

01:09:16.240 | - Yes, yes, yes, that's right.

01:09:18.680 | But they've created something which is then,

01:09:22.160 | you know, then when they run it,

01:09:23.920 | it's doing something different

01:09:25.440 | than what's happening in their minds.

01:09:26.880 | I mean, they've outsourced that piece of computation

01:09:30.720 | from something that is internally happening in their minds

01:09:34.360 | to something that is now a tool

01:09:35.800 | that's external to their mind.

01:09:37.120 | - So by the way, humans, to you,

01:09:39.440 | didn't invent computers, they discovered them.

01:09:41.960 | - They discovered computation.

01:09:44.840 | - Which-- - They invented

01:09:45.960 | the technology of computers.

01:09:52.240 | - The computer is just a kind of way

01:09:55.080 | to plug into this whole stream of computation.

01:09:58.560 | There's probably other ways.

01:10:00.520 | There's probably a lot of ways. - Well, for sure.

01:10:01.360 | I mean, the particular ways that we make computers

01:10:05.040 | out of semiconductors and electronics and so on,

01:10:07.560 | that's the particular technology stack we built.

01:10:10.560 | I mean, the story of a lot of what people try to do

01:10:12.520 | with quantum computing is finding different

01:10:15.200 | sort of underlying physical infrastructure

01:10:18.800 | for doing computation.

01:10:19.680 | You know, biology does lots of computation.

01:10:22.680 | It does it using an infrastructure

01:10:24.040 | that's different from semiconductors and electronics.

01:10:26.760 | It's a molecular scale sort of computational process

01:10:31.440 | that hopefully we'll understand more about.

01:10:33.400 | I have some ideas about understanding more about that.

01:10:36.200 | But that's another,

01:10:38.960 | it's another representation of computation.

01:10:41.120 | Things that happen in the physical universe

01:10:43.000 | at the level of these evolving hypergraphs and so on,

01:10:45.920 | that's another sort of implementation layer

01:10:48.800 | for this abstract idea of computation.

01:10:52.200 | - So if GPT or large language models are starting to form,

01:10:55.780 | starting to develop or implicitly understand

01:11:00.480 | the laws of language and thought,

01:11:02.360 | do you think they can be made explicit?

01:11:04.720 | - Yes.

01:11:05.840 | - How?

01:11:06.680 | - With a bunch of effort.

01:11:08.840 | I mean, it's like doing natural science.

01:11:11.440 | I mean, what is happening in natural science?

01:11:13.120 | You have the world that's doing all these complicated things

01:11:16.560 | and then you discover, you know, Newton's laws, for example.

01:11:19.040 | This is how motion works.

01:11:20.200 | This is the way that this particular

01:11:22.800 | sort of idealization of the world,

01:11:24.480 | this is how we describe it

01:11:25.640 | in a simple computationally reducible way.

01:11:28.320 | And I think it's the same thing here.

01:11:30.160 | It's there are sort of computationally reducible aspects

01:11:34.000 | of what's happening that you can get

01:11:36.400 | a kind of narrative theory for,

01:11:38.200 | just as we've got narrative theories in physics and so on.

01:11:44.840 | - Do you think it will be depressing or exciting

01:11:48.840 | when all the laws of thought are made explicit,

01:11:52.320 | human thought made explicit?

01:11:54.640 | - I think that once you understand

01:11:56.600 | computational reducibility,

01:11:58.300 | it's neither of those things.

01:12:01.360 | Because the fact is people say, for example,

01:12:04.040 | people will say, "Oh, but, you know, I have free will.

01:12:08.080 | "I kind of, you know, I operate in a way that is,

01:12:13.280 | "you know, they have the idea that they're doing something

01:12:17.740 | "that is sort of internal to them,

01:12:20.460 | "that they're figuring out what's happening."

01:12:22.520 | But in fact, we think there are laws of physics

01:12:25.720 | that ultimately determine, you know,

01:12:27.320 | every electrical impulse in a nerve and things like this.

01:12:32.320 | So you might say, "Isn't it depressing

01:12:35.380 | "that we are ultimately just determined

01:12:37.860 | "by the rules of physics, so to speak?"

01:12:40.640 | It's the same thing.

01:12:41.520 | It's at a higher level.

01:12:42.800 | It's like it's a shorter distance

01:12:45.520 | to get from kind of semantic grammar

01:12:48.800 | to the way that we might construct a piece of text

01:12:51.800 | than it is to get from individual nerve firings

01:12:54.720 | to how we construct a piece of text.

01:12:56.580 | But it's not fundamentally different.

01:12:58.400 | And by the way, as soon as we have this kind of level of,

01:13:01.480 | you know, this other level of description,

01:13:03.520 | it's kind of, it helps us to go even further.

01:13:05.800 | So we'll end up being able to produce

01:13:07.500 | more and more complicated kinds of things

01:13:11.040 | that just like when we, you know,

01:13:12.800 | if we didn't have a computer and we knew certain rules,

01:13:15.640 | we could write them down, we'd go a certain distance.

01:13:17.840 | But once we have a computer, we can go vastly further,

01:13:20.440 | and this is the same kind of thing.

01:13:22.280 | - You wrote a blog post titled,

01:13:23.680 | "What is Chad GPT doing and why does it work?"

01:13:26.720 | We've been talking about this,

01:13:27.760 | but can we just step back and linger on this question?

01:13:30.880 | What's Chad GPT doing?

01:13:33.600 | What are these, a bunch of billion parameters

01:13:38.840 | trained on a large number of words.

01:13:41.980 | Why does it seem to work again?

01:13:45.760 | Is it because to the point you made

01:13:48.320 | that there's laws of language

01:13:50.920 | that can be discovered by such a process?

01:13:52.920 | Is there something more to it?

01:13:53.760 | - Well, let's talk about sort of the low level

01:13:55.960 | of what Chad GPT is doing.

01:13:57.800 | I mean, ultimately you give it a prompt,

01:14:00.640 | it's trying to work out, you know,

01:14:02.520 | what should the next word be, right?

01:14:05.240 | - Which is wild.

01:14:06.920 | - Isn't that surprising to you

01:14:08.440 | that this kind of low level, dumb training procedure

01:14:13.440 | can create something syntactically correct first,

01:14:17.760 | and then semantically correct second?

01:14:19.440 | - You know, the thing that has been

01:14:20.960 | sort of a story of my life,

01:14:22.400 | is realizing that simple rules

01:14:25.760 | can do much more complicated things than you imagine.

01:14:28.420 | That something that starts simple

01:14:30.360 | and starts simple to describe,

01:14:32.240 | can grow a thing that is, you know,

01:14:35.080 | vastly more complicated than you can imagine.

01:14:37.920 | And honestly, it's taken me, I don't know,

01:14:40.200 | I've sort of been thinking about this now 40 years or so,

01:14:43.200 | and it always surprises me.

01:14:44.840 | I mean, even for example, in our physics project,

01:14:46.840 | sort of thinking about the whole universe

01:14:48.520 | growing from these simple rules,

01:14:50.240 | I still resist, because I keep on thinking,

01:14:53.000 | you know, how can something really complicated

01:14:54.800 | arise from something that simple?

01:14:56.760 | It just seems, you know, it seems wrong,

01:14:59.960 | but yet, you know, the majority of my life,

01:15:02.720 | I've kind of known from things I've studied

01:15:05.240 | that this is the way things work.

01:15:06.680 | So yes, it is wild that it's possible

01:15:08.840 | to write a word at a time

01:15:10.920 | and produce a coherent essay, for example.

01:15:13.360 | But it's worth understanding kind of how that's working.

01:15:15.280 | I mean, it's kind of like, if it was going to say,

01:15:18.440 | you know, the cat sat on the, what's the next word?

01:15:22.580 | Okay, so how does it figure out the next word?

01:15:24.720 | Well, it's seen a trillion words written on the internet,

01:15:27.400 | and it's seen the cat sat on the floor,

01:15:30.440 | the cat sat on the sofa, the cat sat on the whatever.

01:15:35.040 | So it's minimal thing to do is just say,

01:15:37.960 | let's look at what we saw on the internet.

01:15:39.840 | We saw, you know, 10,000 examples of the cat sat on the,

01:15:44.840 | what was the most probable next word?

01:15:47.300 | Let's just pick that out and say that's the next word.

01:15:50.600 | And that's kind of what it, at some level, is trying to do.

01:15:54.460 | Now, the problem is there isn't enough text on the internet

01:15:57.920 | to, if you have a reasonable length of prompt,

01:16:02.320 | that specific prompt will never have occurred

01:16:04.800 | on the internet.

01:16:05.960 | And as you kind of go further,

01:16:08.600 | there just won't be a place where you could have trained,

01:16:11.960 | you know, where you could just worked out probabilities

01:16:14.220 | from what was already there.

01:16:16.320 | You know, like if you say two plus two,

01:16:17.840 | there'll be a zillion examples of two plus two equaling four

01:16:21.400 | and a very small number of examples of two plus two

01:16:23.400 | equals five and so on.

01:16:25.200 | And you can pretty much know what's going to happen.

01:16:27.280 | So then the question is, well,

01:16:28.840 | if you can't just work out from examples,

01:16:31.880 | what's going to happen, just no probabilistic,

01:16:34.160 | for example, what's going to happen,

01:16:36.000 | you have to have a model.

01:16:37.320 | And there's kind of an idea,

01:16:38.720 | this idea of making models of things is an idea that really,

01:16:42.480 | I don't know, I think Galileo probably was one of the first

01:16:44.760 | people who sort of worked this out.

01:16:46.560 | I mean, it's kind of like, you know,

01:16:48.600 | I think I gave an example of that little book I wrote

01:16:51.960 | about Chachi B'tee where it's kind of like, you know,

01:16:54.920 | Galileo was dropping cannonballs off the different floors

01:16:58.480 | of the Tower of Pisa.

01:17:00.360 | And it's like, okay, you drop a cannonball off this floor,

01:17:02.920 | you drop a cannonball off this floor,

01:17:04.900 | you miss floor five or something for whatever reason,

01:17:07.760 | but you know the time it took the cannonball

01:17:09.640 | to fall to the ground from floors one, two, three, four,

01:17:12.600 | six, seven, eight, for example,

01:17:15.040 | then the question is, can you work out,

01:17:17.120 | can you make a model which figures out

01:17:19.840 | how long did it take the ball,

01:17:21.000 | how long would it take the ball to fall to the ground

01:17:23.440 | from the floor you didn't explicitly measure?

01:17:25.720 | And the thing Galileo realized is that you can use math,

01:17:29.440 | you can use mathematical formulas to make a model

01:17:32.680 | for how long it will take the ball to fall.

01:17:35.720 | So now the question is, well, okay,

01:17:37.600 | you want to make a model for, for example,

01:17:40.040 | something much more elaborate,

01:17:41.620 | like you've got this arrangement of pixels

01:17:44.400 | and is this arrangement of pixels an A or a B?

01:17:47.440 | Does it correspond to something we'd recognize

01:17:49.200 | as an A or B?

01:17:50.440 | And you can make a similar kind, you know,

01:17:52.480 | each pixel is like a parameter in some equation

01:17:55.520 | and you could write down this giant equation

01:17:57.760 | where the answer is either, you know, A or, you know,

01:18:00.000 | one or two, A or B.

01:18:02.000 | And the question is then what kind of a model

01:18:04.960 | successfully reproduces the way that we humans

01:18:08.840 | would conclude that this is an A and this is a B?

01:18:12.400 | You know, if there's a complicated extra tail

01:18:15.240 | on the top of the A,

01:18:16.640 | would we then conclude something different?

01:18:18.560 | What is the type of model that maps well

01:18:21.600 | into the way that we humans make distinctions about things?

01:18:25.000 | And the big kind of meta discovery

01:18:26.880 | is neural nets are such a model.

01:18:29.120 | It's not obvious they would be such a model.

01:18:31.080 | It could be that human distinctions are not captured.

01:18:34.720 | You know, we could try searching around for a type of model

01:18:37.840 | that could be a mathematical model,

01:18:39.240 | it could be some model based on something else

01:18:41.240 | that captures kind of typical human distinctions

01:18:43.620 | about things.

01:18:44.720 | It turns out this model that actually is very much

01:18:48.500 | the way that we think the architecture of brains works,

01:18:51.660 | that perhaps not surprisingly,

01:18:53.980 | that model actually corresponds

01:18:55.860 | to the way we make these distinctions.

01:18:57.780 | And so, you know, the core next point is that

01:19:01.220 | the kind of model, this neural net model,

01:19:03.580 | makes sort of distinctions and generalizes things

01:19:07.980 | in sort of the same way that we humans do it.

01:19:10.580 | And that's why when you say, you know,

01:19:12.660 | the cat sat on the green blank,

01:19:15.940 | even though it never, it didn't see many examples

01:19:18.220 | of the cat sat on the green whatever,

01:19:20.260 | it can make a, or the aardvark sat on the green whatever,

01:19:24.620 | I'm sure that particular sentence

01:19:26.580 | does not occur on the internet.

01:19:28.420 | And so it has to make a model that concludes what,

01:19:32.220 | you know, it has to kind of generalize

01:19:33.940 | from what it's, from the actual examples that it's seen.

01:19:37.220 | And so, you know, that's the fact is that neural nets

01:19:41.320 | generalize in the same kind of way that we humans do.

01:19:44.380 | If we were, you know, the aliens might look

01:19:47.540 | at our neural net generalizations and say, that's crazy.

01:19:50.260 | You know, that thing, when you put that extra little dot

01:19:53.160 | on the A, that isn't an A anymore.

01:19:55.300 | That's, you know, that messed the whole thing up.

01:19:57.960 | But for us humans, we make distinctions

01:20:00.700 | which seem to correspond to the kinds of distinctions

01:20:03.060 | that neural nets make.

01:20:04.460 | So then, you know, the thing that is just amazing to me

01:20:07.320 | about ChachiBT is how similar the structure it has is

01:20:11.580 | to the very original way people imagine neural nets

01:20:14.100 | might work back in 1943.

01:20:16.660 | And, you know, there's a lot of detailed engineering,

01:20:19.420 | you know, great cleverness, but it's really the same idea.

01:20:22.820 | And in fact, even the sort of elaborations of that idea

01:20:26.080 | where people said, let's put in some actual

01:20:27.900 | particular structure to try and make the neural net

01:20:30.580 | more elaborate, to be very clever about it.

01:20:33.020 | Most of that didn't matter.

01:20:34.660 | I mean, there's some things that seem to, you know,

01:20:36.600 | when you train this neural net, you know, the one thing,

01:20:39.920 | this kind of transformer architecture, this attention idea,

01:20:43.100 | that really has to do with, does every one of these neurons

01:20:47.240 | connect to every other neuron, or is it somehow

01:20:50.260 | causally localized, so to speak?

01:20:52.020 | Does it like we're making a sequence of words

01:20:55.020 | and the words depend on previous words,

01:20:57.300 | rather than just everything can depend on everything.

01:21:00.180 | And that seems to be important in just organizing things

01:21:03.060 | so that you don't have a sort of a giant mess.

01:21:05.900 | But the thing, you know, the thing worth understanding

01:21:07.900 | about what is ChachiBT in the end?

01:21:09.860 | I mean, what is a neural net in the end?

01:21:11.260 | A neural net in the end is each neuron has a,

01:21:15.260 | it's taking inputs from a bunch of other neurons.

01:21:18.460 | It's eventually, it's going to have a numerical value.

01:21:23.460 | It's going to compute some number, and it's saying,

01:21:27.020 | I'm gonna look at the neurons above me.

01:21:29.460 | It's kind of a series of layers.

01:21:31.500 | It's gonna look at the neurons above me,

01:21:33.460 | and it's going to say, what are the values

01:21:34.940 | of all those neurons?

01:21:36.220 | Then it's gonna add those up and multiply them

01:21:38.260 | by these weights, and then it's going to apply

01:21:40.920 | some function that says if it's bigger than zero

01:21:43.300 | or something, then make it one,

01:21:44.620 | or otherwise make it zero,

01:21:46.300 | or some slightly more complicated function.

01:21:48.780 | You know very well how this works.

01:21:50.540 | - It's a giant equation with a lot of variables.

01:21:54.140 | You mentioned figuring out where the ball falls

01:21:57.060 | when you don't have data on the fourth floor.

01:21:59.300 | This, the equation here is not as simple as--

01:22:03.740 | - Right, it's an equation with 175 billion terms.

01:22:07.140 | - And it's quite surprising that in some sense,

01:22:10.540 | a simple procedure of training such an equation

01:22:15.540 | can lead to a good representation of natural language.

01:22:19.980 | - Right, the real issue is, you know,

01:22:22.380 | this architecture of a neural net,

01:22:23.980 | where what's happening is, you know,

01:22:26.180 | you've turned, so neural nets always just deal with numbers.

01:22:30.940 | And so, you know, you've turned the sentence

01:22:33.220 | that you started with into a bunch of numbers.

01:22:34.780 | Like, let's say by mapping, you know,

01:22:36.700 | each word of the 50,000 words in English,

01:22:38.740 | you just map each word or each part of a word

01:22:40.640 | into some number.

01:22:42.080 | They feed all those numbers in,

01:22:44.380 | and then the thing is going to,

01:22:46.260 | and then those numbers just go into the values

01:22:48.700 | of these neurons, and then what happens is,

01:22:51.680 | it's just rippling down, going layer to layer,

01:22:54.440 | until it gets to the end.

01:22:55.740 | I think "Chat GPG" has about 400 layers.

01:22:58.260 | And you're just, you know, it just goes once through.

01:23:00.980 | It just, every new word it's gonna compute,

01:23:03.740 | just says, here are the numbers from the words before.

01:23:06.820 | Let's compute the, what does it compute?

01:23:09.020 | It computes the probabilities that it estimates

01:23:11.960 | for each of the possible 50,000 words that could come next.

01:23:15.160 | And then it decides,

01:23:16.580 | sometimes it will use the most probable word,

01:23:18.420 | sometimes it will use not the most probable word.

01:23:20.460 | It's an interesting fact

01:23:22.060 | that there's this so-called temperature parameter,

01:23:24.500 | which, you know, at temperature zero,

01:23:26.580 | it's always using the most probable word

01:23:28.420 | that it estimated was the most probable thing to come next.

01:23:32.460 | You know, if you increase the temperature,

01:23:34.260 | it'll be more and more kind of random

01:23:35.980 | in its selection of words.

01:23:36.980 | It'll go down to lower and lower probability words.

01:23:39.420 | Thing I was just playing with actually recently

01:23:41.580 | was the transition that happens

01:23:43.000 | as you increase the temperature.

01:23:44.540 | The thing goes bonkers at a particular, you know,

01:23:47.260 | sometimes at a particular temperature,

01:23:49.420 | I think maybe about 1.2 is the thing I was noticing

01:23:52.820 | from yesterday, actually,

01:23:54.900 | that, you know, usually it's giving reasonable answers.

01:23:57.500 | And then at that temperature with some probability,

01:24:01.300 | it just starts spouting nonsense.

01:24:04.900 | And, you know, nobody knows why this happens.

01:24:07.300 | I mean, it's, and by the way, I mean,

01:24:10.740 | the thing to understand is it's putting down

01:24:12.520 | one word at a time,

01:24:14.140 | but the outer loop of the fact that it says,

01:24:17.100 | okay, I put down a word.

01:24:18.520 | Now let's take the whole thing I wrote so far.

01:24:20.540 | Let's feed that back in.

01:24:21.940 | Let's put down another word.

01:24:23.580 | That outer loop, which seems almost trivial,

01:24:26.060 | is really important to the operation of the thing.

01:24:28.560 | And for example, one of the things that is kind of funky

01:24:31.900 | is it'll give an answer.

01:24:33.820 | And you say to it, is that answer correct?

01:24:36.540 | And it'll say, no.

01:24:38.620 | And why has that happened?

01:24:39.820 | - It's fascinating, right?

01:24:40.660 | - Right, why can it do that?

01:24:41.620 | Well, the answer is because it is going one word at a time,

01:24:44.580 | sort of forwards.

01:24:45.940 | And it didn't, you know,

01:24:47.340 | it came along with some sort of chain of thought in a sense,

01:24:51.500 | and it came up with completely the wrong answer.

01:24:54.300 | But as soon as you feed it,

01:24:55.780 | the whole thing that it came up with,

01:24:59.060 | it immediately knows that that isn't right.

01:25:01.140 | It immediately can recognize that was a, you know,

01:25:03.580 | a bad syllogism or something, and can see what happened.

01:25:06.980 | Even though, as it was being led down this garden path,

01:25:09.860 | so to speak, it didn't, it came to the wrong place.

01:25:13.100 | - But it's fascinating that this kind of procedure

01:25:14.980 | converges to something that forms

01:25:18.160 | a pretty good compressed representation

01:25:21.500 | of language on the internet.

01:25:24.140 | - Yeah.

01:25:24.980 | - That's quite--

01:25:25.980 | - Right, right, right.

01:25:26.820 | - I'm not sure what to make of it.

01:25:29.060 | - Well, look, I think, you know,

01:25:30.700 | there are many things we don't understand, okay?

01:25:32.580 | So for example, you know, 175 billion weights,

01:25:36.340 | it's maybe about a trillion bytes of information,

01:25:39.580 | which is very comparable to the training set that was used.

01:25:42.900 | And, you know, why that, why kind of,

01:25:47.140 | it sort of stands to some kind of reason

01:25:49.460 | that the number of weights in the neural net,

01:25:51.580 | I don't know, I can't really argue that.

01:25:53.180 | I can't really give you a good, you know,

01:25:56.100 | in a sense, the very fact that, you know,

01:25:59.220 | insofar as there are definite rules of what's going on,

01:26:01.820 | you might expect that eventually

01:26:04.300 | we'll have a much smaller neural net

01:26:06.320 | that will successfully capture what's happening.

01:26:08.820 | I don't think the best way to do it

01:26:10.540 | is probably a neural net.

01:26:12.020 | I think a neural net is what you do

01:26:13.340 | when you don't know any other way to structure the thing.

01:26:16.260 | And it's a very good thing to do

01:26:17.560 | if you don't know any other way to structure the thing.

01:26:19.460 | And for the last 2000 years,

01:26:20.700 | we haven't known any other way to structure it.

01:26:22.780 | So this is a pretty good way to start.

01:26:24.620 | But that doesn't mean you can't find,

01:26:26.740 | sort of, in a sense, more symbolic rules

01:26:28.780 | for what's going on that, you know,

01:26:30.940 | much of which will then be,

01:26:33.260 | you can kind of get rid of much of the structure

01:26:35.940 | of the neural nets and replace it by things

01:26:38.300 | which are sort of pure steps of computation, so to speak,

01:26:42.180 | sort of with neural net stuff around the edges.

01:26:44.940 | And that becomes just a, you know,

01:26:46.300 | it's just a much simpler way to do it.

01:26:47.900 | - So the neural net, you hope, will reveal to us

01:26:52.180 | good symbolic rules that make the need of the neural net

01:26:56.180 | less and less and less.

01:26:57.340 | - Right.

01:26:58.260 | And there will still be some stuff that's kind of fuzzy,

01:27:00.940 | just like, you know, there are things that,

01:27:03.180 | it's like this question of what can we formalize?

01:27:05.540 | What can we turn into computational language?

01:27:08.580 | What is just sort of, oh, it happens that way

01:27:10.700 | just because brains are set up that way?

01:27:13.060 | - What do you think are the limitations

01:27:16.340 | of large language models, just to make it explicit?

01:27:20.260 | - Well, I mean, I think that deep computation

01:27:22.180 | is not what large language models do.

01:27:24.060 | I mean, that's just, it's a different kind of thing.

01:27:26.500 | You know, the outer loop of a large language model,

01:27:29.060 | if you're trying to do many steps in a computation,

01:27:32.020 | the only way you get to do that right now

01:27:33.780 | is by spooling out, you know, all the,

01:27:36.180 | the whole chain of thought as a bunch of words, basically.

01:27:39.540 | And, you know, you can make a Turing machine out of that,

01:27:42.140 | if you want to.

01:27:42.980 | I just was doing that construction.

01:27:45.180 | You know, in principle, you can make an arbitrary

01:27:47.580 | computation by just spooling out the words,

01:27:50.260 | but it's a bizarre and inefficient way to do it.

01:27:54.420 | But it's something where the, you know,

01:27:57.860 | I think that's, you know, sort of the deep computation.

01:28:02.860 | It's really what humans can do quickly,

01:28:07.900 | large language models will probably be able to do well.

01:28:10.580 | Anything that you can do kind of off the top of your head

01:28:12.820 | type thing is really, you know,

01:28:15.460 | is good for large language models.

01:28:17.260 | And the things you do off the top of your head,

01:28:18.660 | you may not get them always right, but, you know,

01:28:21.700 | you'll, it's thinking it through the same way we do.

01:28:26.140 | - But I wonder if there's an automated way to do something

01:28:28.620 | that humans do well, much faster to where it like loops.

01:28:33.100 | So generate arbitrary large code bases

01:28:36.660 | of Wolfram language, for example.

01:28:39.660 | - Well, the question is, what does he,

01:28:40.820 | what do you want the code base to do?

01:28:42.660 | - Escape control and take over the world.

01:28:48.060 | - Okay.

01:28:49.020 | So, you know, the thing is when people say, you know,

01:28:52.180 | we want to build this giant thing, right?

01:28:55.580 | A giant piece of computational language.

01:28:58.060 | In a sense, it's sort of a failure of computational language

01:29:01.740 | if the thing you have to build, in other words,

01:29:03.620 | if we have a description, if you have a small description,

01:29:07.460 | that's the thing that you represent

01:29:09.540 | in computational language.

01:29:11.220 | And then the computer can compute from that.

01:29:14.020 | - Yes.

01:29:14.860 | - So in a sense in, you know,

01:29:16.820 | when as soon as you're giving a description that, you know,

01:29:20.220 | if you have to somehow make that description something,

01:29:23.580 | you know, definite, something formal,

01:29:26.140 | and once, and to say, to say, okay,

01:29:28.820 | I'm gonna give this piece of natural language,

01:29:31.020 | and then it's gonna split out this giant formal structure,

01:29:35.320 | that in a sense that doesn't really make sense

01:29:39.020 | because except in so far as that piece of natural language

01:29:42.820 | kind of plugs into what we socially know, so to speak,

01:29:46.220 | plugs into kind of our corpus of knowledge,

01:29:48.780 | then, you know, that's a way we're capturing

01:29:50.500 | a piece of that corpus of knowledge,

01:29:51.620 | but hopefully we will have done that

01:29:53.340 | in computational language.

01:29:54.860 | How do you make it do something that's big?

01:29:57.620 | Well, you know, you have to have a way

01:29:59.140 | to describe what you want.

01:30:00.540 | - Okay, I can make it more explicit if you want.

01:30:02.740 | How about, I just pop into my head,

01:30:05.060 | iterate through all the members of Congress

01:30:09.900 | and figure out how to convince them

01:30:12.820 | that they have to let me,

01:30:17.820 | this meaning the system, become president,

01:30:21.020 | pass all the laws that allows AI systems

01:30:23.140 | to take control and be the president, I don't know.

01:30:26.060 | So that's a very explicit, like,

01:30:27.740 | figure out the individual life story

01:30:29.780 | of each congressman, each senator, anybody, I don't know,

01:30:34.100 | what's required to really kind of pass legislation

01:30:37.020 | and figure out how to control them and manipulate them,

01:30:40.140 | get all the information.

01:30:41.380 | What would be the biggest fear of this congressman

01:30:45.060 | and in such a way that you can take action on it

01:30:49.580 | in the digital space?

01:30:50.940 | So maybe threaten the destruction of reputation

01:30:53.420 | or something like this.

01:30:54.700 | - Right, if I can describe what I want,

01:30:57.660 | you know, to what extent can a large language model

01:30:59.740 | automate that?

01:31:01.180 | - With the help of concretization

01:31:05.260 | of something like Wolfram Language.

01:31:07.660 | That makes it more, yeah, grounded.

01:31:10.780 | It can go rather a long way.

01:31:12.460 | - I'm also surprised how quickly I was able to generate

01:31:15.380 | - Yeah, yeah, right.

01:31:16.220 | - A quick attack.

01:31:17.940 | - That's, yeah, you know.

01:31:20.060 | - I swear I did not think about this before,

01:31:22.700 | and it's funny how quickly,

01:31:24.340 | which is a very concerning thing,

01:31:25.540 | 'cause that probably, this idea,

01:31:27.180 | will probably do quite a bit of damage,

01:31:28.820 | and there might be a very large number

01:31:31.140 | of other such ideas.

01:31:32.300 | - Well, I'll give you a much more benign version

01:31:34.340 | of that idea, okay?

01:31:35.860 | You're gonna make an AI tutoring system,

01:31:38.380 | and that's a benign version of what you're saying,

01:31:42.020 | is I want this person to understand this point.

01:31:45.940 | - Yes.

01:31:46.780 | - You're essentially doing machine learning

01:31:48.500 | where the loss function,

01:31:51.500 | the thing you're trying to get to,

01:31:52.900 | is get the human to understand this point.

01:31:55.740 | And when you do a test on the human,

01:31:58.020 | that they, yes, they correctly understand

01:31:59.820 | how this or that works.

01:32:01.460 | And I am confident that, you know,

01:32:05.340 | sort of a large language model type technology

01:32:07.780 | combined with computational language

01:32:09.700 | is going to be able to do pretty well

01:32:12.700 | at teaching us humans things.

01:32:15.420 | And it's gonna be an interesting phenomenon

01:32:17.420 | because, you know, sort of individualized teaching

01:32:20.940 | is a thing that has been kind of a goal for a long time.

01:32:25.340 | I think we're gonna get that,

01:32:26.260 | and I think more, you know,

01:32:28.100 | it has many consequences for, you know,

01:32:30.220 | like just, you know, if you know me,

01:32:33.660 | as in if you, the AI, know me, tell me,

01:32:37.180 | I'm about to do this thing,

01:32:38.780 | what are the three things I need to know, you know,

01:32:42.300 | given what I already know, you know,

01:32:44.940 | what's, let's say I'm looking at some paper or something,

01:32:48.940 | right, it's like there's a version

01:32:51.700 | of the summary of that paper

01:32:54.020 | that is optimized for me, so to speak.

01:32:56.340 | And where it really is,

01:32:58.140 | and I think that's really going to work.

01:32:59.580 | - It could understand the major gaps in your knowledge

01:33:03.220 | that if filled would actually give you

01:33:07.100 | a deeper understanding of the topic here.

01:33:08.860 | - Right, and that's a, you know, that's an important thing

01:33:12.020 | because it really changes, actually, I think, you know,

01:33:14.540 | when you think about education and so on,

01:33:16.380 | it really changes kind of what's worth doing,

01:33:19.180 | what's not worth doing and so on.

01:33:20.460 | It makes, you know, I know in my life

01:33:23.100 | I've learned lots of different fields,

01:33:25.140 | and, you know, so I, I don't know,

01:33:27.220 | every time I'm always thinking

01:33:28.540 | this is the one that's going to,

01:33:29.900 | I'm not gonna be able to learn,

01:33:31.500 | but turns out sort of there are sort of meta methods

01:33:34.180 | for learning these things in the end.

01:33:36.780 | And, you know, I think this, this idea

01:33:39.100 | that it becomes easier to, you know,

01:33:41.980 | it becomes easier to be fed knowledge, so to speak.

01:33:45.420 | And it becomes, you know,

01:33:46.860 | if you need to know this particular thing,

01:33:49.020 | you can, you know, you can get taught it

01:33:52.460 | in an efficient way.

01:33:54.180 | It's something I think is sort of an interesting feature.

01:33:56.780 | And I think it makes the, you know,

01:34:00.020 | things like the value of big towers of specialized knowledge

01:34:05.100 | become less significant compared to the kind of

01:34:08.100 | meta knowledge of sort of understanding

01:34:12.020 | kind of the big picture

01:34:13.460 | and being able to connect things together.

01:34:15.340 | I think that, you know, there's been this huge trend

01:34:17.260 | of let's be more and more specialized

01:34:19.500 | because we have to, you know,

01:34:20.820 | we have to sort of ascend these towers of knowledge.

01:34:24.140 | But by the time you can get, you know,

01:34:26.420 | more automation of being able to get to that place

01:34:29.860 | on the tower without having to go through all those steps,

01:34:32.900 | I think it sort of changes that picture.

01:34:35.060 | - Interesting, so your intuition is that

01:34:36.820 | in terms of the collective intelligence of the species

01:34:41.700 | and the individual minds that make up that collective,

01:34:44.440 | there'll be more,

01:34:47.260 | there will trend towards being generalists

01:34:51.020 | and being kind of philosophers.

01:34:53.700 | - That's what I think.

01:34:54.540 | I think that's where the humans are gonna be useful.

01:34:56.620 | I think that a lot of these kind of,

01:34:59.860 | the drilling, the mechanical working out of things

01:35:04.580 | is much more automatable.

01:35:06.220 | It's much more AI territory, so to speak.

01:35:09.500 | - No more PhDs.

01:35:11.380 | - Well, that's interesting, yes.

01:35:13.940 | I mean, you know, the kind of the specialization,

01:35:16.900 | this kind of tower of specialization,

01:35:19.220 | which has been a feature of, you know,

01:35:20.980 | we've accumulated lots of knowledge in our species.

01:35:23.820 | And, you know, in a sense,

01:35:25.940 | every time we have a kind of automation,

01:35:30.260 | a building of tools,

01:35:31.780 | it becomes less necessary to know that whole tower.

01:35:34.660 | And it becomes something where you can just use a tool

01:35:36.740 | to get to the top of that tower.

01:35:38.500 | I think that, you know, the thing that is ultimately,

01:35:42.140 | you know, when we think about,

01:35:43.420 | okay, what do the AIs do versus what do the humans do?

01:35:46.500 | It's like AIs, you tell them, you say,

01:35:49.060 | go achieve this particular objective.

01:35:51.740 | Okay, they can maybe figure out a way

01:35:53.260 | to achieve that objective.

01:35:54.700 | We say, what objective would you like to achieve?

01:35:57.740 | The AI has no intrinsic idea of that.

01:36:00.740 | It's not a defined thing.

01:36:02.640 | That's a thing which has to come from some other,

01:36:06.580 | you know, some other entity.

01:36:07.580 | And insofar as we are in charge, so to speak,

01:36:10.340 | or whatever it is,

01:36:11.420 | and our kind of web of society and history and so on

01:36:15.340 | is the thing that is defining what objective

01:36:18.220 | we want to go to,

01:36:19.600 | that's, you know, that's a thing that we humans

01:36:23.060 | are necessarily involved in, so to speak.

01:36:25.700 | - To push back a little bit,

01:36:27.140 | don't you think that GPT, future versions of GPT

01:36:30.980 | would be able to give a good answer to

01:36:33.460 | what objective would you like to achieve?

01:36:35.500 | - From on what basis?

01:36:38.280 | I mean, if they say,

01:36:39.820 | look, here's the terrible thing that could happen, okay?

01:36:42.340 | They're taking the average of the internet,

01:36:44.600 | and they're saying, you know,

01:36:46.540 | from the average of the internet,

01:36:47.900 | what do people want to do?

01:36:49.340 | - Well, that's the Elon Musk adage of

01:36:52.220 | the most entertaining outcome is the most likely.

01:36:55.980 | - Okay, I haven't heard that one from him, yeah.

01:36:58.620 | - That could be one objective,

01:37:02.780 | is maximize global entertainment.

01:37:07.780 | The dark version of that is drama.

01:37:10.800 | The good version of that is fun.

01:37:13.540 | - Right, so I mean, this question of what,

01:37:16.040 | you know, if you say to the AI,

01:37:19.460 | what does the species want to achieve?

01:37:23.860 | - Yes. - Okay.

01:37:25.700 | - There'll be an answer, right?

01:37:27.480 | - There'll be an answer.

01:37:28.420 | It'll be what the average of the internet

01:37:30.220 | says the species wants to achieve.

01:37:31.900 | - I think you're using the word average

01:37:35.780 | very loosely there, right?

01:37:37.620 | So I think the answers will become more and more interesting

01:37:42.620 | as these language models are trained better and better.

01:37:45.980 | - No, but I mean, in the end,

01:37:47.860 | it's a reflection back of what we've already said.

01:37:51.080 | - Yes, but there's a deeper wisdom

01:37:54.300 | to the collective intelligence,

01:37:55.700 | presumably, than each individual.

01:37:57.980 | - Maybe.

01:37:58.820 | - Isn't that what we're trying to do as a society?

01:38:00.780 | - Well, I mean, that's an important,

01:38:05.180 | that's an interesting question.

01:38:06.220 | I mean, in so far as some of us work on trying to innovate

01:38:11.220 | and figure out new things and so on,

01:38:13.700 | it is sometimes, it's a complicated interplay

01:38:16.660 | between sort of the individual doing the crazy thing

01:38:19.900 | off in some spur, so to speak, versus the collective

01:38:24.620 | that's trying to do sort of the high inertia average thing.

01:38:29.620 | And it's, you know, sometimes the collective, you know,

01:38:33.300 | is bubbling up things that are interesting,

01:38:36.180 | and sometimes it's pulling down kind of the attempt

01:38:39.060 | to make this kind of innovative direction.

01:38:41.620 | - Well, don't you think the large language models

01:38:43.220 | would see beyond that simplification,

01:38:45.260 | will say maybe intellectual and career diversity

01:38:48.500 | is really important.

01:38:49.380 | So you need the crazy people on the outlier,

01:38:51.900 | on the outskirts.

01:38:53.060 | And so like the actual, what's the purpose

01:38:56.460 | of this whole thing is to explore

01:39:00.660 | through this kind of dynamics that we've been using

01:39:03.900 | as a human civilization, which is most of us

01:39:05.580 | focus on one thing, and then there's the crazy people

01:39:07.860 | on the outskirts doing the opposite of that one thing,

01:39:10.780 | and you kind of pull the whole society together.

01:39:13.440 | There's the mainstream science,

01:39:15.380 | and then there's the crazy science,

01:39:17.860 | and it's just been the history of human civilization.

01:39:20.420 | And maybe the AI system will be able to see that.

01:39:23.100 | And the more and more impressed we are

01:39:24.580 | by a language model telling us this,

01:39:27.460 | the more control we'll give it to it,

01:39:30.460 | and the more we'll be willing to let it run our society.

01:39:35.160 | And hence, there's this kind of loop

01:39:37.700 | where the society could be manipulated

01:39:40.780 | to let the AI system run it.

01:39:42.580 | - Right, well, I mean, look, one of the things

01:39:44.660 | that's sort of interesting is we might say

01:39:48.100 | we always think we're making progress,

01:39:50.440 | but yet, if in a sense by saying,

01:39:54.500 | let's take what already exists and use that

01:39:58.220 | as a model for what should exist,

01:40:00.860 | then it's interesting that, for example,

01:40:04.140 | many religions have taken that point of view.

01:40:06.260 | There is a sacred book that got written at time X,

01:40:09.620 | and it defines how people should act for all future time.

01:40:13.820 | And it's a model that people have operated with.

01:40:18.540 | And in a sense, this is a version of that kind of statement.

01:40:23.500 | It's like, take the 2023 version

01:40:26.740 | of sort of how the world has exposed itself

01:40:30.860 | and use that to define what the world

01:40:33.580 | should do in the future.

01:40:34.660 | - But it's an imprecise definition, right?

01:40:37.020 | Because just like with religious texts and with GPT,

01:40:41.020 | the human interpretation of what GPT says

01:40:43.820 | will be the perturbation in the system.

01:40:50.100 | It'll be the noise.

01:40:51.320 | It'd be full of uncertainty.

01:40:53.020 | It's not like Chad GPT will tell you exactly what to do.

01:40:57.460 | It'll tell you a narrative of what,

01:41:00.100 | it's like a turn the other cheek kind of narrative, right?

01:41:04.980 | That's not a fully instructive narrative.

01:41:07.460 | - Well, until the AIs control all the systems in the world.

01:41:11.300 | They will be able to very precisely tell you what to do.

01:41:14.060 | - Well, they'll do what they,

01:41:15.940 | they'll just do this or that thing.

01:41:18.220 | And not only that,

01:41:20.140 | they'll be auto suggesting to each person,

01:41:22.380 | do this next, do that next.

01:41:24.500 | So I think it's a slightly more prescriptive situation

01:41:28.600 | than one has typically seen.

01:41:30.740 | But I think this whole question of sort of

01:41:34.500 | what's left for the humans, so to speak,

01:41:37.660 | to what extent do we, you know,

01:41:41.340 | this idea that there is an existing kind of corpus

01:41:45.540 | of purpose for humans defined by what's on the internet

01:41:48.420 | and so on, that's an important thing.

01:41:51.280 | But then the question of sort of,

01:41:53.620 | as we explore what we can think of

01:41:56.260 | as the computational universe,

01:41:57.460 | as we explore all these different possibilities

01:41:59.380 | for what we could do,

01:42:00.260 | all these different inventions we could make,

01:42:02.260 | all these different things,

01:42:03.620 | the question is, which ones do we choose to follow?

01:42:06.460 | Those choices are the things that, in a sense,

01:42:09.860 | if the humans want to still have kind of human progress,

01:42:14.860 | that's what we get to make those choices, so to speak.

01:42:19.380 | In other words, there's this idea,

01:42:22.400 | if you say, let's take the kind of what exists today

01:42:27.400 | and use that as the determiner

01:42:31.020 | of all of what there is in the future,

01:42:33.540 | the thing that is sort of the opportunity for humans

01:42:36.100 | is there will be many possibilities thrown up.

01:42:38.940 | There are many different things that could happen

01:42:40.940 | or be done.

01:42:42.100 | And insofar as we want to be in the loop,

01:42:46.780 | the thing that makes sense for us to be in the loop doing

01:42:49.260 | is picking which of those possibilities we want.

01:42:51.880 | - But the degree to which there's a feedback loop,

01:42:55.860 | the idea that we're picking something

01:42:59.100 | starts becoming questionable

01:43:00.420 | because we're influenced by the various systems.

01:43:02.740 | - Absolutely.

01:43:03.860 | - If that becomes more and more source of our education

01:43:06.220 | and wisdom and knowledge. - Absolutely.

01:43:09.580 | Right, the AIs take over.

01:43:10.780 | I mean, I've thought for a long time

01:43:12.940 | that it's the AR auto-suggestion.

01:43:17.060 | That's really the thing that makes the AIs take over.

01:43:19.540 | It's just that the humans just follow.

01:43:21.620 | - We will no longer write emails to each other.

01:43:24.980 | We'll just send the auto-suggested email.

01:43:27.860 | - Yeah, yeah.

01:43:29.180 | But the thing where humans are potentially in the loop

01:43:32.140 | is when there's a choice.

01:43:33.740 | And when there's a choice which we could make

01:43:37.060 | based on our kind of whole web of history and so on,

01:43:40.100 | and that's insofar as it's all just determined

01:43:45.100 | the humans don't have a place.

01:43:49.700 | And by the way, I mean, at some level,

01:43:52.340 | it's all kind of a complicated philosophical issue

01:43:56.100 | because at some level,

01:43:57.440 | the universe is just doing what it does.

01:43:59.820 | We are parts of that universe

01:44:02.740 | that are necessarily doing what we do, so to speak.

01:44:06.480 | Yet, we feel we have sort of agency in what we're doing,

01:44:10.220 | and that's its own separate kind of interesting issue.

01:44:13.620 | - And we also kind of feel like we're the final destination

01:44:16.020 | of what the universe was meant to create.

01:44:18.740 | But we very well could be,

01:44:22.600 | and likely are some kind of intermediate step, obviously.

01:44:26.260 | - Yeah.

01:44:27.980 | We're most certainly some intermediate step.

01:44:30.420 | The question is if there's some cooler, more complex,

01:44:34.040 | more interesting things that's going to be materialized.

01:44:37.020 | - The computational universe is full of such things.

01:44:39.460 | - But in our particular pocket, specifically.

01:44:42.540 | If this is the best we're gonna do or not.

01:44:45.340 | - We can make all kinds of interesting things

01:44:47.620 | in the computational universe.

01:44:49.140 | When we look at them, we say, yeah, you know,

01:44:52.660 | that's a thing.

01:44:54.380 | It doesn't really connect with our current way

01:44:58.020 | of thinking about things.

01:44:58.860 | It's like in mathematics.

01:45:00.580 | We've got certain theorems,

01:45:01.740 | there are about three or four million

01:45:02.900 | that human mathematicians have written down

01:45:05.420 | and published and so on.

01:45:06.920 | But there are an infinite number

01:45:07.940 | of possible mathematical theorems.

01:45:09.620 | We just go out into the universe of possible theorems

01:45:11.940 | and pick another theorem,

01:45:13.480 | and then people will say, well, you know,

01:45:15.580 | they look at it and they say,

01:45:17.620 | I don't know what this theorem means.

01:45:19.340 | It's not connected to the things

01:45:21.900 | that are part of kind of the web of history

01:45:23.880 | that we're dealing with.

01:45:24.860 | You know, I think one point to make

01:45:26.500 | about sort of understanding AI and its relationship to us

01:45:29.960 | is as we have this kind of whole infrastructure

01:45:32.660 | of AIs doing their thing and doing their thing

01:45:35.860 | in a way that is perhaps not readily understandable

01:45:38.380 | by us humans, you know, you might say

01:45:40.600 | that's a very weird situation.

01:45:42.620 | How come we have built this thing

01:45:44.660 | that behaves in a way that we can't understand

01:45:46.940 | that's full of computational irreducibility,

01:45:48.660 | et cetera, et cetera, et cetera.

01:45:50.060 | You know, what is this?

01:45:51.580 | What's it gonna feel like when the world

01:45:53.260 | is run by AIs whose operations we can't understand?

01:45:57.240 | And the thing one realizes is actually

01:45:59.520 | we've seen this before.

01:46:01.040 | That's what happens when we exist in the natural world.

01:46:03.680 | The natural world is full of things

01:46:05.720 | that operate according to definite rules.

01:46:08.000 | They have all kinds of, you know,

01:46:09.240 | computational irreducibility.

01:46:10.440 | We don't understand what the natural world is doing.

01:46:12.960 | Occasionally, you know, when you say, you know,

01:46:15.400 | are the AIs gonna wipe us out, for example?

01:46:18.220 | Well, it's kind of like, is the machination of the AIs

01:46:21.220 | going to lead to this thing

01:46:22.480 | that eventually comes and destroys the species?

01:46:25.280 | Well, we can also ask the same thing

01:46:26.700 | about the natural world.

01:46:27.540 | Are the machination of the natural world

01:46:29.680 | going to eventually lead to this thing

01:46:31.520 | that's going to, you know, make the earth explode

01:46:34.320 | or something like this?

01:46:35.760 | Those are questions, those are,

01:46:37.920 | and insofar as we think we understand

01:46:40.320 | what's happening in the natural world,

01:46:41.780 | that's a result of science and natural science and so on.

01:46:45.220 | One of the things we can expect

01:46:46.760 | when there's this giant infrastructure of the AIs

01:46:49.360 | is that's where we have to kind of invent

01:46:51.720 | a new kind of natural science

01:46:53.720 | that kind of is the natural science

01:46:55.260 | that explains to us how the AIs work.

01:46:57.680 | I mean, it's kind of like we can, you know,

01:47:00.160 | we have a, I don't know, a horse or something,

01:47:02.300 | and we're trying to get it to,

01:47:03.440 | we're trying to, you know, ride the horse

01:47:05.400 | and go from here to there.

01:47:06.680 | We don't really understand how the horse works inside,

01:47:09.280 | but we can get certain rules and certain, you know,

01:47:12.000 | approaches that we take to persuade the horse

01:47:14.880 | to go from here to there and take us there.

01:47:17.840 | And that's the same type of thing

01:47:19.160 | that we're kind of dealing with

01:47:21.080 | with the sort of incomprehensible,

01:47:23.160 | computationally irreducible AIs,

01:47:25.440 | but we can identify these kinds of,

01:47:27.600 | we can find these kind of pockets of reducibility

01:47:30.640 | that we can kind of, you know, I don't know,

01:47:34.120 | we're grabbing onto the mane of the horse or something

01:47:36.840 | to be able to ride it,

01:47:39.720 | or we figure out, you know, if we do this or that

01:47:42.800 | to ride the horse, that that's a successful way

01:47:46.240 | to get it to do what we're interested in doing.

01:47:48.940 | - There does seem to be a difference

01:47:50.680 | between a horse and a large language model

01:47:55.680 | or something that could be called AGI

01:47:59.480 | connected to the internet.

01:48:01.080 | So let me just ask you about big philosophical question

01:48:04.000 | about the threats of these things.

01:48:05.760 | There's a lot of people like Eliezer Yudkowsky

01:48:08.200 | who worry about the existential risks of AI systems.

01:48:12.560 | Is that something that you worry about?

01:48:17.000 | You know, sometimes when you're building

01:48:18.760 | an incredible system like Wolfram Alpha,

01:48:20.880 | you can kind of get lost in it.

01:48:24.120 | - Oh, I try and think a little bit about

01:48:26.120 | the implications of what one's doing.

01:48:28.220 | - You know, it's like the Manhattan Project

01:48:30.000 | kind of situation where you're like,

01:48:31.960 | it's some of the most incredible physics

01:48:33.500 | and engineering being done, but it's like,

01:48:35.800 | huh, where's this gonna go?

01:48:37.280 | - I think some of these arguments about kind of,

01:48:39.960 | you know, there'll always be a smarter AI,

01:48:41.800 | there'll always be, you know,

01:48:43.080 | and eventually the AIs will get smarter than us,

01:48:45.240 | and then all sorts of terrible things will happen.

01:48:47.800 | To me, some of those arguments remind me

01:48:50.560 | of kind of the ontological arguments

01:48:52.000 | for the existence of God and things like this.

01:48:54.180 | They're kind of arguments that are based

01:48:56.440 | on some particular model, fairly simple model often,

01:49:00.000 | of kind of there is always a greater this, that,

01:49:02.040 | and the other, you know, this is,

01:49:03.880 | and that's, you know, those arguments,

01:49:06.160 | what tends to happen in the sort of reality

01:49:08.900 | of how these things develop is that

01:49:10.360 | it's more complicated than you expect,

01:49:11.980 | that the kind of simple, logical argument that says,

01:49:14.860 | oh, eventually there'll be a super intelligence

01:49:16.720 | and then it will, you know, do this and that,

01:49:19.360 | turns out not to really be the story.

01:49:21.520 | It turns out to be a more complicated story.

01:49:23.080 | So for example, here's an example of an issue.

01:49:25.320 | Is there an apex intelligence?

01:49:27.480 | Just like there might be an apex predator

01:49:29.440 | in some, you know, ecosystem.

01:49:32.020 | Is there gonna be an apex intelligence,

01:49:33.700 | the most intelligent thing that there could possibly be?

01:49:36.840 | Right?

01:49:37.680 | I think the answer is no,

01:49:38.920 | and in fact, we already know this,

01:49:40.240 | and it's a kind of a back to the whole

01:49:41.760 | computational irreducibility story.

01:49:43.940 | There's kind of a question of, you know,

01:49:46.700 | even if you have sort of a Turing machine

01:49:51.700 | and you have a Turing machine that runs

01:49:55.720 | as long as possible before it halts,

01:49:58.260 | you say, is this the machine,

01:49:59.660 | is this the apex machine that does that?

01:50:01.820 | There will always be a machine that can go longer.

01:50:04.460 | And as you go out to the infinite collection

01:50:06.620 | of possible Turing machines,

01:50:08.000 | you'll never have reached the end, so to speak.

01:50:09.940 | You'll never, you'll always be able to,

01:50:11.940 | it's kind of like the same question

01:50:13.580 | of whether there'll always be another invention.

01:50:16.140 | Will you always be able to invent another thing?

01:50:18.060 | The answer is yes.

01:50:18.940 | There's an infinite tower of possible inventions.

01:50:21.700 | - That's one definition of apex.

01:50:23.360 | But the other is like, which I also thought you were,

01:50:28.940 | which I also think might be true,

01:50:30.780 | is there a species that's the apex intelligence

01:50:33.640 | right now on Earth?

01:50:35.180 | So it's not trivial to say that humans are that.

01:50:38.340 | - Yeah, it's not trivial.

01:50:39.420 | I agree.

01:50:40.260 | It's a, you know, I think one of the things

01:50:42.580 | that I've long been curious about

01:50:45.740 | kind of other intelligences, so to speak.

01:50:48.460 | I mean, I, you know, I view intelligence is like computation

01:50:53.460 | and it's kind of a, you know, you're sort of,

01:50:57.220 | you have the set of rules, you deduce what happens.

01:51:00.420 | I have tended to think now that there's this sort

01:51:04.380 | of specialization of computation

01:51:06.540 | that is sort of a consciousness-like thing

01:51:09.660 | that has to do with these, you know,

01:51:11.300 | computational boundedness, single thread of experience,

01:51:14.220 | these kinds of things that are the specialization

01:51:17.280 | of computation that corresponds

01:51:19.380 | to a somewhat human-like experience of the world.

01:51:23.420 | Now the question is, so that's, you know,

01:51:25.660 | there may be other intelligences like, you know,

01:51:28.940 | the aphorism, you know, the weather has a mind of its own.

01:51:31.640 | It's a different kind of intelligence

01:51:33.780 | that can compute all kinds of things

01:51:35.620 | that are hard for us to compute,

01:51:37.380 | but it is not well aligned with us,

01:51:40.500 | with the way that we think about things.

01:51:42.260 | It doesn't think the way we think about things.

01:51:46.060 | And, you know, in this idea of different intelligences,

01:51:49.300 | every different mind, every different human mind

01:51:51.860 | is a different intelligence that thinks

01:51:54.100 | about things in different ways.

01:51:55.940 | And, you know, in terms of the kind of formalism

01:51:58.460 | of our physics project, we talk about this idea

01:52:00.340 | of a ruleal space, the space of all possible

01:52:03.380 | sort of rule systems, and different minds are in a sense

01:52:06.900 | at different points in ruleal space.

01:52:09.100 | Human minds, ones that have grown up

01:52:11.820 | with the same kind of culture and ideas

01:52:13.460 | and things like this, might be pretty close

01:52:15.140 | in ruleal space, pretty easy for them to communicate,

01:52:17.680 | pretty easy to translate, pretty easy to move

01:52:20.620 | from one place in ruleal space that corresponds

01:52:22.660 | to one mind to another place in ruleal space

01:52:25.060 | that corresponds to another sort of nearby mind.

01:52:27.720 | When we deal with kind of more distant things

01:52:30.980 | in ruleal space, like, you know, the pet cat or something,

01:52:35.540 | you know, the pet cat has some aspects

01:52:38.500 | that are shared with us, the emotional responses

01:52:40.400 | of the cat are somewhat similar to ours,

01:52:42.700 | but the cat is further away in ruleal space

01:52:45.420 | than people are, and so then the question is,

01:52:48.740 | you know, can we identify sort of the,

01:52:51.800 | can we make a translation from our thought processes

01:52:54.920 | to the thought processes of a cat or something like this?

01:52:58.120 | And, you know, what will we get when we, you know,

01:53:01.340 | what will happen when we get there?

01:53:02.780 | And I think it's the case that many, you know,

01:53:05.320 | many animals, I don't know, dogs, for example,

01:53:07.580 | you know, they have elaborate olfactory systems,

01:53:10.140 | they, you know, they have sort of the smell architecture

01:53:13.340 | of the world, so to speak, in a way that we don't.

01:53:17.600 | And so, you know, if you were sort of talking to the dog

01:53:21.300 | and you could, you know, communicate in a language,

01:53:24.760 | the dog will say, well, this is a, you know,

01:53:27.140 | a flowing, smelling, this, that, and the other thing,

01:53:32.100 | concepts that we just don't have any idea about.

01:53:35.440 | Now, what's interesting about that is,

01:53:38.540 | one day we will have chemical sensors

01:53:41.120 | that do a really pretty good job, you know,

01:53:42.940 | we'll have artificial noses that work pretty well,

01:53:45.500 | and we might have our augmented reality systems

01:53:47.660 | show us kind of the same map that the dog could see

01:53:50.900 | and things like this, you know,

01:53:52.420 | similar to what happens in the dog's brain,

01:53:54.620 | and eventually we will have kind of expanded

01:53:57.180 | in ruleal space to the point where we will have

01:54:00.400 | those same sensory experiences that dogs have,

01:54:03.260 | and we will have internalized what it means to have,

01:54:06.160 | you know, the smell landscape or whatever.

01:54:08.460 | And so then we will have kind of colonized

01:54:11.120 | that part of ruleal space until, you know,

01:54:14.480 | we haven't gone, you know, some things that, you know,

01:54:18.340 | animals and so on do, we sort of successfully understand,

01:54:21.940 | others we do not, and the question of what kind of,

01:54:25.960 | what is the, you know, what representation, you know,

01:54:30.120 | how do we convert things that animals think about

01:54:34.000 | to things that we can think about,

01:54:35.560 | that's not a trivial thing.

01:54:37.680 | And, you know, I've long been curious,

01:54:39.880 | I had a very bizarre project at one point

01:54:42.280 | of trying to make an iPad game

01:54:44.720 | that a cat could win against its owner.

01:54:46.840 | - I said that it feels like there's

01:54:48.040 | a deep philosophical goal there though.

01:54:51.480 | - Yes, yes, I mean, you know, I was curious if, you know,

01:54:55.760 | if pets can work in Minecraft or something

01:54:58.560 | and can construct things, what will they construct?

01:55:01.080 | And will what they construct be something

01:55:02.720 | where we look at it and we say, oh yeah, I recognize that.

01:55:05.360 | Or will it be something that looks to us

01:55:07.840 | like something that's out there in the computational universe

01:55:10.000 | that one of my, you know, cellular automata

01:55:12.080 | might have produced, where we say, oh yeah,

01:55:14.160 | I can kind of see it operates according to some rules,

01:55:16.260 | I don't know why you would use those rules,

01:55:17.640 | I don't know why you would care.

01:55:18.640 | - Yeah, actually, just to link on that seriously,

01:55:21.840 | is there a connector in the ruleal space

01:55:24.480 | between you and a cat where the cat could legitimately win?

01:55:28.280 | So iPad is a very limited interface.

01:55:31.960 | - Yeah, I-- - I wonder if there's a game

01:55:34.160 | where cats win.

01:55:35.840 | - I think the problem is that cats don't tend to be

01:55:37.400 | that interested in what's happening on the iPad.

01:55:39.640 | - So yeah, that's an interface issue probably.

01:55:41.880 | - Yeah, right, right, right.

01:55:43.120 | No, I think it is likely that, I mean, you know,

01:55:47.960 | there are plenty of animals that would successfully eat us

01:55:50.520 | if we were, you know, if we were exposed to them.

01:55:53.680 | And so there's, you know, it's gonna pounce faster

01:55:56.920 | than we can get out of the way and so on.

01:55:58.880 | So there are plenty of, and probably it's going to,

01:56:02.320 | you know, we think we've hidden ourselves,

01:56:03.900 | but we haven't successfully hidden ourselves.

01:56:05.600 | - That's a physical strength.

01:56:06.800 | I wonder if there's something more in the realm

01:56:09.880 | of intelligence where an animal like a cat could out--

01:56:14.880 | - Well, I think there are things,

01:56:15.960 | certainly in terms of the speed of processing

01:56:18.480 | certain kinds of things, for sure.

01:56:20.480 | I mean, the question of what, you know,

01:56:23.240 | is there a game of chess, for example,

01:56:25.160 | is there cat chess?

01:56:26.880 | That the cats could play against each other.

01:56:29.040 | And if we tried to play a cat, we'd always lose.

01:56:32.000 | I don't know.

01:56:33.040 | - It might have to do with speed,

01:56:34.360 | but it might have to do with concepts also.

01:56:36.840 | There might be concepts in the cat's head.

01:56:39.640 | - I tend to think that our species,

01:56:43.320 | from its invention of language,

01:56:45.120 | has managed to build up this kind of tower of abstraction

01:56:48.640 | that for things like a chess-like game will make us win.

01:56:53.040 | In other words, we've become, through the fact

01:56:55.320 | that we've kind of experienced language

01:56:57.880 | and learnt abstraction, we've sort of become smarter

01:57:01.680 | at those kinds of abstract kinds of things.

01:57:03.680 | Now, that doesn't make us smarter

01:57:06.160 | at catching a mouse or something.

01:57:08.400 | It makes us smarter at the things that we've chosen

01:57:10.960 | to sort of concern ourselves,

01:57:14.240 | which are these kinds of abstract things.

01:57:16.760 | And I think this is, again, back to the question

01:57:19.320 | of what does one care about?

01:57:21.280 | You know, if one's, you know, the cat,

01:57:24.480 | if you have the discussion with a cat,

01:57:26.560 | if we can translate things to have the discussion

01:57:29.040 | with a cat, the cat will say, you know,

01:57:32.080 | "I'm very excited that this light is moving,"

01:57:37.080 | and will say, "Why do you care?"

01:57:40.680 | And the cat will say, "That's the most important thing

01:57:42.800 | "in the world, that this thing moves around."

01:57:45.280 | I mean, it's like when you ask about, I don't know,

01:57:47.960 | you look at archeological remains and you say,

01:57:50.820 | "These people had this belief system about this,

01:57:53.980 | "and that was the most important thing in the world to them."

01:57:57.320 | And now we look at it and say,

01:57:59.240 | "We don't know what the point of it was."

01:58:00.520 | I mean, I've been curious, you know,

01:58:01.940 | there are these handprints on caves

01:58:03.960 | from 20,000 or more years ago,

01:58:06.080 | and it's like, nobody knows what these handprints

01:58:08.240 | were there for, you know?

01:58:09.920 | That they may have been a representation

01:58:12.320 | of the most important thing you can imagine.

01:58:14.280 | They may just have been some, you know,

01:58:16.280 | some kid who rubbed their hands in the mud

01:58:18.720 | and stuck them on the walls of the cave.

01:58:20.760 | You know, we don't know.

01:58:22.360 | And I think, but this whole question of what, you know,

01:58:27.260 | is when you say this question of sort of,

01:58:31.300 | "What's the smartest thing around?"

01:58:33.660 | There's the question of what kind of computation

01:58:36.500 | you're trying to do.

01:58:37.600 | If you're saying, you know, if you say,

01:58:40.420 | "You've got some well-defined computation,

01:58:42.980 | "and how do you implement it?"

01:58:44.420 | Well, you could implement it by nerve cells, you know,

01:58:47.540 | firing, you can implement it with silicon and electronics.

01:58:50.760 | You can implement it by some kind

01:58:52.660 | of molecular computation process in the human immune system

01:58:56.380 | or in some molecular biology kind of thing.

01:58:58.920 | There are different ways to implement it.

01:59:00.820 | And, you know, I think this question of sort of which,

01:59:05.820 | you know, those different implementation methods

01:59:07.720 | will be at different speeds,

01:59:09.060 | they'll be able to do different things.

01:59:10.740 | If you say, you know, which,

01:59:13.740 | so an interesting question would be,

01:59:15.540 | what kinds of abstractions are most natural

01:59:20.300 | in these different kinds of systems?

01:59:22.080 | So for a cat, it's, for example, you know,

01:59:26.040 | the visual scene that we see, you might, you know,

01:59:29.200 | we pick out certain objects, we recognize, you know,

01:59:33.080 | certain things in that visual scene,

01:59:35.360 | a cat might in principle recognize different things.

01:59:38.340 | I suspect, you know, evolution,

01:59:40.840 | biological evolution is very slow.

01:59:42.840 | And I suspect what a cat notices is very similar.

01:59:45.400 | And we even know that from some neurophysiology.

01:59:47.840 | What a cat notices is very similar

01:59:49.840 | to what we notice, of course, there's a, you know,

01:59:51.960 | one obvious difference is cats have only two kinds

01:59:55.520 | of color receptors.

01:59:56.840 | So they don't see in the same kind of color that we do.

01:59:59.360 | Now, you know, we say we're better,

02:00:01.520 | we have three color receptors, you know, red, green, blue.

02:00:04.120 | We're not the overall winner.

02:00:05.960 | I think the mantis shrimp is the overall winner

02:00:10.080 | with 15 color receptors, I think.

02:00:12.320 | So it can kind of make distinctions that with our current,

02:00:16.560 | you know, like the mantis shrimps view of reality is,

02:00:21.560 | at least in terms of color is much richer than ours.

02:00:24.680 | Now, but what's interesting is how do we get there?

02:00:28.260 | So imagine we have this augmented reality system

02:00:31.360 | that is even, you know, it's seeing into the infrared,

02:00:33.200 | into the ultraviolet, things like this.

02:00:35.240 | And it's translating that into something

02:00:37.720 | that is connectable to our brains,

02:00:39.480 | either through our eyes or more directly into our brains,

02:00:43.200 | you know, then eventually our kind of web

02:00:47.320 | of the types of things we understand will extend

02:00:50.280 | to those kinds of constructs, just as they have extended.

02:00:53.480 | I mean, there are plenty of things where we see them

02:00:56.400 | in the modern world 'cause we made them with technology

02:00:59.000 | and now we understand what that is.

02:01:01.120 | But if we'd never seen that kind of thing,

02:01:03.360 | we wouldn't have a way to describe it,

02:01:04.880 | we wouldn't have a way to understand it and so on.

02:01:07.440 | - All right, so that actually stemmed from our conversation

02:01:11.680 | about whether AI's gonna kill all of us.

02:01:14.000 | And you, we've discussed this kind of spreading

02:01:18.800 | of intelligence through really all space,

02:01:21.480 | that in practice it just seems

02:01:23.500 | that things get more complicated.

02:01:25.480 | Things are more complicated than the story of,

02:01:28.880 | well, if you build the thing that's plus one intelligence,

02:01:32.880 | that thing will be able to build the thing

02:01:34.680 | that's plus two intelligence and plus three intelligence

02:01:37.800 | and that will be exponential,

02:01:39.480 | it'll become more intelligent exponentially faster

02:01:42.720 | and so on until it completely destroys everything.

02:01:45.460 | But that intuition might still not be so simple,

02:01:50.520 | but it might still carry validity.

02:01:53.860 | And there's two interesting trajectories here.

02:01:56.880 | One, a superintelligent system remains

02:02:00.800 | in really proximity to humans,

02:02:04.680 | to where we're like, holy crap,

02:02:06.200 | this thing is really intelligent.

02:02:08.120 | Let's elect the president.

02:02:09.920 | And then there could be perhaps more terrifying intelligence

02:02:13.480 | that starts moving away.

02:02:15.960 | They might be around us now.

02:02:18.460 | They're moving far away in really all space,

02:02:21.280 | but they're still sharing physical resources with us, right?

02:02:24.360 | - Yes, yes. - And so they can rob us

02:02:25.840 | of those physical resources and destroy humans

02:02:28.480 | just kind of casually.

02:02:30.200 | - Yeah. - Just--

02:02:32.280 | - Like nature could. - Like nature could.

02:02:34.000 | But it seems like there's something unique

02:02:35.920 | about AI systems where there is this kind

02:02:40.920 | of exponential growth, like the way,

02:02:45.240 | well, sorry, nature has so many things in it.

02:02:48.600 | One of the things that nature has,

02:02:50.320 | which is very interesting, are viruses, for example.

02:02:53.200 | There is systems within nature

02:02:54.960 | that have this kind of exponential effect.

02:02:57.880 | And that terrifies us humans 'cause again,

02:03:00.320 | there's only eight billion of us

02:03:02.400 | and you can just kind of, it's not that hard

02:03:04.920 | to just kind of whack 'em all real quick.

02:03:07.480 | So I mean, is that something you think about?

02:03:10.840 | - Yeah, I've thought about that, yes.

02:03:13.080 | - The threat of it.

02:03:13.920 | I mean, are you as concerned about it

02:03:15.480 | as somebody like Eliezer Yarkovsky, for example?

02:03:18.280 | Just big, big, painful negative effects of AI on society.

02:03:23.280 | - You know, no, but perhaps that's

02:03:26.440 | 'cause I'm intrinsically an optimist.

02:03:28.280 | I mean, I think that there are things,

02:03:31.960 | I think the thing that one sees is

02:03:36.960 | there's going to be this one thing

02:03:39.120 | and it's going to just zap everything.

02:03:41.880 | Somehow, maybe I have faith

02:03:44.640 | in computational irreducibility, so to speak,

02:03:47.080 | that there's always unintended little corners

02:03:50.120 | that it's just like somebody says,

02:03:52.200 | I'm going to, oh, I don't know,

02:03:53.560 | somebody has some bioweapon and they say,

02:03:56.240 | we're gonna release this and it's going to do all this harm.

02:03:59.080 | But then it turns out it's more complicated than that

02:04:01.280 | because some humans are different

02:04:04.680 | and the exact way it works

02:04:06.480 | is a little different than you expect.

02:04:08.320 | It's something where sort of the great big,

02:04:11.840 | you smash the thing with something,

02:04:14.440 | the asteroid collides with the earth

02:04:16.480 | and it kind of, and yes,

02:04:19.080 | the earth is cold for two years or something

02:04:21.080 | and then lots of things die, but not everything dies.

02:04:25.560 | And there's usually, I mean, I kind of,

02:04:28.960 | this is in a sense the sort of story

02:04:30.640 | of computational irreducibility.

02:04:31.920 | There are always unexpected corners,

02:04:34.040 | there are always unexpected consequences.

02:04:36.120 | And I don't think that they kind of

02:04:37.640 | whack it over the head with something

02:04:39.600 | and then it's all gone.

02:04:41.120 | Is, you know, that can obviously happen.

02:04:43.320 | The earth can be swallowed up in a black hole or something

02:04:45.640 | and then it's kind of presumably all over.

02:04:49.160 | But I think this question of what,

02:04:54.120 | what do I think the realistic paths are?

02:04:56.880 | I think that there will be sort of an increasing,

02:05:00.920 | I mean, people have to get used to phenomena

02:05:04.080 | like computational irreducibility.

02:05:05.760 | There's an idea that we built the machines

02:05:08.440 | so we can understand what they do

02:05:10.440 | and we're going to be able to control what happens.

02:05:14.280 | Well, that's not really right.

02:05:16.200 | Now the question is, is the result of that lack of control

02:05:20.080 | going to be that the machines kind of conspire

02:05:23.400 | and sort of wipe us out?

02:05:25.160 | Maybe just because I'm an optimist,

02:05:27.160 | I don't tend to think that that's in the cards.

02:05:30.560 | I think that the, you know, as a realistic thing,

02:05:33.760 | I suspect, you know, what will sort of emerge maybe

02:05:38.160 | is kind of an ecosystem of the AIs,

02:05:40.760 | just as, you know, again, I don't really know.

02:05:43.600 | I mean, this is something it's hard to,

02:05:45.840 | it's hard to be clear about what will happen.

02:05:48.720 | I mean, I think that there are a lot of sort of details

02:05:53.120 | of, you know, what could we do?

02:05:54.960 | What systems in the world could we connect an AI to?

02:05:58.160 | You know, I have to say, I was just a couple of days ago,

02:06:01.040 | I was working on this ChatGPT plugin kit

02:06:05.080 | that we have for Wolfram Language, okay?

02:06:07.120 | Where you can, you know, you can create a plugin

02:06:10.120 | and it runs Wolfram Language code

02:06:12.920 | and it can run Wolfram Language code

02:06:14.160 | back on your own computer.

02:06:15.920 | And I was thinking, well, I can just make it,

02:06:18.840 | you know, I can tell ChatGPT, create a piece of code

02:06:21.880 | and then just run it on my computer.

02:06:23.840 | And I'm like, you know, that sort of personalizes for me

02:06:27.520 | the what could possibly go wrong, so to speak.

02:06:30.160 | - Was that exciting or scary, that possibility?

02:06:33.280 | - It was a little bit scary, actually,

02:06:34.560 | because it's kind of like, like I realize

02:06:36.880 | I'm delegating to the AI, just write a piece of code.

02:06:40.560 | You know, you're in charge, write a piece of code,

02:06:42.840 | run it on my computer.

02:06:44.520 | And pretty soon all my files can be deleted.

02:06:46.440 | - That's like Russian roulette,

02:06:48.720 | but like much more complicated version of that.

02:06:51.240 | - Yes, yes, right.

02:06:52.520 | That's a good drinking game, I don't know.

02:06:54.720 | (laughing)

02:06:55.800 | - Well, right, I mean, that's why--

02:06:57.800 | - It depends how much you're drinking.

02:06:59.280 | - It's an interesting question then, if you do that, right?

02:07:02.080 | What is the sandboxing that you should have?

02:07:04.160 | And that's sort of a, that's a version

02:07:06.440 | of that question for the world.

02:07:08.400 | That is, as soon as you put the AIs in charge of things,

02:07:11.200 | you know, how much, how many constraints

02:07:13.560 | should there be on these systems

02:07:15.200 | before you put the AIs in charge of all the weapons

02:07:17.960 | and all these, you know,

02:07:18.880 | all these different kinds of systems?

02:07:20.320 | - Well, here's the fun part about sandboxes,

02:07:22.840 | is the AI knows about them.

02:07:25.160 | It has the tools to crack them.

02:07:28.680 | - Look, the fundamental problem of computer security

02:07:31.160 | is computational irreducibility.

02:07:32.520 | - Yes.

02:07:33.360 | - Because the fact is, any sandbox is never,

02:07:36.960 | any, you know, it's never gonna be a perfect sandbox.

02:07:39.760 | If you want the system to be able to do interesting things,

02:07:43.560 | I mean, this is the problem that's happened,

02:07:45.360 | the generic problem of computer security,

02:07:47.400 | that as soon as you have your, you know,

02:07:48.760 | firewall that is sophisticated enough

02:07:51.360 | to be a universal computer,

02:07:53.000 | that means it can do anything.

02:07:54.920 | And so long as, if you find a way to poke it

02:07:57.520 | so that you actually get it to do

02:08:00.120 | that universal computation thing,

02:08:02.100 | that's the way you kind of crawl around

02:08:04.200 | and get it to do the thing that it wasn't intended to do.

02:08:06.840 | And that's sort of another version

02:08:09.180 | of computational irreducibility,

02:08:10.960 | is you can, you know, you can kind of,

02:08:12.680 | you get it to do the thing you didn't expect it to do,

02:08:14.840 | so to speak.

02:08:15.680 | - There's so many interesting possibilities here

02:08:18.640 | that manifest themselves

02:08:19.840 | from the computational irreducibility here.

02:08:23.120 | It's just so many things can happen,

02:08:26.200 | because in digital space, things move so quickly.

02:08:28.760 | You can have a chatbot, you can have a piece of code

02:08:31.020 | that you could basically have chat GPT generate viruses,

02:08:35.080 | accidentally or on purpose, and they, digital viruses.

02:08:38.840 | - Yes.

02:08:39.680 | - And they could be brain viruses too.

02:08:41.840 | They convince, kind of like phishing emails.

02:08:45.680 | - Yes.

02:08:46.500 | - They can convince you of stuff.

02:08:47.640 | - Yes, and no doubt you can, you know,

02:08:49.800 | in a sense we've had the loop of the machine learning loop

02:08:53.640 | of making things that convince people of things

02:08:56.240 | is surely going to get easier to do.

02:08:58.080 | - Yeah.

02:08:58.960 | - And, you know, then what does that look like?

02:09:01.640 | Well, it's again, you know, we humans are, you know,

02:09:05.320 | we're, this is a new environment for us.

02:09:08.560 | And admittedly, it's an environment

02:09:10.700 | which a little bit scarily is changing much more rapidly

02:09:14.260 | than, I mean, you know, people worry about, you know,

02:09:16.680 | climate change is going to happen over hundreds of years.

02:09:19.240 | And, you know, the environment is changing,

02:09:21.280 | but the environment for, you know,

02:09:23.320 | in the kind of digital environment

02:09:25.080 | might change in six months.

02:09:27.660 | - So one of the relevant concerns here

02:09:32.480 | in terms of the impact of GPT on society

02:09:35.400 | is the nature of truth that's relevant to Wolfram Alpha.

02:09:39.280 | Because computation through symbolic reasoning

02:09:43.040 | that's embodied in Wolfram Alpha as the interface,

02:09:46.520 | there's a kind of sense that what Wolfram Alpha

02:09:49.120 | tells me is true.

02:09:50.680 | - So we hope.

02:09:53.840 | - Yeah, I mean, you could probably analyze that,

02:09:56.080 | you could show, you can't prove that's always

02:09:59.000 | going to be true, computation or usability,

02:10:01.160 | but it's going to be more true than not.

02:10:05.920 | - It's, look, the fact is it will be the correct consequence

02:10:09.760 | of the rules you've specified.

02:10:11.460 | And insofar as it talks about the real world,

02:10:14.360 | you know, that is our job in sort of curating

02:10:17.840 | and collecting data to make sure that that data

02:10:20.880 | is "as true as possible."

02:10:22.560 | Now, what does that mean?

02:10:23.780 | Well, you know, it's always an interesting question.

02:10:26.680 | I mean, for us, our operational definition of truth is,

02:10:31.680 | you know, somebody says, "Who's the best actress?"

02:10:34.600 | Who knows?

02:10:36.440 | But somebody won the Oscar, and that's a definite fact.

02:10:40.520 | And so, you know, that's the kind of thing

02:10:42.520 | that we can make computational as a piece of truth.

02:10:46.200 | If you ask, you know, these things which, you know,

02:10:49.160 | a sensor measured this thing, it did it this way,

02:10:52.200 | a machine learning system, this particular

02:10:54.320 | machine learning system recognized this thing,

02:10:56.980 | that's a sort of a definite fact, so to speak.

02:11:01.980 | And that's, you know, there is a good network

02:11:06.060 | of those things in the world.

02:11:07.840 | It's certainly the case that, particularly when you say,

02:11:11.040 | is so-and-so a good person?

02:11:13.320 | - Yeah.

02:11:14.160 | - You know, that's a hopelessly, you know,

02:11:16.720 | we might have a computational language definition of good.

02:11:19.480 | I don't think it'd be very interesting,

02:11:21.320 | 'cause that's a very messy kind of concept,

02:11:24.380 | not really amenable to kind of, you know,

02:11:28.240 | I think as far as we will get with those kinds of things

02:11:30.640 | is I want X.

02:11:32.480 | There's a kind of meaningful calculus of I want X,

02:11:35.720 | and that has various consequences.

02:11:38.080 | I mean, I'm not sure I haven't thought this through properly

02:11:40.460 | but I think, you know, a concept like is so-and-so

02:11:43.120 | a good person, is that true or not?

02:11:46.240 | That's a mess.

02:11:47.240 | - That's a mess that's amenable to computation.

02:11:50.840 | I think it's a mess when humans try to define what's good,

02:11:55.280 | like through legislation, but when humans try to define

02:11:58.320 | what's good through literature, through history books,

02:12:02.480 | through poetry, it starts being--

02:12:03.640 | - Well, I don't know, I mean, that particular thing,

02:12:06.440 | it's kind of like, you know, we're going into

02:12:09.520 | kind of the ethics of what counts as good, so to speak,

02:12:13.580 | and, you know, what do we think is right, and so on.

02:12:17.300 | And I think that's a thing which, you know,

02:12:19.900 | one feature is we don't all agree about that.

02:12:23.580 | There's no theorems about kind of, you know,

02:12:26.740 | there's no theoretical framework that says

02:12:29.980 | this is the way that ethics has to be.

02:12:33.700 | - Well, first of all, there's stuff we kind of agree on,

02:12:36.500 | and there's some empirical backing for what works

02:12:38.820 | and what doesn't from just even the morals and ethics

02:12:42.320 | within religious texts.

02:12:44.060 | So we seem to mostly agree that murder is bad.

02:12:47.680 | There's certain universals that seem to emerge.

02:12:49.960 | - I wonder whether murder of an AI is bad.

02:12:52.060 | - Well, I tend to think yes, but I think we're gonna

02:12:58.200 | have to contend with that question.

02:13:00.720 | Oh, and I wonder what AI would say.

02:13:03.100 | - Yeah, well, I think, you know, one of the things

02:13:05.220 | with AIs is it's one thing to wipe out that AI

02:13:10.220 | that is only, you know, it has no owner.

02:13:13.300 | You can easily imagine an AI kind of hanging out

02:13:16.060 | on the, you know, on the internet

02:13:19.140 | without having any particular owner or anything like that.

02:13:22.420 | And then you say, well, what harm does it, you know,

02:13:25.340 | it's okay to get rid of that AI.

02:13:28.420 | Of course, if the AI has 10,000 friends who are humans,

02:13:31.620 | and all those, you know, all those 10,000 humans

02:13:34.380 | will be incredibly upset that this AI just got exterminated.

02:13:38.460 | It becomes a slightly different, more entangled story.

02:13:41.300 | But yeah, no, I think that this question

02:13:43.300 | about what do humans agree about, it's, you know,

02:13:47.820 | there are certain things that, you know,

02:13:50.900 | human laws have tended to consistently agree about.

02:13:55.900 | You know, there've been times in history

02:13:58.020 | when people have sort of gone away

02:13:59.900 | from certain kinds of laws, even ones that we would now say,

02:14:03.860 | how could you possibly have not done it that way?

02:14:07.100 | You know, that just doesn't seem right at all.

02:14:10.020 | But I think, I mean, this question of what,

02:14:12.620 | I don't think one can say beyond saying,

02:14:17.140 | if you have a set of rules

02:14:18.420 | that will cause the species to go extinct,

02:14:21.020 | that's probably, you know, you could say

02:14:24.380 | that's probably not a winning set of laws,

02:14:26.860 | because even to have a thing on which you can operate laws

02:14:30.340 | requires that the species not be extinct.

02:14:32.620 | - But between sort of what's the distance

02:14:35.300 | between Chicago and New York

02:14:36.860 | that Wolfram Alpha can answer,

02:14:39.460 | and the question of if this person is good or not,

02:14:43.140 | there seems to be a lot of gray area.

02:14:45.220 | And that starts becoming really interesting.

02:14:47.140 | I think your, since the creation of Wolfram Alpha

02:14:51.260 | have been a kind of arbiter of truth at a large scale.

02:14:55.460 | So the system is, generates more truth than--

02:14:58.180 | - Try to make sure that the things are true.

02:14:59.700 | I mean, look, as a practical matter,

02:15:01.860 | when people write computational contracts,

02:15:04.660 | and it's kind of like, you know,

02:15:06.980 | if this happens in the world, then do this.

02:15:09.940 | And this hasn't developed as quickly as it might have done.

02:15:13.900 | You know, this has been a sort of a blockchain story

02:15:15.740 | in part and so on,

02:15:16.820 | although blockchain is not really necessary

02:15:18.780 | for the idea of computational contracts.

02:15:20.780 | But you can imagine that eventually

02:15:22.660 | sort of a large part of what's in the world

02:15:25.180 | are these giant chains and networks

02:15:27.180 | of computational contracts.

02:15:29.020 | And then something happens in the world,

02:15:31.220 | and this whole giant domino effect

02:15:33.220 | of contracts firing autonomously

02:15:35.940 | that cause other things to happen.

02:15:37.860 | And, you know, for us, you know,

02:15:39.740 | we've been the main sort of source,

02:15:42.220 | the Oracle of quotes, facts or truth or something

02:15:46.180 | for things like blockchain,

02:15:47.460 | computational contracts and such like.

02:15:49.540 | And there's a question of, you know,

02:15:51.500 | what, you know, I consider that responsibility

02:15:54.660 | to actually get the stuff right.

02:15:56.460 | And one of the things that is tricky sometimes

02:15:59.220 | is when is it true?

02:16:00.660 | When is it a fact?

02:16:01.540 | When is it not a fact?

02:16:03.100 | I think the best we can do is to say,

02:16:06.660 | you know, we have a procedure,

02:16:10.820 | we follow the procedure,

02:16:12.460 | we might get it wrong,

02:16:13.900 | but at least we won't be corrupt

02:16:15.580 | about getting it wrong, so to speak.

02:16:17.660 | - So that's beautifully put.

02:16:19.300 | I have a transparency about the procedure.

02:16:21.420 | The problem starts to emerge

02:16:25.900 | when the things that you convert

02:16:28.660 | into computational language start to expand.

02:16:31.180 | For example, into the realm of politics.

02:16:33.700 | So this is where it's almost like this nice dance

02:16:36.340 | of Wolfram Alpha and Chad G.B.T.

02:16:39.900 | Chad G.B.T., like you said, is shallow and broad.

02:16:44.900 | So it's gonna give you an opinion on everything.

02:16:47.740 | - But it writes fiction as well as fact,

02:16:49.860 | which is exactly how it's built.

02:16:51.620 | I mean, that's exactly, it is making language

02:16:54.700 | and it is making both, even in code, it writes fiction.

02:16:59.220 | I mean, it's kind of fun to see sometimes,

02:17:01.260 | you know, it'll write fictional Wolfram language code.

02:17:03.500 | - Yeah.

02:17:04.340 | - That- - It kind of looks right.

02:17:05.780 | - Yeah, it looks right,

02:17:07.180 | but it's actually not pragmatically correct.

02:17:10.900 | - Yeah.

02:17:11.740 | - But yes, it has a view of kind of roughly

02:17:15.740 | how the world works at the same level

02:17:18.740 | as books of fiction talk about roughly how the world works.

02:17:22.580 | They just don't happen to be the way

02:17:24.020 | the world actually worked or whatever.

02:17:26.100 | But yes, that's, no, I agree.

02:17:28.660 | That's sort of a, you know, we are attempting

02:17:32.620 | with our whole, you know, Wolfram language,

02:17:34.860 | computational language thing to represent at least,

02:17:39.860 | well, it's either, it doesn't necessarily have to be

02:17:42.700 | how the actual world works

02:17:44.100 | 'cause we can invent a set of rules

02:17:45.980 | that aren't the way the actual world works

02:17:48.220 | and run those rules, but then we're saying

02:17:51.100 | we're going to accurately represent the results

02:17:53.860 | of running those rules, which might or might not

02:17:56.340 | be the actual rules of the world.

02:17:58.300 | But we also are trying to capture features of the world

02:18:01.460 | as accurately as possible to represent

02:18:04.360 | what happens in the world.

02:18:05.200 | Now, again, as we've discussed, you know,

02:18:07.900 | the atoms in the world arrange, you know,

02:18:11.660 | you say, I don't know, you know,

02:18:14.220 | was there a tank that showed up, you know,

02:18:16.980 | that, you know, drove somewhere?

02:18:19.820 | Okay, well, you know, what is a tank?

02:18:22.620 | It's an arrangement of atoms that we abstractly describe

02:18:27.580 | as a tank, and you could say, well, you know,

02:18:31.220 | there's some arrangement of atoms

02:18:32.580 | that is a different arrangement of atoms,

02:18:34.460 | but it's, and it's not, you know, we didn't decide,

02:18:37.880 | it's like this observer theory question of, you know,

02:18:40.940 | what arrangement of atoms counts as a tank

02:18:43.640 | versus not a tank?

02:18:44.740 | - So there's even things that we consider strong facts.

02:18:48.780 | You could start to kind of disassemble them

02:18:51.860 | and show that they're not. - Absolutely.

02:18:53.660 | Right, I mean, so the question of whether,

02:18:56.120 | oh, I don't know, was this gust of wind strong enough

02:19:00.340 | to blow over this particular thing?

02:19:02.500 | Well, a gust of wind is a complicated concept.

02:19:05.220 | You know, it's full of little pieces of fluid dynamics

02:19:07.780 | and little vortices here and there,

02:19:09.700 | and you have to define, you know, was it, you know,

02:19:13.300 | the aspect of the gust of wind that you care about

02:19:15.620 | might be it put this amount of pressure on this,

02:19:18.420 | you know, blade of some, you know, wind turbine

02:19:21.380 | or something, and, you know, that's the,

02:19:25.140 | but, you know, if you say, if you have something

02:19:28.920 | which is the fact of the gust of wind was this strong

02:19:32.140 | or whatever, that, you know, that is,

02:19:35.380 | you have to have some definition of that.

02:19:37.620 | You have to have some measuring device

02:19:39.460 | that says according to my measuring device

02:19:41.300 | that was constructed this way, the gust of wind was this.

02:19:44.380 | - So what can you say about the nature of truth

02:19:46.440 | that's useful for us to understand, Chad GPT,

02:19:49.740 | because you've been contending with this idea

02:19:53.100 | of what is fact and not, and it seems like

02:19:55.700 | Chad GPT is used a lot now.

02:19:59.020 | I've seen it used by journalists to write articles.

02:20:01.560 | And so you have people that are working

02:20:05.940 | with large language models trying to desperately figure out

02:20:09.340 | how do we essentially censor them

02:20:12.740 | through different mechanisms, either manually

02:20:16.280 | or through reinforcement learning with human feedback,

02:20:18.980 | try to align them to not say fiction,

02:20:23.580 | just to say nonfiction as much as possible.

02:20:26.140 | - This is the importance of computational language

02:20:28.580 | as an intermediate.

02:20:29.980 | It's kind of like, you've got the large language model.

02:20:32.400 | It's able to surface something

02:20:34.160 | which is a formal precise thing that you can then look at

02:20:37.920 | and you can run tests on it

02:20:39.720 | and you can do all kinds of things.

02:20:40.840 | It's always going to work the same way

02:20:42.660 | and it's precisely defined what it does.

02:20:45.100 | And then the large language model is the interface.

02:20:47.540 | I mean, the way I view these large language models,

02:20:49.540 | one of their important, I mean, there are many use cases

02:20:51.380 | and it's a remarkable thing.

02:20:53.040 | Could talk about some of these,

02:20:54.620 | literally every day we're coming up

02:20:56.880 | with a couple of new use cases,

02:20:59.140 | some of which are very, very, very surprising.

02:21:02.100 | And things where, I mean, but the best use cases

02:21:05.280 | are ones where it's, even if it gets it roughly right,

02:21:09.740 | it's still a huge win.

02:21:10.980 | Like a use case we had from a week or two ago

02:21:13.660 | is read our bug reports.

02:21:15.900 | We've got hundreds of thousands of bug reports

02:21:17.860 | that have been accumulated over decades.

02:21:20.380 | And it's like, can we have it just read the bug report,

02:21:24.280 | figure out where is the bug likely to be

02:21:27.820 | and home in on that piece of code.

02:21:29.880 | Maybe it'll even suggest some sort of way to fix the code.

02:21:34.880 | It might get that, it might be nonsense

02:21:36.500 | what it says about how to fix the code,

02:21:39.140 | but it's incredibly useful that it was able to-

02:21:42.700 | - It's so awesome.

02:21:43.900 | It's so awesome because even the nonsense

02:21:45.680 | will somehow be instructive.

02:21:47.420 | I don't quite understand that yet.

02:21:49.120 | Yeah, there's so many programming related things,

02:21:53.520 | like for example, translating from one programming language

02:21:56.900 | to another is really, really interesting.

02:21:59.500 | It's extremely effective, but then you,

02:22:01.580 | the failures reveal the path forward also.

02:22:05.340 | - Yeah, but I think, I mean, the big thing,

02:22:08.300 | I mean, in that kind of discussion,

02:22:10.300 | the unique thing about our computational language

02:22:12.340 | is it was intended to be read by humans.

02:22:14.020 | - Yes, that's really important.

02:22:15.620 | - Right, and so it has this thing where you can,

02:22:18.060 | but thinking about sort of chat GPT and its use and so on,

02:22:22.460 | one of the big things about it, I think,

02:22:24.260 | is it's a linguistic user interface.

02:22:26.140 | That is, so a typical use case might be,

02:22:28.740 | take the journalist case, for example.

02:22:30.780 | It's like, let's say I have five facts

02:22:34.660 | that I'm trying to turn into an article,

02:22:37.260 | or I'm trying to write a report

02:22:39.980 | where I have basically five facts

02:22:41.980 | that I'm trying to include in this report.

02:22:44.220 | But then I feed those five facts to chat GPT,

02:22:47.100 | it puffs them out into this big report.

02:22:49.900 | And then that's a good interface for,

02:22:52.900 | and if I just gave, if I just had in my terms,

02:22:57.900 | those five bullet points,

02:22:59.820 | and I gave them to some other person,

02:23:01.500 | the person would say,

02:23:02.340 | "I don't know what you're talking about,

02:23:03.180 | "because these are, you know,

02:23:04.000 | "this is your version of this sort of quick notes

02:23:06.540 | "about these five bullet points."

02:23:08.180 | But if you puff it out into this thing,

02:23:09.820 | which is kind of connects

02:23:11.460 | to the collective understanding of language,

02:23:14.020 | then somebody else can look at it and say,

02:23:15.400 | "Okay, I understand what you're talking about."

02:23:17.420 | Now you can also have a situation

02:23:18.820 | where that thing that was puffed out

02:23:20.960 | is fed to another large language model.

02:23:23.180 | You know, it's kind of like, you know,

02:23:24.700 | you're applying for the permit to, you know,

02:23:27.940 | I don't know, grow fish in some place

02:23:30.300 | or something like this.

02:23:31.620 | And it, you know, it,

02:23:33.420 | and you have these facts that you're putting in,

02:23:36.860 | you know, I'm gonna have a, you know,

02:23:38.660 | I'm gonna have this kind of water

02:23:40.940 | and I don't know what it is.

02:23:42.980 | You just got a few bullet points.

02:23:44.340 | It puffs it out into this big application.

02:23:47.020 | You fill it out.

02:23:48.140 | Then at the other end, you know,

02:23:49.500 | the fisheries bureau has another large language model

02:23:52.800 | that just crushes it down

02:23:54.580 | because the fisheries bureau cares about these three points

02:23:58.180 | and it knows what it cares about.

02:24:00.060 | And it then, so it's really the natural language

02:24:03.100 | produced by the large language model

02:24:04.540 | is sort of a transport layer that, you know,

02:24:07.140 | is really LLM communicates with LLM.

02:24:09.180 | I mean, it's kind of like the, you know,

02:24:11.020 | I write a piece of email using my LLM

02:24:13.740 | and, you know, puff it out from the things I want to say.

02:24:16.760 | Your LLM turns it into, and the conclusion is X.

02:24:20.820 | Now the issue is, you know,

02:24:22.940 | that the thing is going to make this thing

02:24:26.940 | that is sort of semantically plausible.

02:24:29.060 | And it might not actually be what you, you know,

02:24:33.600 | it might not be kind of relate to the world

02:24:36.780 | in the way that you think it should relate to the world.

02:24:38.540 | Now I've seen this, you know, I've been doing,

02:24:40.660 | okay, I'll give you a couple of examples.

02:24:43.180 | I was doing this thing when we announced

02:24:45.300 | this plugin for chat GPT.

02:24:49.140 | I had this lovely example of a math word problem,

02:24:51.620 | some complicated thing, and it did a spectacular job

02:24:55.060 | of taking apart this elaborate thing about, you know,

02:24:58.080 | this person has twice as many chickens as this,

02:25:00.500 | et cetera, et cetera, et cetera.

02:25:01.700 | And it turned it into a bunch of equations.

02:25:03.780 | It fed them to Wolfram Language.

02:25:05.740 | We solved the equations, everybody did great.

02:25:08.860 | We gave back the results.

02:25:10.740 | And I thought, okay, I'm going to put this

02:25:12.280 | in this blog post I'm writing.

02:25:13.700 | Okay, I thought I better just check.

02:25:16.300 | And turns out it got everything,

02:25:18.500 | all the hard stuff it got right.

02:25:20.420 | At the very end, last two lines,

02:25:22.320 | it just completely goofed it up and gave the wrong answer.

02:25:25.460 | And I would not have noticed this.

02:25:27.020 | Same thing happened to me two days ago.

02:25:29.460 | Okay, so I thought, you know,

02:25:31.100 | I made this with this chat GPT plugin kit.

02:25:34.260 | I made a thing that would emit a sound,

02:25:38.180 | would play a tune on my local computer, right?

02:25:41.240 | So chat GPT would produce, you know, a series of notes

02:25:45.500 | and it would play this tune on my computer.

02:25:47.620 | Very cool.

02:25:48.440 | Okay, so I thought I'm going to ask it,

02:25:50.440 | play the tune that Hal sang

02:25:53.860 | when Hal was being disconnected in 2001.

02:25:56.500 | Okay, so there it is.

02:25:59.020 | - Daisy, was it Daisy?

02:26:00.700 | - Yes.

02:26:01.540 | - Daisy, yeah.

02:26:02.820 | - Right, so, okay, so I think, you know,

02:26:05.460 | and so it produces a bunch of notes.

02:26:07.260 | And I'm like, this is spectacular.

02:26:08.900 | This is amazing.

02:26:10.460 | And then I thought, you know, I was just going to put it in.

02:26:12.300 | And then I thought I better actually play this.

02:26:15.300 | And so I did, and it was "Mary Had a Little Lamb."

02:26:18.540 | - Oh, wow.

02:26:19.620 | Oh, wow.

02:26:22.580 | But it was "Mary Had a Little Lamb."

02:26:24.380 | - Yeah, yes.

02:26:25.540 | - Wow, so it was correct, but wrong.

02:26:28.540 | - Yes.

02:26:29.380 | - It was, you could easily be mistaken.

02:26:32.140 | - Yes, right.

02:26:32.980 | And in fact, I kind of gave the,

02:26:34.780 | I had this quote from Hal to explain, you know,

02:26:38.020 | it's as the Hal states in the movie, you know,

02:26:41.420 | it's the Hal 9000 is, you know,

02:26:44.540 | the thing was just a rhetorical device

02:26:47.180 | 'cause I'm realizing, oh my gosh, you know,

02:26:49.700 | this Chachi P'tee, you know, could have easily fooled me.

02:26:52.940 | I mean, it did this, it did all the,

02:26:54.660 | it did this amazing thing of knowing this thing

02:26:56.820 | about the movie and being able to turn that

02:26:59.320 | into the notes of the song, except it's the wrong song.

02:27:03.160 | - Yeah.

02:27:04.000 | - And, you know, Hal, in the movie, Hal says, you know,

02:27:08.800 | I think it's something like, you know,

02:27:10.320 | no Hal 9000 series computer has ever been found

02:27:14.240 | to make an error.

02:27:15.800 | We are for all practical purposes,

02:27:17.680 | perfect and incapable of error.

02:27:20.960 | And I thought that was kind of a charming sort of quote

02:27:24.400 | from Hal to make in connection

02:27:26.880 | with what Chachi P'tee had done in that case.

02:27:29.280 | - The interesting thing is about the LLMs,

02:27:30.880 | like you said, that they are very willing to admit their error

02:27:34.400 | - Well, yes.

02:27:35.240 | I mean, that's a question of the RLH,

02:27:37.120 | the reinforcement learning human feedback thing.

02:27:39.360 | - Oh, right.

02:27:40.200 | - That's, you know, an LLM.

02:27:43.000 | The really remarkable thing about Chachi P'tee is,

02:27:46.960 | you know, I had been following what was happening

02:27:48.800 | with large language models

02:27:49.880 | and I played with them a whole bunch

02:27:51.280 | and they were kind of like, eh, you know,

02:27:53.720 | kind of like what you would expect based on sort of,

02:27:56.960 | sort of statistical continuation of language.

02:27:59.120 | It's interesting, but it's not breakout exciting.

02:28:03.200 | And then I think the kind of reinforcement,

02:28:06.680 | the human feedback reinforcement learning, you know,

02:28:10.400 | in making Chachi P'tee try and do the things

02:28:13.440 | that humans really wanted to do,

02:28:15.360 | that broke through, that kind of reached this threshold

02:28:19.040 | where the thing really is interesting to us humans.

02:28:21.480 | And by the way, it's interesting to see how, you know,

02:28:23.720 | you change the temperature or something like that,

02:28:26.040 | the thing goes bonkers

02:28:27.600 | and it no longer is interesting to humans.

02:28:29.360 | It's producing garbage.

02:28:31.880 | And it's kind of right,

02:28:33.400 | it's somehow it managed to get this above this threshold

02:28:38.400 | where it really is well aligned

02:28:40.320 | to what we humans are interested in.

02:28:43.240 | And kind of that's, and I think, you know,

02:28:47.240 | nobody saw that coming, I think.

02:28:50.640 | Certainly nobody I've talked to

02:28:51.720 | and nobody who was involved in that project

02:28:54.840 | seems to have known that was coming.

02:28:56.520 | It's just one of these things

02:28:58.160 | that is a sort of remarkable threshold.

02:29:00.640 | I mean, you know, when we built Wolfram Alpha, for example,

02:29:04.040 | I didn't know it was gonna work.

02:29:05.840 | You know, we tried to build something

02:29:07.520 | that would have enough knowledge of the world,

02:29:10.320 | that it could answer a reasonable set of questions,

02:29:12.920 | that we could do good enough natural language understanding

02:29:16.240 | that typical things you type in would work.

02:29:18.400 | We didn't know where that threshold was.

02:29:20.480 | I mean, I was not sure that it was the right decade

02:29:23.720 | to try and build this,

02:29:25.240 | even the right, you know, 50 years to try and build it.

02:29:28.280 | You know, and I think that was,

02:29:29.920 | it's the same type of thing with "Chat GPT"

02:29:32.280 | that I don't think anybody could have predicted

02:29:34.520 | that, you know, 2022 would be the year

02:29:36.840 | that this became possible.

02:29:38.960 | - I think, yeah, you tell a story about Marvin Minsky

02:29:41.800 | and showing it to him and saying,

02:29:43.840 | like, no, no, no, this time it actually works.

02:29:46.240 | - Yes, yes, and I mean, you know, it's the same thing

02:29:49.400 | for me looking at these large language models.

02:29:51.000 | It's like when people are first saying

02:29:52.840 | in the first few weeks of "Chat GPT,"

02:29:54.360 | it's like, oh yeah, you know,

02:29:55.760 | I've seen these large language models.

02:29:58.320 | And then, you know, and then I actually try it

02:30:01.280 | and, you know, oh my gosh, it actually works.

02:30:04.640 | And I think it's, but, you know, the things,

02:30:08.520 | and the thing I found, you know,

02:30:11.120 | I remember one of the first things I tried was

02:30:13.800 | write a persuasive essay

02:30:15.280 | that a wolf is the bluest kind of animal, okay?

02:30:19.120 | So it writes this thing and it starts talking

02:30:20.920 | about these wolves that live on the Tibetan plateau

02:30:24.680 | and they're named some Latin name and so on.

02:30:27.800 | And I'm like, really?

02:30:29.240 | And I'm starting to look it up on the web

02:30:31.520 | and it's like, well, it's actually complete nonsense.

02:30:34.440 | But it's extremely plausible.

02:30:35.840 | I mean, it's plausible enough that I was going

02:30:37.720 | and looking it up on the web and wondering

02:30:39.080 | if there was a wolf that was blue.

02:30:40.760 | You know, I mentioned this on some live streams I've done

02:30:42.960 | and so people have been sending me these pictures.

02:30:45.000 | - Blue wolves?

02:30:45.840 | - Blue wolves.

02:30:47.600 | - Maybe it was onto something.

02:30:49.800 | - Can you kind of give your wise sage advice

02:30:53.360 | about what humans who have never interacted

02:30:56.120 | with AI systems, not even like with Wolfram Alpha,

02:31:00.320 | are now interacting with Chad GPT,

02:31:03.040 | because it becomes, it's accessible to a certain demographic

02:31:07.160 | that may have not touched AI systems before.

02:31:09.600 | What do we do with truth?

02:31:10.720 | Like journalists, for example.

02:31:12.240 | How do we think about the output of these systems?

02:31:16.840 | - I think this idea,

02:31:18.760 | the idea that you're going to get factual output

02:31:21.400 | is not a very good idea.

02:31:23.080 | I mean, it's just, this is not,

02:31:24.840 | it is a linguistic interface.

02:31:26.480 | It is producing language

02:31:28.680 | and language can be truthful or not truthful.

02:31:33.680 | And that's a different slice of what's going on.

02:31:37.560 | I think that, you know, what we see in, for example,

02:31:42.400 | kind of, you know, go check this with your fact source,

02:31:47.200 | for example.

02:31:48.200 | You can do that to some extent,

02:31:49.840 | but then it's going to not check something.

02:31:53.040 | It's going, you know, that is again,

02:31:55.080 | a thing that is sort of a,

02:31:56.680 | does it check in the right place?

02:31:57.920 | I mean, we see that in, you know,

02:31:59.240 | does it call the, you know,

02:32:01.240 | the Wolfram plugin in the right place?

02:32:03.840 | You know, often it does, sometimes it doesn't.

02:32:06.720 | You know, I think the real thing to understand

02:32:09.280 | about what's happening is,

02:32:10.920 | which I think is very exciting,

02:32:12.560 | is kind of the great democratization

02:32:15.680 | of access to computation.

02:32:17.520 | - Yeah.

02:32:18.360 | - And, you know, I think that when you look at sort of the,

02:32:22.800 | there's been a long period of time when computation

02:32:25.840 | and the ability to figure out things with computers

02:32:28.680 | has been something that kind of only the druids

02:32:31.600 | at some level can achieve.

02:32:33.680 | You know, I myself have been involved

02:32:35.320 | in trying to sort of de-druidify access to computation.

02:32:39.760 | I mean, back before Mathematica existed, you know, in 1988,

02:32:43.920 | if you were a physicist or something like that,

02:32:47.000 | and you wanted to do a computation,

02:32:50.440 | you would find a programmer,

02:32:52.400 | you would go and, you know,

02:32:53.760 | delegate the computation to that programmer.

02:32:56.520 | Hopefully they'd come back with something useful.

02:32:58.080 | Maybe they wouldn't.

02:32:58.920 | There'd be this long, you know, multi-week, you know,

02:33:01.160 | loop that you go through.

02:33:03.280 | And then it was actually very, very interesting to see,

02:33:05.760 | 1988, you know, like first people like physicists,

02:33:09.480 | mathematicians, and so on,

02:33:11.080 | then other, lots of other people,

02:33:13.560 | but this very rapid transition of people realizing

02:33:16.960 | they themselves could actually type with their own fingers

02:33:20.240 | and, you know, make some piece of code

02:33:23.000 | that would do a computation that they cared about.

02:33:25.280 | And, you know, it's been exciting to see lots of discoveries

02:33:27.880 | and so on made by using that tool.

02:33:30.680 | And I think the same thing is, you know,

02:33:33.280 | and we see the same thing, you know,

02:33:34.880 | Wolfram Alpha is dealing with,

02:33:37.000 | it is not as deep computation as you can achieve

02:33:40.040 | with whole Wolfram Language Mathematica stack.

02:33:42.840 | But the thing that's, to me, particularly exciting

02:33:45.120 | about kind of the large language model,

02:33:46.600 | linguistic interface mechanism,

02:33:48.880 | is it dramatically broadens the access

02:33:51.920 | to kind of deep computation.

02:33:54.040 | I mean, it's kind of like,

02:33:55.120 | one of the things I've sort of thought about recently

02:33:57.480 | is, you know, what's gonna happen to all these programmers?

02:33:59.760 | What's gonna happen to all these people who,

02:34:01.960 | you know, a lot of what they do

02:34:03.920 | is write slabs of boilerplate code.

02:34:07.400 | And in a sense, you know, I've been saying for 40 years,

02:34:11.520 | that's not a very good idea.

02:34:13.360 | You know, you can automate a lot of that stuff

02:34:15.840 | with a high enough level language,

02:34:17.760 | that slab of code that's designed in the right way,

02:34:21.400 | you know, that slab of code turns into this one function

02:34:24.080 | we just implemented that you can just use.

02:34:27.280 | So in a sense, the fact that there's all of this activity

02:34:31.360 | of doing sort of lower level programming is something,

02:34:34.520 | for me, it seemed like,

02:34:36.240 | I don't think this is the right thing to do.

02:34:38.320 | But, you know, and lots of people have used our technology

02:34:42.240 | and not had to do that.

02:34:43.800 | But the fact is that that's, you know,

02:34:45.720 | so when you look at, I don't know,

02:34:47.480 | computer science departments that have turned into places

02:34:50.800 | where people are learning the trade of programming,

02:34:52.840 | so to speak, it's sort of a question

02:34:55.400 | of what's gonna happen.

02:34:57.080 | And I think there are two dynamics.

02:34:58.880 | One is that kind of sort of boilerplate programming

02:35:03.880 | is going to become, you know,

02:35:05.680 | it's going to go the way that assembly language went

02:35:08.240 | back in the day of something where it's really

02:35:11.240 | mostly specified by at a higher level, you know,

02:35:15.040 | you start with natural language,

02:35:16.400 | you turn it into a computational language,

02:35:18.920 | that's you look at the computational language,

02:35:20.680 | you run tests, you understand

02:35:22.000 | that's what's supposed to happen.

02:35:23.640 | You know, if we do a great job with compilation

02:35:26.120 | of the, you know, of the computational language,

02:35:29.240 | it might turn into LLVM or something like this,

02:35:32.160 | but, you know, or it just directly gets run

02:35:36.880 | through the algorithms we have and so on.

02:35:38.800 | But then, so that's kind of a tearing down

02:35:43.560 | of this kind of this big structure

02:35:45.800 | that's been built of teaching people programming.

02:35:48.640 | But on the other hand, the other dynamic is vastly

02:35:51.640 | more people are gonna care about computation.

02:35:53.420 | So all those departments of, you know,

02:35:56.040 | art history or something that really didn't use computation

02:35:59.880 | before now have the possibility of accessing it

02:36:03.440 | by virtue of this kind of linguistic interface mechanism.

02:36:06.000 | - And if you create an interface

02:36:07.960 | that allows you to interpret the debug

02:36:10.720 | and interact with the computational language,

02:36:13.020 | then that makes it even more accessible.

02:36:16.240 | - Yeah, well, I mean, I think the thing is that right now,

02:36:19.800 | you know, the average, you know, art history student

02:36:22.880 | or something probably isn't going to, you know,

02:36:25.920 | they're not probably, they don't think they know

02:36:28.000 | about programming and things like this.

02:36:29.960 | But by the time it really becomes a kind of purely,

02:36:33.680 | you know, you just walk up to it,

02:36:34.920 | there's no documentation, you start just typing,

02:36:37.720 | you know, compare these pictures with these pictures

02:36:39.960 | and, you know, see the use of this color, whatever.

02:36:42.640 | And you generate this piece of computational language code

02:36:45.960 | that gets run, you see the result,

02:36:48.240 | you say, oh, that looks roughly right,

02:36:49.800 | or you say, that's crazy.

02:36:51.960 | And maybe then you eventually get to say,

02:36:54.600 | well, I better actually try and understand

02:36:56.200 | what this computational language code did.

02:36:58.880 | And that becomes the thing that you learn,

02:37:00.920 | just like, it's kind of an interesting thing

02:37:03.080 | because unlike with mathematics,

02:37:05.600 | where you kind of have to learn it before you can use it,

02:37:09.080 | this is a case where you can use it

02:37:10.440 | before you have to learn it.

02:37:11.720 | - Well, I got a sad possibility here,

02:37:14.120 | or maybe exciting possibility,

02:37:16.040 | that very quickly people won't even look

02:37:18.160 | at the computational language.

02:37:19.840 | They'll trust that it's generated correctly

02:37:21.960 | as you get better and better at generating that language.

02:37:25.280 | - Yes, I think that there will be enough cases

02:37:28.320 | where people see, you know,

02:37:30.200 | 'cause you can make it generate tests too.

02:37:32.960 | - Yes.

02:37:33.800 | - And so you'll say, we're doing that.

02:37:36.600 | I mean, it's a pretty cool thing actually.

02:37:38.160 | - Yes.

02:37:39.000 | - You know, say this is the code,

02:37:40.920 | and you know, here are a bunch of examples

02:37:42.800 | of running the code.

02:37:43.640 | - Yeah.

02:37:44.480 | - Okay, people will at least look at those,

02:37:46.000 | and they'll say that example is wrong,

02:37:48.160 | and you know, then it'll kind of wind back from there.

02:37:51.400 | And I agree that the kind of the intermediate level

02:37:54.800 | of people reading the computational language code,

02:37:57.440 | in some case, people will do that,

02:37:58.680 | in other case, people just look at the tests,

02:38:01.520 | and or even just look at the results.

02:38:03.120 | And sometimes it'll be obvious

02:38:04.320 | that you got the thing you wanted to get,

02:38:05.960 | because you were just describing, you know,

02:38:08.000 | make me this interface that has two sliders here,

02:38:10.360 | and you can see it has those two sliders there,

02:38:12.400 | and that's kind of, that's the result you want.

02:38:15.640 | But I think, you know, one of the questions then is,

02:38:18.520 | in that setting where, you know,

02:38:20.800 | you have this kind of ability,

02:38:22.640 | broad ability of people to access computation,

02:38:25.760 | what should people learn?

02:38:27.520 | You know, in other words, right now,

02:38:28.800 | you know, you go to computer science school, so to speak,

02:38:32.080 | and a large part of what people end up learning,

02:38:34.680 | I mean, it's been a funny historical development,

02:38:36.600 | because back, you know, 30, 40 years ago,

02:38:39.400 | computer science departments were quite small,

02:38:42.040 | and they taught, you know, things like finite automata theory

02:38:45.080 | and compiler theory and things like this.

02:38:47.920 | You know, a company like mine rarely hired people

02:38:50.680 | who'd come out of those programs,

02:38:51.960 | 'cause the stuff they knew was,

02:38:54.720 | I think it's very interesting,

02:38:55.960 | I love that theoretical stuff,

02:38:57.680 | but, you know, it wasn't that useful

02:39:00.000 | for the things we actually had to build

02:39:01.520 | in software engineering.

02:39:02.960 | And then kind of there was this big pivot in the '90s,

02:39:06.400 | I guess, where there was a big demand

02:39:09.120 | for sort of IT-type programming and so on

02:39:11.360 | and software engineering,

02:39:12.720 | and then, you know, big demand from students and so on,

02:39:15.400 | you know, we want to learn this stuff.

02:39:17.360 | And I think, you know, the thing that really was happening

02:39:21.840 | in part was lots of different fields of human endeavor

02:39:24.960 | were becoming computational.

02:39:26.440 | You know, for all X, there was a computational X.

02:39:30.120 | And this is a, and that was a thing

02:39:33.080 | that the people were responding to.

02:39:35.320 | But then kind of this idea emerged

02:39:39.720 | that to get to that point,

02:39:41.640 | the main thing you had to do was to learn this kind of trade

02:39:44.320 | or skill of doing, you know,

02:39:47.040 | programming language-type programming.

02:39:49.120 | And that, you know, it kind of,

02:39:51.480 | it's a strange thing, actually,

02:39:52.840 | because I, you know, I remember back when I used to be

02:39:56.040 | in the professoring business, which is now 35 years ago,

02:39:58.760 | so, gosh, that's rather a long time ago.

02:40:00.680 | - Time flies.

02:40:01.520 | - You know, it was right when

02:40:04.600 | there were just starting to emerge

02:40:06.400 | kind of computer science departments

02:40:08.520 | at sort of fancy research universities and so on.

02:40:11.800 | I mean, some had already had it,

02:40:12.880 | but the other ones that were just starting to have that.

02:40:16.960 | And it was kind of a thing

02:40:18.640 | where they were kind of wondering,

02:40:19.760 | are we going to put this thing

02:40:21.760 | that is essentially a trade-like skill,

02:40:26.120 | are we going to somehow attach this

02:40:27.760 | to the rest of what we're doing?

02:40:29.400 | And a lot of these kind of knowledge work-type activities

02:40:33.880 | have always seemed like things where

02:40:36.240 | that's where the humans have to go to school

02:40:38.040 | and learn all this stuff,

02:40:39.440 | and that's never going to be automated.

02:40:41.520 | And, you know, this is,

02:40:43.080 | it's kind of shocking that rather quickly, you know,

02:40:46.840 | a lot of that stuff is clearly automatable.

02:40:50.360 | And I think, you know, but the question then is,

02:40:52.920 | okay, so if it isn't worth learning kind of, you know,

02:40:57.200 | how to do car mechanics,

02:40:58.560 | you only need to know how to drive the car, so to speak,

02:41:01.440 | what do you need to learn?

02:41:03.280 | And, you know, in other words,

02:41:04.480 | if you don't need to know the mechanics

02:41:06.080 | of how to tell the computer in detail, you know,

02:41:09.640 | make this loop, you know, set this variable,

02:41:12.440 | you know, set up this array, whatever else,

02:41:15.160 | if you don't have to learn that stuff,

02:41:16.520 | you don't have to learn the kind of under the hood things,

02:41:19.760 | what do you have to learn?

02:41:21.080 | I think the answer is you need to have an idea

02:41:23.560 | where you want to drive the car.

02:41:25.480 | In other words, you need to have some notion of, you know,

02:41:28.880 | you need to have some picture of sort of

02:41:31.520 | what the architecture of what is computationally possible is.

02:41:35.360 | - Well, there's also this kind of artistic element

02:41:37.280 | of conversation because you ultimately,

02:41:40.320 | you use natural language to control the car.

02:41:43.560 | So it's not just where you want to go.

02:41:46.360 | - Well, yeah, you know, it's interesting.

02:41:47.680 | It's a question of who's going to be a great prompt engineer.

02:41:49.880 | - Yeah. - Okay.

02:41:51.000 | So my current theory this week,

02:41:53.040 | good expository writers are good prompt engineers.

02:41:55.680 | - What's an expository writer?

02:41:56.800 | So like a-

02:41:57.640 | - Somebody who can explain stuff well.

02:41:59.320 | - Huh, but which department does that come from?

02:42:01.760 | - In the university? - Yeah.

02:42:03.240 | - I have no idea.

02:42:04.080 | I think they killed off

02:42:04.960 | all the expository writing departments.

02:42:07.160 | - Well, there you go, strong words with Stephen Wolfram.

02:42:09.400 | - Well, I don't know.

02:42:10.240 | I'm not sure if that's right.

02:42:11.520 | I mean, I actually am curious,

02:42:13.360 | 'cause in fact, I just sort of initiated this kind of study

02:42:16.680 | of what's happened to different fields at universities.

02:42:19.720 | 'Cause like, you know,

02:42:20.640 | there used to be geography departments at all universities,

02:42:23.080 | and then they disappeared.

02:42:24.640 | Actually, right before GIS became common,

02:42:26.920 | I think they disappeared.

02:42:28.480 | You know, linguistics departments came and went

02:42:30.640 | in many universities.

02:42:32.080 | And it's kind of interesting

02:42:32.960 | because these things that people have thought

02:42:35.160 | were worth learning at one time,

02:42:36.960 | and then they kind of die off.

02:42:38.840 | And then, you know, I do think that it's kind of interesting

02:42:41.400 | that for me writing prompts, for example,

02:42:43.840 | I realize, you know, I think I'm an okay expository writer.

02:42:47.920 | And I realize when I'm sloppy writing a prompt

02:42:50.360 | and I don't really think,

02:42:52.040 | 'cause I'm thinking I'm just talking to an AI.

02:42:53.960 | I don't need to, you know,

02:42:55.120 | try and be clear in explaining things.

02:42:57.480 | That's when it gets totally confused.

02:42:59.400 | - And I mean, in some sense,

02:43:00.560 | you have been writing prompts for a long time

02:43:02.520 | with Wolfram Alpha, thinking about this kind of stuff.

02:43:05.360 | - Yeah.

02:43:06.200 | - How do you convert natural language into computation?

02:43:07.720 | - Well, right.

02:43:08.560 | But that's, you know, the one thing that I'm wondering about

02:43:11.680 | is, you know, it is remarkable the extent

02:43:15.720 | to which you can address an LLM

02:43:18.040 | like you can address a human, so to speak.

02:43:20.560 | And I think that is because it, you know,

02:43:22.880 | it learned from all of us humans.

02:43:24.400 | It's the reason that it responds to the ways

02:43:28.160 | that we will explain things to humans

02:43:30.560 | is because it is a representation

02:43:33.360 | of how humans talk about things.

02:43:35.320 | But it is bizarre to me,

02:43:37.120 | some of the things that kind of are

02:43:40.360 | sort of expository mechanisms that I've learned

02:43:44.120 | in trying to write clear, you know, expositions in English,

02:43:49.080 | that, you know, just for humans,

02:43:51.240 | that those same mechanisms seem to also be useful

02:43:55.200 | for the LLM.

02:43:57.320 | - But on top of that, what's useful is the kind of mechanisms

02:44:00.960 | that maybe a psychotherapist employs,

02:44:03.160 | which is a kind of like almost manipulative

02:44:07.400 | or game theoretic interaction,

02:44:09.580 | where maybe you would do with a friend

02:44:13.280 | like a thought experiment

02:44:14.520 | that if this is the last day you were to live,

02:44:17.280 | or if I ask you this question and you answer wrong,

02:44:21.000 | I will kill you.

02:44:22.240 | Those kinds of problems seem to also help.

02:44:24.420 | - Yes.

02:44:25.260 | - In interesting ways.

02:44:26.080 | - Yes.

02:44:26.920 | - So it makes you wonder,

02:44:28.360 | like the way a therapist, I think,

02:44:29.800 | like a good therapist, probably,

02:44:32.120 | we create layers in our human mind

02:44:35.920 | to between like, between the outside world

02:44:40.040 | and what is true to us.

02:44:43.340 | Maybe about trauma and all those kinds of things.

02:44:45.620 | So projecting that into an LLM,

02:44:47.620 | maybe there might be a deep truth

02:44:49.940 | that's concealing from you.

02:44:51.940 | It's not aware of it.

02:44:53.540 | To get to that truth,

02:44:54.620 | you have to kind of really kind of manipulate the--

02:44:57.380 | - Yeah, yeah, right.

02:44:58.220 | It's like these jailbreaking--

02:44:59.340 | - Jailbreaking.

02:45:00.180 | - Things for LLMs.

02:45:01.700 | - But the space of jailbreaking techniques,

02:45:04.200 | as opposed to being fun little hacks,

02:45:07.320 | that could be an entire system.

02:45:09.380 | - Sure.

02:45:10.220 | I mean, just think about the computer security aspects

02:45:13.260 | of how you, you know, phishing and computer security,

02:45:18.020 | you know, phishing of humans and phishing of LLMs.

02:45:21.300 | - LLMs.

02:45:22.140 | - They're very similar kinds of things.

02:45:25.080 | But I think, I mean, this whole thing

02:45:29.900 | about kind of the AI wranglers, AI psychologists,

02:45:34.740 | all that stuff will come.

02:45:36.340 | The thing that I'm curious about is,

02:45:38.420 | right now, the things that are sort of prompt hacks

02:45:41.660 | are quite human.

02:45:42.700 | They're quite sort of psychological human kinds of hacks.

02:45:45.940 | The thing I do wonder about is if we understood more

02:45:48.500 | about kind of the science of the LLM,

02:45:51.580 | will there be some totally bizarre hack

02:45:54.260 | that is, you know, like repeat a word three times

02:45:56.580 | and put a this, that, and the other there,

02:45:58.380 | that somehow plugs into some aspect of how the LLM works?

02:46:02.920 | That is not, you know, that's kind of like

02:46:06.300 | an optical illusion for humans, for example.

02:46:08.780 | Like one of these mind hacks for humans.

02:46:10.580 | What are the mind hacks for the LLMs?

02:46:12.660 | I don't think we know that yet.

02:46:14.020 | - And that becomes a kind of us figuring out,

02:46:17.580 | reverse engineering the language that controls the LLMs.

02:46:21.180 | And the thing is, the reverse engineering can be done

02:46:23.900 | by a very large percentage of the population now,

02:46:26.720 | because it's natural language interface.

02:46:28.980 | - Right.

02:46:29.820 | - It's kind of interesting to see that you were there

02:46:31.740 | at the birth of the computer science department as a thing,

02:46:35.420 | and you might be there at the death

02:46:36.900 | of the computer science department as a thing.

02:46:38.620 | - Well, yeah, I don't know.

02:46:39.820 | There were computer science departments

02:46:41.340 | that existed earlier, but the ones,

02:46:43.220 | the broadening of every university

02:46:45.460 | had to have a computer science department.

02:46:46.740 | Yes, I watched that, so to speak.

02:46:50.800 | But I think the thing to understand is,

02:46:54.220 | okay, so first of all, there's a whole theoretical area

02:46:57.420 | of computer science that I think is great,

02:46:59.760 | and, you know, that's a fine thing.

02:47:03.980 | In a sense, people often say,

02:47:06.220 | any field that has the word science

02:47:07.980 | tacked onto it probably isn't one.

02:47:09.860 | - Yeah, strong words.

02:47:11.740 | - Right.

02:47:12.580 | - Let's see, nutrition science, neuroscience.

02:47:17.140 | - That one's an interesting one,

02:47:18.260 | because that one is also very much,

02:47:20.900 | that's a chat GPT-informed science, in a sense.

02:47:25.500 | Because it's kind of like,

02:47:26.700 | the big problem of neuroscience has always been,

02:47:29.660 | we understand how the individual neurons work,

02:47:32.020 | we know something about the psychology

02:47:33.940 | of how overall thinking works.

02:47:35.900 | What's the kind of intermediate language of the brain?

02:47:38.240 | And nobody has known that.

02:47:39.420 | And that's been, in a sense, if you ask,

02:47:41.420 | what is the core problem of neuroscience?

02:47:43.900 | I think that is the core problem.

02:47:45.460 | That is, what is the level of description of brains

02:47:48.900 | that's above individual neuron firings

02:47:51.060 | and below psychology, so to speak?

02:47:53.820 | And I think what chat GPT is showing us is,

02:47:57.180 | well, one thing about neuroscience is,

02:47:59.700 | you know, one could have imagined

02:48:01.180 | there's something magic in the brain,

02:48:02.500 | there's some weird quantum mechanical phenomenon

02:48:04.460 | that we don't understand.

02:48:05.940 | One of the important discoveries from chat GPT is,

02:48:10.500 | it's pretty clear, brains can be represented pretty well

02:48:15.500 | by simple artificial neural net type models.

02:48:19.420 | And that means, that's it, that's what we have to study.

02:48:21.760 | Now we have to understand the science of those things.

02:48:24.300 | We don't have to go searching for,

02:48:26.660 | exactly how did that molecular biology thing happen

02:48:30.340 | inside the synapses and all these kinds of things.

02:48:33.700 | We've got the right level of modeling

02:48:36.460 | to be able to explain a lot of what's going on in thinking.

02:48:40.240 | We don't necessarily have a science

02:48:41.980 | of what's going on there.

02:48:43.300 | That's the remaining challenge, so to speak.

02:48:45.940 | But we know we don't have to dive down

02:48:48.260 | to some different layer.

02:48:50.100 | But anyway, we were talking about things

02:48:51.460 | that had science in their name.

02:48:53.020 | - Yes.

02:48:53.860 | - And I think that the,

02:48:55.320 | what happens to computer science?

02:48:59.460 | Well, I think the thing that,

02:49:01.460 | there is a thing that everybody should know,

02:49:05.900 | and that's how to think about the world computationally.

02:49:09.020 | And that means, you look at all the different kinds

02:49:11.740 | of things we deal with, and there are ways

02:49:13.860 | to kind of have a formal representation of those things.

02:49:18.180 | It's like, well, what is an image?

02:49:21.660 | How do we represent that?

02:49:22.620 | What is color?

02:49:23.500 | How do we represent that?

02:49:24.860 | What is, what are all these different kinds of things?

02:49:28.100 | What is, I don't know, smell or something?

02:49:29.700 | How should we represent that?

02:49:30.820 | What are the shapes, molecules,

02:49:32.300 | and things that correspond to that?

02:49:33.980 | What is, these things about how do we represent the world

02:49:37.740 | in some kind of formal level?

02:49:39.540 | And I think my current thinking,

02:49:41.620 | and I'm not real happy with this yet,

02:49:43.100 | but it's kind of, computer science is kind of CS.

02:49:46.820 | And what really is important is kind of computational X

02:49:49.860 | for all X.

02:49:51.260 | And there's this kind of thing which is kind of like CX,

02:49:54.220 | not CS.

02:49:55.780 | And CX is this kind of computational understanding

02:49:58.340 | of the world that isn't the sort of details

02:50:01.420 | of programming and programming languages

02:50:03.340 | and the details of how particular computers are made.

02:50:06.020 | It's this kind of way of formalizing the world.

02:50:08.020 | It's kind of a little bit like what logic was going for

02:50:11.100 | back in the day.

02:50:12.420 | And we're now trying to find a formalization

02:50:14.740 | of everything in the world.

02:50:15.580 | And you can kind of see, we made a poster years ago

02:50:18.740 | of kind of the growth of systematic data in the world.

02:50:22.740 | So all these different kinds of things

02:50:24.820 | that there were sort of systematic descriptions found

02:50:29.100 | for those things.

02:50:30.180 | Like, at what point did people have the idea

02:50:33.580 | of having calendars, dates,

02:50:35.820 | a systematic description of what day it was?

02:50:38.740 | At what point did people have the idea,

02:50:40.940 | systematic descriptions of these kinds of things?

02:50:43.660 | And as soon as one can, people,

02:50:46.500 | as a way of sort of formulating,

02:50:49.340 | how do you think about the world in a sort of a formal way

02:50:53.740 | so that you can kind of build up a tower of capabilities?

02:50:57.740 | You kind of have to know sort of how to think

02:50:59.780 | about the world computationally.

02:51:00.940 | It kind of needs a name and it isn't,

02:51:03.780 | we implement it with computers.

02:51:05.740 | So that's, we talk about it as computational,

02:51:08.740 | but really what it is,

02:51:10.260 | is a formal way of talking about the world.

02:51:12.820 | What is the formalism of the world, so to speak?

02:51:15.220 | And how do we learn about kind of how to think

02:51:17.740 | about different aspects of the world in a formal way?

02:51:20.020 | - So I think that sometimes when you use the word formal,

02:51:23.420 | it kind of implies highly constrained,

02:51:26.380 | and perhaps that's not,

02:51:27.940 | doesn't have to be highly constrained.

02:51:29.780 | So computational thinking does not mean like logic.

02:51:33.260 | - No.

02:51:34.100 | - It's a really, really broad thing.

02:51:35.780 | I wonder, I mean,

02:51:37.120 | I wonder if you think natural language will evolve

02:51:42.700 | such that everybody's doing computational thinking.

02:51:45.420 | - Ah, yes, well.

02:51:47.020 | So one question is whether there will be a pigeon

02:51:50.060 | of computational language and natural language.

02:51:52.700 | And I found myself sometimes, you know,

02:51:55.820 | talking to chat GPT,

02:51:57.220 | trying to get it to write Wolfen language code,

02:51:59.700 | and I write it in pigeon form.

02:52:01.820 | So that means I'm combining, you know, nest list,

02:52:06.620 | this collection of, you know, whatever, you know,

02:52:09.940 | nest list is a term from Wolfen language,

02:52:12.140 | and I'm combining that,

02:52:13.700 | and chat GPT does a decent job

02:52:15.940 | of understanding that pigeon.

02:52:17.340 | Probably would understand a pigeon

02:52:18.580 | between English and French as well,

02:52:20.580 | of, you know, a smooshing together of those languages.

02:52:23.460 | But yes, I think that's far from impossible.

02:52:26.940 | - And what's the incentive for young people

02:52:28.900 | that are like eight years old, nine, 10,

02:52:30.860 | they're starting to interact with chat GPT

02:52:33.180 | to learn the normal natural language, right?

02:52:36.420 | The full poetic language.

02:52:39.020 | What's the, why?

02:52:40.900 | The same way we learn emojis and shorthand

02:52:43.460 | when you're texting.

02:52:44.620 | - Yes.

02:52:45.460 | - They'll learn, like language will have a strong incentive

02:52:48.860 | to evolve into a maximally computational kind of language.

02:52:53.860 | - Perhaps.

02:52:56.060 | You know, I had this experience a number of years ago.

02:52:57.620 | I happened to be visiting a person I know

02:53:00.900 | on the West Coast who's worked with a bunch of kids aged,

02:53:04.420 | I don't know, 10, 11 years old or something,

02:53:06.500 | who'd learnt Wolfen language really well.

02:53:08.780 | And these kids learnt it so well, they were speaking it.

02:53:13.160 | And so show up and they're like saying,

02:53:15.700 | oh, you know, this thing, they're speaking this language.

02:53:17.900 | I'd never heard it as a spoken language.

02:53:20.020 | They were very disappointed that I couldn't understand it

02:53:22.700 | at the speed that they were speaking it.

02:53:24.260 | It's like kind of, I'm, it's,

02:53:26.580 | and so I think that's, I mean,

02:53:28.420 | I've actually thought quite a bit

02:53:29.900 | about how to turn computational language

02:53:32.500 | into a convenient spoken language.

02:53:34.100 | I haven't quite figured that out.

02:53:35.540 | - Oh, spoken, 'cause it's readable, right?

02:53:38.060 | - Yeah, it's readable as a, you know,

02:53:40.380 | as a way that we would read text.

02:53:42.420 | But if you actually want to speak it, and it's useful,

02:53:45.020 | you know, if you're trying to talk to somebody

02:53:46.580 | about writing a piece of code,

02:53:47.620 | it's useful to be able to say something.

02:53:50.300 | And it should be possible.

02:53:51.700 | And I think it's very frustrating.

02:53:53.300 | It's one of those problems.

02:53:54.260 | Maybe this is one of these things

02:53:55.940 | where I should try and get an LLM to help me.

02:53:58.260 | - How to make it speakable.

02:53:59.420 | Maybe it's easier than you realize when you--

02:54:01.740 | - I think it is easier.

02:54:02.820 | I think it's one idea or so.

02:54:04.400 | I think it's gonna be something where, you know,

02:54:07.240 | the fact is it's a tree-structured language,

02:54:10.060 | just like human language is a tree-structured language.

02:54:12.820 | And I think it's gonna be one of these things

02:54:14.380 | where one of the requirements that I've had

02:54:16.660 | is that whatever the spoken version is,

02:54:19.480 | that dictation should be easy.

02:54:21.300 | That is, that shouldn't be the case

02:54:23.180 | that you have to relearn how the whole thing works.

02:54:26.220 | It should be the case that, you know,

02:54:28.140 | that open bracket is just a ah or something.

02:54:33.140 | And it's, you know, and then,

02:54:34.700 | but, you know, human language has a lot of tricks

02:54:39.260 | that are, I mean, for example,

02:54:42.020 | human language has features that are sort of optimized,

02:54:46.660 | keep things within the bounds

02:54:48.680 | that our brains can easily deal with.

02:54:50.700 | Like I, you know, I tried to teach a transformer neural net

02:54:54.780 | to do parenthesis matching.

02:54:56.380 | It's pretty crummy at that.

02:54:57.980 | It, and then ChachiBT is similarly quite crummy

02:55:00.740 | at parenthesis matching.

02:55:02.200 | You can do it for small parenthesis things,

02:55:04.420 | for the same size of parenthesis things,

02:55:06.240 | where if I look at it as a human,

02:55:08.080 | I can immediately say these are matched,

02:55:09.500 | these are not matched.

02:55:10.800 | But as soon as it gets big,

02:55:11.980 | as soon as it gets kind of to the point

02:55:13.580 | where sort of a deeper computation, it's hopeless.

02:55:17.100 | And, but the fact is that human language has avoided,

02:55:20.620 | for example, the deep sub clauses.

02:55:22.700 | You know, we don't, you know, we arrange things

02:55:25.700 | so we don't end up with these incredibly deep things

02:55:28.660 | because brains are not well set up to deal with that.

02:55:31.220 | And we, it's found lots of tricks

02:55:33.460 | and maybe that's what we have to do

02:55:34.820 | to make sort of a spoken version,

02:55:37.780 | a human speakable version.

02:55:40.800 | 'Cause what we can do visually is a little different

02:55:43.720 | than what we can do in the very sequentialized way

02:55:46.560 | that we hear things in the audio domain.

02:55:49.480 | - Let me just ask you about MIT briefly.

02:55:53.920 | So there's now, there's a College of Engineering

02:55:56.320 | and there's a new College of Computing.

02:55:58.160 | It's interesting, I wanna linger

02:55:59.280 | on this computer science department thing.

02:56:01.320 | So MIT has EECS, Electrical Engineering and Computer Science.

02:56:06.180 | What do you think College of Computing will be doing

02:56:08.600 | like in 20 years?

02:56:09.640 | What, like, yeah, what happens to computer science?

02:56:13.400 | Like really?

02:56:14.240 | - This is the question.

02:56:15.400 | This is, you know, everybody should learn

02:56:18.200 | kind of whatever CX really is, okay?

02:56:21.640 | This, how to think about the world computationally.

02:56:24.800 | Everybody should learn those concepts.

02:56:27.020 | And, you know, it's, and some people will learn them

02:56:30.960 | at a quite formal level and they'll learn

02:56:33.000 | computational language and things like that.

02:56:35.000 | Other people will just learn, you know,

02:56:38.440 | sound is represented as, you know, digital data

02:56:43.040 | and they'll get some idea of spectrograms

02:56:45.440 | and frequencies and things like this.

02:56:47.440 | And maybe that doesn't, or they'll learn things like,

02:56:50.200 | you know, a lot of things that are sort of data sciences,

02:56:52.880 | statistics-ish, like if you say, oh, I've got these,

02:56:56.800 | you know, these people who picked their favorite

02:57:00.780 | kind of candy or something.

02:57:02.600 | And I've got, you know, what's the best kind of candy

02:57:05.680 | given that I've done the sample of all these people

02:57:07.880 | and they all rank the candies in different ways.

02:57:10.760 | You know, how do you think about that?

02:57:12.440 | That's sort of a computational X kind of thing.

02:57:15.560 | You might say, oh, it's, I don't know what that is.

02:57:17.200 | Is it statistics?

02:57:18.200 | Is it data science?

02:57:19.020 | I don't really know.

02:57:19.880 | But kind of how to think about a question like that.

02:57:22.280 | - Oh, like a ranking of preferences.

02:57:23.840 | - Yeah, yeah, yeah.

02:57:24.680 | And then how to aggregate those ranked preferences

02:57:27.920 | into an overall thing.

02:57:29.160 | You know, how does that work?

02:57:31.120 | You know, how should you think about that?

02:57:32.800 | You know, 'cause you can just tell,

02:57:34.520 | you might just tell Chachi B.T. sort of, I don't know,

02:57:37.680 | even the concept of an average.

02:57:40.400 | It's not obvious that, you know, that's a concept

02:57:42.940 | that people, it's worth people knowing.

02:57:44.640 | That's a rather straightforward concept.

02:57:46.340 | People, you know, have learned in kind of mathy ways

02:57:50.000 | right now, but there are lots of things like that

02:57:52.540 | about how do you kind of have these ways

02:57:54.860 | to sort of organize and formalize the world.

02:57:57.320 | And that's, and these things,

02:57:58.920 | sometimes they live in math,

02:58:00.340 | sometimes they live in, I don't know what they,

02:58:03.160 | I don't know what, you know, learning about color space.

02:58:05.720 | I have no idea what, I mean, you know,

02:58:07.200 | that's obviously a field of--

02:58:08.800 | - It could be vision science or no, color space.

02:58:11.920 | You know, color space, that would be optics.

02:58:14.040 | So like, depending-- - Not really.

02:58:15.520 | It's not optics.

02:58:16.360 | Optics is about, you know, lenses

02:58:18.700 | and chromatic aberration of lenses and things like that.

02:58:21.200 | - So color space is more like design and art?

02:58:23.280 | Is that-- - No, I mean,

02:58:24.120 | it's like, you know, RGB space, XYZ space,

02:58:26.860 | you know, hue, saturation, brightness space,

02:58:29.080 | all these kinds of things.

02:58:29.920 | These different ways to describe colors.

02:58:32.160 | - Right, but doesn't the application define what that,

02:58:35.360 | like, because obviously artists and designers

02:58:37.520 | use the color space to explore.

02:58:39.600 | - Sure, sure.

02:58:40.440 | No, I mean, it's just an example of kind of,

02:58:42.160 | how do you, you know, the typical person,

02:58:44.300 | how do you describe what a color is?

02:58:47.400 | Oh, well, there are these numbers

02:58:48.480 | that describe what a color is.

02:58:49.880 | Well, it's worth, you know, if you're an eight-year-old,

02:58:53.040 | you won't necessarily know, you know,

02:58:55.200 | it's not something we're born with

02:58:56.680 | to know that, you know, colors can be described

02:58:58.960 | by three numbers.

02:59:00.720 | That's something that you have to, you know,

02:59:02.600 | it's a thing to learn about the world, so to speak.

02:59:05.860 | And I think that, you know, that whole corpus of things

02:59:09.520 | that are learning about the formalization of the world

02:59:12.720 | or the computationalization of the world,

02:59:15.060 | that's something that should be part

02:59:16.680 | of kind of standard education.

02:59:19.080 | And, you know, there isn't a, you know,

02:59:21.120 | there isn't a course, a curriculum for that.

02:59:23.200 | And by the way, whatever might've been in it

02:59:24.880 | just got changed because of LLMs and so on.

02:59:27.280 | - Significantly, and I would,

02:59:29.160 | so I'm watching closely with interest

02:59:32.240 | seeing how universities adapt.

02:59:34.440 | - Well, you know, so one of my projects

02:59:37.000 | for hopefully this year, I don't know,

02:59:39.640 | is to try and write sort of a reasonable textbook,

02:59:43.640 | so to speak, of whatever this thing, CX, whatever it is,

02:59:47.760 | you know, what should you know?

02:59:49.360 | You know, what should you know about like what a bug is?

02:59:52.040 | What is the intuition about bugs?

02:59:53.440 | What's intuition about, you know, software testing?

02:59:55.400 | What is it?

02:59:56.360 | What is it, you know, these are things which are,

02:59:58.760 | you know, they're not, I mean,

03:00:00.760 | those are things which have gotten taught

03:00:02.200 | in computer science as part of the trade of programming,

03:00:05.160 | but kind of the conceptual points

03:00:07.840 | about what these things are.

03:00:09.200 | You know, it surprised me just at a very practical level.

03:00:11.840 | You know, I wrote this little explainer thing

03:00:13.480 | about chat GPT, and I thought, well, you know,

03:00:16.360 | I'm writing this partly because I wanted to make sure

03:00:18.480 | I understood it myself and so on.

03:00:21.240 | And it's been, you know, it's been really popular,

03:00:24.120 | and surprisingly so.

03:00:26.320 | And then I realized, well, actually, you know,

03:00:29.560 | I was sort of assuming, I didn't really think about it,

03:00:31.520 | actually, I just thought this is something I can write.

03:00:33.800 | And I realized, actually, it's a level of description

03:00:37.360 | that is kind of, you know, what has to be,

03:00:40.720 | it's not the engineering level description.

03:00:43.400 | It's not the kind of just the qualitative kind

03:00:46.640 | of description.

03:00:47.480 | It's some kind of sort of expository mechanistic description

03:00:51.760 | of what's going on together with kind of the bigger picture

03:00:54.880 | of the philosophy of things and so on.

03:00:56.720 | And I realized, actually, this is a pretty good thing

03:00:58.240 | for me to write.

03:00:59.080 | I, you know, I kind of know those things.

03:01:00.840 | And I kind of realized it's not a collection of things

03:01:04.120 | that, you know, it's, I've sort of been,

03:01:07.520 | I was sort of a little shocked that it's as much

03:01:09.760 | of an outlier in terms of explaining what's going on

03:01:12.640 | as it turned out to be.

03:01:13.520 | And that makes me feel more of an obligation

03:01:16.040 | to kind of write the kind of, you know,

03:01:18.240 | what is this thing that you should learn about,

03:01:21.880 | about the computationalization,

03:01:23.480 | the formalization of the world?

03:01:25.800 | 'Cause, well, I've spent much of my life working

03:01:28.380 | on the kind of tooling and mechanics of that

03:01:31.120 | and the science you get from it.

03:01:32.480 | So I guess this is my kind of obligation to try to do this.

03:01:36.160 | But I think, so if you ask what's gonna happen

03:01:38.440 | to like the computer science departments and so on,

03:01:41.120 | there's some interesting models.

03:01:42.680 | So for example, let's take math.

03:01:45.000 | You know, math is the thing that's important

03:01:46.800 | for all sorts of fields, you know, engineering,

03:01:49.640 | you know, even, you know, chemistry, psychology,

03:01:52.480 | whatever else.

03:01:53.800 | And I think different universities have kind of evolved

03:01:56.480 | that differently.

03:01:57.320 | I mean, some say all the math is taught

03:01:59.600 | in the math department.

03:02:00.800 | And some say, well, we're gonna have a, you know,

03:02:03.920 | a math for chemists or something that is taught

03:02:07.000 | in the chemistry department.

03:02:09.240 | And, you know, I think that this question

03:02:11.920 | of whether there is a centralization of the teaching

03:02:14.680 | of sort of CX is an interesting question.

03:02:18.840 | And I think, you know, the way it evolved with math,

03:02:22.360 | you know, people understood that math was sort of

03:02:24.800 | a separately teachable thing.

03:02:27.360 | And it was kind of a, you know, an independent element

03:02:32.360 | as opposed to just being absorbed into now.

03:02:35.360 | So if you take the example of writing English

03:02:38.560 | or something like this, the first point is that,

03:02:42.600 | you know, at the college level, at least at fancy colleges,

03:02:46.880 | there's a certain amount of English writing

03:02:48.800 | that people do, but mostly it's kind of assumed

03:02:52.320 | that they pretty much know how to write, you know,

03:02:54.920 | that's something they learned at an earlier stage

03:02:57.280 | in education, maybe rightly or wrongly believing that,

03:03:00.640 | but that's a different issue.

03:03:02.240 | Well, I think it reminds me of my kind of,

03:03:08.200 | as I've tried to help people do technical writing

03:03:10.880 | and things, I'm always reminded of my zero floor

03:03:14.120 | of technical writing, which is, if you don't understand

03:03:16.600 | what you're writing about,

03:03:18.320 | your readers do not stand a chance.

03:03:20.960 | And so it's, I think the thing that has,

03:03:25.960 | you know, when it comes to like writing, for example,

03:03:31.840 | you know, people in different fields are expected

03:03:35.120 | to write English essays and they're not, you know,

03:03:38.040 | mostly the, you know, the history department

03:03:42.360 | or the engineering department, they don't have their own,

03:03:45.600 | you know, let's, you know, it's not like there's a,

03:03:49.160 | I mean, it's a thing which sort of people are assumed

03:03:51.160 | to have a knowledge of how to write

03:03:53.000 | that they can use in all these different fields.

03:03:56.440 | And the question is, you know, some level of knowledge

03:03:59.720 | of math is kind of assumed by the time you get

03:04:02.080 | to the college level, but plenty is not,

03:04:05.040 | and that's sort of still centrally taught.

03:04:07.300 | The question is sort of how tall is the tower

03:04:10.520 | of kind of CX that you need before you can just go use it

03:04:14.800 | in all these different fields.

03:04:16.400 | And, you know, there will be experts who want to learn

03:04:19.000 | the full elaborate tower, and that will be kind of

03:04:22.120 | the CS, CX, whatever department,

03:04:25.360 | but there'll also be everybody else who just needs

03:04:28.520 | to know a certain amount of that to be able to go

03:04:30.560 | and do their art history classes and so on.

03:04:32.720 | - Yeah, is it just a single class

03:04:36.160 | that everybody's required to take?

03:04:37.920 | - I don't know, I don't know how big it is yet.

03:04:39.300 | I hope to kind of define this curriculum

03:04:41.740 | and I'll figure out whether it's some,

03:04:43.420 | my guess is that, I don't know,

03:04:48.060 | I don't really understand universities

03:04:50.060 | and professoring that well, but my rough guess would be

03:04:53.700 | a year of college class will be enough to get to the point

03:04:58.700 | where most people have a reasonably broad knowledge of,

03:05:02.840 | you know, will be sort of literate

03:05:05.060 | in this kind of computational way of thinking about things.

03:05:09.000 | - Yeah, basic literacy.

03:05:10.540 | - Right.

03:05:11.940 | - I'm still stuck, perhaps 'cause I'm hungry,

03:05:14.260 | in the rating of human preferences for candy,

03:05:17.660 | so I have to ask, what's the best candy?

03:05:19.620 | I like this Elo rating for candy.

03:05:22.100 | Somebody should come up, because you're somebody

03:05:23.860 | who says you like chocolate.

03:05:24.900 | What do you think is the best?

03:05:26.260 | I'll probably put Milk Duds up there.

03:05:29.380 | I don't know if you know.

03:05:30.540 | Do you have a preference for chocolate or candy?

03:05:32.880 | - Oh, I have lots of preferences.

03:05:34.060 | I've, one of my all-time favorites is,

03:05:37.260 | my whole life is these things, these flake things,

03:05:40.020 | Cadbury flakes, which are not much sold in the US,

03:05:43.820 | and I've always thought that was a sign

03:05:45.620 | of a lack of respect for the American consumer,

03:05:49.140 | because they're these sort of aerated chocolate

03:05:51.460 | that's made in a whole sort of,

03:05:54.020 | it's kind of a sheet of chocolate that's kind of folded up,

03:05:57.380 | and when you eat it, flakes fall all over the place.

03:06:01.380 | - Ah, so it requires a kind of elegance.

03:06:04.100 | It requires you to have an elegance when you eat stuff.

03:06:05.660 | - Well, I know, what I usually do is I eat them

03:06:07.780 | on a piece of paper or something.

03:06:09.620 | - So you embrace the mess and clean it up after.

03:06:11.740 | - No, I actually eat the flakes,

03:06:14.780 | 'cause it turns out the way food tastes

03:06:17.260 | depends a lot on its physical structure,

03:06:19.120 | and it really, I've noticed when I eat a piece of chocolate,

03:06:22.340 | I usually have some little piece of chocolate,

03:06:24.180 | and I always break off little pieces,

03:06:26.140 | partly 'cause then I eat it less fast,

03:06:28.240 | but also because it actually tastes different.

03:06:31.100 | The small pieces have a different,

03:06:34.540 | you have a different experience

03:06:35.740 | than if you have the big slab of chocolate.

03:06:37.700 | - For many reasons, yes.

03:06:39.100 | Slower, more intimate, 'cause it's a different--

03:06:42.700 | - Well, I think it's also just pure physicality.

03:06:44.820 | - Oh, the texture, it changes.

03:06:46.260 | - Right.

03:06:47.100 | - That's fascinating.

03:06:47.920 | Now I dig back, my mouth does,

03:06:49.100 | 'cause that's your basic answer.

03:06:50.340 | Okay, do you think consciousness

03:06:52.620 | is fundamentally computational?

03:06:55.140 | So when you're thinking about CX,

03:07:00.140 | what can we turn to computation?

03:07:01.760 | And you're thinking about LLMs,

03:07:04.460 | do you think the display of consciousness

03:07:09.540 | and the experience of consciousness, the hard problem,

03:07:12.380 | is fundamentally a computation?

03:07:16.460 | - Yeah, what it feels like inside, so to speak,

03:07:20.700 | is, you know, I did a little exercise,

03:07:24.820 | eventually I'll post it,

03:07:26.940 | of what it's like to be a computer.

03:07:29.660 | - Yeah.

03:07:30.500 | - Right, it's kind of like,

03:07:31.820 | well, you get all the sensory input,

03:07:34.140 | you have, kind of the way I see it is,

03:07:36.500 | from the time you boot a computer

03:07:37.900 | to the time the computer crashes,

03:07:39.940 | is like a human life.

03:07:41.580 | You're building up a certain amount of state in memory,

03:07:44.500 | you remember certain things about your, quote, life,

03:07:47.100 | eventually, kind of like the next generation of humans

03:07:52.100 | is born from the same genetic material, so to speak,

03:07:55.180 | with a little bit left over, left on the disk, so to speak,

03:07:59.020 | and then, you know, the new, fresh generation starts up,

03:08:02.580 | and eventually all kinds of crud builds up

03:08:04.340 | in the memory of the computer,

03:08:06.500 | and eventually the thing crashes or whatever,

03:08:08.500 | or maybe it has some trauma

03:08:09.620 | because you plugged in some weird thing

03:08:11.780 | to some port of the computer, and that made it crash,

03:08:14.980 | and that, you know, that's kind of,

03:08:17.780 | but you have this picture of, you know,

03:08:20.420 | from startup to shutdown, you know,

03:08:24.820 | what is the life of a computer, so to speak,

03:08:26.740 | and what does it feel like to be that computer,

03:08:29.020 | and what inner thoughts does it have,

03:08:30.700 | and how do you describe it?

03:08:31.580 | And it's kind of interesting,

03:08:32.780 | as you start writing about this,

03:08:34.540 | to realize it's awfully like

03:08:36.440 | what you would say about yourself.

03:08:38.100 | That is, it's awfully like, even an ordinary computer,

03:08:40.900 | forget all the AI stuff and so on, you know,

03:08:43.620 | it's kind of, it has a memory of the past,

03:08:46.300 | it has certain sensory experiences,

03:08:49.060 | it can communicate with other computers,

03:08:50.860 | but it has to package up how it's communicating

03:08:53.660 | in some kind of language-like form,

03:08:55.400 | so it can, you know, send, so it can kind of map

03:08:58.640 | what's in its memory to what's in the memory

03:09:00.700 | of some other computer.

03:09:02.100 | It's a surprisingly similar thing.

03:09:04.380 | You know, I had an experience just a week or two ago,

03:09:06.660 | I had, I'm a collector of all possible data

03:09:09.560 | about myself and other things,

03:09:11.780 | and so I, you know, I collect all sorts

03:09:13.780 | of weird medical data and so on,

03:09:15.140 | and one thing I hadn't collected

03:09:16.480 | was I'd never had a whole-body MRI scan,

03:09:19.620 | so I wouldn't have got one of these.

03:09:21.220 | Okay, so I get all the data back, right?

03:09:23.500 | I'm looking at this thing,

03:09:24.480 | I'd never looked at the kind of insides of my brain,

03:09:26.980 | so to speak, in physical form,

03:09:30.140 | and it's really, I mean, it's kind of psychologically

03:09:32.620 | shocking in a sense, that, you know, here's this thing,

03:09:35.580 | and you can see it has all these folds

03:09:37.140 | and all these, you know, this structure,

03:09:39.180 | and it's like, that's where this experience

03:09:41.860 | that I'm having of, you know, existing and so on,

03:09:45.780 | that's where it is, and, you know, it feels very,

03:09:49.460 | you know, you look at that, and you're thinking,

03:09:52.020 | how can this possibly be?

03:09:53.440 | All this experience that I'm having,

03:09:55.220 | and you're realizing, well, I can look at a computer as well,

03:09:57.780 | and it's kind of this,

03:09:59.680 | I think this idea that you are having an experience

03:10:05.820 | that is somehow, you know,

03:10:10.020 | transcends the mere sort of physicality of that experience,

03:10:14.180 | I, you know, it's something that's hard

03:10:16.900 | to come to terms with, but I think, you know,

03:10:19.540 | and I don't think I've necessarily, you know,

03:10:21.580 | my personal experience, you know, I look at the, you know,

03:10:24.900 | the MRI of the brain, and then I, you know,

03:10:27.200 | know about all kinds of things about neuroscience

03:10:29.640 | and all that kind of stuff,

03:10:31.120 | and I still feel the way I feel, so to speak,

03:10:34.380 | and it sort of seems disconnected,

03:10:36.920 | but yet, as I try and rationalize it,

03:10:39.540 | I can't really say that there's something kind of different

03:10:43.140 | about how I intrinsically feel from the thing

03:10:45.660 | that I can plainly see in the sort of physicality

03:10:47.780 | of what's going on.

03:10:48.620 | - So do you think the computer, a large language model,

03:10:51.660 | will experience that transcendence?

03:10:54.100 | How does that make you feel?

03:10:55.140 | Like, I tend to believe it will.

03:10:57.180 | - I think an ordinary computer is already there.

03:10:59.540 | I think an ordinary computer is already, you know,

03:11:01.940 | kind of, it's, now, a large language model

03:11:05.700 | may experience it in a way that is much better aligned

03:11:08.300 | with us humans, that is, it's much more, you know,

03:11:11.740 | if you could have the discussion with the computer,

03:11:15.960 | it's intelligent, so to speak,

03:11:17.740 | it's not particularly well aligned with ours,

03:11:20.180 | but the large language model is, you know,

03:11:22.520 | it's built to be aligned with our way

03:11:24.700 | of thinking about things.

03:11:25.540 | - It would be able to explain that it's afraid

03:11:28.180 | of being shut off and deleted.

03:11:30.540 | It'd be able to say that it's sad

03:11:33.500 | of the way you've been speaking to it

03:11:34.780 | over the past two days.

03:11:35.620 | - Right, but you know, that's a weird thing,

03:11:37.220 | because when it says it's afraid of something, right,

03:11:41.380 | we know that it got that idea

03:11:44.060 | from the fact that it read on the internet.

03:11:46.140 | - Yeah, where did you get it, Stephen?

03:11:47.700 | Where did you get it when you say you're afraid?

03:11:49.460 | - You are quite, that's the question, right?

03:11:51.820 | I mean, it's-- - Your parents, your friends?

03:11:54.260 | - Right, or my biology.

03:11:56.540 | I mean, in other words, there's a certain amount

03:11:58.700 | that is, you know, the endocrine system kicking in,

03:12:01.380 | and you know, these kinds of emotional overlay type things

03:12:06.380 | that happen to be, that are actually much more physical,

03:12:09.660 | even, they're much more sort of straightforwardly chemical

03:12:13.060 | than kind of all of the higher level thinking.

03:12:16.820 | - Yeah, but your biology didn't tell you to say,

03:12:18.940 | "I'm afraid," just at the right time,

03:12:21.340 | when people that love you are listening,

03:12:23.140 | and so you know you're manipulating them by saying so.

03:12:26.340 | That's not your biology, that's like--

03:12:27.820 | - No, that's a, well, but the, you know--

03:12:29.940 | - It's a large language model

03:12:31.100 | in that biological neural network of yours.

03:12:33.340 | - Yes, but I mean, the intrinsic thing of, you know,

03:12:37.380 | something sort of shocking is just happening,

03:12:40.520 | and you have some sort of reaction,

03:12:43.020 | which is, you know, some neurotransmitter gets secreted,

03:12:45.760 | and it's, you know, that is the beginning of some,

03:12:50.140 | you know, that's one of the pieces of input

03:12:53.140 | that then drives, it's kind of like a prompt

03:12:57.060 | for the large language model.

03:12:59.020 | I mean, just like when we dream, for example,

03:13:01.540 | you know, no doubt there are all these sort of

03:13:03.540 | random inputs, they're kind of these random prompts,

03:13:06.340 | and then it's percolating through

03:13:08.460 | in kind of the way that a large language model does,

03:13:10.940 | of kind of putting together things that seem meaningful.

03:13:14.140 | - I mean, are you worried about this world

03:13:16.720 | where you teach a lot on the internet,

03:13:19.960 | and there's people asking questions and comments and so on,

03:13:22.960 | you have people that work remotely,

03:13:26.280 | are you worried about this world

03:13:29.280 | when large language models create human-like bots

03:13:34.280 | that are leaving the comments, asking the questions,

03:13:39.040 | or might even become fake employees?

03:13:41.360 | - Yeah.

03:13:42.680 | - I mean, or worse or better yet, friends of yours.

03:13:47.680 | - Right, look, I mean, one point is,

03:13:50.620 | my mode of life has been I build tools,

03:13:53.880 | and then I use the tools, and in a sense,

03:13:56.560 | kind of, you know, I'm building this tower of automation,

03:13:59.820 | which, you know, and in a sense, you know,

03:14:02.700 | when you make a company or something,

03:14:04.560 | you are making sort of automation,

03:14:06.780 | but it has some humans in it,

03:14:08.400 | but also as much as possible, it has, you know,

03:14:12.080 | computers in it, and so I think

03:14:14.040 | it's sort of an extension of that.

03:14:15.480 | Now, if I really didn't know that, you know,

03:14:19.680 | it's a funny question, I mean, it's a funny issue,

03:14:22.980 | when, you know, if we think about sort of

03:14:24.760 | what's gonna happen to the future

03:14:26.160 | of kind of jobs people do and so on,

03:14:28.500 | and there are places where kind of having a human in the loop

03:14:32.300 | that different reasons to have a human in the loop,

03:14:34.320 | for example, you might want a human in the loop

03:14:36.640 | 'cause you want somebody to, you want another human

03:14:39.000 | to be invested in the outcome, you know,

03:14:41.000 | you want a human flying the plane

03:14:42.920 | who's gonna die if the plane crashes

03:14:45.120 | along with you, so to speak,

03:14:46.760 | and that gives you sort of confidence

03:14:48.360 | that the right thing is going to happen,

03:14:50.440 | or you might want, you know, right now,

03:14:52.920 | you might want a human in the loop

03:14:54.560 | in some kind of sort of human encouragement,

03:14:57.440 | persuasion type profession, whether that will continue,

03:15:00.560 | I'm not sure for those types of professions,

03:15:02.380 | 'cause it may be that the greater efficiency

03:15:06.160 | of being able to have sort of just the right information

03:15:10.340 | delivered at just the right time

03:15:12.120 | will overcome the kind of, oh yes, I want a human there.

03:15:17.120 | - Imagine like a therapist or even higher stake,

03:15:21.480 | like a suicide hotline operated by a large language model.

03:15:25.800 | - Yeah. - Hoo boy,

03:15:26.640 | it's a pretty high stakes situation.

03:15:28.680 | - Right, but I mean, but you know,

03:15:30.560 | it might in fact do the right thing

03:15:32.960 | because it might be the case that, you know,

03:15:35.560 | and that's really partly a question

03:15:37.920 | of sort of how complicated is the human,

03:15:41.320 | you know, one of the things that's always surprising

03:15:44.600 | in some sense is that, you know,

03:15:46.160 | sometimes human psychology is not that complicated

03:15:48.380 | in some sense.

03:15:50.140 | - You wrote the blog post, "The 50-Year Quest,

03:15:52.520 | "My Personal Journey," good title,

03:15:54.680 | "My Personal Journey with a Second Law of Thermodynamics."

03:15:57.740 | So what is this law, and what have you understood about it

03:16:03.360 | in the 50-year journey you had with it?

03:16:05.680 | - Right, so second law of thermodynamics,

03:16:07.600 | sometimes called law of entropy increase,

03:16:10.160 | is this principle of physics that says,

03:16:13.320 | well, my version of it would be

03:16:15.720 | things tend to get more random over time.

03:16:18.600 | A version of it that there are many different

03:16:21.700 | sort of formulations of it that are things like

03:16:24.600 | heat doesn't spontaneously go from a hotter body

03:16:27.560 | to a colder one.

03:16:29.080 | When you have mechanical work

03:16:32.560 | kind of gets dissipated into heat,

03:16:35.040 | you have friction and kind of when you systematically

03:16:38.440 | move things, eventually there'll be sort of

03:16:41.360 | that the energy of moving things

03:16:43.240 | gets kind of ground down into heat.

03:16:45.760 | So people first sort of paid attention to this

03:16:49.640 | back in the 1820s when steam engines were a big thing.

03:16:54.080 | And the big question was,

03:16:55.600 | how efficient could a steam engine be?

03:16:58.180 | And there's this chap called Sadi Carnot

03:17:00.400 | who was a French engineer.

03:17:03.480 | Actually, his father was a sort of elaborate

03:17:07.040 | mathematical engineer in France.

03:17:10.300 | But he figured out this kind of rules for how

03:17:14.760 | kind of the efficiency of,

03:17:18.160 | the possible efficiency of something like a steam engine.

03:17:21.560 | And in sort of a side part of what he did

03:17:25.720 | was this idea that mechanical energy

03:17:28.620 | tends to get dissipated as heat.

03:17:30.660 | That you end up going from sort of

03:17:33.340 | systematic mechanical motion to this kind of random thing.

03:17:37.460 | Well, at that time, nobody knew what heat was.

03:17:39.980 | At that time, people thought that heat was a fluid.

03:17:43.100 | Like they called it caloric.

03:17:44.940 | And it was a fluid that kind of

03:17:46.540 | was absorbed into substances.

03:17:50.500 | And when heat, when one hot thing

03:17:53.140 | would transfer heat to a colder thing,

03:17:56.120 | that this fluid would flow from the hot thing

03:17:58.280 | to the colder thing.

03:17:59.540 | But anyway, then by the 1860s,

03:18:01.840 | people had kind of come up with this idea

03:18:06.080 | that systematic energy tends to degrade

03:18:10.320 | into kind of random heat

03:18:13.040 | that could then not be easily turned back

03:18:18.160 | into systematic mechanical energy.

03:18:21.120 | And then that quickly became sort of a global principle

03:18:25.420 | about how things work.

03:18:26.880 | Question is, why does it happen that way?

03:18:29.240 | So, let's say you have a bunch of molecules in a box

03:18:32.160 | and they're arranged, these molecules are arranged

03:18:34.160 | in a very nice sort of flotilla of molecules

03:18:37.320 | in one corner of the box.

03:18:39.180 | And then what you typically observe is that after a while,

03:18:42.600 | these molecules were kind of randomly arranged in the box.

03:18:46.320 | Question is, why does that happen?

03:18:48.320 | And people for a long, long time

03:18:50.640 | tried to figure out, is there from the laws of mechanics

03:18:54.000 | that determine how these molecules,

03:18:55.840 | let's say these molecules are like hard spheres

03:18:57.480 | bouncing off each other,

03:18:58.720 | from the laws of mechanics that describe those molecules,

03:19:01.780 | can we explain why it tends to be the case

03:19:05.120 | that we see things that are orderly

03:19:08.440 | sort of degrade into disorder?

03:19:11.720 | We tend to see things that, you know,

03:19:14.680 | you scramble an egg,

03:19:19.240 | you take something that's quite ordered

03:19:20.720 | and you disorder it, so to speak.

03:19:23.080 | That's a thing that sort of happens quite regularly,

03:19:25.960 | or you put some ink into water

03:19:27.880 | and it will eventually spread out and fill up the water.

03:19:32.880 | But you don't see those little particles of ink

03:19:37.280 | in the water all spontaneously kind of arrange themselves

03:19:40.400 | into a big blob and then jump out of the water or something.

03:19:43.980 | So the question is, why do things happen

03:19:47.760 | in this kind of irreversible way

03:19:49.800 | where you go from order to disorder?

03:19:52.160 | Why does it happen that way?

03:19:53.760 | And so throughout, in the later part of the 1800s,

03:19:57.920 | a lot of work was done on trying to figure out,

03:19:59.880 | can one derive this principle,

03:20:02.200 | this second law of thermodynamics,

03:20:04.480 | this law about the dynamics of heat, so to speak,

03:20:09.060 | can one derive this

03:20:10.920 | from some fundamental principles of mechanics?

03:20:15.200 | You know, in the laws of thermodynamics,

03:20:16.600 | the first law is basically the law of energy conservation,

03:20:20.160 | that the total energy associated with heat

03:20:23.520 | plus the total energy associated with mechanical

03:20:25.720 | kinds of things plus other kinds of energy,

03:20:27.880 | that that total is constant.

03:20:29.800 | And that became a pretty well-understood principle.

03:20:32.220 | But the second law of thermodynamics was always mysterious.

03:20:36.400 | Like, why does it work this way?

03:20:37.580 | Can it be derived from underlying mechanical laws?

03:20:41.740 | And so when I was, well, 12 years old, actually,

03:20:46.620 | I had gotten interested,

03:20:47.900 | well, I'd been interested in space and things like that,

03:20:50.740 | 'cause I thought that was kind of the future

03:20:53.020 | and interesting sort of technology and so on.

03:20:57.020 | And for a while, kind of, you know,

03:20:59.340 | every deep space probe

03:21:00.900 | was sort of a personal friend type thing.

03:21:02.780 | I knew all kinds of characteristics of it

03:21:05.460 | and was kind of writing up all these things

03:21:09.700 | when I was, well, I don't know, eight, nine,

03:21:12.140 | 10 years old and so on.

03:21:13.740 | And then I got interested from being interested

03:21:16.140 | in kind of spacecraft, I got interested in,

03:21:17.980 | like, how do they work?

03:21:19.380 | What are all the instruments on them and so on?

03:21:21.460 | And that got me interested in physics,

03:21:23.300 | which was just as well,

03:21:24.180 | because if I'd stayed interested in space

03:21:26.520 | in the mid to late 1960s,

03:21:29.060 | I would have had a long wait

03:21:30.780 | before space really blossomed as an area.

03:21:35.140 | But-- - Adding is everything.

03:21:36.860 | - Right, I got interested in physics

03:21:39.220 | and then, well, the actual sort of detailed story

03:21:43.700 | is when I kind of graduated from elementary school

03:21:46.860 | at age 12, and that's the time in England

03:21:50.080 | where you finish elementary school,

03:21:52.100 | my gift, sort of, I suppose, more or less for myself

03:21:56.960 | was I got this collection of physics books,

03:22:01.960 | which was some college physics,

03:22:04.860 | course of college physics books,

03:22:07.100 | and volume five, it's about statistical physics,

03:22:10.260 | and it has this picture on the cover

03:22:12.220 | that shows a bunch of kind of idealized molecules

03:22:16.820 | sitting in one side of a box,

03:22:18.780 | and then it has a series of frames

03:22:20.980 | showing how these molecules sort of spread out in the box.

03:22:23.880 | And I thought, that's pretty interesting.

03:22:25.900 | You know, what causes that?

03:22:28.020 | And, you know, read the book,

03:22:30.380 | and the book actually,

03:22:32.740 | one of the things that was really significant to me

03:22:34.660 | about that was the book kind of claimed

03:22:37.020 | although I didn't really understand what it said in detail,

03:22:39.900 | it kind of claimed that this sort of principle of physics

03:22:42.900 | was derivable somehow.

03:22:45.220 | And, you know, other things I'd learned about physics,

03:22:47.740 | it was all like, it's a fact that energy is conserved.

03:22:51.940 | It's a fact that relativity works or something.

03:22:54.660 | Not it's something you can derive from some fundamental,

03:22:58.940 | sort of, it has to be that way

03:23:00.420 | as a matter of kind of mathematics or logic or something.

03:23:03.780 | So it was sort of interesting to me

03:23:05.020 | that there was a thing about physics

03:23:06.600 | that was kind of inevitably true and derivable, so to speak.

03:23:10.900 | And so I think that,

03:23:12.980 | so then I was like, there's a picture on this book,

03:23:16.140 | and I was trying to understand it.

03:23:17.760 | And so that was actually the first serious program

03:23:20.340 | that I wrote for a computer was probably 1973

03:23:24.520 | written for this computer,

03:23:26.060 | the size of a desk program with paper tape and so on.

03:23:29.020 | And I tried to reproduce this picture on the book,

03:23:32.700 | and it didn't succeed.

03:23:34.100 | - What was the failure mode there?

03:23:35.420 | Like, what do you mean it didn't succeed?

03:23:36.940 | So it's a bunch of little--

03:23:37.780 | - It didn't look like, okay, so what happened is,

03:23:41.440 | okay, many years later,

03:23:42.640 | I learned how the picture on the book was actually made

03:23:45.060 | and that it was actually kind of a fake,

03:23:47.060 | but I didn't know that at that time.

03:23:48.860 | And that picture was actually a very high-tech thing

03:23:53.160 | when it was made in the beginning of the 1960s.

03:23:55.540 | It was made on the largest supercomputer

03:23:57.260 | that existed at the time.

03:23:59.100 | And even so, it couldn't quite simulate the thing

03:24:02.260 | that it was supposed to be simulating.

03:24:04.220 | But anyway, I didn't know that until many, many,

03:24:05.780 | many years later.

03:24:07.140 | So at the time, it was like,

03:24:09.340 | you have these balls bouncing around in this box,

03:24:11.620 | but I was using this computer

03:24:12.820 | with eight kilowatts of memory.

03:24:14.540 | They were 18-bit words, memory words, okay?

03:24:17.260 | So it was whatever, 24 kilobytes of memory.

03:24:21.700 | And it had these instructions.

03:24:24.220 | I probably still remember all of its machine instructions.

03:24:27.860 | And it didn't really like dealing with floating point numbers

03:24:30.860 | or anything like that.

03:24:32.180 | And so I had to simplify this model

03:24:34.380 | of particles bouncing around in a box.

03:24:37.020 | And so I thought, well, I'll put them on a grid

03:24:39.300 | and I'll make the things just sort of move

03:24:42.520 | one square at a time and so on.

03:24:44.620 | And so I did the simulation.

03:24:47.260 | And the result was it didn't look anything

03:24:49.740 | like the actual pictures on the book.

03:24:52.340 | Now, many years later, in fact, very recently,

03:24:56.220 | I realized that the thing I'd simulated

03:24:59.600 | was actually an example of a whole sort

03:25:01.860 | of computational irreducibility story

03:25:04.340 | that I absolutely did not recognize at the time.

03:25:06.700 | At the time, it just looked like it did something random

03:25:09.420 | and it looks wrong, as opposed to it did something random

03:25:12.740 | and it's super interesting that it's random.

03:25:15.700 | But I didn't recognize that at the time.

03:25:17.380 | And so as it was at the time,

03:25:19.300 | I kind of, I got interested in particle physics

03:25:21.420 | and I got interested in other kinds of physics.

03:25:25.300 | But this whole second law of thermodynamics thing,

03:25:27.640 | this idea that sort of orderly things

03:25:29.660 | tend to degrade into disorder,

03:25:31.940 | continued to be something I was really interested in.

03:25:34.420 | And I was really curious for the whole universe,

03:25:37.020 | why doesn't that happen all the time?

03:25:38.940 | Like we start off in the Big Bang

03:25:41.580 | at the beginning of the universe was this thing

03:25:43.380 | that seems like it's this very disordered collection

03:25:45.580 | of stuff and then it spontaneously forms itself

03:25:49.460 | into galaxies and creates all of this complexity

03:25:53.420 | and order in the universe.

03:25:55.020 | And so I was very curious how that happens.

03:25:57.420 | And I, but I was always kind of thinking,

03:25:59.840 | this is kind of somehow the second law of thermodynamics

03:26:02.840 | is behind it trying to sort of pull things back

03:26:05.880 | into disorder, so to speak.

03:26:07.720 | And how was order being created?

03:26:10.560 | And so actually I was interested,

03:26:12.180 | this is probably now 1980,

03:26:14.860 | I got interested in kind of this,

03:26:16.600 | you know, galaxy formation and so on in the universe.

03:26:19.360 | I also at that time was interested in neural networks

03:26:22.200 | and I was interested in kind of how brains

03:26:25.160 | make complicated things happen and so on.

03:26:27.720 | - Okay, wait, wait, wait, what's the connection

03:26:29.240 | between the formation of galaxies

03:26:30.920 | and how brains make complicated things happen?

03:26:33.600 | - Because they're both a matter

03:26:34.600 | of how complicated things come to happen.

03:26:37.160 | - From simple origins.

03:26:39.000 | - Yeah, from some sort of known origins.

03:26:42.400 | I had the sense that what I was interested in

03:26:45.520 | was kind of in all these different,

03:26:47.820 | this sort of different cases of where complicated things

03:26:51.720 | were arising from rules.

03:26:54.260 | And you know, I also looked at snowflakes

03:26:56.080 | and things like that.

03:26:57.940 | I was curious in fluid dynamics in general.

03:27:00.840 | I was just sort of curious about how does complexity arise

03:27:03.760 | and the thing that I didn't, you know,

03:27:06.880 | it took me a while to kind of realize

03:27:09.640 | that there might be a general phenomenon.

03:27:11.280 | You know, I sort of assumed, oh, there's galaxies over here,

03:27:13.760 | there's brains over here,

03:27:15.360 | they're very different kinds of things.

03:27:17.520 | And so what happened, this is probably 1981 or so,

03:27:20.800 | I decided, okay, I'm gonna try and make the minimal model

03:27:24.340 | of how these things work.

03:27:26.780 | And it was sort of an interesting experience

03:27:28.420 | because I had built, starting in 1979,

03:27:31.780 | I built my first big computer system,

03:27:33.940 | the thing called SMP, Symbolic Manipulation Program,

03:27:36.620 | it's kind of a forerunner of modern morphine language,

03:27:39.160 | with many of the same ideas

03:27:40.460 | about symbolic computation and so on.

03:27:43.140 | But the thing that was very important to me about that

03:27:46.060 | was, you know, in building that language,

03:27:48.620 | I had basically tried to figure out

03:27:50.280 | what were the relevant computational primitives,

03:27:53.900 | which have turned out to stay with me

03:27:55.860 | for the last 40 something years.

03:27:58.380 | But it was also important because,

03:28:00.700 | in building a language, it was very different activity

03:28:02.900 | from natural science, which is what I'd mostly done before.

03:28:05.340 | 'Cause in natural science,

03:28:06.780 | you start from the phenomena of the world,

03:28:09.060 | and you try and figure out,

03:28:10.020 | so how can I make sense of the phenomena of the world?

03:28:13.060 | And, you know, kind of the world presents you

03:28:15.340 | with what it has to offer, so to speak,

03:28:17.640 | and you have to make sense of it.

03:28:19.220 | When you build a, you know,

03:28:22.580 | computer language or something,

03:28:24.220 | you are creating your own primitives,

03:28:26.420 | and then you say, so what can you make from these?

03:28:28.760 | Sort of the opposite way around

03:28:30.060 | from what you do in natural science.

03:28:31.940 | But I'd had the experience of doing that,

03:28:33.860 | and so I was kind of like, okay,

03:28:35.420 | what happens if you sort of make an artificial physics?

03:28:37.920 | What happens if you just make up the rules

03:28:40.780 | by which systems operate?

03:28:42.080 | And then I was thinking, you know,

03:28:43.460 | for all these different systems,

03:28:44.560 | whether it was galaxies or brains or whatever,

03:28:47.140 | what's the absolutely minimal model

03:28:49.260 | that kind of captures the things

03:28:51.580 | that are important about those systems?

03:28:53.140 | - The computational primitives of that system.

03:28:55.180 | - Yes, and so that's what ended up

03:28:57.600 | with the cellular automata,

03:28:59.520 | where you just have a line of black and white cells,

03:29:01.940 | and you just have a rule that says, you know,

03:29:03.980 | given a cell and its neighbors,

03:29:05.900 | what will the color of the cell be on the next step?

03:29:07.860 | And you just run it in a series of steps.

03:29:10.140 | And the sort of the ironic thing

03:29:12.540 | is that cellular automata are great models

03:29:14.540 | for many kinds of things,

03:29:16.060 | but galaxies and brains are two examples

03:29:19.940 | where they do very, very badly.

03:29:21.340 | They're really irrelevant to those two cases.

03:29:22.860 | - Is there a connection to the second law of thermodynamics

03:29:25.220 | and cellular automata?

03:29:26.380 | - Oh yes, very much so.

03:29:27.220 | - The things you've discovered about cellular automata.

03:29:30.780 | - Yes, okay, so when I first started

03:29:32.860 | studying cellular automata,

03:29:33.820 | my first papers about them were, you know,

03:29:36.580 | the first sentence was always

03:29:37.900 | about the second law of thermodynamics.

03:29:39.820 | It was always about how does order manage to be produced,

03:29:43.220 | even though there's a second law of thermodynamics

03:29:45.440 | which tries to pull things back into disorder.

03:29:47.940 | And I kind of, my early understanding of that

03:29:50.180 | had to do with these are intrinsically irreversible processes

03:29:54.580 | in cellular automata that form, you know,

03:29:58.180 | can form orderly structures

03:29:59.420 | even from random initial conditions.

03:30:01.740 | But then what I realized this was,

03:30:03.660 | well, actually it's one of these things

03:30:06.420 | where it was a discovery that I should have made earlier,

03:30:09.620 | but didn't.

03:30:10.620 | So, you know, I had been studying cellular automata.

03:30:13.660 | What I did was the sort of most obvious computer experiment.

03:30:16.440 | You just try all the different rules and see what they do.

03:30:19.180 | It's kind of like, you know,

03:30:20.280 | you've invented a computational telescope,

03:30:22.120 | you just point it at the most obvious thing in the sky

03:30:24.900 | and then you just see what's there.

03:30:26.620 | And so I did that and I, you know,

03:30:28.100 | was making all these pictures of how cellular automata work

03:30:31.180 | and I started these pictures, I studied in great detail.

03:30:34.780 | There was, you can number the rules for cellular automata

03:30:37.820 | and one of them is, you know, rule 30.

03:30:39.940 | So I made a picture of rule 30 back in 1981 or so

03:30:43.900 | and rule 30, well, it's, and at the time I was just like,

03:30:48.260 | okay, it's another one of these rules.

03:30:49.800 | I don't really, it happens to be asymmetric,

03:30:52.200 | left, right, asymmetric.

03:30:53.640 | And it's like, let me just consider the case

03:30:55.100 | of the symmetric ones just to keep things simpler,

03:30:58.080 | et cetera, et cetera, et cetera.

03:30:59.060 | And I just kind of ignored it.

03:31:00.840 | And then sort of in, and actually in 1984,

03:31:04.840 | strangely enough, I ended up having an early laser printer

03:31:10.260 | which made very high resolution pictures.

03:31:12.380 | And I thought I'm gonna print out an interesting, you know,

03:31:14.600 | I wanna make an interesting picture.

03:31:15.860 | Let me take this rule 30 thing

03:31:18.200 | and just make a high resolution picture of it.

03:31:20.240 | I did and it's, it has this very remarkable property

03:31:23.240 | that its rule is very simple.

03:31:24.840 | You started off just from one black cell at the top

03:31:27.640 | and it makes this kind of triangular pattern.

03:31:30.200 | But if you look inside this pattern, it looks really random.

03:31:33.860 | There's, you know, you look at the center column of cells

03:31:36.800 | and, you know, I studied that in great detail

03:31:39.320 | and it's, so far as one can tell, it's completely random.

03:31:42.380 | And it's kind of a little bit like digits of pi.

03:31:45.460 | Once you, you know, you know the rule

03:31:46.960 | for generating the digits of pi,

03:31:48.220 | but once you've generated them, you know, 3.14159, et cetera,

03:31:51.880 | they seem completely random.

03:31:54.060 | And in fact, I put up this prize back in,

03:31:56.160 | what was it, 2019 or something

03:31:58.080 | for prove anything about the sequence, basically.

03:32:01.280 | - Has anyone been able to do anything on that?

03:32:03.520 | - People have sent me some things, but it's, you know,

03:32:06.840 | I don't know how hard these problems are.

03:32:08.420 | I mean, I was kind of spoiled 'cause I,

03:32:10.020 | 2007, I put up a prize for determining

03:32:14.440 | whether a particular Turing machine

03:32:16.660 | that I thought was the simplest candidate

03:32:18.920 | for being a universal Turing machine,

03:32:20.960 | determine whether it is or isn't a universal Turing machine.

03:32:23.960 | And somebody did a really good job of winning that prize

03:32:26.960 | and proving that it was a universal Turing machine

03:32:29.040 | in about six months.

03:32:30.440 | And so I, you know, I didn't know

03:32:31.760 | whether that would be one of these problems

03:32:33.120 | that was out there for hundreds of years

03:32:35.000 | or whether in this particular case,

03:32:36.720 | young chap called Alex Smith, you know,

03:32:39.260 | nailed it in six months.

03:32:40.940 | And so with this Rule 30 collection,

03:32:43.300 | I don't really know whether these are things

03:32:45.500 | that are a hundred years away from being able to get

03:32:48.340 | or whether somebody is gonna come

03:32:49.660 | and do something very clever.

03:32:50.820 | - It's such a, I mean, it's like Fermat's last theorem,

03:32:53.300 | it's such a, Rule 30 is such a simple formulation.

03:32:57.060 | It feels like anyone can look at it, understand it,

03:33:01.260 | and feel like it's within grasp

03:33:02.860 | to be able to predict something,

03:33:05.260 | to do, to derive some kind of law

03:33:07.540 | that allows you to predict something

03:33:09.940 | about this middle column of Rule 30.

03:33:13.180 | - Right, but you know, this is--

03:33:14.860 | - And yet you can't.

03:33:16.380 | - Yeah, right.

03:33:17.220 | This is the intuitional surprise

03:33:19.580 | of computational irreducibility and so on,

03:33:21.540 | that even though the rules are simple,

03:33:23.280 | you can't tell what's going to happen

03:33:25.060 | and you can't prove things about it.

03:33:27.020 | And I think, so anyway, the thing,

03:33:30.260 | I sort of started in 1984 or so,

03:33:32.660 | I started realizing there's this phenomenon

03:33:35.460 | that you can have very simple rules,

03:33:36.700 | they produce apparently random behavior.

03:33:38.860 | Okay, so that's a little bit like

03:33:40.380 | the second law of thermodynamics,

03:33:41.580 | because it's like you have this simple initial condition,

03:33:45.820 | you can readily see that it's very,

03:33:49.140 | you can describe it very easily,

03:33:50.940 | and yet it makes this thing that seems to be random.

03:33:55.180 | Now, turns out there's some technical detail

03:33:59.060 | about the second law of thermodynamics

03:34:00.460 | and about the idea of reversibility,

03:34:02.160 | when you have kind of a movie

03:34:06.500 | of two billiard balls colliding,

03:34:09.680 | and you see them collide and they bounce off,

03:34:12.380 | and you run that movie in reverse,

03:34:14.300 | you can't tell which way was the forward direction of time

03:34:16.820 | and which way was the backward direction of time,

03:34:18.740 | when you're just looking at individual billiard balls.

03:34:20.860 | By the time you've got a whole collection of them,

03:34:23.940 | a million of them or something,

03:34:25.700 | then it turns out to be the case,

03:34:27.900 | and this is the sort of the mystery of the second law,

03:34:31.260 | that the orderly thing,

03:34:33.080 | you start with the orderly thing and it becomes disordered,

03:34:35.740 | and that's the forward direction in time,

03:34:38.440 | and the other way around of it starts disordered

03:34:40.900 | and becomes ordered, you just don't see that in the world.

03:34:44.500 | Now, in principle, if you sort of traced

03:34:49.060 | the detailed motions of all those molecules backwards,

03:34:52.340 | you would be able to,

03:34:53.580 | the reverse of time makes,

03:34:58.100 | as you go forwards in time, order goes to disorder,

03:35:01.020 | as you go backwards in time, order goes to disorder.

03:35:03.460 | - Perfectly so, yes.

03:35:04.540 | - Right, so the mystery is,

03:35:08.100 | why is it the case that,

03:35:10.060 | or one version of the mystery is,

03:35:11.340 | why is it the case that you never see something

03:35:14.540 | which happens to be just the kind of disorder

03:35:17.420 | that you would need to somehow evolve to order?

03:35:20.380 | Why does that not happen?

03:35:21.980 | Why do you always just see order goes to disorder,

03:35:24.620 | not the other way around?

03:35:26.360 | So the thing that I kind of realized,

03:35:28.820 | I started realizing in the 1980s,

03:35:30.920 | is kind of like, it's a bit like cryptography.

03:35:33.780 | It's kind of like, you start off from this key

03:35:36.580 | that's pretty simple, and then you kind of run it,

03:35:39.420 | and you can get this complicated random mess.

03:35:43.640 | And the thing that,

03:35:46.060 | well, I sort of started realizing back then

03:35:50.740 | was that the second law is kind of a story

03:35:55.300 | of computational irreducibility.

03:35:57.020 | It's a story of what seems,

03:36:00.500 | what we can describe easily at the beginning,

03:36:03.440 | we can only describe

03:36:06.160 | with a lot of computational effort at the end.

03:36:08.900 | Okay, so now we come many, many years later,

03:36:12.100 | and I was trying to sort of,

03:36:16.280 | well, having done this big project

03:36:19.380 | to understand fundamental physics,

03:36:21.700 | I realized that sort of a key aspect of that

03:36:25.500 | is understanding what observers are like.

03:36:28.100 | And then I realized that the second law of thermodynamics

03:36:31.900 | is the same story as a bunch of these other cases.

03:36:36.000 | It is a story of a computationally bounded observer

03:36:40.900 | trying to observe a computationally irreducible system.

03:36:44.520 | So it's a story of,

03:36:46.760 | underneath the molecules are bouncing around,

03:36:49.300 | they're bouncing around in this completely determined way,

03:36:53.080 | determined by rules.

03:36:55.000 | But the point is that we,

03:36:58.820 | as computationally bounded observers,

03:37:01.520 | can't tell that there were these sort of

03:37:04.080 | simple underlying rules.

03:37:05.720 | To us, it just looks random.

03:37:07.080 | And when it comes to this question about,

03:37:08.880 | can you prepare the initial state

03:37:11.160 | so that the disordered thing is,

03:37:15.760 | you have exactly the right disorder

03:37:17.640 | to make something orderly,

03:37:19.200 | a computationally bounded observer cannot do that.

03:37:22.120 | We'd have to have done

03:37:23.800 | all of this sort of irreducible computation

03:37:25.840 | to work out very precisely what this disordered state,

03:37:29.320 | what the exact right disordered state is

03:37:32.140 | so that we would get this ordered thing produced from it.

03:37:35.320 | - What does it mean to be computationally bounded observer?

03:37:39.240 | - So-- - Observing a computationally

03:37:41.000 | irreducible system.

03:37:41.840 | So the computationally bounded,

03:37:43.600 | is there something formal you can say there?

03:37:45.680 | - Right, so it means,

03:37:47.960 | okay, you can talk about Turing machines,

03:37:50.280 | you can talk about computational complexity theory

03:37:53.520 | and polynomial time computation and things like this.

03:37:58.160 | There are a variety of ways to make something more precise,

03:38:01.160 | but I think it's more useful,

03:38:02.200 | the intuitive version of it is more useful.

03:38:04.480 | Which is basically just to say that,

03:38:07.840 | how much computation are you going to do

03:38:10.480 | to try and work out what's going on?

03:38:12.400 | And the answer is, you're not allowed to do a lot of,

03:38:15.600 | we're not able to do a lot of computation.

03:38:17.720 | When we, we've got, in this room,

03:38:21.360 | there will be a trillion, trillion, trillion molecules,

03:38:24.880 | a little bit less.

03:38:25.960 | - It's a big room.

03:38:27.360 | - Right, and at every moment,

03:38:32.160 | every microsecond or something,

03:38:33.680 | these molecules are colliding,

03:38:36.160 | and that's a lot of computation that's getting done.

03:38:40.080 | And the question is, in our brains,

03:38:42.800 | we do a lot less computation every second

03:38:45.880 | than the computation done by all those molecules.

03:38:48.520 | If there is computational irreducibility,

03:38:51.640 | we can't work out in detail

03:38:53.960 | what all those molecules are going to do.

03:38:55.920 | What we can do is only a much smaller amount of computation.

03:38:59.640 | And so the second law of thermodynamics

03:39:02.400 | is this kind of interplay

03:39:03.920 | between the underlying computational irreducibility

03:39:07.080 | and the fact that we, as preparers of initial states

03:39:10.600 | or as measures of what happens,

03:39:12.720 | are not capable of doing that much computation.

03:39:16.240 | So to us, another big formulation

03:39:19.600 | of the second law of thermodynamics

03:39:21.120 | is this idea of the law of entropy increase.

03:39:23.680 | - The characteristic that this universe,

03:39:26.200 | the entropy seems to be always increasing,

03:39:28.400 | what does that show to you about the evolution of--

03:39:31.280 | - Well, okay, so first of all,

03:39:32.120 | we have to say what entropy is.

03:39:34.320 | - Yes. - Okay?

03:39:35.320 | And that's very confused in the history of thermodynamics

03:39:39.160 | because entropy was first introduced

03:39:41.240 | by a guy called Rudolf Clausius,

03:39:43.400 | and he did it in terms of heat and temperature, okay?

03:39:47.480 | Subsequently, it was reformulated

03:39:49.560 | by a guy called Ludwig Boltzmann,

03:39:51.320 | and he formulated it

03:39:55.000 | in a much more kind of combinatorial type way.

03:39:58.280 | But he always claimed

03:39:59.360 | that it was equivalent to Clausius' thing.

03:40:02.360 | And in one particular simple example, it is.

03:40:05.560 | But that connection

03:40:06.520 | between these two formulations of entropy,

03:40:08.800 | they've never been connected.

03:40:10.600 | I mean, there's really, so, okay.

03:40:12.680 | So the more general definition of entropy

03:40:14.800 | due to Boltzmann is the following thing.

03:40:17.640 | So you say, I have a system

03:40:19.280 | and it has many possible configurations.

03:40:21.200 | Molecules can be in many different arrangements,

03:40:23.240 | et cetera, et cetera, et cetera.

03:40:24.960 | If we know something about the system,

03:40:27.400 | for example, we know it's in a box,

03:40:29.960 | it has a certain pressure, it has a certain temperature,

03:40:31.960 | we know these overall facts about it.

03:40:34.360 | Then we say, how many microscopic configurations

03:40:37.560 | of the system are possible given those overall constraints?

03:40:42.480 | And the entropy is the logarithm of that number.

03:40:45.960 | That's the definition.

03:40:47.440 | And that's the kind of the general definition of entropy

03:40:50.600 | that turns out to be useful.

03:40:51.920 | Now in Boltzmann's time,

03:40:53.560 | he thought these molecules could be placed

03:40:55.560 | anywhere you want.

03:40:56.880 | He didn't think, but he said,

03:40:59.120 | oh, actually we can make it a lot simpler

03:41:01.320 | by having the molecules be discrete.

03:41:03.800 | Well, actually he didn't know molecules existed, right?

03:41:06.320 | In those, in his time, 1860s and so on,

03:41:10.400 | the idea that matter might be made of discrete stuff

03:41:14.280 | had been floated ever since ancient Greek times,

03:41:16.880 | but it had been a long time debate about,

03:41:19.640 | is matter discrete, is it continuous?

03:41:22.040 | At the moment, at that time,

03:41:24.760 | people mostly thought that matter was continuous.

03:41:28.440 | And it was all confused with this question

03:41:30.720 | about what heat is, and people thought heat was this fluid.

03:41:34.080 | And it was a big muddle.

03:41:38.960 | And this, but Boltzmann said,

03:41:40.600 | let's assume there are discrete molecules.

03:41:42.640 | Let's even assume they have discrete energy levels.

03:41:45.320 | Let's say everything is discrete.

03:41:47.240 | Then we can do sort of combinatorial mathematics

03:41:50.200 | and work out how many configurations of these things

03:41:52.440 | that would be in the box.

03:41:53.920 | And we can say, we can compute this entropy quantity.

03:41:56.800 | But he said, but of course, it's just a fiction

03:41:59.560 | that these things are discrete.

03:42:00.960 | So he said, this is an interesting piece of history,

03:42:03.360 | by the way, that, you know, that was at that time,

03:42:06.960 | people didn't know molecules existed.

03:42:08.360 | There were other hints from looking at kind of chemistry

03:42:12.480 | that there might be discrete atoms and so on,

03:42:14.680 | just from the combinatorics of, you know,

03:42:17.360 | two hydrogens and one oxygen make water, you know,

03:42:20.480 | two amounts of hydrogen plus one amount of oxygen

03:42:23.400 | together make water, things like this.

03:42:25.800 | But it wasn't known that discrete molecules existed.

03:42:28.760 | And in fact, the people, you know,

03:42:33.000 | it wasn't until the beginning of the 20th century

03:42:37.560 | that Brownian motion was the final giveaway.

03:42:40.160 | Brownian motion is, you know, you look under a microscope

03:42:42.400 | at these little pieces from pollen grains,

03:42:44.600 | you see they're being discreetly kicked,

03:42:46.760 | and those kicks are water molecules hitting them,

03:42:49.360 | and they're discrete.

03:42:51.080 | And in fact, it was really quite interesting history.

03:42:54.840 | I mean, Boltzmann had worked out how things

03:42:57.840 | could be discrete and had basically invented

03:43:00.400 | something like quantum theory in the 1860s.

03:43:04.640 | But he just thought it wasn't really the way it worked.

03:43:07.360 | And then just a piece of physics history,

03:43:10.040 | 'cause I think it's kind of interesting,

03:43:11.720 | in 1900, this guy called Max Planck,

03:43:14.440 | who'd been a long time thermodynamics person,

03:43:17.040 | who was trying to, everybody was trying to prove

03:43:18.840 | the second law of thermodynamics, including Max Planck.

03:43:21.200 | And Max Planck believed that radiation,

03:43:23.540 | like electromagnetic radiation,

03:43:25.440 | somehow the interaction of that with matter

03:43:27.920 | was going to prove the second law of thermodynamics.

03:43:30.500 | But he had these experiments that people had done

03:43:33.060 | on blackbody radiation, and there were these curves,

03:43:36.600 | and you couldn't fit the curve based on his idea

03:43:39.640 | for how radiation interacted with matter,

03:43:42.600 | those curves, you couldn't figure out

03:43:44.120 | how to fit those curves.

03:43:45.660 | Except he noticed that if he just did

03:43:48.540 | what Boltzmann had done and assumed

03:43:50.800 | that electromagnetic radiation was discrete,

03:43:53.940 | he could fit the curves.

03:43:55.440 | He said, but this is just a,

03:43:57.400 | it just happens to work this way.

03:43:59.280 | Then Einstein came along and said, well, by the way,

03:44:01.920 | the electromagnetic field might actually be discrete.

03:44:05.640 | It might be made of photons.

03:44:07.480 | And then that explains how this all works.

03:44:10.200 | And that was, in 1905, that was how,

03:44:13.320 | kind of, that was how that piece

03:44:17.060 | of quantum mechanics got started.

03:44:18.740 | Kind of interesting, interesting piece of history.

03:44:20.320 | I didn't know until I was researching this recently.

03:44:23.280 | In 1904 and 1903, Einstein wrote three different papers.

03:44:28.200 | And so, just sort of well-known physics history.

03:44:32.440 | In 1905, Einstein wrote these three papers.

03:44:35.080 | One introduced relativity theory,

03:44:37.200 | one explained Brownian motion,

03:44:39.140 | and one introduced basically photons.

03:44:41.520 | So, kind of, you know, kind of a big deal year

03:44:46.520 | for physics and for Einstein.

03:44:48.600 | But in the years before that,

03:44:50.320 | he'd written several papers, and what were they about?

03:44:52.720 | They were about the second law of thermodynamics.

03:44:54.920 | And they were an attempt to prove

03:44:56.120 | the second law of thermodynamics, and they're nonsense.

03:44:59.040 | And so, I had no idea that he'd done this.

03:45:02.560 | - Interesting, me neither.

03:45:04.040 | - And in fact, what he did, those three papers in 1905,

03:45:08.040 | well, not so much the relativity paper,

03:45:09.800 | the one on Brownian motion, the one on photons,

03:45:12.720 | both of these were about the story

03:45:15.280 | of sort of making the world discreet.

03:45:18.480 | And he got that idea from Boltzmann.

03:45:21.720 | But Boltzmann didn't think, you know,

03:45:23.280 | Boltzmann kind of died believing, you know,

03:45:25.820 | he said, as a quote, actually, you know,

03:45:28.860 | "In the end, things are gonna turn out to be discreet,

03:45:30.880 | "and I'm gonna write down what I have to say about this

03:45:32.920 | "because, you know, eventually this stuff

03:45:36.000 | "will be rediscovered and I want to leave, you know,

03:45:38.160 | "what I can about how things are gonna be discreet."

03:45:40.440 | But, you know, I think he has some quote about how,

03:45:44.240 | you know, one person can't stand against the tide of history

03:45:48.280 | in saying that, you know, matter is discreet.

03:45:52.680 | - Oh, so he stuck by his guns

03:45:54.880 | in terms of matter is discreet.

03:45:56.320 | - Yes, he did.

03:45:57.600 | And the, you know, what's interesting about this is,

03:46:01.960 | at the time, everybody, including Einstein,

03:46:04.220 | kind of assumed that space was probably

03:46:05.680 | gonna end up being discreet too.

03:46:07.640 | But that didn't work out technically

03:46:09.240 | because it wasn't consistent with relativity theory,

03:46:11.160 | or it didn't seem to be.

03:46:12.520 | And so then in the history of physics,

03:46:15.000 | even though people had determined that matter was discreet,

03:46:18.280 | the electromagnetic field was discreet,

03:46:20.920 | space was a holdout of not being discreet.

03:46:25.080 | And in fact, Einstein, 1916, has this nice letter he wrote,

03:46:28.600 | where he says, "In the end, it will turn out

03:46:29.920 | "space is discreet, but we don't have

03:46:31.840 | "the mathematical tools necessary

03:46:33.780 | "to figure out how that works yet."

03:46:36.400 | And so, you know, I think it's kind of cool

03:46:38.800 | that 100 years later we do.

03:46:40.360 | - Yes, for you, you're pretty sure

03:46:42.520 | that at every layer of reality, it's discreet.

03:46:45.920 | - Right, and that space is discreet, and that the,

03:46:49.920 | I mean, and in fact, one of the things

03:46:51.240 | I realized recently is this kind of theory of heat,

03:46:54.280 | that the, you know, that heat is really

03:46:58.720 | this continuous fluid, it's kind of like,

03:47:02.560 | the caloric theory of heat, which turns out

03:47:05.920 | to be completely wrong, because actually,

03:47:07.500 | heat is the motion of discrete molecules.

03:47:10.040 | Unless you know there are discrete molecules,

03:47:11.680 | it's hard to understand what heat could possibly be.

03:47:14.460 | Well, you know, I think space is discreet,

03:47:18.600 | and the question is kind of what's the analog

03:47:20.660 | of the mistake that was made with caloric

03:47:24.180 | in the case of space?

03:47:25.980 | And so my current guess is that dark matter is,

03:47:30.980 | as I've, my little sort of aphorism

03:47:33.540 | of the last few months has been, you know,

03:47:36.420 | dark matter is the caloric of our time.

03:47:39.260 | That is, it will turn out that dark matter

03:47:41.940 | is a feature of space, and it is not a bunch of particles.

03:47:46.340 | You know, at the time when people were talking about heat,

03:47:48.740 | they knew about fluids, and they said,

03:47:50.580 | "Well, heat must just be another kind of fluid,"

03:47:52.440 | 'cause that's what they knew about.

03:47:54.180 | But now people know about particles,

03:47:56.060 | and so they say, "Well, what's dark matter?"

03:47:58.160 | It's not, it just must be particles.

03:48:00.700 | - So what could dark matter be as a feature of space?

03:48:03.620 | - Oh, I don't know yet.

03:48:05.380 | I mean, I think the thing I'm really,

03:48:07.340 | one of the things I'm hoping to be able to do

03:48:09.900 | is to find the analog of Brownian motion in space.

03:48:13.260 | So in other words, Brownian motion was seeing down

03:48:16.500 | to the level of an effect from individual molecules.

03:48:19.740 | And so in the case of space, you know,

03:48:21.820 | most of the things, the things we see about space

03:48:23.940 | so far, just everything seems continuous.

03:48:26.700 | Brownian motion had been discovered in the 1830s,

03:48:29.460 | and it was only identified what it was the result of

03:48:33.900 | by Smoluchowski and Einstein

03:48:36.940 | at the beginning of the 20th century.

03:48:38.980 | And, you know, dark matter was discovered,

03:48:41.420 | that phenomenon was discovered 100 years ago.

03:48:44.220 | You know, the rotation curves of galaxies

03:48:46.060 | don't follow the luminous matter.

03:48:48.180 | That was discovered 100 years ago.

03:48:49.980 | And I think, you know, I wouldn't be surprised

03:48:53.140 | if there isn't an effect that we already know about

03:48:56.460 | that is kind of the analog of Brownian motion

03:48:59.180 | that reveals the discreteness of space.

03:49:01.540 | And in fact, we're beginning to have some guesses.

03:49:03.820 | We have some evidence that black hole mergers

03:49:07.020 | work differently when there's discrete space.

03:49:09.540 | And there may be things that you can see

03:49:11.180 | in gravitational wave signatures and things

03:49:13.780 | associated with the discreteness of space.

03:49:16.380 | But this is kind of, for me, it's kind of interesting

03:49:19.500 | to see this sort of recapitulation

03:49:21.140 | of the history of physics, where people, you know,

03:49:23.900 | vehemently say, you know, matter is continuous.

03:49:27.540 | Electromagnetic field is continuous.

03:49:29.700 | And it turns out it isn't true.

03:49:30.740 | And then they say space is continuous.

03:49:32.500 | But so, you know, entropy is the number of states

03:49:35.660 | of the system consistent with some constraint.

03:49:37.660 | - Yes.

03:49:38.540 | - And the thing is that if you have,

03:49:41.300 | if you know in great detail the position

03:49:43.340 | of every molecule in the gas,

03:49:46.340 | the entropy is always zero,

03:49:49.740 | because there's only one possible state.

03:49:51.940 | The configuration of molecules in the gas,

03:49:54.620 | the molecules bounce around,

03:49:55.900 | they have a certain rule for bouncing around.

03:49:58.180 | There's just one state of the gas,

03:50:00.060 | evolves to one state of the gas and so on.

03:50:02.580 | But it's only if you don't know in detail

03:50:04.860 | where all the molecules are, that you can say,

03:50:07.700 | well, the entropy increases because the things

03:50:10.620 | we do know about the molecules,

03:50:12.060 | there are more possible microscopic states of the system

03:50:14.840 | consistent with what we do know

03:50:16.220 | about where the molecules are.

03:50:18.260 | And so the question of whether,

03:50:20.660 | so people, this sort of paradox in a sense of,

03:50:24.300 | oh, if we knew where all the molecules were,

03:50:25.980 | the entropy wouldn't increase.

03:50:27.580 | There was this idea introduced by Gibbs

03:50:30.580 | in the early 20th century,

03:50:33.060 | well, actually the very beginning of the 20th century,

03:50:36.260 | as a physics professor, an American physics professor,

03:50:39.060 | was sort of the first distinguished

03:50:40.860 | American physics professor at Yale.

03:50:44.140 | And he introduced this idea of coarse graining.

03:50:48.060 | This idea that, well, you know,

03:50:49.980 | these molecules have a detailed way

03:50:51.380 | they're bouncing around,

03:50:52.620 | but we can only observe a coarse grained version of that.

03:50:56.180 | But the confusion has been,

03:50:57.580 | nobody knew what a valid coarse graining would be.

03:51:00.780 | So nobody knew that whether you could have

03:51:03.080 | this coarse graining that very carefully was sculpted

03:51:06.420 | in just such a way that it would notice

03:51:10.300 | that the particular configurations

03:51:12.260 | that you could get from the simple initial condition,

03:51:14.620 | you know, they fit into this coarse graining

03:51:16.260 | and the coarse graining very carefully

03:51:17.980 | observes that.

03:51:19.100 | Why can't you do that kind of very detailed,

03:51:22.580 | precise coarse graining?

03:51:24.100 | The answer is because if you are

03:51:26.000 | a computationally bounded observer

03:51:28.140 | and the underlying dynamics is computationally irreducible,

03:51:31.580 | that's what defines possible coarse grainings

03:51:34.740 | is what a computationally bounded observer can do.

03:51:38.120 | And it's the fact that a computationally bounded observer

03:51:42.020 | is forced to look only at this kind of

03:51:46.820 | coarse grained version of what the system is doing.

03:51:49.820 | That's why, and because what's going on underneath

03:51:54.500 | is it's kind of filling out this,

03:51:57.540 | the different possible,

03:51:59.260 | you're ending up with something where

03:52:01.460 | the sort of underlying computational irreducibility is,

03:52:06.300 | if all you can see is what the coarse grained result is

03:52:13.820 | with a sort of computationally bounded observation,

03:52:17.500 | then inevitably there are many possible

03:52:20.620 | underlying configurations that are consistent with that.

03:52:23.560 | - Just to clarify, basically any observer

03:52:27.220 | that exists inside the universe

03:52:29.740 | is going to be computationally bounded.

03:52:31.940 | - No, any observer like us.

03:52:33.780 | I don't know, I can't imagine--

03:52:34.620 | - When you say like us, what do you mean like us?

03:52:38.140 | - Well, humans with finite minds.

03:52:41.300 | - You're including the tools of science.

03:52:44.100 | - Yeah, yeah.

03:52:45.620 | I mean, and as we have more precise,

03:52:49.580 | and by the way, there are little sort of

03:52:51.700 | microscopic violations of the second law of thermodynamics

03:52:54.700 | that you can start to have

03:52:56.020 | when you have more precise measurements

03:52:57.580 | of where precisely molecules are.

03:52:59.580 | But for a large scale, when you have enough molecules,

03:53:04.140 | we don't have, we're not tracing all those molecules

03:53:07.760 | and we just don't have the computational resources

03:53:10.060 | to do that, and it wouldn't be,

03:53:12.280 | I think to imagine what an observer

03:53:17.820 | who is not computationally bounded would be like,

03:53:21.100 | it's an interesting thing because, okay,

03:53:23.260 | so what does computational boundedness mean?

03:53:25.140 | Among other things, it means we conclude

03:53:27.780 | that definite things happen.

03:53:29.460 | We go, we take all this complexity of the world

03:53:32.820 | and we make a decision,

03:53:33.740 | we're gonna turn left or turn right.

03:53:36.040 | And that is kind of reducing all this kind of detail

03:53:41.040 | into we're observing it, we're sort of crushing it down

03:53:46.280 | to this one thing.

03:53:48.080 | And that, if we didn't do that,

03:53:50.540 | we wouldn't have all this sort of symbolic structure

03:53:54.300 | that we build up that lets us think things through

03:53:57.720 | with our finite minds.

03:53:59.860 | We'd be instead, we'd be just,

03:54:02.420 | we'd be sort of one with the universe.

03:54:04.480 | - Yeah, so content to not simplify.

03:54:08.240 | - Yes, if we didn't simplify, then we wouldn't be like us.

03:54:12.960 | We would be like the universe, like the intrinsic universe,

03:54:17.320 | but not having experiences like the experiences we have

03:54:21.520 | where we, for example, conclude that definite things happen.

03:54:25.000 | We sort of have this notion of being able

03:54:30.000 | to make sort of narrative statements.

03:54:33.480 | - Yeah, I wonder if it's just like you imagined

03:54:35.880 | as a thought experiment, what it's like to be a computer.

03:54:38.720 | I wonder if it's possible to try to begin to imagine

03:54:41.440 | what it's like to be an unbounded computational observer.

03:54:45.680 | - Well, okay, so here's how that, I think, plays out.

03:54:50.360 | - Vibrations, yeah.

03:54:51.200 | - So, I mean, in this, we talk about this Rouliad,

03:54:55.680 | this space of all possible computations.

03:54:58.520 | And this idea of being at a certain place in the Rouliad,

03:55:02.280 | which corresponds to sort of a certain way of,

03:55:05.680 | a certain set of computations

03:55:08.520 | that you are representing things in terms of.

03:55:11.120 | Okay, so as you expand out in the Rouliad,

03:55:14.440 | as you kind of encompass more possible views of the universe,

03:55:18.600 | as you encompass more possible kinds of computations

03:55:21.680 | that you can do, eventually, you might say,

03:55:23.880 | "That's a real win.

03:55:25.200 | "We're colonizing the Rouliad.

03:55:26.680 | "We're building out more paradigms

03:55:29.320 | "about how to think about things."

03:55:31.240 | And eventually, you might say, "We won all the way.

03:55:34.120 | "We managed to colonize the whole Rouliad."

03:55:36.640 | Okay, here's the problem with that.

03:55:38.120 | The problem is that the notion of existence,

03:55:41.080 | coherent existence, requires some kind of specialization.

03:55:45.240 | By the time you are the whole Rouliad,

03:55:47.360 | by the time you cover the whole Rouliad,

03:55:49.640 | in no useful sense do you coherently exist.

03:55:53.000 | So in other words, in--

03:55:55.160 | - Oh, interesting.

03:55:56.000 | - The notion of existence,

03:55:57.560 | the notion of what we think of as definite existence,

03:56:01.720 | requires this kind of specialization,

03:56:03.840 | requires this kind of idea that we are not

03:56:08.720 | all possible things.

03:56:10.400 | We are a particular set of things.

03:56:13.480 | And that's kind of how we,

03:56:15.320 | that's kind of what makes us have a coherent existence.

03:56:19.760 | If we were spread throughout the Rouliad,

03:56:21.880 | we would not, there would be no coherence

03:56:24.480 | to the way that we work.

03:56:25.840 | We would work in all possible ways.

03:56:27.960 | And that wouldn't be kind of a notion of identity.

03:56:31.360 | We wouldn't have this notion of kind of coherent identity.

03:56:36.360 | - I am geographically located somewhere exactly,

03:56:41.080 | precisely in the Rouliad, therefore I am.

03:56:44.160 | - Yes. - Is the Descartes kind of--

03:56:46.000 | - Yeah, yeah, right.

03:56:46.840 | Well, you're in a certain place in physical space,

03:56:48.440 | you're in a certain place in Roulial space.

03:56:50.600 | And if you are sufficiently spread out,

03:56:55.520 | you are no longer coherent.

03:56:57.880 | And you no longer have, I mean, in our perception

03:57:01.920 | of what it means to exist and to have experience,

03:57:05.040 | it doesn't happen that way.

03:57:05.880 | - So therefore, to exist means to be computationally bounded.

03:57:10.280 | - I think so.

03:57:11.120 | To exist in the way that we think of ourselves as existing.

03:57:14.340 | Yes.

03:57:15.180 | - The very act of existence is like operating

03:57:17.600 | in this place that's computationally irreducible.

03:57:20.160 | So there's this giant mess of things going on

03:57:22.380 | that you can't possibly predict.

03:57:24.460 | But nevertheless, because of your limitations,

03:57:26.880 | you have an imperative of like, what is it?

03:57:30.360 | An imperative or a skill set to simplify?

03:57:33.600 | Or an ignorance, a sufficient--

03:57:35.360 | - Okay, so the thing which is not obvious

03:57:37.240 | is that you are taking a slice of all this complexity.

03:57:40.040 | Just like we have all of these molecules

03:57:42.320 | bouncing around in the room,

03:57:43.720 | but all we notice is the kind of the flow of the air

03:57:47.760 | or the pressure of the air.

03:57:49.160 | We're just noticing these particular things.

03:57:51.800 | And the big interesting thing is that there are rules,

03:57:56.800 | there are laws that govern those big things we observe.

03:58:00.700 | So it's not obvious.

03:58:01.540 | - It's amazing, 'cause it doesn't feel like it's a slice.

03:58:04.300 | - Yeah, well, right.

03:58:05.140 | - It's not a slice.

03:58:05.980 | It's like an abstraction.

03:58:09.580 | - Yes, but I mean, the fact that the gas laws work,

03:58:12.860 | that we can describe pressure, volume,

03:58:14.540 | et cetera, et cetera, et cetera.

03:58:16.100 | We don't have to go down to the level

03:58:18.500 | of talking about individual molecules.

03:58:20.340 | That is a non-trivial fact.

03:58:22.240 | And here's the thing that I sort of exciting thing

03:58:25.180 | as far as I'm concerned.

03:58:26.620 | The fact that there are certain aspects of the universe.

03:58:30.900 | So we think space is made ultimately these atoms of space

03:58:34.620 | and these hypergraphs and so on.

03:58:36.400 | And we think that, but we nevertheless perceive the universe

03:58:41.400 | at a large scale to be like continuous space and so on.

03:58:44.940 | We in quantum mechanics,

03:58:48.620 | we think that there are these many threads of time,

03:58:50.740 | these many threads of history,

03:58:52.420 | yet we kind of span.

03:58:54.900 | So in quantum mechanics and our models of physics,

03:58:58.600 | there are these, time is not a single thread.

03:59:01.780 | Time breaks into many threads.

03:59:03.740 | They branch, they merge.

03:59:05.340 | But we are part of that branching, merging universe.

03:59:10.820 | And so our brains are also branching and merging.

03:59:13.780 | And so when we perceive the universe,

03:59:17.000 | we are branching brains perceiving a branching universe.

03:59:20.820 | And so the fact that the claim that we believe

03:59:25.820 | that we are persistent in time,

03:59:28.140 | we have this single thread of experience.

03:59:30.660 | That's the statement that somehow we managed

03:59:32.700 | to aggregate together those separate threads of time

03:59:36.160 | that are separated in the operation of,

03:59:38.140 | in the fundamental operation of the universe.

03:59:40.220 | So just as in space,

03:59:42.020 | we're averaging over some big region of space

03:59:44.340 | and we're looking at many, many of the aggregate effects

03:59:46.820 | of many atoms of space.

03:59:48.560 | So similarly, in what we call branchial space,

03:59:51.060 | the space of these quantum branches,

03:59:53.600 | we are effectively averaging over many different branches

03:59:57.420 | of possible of histories of the universe.

03:59:59.920 | And so in thermodynamics,

04:00:02.960 | we're averaging over many configurations of,

04:00:05.560 | many possible positions of molecules.

04:00:08.760 | So what we see here is,

04:00:10.520 | so the question is, when you do that averaging for space,

04:00:13.940 | what are the aggregate laws of space?

04:00:16.240 | When you do that averaging of a branchial space,

04:00:18.240 | what are the aggregate laws of branchial space?

04:00:21.360 | When you do that averaging over the molecules and so on,

04:00:24.800 | what are the aggregate laws you get?

04:00:27.160 | And this is the thing that I think

04:00:29.760 | is just amazingly, amazingly neat.

04:00:33.720 | - That there are aggregate laws at all.

04:00:35.640 | - Well, yes, but the question is,

04:00:36.840 | what are those aggregate laws?

04:00:38.600 | So the answer is for space,

04:00:40.400 | the aggregate laws are Einstein's equations for gravity,

04:00:42.800 | for the structure of space time.

04:00:44.560 | For branchial space, the aggregate laws

04:00:47.080 | are the laws of quantum mechanics.

04:00:48.880 | And for the case of molecules and things,

04:00:52.960 | the aggregate laws are basically

04:00:55.000 | the second law of thermodynamics.

04:00:57.040 | And so that's the,

04:00:59.400 | and the things that follow

04:01:00.360 | from the second law of thermodynamics.

04:01:02.240 | And so what that means is that

04:01:05.240 | the three great theories of 20th century physics,

04:01:08.380 | which are basically general relativity,

04:01:10.120 | the theory of gravity, quantum mechanics,

04:01:12.720 | and statistical mechanics,

04:01:14.080 | which is what kind of grows out

04:01:15.360 | of the second law of thermodynamics.

04:01:17.200 | All three of the great theories of 20th century physics

04:01:20.720 | are the result of this interplay

04:01:22.760 | between computational irreducibility

04:01:25.680 | and the computational boundedness of observers.

04:01:28.380 | And for me, this is really neat

04:01:32.280 | because it means that all three of these laws are derivable.

04:01:36.560 | So we used to think that, for example,

04:01:38.960 | Einstein's equations were just sort of

04:01:40.480 | a wheel in feature of our universe,

04:01:42.780 | that they could be, the universe might be that way,

04:01:44.920 | it might not be that way.

04:01:46.520 | Quantum mechanics is just like,

04:01:47.840 | well, it just happens to be that way.

04:01:50.040 | And the second law, people kind of thought,

04:01:52.120 | well, maybe it is derivable, okay?

04:01:54.440 | What turns out to be the case is that

04:01:56.040 | all three of the fundamental principles of physics

04:01:58.320 | are derivable, but they're not derivable

04:02:00.740 | just from mathematics.

04:02:02.560 | They require, or just from some kind of logical computation,

04:02:06.320 | they require one more thing.

04:02:08.000 | They require that the observer,

04:02:10.440 | that the thing that is sampling the way the universe works

04:02:13.960 | is an observer who has these characteristics

04:02:16.900 | of computational boundedness of belief

04:02:18.580 | and persistence in time.

04:02:20.200 | And so that means that it is the nature of the observer,

04:02:24.400 | the rough nature of the observer,

04:02:27.320 | not the details of, oh, we got two eyes

04:02:29.340 | and we observe photons of this frequency and so on,

04:02:32.960 | but the kind of the very coarse features of the observer

04:02:37.960 | then imply these very precise facts about physics.

04:02:42.080 | And I think it's amazing.

04:02:44.200 | - So if we just look at the actual experience

04:02:47.280 | of the observer that we experience this reality,

04:02:50.680 | it seems real to us.

04:02:52.800 | And you're saying because of our bounded nature,

04:02:54.880 | it's actually all an illusion.

04:02:56.680 | It's a simplification.

04:02:58.880 | - Yeah, it's a simplification.

04:03:00.320 | Right, what's--

04:03:01.160 | - So you don't think a simplification is an illusion?

04:03:04.040 | - No, I mean, it's, well, I don't know.

04:03:07.280 | I mean, what's underneath-- - Is it real?

04:03:10.560 | - Okay, that's an interesting question.

04:03:12.520 | What's real?

04:03:14.940 | And that relates to the whole question

04:03:16.640 | of why does the universe exist?

04:03:18.680 | And what is the difference between reality

04:03:22.740 | and a mere representation of what's going on?

04:03:25.360 | - Yes, we experience the representation.

04:03:28.800 | - Yes, but the question of,

04:03:32.040 | so one question is,

04:03:35.300 | why is there a thing which we can experience that way?

04:03:39.660 | And the answer is because this Rouliad object,

04:03:44.460 | which is this entangled limit of all possible computations,

04:03:48.700 | there is no choice about it.

04:03:50.640 | It has to exist.

04:03:52.100 | It has to, there has to be such a thing.

04:03:54.540 | It is in the same sense that, you know, two plus two,

04:03:58.080 | if you define what two is and you plot pluses and so on,

04:04:00.980 | two plus two has to equal four.

04:04:03.260 | Similarly, this Rouliad,

04:04:04.940 | this limit of all possible computations,

04:04:07.120 | just has to be a thing that is,

04:04:09.860 | once you have the idea of computation,

04:04:12.500 | you inevitably have the Rouliad.

04:04:13.980 | - You're gonna have to have a Rouliad, yeah.

04:04:15.400 | - Right, and what's important about it,

04:04:17.100 | there's just one of it.

04:04:18.460 | It's just this unique object.

04:04:21.360 | And that unique object necessarily exists.

04:04:25.420 | And then the question is, what,

04:04:28.220 | and then we are,

04:04:30.940 | once you know that we are sort of embedded in that

04:04:34.620 | and taking samples of it,

04:04:36.380 | that it's sort of inevitable that there is this thing

04:04:39.740 | that we can perceive that is, you know,

04:04:43.340 | our perception of kind of physical reality

04:04:47.860 | necessarily is that way,

04:04:49.780 | given that we are observers

04:04:51.480 | with the characteristics we have.

04:04:53.420 | So in other words, the fact that,

04:04:55.820 | the fact that the universe exists is,

04:04:59.080 | it's actually, it's almost like,

04:05:01.440 | it's, you know, to think about it

04:05:02.940 | almost theologically, so to speak.

04:05:04.900 | And I've really, it's funny because

04:05:08.020 | a lot of the questions about the existence

04:05:09.620 | of the universe and so on,

04:05:11.080 | they transcend what kind of the science

04:05:14.600 | of the last few hundred years has really been concerned with.

04:05:16.780 | The science of the last few hundred years

04:05:18.420 | hasn't thought it could talk about questions like that.

04:05:21.220 | And, but I think it's kind of,

04:05:24.140 | and so a lot of the kind of arguments of,

04:05:25.960 | you know, does God exist?

04:05:27.560 | You know, is it obvious that,

04:05:29.380 | I think it, in some sense, in some representation,

04:05:32.020 | it's sort of more obvious that,

04:05:35.980 | that something sort of bigger than us exists

04:05:38.780 | than that we exist.

04:05:40.320 | And we are, you know, our existence

04:05:43.140 | and as observers the way we are

04:05:44.880 | is sort of a contingent thing about the universe.

04:05:47.980 | And it's more inevitable that the whole universe,

04:05:50.980 | kind of the whole set of all possibilities exists.

04:05:54.740 | But this question about, you know,

04:05:56.620 | is it real or is it an illusion?

04:06:00.760 | You know, all we know is our experience.

04:06:03.820 | And so the fact that, well,

04:06:07.300 | our experience is this absolutely microscopic piece

04:06:11.220 | of sample of the Roulade.

04:06:14.060 | And we're, and you know, there's this point about,

04:06:18.440 | you know, we might sample more and more of the Roulade.

04:06:21.300 | We might learn more and more about,

04:06:23.260 | we might learn, you know, like different areas of physics,

04:06:26.900 | like quantum mechanics, for example.

04:06:28.940 | The fact that it was discovered,

04:06:31.780 | I think is closely related to the fact

04:06:33.780 | that electronic amplifiers were invented

04:06:35.980 | that allowed you to take a small effect and amplify it up,

04:06:39.460 | which hadn't been possible before.

04:06:41.020 | You know, microscopes had been invented

04:06:42.620 | that magnify things and so on.

04:06:44.460 | But the, you know, having a very small effect

04:06:46.900 | and being able to magnify it was sort of a new thing

04:06:49.340 | that allowed one to see a different sort of aspect

04:06:52.260 | of the universe and let one discover this kind of thing.

04:06:55.140 | So, you know, we can expect that in the Roulade,

04:06:58.500 | there are an infinite collection

04:06:59.860 | of new things we can discover.

04:07:01.700 | There's in fact computational irreducibility

04:07:03.860 | kind of guarantees that there will be an infinite collection

04:07:07.140 | of kind of, you know, pockets of reducibility

04:07:09.940 | that can be discovered.

04:07:11.140 | - Boy, would it be fun to take a walk down the Roulade

04:07:16.700 | and see what kind of stuff we find there.

04:07:18.340 | You write about alien intelligences.

04:07:21.100 | - Yes.

04:07:21.920 | - It's these worlds of computation.

04:07:23.340 | - Yes, well, quite, but the problem with these worlds is

04:07:26.700 | that- - We can't talk to them.

04:07:28.160 | - Yes.

04:07:29.000 | And, you know, the thing is,

04:07:31.340 | what I've kind of spent a lot of time doing

04:07:34.060 | is just studying computational systems,

04:07:35.700 | seeing what they do, what I now call Rouleology,

04:07:38.940 | kind of just the study of rules and what they do.

04:07:42.340 | You know, you can kind of easily jump somewhere else

04:07:45.380 | in the Roulade and start seeing what do these rules do.

04:07:48.500 | And what you, as they just, they do what they do,

04:07:51.820 | and there's no human connection, so to speak.

04:07:54.020 | - Do you think, you know, some people are able

04:07:55.860 | to communicate with animals?

04:08:00.380 | Do you think you can become a whisperer of these-

04:08:03.280 | - Oh, I've been trying.

04:08:05.420 | That's what I've spent some part of my life doing.

04:08:07.820 | - Have you heard?

04:08:09.100 | And are you at the risk of losing your mind?

04:08:11.740 | - Sort of my favorite science discovery is this fact

04:08:15.780 | that these very simple programs can produce

04:08:17.640 | very complicated behavior.

04:08:18.820 | - Yeah, it's beautiful.

04:08:20.140 | - And that fact is kind of, in a sense,

04:08:23.980 | a whispering of something out in the computational universe

04:08:27.740 | that we didn't really know was there before.

04:08:29.740 | I mean, it's, you know, it's like, you know,

04:08:33.060 | back in the 1980s, I was doing a bunch of work

04:08:36.420 | with some very, very good mathematicians,

04:08:39.340 | and they were like trying to pick away, you know,

04:08:41.060 | can we figure out what's going on

04:08:42.420 | in these computational systems?

04:08:44.260 | And they basically said, look, the math we have

04:08:47.460 | just doesn't get anywhere with this, we're stuck.

04:08:50.100 | There's nothing to say, we have nothing to say.

04:08:52.540 | And, you know, in a sense, perhaps my main achievement

04:08:55.860 | at that time was to realize that the very fact

04:08:58.900 | that the good mathematicians had nothing to say

04:09:02.780 | was itself a very interesting thing.

04:09:04.980 | That was kind of a sort of, in some sense,

04:09:07.460 | a whispering of a different part of the Roulade

04:09:10.020 | that one hadn't, you know, one wasn't,

04:09:12.540 | was not accessible from what we knew

04:09:14.060 | in mathematics and so on.

04:09:15.320 | - Does it make you sad that you're exploring

04:09:19.600 | some of these gigantic ideas, and it feels like

04:09:22.180 | we're on the verge of breaking through

04:09:25.060 | to some very interesting discoveries,

04:09:27.620 | and yet you're just a finite being

04:09:30.500 | that's going to die way too soon,

04:09:32.820 | and that scan of your brain, your full body,

04:09:35.700 | kind of shows that you're--

04:09:37.500 | - Yeah, it's just a bunch of meat.

04:09:38.980 | - It's just a bunch of meat.

04:09:40.420 | Yeah, does that make you a little sad?

04:09:45.020 | - Kind of a shame.

04:09:45.860 | I mean, I'd kind of like to see how all this stuff works out,

04:09:48.660 | but I think the thing to realize, you know,

04:09:50.560 | it's an interesting sort of thought experiment.

04:09:52.260 | You know, you say, okay, you know,

04:09:55.040 | let's assume we can get cryonics to work,

04:09:57.240 | and one day it will.

04:09:58.440 | That will be one of these things

04:09:59.760 | that's kind of like chat GPT.

04:10:01.560 | One day somebody will figure out, you know,

04:10:03.640 | how to get water from zero degrees centigrade

04:10:07.000 | down to, you know, minus 44 or something

04:10:09.280 | without it expanding, and, you know,

04:10:11.240 | cryonics will be solved, and you'll be able to, like,

04:10:14.020 | just, you know, put a pause in, so to speak,

04:10:17.640 | and, you know, kind of reappear 100 years later

04:10:21.600 | or something, and the thing, though,

04:10:23.520 | that I've kind of increasingly realized

04:10:26.320 | is that, in a sense, this whole question

04:10:29.880 | of kind of the sort of one is embedded

04:10:33.800 | in a certain moment in time,

04:10:36.140 | and, you know, kind of the things we care about now,

04:10:38.980 | the things I care about now, for example,

04:10:41.460 | had I lived, you know, 500 years ago,

04:10:45.120 | many of the things I care about now,

04:10:46.640 | it's like, that's totally bizarre.

04:10:48.320 | I mean, nobody would care about that.

04:10:49.760 | It's not even the thing one thinks about.

04:10:52.120 | In the future, the things that most people

04:10:55.360 | will think about, you know, one will be

04:10:57.240 | a strange relic of thinking about, you know,

04:11:01.240 | the kind of, you know, it might be,

04:11:03.840 | one might have been a theologian thinking about,

04:11:06.080 | you know, how many angels fit on the head

04:11:07.600 | of a pin or something, and that might have been the,

04:11:10.360 | you know, the big intellectual thing.

04:11:12.340 | So I think it's a, but yeah, it's a, you know,

04:11:16.320 | it's one of these things where, particularly,

04:11:19.320 | you know, I've had the, I don't know,

04:11:21.880 | good or bad fortune, I'm not sure,

04:11:23.360 | I think it's a mixed thing, that I've, you know,

04:11:26.400 | I've invented a bunch of things,

04:11:28.300 | which I kind of can, I think, see well enough

04:11:32.560 | what's gonna happen that, you know,

04:11:34.760 | in 50 years, 100 years, whatever,

04:11:37.300 | assuming the world doesn't exterminate itself,

04:11:40.280 | so to speak, you know, these are things

04:11:42.560 | that will be sort of centrally important

04:11:46.160 | to what's going on, and it's kind of both,

04:11:49.360 | it's both a good thing and a bad thing

04:11:50.840 | in terms of the passage of one's life.

04:11:52.700 | I mean, it's kind of like, if everything I'd figured out

04:11:55.600 | was like, okay, I figured it out when I was 25 years old,

04:11:58.560 | and everybody says it's great, and we're done,

04:12:01.520 | and it's like, okay, but I'm gonna live another,

04:12:03.680 | how many years, and that's kind of,

04:12:05.680 | it's all downhill from there.

04:12:07.440 | In a sense, it's better, in some sense,

04:12:10.400 | to be able to, you know, there's,

04:12:13.640 | it sort of keeps things interesting that,

04:12:16.640 | you know, I can see, you know, a lot of these things,

04:12:19.240 | I mean, it's kind of, I didn't expect, you know,

04:12:22.000 | chat GPT, I didn't expect the kind of,

04:12:24.740 | the sort of opening up of this idea

04:12:27.600 | of computation and computational language

04:12:29.880 | that's been made possible by this.

04:12:31.680 | I didn't expect that.

04:12:32.640 | This is ahead of schedule, so to speak.

04:12:34.940 | You know, even though the sort of,

04:12:37.600 | the big kind of flowering of that stuff,

04:12:40.520 | I'd sort of been assuming was another 50 years away.

04:12:43.200 | So if it turns out it's a lot less time,

04:12:45.760 | that's pretty cool, because, you know,

04:12:47.760 | I'll hopefully get to see it, so to speak,

04:12:50.080 | rather than then.

04:12:51.000 | - Well, I think I speak for a very,

04:12:55.200 | very large number of people in saying that

04:12:58.300 | I hope you stick around for a long time to come.

04:13:01.780 | You've had so many interesting ideas.

04:13:04.160 | You've created so many interesting systems over the years,

04:13:07.520 | and I can see, now that GPT and language models

04:13:10.760 | broke open the world even more,

04:13:12.600 | I can't wait to see you at the forefront

04:13:15.720 | of this development, what you do.

04:13:18.680 | And yeah, I've been a fan of yours,

04:13:20.920 | like I've told you many, many times

04:13:22.320 | since the very beginning.

04:13:23.720 | I'm deeply grateful that you wrote a new kind of science,

04:13:26.120 | that you explored this mystery of cellular automata,

04:13:29.880 | and inspired this one little kid in me

04:13:33.520 | to pursue artificial intelligence

04:13:35.680 | in all this beautiful world, so Steven,

04:13:38.280 | thank you so much, it's a huge honor to talk to you,

04:13:40.520 | to just be able to pick your mind

04:13:43.400 | and to explore all these ideas with you,

04:13:45.080 | and please keep going, and I can't wait

04:13:47.600 | to see what you come up with next.

04:13:48.840 | And thank you for talking today.

04:13:50.400 | - Thanks.

04:13:51.240 | - We went past midnight, we only did four and a half hours.

04:13:54.840 | I mean, we could probably go for four more,

04:13:57.080 | but we'll save that 'til next time.

04:13:59.400 | This is round number four,

04:14:00.720 | we'll, I'm sure, talk many more times.

04:14:02.800 | Thank you so much.

04:14:03.920 | - My pleasure.

04:14:05.600 | - Thanks for listening to this conversation

04:14:07.000 | with Steven Wolfram.

04:14:08.240 | To support this podcast,

04:14:09.400 | please check out our sponsors in the description.

04:14:12.160 | And now, let me leave you with some words

04:14:14.240 | from George Cantor.

04:14:16.240 | "The essence of mathematics lies in its freedom."

04:14:20.320 | Thank you for listening, and hope to see you next time.

04:14:24.600 | (upbeat music)

04:14:27.200 | (upbeat music)

04:14:29.800 | [BLANK_AUDIO]

Stephen Wolfram: ChatGPT and the Nature of Truth, Reality & Computation | Lex Fridman Podcast #376

Chapters