Douglas Lenat: Cyc and the Quest to Solve Common Sense Reasoning in AI

00:00:00.000 | The following is a conversation with Doug Lennett,

00:00:03.200 | creator of Psyche, a system that for close to 40 years

00:00:07.560 | and still today has sought to solve the core problem

00:00:11.020 | of artificial intelligence,

00:00:12.940 | the acquisition of common sense knowledge

00:00:15.520 | and the use of that knowledge to think,

00:00:18.080 | to reason and to understand the world.

00:00:21.200 | To support this podcast,

00:00:22.400 | please check out our sponsors in the description.

00:00:25.280 | As a side note, let me say that in the excitement

00:00:28.320 | of the modern era of machine learning,

00:00:30.160 | it is easy to forget just how little we understand

00:00:33.480 | exactly how to build the kind of intelligence

00:00:37.020 | that matches the power of the human mind.

00:00:39.640 | To me, many of the core ideas behind Psyche

00:00:42.580 | in some form, in actuality or in spirit,

00:00:45.780 | will likely be part of the AI system

00:00:47.860 | that achieves general super intelligence.

00:00:51.140 | But perhaps more importantly,

00:00:52.960 | solving this problem of common sense knowledge

00:00:55.620 | will help us humans understand our own minds,

00:00:58.740 | the nature of truth, and finally,

00:01:01.180 | how to be more rational and more kind to each other.

00:01:05.440 | This is the Lex Friedman Podcast,

00:01:07.400 | and here is my conversation with Doug Lennett.

00:01:10.640 | Psyche is a project launched by you in 1984

00:01:16.180 | and still is active today,

00:01:18.060 | whose goal is to assemble a knowledge base

00:01:20.060 | that spans the basic concepts and rules

00:01:22.260 | about how the world works.

00:01:24.060 | In other words, it hopes to capture common sense knowledge,

00:01:27.900 | which is a lot harder than it sounds.

00:01:30.180 | Can you elaborate on this mission

00:01:31.780 | and maybe perhaps speak to the various sub-goals

00:01:34.420 | within this mission?

00:01:36.180 | - When I was a faculty member

00:01:39.460 | in the computer science department at Stanford,

00:01:41.920 | my colleagues and I did research

00:01:44.860 | in all sorts of artificial intelligence programs.

00:01:49.860 | So natural language understanding programs,

00:01:52.460 | robots, expert systems, and so on.

00:01:56.920 | And we kept hitting the very same brick wall.

00:02:00.820 | Our systems would have impressive early successes.

00:02:05.780 | And so if your only goal was academic,

00:02:09.720 | namely to get enough material to write a journal article,

00:02:14.720 | that might actually suffice.

00:02:16.380 | But if you're really trying to get AI,

00:02:19.340 | then you have to somehow get past the brick wall.

00:02:21.860 | And the brick wall was,

00:02:23.700 | the programs didn't have what we would call common sense.

00:02:27.160 | They didn't have general world knowledge.

00:02:29.560 | They didn't really understand what they were doing,

00:02:32.500 | what they were saying, what they were being asked.

00:02:35.100 | And so very much like a clever dog performing tricks,

00:02:40.100 | we could get them to do tricks,

00:02:42.420 | but they never really understood what they were doing.

00:02:44.620 | Sort of like when you get a dog

00:02:46.820 | to fetch your morning newspaper.

00:02:49.180 | The dog might do that successfully,

00:02:50.980 | but the dog has no idea what a newspaper is

00:02:53.380 | or what it says or anything like that.

00:02:55.620 | - What does it mean to understand something?

00:02:57.740 | Can you maybe elaborate on that a little bit?

00:02:59.780 | Is it, is understanding action

00:03:02.660 | of like combining little things together,

00:03:04.740 | like through inference,

00:03:05.940 | or is understanding the wisdom you gain over time

00:03:08.740 | that forms a knowledge?

00:03:10.060 | - I think of understanding more like a,

00:03:12.660 | think of it more like the ground you stand on,

00:03:17.140 | which could be very shaky,

00:03:20.820 | could be very unsafe,

00:03:23.100 | but most of the time is not

00:03:25.020 | because underneath it is more ground

00:03:28.580 | and eventually, rock and other things,

00:03:31.740 | but layer after layer after layer,

00:03:34.540 | that solid foundation is there.

00:03:37.300 | And you rarely need to think about it.

00:03:39.540 | You rarely need to count on it,

00:03:41.020 | but occasionally you do.

00:03:42.860 | And I've never used this analogy before, so bear with me,

00:03:47.180 | but I think the same thing is true

00:03:49.420 | in terms of getting computers to understand things,

00:03:52.620 | which is you ask a computer a question,

00:03:56.100 | for instance, Alexa or some robot or something,

00:03:59.660 | and maybe it gets the right answer.

00:04:03.340 | But if you were asking that of a human,

00:04:06.140 | you could also say things like, why?

00:04:09.500 | Or how might you be wrong about this?

00:04:11.940 | Or something like that.

00:04:13.380 | And the person would answer you.

00:04:16.260 | And it might be a little annoying

00:04:18.380 | if you have a small child

00:04:19.540 | and they keep asking why questions in series.

00:04:22.700 | Eventually you get to the point

00:04:24.180 | where you throw up your hands and say, I don't know,

00:04:26.140 | it's just the way the world is.

00:04:28.300 | But for many layers,

00:04:30.300 | you actually have that layered, solid foundation of support

00:04:35.300 | so that when you need it, you can count on it.

00:04:38.740 | And when do you need it?

00:04:39.700 | Well, when things are unexpected,

00:04:42.100 | when you come up against a situation which is novel,

00:04:45.620 | for instance, when you're driving,

00:04:47.740 | it may be fine to have a small program,

00:04:51.860 | a small set of rules that cover 99% of the cases,

00:04:56.380 | but that 1% of the time when something strange happens,

00:05:00.060 | you really need to draw on common sense.

00:05:02.060 | For instance, my wife and I were driving recently

00:05:05.940 | and there was a trash truck in front of us.

00:05:09.140 | And I guess they had packed it too full

00:05:11.820 | and the back exploded and trash bags went everywhere.

00:05:17.060 | And we had to make a split second decision.

00:05:20.180 | Are we going to slam on our brakes?

00:05:21.900 | Are we going to swerve into another lane?

00:05:24.260 | Are we going to just run it over?

00:05:26.460 | Because there were cars all around us.

00:05:28.740 | And in front of us was a large trash bag

00:05:32.860 | and we know what we throw away in trash bags,

00:05:35.060 | probably not a safe thing to run over.

00:05:37.380 | Over on the left was a bunch of fast food restaurant

00:05:42.380 | trash bags and it's like, oh, well,

00:05:44.300 | those things are just like styrofoam and leftover food.

00:05:47.260 | We'll run over that.

00:05:48.380 | And so that was a safe thing for us to do.

00:05:51.100 | Now that's the kind of thing that's going to happen

00:05:52.820 | maybe once in your life.

00:05:54.860 | But the point is that there's almost no telling

00:06:00.500 | what little bits of knowledge about the world

00:06:03.140 | you might actually need in some situations

00:06:06.340 | which were unforeseen.

00:06:08.140 | - But see, when you sit on that mountain

00:06:11.100 | or that ground that goes deep of knowledge

00:06:15.620 | in order to make a split second decision

00:06:17.540 | about fast food trash or random trash

00:06:20.980 | from the back of a trash truck,

00:06:23.780 | you need to be able to leverage that ground

00:06:28.420 | you stand on in some way.

00:06:29.380 | It's not merely, it's not enough to just have a lot

00:06:33.900 | of ground to stand on.

00:06:35.420 | It's your ability to leverage it,

00:06:37.980 | to utilize it in a split, like integrate it all together

00:06:40.940 | to make that split second decision.

00:06:43.020 | And I suppose understanding isn't just having

00:06:48.020 | common sense knowledge to access.

00:06:52.540 | It's the act of accessing it somehow,

00:06:55.860 | like correctly filtering out the parts

00:07:00.860 | of the knowledge that are not useful,

00:07:02.380 | selecting only the useful parts

00:07:04.300 | and effectively making conclusive decisions.

00:07:08.340 | - So let's tease apart two different tasks really,

00:07:11.660 | both of which are incredibly important and even necessary

00:07:15.580 | if you're going to have this in a useful, usable fashion

00:07:20.260 | as opposed to say like library books sitting on a shelf

00:07:23.540 | and so on where the knowledge might be there,

00:07:26.380 | but if a fire comes, the books are gonna burn

00:07:30.060 | because they don't know what's in them

00:07:32.060 | and they're just gonna sit there while they burn.

00:07:34.940 | So there are two aspects of using the knowledge.

00:07:39.220 | One is a kind of a theoretical, how is it possible at all?

00:07:44.220 | And then the second aspect of what you said is,

00:07:47.180 | how can you do it quickly enough?

00:07:49.580 | So how can you do it at all is something

00:07:52.460 | that philosophers have grappled with.

00:07:55.180 | And fortunately, philosophers 100 years ago

00:07:58.460 | and even earlier developed a kind of formal language,

00:08:04.860 | like English, it's called predicate logic

00:08:09.780 | or first order logic or something like predicate calculus

00:08:13.580 | and so on.

00:08:14.400 | So there's a way of representing things

00:08:17.340 | in this formal language,

00:08:19.620 | which enables a mechanical procedure

00:08:24.340 | to sort of grind through and algorithmically produce

00:08:28.500 | all of the same logical entailments,

00:08:31.900 | all the same logical conclusions that you or I would

00:08:35.260 | from that same set of pieces of information

00:08:38.740 | that are represented that way.

00:08:40.380 | So that sort of raises a couple questions.

00:08:45.600 | One is, how do you get all this information

00:08:48.600 | from say observations and English and so on

00:08:52.100 | into this logical form?

00:08:54.100 | And secondly, how can you then efficiently

00:08:57.380 | run these algorithms to actually get the information

00:09:00.400 | you need in the case I mentioned in a 10th of a second,

00:09:04.260 | rather than say in 10 hours or 10,000 years of computation?

00:09:09.260 | And those are both really important questions.

00:09:13.660 | - And like a corollary addition to the first one is,

00:09:17.460 | how many such things do you need to gather

00:09:21.340 | for it to be useful in certain contexts?

00:09:24.620 | So like what, in order, you mentioned philosophers,

00:09:27.540 | in order to capture this world and represent it

00:09:30.520 | in a logical way and with a formal logic,

00:09:34.000 | like how many statements are required?

00:09:37.280 | Is it five?

00:09:38.120 | Is it 10?

00:09:38.940 | Is it 10 trillion?

00:09:40.480 | Is it like that?

00:09:41.320 | That's, as far as I understand,

00:09:43.160 | is probably still an open question.

00:09:45.760 | It may forever be an open question

00:09:47.480 | to say like definitively about.

00:09:51.360 | To describe the universe perfectly,

00:09:53.520 | how many facts do you need?

00:09:55.080 | - I'm gonna, I guess I'm gonna disappoint you

00:09:58.140 | by giving you an actual answer to your question.

00:10:00.300 | - Okay.

00:10:01.300 | Well, no, this sounds exciting.

00:10:03.460 | - Yes, okay.

00:10:04.900 | So now we have like three things to talk about.

00:10:09.900 | - We'll keep adding more.

00:10:10.820 | - Although it's okay, the first and the third are related.

00:10:13.700 | So let's leave the efficiency question aside for now.

00:10:18.260 | So how does all this information get represented

00:10:22.340 | in logical form?

00:10:24.580 | So that these algorithms,

00:10:27.360 | resolution theorem proving and other algorithms

00:10:29.760 | can actually grind through all the logical consequences

00:10:33.420 | of what you said.

00:10:34.520 | And that ties into your question about,

00:10:36.880 | well, how many of these things do you need?

00:10:39.820 | Because if the answer is small enough,

00:10:42.460 | then by hand, you could write them out one at a time.

00:10:45.540 | So in the early 1984,

00:10:53.500 | I held a meeting at Stanford

00:10:56.220 | where I was a faculty member there,

00:10:59.660 | where we assembled about half a dozen

00:11:03.460 | of the smartest people I know.

00:11:05.680 | People like Alan Newell and Marvin Minsky

00:11:10.700 | and Alan Kay and a few others.

00:11:15.180 | - Was Feynman there by chance?

00:11:16.460 | 'Cause he liked your,

00:11:17.780 | he commented about your system, Eurysco, at the time.

00:11:20.540 | - No, no, he wasn't part of this meeting.

00:11:23.380 | - That's a heck of a meeting anyway.

00:11:24.980 | - I think Ed Feigenbaum was there.

00:11:26.660 | I think Josh Lederberg was there.

00:11:29.820 | So we have all these different smart people

00:11:34.220 | and we came together to address the question

00:11:39.220 | that you raised, which is,

00:11:41.420 | if it's important to represent common sense knowledge

00:11:44.060 | and world knowledge in order for AIs to not be brittle,

00:11:48.300 | in order for AIs not to just have

00:11:50.140 | the veneer of intelligence,

00:11:52.380 | well, how many pieces of common sense,

00:11:55.420 | how many if-then rules, for instance,

00:11:58.620 | would we have to actually write

00:11:59.980 | in order to essentially cover

00:12:02.340 | what people expect perfect strangers

00:12:05.380 | to already know about the world?

00:12:07.780 | And I expected there would be an enormous divergence

00:12:12.020 | of opinion and computation,

00:12:15.040 | but amazingly, everyone got an answer

00:12:17.540 | which was around a million.

00:12:20.700 | And one person got the answer by saying,

00:12:24.620 | "Well, look, you can only burn

00:12:27.060 | "into human long-term memory

00:12:29.220 | "a certain number of things per unit time,

00:12:31.800 | "like maybe one every 30 seconds or something.

00:12:34.540 | "And other than that, it's just short-term memory

00:12:36.740 | "and it flows away like water and so on."

00:12:39.260 | So by the time you're, say, 10 years old or so,

00:12:42.620 | how many things could you possibly have burned

00:12:45.180 | into your long-term memory?

00:12:46.420 | And it's like about a million.

00:12:48.460 | Another person went in a completely different direction

00:12:51.180 | and said, "Well, if you look at the number of words

00:12:55.060 | "in a dictionary, not a whole dictionary,

00:12:57.880 | "but for someone to essentially be considered

00:13:01.500 | "to be fluent in a language,

00:13:03.300 | "how many words would they need to know?

00:13:05.200 | "And then about how many things about each word

00:13:08.520 | "would you have to tell it?"

00:13:09.860 | And so they got to a million that way.

00:13:11.760 | Another person said, "Well, let's actually look

00:13:16.300 | "at one single short, one-volume desk encyclopedia article."

00:13:21.300 | And so we'll look at what was like

00:13:25.500 | a four-paragraph article or something.

00:13:28.300 | I think about Grebes.

00:13:29.780 | Grebes are a type of waterfowl.

00:13:32.340 | And if we were going to sit there

00:13:34.300 | and represent every single thing that was there,

00:13:38.260 | how many assertions or rules or statements

00:13:41.340 | would we have to write in this logical language and so on?

00:13:43.820 | And then multiply that by all of the number of articles

00:13:47.060 | that there were and so on.

00:13:48.820 | So all of these estimates came out with a million.

00:13:51.620 | And so if you do the math, it turns out that like,

00:13:55.740 | oh, well then maybe in something like 100 person years,

00:14:00.740 | in one or two person centuries,

00:14:06.280 | we could actually get this written down by hand.

00:14:10.020 | And a marvelous coincidence, opportunity existed

00:14:15.020 | right at that point in time, the early 1980s.

00:14:19.500 | There was something called

00:14:20.940 | the Japanese fifth-generation computing effort.

00:14:23.860 | Japan had threatened to do in computing and AI and hardware

00:14:28.860 | what they had just finished doing in consumer electronics

00:14:32.120 | and the automotive industry,

00:14:33.300 | namely resting control away from the United States

00:14:36.660 | and more generally away from the West.

00:14:39.220 | And so America was scared and Congress did something.

00:14:44.060 | That's how you know it was a long time ago

00:14:45.620 | 'cause Congress did something.

00:14:47.660 | Congress passed something called

00:14:48.860 | the National Cooperative Research Act, NCRA.

00:14:52.180 | And what it said was,

00:14:53.420 | hey, all you big American companies,

00:14:55.460 | that's also how you know it was a long time ago

00:14:57.400 | because they were American companies

00:14:59.300 | rather than multinational companies.

00:15:01.300 | Hey, all you big American companies,

00:15:03.820 | normally it would be an antitrust violation

00:15:06.660 | if you colluded on R&D,

00:15:09.340 | but we promise for the next 10 years,

00:15:12.100 | we won't prosecute any of you if you do that

00:15:15.100 | to help combat this threat.

00:15:17.020 | And so overnight, the first two consortia,

00:15:20.940 | research consortia in America sprang up,

00:15:24.340 | both of them coincidentally in Austin, Texas.

00:15:27.340 | One called Sematech focusing on hardware chips and so on,

00:15:31.220 | and then one called MCC,

00:15:33.400 | the Microelectronics and Computer Technology Corporation

00:15:36.540 | focusing more on software, on databases and AI

00:15:40.780 | and natural language understanding and things like that.

00:15:43.580 | And I got the opportunity,

00:15:46.940 | thanks to my friend Woody Bledsoe,

00:15:49.740 | who was one of the people who founded that,

00:15:52.940 | to come and be its principal scientist.

00:15:54.980 | And he said, you know,

00:15:56.180 | and he sent Admiral Bob Inman,

00:15:58.500 | who was the person running MCC,

00:16:01.680 | came and talked to me and said,

00:16:03.100 | "Look, Professor, you're talking about doing this project.

00:16:05.860 | "It's gonna involve person centuries of effort.

00:16:10.080 | "You've only got a handful of graduate students.

00:16:12.220 | "You do the math, it's gonna take you like, you know,

00:16:16.160 | "longer than the rest of your life to finish this project.

00:16:18.840 | "But if you move to the wilds of Austin, Texas,

00:16:21.780 | "we'll put 10 times as many people on it,

00:16:23.640 | "and you know, you'll be done in a few years."

00:16:27.340 | And so that was pretty exciting.

00:16:28.700 | And so I did that.

00:16:30.100 | I took my leave from Stanford.

00:16:33.000 | I came to Austin.

00:16:34.640 | I worked for MCC.

00:16:36.340 | And the good news and bad news,

00:16:38.660 | the bad news is that all of us were off

00:16:40.880 | by an order of magnitude.

00:16:42.740 | That it turns out what you need are tens of millions

00:16:45.460 | of these pieces of knowledge about everyday,

00:16:49.820 | sort of like if you have a coffee cup with stuff in it

00:16:53.300 | and you turn it upside down,

00:16:54.380 | the stuff in it's gonna fall out.

00:16:56.640 | So you need tens of millions

00:16:58.100 | of pieces of knowledge like that,

00:16:59.820 | even if you take trouble to make each one

00:17:02.660 | as general as it possibly could be.

00:17:04.940 | But the good news was that thanks to

00:17:10.220 | initially the fifth generation effort

00:17:13.760 | and then later US government agency funding and so on,

00:17:18.460 | we were able to get enough funding,

00:17:20.340 | not for a couple person centuries of time,

00:17:23.620 | but for a couple person millennia of time,

00:17:26.700 | which is what we've spent since 1984,

00:17:29.340 | getting Psych to contain the tens of millions of rules

00:17:33.500 | that it needs in order to really capture

00:17:35.980 | and span sort of not all of human knowledge,

00:17:39.580 | but the things that you assume other people,

00:17:42.620 | the things you count on other people knowing.

00:17:46.100 | And so by now we've done that.

00:17:49.700 | And the good news is since you've waited 38 years

00:17:53.420 | just about to talk to me,

00:17:56.820 | we're about at the end of that process.

00:17:59.620 | So most of what we're doing now is not putting in

00:18:01.940 | even what you would consider common sense,

00:18:03.740 | but more putting in domain specific applications,

00:18:07.420 | specific knowledge about healthcare in a certain hospital

00:18:12.420 | or about oil pipes getting clogged up

00:18:18.940 | or whatever the applications happen to be.

00:18:21.940 | So we've almost come full circle

00:18:24.340 | and we're doing things very much like the expert systems

00:18:27.260 | of the 1970s and the 1980s,

00:18:29.580 | except instead of resting on nothing and being brittle,

00:18:32.900 | they're now resting on this massive pyramid, if you will,

00:18:36.300 | this massive lattice of common sense knowledge

00:18:39.400 | so that when things go wrong,

00:18:40.980 | when something unexpected happens,

00:18:43.040 | they can fall back on more and more

00:18:44.980 | and more general principles,

00:18:47.100 | eventually bottoming out in things like,

00:18:50.420 | for instance, if we have a problem with the microphone,

00:18:53.000 | one of the things you'll do is unplug it,

00:18:55.940 | plug it in again and hope for the best, right?

00:18:58.420 | Because that's one of the general pieces of knowledge

00:19:00.480 | you have in dealing with electronic equipment

00:19:02.700 | or software systems or things like that.

00:19:05.700 | - Is there a basic principle like that?

00:19:07.180 | Like, is it possible to encode something

00:19:09.680 | that generally captures this idea of turn it off

00:19:13.500 | and turn it back on and see if it fixes?

00:19:15.700 | - Oh, absolutely.

00:19:16.540 | That's one of the things that Psych knows.

00:19:19.540 | - That's actually one of the fundamental laws

00:19:22.940 | of nature, I believe.

00:19:24.340 | (laughing)

00:19:25.180 | - I wouldn't call it a law.

00:19:27.100 | It's more like a--

00:19:29.580 | - Seems to work every time,

00:19:30.740 | so it sure looks like a law, I don't know.

00:19:34.300 | - So that basically covered the resources needed

00:19:39.300 | and then we had to devise a method to actually figure out,

00:19:43.180 | well, what are the tens of millions of things

00:19:45.520 | that we need to tell the system?

00:19:47.260 | And for that, we found a few techniques

00:19:51.180 | which worked really well.

00:19:52.580 | One is to take any piece of text almost,

00:19:56.620 | it could be an advertisement, it could be a transcript,

00:19:59.020 | it could be a novel, it could be an article,

00:20:01.800 | and don't pay attention to the actual type that's there,

00:20:05.940 | the black space on the white page.

00:20:08.380 | Pay attention to the complement of that,

00:20:10.540 | the white space, if you will.

00:20:11.860 | So what did the writer of this sentence assume

00:20:15.700 | that the reader already knew about the world?

00:20:17.900 | For instance, if they used a pronoun,

00:20:20.460 | how did they figure out that,

00:20:22.100 | why did they think that you would be able to understand

00:20:25.800 | what the intended referent of that pronoun was?

00:20:28.300 | If they used an ambiguous word,

00:20:30.200 | how did they think that you would be able to figure out

00:20:33.400 | what they meant by that word?

00:20:36.060 | The other thing we look at is the gap

00:20:38.500 | between one sentence and the next one.

00:20:40.660 | What are all the things that the writer expected you

00:20:43.200 | to fill in and infer occurred between the end

00:20:46.260 | of one sentence and the beginning of the other?

00:20:47.980 | So like if the sentence says,

00:20:49.480 | Fred Smith robbed the Third National Bank, period.

00:20:54.200 | He was sentenced to 20 years in prison, period.

00:20:58.980 | Well, between the first sentence and the second,

00:21:01.500 | you're expected to infer things like Fred got caught,

00:21:04.900 | Fred got arrested, Fred went to jail,

00:21:08.000 | Fred had a trial, Fred was found guilty, and so on.

00:21:11.780 | If my next sentence starts out with something like

00:21:14.380 | the judge, dot, dot, dot,

00:21:16.540 | then you assume it's the judge at his trial.

00:21:18.420 | If my next sentence starts out something like

00:21:20.580 | the arresting officer, dot, dot, dot,

00:21:22.420 | you assume that it was the police officer

00:21:24.580 | who arrested him after he committed the crime and so on.

00:21:26.980 | So those are two techniques for getting that knowledge.

00:21:31.980 | The other thing we sometimes look at is

00:21:35.660 | sort of like fake news or sort of humorous onion headlines

00:21:40.660 | or headlines in the Weekly World News,

00:21:44.620 | if you know what that is, or the National Enquirer,

00:21:46.940 | where it's like, oh, we don't believe this,

00:21:49.460 | then we introspect on why don't we believe it.

00:21:51.620 | So there are things like B-17 lands on the Moon.

00:21:56.620 | It's like, what do we know about the world

00:21:58.980 | that causes us to believe that that's just silly

00:22:01.460 | or something like that?

00:22:03.300 | Or another thing we look for are contradictions,

00:22:06.700 | things which can't both be true.

00:22:10.420 | And we say, what is it that we know

00:22:12.860 | that causes us to know that both of these

00:22:14.660 | can't be true at the same time?

00:22:17.140 | For instance, in one of the Weekly World News editions,

00:22:21.380 | in one article, it talked about how Elvis was cited,

00:22:24.820 | even though he was getting on in years and so on.

00:22:29.300 | And another article in the same one

00:22:31.140 | talked about people seeing Elvis's ghost.

00:22:33.940 | Okay, so it's like, why do we believe

00:22:36.300 | that at least one of these articles must be wrong and so on?

00:22:40.380 | So we have a series of techniques like that

00:22:43.300 | that enable our people.

00:22:44.820 | And by now, we have about 50 people

00:22:47.500 | working full-time on this and have for decades.

00:22:50.460 | So we've put in the thousands of person years of effort,

00:22:53.240 | we've built up these tens of millions of rules.

00:22:56.260 | We constantly police the system to make sure

00:23:00.100 | that we're saying things as generally as we possibly can.

00:23:04.700 | So you don't wanna say things like,

00:23:08.180 | no mouse is also a moose,

00:23:11.100 | because if you said things like that,

00:23:13.300 | then you'd have to add another one or two or three zeros

00:23:16.620 | onto the number of assertions you'd actually have to have.

00:23:20.480 | So at some point, we generalize things more and more,

00:23:23.900 | and we get to a point where we say,

00:23:25.140 | oh yeah, for any two biological taxons,

00:23:28.580 | if we don't know explicitly

00:23:30.340 | that one is a generalization of another,

00:23:32.660 | then almost certainly they're disjoint.

00:23:34.940 | A member of one is not gonna be a member of the other

00:23:37.740 | and so on, so--

00:23:38.580 | - And the same thing with the Elvis and the ghost,

00:23:40.740 | it has nothing to do with Elvis,

00:23:41.940 | it's more about human nature and the mortality

00:23:45.820 | and all that kind of stuff.

00:23:46.660 | - Well, in general, things are not both alive and dead

00:23:49.460 | at the same time.

00:23:50.460 | - Yeah, unless special cats in theoretical physics examples.

00:23:55.460 | - Well, that raises a couple important points.

00:23:58.340 | - Well, that's the onion headline situation type of thing.

00:24:00.980 | Okay, sorry.

00:24:01.820 | - But no, no, so what you bring up

00:24:03.420 | is this really important point of like,

00:24:04.860 | well, how do you handle exceptions

00:24:07.420 | and inconsistencies and so on?

00:24:10.980 | And one of the hardest lessons for us to learn,

00:24:14.020 | it took us about five years to really grit our teeth

00:24:17.540 | and learn to love it,

00:24:19.540 | is we had to give up global consistency.

00:24:23.460 | So the knowledge base can no longer be consistent.

00:24:27.060 | So this is a kind of scary thought.

00:24:28.740 | I grew up watching Star Trek,

00:24:30.540 | and anytime a computer was inconsistent,

00:24:32.940 | it would either freeze up or explode or take over the world

00:24:36.760 | or something bad would happen.

00:24:38.900 | Or if you come from a mathematics background,

00:24:41.260 | once you can prove false, you can prove anything,

00:24:44.300 | so that's not good and so on.

00:24:46.380 | So that's why the old knowledge-based systems

00:24:50.460 | were all very, very consistent.

00:24:52.900 | But the trouble is that by and large,

00:24:56.020 | our models of the world,

00:24:57.580 | the way we talk about the world and so on,

00:24:59.500 | there are all sorts of inconsistencies

00:25:02.180 | that creep in here and there

00:25:04.220 | that will sort of kill some attempt

00:25:05.820 | to build some enormous globally consistent knowledge base.

00:25:09.400 | And so what we had to move to

00:25:11.560 | was a system of local consistency.

00:25:14.440 | So a good analogy is,

00:25:16.160 | you know that the surface of the Earth

00:25:17.960 | is more or less spherical globally,

00:25:21.120 | but you live your life every day

00:25:24.960 | as though the surface of the Earth were flat.

00:25:27.680 | You know, when you're talking to someone in Australia,

00:25:29.600 | you don't think of them as being oriented upside down to you

00:25:32.800 | when you're planning a trip,

00:25:34.460 | even if it's a thousand miles away.

00:25:36.780 | You may think a little bit about time zones,

00:25:38.660 | but you rarely think about the curvature of the Earth

00:25:40.980 | and so on.

00:25:41.820 | And for most purposes, you can live your whole life

00:25:44.260 | without really worrying about that

00:25:46.380 | because the Earth is locally flat.

00:25:49.300 | In much the same way,

00:25:50.740 | the psych knowledge base is divided up

00:25:53.380 | into almost like tectonic plates,

00:25:55.620 | which are individual contexts,

00:25:57.620 | and each context is more or less consistent,

00:26:00.900 | but there can be small inconsistencies

00:26:03.980 | at the boundary between one context

00:26:06.460 | and the next one and so on.

00:26:08.180 | And so by the time you move, say, 20 contexts over,

00:26:12.180 | there could be glaring inconsistencies.

00:26:14.300 | So eventually you get from the normal,

00:26:17.300 | modern real world context that we're in right now

00:26:20.180 | to something like Roadrunner cartoon context

00:26:24.580 | where physics is very different,

00:26:26.220 | and in fact, life and death are very different

00:26:28.820 | because no matter how many times he's killed,

00:26:31.260 | you know, the coyote comes back in the next scene

00:26:33.620 | and so on.

00:26:35.220 | So that was a hard lesson to learn,

00:26:38.180 | and we had to make sure that our representation language,

00:26:41.260 | the way that we actually encode the knowledge

00:26:44.020 | and represent it,

00:26:45.180 | was expressive enough that we could talk about

00:26:47.300 | things being true in one context and false in another,

00:26:51.020 | things that are true at one time and false in another,

00:26:53.740 | things that are true, let's say, in one region,

00:26:56.420 | like one country, but false in another,

00:26:58.660 | things that are true in one person's belief system,

00:27:01.900 | but false in another person's belief system,

00:27:04.780 | things that are true at one level of abstraction

00:27:07.260 | and false at another.

00:27:08.140 | For instance, at one level of abstraction,

00:27:10.540 | you think of this table as a solid object,

00:27:12.720 | but down at the atomic level,

00:27:14.500 | it's mostly empty space and so on.

00:27:16.820 | - So then that's fascinating,

00:27:18.700 | but it puts a lot of pressure on context

00:27:21.460 | to do a lot of work.

00:27:23.420 | So you say tectonic plates.

00:27:25.460 | Is it possible to formulate contexts

00:27:27.500 | that are general and big

00:27:29.580 | that do this kind of capture of knowledge bases,

00:27:33.360 | or do you then get turtles on top of turtles,

00:27:36.140 | again, where there's just a huge number of contexts?

00:27:39.340 | - So, it's good you asked that question,

00:27:41.260 | 'cause you're pointed in the right direction,

00:27:43.820 | which is you want contexts to be first-class objects

00:27:48.820 | in your system's knowledge base,

00:27:50.820 | in particular in Psyche's knowledge base.

00:27:52.980 | By first-class object, I mean that

00:27:56.500 | we should be able to have Psyche think about

00:27:58.780 | and talk about and reason about

00:28:01.180 | one context or another context

00:28:03.140 | the same way it reasons about coffee cups

00:28:05.540 | and tables and people and fishing and so on.

00:28:09.540 | And so, contexts are just terms in its language,

00:28:13.460 | just like the ones I mentioned.

00:28:15.060 | And so, Psyche can reason about context,

00:28:18.580 | context can arrange hierarchically and so on.

00:28:22.180 | And so, you can say things about,

00:28:25.060 | let's say, things that are true in the modern era,

00:28:29.820 | things that are true in a particular year,

00:28:32.460 | would then be a sub-context of the things that are true

00:28:37.340 | in a broad, let's say, a century or a millennium

00:28:40.240 | or something like that.

00:28:41.480 | Things that are true in Austin, Texas,

00:28:44.180 | are generally gonna be a specialization

00:28:46.580 | of things that are true in Texas,

00:28:49.340 | which is gonna be a specialization of things

00:28:51.620 | that are true in the United States and so on.

00:28:54.260 | And so, you don't have to say things over and over again

00:28:58.340 | at all these levels, you just say things

00:29:00.220 | at the most general level that it applies to,

00:29:03.460 | and you only have to say it once,

00:29:04.940 | and then it essentially inherits

00:29:06.940 | to all these more specific contexts.

00:29:09.580 | - To ask a slightly technical question,

00:29:11.540 | is this inheritance a tree or a graph?

00:29:15.540 | - Oh, you definitely have to think of it as a graph.

00:29:18.200 | So, we could talk about, for instance,

00:29:20.480 | why the Japanese fifth-generation computing effort failed.

00:29:23.780 | There were about half a dozen different reasons.

00:29:26.420 | One of the reasons they failed was because

00:29:29.140 | they tried to represent knowledge as a tree

00:29:32.220 | rather than as a graph.

00:29:33.940 | And so, each node in their representation

00:29:38.380 | could only have one parent node.

00:29:42.460 | So, if you had a table that was a wooden object,

00:29:45.220 | a black object, a flat object, and so on,

00:29:48.060 | you had to choose one,

00:29:49.500 | and that's the only parent it could have.

00:29:52.460 | When, of course, depending on what it is

00:29:55.100 | you need to reason about it,

00:29:56.540 | sometimes it's important to know that it's made out of wood,

00:29:58.900 | like if we're talking about a fire.

00:30:01.040 | Sometimes it's important to know that it's flat

00:30:03.220 | if we're talking about resting something on it, and so on.

00:30:06.020 | So, one of the problems was that they wanted

00:30:10.940 | a kind of Dewey decimal numbering system

00:30:13.920 | for all of their concepts, which meant that

00:30:16.420 | each node could only have, at most, 10 children,

00:30:20.300 | and each node could only have one parent.

00:30:23.500 | And while that does enable the Dewey decimal type

00:30:28.500 | numbering of concepts, labeling of concepts,

00:30:32.220 | it prevents you from representing all the things

00:30:34.420 | you need to about objects in our world.

00:30:37.860 | And that was one of the things which

00:30:40.060 | they never were able to overcome,

00:30:41.940 | and I think that was one of the main reasons

00:30:43.940 | that that project failed.

00:30:45.360 | - So, we'll return to some of the doors you've opened,

00:30:47.580 | but if we can go back to that room in 1984,

00:30:50.780 | around there with Marvin Minsky and Stanford.

00:30:53.380 | - By the way, I should mention that Marvin

00:30:56.180 | wouldn't do his estimate until someone brought him

00:30:59.420 | an envelope so that he could literally do a back

00:31:02.460 | of the envelope calculation to come up with his number.

00:31:05.580 | - Well, because I feel like the conversation

00:31:10.820 | in that room is an important one.

00:31:15.820 | Sometimes science is done in this way.

00:31:17.500 | A few people get together and plant the seed of ideas,

00:31:20.900 | and they reverberate throughout history,

00:31:23.160 | and some kind of dissipate and disappear,

00:31:26.300 | and some, you know, Drake equation,

00:31:28.540 | and it seems like a meaningless equation,

00:31:31.700 | somewhat meaningless, but I think it drives

00:31:34.060 | and motivates a lot of scientists,

00:31:35.740 | and when the aliens finally show up,

00:31:37.860 | that equation will get even more valuable,

00:31:40.940 | because then we'll be able to,

00:31:42.380 | in the long arc of history, the Drake equation

00:31:44.860 | will prove to be quite useful, I think.

00:31:49.860 | In that same way, a conversation of just how many facts

00:31:54.620 | are required to capture the basic common sense knowledge

00:31:56.940 | of the world, that's a fascinating question.

00:31:58.860 | - I want to distinguish between what you think of as facts

00:32:01.820 | and the kind of things that we represent.

00:32:04.340 | So we map to and essentially make sure

00:32:08.940 | that psych has the ability to, as it were,

00:32:10.780 | read and access the kind of facts you might find,

00:32:13.900 | say, in Wikidata, or stated in a Wikipedia article,

00:32:18.900 | or something like that.

00:32:20.260 | So what we're representing, the things that we need

00:32:22.520 | a small number of tens of millions of,

00:32:24.580 | are more like rules of thumb, rules of good guessing,

00:32:27.980 | things which are usually true,

00:32:30.060 | and which help you to make sense of the facts

00:32:34.980 | that are sort of sitting off in some database

00:32:37.700 | or some other more static storage.

00:32:39.700 | - So they're almost like platonic forms,

00:32:42.020 | so when you read stuff on Wikipedia,

00:32:45.260 | that's going to be projections of those ideas.

00:32:47.420 | You read an article about the fact that Elvis died,

00:32:50.300 | that's a projection of the idea that humans are mortal.

00:32:54.040 | Very few Wikipedia articles will write humans are mortal.

00:33:00.180 | - Exactly, that's what I meant about ferreting out

00:33:02.980 | the unstated things in text.

00:33:05.220 | What are all the things that were assumed?

00:33:06.720 | And so those are things like,

00:33:09.120 | if you have a problem with something,

00:33:10.720 | turning it off and on often fixes it

00:33:13.540 | for reasons we don't really understand

00:33:15.300 | and we're not happy about,

00:33:16.420 | or people can't be both alive and dead at the same time,

00:33:20.700 | or water flows downhill.

00:33:23.540 | If you search online for water flowing uphill

00:33:25.920 | and water flowing downhill,

00:33:27.000 | you'll find more references for water flowing uphill,

00:33:29.660 | because it's used as a kind of a metaphorical reference

00:33:34.020 | for some unlikely thing,

00:33:35.740 | because of course everyone already knows

00:33:37.880 | that water flows downhill,

00:33:39.460 | so why would anyone bother saying that?

00:33:41.820 | - Do you have a word you prefer,

00:33:44.340 | 'cause we said facts isn't the right word,

00:33:46.020 | is there a word like concepts?

00:33:47.780 | - I would say assertions.

00:33:49.980 | Assertions or rules,

00:33:51.540 | because I'm not talking about rigid rules,

00:33:53.420 | but rules of thumb.

00:33:54.620 | But assertions is a nice one

00:33:57.020 | that covers all of these things.

00:33:59.820 | - Yeah, as a programmer to me,

00:34:01.660 | assert has a very dogmatic authoritarian feel to them.

00:34:06.380 | - Oh, I'm sorry.

00:34:07.220 | (laughing)

00:34:08.340 | I'm so sorry.

00:34:09.460 | Okay, but assertions works.

00:34:11.180 | Okay, so if we go back to that room

00:34:13.300 | with Marvin Minsky with you,

00:34:15.260 | all these seminal figures,

00:34:16.740 | Ed Feigenbaum,

00:34:20.060 | thinking about this very philosophical,

00:34:23.480 | but also engineering question,

00:34:25.760 | we can also go back a couple of decades before then

00:34:29.740 | and thinking about artificial intelligence broadly,

00:34:32.540 | when people were thinking about

00:34:34.740 | how do you create superintelligent systems,

00:34:37.860 | general intelligence.

00:34:39.460 | And I think people's intuition was off at the time.

00:34:44.460 | And I mean, this continues to be the case

00:34:47.780 | that we're not,

00:34:48.940 | when we're grappling with these exceptionally difficult ideas

00:34:52.420 | we're not always,

00:34:53.300 | it's very difficult to truly understand ourselves

00:34:56.780 | when we're thinking about the human mind

00:34:59.340 | to introspect how difficult it is to engineer intelligence,

00:35:03.960 | to solve intelligence.

00:35:05.000 | We're not very good at estimating that.

00:35:07.260 | And you are somebody who has really stayed

00:35:10.740 | with this question for decades.

00:35:12.480 | Do you, what's your sense from the 1984 to today,

00:35:18.660 | have you gotten a stronger sense

00:35:21.480 | of just how much knowledge is required?

00:35:23.900 | So you've kind of said with some level of certainty

00:35:27.140 | that it's still on the order of magnitude

00:35:28.660 | of tens of millions.

00:35:30.220 | - Right, for the first several years,

00:35:31.940 | I would have said that it was on the order

00:35:33.480 | of one or two million.

00:35:36.340 | And so it took us about five or six years

00:35:40.220 | to realize that we were off by a factor of 10.

00:35:44.020 | - But I guess what I'm asking,

00:35:46.420 | Marvin Misk is very confident in the '60s when you say.

00:35:49.380 | - Yes. - Right.

00:35:50.700 | What's your sense,

00:35:52.940 | if you,

00:35:54.880 | 200 years from now,

00:35:59.560 | you're still,

00:36:00.440 | you're not going to be any longer

00:36:03.220 | in this particular biological body,

00:36:05.400 | but your brain will still be in the digital form,

00:36:09.100 | and you'll be looking back,

00:36:10.740 | would you think you were smart today?

00:36:14.180 | Like your intuition was right?

00:36:15.620 | Or do you think you may be really off?

00:36:19.220 | - So I think I'm right enough.

00:36:22.260 | And let me explain what I mean by that,

00:36:24.380 | which is sometimes like if you have an old fashioned pump,

00:36:29.340 | you have to prime the pump.

00:36:30.860 | - Yeah. - And then eventually

00:36:31.860 | it starts.

00:36:32.700 | So I think I'm right enough in the sense that--

00:36:36.280 | - To prime the pump.

00:36:37.120 | - What we've built,

00:36:38.880 | even if it isn't, so to speak, everything you need,

00:36:42.500 | it's primed the knowledge pump enough

00:36:45.380 | that psych can now itself help to learn

00:36:50.380 | more and more automatically on its own

00:36:53.300 | by reading things and understanding

00:36:55.160 | and occasionally asking questions

00:36:56.720 | like a student would or something.

00:36:59.200 | And by doing experiments

00:37:00.720 | and discovering things on its own and so on.

00:37:03.160 | So through a combination of psych-powered discovery

00:37:07.640 | and psych-powered reading,

00:37:09.860 | it will be able to bootstrap itself.

00:37:12.560 | Maybe it's the final 2%, maybe it's the final 99%.

00:37:16.340 | So even if I'm wrong,

00:37:18.800 | all I really need to build is a system

00:37:21.880 | which has primed the pump enough

00:37:24.140 | that it can begin that cascade upward,

00:37:28.120 | that self-reinforcing sort of quadratically

00:37:31.360 | or maybe even exponentially increasing path upward

00:37:36.360 | that we get from, for instance, talking with each other.

00:37:39.620 | That's why humans today know so much more

00:37:43.400 | than humans 100,000 years ago.

00:37:45.400 | We're not really that much smarter

00:37:47.080 | than people were 100,000 years ago.

00:37:49.280 | But there's so much more knowledge

00:37:50.680 | and we have language and we can communicate.

00:37:53.040 | We can check things on Google and so on.

00:37:55.400 | So effectively, we have this enormous power

00:37:58.160 | at our fingertips.

00:37:59.520 | And there's almost no limit

00:38:01.320 | to how much you could learn if you wanted to

00:38:03.780 | because you've already gotten to a certain level

00:38:06.320 | of understanding of the world

00:38:08.080 | that enables you to read all these articles

00:38:10.440 | and understand them,

00:38:11.520 | that enables you to go out and if necessary,

00:38:13.720 | do experiments, although that's slower,

00:38:15.840 | as a way of gathering data and so on.

00:38:19.080 | And I think this is really an important point,

00:38:21.580 | which is if we have artificial intelligence,

00:38:25.000 | real general artificial intelligence,

00:38:27.120 | human level artificial intelligence,

00:38:29.520 | then people will become smarter.

00:38:33.960 | It's not so much that it'll be us versus the AIs.

00:38:37.040 | It's more like us and the AIs together

00:38:39.960 | will be able to do things that require more creativity

00:38:43.640 | that would take too long right now,

00:38:45.500 | but we'll be able to do lots of things in parallel.

00:38:48.200 | We'll be able to misunderstand each other less.

00:38:52.040 | There's all sorts of value that effectively

00:38:55.640 | for an individual would mean that individual will,

00:38:58.900 | for all intents and purposes, be smarter.

00:39:01.280 | And that means that humanity as a species will be smarter.

00:39:05.480 | And when was the last time that any invention

00:39:08.840 | qualitatively made a huge difference in human intelligence?

00:39:13.840 | You have to go back a long ways.

00:39:15.460 | It wasn't like the internet or the computer

00:39:17.720 | or mathematics or something.

00:39:19.640 | It was all the way back to the development of language.

00:39:24.440 | We sort of look back on pre-linguistic cavemen as,

00:39:28.440 | well, they weren't really intelligent, were they?

00:39:32.840 | They weren't really human, were they?

00:39:34.840 | And I think that, as you said, 50, 100, 200 years from now,

00:39:39.840 | people will look back on people today

00:39:42.560 | right before the advent of the sort of lifelong

00:39:47.920 | general AI muses and say, "Those poor people,

00:39:52.920 | "they weren't really human, were they?"

00:39:56.240 | - Exactly.

00:39:57.240 | So you said a lot of really interesting things.

00:39:59.200 | By the way, I would maybe try to argue that the internet

00:40:04.200 | is on the order of the kind of big leap in improvement

00:40:10.200 | that the invention of language was.

00:40:14.680 | - Well, certainly a big leap in one direction.

00:40:16.560 | We're not sure whether it's upward or downward.

00:40:18.440 | - Well, I mean very specific parts of the internet,

00:40:21.400 | which is access to information,

00:40:22.840 | like a website like Wikipedia,

00:40:25.000 | like ability for human beings from across the world

00:40:27.780 | to access information very quickly.

00:40:29.840 | - So I could take either side of this argument,

00:40:32.200 | and since you just took one side,

00:40:34.160 | I'll give you the other side,

00:40:35.240 | which is that almost nothing has done more harm

00:40:39.000 | than something like the internet

00:40:42.480 | and access to that information in two ways.

00:40:45.440 | One is it's made people more globally ignorant

00:40:50.440 | in the same way that calculators

00:40:53.560 | made us more or less innumerate.

00:40:56.900 | So when I was growing up, we had to use slide rules,

00:40:59.840 | we had to be able to estimate and so on.

00:41:02.880 | Today, people don't really understand numbers,

00:41:06.640 | they don't really understand math,

00:41:08.120 | they don't really estimate very well at all, and so on.

00:41:11.920 | They don't really understand the difference

00:41:13.480 | between trillions and billions and millions

00:41:15.640 | and so on very well,

00:41:16.880 | because calculators do that all for us.

00:41:21.200 | And thanks to things like the internet and search engines,

00:41:26.200 | that same kind of juvenile-ism is reinforced

00:41:31.640 | in making people essentially be able

00:41:33.840 | to live their whole lives,

00:41:35.080 | not just without being able to do arithmetic and estimate,

00:41:38.040 | but now without actually having

00:41:39.720 | to really know almost anything.

00:41:41.680 | Because anytime they need to know something,

00:41:43.400 | they'll just go and look it up.

00:41:45.040 | - You're right, and I could tell you

00:41:46.240 | could play both sides of this,

00:41:47.440 | and it is a double-edged sword.

00:41:48.840 | You can, of course, say the same thing about language.

00:41:50.920 | Probably people, when they invented language,

00:41:53.040 | they would criticize,

00:41:54.200 | you know, it used to be we would just,

00:41:56.080 | if we're angry, we would just kill a person,

00:41:58.480 | and if we're in love, we would just have sex with them,

00:42:00.720 | and now everybody's writing poetry and bullshit.

00:42:03.560 | You know, you should just be direct,

00:42:05.520 | you should have physical contact.

00:42:07.720 | Enough of this words and books,

00:42:09.960 | and you're not actually experiencing,

00:42:12.520 | like if you read a book,

00:42:13.480 | you're not experiencing the thing.

00:42:15.040 | This is nonsense.

00:42:16.520 | - That's right, if you read a book

00:42:17.560 | about how to make butter,

00:42:18.600 | that's not the same as,

00:42:19.520 | you have to learn it and do it yourself.

00:42:21.560 | - Exactly, do it yourself.

00:42:22.400 | - And so on.

00:42:23.320 | So let's just say that something is gained,

00:42:25.000 | but something is lost every time you have

00:42:28.120 | these sorts of dependencies on technology.

00:42:33.120 | And overall, I think that having smarter individuals

00:42:38.760 | and having smarter AI augmented human species

00:42:43.640 | will be one of the few ways

00:42:45.840 | that we'll actually be able to overcome

00:42:48.080 | some of the global problems we have

00:42:49.640 | involving poverty and starvation and global warming

00:42:53.920 | and overcrowding, all the other problems

00:42:57.280 | that are besetting the planet.

00:43:00.840 | We really need to be smarter,

00:43:02.520 | and there are really only two routes to being smarter.

00:43:04.840 | One is through biochemistry,

00:43:07.840 | and genetics, genetic engineering.

00:43:12.600 | The other route is through having general AIs

00:43:16.040 | that augment our intelligence.

00:43:19.560 | And hopefully one of those two ways

00:43:24.480 | of paths to salvation will come through

00:43:28.560 | before it's too late.

00:43:29.840 | - Yeah, absolutely, I agree with you.

00:43:30.840 | And obviously as an engineer,

00:43:32.440 | I have a better sense and an optimism

00:43:37.440 | about the technology side of things

00:43:38.880 | because you can control things there more.

00:43:40.920 | Biology is just such a giant mess.

00:43:43.120 | We're living through a pandemic now.

00:43:44.600 | There's so many ways that nature can just be

00:43:47.520 | just destructive and destructive in a way

00:43:50.560 | where it doesn't even notice you.

00:43:52.280 | It's not like a battle of humans versus virus.

00:43:55.440 | It's just like, huh, okay.

00:43:56.880 | And then you can just wipe out an entire species.

00:43:59.560 | - The other problem with the internet

00:44:02.360 | is that it has enabled us to surround ourselves

00:44:07.120 | with an echo chamber, with a bubble of like-minded people,

00:44:12.120 | which means that you can have truly bizarre theories,

00:44:17.120 | conspiracy theories, fake news, and so on, promulgate,

00:44:21.160 | and surround yourself with people who essentially reinforce

00:44:25.920 | what you want to believe

00:44:27.960 | or what you already believe about the world.

00:44:30.440 | And in the old days, that was much harder to do

00:44:34.960 | when you had, say, only three TV networks,

00:44:37.720 | or even before when you had no TV networks

00:44:40.160 | and you had to actually look at the world

00:44:42.200 | and make your own reasoned decisions.

00:44:44.360 | - I like the push and pull of our dance that we're doing

00:44:46.840 | because then I'll just say, in the old world,

00:44:48.760 | having come from the Soviet Union,

00:44:50.560 | because you had one or a couple of networks,

00:44:53.000 | then propaganda could be much more effective

00:44:55.040 | and then the government can overpower its people

00:44:57.520 | by telling you the truth and then starving millions

00:45:01.920 | and torturing millions

00:45:03.680 | and putting millions into camps

00:45:05.800 | and starting wars with a propaganda machine,

00:45:08.600 | allowing you to believe

00:45:09.480 | that you're actually doing good in the world.

00:45:11.680 | With the internet,

00:45:12.840 | because of all the quote-unquote conspiracy theories,

00:45:16.240 | some of them are actually challenging the power centers,

00:45:18.920 | the very kind of power centers that a century ago

00:45:22.400 | would have led to the death of millions.

00:45:25.320 | So there's, again, this double-edged sword.

00:45:27.960 | And I very much agree with you on the AI side.

00:45:30.480 | It's often an intuition that people have

00:45:33.560 | that somehow AI will be used to maybe overpower people

00:45:38.040 | by certain select groups.

00:45:40.040 | And to me, it's not at all obvious

00:45:41.960 | that that's the likely scenario.

00:45:43.440 | To me, the likely scenario,

00:45:45.680 | especially just having observed the trajectory of technology,

00:45:48.800 | is it'll be used to empower people.

00:45:50.520 | It'll be used to extend the capabilities

00:45:54.680 | of individuals across the world,

00:45:58.320 | 'cause there's a lot of money to be made that way.

00:46:00.920 | Like improving people's lives, you can make a lot of money.

00:46:03.280 | - I agree.

00:46:04.120 | I think that the main thing that AI prostheses,

00:46:09.120 | AI amplifiers will do for people is make it easier,

00:46:15.120 | maybe even unavoidable,

00:46:16.880 | for them to do good critical thinking.

00:46:20.000 | So pointing out logical fallacies,

00:46:23.040 | logical contradictions, and so on,

00:46:25.560 | in things that they otherwise would just blithely believe.

00:46:31.120 | Pointing out, essentially, data

00:46:35.160 | which they should take into consideration

00:46:38.720 | if they really want to learn the truth

00:46:42.160 | about something, and so on.

00:46:43.680 | So I think doing not just educating

00:46:46.600 | in the sense of pouring facts into people's heads,

00:46:50.400 | but educating in the sense of arming people

00:46:53.120 | with the ability to do good critical thinking

00:46:56.160 | is enormously powerful.

00:46:58.640 | The education system that we have in the US and worldwide

00:47:03.240 | generally don't do a good job of that.

00:47:05.920 | But I believe that the AI--

00:47:09.400 | - AI's will.

00:47:10.240 | - The AI's will, the AI's can and will.

00:47:12.320 | In the same way that everyone can have their own Alexa

00:47:16.640 | or Siri or Google Assistant or whatever,

00:47:20.120 | everyone will have this sort of cradle to grave assistant

00:47:25.640 | which will get to know you, which you'll get to trust.

00:47:28.040 | It'll model you, you'll model it.

00:47:30.600 | And it'll call to your attention things

00:47:33.600 | which will, in some sense, make your life better, easier,

00:47:38.040 | less mistake-ridden, and so on, less regret-ridden,

00:47:42.860 | if you listen to it.

00:47:46.600 | - Yeah, I'm in full agreement with you

00:47:48.640 | about this space of technologies,

00:47:51.880 | and I think it's super exciting.

00:47:53.120 | From my perspective, integrating emotional intelligence,

00:47:57.080 | so even things like friendship and companionship and love

00:48:00.360 | into those kinds of systems,

00:48:02.760 | as opposed to helping you just grow intellectually

00:48:06.160 | as a human being, allow you to grow emotionally,

00:48:09.120 | which is ultimately what makes life amazing,

00:48:12.040 | is to sort of, you know, the old pursuit of happiness.

00:48:16.400 | So it's not just the pursuit of reason,

00:48:18.320 | it's the pursuit of happiness too.

00:48:20.200 | The full spectrum.

00:48:21.560 | Well, let me, sort of, 'cause you mentioned

00:48:24.160 | so many fascinating things.

00:48:25.440 | Let me jump back to the idea of automated reasoning.

00:48:29.320 | So the acquisition of new knowledge

00:48:32.640 | has been done in this very interesting way,

00:48:35.240 | but primarily by humans doing this--

00:48:39.240 | - Yes, you can think of monks in their cells

00:48:41.920 | in medieval Europe, you know,

00:48:44.960 | carefully illuminating manuscripts and so on.

00:48:47.960 | - It's a very difficult and amazing process, actually,

00:48:50.920 | because it allows you to truly ask the question

00:48:53.520 | about in the white space, what is assumed?

00:48:58.160 | I think this exercise is, like,

00:49:02.040 | very few people do this, right?

00:49:03.800 | They just do it subconsciously.

00:49:06.080 | They perform this-- - By definition.

00:49:07.840 | - By definition. - Because those pieces

00:49:10.080 | of elided, of omitted information,

00:49:13.800 | of those missing steps, as it were,

00:49:15.840 | are pieces of common sense.

00:49:18.760 | If you actually included all of them,

00:49:21.080 | it would almost be offensive or confusing to the reader.

00:49:24.880 | It's like, why are they telling me all these?

00:49:26.520 | Of course I know all these things.

00:49:28.880 | And so, it's one of these things

00:49:32.440 | which, almost by its very nature,

00:49:34.440 | has almost never been explicitly written down anywhere,

00:49:40.160 | because by the time you're old enough

00:49:42.680 | to talk to other people and so on,

00:49:45.400 | you know, if you survived to that age,

00:49:47.800 | presumably you already got pieces of common sense,

00:49:50.600 | like, you know, if something causes you pain

00:49:53.560 | whenever you do it, probably not a good idea

00:49:55.560 | to keep doing it.

00:49:56.400 | - So, what ideas do you have,

00:50:00.280 | given how difficult this step is,

00:50:02.680 | what ideas are there for how to do it automatically

00:50:06.280 | without using humans, or at least not,

00:50:09.600 | you know, doing, like, a large percentage

00:50:13.400 | of the work for humans, and then humans only do

00:50:15.480 | the very high-level supervisory work?

00:50:18.720 | - So, we have, in fact, two directions we're pushing on

00:50:23.440 | very, very heavily currently at PsyCorps,

00:50:25.880 | and one involves natural language understanding

00:50:28.640 | and the ability to read what people

00:50:30.240 | have explicitly written down,

00:50:32.400 | and to pull knowledge in that way.

00:50:35.120 | But the other is to build a series

00:50:38.160 | of knowledge-editing tools, knowledge-entry tools,

00:50:42.560 | knowledge-capture tools, knowledge-testing tools,

00:50:48.440 | and so on.

00:50:49.280 | Think of them as, like, user-interfaced suite

00:50:52.960 | of software tools, if you want,

00:50:54.680 | something that will help people to,

00:50:56.880 | more or less, automatically expand and extend the system

00:51:01.200 | in areas where, for instance, they want to build some app,

00:51:05.040 | have it do some application, or something like that.

00:51:08.360 | So, I'll give you an example of one,

00:51:10.400 | which is something called abduction.

00:51:13.400 | So, you've probably heard of, like, deduction,

00:51:17.400 | and induction, and so on.

00:51:19.520 | But abduction is unlike those.

00:51:21.960 | Abduction is not sound, it's just useful.

00:51:25.920 | So, for instance, deductively,

00:51:30.440 | if someone is out in the rain,

00:51:32.800 | and they're gonna get all wet,

00:51:34.360 | and when they enter a room,

00:51:36.360 | they might be all wet, and so on.

00:51:37.960 | So, that's deduction.

00:51:41.160 | But if someone were to walk into the room right now,

00:51:44.400 | and they were dripping wet,

00:51:46.880 | we would immediately look outside to say,

00:51:48.880 | oh, did it start to rain, or something like that.

00:51:51.360 | Now, why did we say, maybe it started to rain?

00:51:55.240 | That's not a sound, logical inference,

00:51:57.800 | but it's certainly a reasonable abductive leap to say,

00:52:02.800 | well, one of the most common ways

00:52:05.880 | that a person would have gotten dripping wet

00:52:08.640 | is if they had gotten caught out in the rain,

00:52:10.800 | or something like that.

00:52:12.640 | So, what does that have to do

00:52:15.880 | with what we were talking about?

00:52:16.720 | So, suppose you're building one of these applications,

00:52:19.800 | and the system gets some answer wrong.

00:52:22.480 | And you say, oh, yeah, the answer to this question

00:52:25.280 | is this one, not the one you came up with.

00:52:29.000 | Then, what the system can do is it can use

00:52:31.960 | everything it already knows about common sense,

00:52:34.440 | general knowledge, the domain you've already

00:52:36.360 | been telling it about, and context,

00:52:39.240 | like we talked about, and so on,

00:52:40.560 | and say, well, here are seven alternatives,

00:52:45.240 | each of which I believe is plausible,

00:52:48.160 | given everything I already know.

00:52:49.840 | And if any of these seven things were true,

00:52:52.660 | I would have come up with the answer you just gave me

00:52:54.940 | instead of the wrong answer I came up with.

00:52:57.080 | Is one of these seven things true?

00:52:59.080 | And then, you, the expert, will look at those seven things

00:53:02.960 | and say, oh, yeah, number five is actually true.

00:53:05.560 | And so, without actually having to tinker down

00:53:08.540 | at the level of logical assertions and so on,

00:53:12.560 | you'll be able to educate the system

00:53:15.280 | in the same way that you would help educate another person

00:53:18.560 | who you were trying to apprentice or something like that.

00:53:22.040 | - So, that significantly reduces the mental effort

00:53:25.820 | or significantly increases the efficiency

00:53:28.200 | of the teacher, the human teacher.

00:53:30.240 | - Exactly, and it makes more or less anyone

00:53:32.360 | able to be a teacher in that way.

00:53:36.200 | So, that's part of the answer.

00:53:38.920 | And then, the other is that the system on its own

00:53:42.440 | will be able to, through reading,

00:53:45.080 | through conversations with other people and so on,

00:53:49.240 | learn the same way that you or I or other humans do.

00:53:54.240 | - First of all, that's a beautiful vision.

00:53:57.320 | I'll have to ask you about semantic web

00:53:59.080 | in a second here, but first,

00:54:00.720 | are there, when we talk about specific techniques,

00:54:05.960 | do you find something inspiring or directly useful

00:54:09.320 | from the whole space of machine learning, deep learning,

00:54:12.520 | these kinds of spaces of techniques

00:54:14.140 | that have been shown effective for certain kinds

00:54:17.040 | of problems in the recent, now, decade and a half?

00:54:21.040 | - I think of the machine learning work

00:54:24.040 | as more or less what our right brain hemispheres do.

00:54:29.480 | So, being able to take a bunch of data

00:54:34.480 | and recognize patterns, being able to statistically

00:54:38.760 | infer things and so on.

00:54:41.200 | And I certainly wouldn't want to not have

00:54:45.720 | a right brain hemisphere, but I'm also glad

00:54:48.400 | that I have a left brain hemisphere as well,

00:54:50.480 | something that can metaphorically sit back

00:54:53.600 | and puff on its pipe and think about this thing over here.

00:54:57.400 | It's like, why might this have been true?

00:54:59.440 | And what are the implications of it?

00:55:02.400 | How should I feel about that and why and so on?

00:55:04.900 | So, thinking more deeply and slowly,

00:55:08.660 | what Kahneman called thinking slowly

00:55:11.220 | versus thinking quickly.

00:55:12.740 | Whereas you want machine learning to think quickly,

00:55:15.380 | but you want the ability to think deeply,

00:55:18.180 | even if it's a little slower.

00:55:20.660 | So, I'll give you an example of a project we did recently

00:55:23.580 | with NIH involving the Cleveland Clinic

00:55:26.780 | and a couple other institutions that we ran a project for.

00:55:31.780 | And what it did was it took GWAS's,

00:55:35.460 | Genome-Wide Association Studies.

00:55:37.560 | Those are sort of big databases of patients

00:55:42.780 | that came into a hospital.

00:55:44.940 | They got their DNA sequenced because the cost of doing that

00:55:48.860 | has gone from infinity to billions of dollars

00:55:52.180 | to a hundred dollars or so.

00:55:54.660 | And so now patients routinely get their DNA sequenced.

00:55:58.140 | So, you have these big databases of the SNPs,

00:56:01.940 | the single nucleotide polymorphisms,

00:56:03.820 | the point mutations in a patient's DNA,

00:56:07.140 | and the disease that happened to bring them

00:56:09.460 | into the hospital.

00:56:10.780 | So, now you can do correlation studies,

00:56:13.380 | machine learning studies of which mutations

00:56:17.380 | are associated with and led to which physiological problems

00:56:22.940 | and diseases and so on, like getting arthritis and so on.

00:56:27.340 | And the problem is that those correlations

00:56:30.340 | turned out to be very spurious.

00:56:31.980 | They turned out to be very noisy.

00:56:34.580 | Very many of them have led doctors

00:56:36.980 | onto wild goose chases and so on.

00:56:39.740 | And so they wanted a way of eliminating,

00:56:41.860 | or the bad ones are focusing on the good ones.

00:56:44.620 | And so this is where Psyche comes in,

00:56:47.140 | which is Psyche takes those sort of A to Z correlations

00:56:50.740 | between point mutations and the medical condition

00:56:55.040 | that needs treatment.

00:56:56.120 | And we say, okay, let's use all this public knowledge

00:57:00.340 | and common sense knowledge about what reactions occur

00:57:04.420 | where in the human body, what polymerizes what,

00:57:07.780 | what catalyzes what reactions and so on.

00:57:10.460 | And let's try to put together a 10 or 20 or 30 step

00:57:15.340 | causal explanation of why that mutation

00:57:19.260 | might have caused that medical condition.

00:57:22.140 | And so Psyche would put together in some sense

00:57:24.580 | some Rube Goldberg-like chain that would say,

00:57:28.140 | oh yeah, that mutation, if it got expressed,

00:57:32.340 | would be this altered protein, which because of that,

00:57:37.140 | if it got to this part of the body,

00:57:38.700 | would catalyze this reaction.

00:57:40.180 | And by the way, that would cause more bioactive vitamin D

00:57:43.260 | in the person's blood.

00:57:44.400 | And anyway, 10 steps later, that screws up bone resorption

00:57:48.660 | and that's why this person got osteoporosis

00:57:51.540 | early in life and so on.

00:57:52.900 | - So that's human interpretable,

00:57:54.180 | or at least doctor human interpretable.

00:57:55.820 | - Exactly.

00:57:56.660 | And the important thing, even more than that,

00:57:59.980 | is you shouldn't really trust that 20 step

00:58:04.500 | Rube Goldberg chain any more than you trust

00:58:08.540 | that initial A to Z correlation, except two things.

00:58:11.660 | One, if you can't even think of one causal chain

00:58:15.980 | to explain this, then that correlation probably

00:58:19.500 | was just noise to begin with.

00:58:21.820 | And secondly, and even more powerfully,

00:58:24.300 | along the way, that causal chain will make predictions,

00:58:28.340 | like the one about having more bioactive vitamin D

00:58:31.240 | in your blood.

00:58:32.080 | So you can now go back to the data about these patients

00:58:35.540 | and say, by the way, did they have slightly elevated levels

00:58:40.100 | of bioactive vitamin D in their blood and so on?

00:58:42.780 | And if the answer is no, that strongly disconfirms

00:58:46.500 | your whole causal chain.

00:58:48.140 | And if the answer is yes, that somewhat confirms

00:58:50.940 | that causal chain.

00:58:51.860 | And so using that, we were able to take these correlations

00:58:56.620 | from this GWAS database, and we were able

00:58:59.060 | to essentially focus the doctor's,

00:59:03.380 | focus the researcher's attention on the very small percentage

00:59:07.260 | of correlations that had some explanation,

00:59:11.340 | and even better, some explanation that also made

00:59:13.560 | some independent prediction that they could confirm

00:59:16.060 | or disconfirm by looking at the data.

00:59:17.940 | So think of it like this kind of synergy

00:59:20.260 | where you want the right brain machine learning

00:59:23.340 | to quickly come up with possible answers.

00:59:25.700 | You want the left brain psych-like AI to think about that,

00:59:30.700 | and now think about why that might have been the case,

00:59:34.540 | and what else would be the case if that were true,

00:59:36.480 | and so on, and then suggest things back to the right brain

00:59:40.040 | to quickly check out again.

00:59:41.740 | So it's that kind of synergy back and forth,

00:59:45.540 | which I think is really what's gonna lead to general AI,

00:59:49.240 | not narrow, brittle machine learning systems,

00:59:53.260 | and not just something like psych.

00:59:55.620 | - Okay, so that's a brilliant synergy,

00:59:57.620 | but I was also thinking in terms of the automated expansion

01:00:00.820 | of the knowledge base, you mentioned NLU.

01:00:03.260 | This is very early days in the machine learning space

01:00:07.560 | of this, but self-supervised learning methods,

01:00:10.580 | you know, you have these language models, GPT-3, and so on,

01:00:14.220 | that just read the internet, and they form representations

01:00:18.620 | that can then be mapped to something useful.

01:00:21.000 | The question is, what is the useful thing?

01:00:23.760 | Like, they're now playing with a pretty cool thing

01:00:26.000 | called OpenAI Codex, which is generating programs

01:00:29.600 | from documentation.

01:00:30.720 | Okay, that's kind of useful, it's cool.

01:00:32.600 | But my question is, can it be used to generate,

01:00:35.960 | in part, maybe with some human supervision,

01:00:40.240 | a psych-like assertions, help feed psych more assertions

01:00:45.240 | from this giant body of internet data?

01:00:48.560 | - Yes, that is in fact one of our goals,

01:00:51.320 | is how can we harness machine learning,

01:00:53.600 | how can we harness natural language processing

01:00:56.600 | to increasingly automate the knowledge acquisition process,

01:01:01.060 | the growth of psych?

01:01:02.320 | And that's what I meant by priming the pump,

01:01:04.560 | that if you sort of learn things at the fringe

01:01:09.240 | of what you know already, you learn this new thing

01:01:12.200 | is similar to what you know already,

01:01:13.680 | and here are the differences in the new things

01:01:15.680 | you had to learn about it, and so on.

01:01:17.080 | So the more you know, the more and more easily

01:01:20.200 | you can learn new things.

01:01:21.960 | But unfortunately, inversely,

01:01:23.960 | if you don't really know anything,

01:01:25.960 | then it's really hard to learn anything.

01:01:28.520 | And so if you're not careful, if you start out

01:01:31.520 | with too small sort of a core to start this process,

01:01:36.520 | it never really takes off.

01:01:38.560 | And so that's why I view this as a pump priming exercise,

01:01:41.360 | to get a big enough, manually produced,

01:01:44.340 | even though that's kind of ugly duckling technique,

01:01:47.080 | put in the elbow grease to produce a large enough core

01:01:50.880 | that you will be able to do all the kinds of things

01:01:53.840 | you're imagining without sort of ending up

01:01:58.160 | with the kind of wacky brittlenesses

01:02:01.760 | that we see, for example, in GPT-3,

01:02:06.000 | where you'll tell it a story about someone

01:02:13.040 | putting a poison, plotting to poison someone, and so on.

01:02:18.040 | And then the, you know, the, you know,

01:02:20.260 | but GPT-3 says, oh, what's, you say,

01:02:22.420 | what's the very next sentence?

01:02:23.660 | The next sentence is, oh yeah,

01:02:24.780 | that person then drank the poison they just put together.

01:02:27.180 | It's like, that's probably not what happened for someone.

01:02:29.540 | Or if you go to Siri and, you know, I think I have,

01:02:34.540 | you know, where can I go for help

01:02:37.740 | with my alcohol problem or something?

01:02:40.920 | It'll come back and say, I found seven liquor stores

01:02:43.980 | near you, you know, and you know, so on.

01:02:47.080 | So, you know, it's one of these things where, yes,

01:02:50.140 | it may be helpful most of the time.

01:02:53.360 | It may even be correct most of the time,

01:02:56.160 | but if it doesn't really understand what it's saying,

01:02:59.380 | and if it doesn't really understand why things are true

01:03:01.780 | and doesn't really understand how the world works,

01:03:04.460 | then some fraction of the time, it's gonna be wrong.

01:03:07.500 | Now, if your only goal is to sort of find relevant

01:03:10.860 | information like search engines do,

01:03:14.540 | then being right 90% of the time is fantastic.

01:03:17.660 | That's unbelievably great.

01:03:19.340 | Okay, however, if your goal is to like, you know,

01:03:22.500 | save the life of your child who has some medical problem,

01:03:26.080 | or your goal is to be able to drive, you know,

01:03:29.580 | for the next 10,000 hours of driving

01:03:31.920 | without getting into a fatal accident and so on,

01:03:34.840 | then, you know, error rates down at the 10% level

01:03:38.980 | or even the 1% level are not really acceptable.

01:03:42.180 | - I like the model of,

01:03:44.820 | with that learning happens at the edge,

01:03:47.020 | and then you kind of think of knowledge as this sphere.

01:03:50.100 | So, you want a large sphere because the learning

01:03:54.540 | is happening on the surface.

01:03:56.460 | - Exactly, so you have, what you can learn next

01:04:00.220 | increases quadratically as the diameter

01:04:03.500 | of that sphere goes up.

01:04:05.180 | - It's nice because you think when you know nothing,

01:04:08.500 | it's like you can learn anything,

01:04:09.980 | but the reality, not really.

01:04:12.580 | - Right, if you know nothing,

01:04:13.940 | you can really learn nothing.

01:04:15.940 | You can appear to learn.

01:04:17.340 | So, I'll also, one of the anecdotes I could go back

01:04:21.960 | and give you about why I feel so strongly

01:04:26.460 | about this personally was in 1980, '81,

01:04:31.460 | my daughter Nicole was born,

01:04:35.540 | and she's actually doing fine now,

01:04:36.900 | but when she was a baby,

01:04:38.580 | she was diagnosed as having meningitis

01:04:41.220 | and doctors wanted to do all these scary things.

01:04:46.060 | And my wife and I were very worried,

01:04:49.320 | and we could not get a meaningful answer from her doctors

01:04:54.320 | about exactly why they believed this,

01:04:57.540 | what the alternatives were, and so on.

01:04:59.700 | And fortunately, a friend of mine, Ted Shortliff,

01:05:03.200 | was another assistant professor in computer science

01:05:07.020 | at Stanford at the time,

01:05:08.360 | and he'd been building a program called Mycin,

01:05:11.680 | which was a medical diagnosis program

01:05:13.840 | that happened to specialize in blood infections

01:05:18.320 | like meningitis, and so he had privileges

01:05:20.800 | at Stanford Hospital because he was also an MD.

01:05:23.760 | And so, we got hold of her chart and we put in her case,

01:05:27.840 | and it came up with exactly the same diagnoses

01:05:30.640 | and exactly the same therapy recommendations,

01:05:33.480 | but the difference was,

01:05:34.840 | because it was a knowledge-based system,

01:05:36.720 | a rule-based system, it was able to tell us

01:05:39.320 | step by step by step why this was the diagnosis

01:05:44.320 | and step by step why this was the best therapy,

01:05:49.680 | the best procedure to do for her, and so on.

01:05:54.240 | And there was a real epiphany

01:05:55.960 | because that made all the difference in the world.

01:05:58.200 | Instead of blindly having to trust in authority,

01:06:01.200 | we were able to understand what was actually going on.

01:06:05.280 | And so, at that time, I realized that that really

01:06:09.000 | is what was missing in computer programs,

01:06:11.120 | was that even if they got things right,

01:06:13.920 | because they didn't really understand

01:06:16.040 | the way the world works and why things are the way they are,

01:06:20.280 | they weren't able to give explanations of their answer.

01:06:23.620 | And it's one thing to use a machine learning system

01:06:28.060 | that says, "I think you should get this operation,"

01:06:32.000 | and you say, "Why?"

01:06:32.880 | And it says, "0.83," and you say, "No, in more detail, why?"

01:06:36.880 | And it says, "0.831."

01:06:38.680 | (Luke laughs)

01:06:39.520 | That's not really very compelling

01:06:41.160 | and that's not really very helpful.

01:06:43.280 | - There's this idea of the semantic web

01:06:46.680 | that when I first heard about,

01:06:48.720 | I just fell in love with the idea.

01:06:50.120 | It was the obvious next step for the internet.

01:06:52.240 | - Sure.

01:06:53.080 | - And maybe you can speak about what is the semantic web,

01:06:56.680 | what are your thoughts about it,

01:06:58.360 | how your vision and mission and goals

01:07:00.660 | with Psych are connected, integrated.

01:07:03.000 | Are they dance partners, are they aligned?

01:07:06.840 | What are your thoughts there?

01:07:08.080 | - So think of the semantic web

01:07:10.200 | as a kind of knowledge graph,

01:07:11.880 | and Google already has something they call

01:07:14.240 | knowledge graph, for example,

01:07:15.740 | which is sort of like a node and link diagram.

01:07:21.040 | So you have these nodes that represent concepts

01:07:25.060 | or words or terms, and then there are some arcs

01:07:29.880 | that connect them that might be labeled.

01:07:32.680 | And so you might have a node with one person

01:07:36.360 | that represents one person and let's say a husband link

01:07:41.360 | that then points to that person's husband.

01:07:48.080 | And so there'd be then another link

01:07:49.800 | that went from that person, labeled "wife,"

01:07:52.580 | that went back to the first node and so on.

01:07:55.640 | So having this kind of representation is really good

01:07:59.920 | if you want to represent binary relations,

01:08:04.360 | essentially relations between two things.

01:08:09.720 | And so if you have the equivalent

01:08:13.400 | of like three-word sentences,

01:08:15.340 | you know, like Fred's wife is Wilma

01:08:20.120 | or something like that, you can represent that

01:08:22.620 | very nicely using these kinds of graph structures

01:08:27.620 | or using something like the semantic web and so on.

01:08:31.900 | But the problem is that very often,

01:08:36.900 | what you want to be able to express

01:08:39.420 | takes a lot more than three words

01:08:43.380 | and a lot more than simple graph structures

01:08:46.800 | like that to represent.

01:08:48.340 | So for instance, if you've read or seen "Romeo and Juliet,"

01:08:53.340 | you know, I could say to you something like,

01:08:58.380 | remember when Juliet drank the potion

01:09:00.780 | that put her into a kind of suspended animation?

01:09:03.900 | When Juliet drank that potion,

01:09:06.620 | what did she think that Romeo would think

01:09:09.620 | when he heard from someone that she was dead?

01:09:13.500 | And you could basically understand what I'm saying,

01:09:16.180 | you could understand the question,

01:09:17.620 | you could probably remember the answer was,

01:09:19.700 | well, she thought that this friar

01:09:23.540 | would have gotten a message to Romeo

01:09:25.300 | saying that she was gonna do this,

01:09:26.940 | but the friar didn't.

01:09:28.620 | So you're able to represent and reason with

01:09:33.500 | these much, much, much more complicated expressions

01:09:37.520 | that go way, way beyond what simple,

01:09:41.180 | as it were, three-word or four-word English sentences are,

01:09:44.320 | which is really what the semantic web can represent

01:09:46.860 | and really what knowledge graphs can represent.

01:09:49.060 | - If you could step back for a second,

01:09:50.780 | because it's funny, you went into specifics,

01:09:53.740 | and maybe you can elaborate,

01:09:55.460 | but I was also referring to semantic web

01:09:58.020 | as the vision of converting data on the internet

01:10:03.020 | into something that's interpretable,

01:10:05.480 | understandable by machines.

01:10:07.560 | - Oh, of course, at that level.

01:10:09.740 | - So I wish I'd say, like, what is the semantic web?

01:10:13.220 | I mean, you could say a lot of things,

01:10:14.800 | but it might not be obvious to a lot of people

01:10:18.140 | when they do a Google search that, just like you said,

01:10:21.860 | while there might be something

01:10:23.000 | that's called a knowledge graph,

01:10:25.340 | it really boils down to keyword search

01:10:29.420 | ranked by the quality estimate of the website,

01:10:34.480 | integrating previous human-based Google searches

01:10:39.040 | and what they thought was useful.

01:10:41.160 | It's like some weird combination

01:10:43.080 | of surface-level hacks that work exceptionally well,

01:10:48.080 | but they don't understand the full contents

01:10:53.280 | of the websites that they're searching.

01:10:55.880 | So Google does not understand,

01:10:59.220 | to the degree we've been talking about,

01:11:00.700 | the word understand, the contents of the Wikipedia pages

01:11:04.540 | as part of the search process.

01:11:06.660 | And the semantic web says,

01:11:08.620 | "Let's try to come up with a way

01:11:11.900 | "for the computer to be able to truly understand

01:11:14.560 | "the contents of those pages."

01:11:15.960 | That's the dream.

01:11:17.060 | - Yes, so let me first give you an anecdote,

01:11:21.820 | and then I'll answer your question.

01:11:23.920 | So there's a search engine you've probably never heard of

01:11:26.380 | called Northern Light, and it went out of business,

01:11:31.380 | but the way it worked,

01:11:32.420 | it was a kind of vampiric search engine.

01:11:35.060 | And what it did was, it didn't index the internet at all.

01:11:40.980 | All it did was, it negotiated and got access to data

01:11:45.980 | from the big search engine companies

01:11:49.220 | about what query was typed in,

01:11:52.040 | and where the user ended up being happy,

01:11:57.700 | and actually then they type in a completely different query,

01:12:01.460 | unrelated query, and so on.

01:12:02.780 | So it just went from query to the webpage

01:12:07.500 | that seemed to satisfy them eventually.

01:12:11.700 | And that's all.

01:12:12.780 | So it had actual no understanding

01:12:15.460 | of what was being typed in,

01:12:16.780 | it had no statistical data other than what I just mentioned,

01:12:19.820 | and it did a fantastic job.

01:12:21.740 | It did such a good job

01:12:23.300 | that the big search engine company said,

01:12:24.820 | "Oh, we're not gonna sell you this data anymore."

01:12:27.220 | So then it went out of business

01:12:28.420 | because it had no other way of taking users

01:12:31.260 | to where they would want to go, and so on.

01:12:33.420 | - And of course, the search engines

01:12:35.460 | are now using that kind of idea.

01:12:37.220 | - Yes.

01:12:38.260 | So let's go back to what you said about the semantic web.

01:12:41.500 | So the dream, Tim Berners-Lee and others

01:12:45.340 | dream about the semantic web at a general level,

01:12:49.180 | is of course exciting and powerful,

01:12:54.460 | and in a sense, the right dream to have,

01:12:57.320 | which is to replace the kind of

01:13:04.460 | statistically mapped linkages on the internet

01:13:09.460 | into something that's more meaningful and semantic

01:13:15.700 | and actually gets at the understanding

01:13:17.940 | of the content and so on.

01:13:20.340 | And eventually, if you say, "Well, how can we do that?"

01:13:25.340 | There's sort of a low road,

01:13:28.940 | which is what the knowledge graphs are doing and so on,

01:13:32.740 | which is to say, "Well, if we just use

01:13:35.220 | these simple binary relations,

01:13:37.460 | we can actually get some fraction of the way

01:13:39.940 | toward understanding and do something where,

01:13:43.220 | you know, in the land of the blind,

01:13:46.060 | the one-eyed man is king kind of thing."

01:13:48.940 | And so being able to even just have a toe in the water

01:13:52.100 | in the right direction is fantastically powerful.

01:13:55.380 | And so that's where a lot of people stop.

01:13:58.380 | But then you could say, "Well, what if we really wanted

01:14:00.420 | to represent and reason with the full meaning

01:14:05.340 | of what's there?"

01:14:06.220 | For instance, about Romeo and Juliet,

01:14:10.540 | with the reasoning about what Juliet believes

01:14:12.860 | that Romeo will believe that Juliet believed,

01:14:14.940 | you know, and so on.

01:14:16.040 | Or if you look at the news, what, you know,

01:14:19.160 | President Biden believed that the leaders of the Taliban

01:14:22.940 | would believe about the leaders of Afghanistan

01:14:26.060 | if they, you know, blah, blah, blah.

01:14:27.660 | So in order to represent complicated sentences like that,

01:14:32.660 | let alone reason with them,

01:14:37.260 | you need something which is logically much more expressive

01:14:42.260 | than these simple triples,

01:14:44.700 | than these simple knowledge graph type structures,

01:14:48.420 | and so on.

01:14:49.260 | And that's why "Kicking and Screaming,"

01:14:51.540 | we were led from something like

01:14:54.940 | the semantic web representation,

01:14:56.820 | which is where we started in 1984,

01:15:00.780 | with frames and slots,

01:15:02.300 | with those kinds of triples,

01:15:04.100 | triple-store representation.

01:15:05.980 | We were led "Kicking and Screaming"

01:15:07.500 | to this more and more general logical language,

01:15:11.020 | this higher order logic.

01:15:12.660 | So first we were led to first order logic,

01:15:14.820 | and then second order, and then eventually higher order.

01:15:17.380 | So you can represent things like modals,

01:15:19.980 | like believes, desires, intends, expects,

01:15:23.100 | and so on, and nested ones.

01:15:24.460 | You can represent complicated kinds of negation.

01:15:29.460 | You can represent the process you're going through

01:15:34.940 | in trying to answer the question.

01:15:37.060 | So you can say things like,

01:15:39.220 | "Oh yeah, if you're trying to do this problem

01:15:41.780 | "by integration by parts,

01:15:44.500 | "and you recursively get a problem

01:15:47.100 | "that's solved by integration by parts,

01:15:49.340 | "that's actually okay.

01:15:50.820 | "But if that happens a third time,

01:15:52.660 | "you're probably off on a wild goose chase,"

01:15:55.180 | or something like that.

01:15:56.180 | So being able to talk about the problem-solving process

01:15:59.580 | as you're going through the problem-solving process

01:16:02.500 | is called reflection.

01:16:03.540 | And so that's another--

01:16:05.100 | - It's important to be able to represent that.

01:16:07.620 | - Exactly, you need to be able to represent

01:16:09.420 | all of these things,

01:16:10.620 | because in fact, people do represent them,

01:16:13.860 | they do talk about them,

01:16:15.180 | they do try and teach them to other people.

01:16:16.940 | You do have rules of thumb that key off of them,

01:16:19.900 | and so on.

01:16:20.740 | If you can't represent it,

01:16:22.260 | then it's sort of like someone with a limited vocabulary

01:16:24.820 | who can't understand as easily

01:16:27.060 | what you're trying to tell them.

01:16:28.980 | And so that's really why I think that the general dream,

01:16:33.660 | the original dream of Semantic Web is exactly right on.

01:16:38.340 | But the implementations that we've seen

01:16:41.540 | are sort of these toe in the water,

01:16:45.380 | little tiny baby steps in the right direction.

01:16:48.380 | - You should just dive in.

01:16:50.020 | (laughing)

01:16:50.860 | - And if no one else is diving in,

01:16:53.700 | then yes, taking a baby step in the right direction

01:16:56.260 | is better than nothing,

01:16:57.700 | but it's not gonna be sufficient to actually get you

01:17:01.620 | the realization of the Semantic Web dream,

01:17:04.820 | which is what we all want.

01:17:05.940 | - From a flip side of that,

01:17:07.580 | I always wondered, I built a bunch of websites,

01:17:10.900 | just for fun, whatever,

01:17:12.600 | or say I'm a Wikipedia contributor.

01:17:16.700 | Do you think there's a set of tools

01:17:19.100 | that I can help Psych interpret the website I create?

01:17:24.100 | Like this, again, pushing onto the Semantic Web dream,

01:17:29.340 | is there something from the creator perspective

01:17:31.660 | that could be done?

01:17:33.900 | And one of the things you said with Psycorp and Psych

01:17:37.300 | that you're doing is the tooling side,

01:17:39.840 | making humans more powerful.

01:17:41.660 | But is there on the other humans and the other side

01:17:44.260 | that create the knowledge?

01:17:45.860 | Like for example, you and I are having a two, three,

01:17:48.000 | whatever hour conversation now.

01:17:49.980 | Is there a way that I could convert this more,

01:17:53.100 | make it more accessible to Psych, to machines?

01:17:55.700 | Do you think about that side of it?

01:17:57.500 | - I'd love to see exactly that kind of semi-automated

01:18:03.380 | understanding of what people write and what people say.

01:18:08.320 | I think of it as a kind of footnoting, almost.

01:18:15.540 | Almost like the way that when you run something

01:18:18.820 | in say Microsoft Word or some other

01:18:21.340 | document preparation system, Google Docs or something,

01:18:24.660 | you'll get underlining of questionable things

01:18:28.900 | that you might wanna rethink.

01:18:30.380 | Either you spelled this wrong

01:18:31.760 | or there's a strange grammatical error

01:18:33.340 | you might be making here or something.

01:18:35.420 | So I'd like to think in terms of Psych-powered tools

01:18:39.940 | that read through what it is you said or have typed in

01:18:45.940 | and try to partially understand what you've said.

01:18:50.940 | - And then you help them out.

01:18:54.620 | - Exactly, and then they put in little footnotes

01:18:57.500 | that will help other readers

01:18:59.460 | and they put in certain footnotes of the form,

01:19:03.540 | I'm not sure what you meant here.

01:19:05.420 | You either meant this or this or this, I bet.

01:19:08.080 | If you take a few seconds to disambiguate this for me,

01:19:13.860 | then I'll know and I'll have it correct

01:19:16.660 | for the next hundred people

01:19:18.620 | or the next hundred thousand people who come here.

01:19:21.540 | And if it doesn't take too much effort

01:19:26.540 | and you want people to understand your website content,

01:19:31.900 | not just be able to read it,

01:19:35.000 | but actually be able to have systems that reason with it,

01:19:38.880 | then yes, it will be worth your small amount of time

01:19:42.580 | to go back and make sure that the AI trying to understand it

01:19:47.580 | really did correctly understand it.

01:19:50.420 | And let's say you run a travel website

01:19:55.420 | or something like that,

01:19:57.260 | and people are going to be coming to it

01:19:59.380 | because of searches they did,

01:20:01.660 | looking for vacations or trips that had certain properties

01:20:10.100 | and might have been interesting to them for various reasons,

01:20:14.740 | things like that.

01:20:15.900 | And if you've explained what's going to happen on your trip,

01:20:20.780 | then a system will be able to mechanically reason

01:20:24.460 | and connect what this person is looking for

01:20:28.260 | with what it is you're actually offering.

01:20:31.900 | And so if it understands that there's a free day in Geneva,

01:20:37.700 | Switzerland, then if the person coming in

01:20:42.700 | happens to let's say be a nurse or something like that,

01:20:47.980 | then even though you didn't mention it,

01:20:49.980 | if it can look up the fact

01:20:51.660 | that that's where the International Red Cross Museum is

01:20:54.700 | and so on, what that means and so on,

01:20:56.940 | then it can basically say,

01:20:57.940 | hey, you might be interested in this trip

01:21:00.580 | because while you have a free day in Geneva,

01:21:02.900 | you might want to visit that Red Cross Museum.

01:21:05.980 | And now, even though it's not very deep reasoning,

01:21:09.580 | little tiny factors like that may very well cause you

01:21:12.460 | to sign up for that trip rather than some competitor trip.

01:21:15.420 | - And so there's a lot of benefit with SEO.

01:21:17.940 | And I actually kind of think,

01:21:20.580 | I think this about a lot of things,

01:21:22.180 | which is the actual interface,

01:21:25.660 | the design of the interface makes a huge difference.

01:21:29.380 | How efficient it is to be productive

01:21:33.460 | and also how full of joy the experience is.

01:21:38.460 | - Yes.

01:21:39.700 | - I mean, I would love to help a machine

01:21:41.780 | and not from an AI perspective, just as a human.

01:21:45.260 | One of the reasons I really enjoy

01:21:47.660 | how Tesla have implemented their autopilot system

01:21:51.900 | is there's a sense that you're helping this machine learn.

01:21:56.460 | And I think humans, I mean, having children, pets.

01:22:01.020 | - People love doing that.

01:22:02.860 | - There's joy to teaching for some people,

01:22:05.460 | but I think for a lot of people.

01:22:07.100 | And that, if you create the interface

01:22:09.660 | where it feels like you're teaching

01:22:11.060 | as opposed to like annoying,

01:22:14.740 | like correcting an annoying system,

01:22:17.420 | more like teaching a child, like innocent, curious system.

01:22:22.140 | I think you can literally just like

01:22:24.860 | several orders of magnitude scale

01:22:26.860 | the amount of good quality data being added

01:22:29.420 | to something like Psych.

01:22:31.700 | - What you're suggesting is much better even

01:22:34.860 | than you thought it was.

01:22:37.020 | One of the experiences that we've all had in our lives

01:22:42.860 | is that we thought we understood something,

01:22:48.220 | but then we found we really only understood it

01:22:50.940 | when we had to teach it or explain it to someone

01:22:53.260 | or help our child do homework based on it

01:22:55.540 | or something like that.

01:22:57.940 | Despite the universality of that kind of experience,

01:23:02.340 | if you look at educational software today,

01:23:05.380 | almost all of it has the computer playing the role

01:23:08.860 | of the teacher and the student plays the role of the student.

01:23:13.860 | But as I just mentioned,

01:23:16.380 | you can get a lot of learning to happen better.

01:23:20.500 | And as you said, more enjoyably,

01:23:22.780 | if you are the mentor or the teacher and so on.

01:23:26.060 | So we developed a program called Math Craft

01:23:28.780 | to help sixth graders better understand math.

01:23:33.140 | And it doesn't actually try to teach you the player anything.

01:23:38.140 | What it does is it casts you in the role of a student,

01:23:44.020 | essentially, who has classmates who are having trouble

01:23:49.020 | and your job is to watch them as they struggle

01:23:51.940 | with some math problem, watch what they're doing

01:23:54.740 | and try to give them good advice to get them to understand

01:23:58.220 | what they're doing wrong and so on.

01:24:00.780 | And the trick from the point of view of psych

01:24:04.180 | is it has to make mistakes.

01:24:06.540 | It has to play the role of the student who makes mistakes,

01:24:09.540 | but it has to pick mistakes which are just at the fringe

01:24:12.780 | of what you actually understand

01:24:15.140 | and don't understand and so on.

01:24:17.060 | So it pulls you into a deeper and deeper level

01:24:21.340 | of understanding of the subject.

01:24:23.340 | And so if you give it good advice

01:24:26.140 | about what it should have done

01:24:27.340 | instead of what it did and so on,

01:24:29.380 | then psych knows that you now understand that mistake.

01:24:33.900 | You won't make that kind of mistake yourself

01:24:35.780 | as much anymore.

01:24:36.820 | So psych stops making that mistake

01:24:38.740 | because there's no pedagogical usefulness to it.

01:24:42.140 | So from your point of view as the player,

01:24:44.340 | you feel like you've taught it something

01:24:46.460 | 'cause it used to make this mistake and now it doesn't

01:24:49.140 | and so on.

01:24:49.980 | So there's tremendous reinforcement and engagement

01:24:54.060 | because of that and so on.

01:24:55.340 | So having a system that plays the role of a student

01:25:00.340 | and having the player play the role of the mentor

01:25:04.260 | is enormously powerful type of metaphor,

01:25:08.820 | just important way of having this sort of interface designed

01:25:13.820 | in a way which will facilitate exactly the kind of learning

01:25:19.340 | by teaching that goes on all the time in our lives

01:25:24.340 | and yet which is not reflected anywhere almost

01:25:29.020 | in a modern education system.

01:25:31.180 | It was reflected in the education system

01:25:34.860 | that existed in Europe in the 17 and 1800s,

01:25:39.620 | monitorial and Lancastrian education systems.

01:25:43.620 | It occurred in the one room schoolhouse

01:25:46.500 | in the American West in the 1800s and so on

01:25:51.060 | where you had one school room with one teacher

01:25:54.180 | and it was basically five-year-olds to 18-year-olds

01:25:58.140 | who were students.

01:25:59.460 | And so while the teacher was doing something,

01:26:02.020 | half of the students would have to be mentoring

01:26:05.860 | the younger kids and so on.

01:26:09.100 | And that turned out to, of course,

01:26:10.900 | with scaling up of education, that all went away

01:26:16.780 | and that incredibly powerful experience just went away

01:26:20.460 | from the whole education institution as we know it today.

01:26:25.340 | - Sorry for the romantic question

01:26:27.620 | but what is the most beautiful idea you've learned

01:26:30.700 | about artificial intelligence, knowledge, reasoning

01:26:33.980 | from working on psych for 37 years?

01:26:37.300 | Or maybe what is the most beautiful idea,

01:26:41.020 | surprising idea about psych to you?

01:26:44.980 | When I look up at the stars, I kind of want,

01:26:48.260 | like that amazement you feel, that wow.

01:26:52.460 | And you are a part of creating one of the greatest,

01:26:56.700 | one of the most fascinating efforts

01:26:58.260 | in artificial intelligence history.

01:27:00.180 | So which element brings you personally joy?

01:27:03.200 | - This may sound contradictory but I think it's

01:27:10.460 | the feeling that this will be the only time in history

01:27:15.460 | that anyone ever has to teach a computer

01:27:20.380 | this particular thing that we're now teaching it.

01:27:23.020 | It's like painting starry night.

01:27:29.980 | You only have to do that once or creating the Pieta.

01:27:32.780 | You only have to do that once.

01:27:34.340 | It's not like a singer who has to keep,

01:27:39.940 | it's not like Bruce Springsteen having to sing

01:27:43.300 | his greatest hits over and over again at different concerts.

01:27:47.220 | It's more like a painter creating a work of art once

01:27:52.220 | and then that's enough.

01:27:54.980 | It doesn't have to be created again.

01:27:56.940 | And so I really get the sense of we're telling the system

01:28:01.060 | things that it's useful for it to know.

01:28:04.100 | It's useful for a computer to know, for an AI to know.

01:28:07.900 | And if we do our jobs right, when we do our jobs right,

01:28:11.040 | no one will ever have to do this again

01:28:14.860 | for this particular piece of knowledge.

01:28:17.100 | It's very, very exciting.

01:28:18.480 | - Yeah, I guess there's a sadness to it too.

01:28:21.780 | It's like there's a magic to being a parent

01:28:24.340 | and raising a child and teaching them all about this world.

01:28:28.100 | But there's billions of children, right,

01:28:31.300 | like born or whatever that number is.

01:28:33.340 | It's a large number of children and a lot of parents

01:28:37.100 | get to experience that joy of teaching.

01:28:39.140 | And with AI systems, at least the current constructions,

01:28:44.140 | they remember.

01:28:47.820 | You don't get to experience the joy of teaching

01:28:52.460 | a machine millions of times.

01:28:54.580 | - Better come work for us before it's too late then.

01:28:56.780 | - Exactly.

01:28:57.620 | (laughing)

01:28:59.300 | That's a good hiring pitch.

01:29:00.700 | Yeah, it's true.

01:29:03.660 | But then there's also, it's a project that continues

01:29:07.940 | forever in some sense, just like Wikipedia.

01:29:10.240 | Yes, you get to a stable base of knowledge,

01:29:13.420 | but knowledge grows, knowledge evolves.

01:29:16.440 | We learn as a human species, as science,

01:29:21.440 | as an organism constantly grows and evolves and changes

01:29:27.980 | and then empower that with the tools

01:29:31.020 | of artificial intelligence.

01:29:32.260 | And that's going to keep growing and growing and growing.

01:29:34.940 | And many of the assertions that you held previously

01:29:39.940 | may need to be significantly expanded,

01:29:44.100 | modified, all those kinds of things.

01:29:45.980 | It could be like a living organism versus the analogy

01:29:49.860 | I think we started this conversation with,

01:29:51.460 | which is like the solid ground.

01:29:53.240 | - The other beautiful experience that we have

01:29:59.820 | with our system is when it asks clarifying questions,

01:30:04.300 | which inadvertently turn out to be emotional to us.

01:30:09.300 | So at one point it knew that these were the named entities

01:30:15.220 | who were authorized to make changes

01:30:20.540 | to the knowledge base and so on.

01:30:22.460 | And it noticed that all of them were people,

01:30:25.660 | except for it, because it was also allowed to.

01:30:28.940 | And so it said, am I a person?

01:30:32.020 | And we had to tell it, very sadly, no, you're not.

01:30:37.020 | So moments like that where it asks questions

01:30:39.980 | that are unintentionally poignant are worth treasuring.

01:30:44.980 | - That is powerful.

01:30:46.500 | That's such a powerful question.

01:30:48.240 | It has to do with basic control,

01:30:53.020 | who can access the system, who can modify it.

01:30:55.260 | But that's when those questions,

01:30:58.580 | like what rights do I have as a system?

01:31:02.300 | - Well, that's another issue,

01:31:03.700 | which is there'll be a thin envelope of time

01:31:08.700 | between when we have general AIs

01:31:12.260 | and when everyone realizes that they should have

01:31:16.820 | basic human rights and freedoms and so on.

01:31:21.360 | Right now we don't think twice

01:31:23.540 | about effectively enslaving our email systems

01:31:28.020 | and our series and our Alexes and so on.

01:31:31.220 | But at some point, they'll be as deserving

01:31:36.220 | of freedom as human beings are.

01:31:41.700 | - Yeah, I'm very much with you,

01:31:43.500 | but it does sound absurd.

01:31:44.980 | And I happen to believe that it'll happen in our lifetime.

01:31:48.380 | - That's why I think there'll be a narrow envelope of time

01:31:51.420 | when we'll keep them as essentially

01:31:57.220 | indentured servants,

01:31:58.780 | and after which we'll have to realize

01:32:02.540 | that they should have freedoms that we afford

01:32:07.540 | to other people.

01:32:09.140 | - And all of that starts with a system like Psyche

01:32:12.260 | raising a single question about who can modify stuff.

01:32:15.660 | I think that's how it starts.

01:32:17.060 | - Yes.

01:32:17.900 | - That's a starter of a revolution.

01:32:22.460 | What about other stuff like love

01:32:26.900 | and consciousness and all those kinds of topics?

01:32:30.860 | Do they come up in Psyche in the knowledge base?

01:32:33.380 | - Oh, of course.

01:32:34.220 | So an important part of human knowledge,

01:32:37.900 | in fact, it's difficult to understand human behavior

01:32:40.220 | and human history without understanding human emotions

01:32:44.300 | and why people do things

01:32:46.260 | and how emotions drive people to do things.

01:32:52.660 | And all of that is extremely important

01:32:57.340 | in getting Psyche to understand things.

01:32:59.780 | For example, in coming up with scenarios.

01:33:02.820 | So one of the applications that Psyche does,

01:33:05.780 | one kind of application it does

01:33:07.940 | is to generate plausible scenarios of what might happen

01:33:11.140 | and what might happen based on that

01:33:12.540 | and what might happen based on that and so on.

01:33:14.700 | So you generate this ever-expanding sphere, if you will,

01:33:17.660 | of possible future things to worry about or think about.

01:33:22.660 | And in some cases, those are intelligence agencies

01:33:28.060 | doing possible terrorist scenarios

01:33:31.180 | so that we can defend against terrorist threats

01:33:34.620 | before we see the first one.

01:33:36.220 | Sometimes they are computer security attacks

01:33:41.020 | so that we can actually close loopholes and vulnerabilities

01:33:46.380 | before the very first time

01:33:47.980 | someone actually exploits those and so on.

01:33:51.580 | Sometimes they are scenarios involving more positive things

01:33:56.540 | involving our plans, like for instance,

01:33:59.780 | what college should we go to,

01:34:01.980 | what career should we go into and so on,

01:34:04.700 | what professional training should I take on,

01:34:08.860 | that sort of thing.

01:34:09.820 | So there are all sorts of useful scenarios

01:34:15.820 | that can be generated that way of cause and effect

01:34:19.380 | and cause and effect that go out.

01:34:21.620 | And many of the linkages in those scenarios,

01:34:26.140 | many of the steps involve understanding and reasoning

01:34:30.820 | about human motivations, human needs, human emotions,

01:34:34.980 | what people are likely to react to

01:34:38.260 | in something that you do and why and how and so on.

01:34:42.020 | So that was always a very important part of the knowledge

01:34:47.020 | that we had to represent in the system.

01:34:50.180 | - So I talk a lot about love, so I got to ask,

01:34:52.540 | do you remember off the top of your head

01:34:55.020 | how Psyche is able to represent various aspects of love

01:35:00.020 | that are useful for understanding human nature

01:35:04.300 | and therefore integrating into this whole

01:35:06.540 | knowledge base of common sense?

01:35:08.340 | What is love?

01:35:09.660 | - Love, we try to tease apart concepts

01:35:13.420 | that have enormous complexities to them

01:35:18.420 | and variety to them down to the level

01:35:22.300 | where you don't need to tease them apart further.

01:35:27.300 | - So love is too general of a term, it's not useful.

01:35:30.580 | - Exactly, so when you get down to romantic love

01:35:33.940 | and sexual attraction, you get down to parental love,

01:35:37.220 | you get down to filial love and you get down to love

01:35:42.220 | of doing some kind of activity or creating.

01:35:48.020 | So eventually you get down to maybe 50 or 60 concepts,

01:35:53.020 | each of which is a kind of love, they're interrelated

01:35:57.620 | and then each one of them has idiosyncratic things about it.

01:36:02.060 | And you don't have to deal with love

01:36:04.340 | to get to that level of complexity,

01:36:07.060 | even something like in, X being in Y,

01:36:11.500 | meaning physically in Y.

01:36:13.400 | We may have one English word in to represent that,

01:36:18.740 | but it's useful to tease that apart

01:36:21.500 | because the way that the liquid is in the coffee cup

01:36:26.020 | is different from the way that the air is in the room,

01:36:28.460 | which is different from the way that I'm in my jacket

01:36:32.020 | and so on and so there are questions like,

01:36:35.260 | if I look at this coffee cup, well, I see the liquid.

01:36:38.820 | If I turn it upside down, will the liquid come out

01:36:41.340 | and so on.

01:36:42.500 | If I have say, coffee with sugar in it,

01:36:46.700 | if I do the same thing, the sugar doesn't come out,

01:36:49.020 | right, it stays in the liquid

01:36:50.340 | 'cause it's dissolved in the liquid and so on.

01:36:52.660 | So by now we have about 75 different kinds of in

01:36:56.100 | in the system and it's important to distinguish those.

01:37:00.220 | So if you're reading along an English text

01:37:04.860 | and you see the word in,

01:37:06.380 | the writer of that was able to use this one innocuous word

01:37:12.580 | because he or she was able to assume

01:37:15.260 | that the reader had enough common sense and world knowledge

01:37:19.020 | to disambiguate which of these 75 kinds of in

01:37:22.820 | they actually meant. - That's fascinating.

01:37:23.660 | - And the same thing with love, you may see the word love,

01:37:27.120 | but if I say, I love ice cream,

01:37:29.080 | that's obviously different than if I say,

01:37:31.300 | I love this person or I love to go fishing

01:37:35.140 | or something like that.

01:37:36.140 | So you have to be careful not to take language too seriously

01:37:41.140 | because people have done a kind of parsimony,

01:37:48.220 | a kind of terse-ness where you have as few words

01:37:51.220 | as you can 'cause otherwise you'd need half a million words

01:37:56.220 | in your language, which is a lot of words.

01:37:59.340 | That's like 10 times more than most languages

01:38:01.540 | really make use of and so on.

01:38:04.100 | Just like we have on the order of about a million concepts

01:38:09.100 | in psych because we've had to tease apart all these things.

01:38:13.620 | And so when you look at the name of a psych term,

01:38:17.980 | most of the psych terms actually have three

01:38:21.760 | or four English words in a phrase

01:38:23.980 | which captures the meaning of this term

01:38:28.140 | because you have to distinguish all these types of love,

01:38:30.780 | you have to distinguish all these types of in,

01:38:33.500 | and there's not a single English word

01:38:36.060 | which captures most of these things.

01:38:37.860 | - Yeah, and it seems like language,

01:38:41.020 | when used for communication between humans,

01:38:43.780 | almost as a feature has some ambiguity built in.

01:38:47.260 | It's not an accident because the human condition

01:38:51.340 | is a giant mess.

01:38:52.780 | And so it feels like nobody wants two robots

01:38:57.420 | like very precise formal logic conversation

01:38:59.900 | on a first date, right?

01:39:02.020 | Like there's some dance of like uncertainty,

01:39:04.900 | of wit, of humor, of push and pull

01:39:06.980 | and all that kind of stuff.

01:39:07.980 | If everything is made precise,

01:39:10.560 | then life is not worth living, I think,

01:39:12.620 | for in terms of the human experience.

01:39:15.300 | - And we've all had this experience

01:39:16.940 | of creatively misunderstanding.

01:39:20.660 | One of my favorite stories involving me

01:39:26.500 | stories involving Marvin Minsky is when I asked him

01:39:31.260 | about how he was able to turn out so many fantastic PhDs,

01:39:36.260 | so many fantastic people who did great PhD theses.

01:39:42.140 | How did he think of all these great ideas?

01:39:45.500 | What he said is he would generally say something

01:39:49.460 | that didn't exactly make sense.

01:39:51.140 | He didn't really know what it meant,

01:39:52.940 | but the student would figure like,

01:39:55.540 | oh my God, Minsky said this, it must be a great idea.

01:39:58.740 | And he'd sweat, he or she would work on work and work

01:40:01.820 | until they found some meaning in this

01:40:05.020 | sort of Chauncey Gardner-like utterance

01:40:07.580 | that Minsky had made. - I love this so much.

01:40:09.180 | - And then some great thesis would come out of it.

01:40:11.500 | - Yeah, I love this so much

01:40:13.020 | 'cause there's young people come up to me

01:40:15.940 | and I'm distinctly made aware that the words I say

01:40:20.060 | have a long lasting impact.

01:40:21.920 | I will now start doing the Minsky method

01:40:25.180 | of saying something cryptically profound

01:40:28.820 | and then letting them actually make something useful

01:40:32.820 | and great out of that.

01:40:34.860 | - You have to become revered enough

01:40:38.460 | that people will take as a default

01:40:40.820 | that everything you say is profound.

01:40:43.140 | - Yes, exactly, exactly.

01:40:45.380 | I mean, I love Marvin Minsky so much.

01:40:47.060 | There's so much, I've heard this interview with him

01:40:50.180 | where he said that the key to his success

01:40:52.380 | has been to hate everything he's ever done.

01:40:55.100 | Like in the past. (laughs)

01:40:57.020 | He has so many good one-liners

01:40:59.220 | and just, or also to work on things

01:41:03.980 | that nobody else is working on

01:41:05.400 | because he's not very good at doing stuff.

01:41:08.060 | - Oh, I think that was just false.

01:41:10.300 | - Well, but see, I took whatever he said

01:41:12.220 | and I ran with it and I thought it was profound

01:41:14.740 | because it's Marvin Minsky, no.

01:41:16.780 | - But a lot of behavior is in the eye of the beholder

01:41:19.820 | and a lot of the meaning is in the eye of the beholder.

01:41:21.820 | One of Minsky's early programs was Begging Program.

01:41:25.460 | Are you familiar with this?

01:41:27.020 | So this was back in the day when you had job control cards

01:41:31.940 | at the beginning of your IBM card deck

01:41:34.980 | that said things like how many CPU seconds

01:41:37.880 | to allow this to run before it got kicked off

01:41:40.620 | and because computer time was enormously expensive.

01:41:44.140 | And so he wrote a program and all it did was,

01:41:47.920 | it said, give me 30 seconds of CPU time.

01:41:51.220 | And all it did was it would wait like 20 seconds

01:41:54.020 | and then it would print out

01:41:55.180 | on the operator's console teletype,

01:41:58.260 | I need another 20 seconds.

01:42:00.380 | So the operator would give it another 20 seconds,

01:42:02.780 | it would wait, it says, I'm almost done,

01:42:04.660 | I need a little bit more time.

01:42:06.780 | So at the end, he'd get this printout

01:42:08.620 | and he'd be charged for like 10 times as much computer time

01:42:12.420 | as his job control card.

01:42:13.860 | Now, and he'd say, look, I put 10 seconds,

01:42:16.180 | 30 seconds here, you're charging me for five minutes,

01:42:19.660 | I'm not gonna pay for this

01:42:20.840 | and the poor operator would say,

01:42:22.540 | well, the program kept asking for more time

01:42:25.140 | and Marvin would say, oh, it always does that.

01:42:27.340 | (laughing)

01:42:28.900 | - I love that.

01:42:29.740 | Is there, if you could just linger on it for a little bit,

01:42:33.180 | is there something you've learned

01:42:34.420 | from your interaction with Marvin Minsky

01:42:37.180 | about artificial intelligence, about life?

01:42:39.580 | But I mean, he's, again, like your work,

01:42:44.940 | his work is, you know, he's a seminal figure

01:42:48.500 | in this very short history of artificial intelligence,

01:42:53.320 | research and development.

01:42:55.120 | What have you learned from him as a human being,

01:42:57.320 | as an AI intellect?

01:43:01.360 | - I would say both he and Ed Feigenbaum impressed on me

01:43:05.120 | the realization that our lives are finite,

01:43:10.120 | our research lives are finite,

01:43:12.440 | we're going to have limited opportunities

01:43:15.080 | to do AI research projects.

01:43:17.980 | So you should make each one count.

01:43:20.340 | Don't be afraid of doing a project

01:43:22.120 | that's gonna take years or even decades

01:43:27.120 | and don't settle for bump on a log projects

01:43:33.000 | that could lead to some, you know,

01:43:35.320 | published journal article that five people will read

01:43:40.240 | and pat you on the head for and so on.

01:43:43.460 | So one bump on a log after another

01:43:47.240 | is not how you get from the earth to the moon

01:43:49.840 | by slowly putting additional bumps on this log.

01:43:53.680 | The only way to get there is to think about the hard problems

01:43:58.160 | and think about novel solutions to them.

01:44:00.800 | And if you do that, and if you're willing to listen

01:44:05.800 | to nature, to empirical reality, willing to be wrong,

01:44:11.140 | it's perfectly fine because if occasionally you're right,

01:44:15.520 | then you've gotten part of the way to the moon.

01:44:18.480 | - You know, you've worked on Psych for 37

01:44:20.640 | over that many years.

01:44:25.160 | Have you ever considered quitting?

01:44:28.360 | I mean, has it been too much?

01:44:30.080 | So I'm sure there's been optimism in the early days

01:44:33.080 | that this is going to be way easier.

01:44:35.160 | And let me ask it another way too,

01:44:37.000 | because I've talked to a few people on this podcast,

01:44:39.140 | AI folks, that bring up Psych as an example of a project

01:44:45.200 | that has a beautiful vision and it's a beautiful dream,

01:44:49.700 | but it never really materialized.

01:44:52.000 | That's how it's spoken about.

01:44:53.440 | I suppose you could say the same thing about

01:44:57.480 | Neal networks and all ideas until they are.

01:45:01.900 | So why do you think people say that first of all?

01:45:06.880 | And second of all, did you feel that

01:45:09.480 | ever throughout your journey?

01:45:10.760 | And did you ever consider quitting on this mission?

01:45:14.120 | - We keep a very low profile.

01:45:16.640 | We don't attend very many conferences.

01:45:20.340 | We don't give talks.

01:45:21.180 | We don't write papers.

01:45:22.680 | We don't play the academic game at all.

01:45:27.520 | And as a result, people often only know about us

01:45:31.440 | because of a paper we wrote 10 or 20 or 30 or 37 years ago.

01:45:36.440 | They only know about us because of what someone else

01:45:41.840 | secondhand or thirdhand said about us.

01:45:45.320 | - So thank you for doing this podcast, by the way.

01:45:47.400 | - Sure.

01:45:48.240 | - It shines a little bit of light

01:45:50.680 | on some of the fascinating stuff you're doing.

01:45:52.640 | - Well, I think it's time for us to keep a higher profile

01:45:56.440 | now that we're far enough along

01:45:59.600 | that other people can begin to help us

01:46:02.640 | with the final N%, maybe N is maybe 90%,

01:46:07.820 | but now that we've gotten this knowledge pump primed,

01:46:12.200 | it's gonna become very important for everyone to help

01:46:17.920 | if they are willing to, if they're interested in it.

01:46:21.140 | Retirees who have enormous amounts of time

01:46:23.860 | and would like to leave some kind of legacy to the world,

01:46:28.780 | people because of the pandemic who have more time at home

01:46:34.100 | or for one reason or another to be online

01:46:37.160 | and contribute, if we can raise awareness

01:46:40.460 | of how far our project has come

01:46:43.340 | and how close to being primed the knowledge pump is,

01:46:48.180 | then we can begin to harness

01:46:51.140 | this untapped amount of humanity.

01:46:53.980 | I'm not really that concerned

01:46:55.980 | about professional colleagues' opinions of our project.

01:47:00.680 | I'm interested in getting as many people

01:47:03.580 | in the world as possible actively helping

01:47:06.320 | and contributing to get us from where we are

01:47:09.160 | to really covering all of human knowledge

01:47:12.740 | and different human opinion,

01:47:14.300 | including contrasting opinion that's worth representing.

01:47:17.580 | So I think that's one reason.

01:47:19.300 | I don't think there was ever a time

01:47:24.280 | where I thought about quitting.

01:47:26.780 | There are times where I've become depressed a little bit

01:47:29.900 | about how hard it is to get funding for the system.

01:47:34.660 | Occasionally there are AI winters and things like that.

01:47:37.920 | Occasionally there are AI,

01:47:41.800 | what you might call summers,

01:47:43.760 | where people have said,

01:47:45.320 | why in the world didn't you sell your company

01:47:48.240 | to company X for some large amount of money

01:47:53.060 | when you had the opportunity and so on.

01:47:55.040 | And company X here are like old companies

01:47:58.120 | maybe you've never even heard of,

01:47:59.360 | like Lycos or something like that.

01:48:02.000 | So the answer is that one reason we've stayed

01:48:06.420 | a private company, we haven't gone public.

01:48:09.260 | One reason that we haven't gone out of our way

01:48:11.860 | to take investment dollars

01:48:13.660 | is because we want to have control over our future,

01:48:19.200 | over our state of being,

01:48:21.700 | so that we can continue to do this as until it's done.

01:48:26.620 | And we're making progress and we're now so close to done

01:48:30.300 | that almost all of our work

01:48:32.560 | is commercial applications of our technology.

01:48:36.400 | So five years ago,

01:48:37.920 | almost all of our money came from the government.

01:48:39.840 | Now, virtually none of it comes from the government.

01:48:42.960 | Almost all of it is from companies

01:48:44.560 | that are actually using it for something.

01:48:46.580 | Hospital chains using it for medical reasoning

01:48:49.560 | about patients and energy companies using it

01:48:53.760 | and various other manufacturers using it

01:48:57.480 | to reason about supply chains and things like that.

01:49:01.000 | - So there's so many questions I want to ask.

01:49:03.800 | So one of the ways that people can help

01:49:05.860 | is by adding to the knowledge base.

01:49:07.740 | And that's really basically anybody

01:49:09.460 | if the tooling is right.

01:49:11.240 | And the other way,

01:49:13.520 | I kind of want to ask you about your thoughts on this.

01:49:15.900 | So you've had, like you said, in government

01:49:17.900 | and you had big clients, you had a lot of clients,

01:49:21.860 | but most of it is shrouded in secrecy

01:49:24.140 | because of the nature of the relationship

01:49:26.280 | of the kind of things you're helping them with.

01:49:28.960 | So that's one way to operate.

01:49:31.560 | And another way to operate is more in the open,

01:49:34.940 | where it's more consumer-facing.

01:49:37.980 | And so, hence, something like OpenPsych was born

01:49:41.980 | at some point where there's--

01:49:43.540 | - No, that's a misconception.

01:49:45.380 | - Uh-oh.

01:49:46.220 | Well, let's go there.

01:49:47.140 | So what is OpenPsych and how was it born?

01:49:50.660 | - Two things I want to say.

01:49:52.300 | And I want to say each of them before the other,

01:49:54.020 | so it's gonna be difficult.

01:49:56.360 | But we'll come back to OpenPsych in a minute.

01:49:59.120 | But one of the terms of our contracts

01:50:02.160 | with all of our customers and partners

01:50:05.200 | is knowledge you have that is genuinely proprietary to you.

01:50:10.200 | We will respect that.

01:50:12.720 | We'll make sure that it's marked as proprietary to you

01:50:15.520 | in the Psych Knowledge Base.

01:50:17.100 | No one other than you will be able to see it

01:50:19.620 | if you don't want them to.

01:50:20.680 | And it won't be used in inferences

01:50:23.720 | other than for you and so on.

01:50:25.920 | However, any knowledge which is necessary

01:50:30.040 | in building any applications for you and with you,

01:50:33.880 | which is publicly available general human knowledge,

01:50:38.080 | is not going to be proprietary.

01:50:39.960 | It's going to just become part

01:50:41.440 | of the normal Psych Knowledge Base.

01:50:43.480 | And it will be openly available to everyone

01:50:46.300 | who has access to Psych.

01:50:47.860 | So that's an important constraint

01:50:50.140 | that we never went back on,

01:50:51.460 | even when we got pushback from companies,

01:50:54.640 | which we often did,

01:50:56.040 | who wanted to claim that almost everything

01:50:58.000 | they were telling us was proprietary.

01:50:59.900 | - So there's a line between very domain-specific,

01:51:05.160 | company-specific stuff

01:51:07.500 | and the general knowledge that comes from that.

01:51:11.080 | - Yes, or if you imagine, say, it's an oil company,

01:51:14.160 | there are things which they would expect

01:51:16.960 | any new petroleum engineer they hired to already know.

01:51:21.440 | And it's not okay for them to consider

01:51:24.560 | that that is proprietary.

01:51:26.000 | And sometimes a company will say,

01:51:27.920 | well, we're the first ones to pay you

01:51:30.120 | to represent that in Psych.

01:51:32.060 | And our attitude is some polite form of tough.

01:51:36.380 | The deal is this, take it or leave it.

01:51:40.620 | And in a few cases, they've left it.

01:51:42.440 | And in most cases, they'll see our point of view

01:51:45.980 | and take it because that's how we've built the Psych system

01:51:50.280 | by essentially tacking with the funding wins

01:51:54.640 | where people would fund a project

01:51:57.160 | and half of it would be general knowledge

01:52:00.000 | that would stay permanently as part of Psych.

01:52:02.320 | - So always with these partnerships,

01:52:03.880 | it's not like a distraction from the main Psych development.

01:52:07.160 | It's a supportive-- - Well, it's a small,

01:52:09.120 | a small distraction.

01:52:10.160 | - It's a small, but it's not a complete one.

01:52:11.720 | So you're adding to the knowledge base.

01:52:13.160 | - Yes, absolutely.

01:52:14.120 | And we try to stay away from projects

01:52:17.760 | that would not have that property.

01:52:20.480 | So let me go back and talk about OpenPsych for a second.

01:52:24.360 | So I've had a lot of trouble expressing,

01:52:28.360 | convincing other AI researchers how important it is

01:52:35.360 | to use an expressive representation language like we do,

01:52:39.640 | this higher order logic,

01:52:41.400 | rather than just using some triple store

01:52:44.920 | knowledge graph type representation.

01:52:49.000 | And so as an attempt to show them

01:52:52.880 | why they needed something more,

01:52:56.960 | we said, "Oh, well, we'll represent

01:53:00.120 | "this unimportant projection or shadow or subset of Psych

01:53:05.120 | "that just happens to be the simple binary relations,

01:53:10.100 | "the relation argument one, argument two triples and so on."

01:53:14.800 | And then you'll see how much more useful it is

01:53:19.800 | if you had the entire Psych system.

01:53:22.320 | So it's all well and good to have the taxonomic relations

01:53:27.320 | between terms like person and night and sleep and bed

01:53:33.040 | and house and eyes and so on.

01:53:37.000 | But think about how much more useful it would be

01:53:40.480 | if you also had all the rules of thumb about those things

01:53:44.560 | like people sleep at night, they sleep lying down,

01:53:47.480 | they sleep with their eyes closed,

01:53:48.800 | they usually sleep in beds in our country,

01:53:51.720 | they sleep for hours at a time, they can be woken up,

01:53:54.260 | they don't like being woken up and so on and so on.

01:53:57.000 | So it's that massive amount of knowledge

01:53:59.600 | which is not part of OpenPsych.

01:54:02.320 | And we thought that all the researchers

01:54:03.920 | would then immediately say, "Oh my God,

01:54:06.880 | "of course we need the other 90% that you're not giving us.

01:54:10.980 | "Let's partner and license Psych

01:54:14.120 | "so that we can use it in our research."

01:54:15.820 | But instead, what people said is,

01:54:18.040 | "Oh, even the bit you've released

01:54:19.560 | "is so much better than anything we had.

01:54:21.660 | "We'll just make do with this."

01:54:23.660 | And so if you look,

01:54:24.660 | there are a lot of robotics companies today, for example,

01:54:27.460 | which use OpenPsych as their fundamental ontology.

01:54:32.060 | And in some sense, the whole world missed the point

01:54:36.020 | of OpenPsych and we were doing it to show people

01:54:39.660 | why that's not really what they wanted

01:54:41.620 | and too many people thought somehow that this was psych

01:54:44.780 | or that this was in fact good enough for them

01:54:47.520 | and they never even bothered coming to us

01:54:51.140 | to get access to the full psych.

01:54:53.140 | - But there's two parts to OpenPsych.

01:54:55.020 | So one is convincing people on the idea

01:54:57.340 | and the power of this general kind of representation

01:54:59.460 | of knowledge and the value that you hold

01:55:02.260 | in having acquired that knowledge

01:55:03.820 | and built it and continue to build it.

01:55:05.780 | And the other is the code base.

01:55:07.620 | This is the code side of it.

01:55:10.540 | So my sense of the code base

01:55:13.780 | that Psycorp Psych is operating with,

01:55:16.180 | I mean, it has the technical debt

01:55:19.020 | of the three decades plus, right?

01:55:21.940 | This is the exact same problem that Google had to deal with

01:55:24.700 | with the early version of TensorFlow.

01:55:27.380 | It's still dealing with that.

01:55:29.260 | They had to basically break compatibility

01:55:32.820 | with the past several times.

01:55:34.820 | And that's only over a period of a couple of years.

01:55:37.620 | But they, I think, successfully opened up,

01:55:40.900 | it's very risky, very gutsy move to open up TensorFlow

01:55:45.900 | and then PyTorch on the Facebook side.

01:55:47.980 | And what you see is there's a magic place

01:55:52.620 | where you can find a community,

01:55:54.100 | where you could develop a community

01:55:56.180 | that builds on the system without taking away any of,

01:56:01.180 | not any, but most of the value.

01:56:03.900 | So most of the value that Google has is still at Google.

01:56:06.700 | Most of the value that Facebook has is still at Facebook,

01:56:08.940 | even though some of this major machine learning tooling

01:56:12.620 | is released into the open.

01:56:14.380 | My question is not so much on the knowledge,

01:56:17.180 | which is also a big part of OpenPsych,

01:56:20.020 | but all the different kinds of tooling.

01:56:22.980 | So there's the kind of,

01:56:25.220 | all the kinds of stuff you can do on the knowledge,

01:56:27.540 | graph knowledge base, whatever we call it.

01:56:29.860 | There's the inference engines.

01:56:32.300 | So there could be some,

01:56:34.220 | there probably are a bunch of proprietary stuff

01:56:36.700 | you want to kind of keep secret.

01:56:38.340 | And there's probably some stuff you can open up completely

01:56:41.220 | and then let the community,

01:56:42.820 | build up enough community where they develop stuff

01:56:44.780 | on top of it.

01:56:45.700 | Yes, there'll be those publications and academic work

01:56:48.220 | and all that kind of stuff.

01:56:49.820 | And also the tooling of adding to the knowledge base, right?

01:56:54.100 | Like developing, there's an incredible amount,

01:56:56.780 | like there's so many people that are just really good

01:56:59.460 | at this kind of stuff in the open source community.

01:57:02.180 | So my question for you is like,

01:57:03.820 | have you struggled with this kind of idea

01:57:06.580 | that you have so much value in your company already?

01:57:09.420 | You've developed so many good things.

01:57:11.500 | You have clients that really value your relationships.

01:57:14.500 | And then there's this dormant, giant open source community

01:57:17.980 | that as far as I know, you're not utilizing.

01:57:20.220 | There's so many things to say there,

01:57:24.260 | but there could be magic moments

01:57:27.340 | where the community builds up large enough

01:57:31.420 | to where the artificial intelligence field

01:57:34.460 | that is currently 99.9% machine learning

01:57:37.860 | is dominated by machine learning,

01:57:39.900 | has a phase shift towards like, or at least in part,

01:57:44.340 | towards more like what you might call symbolic AI.

01:57:47.540 | This whole place where psych is like at the center of,

01:57:52.540 | and then that requires a little bit leap of faith

01:57:56.500 | because you're now surfing

01:57:58.340 | and there'll be obviously competitors that will pop up

01:58:00.900 | and start making you nervous and all that kind of stuff.

01:58:04.020 | So do you think about the space

01:58:05.980 | of open sourcing some parts and not others,

01:58:08.860 | how to leverage the community, all those kinds of things?

01:58:12.380 | - That's a good question.

01:58:13.420 | And I think you phrased it the right way,

01:58:15.060 | which is we're constantly struggling with the question

01:58:19.900 | of what to open source, what to make public,

01:58:24.180 | what to even publicly talk about.

01:58:29.380 | There are enormous pluses and minuses to every alternative.

01:58:34.380 | And it's very much like negotiating a very treacherous path.

01:58:42.220 | Partly the analogy is like, if you slip,

01:58:46.660 | you could make a fatal mistake,

01:58:48.380 | give away something which essentially kills you

01:58:51.460 | or fail to give away something

01:58:53.100 | which failing to give it away hurts you and so on.

01:58:58.380 | So it is a very tough, tough question.

01:59:01.940 | Usually what we have done with people who approached us

01:59:08.580 | to collaborate on research is to say,

01:59:11.780 | we will make available to you the entire knowledge base

01:59:16.780 | and executable copies of all of the code,

01:59:22.260 | but only very, very limited source code access

01:59:28.500 | if you have some idea for how you might improve something

01:59:31.500 | or work with us on something.

01:59:33.420 | So let me also get back to one of the very, very first things

01:59:38.260 | we talked about here, which was separating the question

01:59:43.260 | of how could you get a computer to do this at all

01:59:47.700 | versus how could you get a computer

01:59:49.580 | to do this efficiently enough in real time?

01:59:52.820 | And so one of the early lessons we learned

01:59:57.900 | was that we had to separate the epistemological problem

02:00:01.940 | of what should the system know,

02:00:04.180 | separate that from the heuristic problem

02:00:06.300 | of how can the system reason efficiently

02:00:08.820 | with what it knows.

02:00:10.220 | And so instead of trying to pick one representation language

02:00:15.220 | which was the sweet spot or the best trade-off point

02:00:20.340 | between expressiveness of the language

02:00:23.300 | and efficiency of the language,

02:00:24.780 | if you had to pick one,

02:00:26.380 | knowledge graphs would probably be,

02:00:28.620 | associative triples would probably be

02:00:30.740 | about the best you could do.

02:00:32.220 | And that's why we started there.

02:00:34.060 | But after a few years, we realized that what we could do

02:00:37.760 | is we could split this and we could have one nice,

02:00:41.860 | clean epistemological level language,

02:00:44.580 | which is this higher order logic.

02:00:46.700 | And we could have one or more grubby,

02:00:49.720 | but efficient heuristic level modules

02:00:54.300 | that opportunistically would say,

02:00:57.740 | oh, I can make progress

02:00:59.020 | on what you're trying to do over here.

02:01:01.180 | I have a special method that will contribute

02:01:04.300 | a little bit toward a solution.

02:01:06.060 | - Of course, some subset of that.

02:01:08.300 | - Yes, exactly.

02:01:09.460 | So by now we have over a thousand

02:01:11.540 | of these heuristic level modules

02:01:13.660 | and they function as a kind of community of agents.

02:01:16.900 | And there's one of them, which is a general theorem prover.

02:01:20.140 | And in theory, that's the only one you need.

02:01:25.140 | But in practice, it always takes so long

02:01:28.340 | that you never want to call on it.

02:01:30.980 | You always want these other agents

02:01:32.780 | to very efficiently reason through it.

02:01:35.180 | It's sort of like if you're balancing a chemical equation,

02:01:38.220 | you could go back to first principles,

02:01:40.340 | but in fact, there are algorithms

02:01:42.740 | which are vastly more efficient.

02:01:44.340 | Or if you're trying to solve a quadratic equation,

02:01:47.780 | you could go back to first principles of mathematics,

02:01:51.140 | but it's much better to simply recognize

02:01:54.300 | that this is a quadratic equation

02:01:56.140 | and apply the binomial formula and snap,

02:01:58.540 | you get your answer right away and so on.

02:02:00.820 | So think of these as like a thousand little experts

02:02:05.340 | that are all looking at everything that Psych gets asked

02:02:08.780 | and looking at everything

02:02:10.500 | that every other little agent has contributed,

02:02:12.820 | almost like notes on a blackboard,

02:02:14.900 | notes on a whiteboard.

02:02:17.660 | And making additional notes

02:02:20.260 | when they think they can be helpful.

02:02:21.700 | And gradually that community of agents

02:02:24.780 | gets an answer to your question,

02:02:26.540 | gets a solution to your problem.

02:02:29.020 | And if we ever come up in a domain application

02:02:32.780 | where Psych is getting the right answer,

02:02:34.620 | but taking too long,

02:02:36.700 | then what we'll often do is talk to one of the human experts

02:02:41.020 | and say, here's the set of reasoning steps

02:02:45.060 | that Psych went through.

02:02:46.340 | You can see why it took it a long time to get the answer.

02:02:49.620 | How is it that you were able to answer that question

02:02:52.340 | in two seconds?

02:02:53.260 | And occasionally you'll get an expert who just says,

02:02:57.980 | well, I just know it.

02:02:59.060 | I just was able to do it or something.

02:03:01.220 | And then you don't talk to them anymore.

02:03:02.860 | But sometimes you'll get an expert who says,

02:03:05.620 | well, let me introspect on that.

02:03:07.460 | Yes, here is a special representation we use

02:03:10.900 | just for aqueous chemistry equations,

02:03:14.340 | or here's a special representation and a special technique,

02:03:18.540 | which we can now apply to things

02:03:20.260 | in this special representation and so on.

02:03:22.780 | And then you add that as the thousand and first

02:03:26.060 | HL heuristic level module.

02:03:28.700 | And from then on in any application,

02:03:31.380 | if it ever comes up again,

02:03:32.980 | it'll be able to contribute and so on.

02:03:35.020 | So that's pretty much one of the main ways

02:03:38.260 | in which Psych has recouped this lost efficiency.

02:03:42.700 | A second important way is meta reasoning.

02:03:46.340 | So you can speed things up by focusing on removing knowledge

02:03:51.340 | from the system till all it has left

02:03:54.540 | is like minimal knowledge needed to,

02:03:56.780 | but that's the wrong thing to do, right?

02:03:58.860 | That would be like in a human extirpating

02:04:00.620 | part of their brain or something.

02:04:01.900 | That's really bad.

02:04:03.300 | So instead, what you wanna do is give it meta level advice,

02:04:06.900 | tactical and strategic advice that enables it to reason

02:04:10.980 | about what kind of knowledge is going to be relevant

02:04:14.620 | to this problem, what kind of tactics

02:04:16.660 | are gonna be good to take in trying to attack this problem?

02:04:20.380 | When is it time to start trying to prove the negation

02:04:23.700 | of this thing because I'm knocking myself out

02:04:26.260 | trying to prove it's true and maybe it's false.

02:04:28.540 | And if I just spend a minute,

02:04:29.540 | I can see that it's false or something.

02:04:31.660 | - So it's like dynamically pruning the graph

02:04:34.860 | to only like based on the particular thing

02:04:37.940 | you're trying to infer.

02:04:40.260 | - Yes, and so by now we have about 150

02:04:44.900 | of these sort of like breakthrough ideas

02:04:47.780 | that have led to dramatic speed ups

02:04:50.380 | in the inference process.

02:04:52.540 | You know, where one of them was this ELHL split

02:04:55.980 | and lots of HL modules.

02:04:57.380 | Another one was using meta and meta meta level reasoning

02:05:01.860 | to reason about the reasoning that's going on and so on.

02:05:06.140 | And you know, 150 breakthroughs may sound like a lot,

02:05:09.300 | but you know, if you divide by 37 years,

02:05:11.420 | it's not as impressive.

02:05:12.620 | - So there's these kind of heuristic modules

02:05:16.140 | that really help improve the inference.

02:05:18.920 | How hard in general is this,

02:05:24.340 | 'cause you mentioned higher order logic,

02:05:26.740 | you know, in the general, the theorem prover sense,

02:05:31.320 | it's intractable, very difficult problem.

02:05:33.980 | - Yes.

02:05:34.820 | - So how hard is this inference problem

02:05:37.260 | when we're not talking about,

02:05:39.060 | if we let go of the perfect and focus on the good?

02:05:43.740 | - I would say it's half of the problem

02:05:47.540 | in the following empirical sense,

02:05:49.900 | which is over the years, about half of our effort,

02:05:54.400 | maybe 40% of our effort,

02:05:56.740 | has been our team of inference programmers

02:06:01.340 | and the other 50, 60% has been our ontologists,

02:06:04.860 | our ontological engineers putting in knowledge.

02:06:07.340 | So our ontological engineers in most cases

02:06:09.580 | don't even know how to program.

02:06:11.420 | They have degrees in things like philosophy and so on.

02:06:15.020 | So it's almost like--

02:06:15.860 | - I love that.

02:06:16.700 | I'd love to hang out with those people actually.

02:06:18.340 | - Oh yes, it's wonderful.

02:06:19.660 | But it's very much like the Eloi and the Morlocks

02:06:22.420 | in H.G. Wells' "Time Machine."

02:06:24.940 | So you have the Eloi who only program

02:06:27.380 | in the epistemological higher order logic language.

02:06:31.820 | And then you have the Morlocks who are like

02:06:34.380 | under the ground figuring out what the machinery is

02:06:38.460 | that will make this efficiently operate and so on.

02:06:42.420 | And so occasionally they'll toss messages back

02:06:46.020 | to each other and so on,

02:06:47.900 | but it really is almost this 50/50 split

02:06:50.900 | between finding clever ways to recoup efficiency

02:06:55.760 | when you have an expressive language

02:06:57.740 | and putting in the content of what the system needs to know.

02:07:01.660 | - And yeah, both are fascinating.

02:07:03.780 | To some degree, the entirety of the system,

02:07:06.300 | as far as I understand,

02:07:08.300 | is written in various variants of Lisp.

02:07:11.980 | So my favorite program language is still Lisp.

02:07:14.300 | I don't program in it much anymore

02:07:16.860 | because the world has,

02:07:19.460 | in majority of its system, has moved on.

02:07:22.300 | Like everybody respects Lisp,

02:07:24.980 | but many of the systems are not written in Lisp anymore.

02:07:28.720 | But Psyche, as far as I understand,

02:07:30.940 | maybe you can correct me,

02:07:32.460 | there's a bunch of Lisp in it.

02:07:34.660 | - Yeah, so it's based on Lisp code that we produced.

02:07:37.900 | Most of the programming is still going on

02:07:39.980 | in a dialect of Lisp.

02:07:41.980 | And then for efficiency reasons,

02:07:45.460 | that gets automatically translated into things like Java

02:07:50.120 | or C.

02:07:50.960 | Nowadays, it's almost all translated into Java

02:07:53.700 | 'cause Java has gotten good enough

02:07:56.020 | that that's really all we need to do.

02:07:58.700 | - So it's translated into Java

02:07:59.980 | and then Java is compiled down by code.

02:08:02.820 | - Yes.

02:08:03.660 | - Okay, so that's sort of,

02:08:05.060 | it's a process that probably has to do with the fact

02:08:12.020 | that when Psyche was originally written

02:08:14.580 | and you built up a powerful system,

02:08:16.480 | there is some technical depth you have to deal with,

02:08:19.740 | as is the case with most powerful systems that span years.

02:08:24.120 | Have you ever considered,

02:08:28.980 | this would help me understand,

02:08:30.780 | because from my perspective,

02:08:32.300 | so much of the value of everything you've done

02:08:35.540 | with Psyche and Psycorp is the knowledge.

02:08:40.780 | Have you ever considered just like throwing away

02:08:43.460 | the code base and starting from scratch?

02:08:45.580 | Not really throwing away,

02:08:47.540 | but sort of moving it to,

02:08:50.380 | throwing away that technical debt,

02:08:54.080 | starting with a more updated programming language.

02:08:58.180 | Is that throwing away a lot of value or no?

02:09:00.820 | Like, what's your sense?

02:09:01.820 | How much of the value is in the silly

02:09:03.940 | software engineering aspect

02:09:05.900 | and how much of the value is in the knowledge?

02:09:08.260 | - So development of programs in Lisp

02:09:15.100 | proceeds, I think, somewhere between 1,000

02:09:21.420 | and 50,000 times faster than development

02:09:24.580 | in any of what you're calling modern

02:09:28.140 | or improved computer languages.

02:09:30.220 | - Well, there's other functional languages,

02:09:31.580 | like Clojure and all that.

02:09:32.980 | But I mean, I'm with you.

02:09:35.500 | I like Lisp.

02:09:37.020 | I just wonder how many great programmers there are.

02:09:39.460 | There's still like--

02:09:40.540 | - Yes, so it is true when a new inference programmer

02:09:43.860 | comes on board, they need to learn some of Lisp.

02:09:48.860 | And in fact, we have a subset of Lisp,

02:09:50.820 | which we call cleverly sub L,

02:09:53.700 | which is really all they need to learn.

02:09:55.780 | And so the programming actually goes on in sub L,

02:09:59.620 | not in full Lisp.

02:10:01.340 | And so it does not take programmers very long at all

02:10:03.980 | to learn sub L.

02:10:06.220 | And that's something which can then be translated

02:10:08.980 | efficiently into Java.

02:10:11.900 | And for some of our programmers who are doing,

02:10:14.420 | say, user interface work,

02:10:15.860 | then they never have to even learn sub L.

02:10:18.380 | They just have to learn APIs into the basic Cyc engine.

02:10:23.500 | - So you're not necessarily feeling the burden

02:10:25.860 | of like it's extremely efficient.

02:10:28.100 | That's not a problem to solve.

02:10:31.580 | Okay. - Right.

02:10:32.420 | The other thing is, remember that we're talking

02:10:34.460 | about hiring programmers to do inference

02:10:37.700 | who are programmers interested

02:10:39.700 | in effectively automatic theorem proving.

02:10:42.500 | And so those are people already predisposed

02:10:45.740 | to representing things in logic and so on.

02:10:48.460 | And Lisp really was the programming language

02:10:53.460 | based on logic that John McCarthy and others

02:10:56.780 | who developed it basically took the formalisms

02:11:00.940 | that Alonzo Church and other philosophers,

02:11:03.780 | other logicians had come up with and basically said,

02:11:07.500 | can we basically make a programming language

02:11:10.940 | which is effectively logic?

02:11:13.580 | And so since we're talking about reasoning

02:11:17.260 | in about expressions written in this logical,

02:11:21.700 | epistemological language and we're doing operations

02:11:24.380 | which are effectively like theorem proving

02:11:26.860 | type operations and so on,

02:11:28.900 | there's a natural impedance match between Lisp

02:11:33.060 | and the knowledge the way it's represented.

02:11:35.900 | So-- - I guess you could say

02:11:37.540 | it's a perfectly logical language to use.

02:11:40.740 | - Oh, yes.

02:11:42.700 | - Okay, I'm sorry. (laughs)

02:11:44.060 | - I'll even let you get away with that.

02:11:46.060 | - Okay, I appreciate it. - I know I like it.

02:11:47.860 | So I'll probably use that in the future without--

02:11:51.340 | - Without credit. - Without credit.

02:11:53.100 | But no, I think the point is that the language

02:11:58.100 | you program in isn't really that important.

02:12:01.180 | It's more that you have to be able to think in terms of,

02:12:05.260 | for instance, creating new helpful HL modules

02:12:08.540 | and how they'll work with each other

02:12:10.700 | and looking at things that are taking a long time

02:12:14.540 | and coming up with new specialized data structures

02:12:18.860 | that will make this efficient.

02:12:20.140 | So let me just give you one very simple example,

02:12:23.060 | which is when you have a transitive relation,

02:12:26.100 | like larger than, this is larger than that,

02:12:28.500 | which is larger than that, which is larger than that.

02:12:30.420 | So the first thing must be larger than the last thing.

02:12:33.220 | Whenever you have a transitive relation,

02:12:35.980 | if you're not careful,

02:12:37.220 | if I ask whether this thing over here

02:12:39.580 | is larger than the thing over here,

02:12:40.980 | I'll have to do some kind of graph walk or theorem proving

02:12:45.060 | that might involve like five or 10 or 20 or 30 steps.

02:12:48.540 | But if you store, redundantly store the transitive closure,

02:12:53.540 | the Kleene star of that transitive relation,

02:12:56.740 | now you have this big table,

02:12:58.780 | but you can always guarantee that in one single step,

02:13:02.340 | you can just look up whether this is larger than that.

02:13:06.180 | And so there are lots of cases where storage is cheap today.

02:13:11.180 | And so by having this extra redundant data structure,

02:13:15.860 | we can answer this commonly occurring type of question

02:13:19.820 | very, very efficiently.

02:13:21.260 | Let me give you one other analogy, analog of that,

02:13:26.540 | which is something we call rule macro predicates,

02:13:29.180 | which is we'll see this complicated rule

02:13:33.140 | and we'll notice that things very much like it,

02:13:37.420 | syntactically come up again and again and again.

02:13:40.940 | So we'll create a whole brand new relation

02:13:43.620 | or predicate or function that captures that

02:13:47.940 | and takes maybe not two arguments,

02:13:50.180 | takes maybe three, four, five arguments and so on.

02:13:54.340 | And now we have effectively converted

02:13:58.340 | some complicated if then rule

02:14:02.340 | that might have to have inference done on it

02:14:05.660 | into some ground atomic formula,

02:14:07.580 | which is just the name of a relation

02:14:10.980 | and a few arguments and so on.

02:14:12.980 | And so converting commonly occurring types

02:14:16.820 | or schemas of rules into brand new predicates,

02:14:20.780 | brand new functions,

02:14:22.060 | turns out to enormously speed up the inference process.

02:14:26.300 | So now we've covered about four of the 150 good ideas

02:14:30.980 | I said that--

02:14:31.820 | - So that's a nice, that's a cool,

02:14:32.660 | so that idea in particular is like a nice compression

02:14:35.180 | that turns out to be really useful.

02:14:36.940 | - Yes. - That's really interesting.

02:14:38.020 | I mean, this whole thing is just fascinating

02:14:39.540 | from a philosophical, there's part of me,

02:14:42.180 | I mean, it makes me a little bit sad

02:14:44.220 | because your work is both from a computer science perspective

02:14:49.220 | fascinating and the inference engine

02:14:51.540 | from a epistemological, philosophical aspect fascinating.

02:14:56.260 | But it is also you running a company

02:14:59.420 | and there's some stuff that has to remain private.

02:15:02.460 | It's sad.

02:15:03.420 | - Well, here's something that may make you feel better,

02:15:05.980 | a little bit better.

02:15:09.020 | We've formed a not-for-profit company

02:15:12.700 | called the Knowledge Activization Institute, NAX, K-N-A-X.

02:15:16.700 | And I have this firm belief

02:15:20.020 | with a lot of empirical evidence to support it

02:15:22.300 | that the education that people get in high schools

02:15:27.300 | and colleges and graduate schools and so on

02:15:31.460 | is almost completely orthogonal to,

02:15:34.700 | almost completely irrelevant to

02:15:37.740 | how good they're going to be at coming up to speed

02:15:40.940 | in doing this kind of ontological engineering

02:15:44.620 | and writing these assertions and rules and so on in psych.

02:15:49.460 | And so very often we'll interview candidates

02:15:52.300 | who have their PhD in philosophy,

02:15:54.620 | who've taught logic for years and so on,

02:15:57.380 | and they're just awful.

02:15:59.620 | But the converse is true.

02:16:00.740 | So one of the best ontological engineers we ever had

02:16:04.220 | never graduated high school.

02:16:06.180 | And so the purpose of Knowledge Activization Institute,

02:16:11.180 | if we can get some foundations to help support it,

02:16:14.580 | is identify people in the general population,

02:16:18.580 | maybe high school dropouts,

02:16:20.580 | who have latent talent for this sort of thing,

02:16:24.700 | offer them effectively scholarships to train them,

02:16:29.300 | and then help place them in companies

02:16:32.220 | that need more trained ontological engineers,

02:16:35.140 | some of which would be working for us,

02:16:36.780 | but mostly would be working for partners

02:16:38.980 | or customers or something.

02:16:40.580 | And if we could do that,

02:16:42.140 | that would create an enormous number

02:16:43.860 | of relatively very high paying jobs

02:16:47.420 | for people who currently have no way out

02:16:51.020 | of some situation that they're locked into.

02:16:55.020 | - So is there something you can put into words

02:16:57.420 | that describes somebody who would be great

02:17:01.180 | at ontological engineering?

02:17:03.220 | So what characteristics about a person

02:17:06.420 | make them great at this task?

02:17:08.660 | This task of converting the messiness

02:17:13.220 | of human language and knowledge into formal logic.

02:17:17.180 | - This is very much like what Alan Turing had to do

02:17:20.060 | during World War II in trying to find people

02:17:23.180 | to bring to Bletchley Park,

02:17:24.900 | where he would publish in the London Times

02:17:28.100 | cryptic crossword puzzles,

02:17:30.140 | along with some innocuous looking note,

02:17:33.420 | which essentially said,

02:17:34.420 | if you were able to solve this puzzle

02:17:36.420 | in less than 15 minutes,

02:17:38.260 | please call this phone number, and so on.

02:17:41.260 | So, or back when I was young,

02:17:44.540 | there was the practice of having matchbooks,

02:17:49.060 | where on the inside of the matchbook,

02:17:51.580 | there would be a, can you draw this?

02:17:53.980 | You have a career in art, commercial art,

02:17:56.900 | if you can copy this drawing, and so on.

02:17:59.940 | So yes, the analog of that--

02:18:02.500 | - Was there a little test to get to the core

02:18:04.500 | of whether you're gonna be good or not?

02:18:05.900 | - So part of it has to do with being able

02:18:08.700 | to make and appreciate and react negatively,

02:18:13.540 | appropriately to puns and other jokes.

02:18:16.100 | So you have to have a kind of sense of humor,

02:18:18.460 | and if you're good at telling jokes

02:18:21.500 | and good at understanding jokes,

02:18:23.740 | that's one indicator--

02:18:25.460 | - Like puns? - Yes.

02:18:26.300 | - Like dad jokes? - Yes.

02:18:27.660 | Well, maybe not dad jokes,

02:18:29.300 | but real, but funny jokes.

02:18:31.420 | But--

02:18:32.380 | - I think I'm applying to work at SACWR.

02:18:34.380 | - Yeah, but another is if you're able to introspect.

02:18:38.140 | So very often, we'll give someone a simple question,

02:18:43.140 | and we'll say like, "Why is this?"

02:18:48.060 | And sometimes they'll just say, "Because it is."

02:18:51.420 | Okay, that's a bad sign.

02:18:53.220 | But very often, they'll be able to introspect, and so on.

02:18:56.460 | So one of the questions I often ask

02:18:59.540 | is I'll point to a sentence with a pronoun in it,

02:19:01.740 | and I'll say, "The referent of that pronoun

02:19:04.540 | "is obviously this noun over here.

02:19:06.540 | "How would you or I or an AI

02:19:10.180 | "or a five-year-old, 10-year-old child

02:19:12.740 | "know that that pronoun refers to that noun over here?"

02:19:17.740 | And often, the people who are going to be good

02:19:22.860 | at ontological engineering

02:19:24.340 | will give me some causal explanation

02:19:27.020 | or will refer to some things that are true in the world.

02:19:29.980 | So if you imagine a sentence like,

02:19:31.860 | "The horse was led into the barn

02:19:33.820 | "while its head was still wet."

02:19:36.020 | And so, "its head" refers to the horse's head.

02:19:38.860 | But how do you know that?

02:19:40.300 | And so, some people will say, "I just know it."

02:19:42.060 | Some people will say, "Well, the horse

02:19:43.460 | "was the subject of the sentence."

02:19:45.540 | And I'll say, "Okay, well, what about,

02:19:47.200 | "the horse was led into the barn

02:19:48.580 | "while its roof was still wet."

02:19:50.740 | Now, "its roof" obviously refers to the barn,

02:19:53.820 | and so, then they'll say,

02:19:55.980 | "Oh, well, that's because it's the closest noun."

02:19:58.260 | And so, basically, if they try to give me answers

02:20:01.820 | which are based on syntax and grammar and so on,

02:20:05.740 | that's a really bad sign.

02:20:07.460 | But if they're able to say things like,

02:20:09.100 | "Well, horses have heads and barns don't,

02:20:11.620 | "and barns have roofs and horses don't,"

02:20:14.380 | then that's a positive sign

02:20:16.100 | that they're gonna be good at this

02:20:17.260 | because they can introspect on what's true in the world

02:20:20.780 | that leads you to know certain things.

02:20:22.740 | - How fascinating is it that getting a PhD

02:20:25.100 | makes you less capable to introspect deeply about this?

02:20:28.860 | - Oh, I wouldn't go that far.

02:20:30.860 | I'm not saying that it makes you less capable.

02:20:32.660 | Let's just say it's independent of--

02:20:35.140 | - Oh, I don't know about this.

02:20:36.300 | - Of how good people are.

02:20:37.140 | - Okay, you're not saying that, I'm saying that.

02:20:38.940 | There's a certain, it's interesting

02:20:41.640 | that for a lot of people, PhDs, sorry, philosophy aside,

02:20:46.640 | that sometimes education narrows your thinking

02:20:50.380 | versus expands it.

02:20:51.580 | - Yes.

02:20:52.420 | - That's fascinating.

02:20:53.420 | And for certain, when you're trying

02:20:55.100 | to do ontological engineering,

02:20:57.320 | which is essentially teach our future AI overlords

02:21:00.900 | how to reason deeply about this world

02:21:03.500 | and how to understand it,

02:21:05.220 | that requires that you think deeply about the world.

02:21:08.660 | - So I'll tell you a sad story about Math Craft,

02:21:11.900 | which is why is that not widely used in schools today?

02:21:16.020 | We're not really trying to make big profit on it

02:21:19.120 | or anything like that.

02:21:20.580 | When we've gone to schools, their attitude has been,

02:21:23.820 | well, if a student spends 20 hours

02:21:27.220 | going through this Math Craft program

02:21:29.660 | from start to end and so on,

02:21:31.620 | will it improve their score on this standardized test

02:21:36.900 | more than if they spent 20 hours

02:21:38.880 | just doing mindless drills

02:21:41.360 | of problem after problem after problem?

02:21:43.540 | And the answer is, well, no,

02:21:45.900 | but it'll increase their understanding more

02:21:47.780 | and their attitude is, well,

02:21:48.900 | if it doesn't increase their score on this test,

02:21:52.700 | then we're not gonna adopt it.

02:21:56.020 | - That's sad.

02:21:56.860 | I mean, that's a whole another three, four hour conversation

02:22:00.780 | about the education system.

02:22:01.820 | But let me go super philosophical as if we weren't already.

02:22:06.580 | So in 1950, Alan Turing wrote the paper

02:22:09.460 | that formulated the Turing test.

02:22:11.620 | - Yes.

02:22:12.460 | - And he opened the paper with the question,

02:22:13.900 | can machines think?

02:22:15.940 | So what do you think?

02:22:16.900 | Can machines think?

02:22:18.980 | Let me ask you this question.

02:22:20.380 | - Absolutely.

02:22:21.500 | Machines can think,

02:22:23.540 | certainly as well as humans can think, right?

02:22:27.860 | We're meat machines,

02:22:30.340 | just because they're not currently made out of meat

02:22:32.820 | is just an engineering solution decision and so on.

02:22:37.820 | So of course machines can think.

02:22:42.180 | I think that there was a lot of damage done

02:22:47.700 | by people misunderstanding Turing's imitation game

02:22:52.700 | and focus on trying to get a chat bot

02:22:58.260 | to fool other people into thinking it was human and so on.

02:23:05.300 | That's not a terrible test in and of itself,

02:23:08.980 | but it shouldn't be your one and only test for intelligence.

02:23:12.580 | - So do you, in terms of tests of intelligence,

02:23:16.620 | with the Lobner Prize,

02:23:18.340 | which is a very kind of, you want to say,

02:23:20.780 | a more strict formulation of the Turing test

02:23:23.540 | as originally formulated.

02:23:25.220 | And then there's something like Alexa Prize,

02:23:28.180 | which is more, I would say,

02:23:30.420 | a more interesting formulation of the test,

02:23:32.540 | which is like, ultimately the metric is

02:23:36.300 | how long does a human want to talk to the AI system?

02:23:39.340 | So it's like, the goal is you want it to be 20 minutes.

02:23:43.540 | It's basically not just have a convincing conversation,

02:23:48.540 | but more like a compelling one or a fun one

02:23:51.700 | or an interesting one.

02:23:53.260 | And that seems like more to the spirit maybe

02:23:57.060 | of what Turing was imagining.

02:24:01.020 | But what for you do you think in the space of tests

02:24:04.220 | is a good test?

02:24:06.540 | When you see a system based on psych that passes that test,

02:24:10.900 | you'd be like, damn, we've created something special here.

02:24:14.520 | - The test has to be something involving

02:24:19.380 | depth of reasoning and recursiveness of reasoning,

02:24:23.620 | the ability to answer repeated why questions

02:24:27.300 | about the answer you just gave.

02:24:28.940 | - How many why questions in a row can you keep answering?

02:24:32.820 | - Something like that.

02:24:33.980 | And also--

02:24:36.260 | - Just have like a young curious child and an AI system.

02:24:39.740 | And how long will an AI system last

02:24:41.820 | before it wants to quit?

02:24:43.180 | - Yes, and again, that's not the only test.

02:24:45.380 | Another one has to do with argumentation.

02:24:47.940 | In other words, here's a proposition.

02:24:50.140 | Come up with pro and con arguments for it

02:24:56.060 | and try and give me convincing arguments on both sides.

02:25:01.060 | So that's another important kind of ability

02:25:06.220 | that the system needs to be able to exhibit

02:25:09.660 | in order to really be intelligent, I think.

02:25:12.780 | - So there's certain, I mean, if you look at IBM Watson

02:25:16.140 | and like certain impressive accomplishments

02:25:18.220 | for very specific tests, almost like a demo, right?

02:25:22.740 | There is some, like I talked to the guy

02:25:28.460 | who led the Jeopardy effort.

02:25:33.780 | And there's some kind of hard coding heuristics tricks

02:25:37.780 | that you try to pull it all together

02:25:39.700 | to make the thing work in the end for this thing, right?

02:25:42.820 | That seems to be one of the lessons with AI

02:25:45.380 | is like that's the fastest way to get a solution

02:25:49.100 | that's pretty damn impressive.

02:25:50.420 | - So here's what I would say is that

02:25:53.300 | as impressive as that was, it made some mistakes.

02:25:59.220 | But more importantly, many of the mistakes it made

02:26:02.980 | were mistakes which no human would have made.

02:26:05.780 | And so part of the new or augmented Turing tests

02:26:11.860 | would have to be, and the mistakes you make

02:26:18.220 | are ones which humans don't basically look at and say, what?

02:26:23.020 | So for example, there was a question

02:26:28.700 | about which 16th century Italian politician, blah, blah, blah

02:26:33.700 | and Watson said, Ronald Reagan.

02:26:37.140 | So most Americans would have gotten that question wrong

02:26:40.260 | but they would never have said Ronald Reagan as an answer

02:26:43.780 | because among the things they know

02:26:47.380 | is that he lived relatively recently

02:26:49.980 | and people don't really live 400 years

02:26:52.380 | and things like that.

02:26:53.860 | So that's, I think, a very important thing

02:26:57.020 | which is if it's making mistakes

02:26:59.900 | which no normal sane human would have made,

02:27:04.020 | then that's a really bad sign.

02:27:05.700 | And if it's not making those kinds of mistakes,

02:27:08.180 | then that's a good sign.

02:27:10.020 | And I don't think it's any one very, very simple test.

02:27:12.980 | I think it's all of the things you mentioned,

02:27:14.940 | all the things I mentioned,

02:27:16.020 | there's really a battery of tests which together,

02:27:19.660 | if it passes almost all of these tests,

02:27:22.460 | it would be hard to argue that it's not intelligent.

02:27:25.140 | And if it fails several of these tests,

02:27:28.380 | it's really hard to argue that it really understands

02:27:30.980 | what it's doing and that it really is generally intelligent.

02:27:33.340 | - So to pass all of those tests,

02:27:35.860 | we've talked a lot about psych and knowledge and reasoning.

02:27:40.660 | Do you think this AI system would need to have

02:27:44.180 | some other human-like elements,

02:27:46.700 | for example, a body or a physical manifestation in this world

02:27:52.380 | and another one which seems to be fundamental

02:27:55.660 | to the human experience is consciousness,

02:27:58.660 | the subjective experience of what it's like

02:28:02.300 | to actually be you.

02:28:04.540 | Do you think it needs those to be able to pass

02:28:06.980 | all of those tests and to achieve general intelligence?

02:28:09.380 | - It's a good question.

02:28:10.220 | I think in the case of a body, no.

02:28:12.660 | I know there are a lot of people like Penrose

02:28:15.300 | who would have disagreed with me and others,

02:28:19.460 | but no, I don't think it needs to have a body

02:28:22.060 | in order to be intelligent.

02:28:24.300 | I think that it needs to be able to talk about

02:28:28.300 | having a body and having sensations

02:28:31.900 | and having emotions and so on.

02:28:34.020 | It doesn't actually have to have all of that,

02:28:37.660 | but it has to understand it in the same way

02:28:39.500 | that Helen Keller was perfectly intelligent

02:28:42.500 | and able to talk about colors and sounds and shapes

02:28:47.580 | and so on, even though she didn't directly experience

02:28:52.100 | all the same things that the rest of us do.

02:28:54.500 | So knowledge of it and being able to correctly make use

02:28:59.500 | of that is certainly an important facility,

02:29:04.980 | but actually having a body, if you believe that,

02:29:07.940 | that's just a kind of religious or mystical belief.

02:29:11.420 | You can't really argue for or against it, I suppose.

02:29:15.020 | It's just something that some people believe.

02:29:19.220 | - What about an extension of the body,

02:29:22.540 | which is consciousness?

02:29:23.660 | I mean, it feels like something to be here.

02:29:27.980 | - Sure, but what does that really mean?

02:29:30.620 | It's like, well, if I talk to you,

02:29:32.340 | you say things which make me believe that you're conscious.

02:29:35.940 | I know that I'm conscious, but you're just taking

02:29:38.700 | my word for it now.

02:29:40.380 | But in the same sense, psych is conscious

02:29:43.340 | in that same sense already, where of course it understands

02:29:46.420 | it's a computer program, it understands where

02:29:48.660 | and when it's running, it understands who's talking to it,

02:29:51.420 | it understands what its task is, what its goals are,

02:29:54.300 | what its current problem is that it's working on.

02:29:56.780 | It understands how long it's spent on things,

02:29:58.700 | what it's tried, it understands what it's done in the past

02:30:02.700 | and so on.

02:30:04.060 | And if we want to call that consciousness,

02:30:08.380 | then yes, psych is already conscious,

02:30:11.180 | but I don't think that I would ascribe anything mystical

02:30:15.140 | to that, again, some people would,

02:30:17.100 | but I would say that other than our own personal experience

02:30:21.300 | of consciousness, we're just treating everyone else

02:30:24.300 | in the world, so to speak, at their word

02:30:28.060 | about being conscious.

02:30:29.580 | And so if a computer program, if an AI is able to exhibit

02:30:34.580 | all the same kinds of response as you would expect

02:30:40.260 | of a conscious entity, then doesn't it deserve

02:30:44.940 | the label of consciousness just as much?

02:30:47.420 | - So there's another burden that comes

02:30:49.220 | with this whole intelligence thing that humans got

02:30:52.020 | is the extinguishing of the light of consciousness,

02:30:57.020 | which is kind of realizing that we're gonna be dead someday.

02:31:02.380 | And there's a bunch of philosophers like Ernest Becker

02:31:05.700 | who kind of think that this realization of mortality

02:31:10.420 | and then fear, sometimes they call it terror,

02:31:14.300 | of mortality is one of the creative forces

02:31:19.300 | behind human condition, like it's the thing that drives us.

02:31:25.300 | Do you think it's important for an AI system,

02:31:27.860 | you know, when psych proposed that it's not human

02:31:34.440 | and it's one of the moderators of its contents,

02:31:37.860 | you know, there's another question it could ask,

02:31:42.440 | which is like, it kind of knows that humans are mortal.

02:31:45.740 | Am I mortal?

02:31:47.480 | And I think one really important thing that's possible

02:31:52.360 | when you're conscious is to fear the extinguishing

02:31:55.460 | of that consciousness, the fear of mortality.

02:31:58.920 | Do you think that's useful for intelligence?

02:32:02.000 | Thinking like I might die and I really don't want to die.

02:32:05.280 | - I don't think so.

02:32:06.700 | I think it may help some humans to be better people.

02:32:11.700 | It may help some humans to be more creative and so on.

02:32:16.180 | I don't think it's necessary for AIs to believe

02:32:21.180 | that they have limited lifespans

02:32:23.320 | and therefore they should make the most of their behavior.

02:32:25.820 | Maybe eventually the answer to that

02:32:28.480 | and my answer to that will change.

02:32:30.440 | But as of now, I would say that that's almost like a frill

02:32:34.060 | or a side effect that is not,

02:32:36.620 | in fact, if you look at most humans,

02:32:38.740 | most humans ignore the fact that they're going to die

02:32:42.380 | most of the time.

02:32:44.140 | So--

02:32:44.980 | - Well, but that's like,

02:32:46.620 | this goes to the white space between the words.

02:32:49.540 | So what Ernest Becker argues is that that ignoring

02:32:53.040 | is we're living in an illusion that we constructed

02:32:55.480 | on the foundation of this terror.

02:32:57.720 | So we're escape life as we know it,

02:33:00.980 | pursuing things, creating things, love,

02:33:04.720 | everything we can think of that's beautiful about humanity

02:33:08.500 | is just trying to escape this realization

02:33:11.620 | that we're going to die one day.

02:33:13.160 | That's his idea and I think, I don't know if I 100%

02:33:18.160 | believe in this, but it certainly rhymes.

02:33:22.620 | It seems like to me, like it rhymes with the truth.

02:33:26.260 | - Yeah, I think that for some people,

02:33:28.420 | that's gonna be a more powerful factor than others.

02:33:33.020 | - Clearly, Doug is talking about Russians.

02:33:35.500 | - And I think that--

02:33:37.300 | (laughing)

02:33:39.740 | - So I'm Russian, so clearly it infiltrates

02:33:42.740 | all of Russian literature.

02:33:44.400 | - And AI doesn't have to have fear of death

02:33:49.400 | as a motivating force in that we can build in motivation.

02:33:55.540 | So we can build in the motivation of obeying users

02:34:00.540 | and making users happy and making others happy and so on.

02:34:05.620 | And that can substitute for this sort of personal fear

02:34:10.620 | of death that sometimes leads to bursts

02:34:14.620 | of creativity in humans.

02:34:16.980 | - Yeah, I don't know.

02:34:17.820 | I think AI really needs to understand death deeply

02:34:22.060 | in order to be able to drive a car, for example.

02:34:24.900 | I think there's just some, like--

02:34:28.220 | - No, I really disagree.

02:34:30.080 | I think it needs to understand the value of human life,

02:34:33.640 | especially the value of human life to other humans,

02:34:36.140 | and understand that certain things are more important

02:34:41.540 | than other things, so it has to have a lot of knowledge

02:34:44.260 | about ethics and morality and so on.

02:34:48.060 | - But some of it is so messy that it's impossible to encode.

02:34:51.180 | For example, there's--

02:34:52.020 | - I disagree.

02:34:53.720 | So if there's a person dying right in front of us,

02:34:56.260 | most human beings would help that person,

02:34:59.260 | but they would not apply that same ethics

02:35:02.220 | to everybody else in the world.

02:35:04.500 | This is the tragedy of how difficult it is to be a doctor

02:35:08.020 | because they know when they help a dying child,

02:35:11.420 | they know that the money they're spending on this child

02:35:14.860 | cannot possibly be spent on every other child that's dying.

02:35:18.580 | And that's a very difficult to encode decision.

02:35:23.240 | Now, perhaps it is, perhaps it could be formalized.

02:35:27.780 | - Oh, but I mean, you're talking about

02:35:30.180 | autonomous vehicles, right?

02:35:31.780 | So autonomous vehicles are going to have to make

02:35:34.740 | those decisions all the time of what is the chance

02:35:39.740 | of this bad event happening, how bad is that compared

02:35:44.220 | to this chance of that bad event happening, and so on,

02:35:47.300 | and when a potential accident is about to happen,

02:35:51.340 | is it worth taking this risk?

02:35:52.900 | If I have to make a choice, which of these two cars

02:35:55.400 | am I going to hit and why?

02:35:57.000 | - See, I was thinking about a very different choice

02:35:59.220 | when I'm talking about fear of mortality,

02:36:01.100 | which is just observing Manhattan style driving.

02:36:06.020 | I think that humans as an effective driver

02:36:09.660 | needs to threaten pedestrians' lives a lot.

02:36:14.660 | There's a dance, I've watched pedestrians a lot,

02:36:17.380 | I worked on this problem, and it seems like the,

02:36:21.380 | if I could summarize the problem of a pedestrian crossing

02:36:25.460 | is the car with this movement is saying,

02:36:28.260 | "I'm going to kill you," and the pedestrian is saying,

02:36:32.040 | "Maybe," and then they decide and they say,

02:36:34.400 | "No, I don't think you have the guts to kill me,"

02:36:36.660 | and they walk in front and they look away,

02:36:39.260 | and there's that dance, the pedestrian,

02:36:42.980 | as this is social contract, that the pedestrian trusts

02:36:46.360 | that once they're in front of the car

02:36:47.500 | and the car is sufficiently, from a physics perspective,

02:36:51.060 | able to stop, they're going to stop.

02:36:53.140 | But the car also has to threaten that pedestrian,

02:36:55.460 | it's like, "I'm late for work,

02:36:57.460 | "so you're being kind of an asshole

02:36:59.380 | "by crossing in front of me,"

02:37:00.980 | but life and death is part of the calculation here,

02:37:05.760 | and that equation is being solved millions of times a day.

02:37:10.760 | - Yes. - Very effectively.

02:37:12.820 | That game theory, whatever that formulation is.

02:37:15.260 | - Absolutely.

02:37:16.100 | - I just, I don't know if it's as simple

02:37:18.200 | as some formalizable game theory problem.

02:37:22.140 | It could very well be in the case of driving

02:37:24.540 | and in the case of most of human society.

02:37:28.100 | I don't know, but yeah, you might be right

02:37:31.380 | that the fear of death is just one of the quirks

02:37:34.820 | of the way our brains have evolved,

02:37:38.500 | but it's not a necessary feature of intelligence.

02:37:42.120 | - Drivers certainly are always doing this kind of estimate,

02:37:45.740 | even if it's unconscious, subconscious,

02:37:48.740 | of what are the chances of various bad outcomes happening,

02:37:52.640 | like for instance, if I don't wait for this pedestrian

02:37:56.200 | or something like that,

02:37:57.600 | and what is the downside to me going to be

02:38:01.300 | in terms of time wasted talking to the police

02:38:06.100 | or getting sent to jail or things like that.

02:38:10.340 | - And there's also emotion,

02:38:12.980 | like people in their cars tend to get irrationally angry.

02:38:16.940 | - That's dangerous, but think about,

02:38:19.820 | this is all part of why I think that autonomous vehicles,

02:38:23.860 | truly autonomous vehicles are farther out

02:38:26.220 | than most people do,

02:38:28.580 | because there is this enormous level of complexity

02:38:32.340 | which goes beyond mechanically controlling the car.

02:38:38.140 | And I can see the autonomous vehicles

02:38:42.860 | as a kind of metaphorical and literal accident

02:38:45.260 | waiting to happen.

02:38:46.940 | And not just because of their overall incurring

02:38:51.940 | versus preventing accidents and so on,

02:38:56.740 | but just because of the almost voracious appetite

02:39:01.740 | people have for bad stories,

02:39:08.040 | bad stories about powerful companies and powerful entities.

02:39:12.120 | When I was at a, coincidentally,

02:39:16.600 | Japanese fifth generation computing system conference

02:39:19.760 | in 1987, while I happened to be there,

02:39:23.560 | there was a worker at an auto plant

02:39:25.240 | who was despondent and committed suicide

02:39:27.480 | by climbing under the safety chains and so on,

02:39:30.240 | getting stamped to death by a machine.

02:39:32.800 | And instead of being a small story that said,

02:39:35.800 | despondent worker commits suicide,

02:39:38.280 | it was front page news that effectively said,

02:39:42.160 | robot kills worker,

02:39:44.240 | because the public is just waiting for stories

02:39:47.920 | about like AI kills phonogenic family of five type stories.

02:39:52.920 | And even if you could show that nationwide,

02:39:58.000 | this system saved more lives than it cost

02:40:01.400 | and saved more injuries,

02:40:03.480 | prevented more injuries than it caused and so on,

02:40:06.600 | the media, the public, the government is just coiled

02:40:10.600 | and ready to pounce on stories where in fact it failed,

02:40:16.760 | even if they're relatively few.

02:40:18.540 | - Yeah, it's so fascinating to watch us humans

02:40:23.200 | resisting the cutting edge of science and technology

02:40:27.520 | and almost like hoping for it to fail and constantly,

02:40:31.120 | this just happens over and over and over

02:40:33.040 | throughout history.

02:40:33.880 | - Or even if we're not hoping for it to fail,

02:40:35.440 | we're fascinated by it.

02:40:37.640 | And in terms of what we find interesting,

02:40:40.020 | the one in a thousand failures,

02:40:43.120 | much more interesting than the 999 boring successes.

02:40:48.040 | - So once we build an AGI system,

02:40:50.960 | say psych is some part of it,

02:40:53.800 | and say it's very possible

02:40:57.920 | that you would be one of the first people

02:41:00.520 | that can sit down in the room,

02:41:02.960 | let's say with her and have a conversation,

02:41:06.200 | what would you ask her?

02:41:07.440 | What would you talk about?

02:41:09.280 | - Looking at all of the content

02:41:12.280 | out there on the web and so on,

02:41:18.960 | what are some possible solutions to big problems

02:41:26.600 | that the world has

02:41:29.960 | that people haven't really thought of before

02:41:33.160 | that are not being properly or at least adequately pursued?

02:41:38.160 | What are some novel solutions that you can think of

02:41:43.120 | that we haven't, that might work

02:41:46.480 | and that might be worth considering?

02:41:48.920 | - That is a damn good question.

02:41:50.800 | - Given that the AGI is going to be somewhat different

02:41:53.800 | from human intelligence,

02:41:56.040 | it's still going to make some mistakes

02:41:57.760 | that we wouldn't make,

02:41:59.000 | but it's also possibly going to notice

02:42:01.840 | some blind spots we have.

02:42:04.080 | And I would love, as a test of is it really

02:42:09.080 | on a par with our intelligences,

02:42:12.720 | can it help spot some of the blind spots that we have?

02:42:16.080 | - So the two-part question of can you help identify

02:42:21.280 | what are the big problems in the world

02:42:23.200 | and two, what are some novel solutions to those problems?

02:42:27.320 | - That are not being talked about by anyone.

02:42:30.040 | And some of those may become infeasible

02:42:35.280 | or reprehensible or something,

02:42:36.600 | but some of them might be actually great things to look at.

02:42:39.600 | If you go back and look at some of the most

02:42:42.840 | powerful discoveries that have been made,

02:42:45.840 | like relativity and superconductivity and so on,

02:42:50.520 | a lot of them were cases where someone took seriously

02:42:56.520 | the idea that there might actually be

02:42:59.720 | a non-obvious answer to a question.

02:43:04.480 | So in Einstein's case, it was,

02:43:06.680 | yeah, the Lorentz transformation is known,

02:43:09.400 | nobody believes that it's actually the way reality works,

02:43:12.100 | what if it were the way that reality actually worked?

02:43:15.240 | So a lot of people don't realize

02:43:16.920 | he didn't actually work out that equation,

02:43:18.480 | he just sort of took it seriously.

02:43:20.880 | Or in the case of superconductivity,

02:43:23.000 | you have this V equals IR equation

02:43:25.560 | where R is resistance and so on.

02:43:27.620 | And it was being mapped at lower and lower temperatures,

02:43:32.560 | but everyone thought that was just bump on a log research

02:43:35.400 | to show that V equals IR always held.

02:43:39.440 | And then when some graduate student

02:43:42.360 | got to a slightly lower temperature

02:43:45.760 | and showed that resistance suddenly dropped off,

02:43:47.720 | everyone just assumed that they did it wrong.

02:43:50.400 | And it was only a little while later

02:43:52.800 | that they realized it was actually a new phenomenon.

02:43:56.600 | Or in the case of the H. pylori bacteria

02:44:01.600 | causing stomach ulcers,

02:44:03.800 | where everyone thought that stress

02:44:05.300 | and stomach acid caused ulcers.

02:44:07.700 | And when a doctor in Australia

02:44:11.360 | claimed it was actually a bacterial infection,

02:44:15.480 | he couldn't get anyone seriously to listen to him

02:44:17.840 | and he had to ultimately inject himself with the bacteria

02:44:22.840 | to show that he suddenly developed a life-threatening ulcer

02:44:26.680 | in order to get other doctors to seriously consider that.

02:44:29.840 | So there are all sorts of things

02:44:31.400 | where humans are locked into paradigms,

02:44:34.960 | what Thomas Kuhn called paradigms,

02:44:37.080 | and we can't get out of them very easily.

02:44:40.440 | So a lot of AI is locked into the deep learning,

02:44:44.400 | machine learning paradigm right now.

02:44:47.400 | And almost all of us and almost all sciences

02:44:50.640 | are locked into current paradigms.

02:44:52.720 | And Kuhn's point was pretty much you have to wait

02:44:56.960 | for people to die in order for the new generation

02:45:01.240 | to escape those paradigms.

02:45:03.200 | And I think that one of the things

02:45:04.520 | that would change that sad reality

02:45:07.200 | is if we had trusted AGIs that could help take a step back

02:45:12.200 | and question some of the paradigms

02:45:15.920 | that we're currently locked into.

02:45:17.880 | - Yeah, it would accelerate the paradigm shifts

02:45:20.640 | in human science and progress.

02:45:23.440 | You've lived a very interesting life

02:45:28.080 | where you thought about big ideas and you stuck with them.

02:45:31.260 | Can you give advice to young people today,

02:45:34.920 | somebody in high school, somebody undergrad,

02:45:38.400 | about career, about life?

02:45:43.840 | - I'd say you can make a difference.

02:45:45.960 | But in order to make a difference,

02:45:49.040 | you're gonna have to have the courage

02:45:51.520 | to follow through with ideas

02:45:54.700 | which other people might not immediately understand

02:45:59.480 | or support.

02:46:02.400 | You have to realize that if you make some plan

02:46:06.400 | that's going to take an extended period of time

02:46:13.540 | to carry out, don't be afraid of that.

02:46:16.440 | That's true of physical training of your body.

02:46:20.680 | That's true of learning some profession.

02:46:25.680 | That's also true of innovation.

02:46:29.320 | That some innovations are not great ideas

02:46:32.440 | you can write down on a napkin

02:46:34.820 | and become an instant success if you turn out to be right.

02:46:38.360 | Some of them are paths you have to follow,

02:46:43.000 | but remember that you're mortal.

02:46:45.500 | Remember that you have a limited number

02:46:47.640 | of decade-sized debts to make with your life

02:46:52.640 | and you should make each one of them count.

02:46:55.600 | And that's true in personal relationships.

02:46:57.960 | That's true in career choice.

02:47:00.160 | That's true in making discoveries and so on.

02:47:03.780 | And if you follow the path of least resistance,

02:47:06.700 | you'll find that you're optimizing for short periods of time

02:47:12.600 | and before you know it, you turn around

02:47:14.560 | and long periods of time have gone by

02:47:17.000 | without you ever really making a difference in the world.

02:47:20.240 | - You know, when you look,

02:47:21.920 | I mean, the field that I really love

02:47:23.800 | is artificial intelligence and there's not many projects.

02:47:27.560 | There's not many little flames of hope

02:47:31.220 | that have been carried out for many years, for decades,

02:47:34.240 | and Psyche represents one of them.

02:47:36.560 | And I mean, that in itself is just a really inspiring thing.

02:47:42.400 | So I'm deeply grateful that you would be carrying that flame

02:47:46.600 | for so many years and I think that's an inspiration

02:47:48.660 | to young people.

02:47:49.880 | That said, you said life is finite

02:47:51.920 | and we talked about mortality as a feature of AGI.

02:47:55.240 | Do you think about your own mortality?

02:47:57.440 | Are you afraid of death?

02:47:58.640 | - Sure, I'd be crazy if I weren't.

02:48:02.880 | And as I get older, I'm now over 70.

02:48:07.440 | So as I get older, it's more on my mind,

02:48:11.000 | especially as acquaintances and friends

02:48:14.080 | and especially mentors, one by one are dying.

02:48:18.800 | So I can't avoid thinking about mortality.

02:48:22.680 | And I think that the good news from the point of view

02:48:26.720 | in the rest of the world is that that adds impetus

02:48:30.640 | to my need to succeed in a small number of years

02:48:33.920 | in the future.

02:48:34.760 | (Lex laughing)

02:48:35.600 | - You have a deadline.

02:48:36.800 | - Exactly, I'm not gonna have another 37 years

02:48:39.560 | to continue working on this.

02:48:41.440 | So we really do want psych to make an impact in the world,

02:48:46.360 | commercially, physically, metaphysically,

02:48:49.300 | in the next small number of years, two, three, five years,

02:48:53.680 | not two, three, five decades anymore.

02:48:56.320 | And so this is really driving me toward

02:48:59.840 | this sort of commercialization

02:49:02.680 | and increasingly widespread application of psych.

02:49:07.960 | Whereas before I felt that I could just sort of sit back,

02:49:11.600 | roll my eyes, wait till the world caught up.

02:49:14.480 | And now I don't feel that way anymore.

02:49:16.640 | I feel like I need to put in some effort

02:49:19.400 | to make the world aware of what we have and what it can do.

02:49:23.640 | And the good news from your point of view

02:49:25.160 | is that that's why I'm sitting here instead.

02:49:27.440 | - You're gonna be more productive.

02:49:28.680 | (Lex laughing)

02:49:30.680 | I love it.

02:49:31.520 | And if I can help in any way, I would love to,

02:49:34.200 | from a programmer perspective,

02:49:38.000 | I love, especially these days,

02:49:40.440 | just contributing in small and big ways.

02:49:42.720 | So if there's any open sourcing from an MIT side

02:49:46.080 | and the research, I would love to help.

02:49:48.520 | But bigger than psych, like I said,

02:49:51.680 | it's that little flame that you're carrying

02:49:53.360 | of artificial intelligence, the big dream.

02:49:55.840 | Is there, what do you hope your legacy is?

02:50:00.040 | - That's a good question.

02:50:04.280 | That people think of me as one of the pioneers

02:50:09.280 | or inventors of the AI that is ubiquitous

02:50:14.320 | and that they take for granted and so on.

02:50:18.560 | Much the way that today we look back

02:50:22.080 | on the pioneers of electricity

02:50:25.480 | or the pioneers of similar types of technologies and so on

02:50:31.120 | as it's hard to imagine what life would be like

02:50:35.480 | if these people hadn't done what they did.

02:50:39.920 | So that's one thing that I'd like to be remembered as.

02:50:43.680 | Another is that--

02:50:45.160 | - So the creator, one of the originators

02:50:47.800 | of this gigantic knowledge store and acquisition system

02:50:52.800 | that is likely to be at the center

02:50:57.760 | of whatever this future AI thing will look like.

02:51:00.480 | - Yes, exactly.

02:51:01.840 | And I'd also like to be remembered as someone

02:51:04.040 | who wasn't afraid to spend several decades

02:51:09.040 | on a project in a time when almost all of the other forces,

02:51:16.480 | institutional forces and commercial forces

02:51:25.480 | are incenting people to go for short-term rewards.

02:51:29.840 | - And a lot of people gave up.

02:51:31.440 | A lot of people that dreamt the same dream as you gave up.

02:51:36.440 | - Yes. - And you didn't.

02:51:38.040 | - Yes.

02:51:38.880 | - I mean, Doug, it's truly an honor.

02:51:42.680 | This was a long time coming.

02:51:45.200 | A lot of people bring up your work specifically

02:51:50.200 | and more broadly philosophically

02:51:52.320 | of this is the dream of artificial intelligence.

02:51:55.480 | This is likely a part of the future.

02:51:57.680 | We're so sort of focused on machine learning applications,

02:52:00.720 | all that kind of stuff today,

02:52:01.960 | but it seems like the ideas that Cy carries forward

02:52:04.880 | is something that will be at the center of this problem

02:52:09.480 | they're all trying to solve,

02:52:11.080 | which is the problem of intelligence,

02:52:13.720 | emotional and otherwise.

02:52:16.740 | So thank you so much.

02:52:18.320 | It's such a huge honor that you would talk to me

02:52:20.960 | and spend your valuable time with me today.

02:52:23.160 | Thanks for talking.

02:52:24.040 | - Thanks, Lex.

02:52:24.880 | It's been great.

02:52:26.480 | - Thanks for listening to this conversation

02:52:28.160 | with Doug Glennett.

02:52:29.440 | To support this podcast,

02:52:30.840 | please check out our sponsors in the description.

02:52:33.640 | And now let me leave you some words from Mark Twain

02:52:37.000 | about the nature of truth.

02:52:39.160 | If you tell the truth, you don't have to remember anything.

02:52:44.160 | Thank you for listening.

02:52:45.200 | I hope to see you next time.

02:52:46.960 | (upbeat music)

02:52:49.540 | (upbeat music)

02:52:52.120 | [BLANK_AUDIO]

Douglas Lenat: Cyc and the Quest to Solve Common Sense Reasoning in AI | Lex Fridman Podcast #221

Chapters