Judea Pearl: Causal Reasoning, Counterfactuals, and the Path to AGI

00:00:00.000 | The following is a conversation with Judea Pearl,

00:00:03.280 | professor at UCLA and a winner of the Turing Award

00:00:06.760 | that's generally recognized as the Nobel Prize of Computing.

00:00:10.720 | He's one of the seminal figures in the field

00:00:12.960 | of artificial intelligence, computer science, and statistics.

00:00:16.720 | He has developed and championed probabilistic approaches

00:00:20.000 | to AI, including Bayesian networks, and profound ideas

00:00:24.040 | and causality in general.

00:00:26.040 | These ideas are important not just to AI,

00:00:29.080 | but to our understanding and practice of science.

00:00:32.800 | But in the field of AI, the idea of causality, cause

00:00:36.160 | and effect, to many, lie at the core

00:00:39.400 | of what is currently missing and what

00:00:41.160 | must be developed in order to build truly intelligent

00:00:44.240 | systems.

00:00:46.080 | For this reason and many others, his work

00:00:48.480 | is worth returning to often.

00:00:50.720 | I recommend his most recent book called Book of Why,

00:00:54.200 | that presents key ideas from a lifetime of work

00:00:57.120 | in a way that is accessible to the general public.

00:01:00.400 | This is the Artificial Intelligence Podcast.

00:01:03.400 | If you enjoy it, subscribe on YouTube,

00:01:05.800 | give it five stars on Apple Podcasts, support on Patreon,

00:01:09.160 | or simply connect with me on Twitter, Alex Friedman,

00:01:12.440 | spelled F-R-I-D-M-A-N. If you leave a review on Apple

00:01:16.920 | Podcasts especially, but also CastBox or comment on YouTube,

00:01:20.920 | consider mentioning topics, people, ideas, questions,

00:01:23.920 | quotes, and science, tech, and philosophy

00:01:26.080 | you find interesting.

00:01:27.440 | And I'll read them on this podcast.

00:01:29.520 | I won't call out names, but I love comments

00:01:31.960 | with kindness and thoughtfulness in them,

00:01:33.840 | so I thought I'd share them with you.

00:01:35.760 | Someone on YouTube highlighted a quote

00:01:37.800 | from the conversation with Noam Chomsky,

00:01:40.000 | where he said that the significance of your life

00:01:42.760 | is something you create.

00:01:44.720 | I like this line as well.

00:01:46.520 | On most days, the existentialist approach to life

00:01:49.800 | is one I find liberating and fulfilling.

00:01:53.560 | I recently started doing ads at the end of the introduction.

00:01:56.560 | I'll do one or two minutes after introducing the episode

00:01:59.320 | and never any ads in the middle that break

00:02:01.480 | the flow of the conversation.

00:02:03.200 | I hope that works for you and doesn't

00:02:05.160 | hurt the listening experience.

00:02:08.200 | This show is presented by Cash App, the number one finance

00:02:11.600 | app in the App Store.

00:02:12.960 | I personally use Cash App to send money to friends,

00:02:15.440 | but you can also use it to buy, sell, and deposit

00:02:17.920 | Bitcoin in just seconds.

00:02:20.040 | Cash App also has a new investing feature.

00:02:22.720 | You can buy fractions of a stock, say $1 worth,

00:02:25.800 | no matter what the stock price is.

00:02:27.960 | Brokerage services are provided by Cash App Investing,

00:02:30.640 | a subsidiary of Square, a member of SIPC.

00:02:34.160 | I'm excited to be working with Cash App

00:02:36.560 | to support one of my favorite organizations called First,

00:02:39.880 | best known for their FIRST Robotics and LEGO competitions.

00:02:43.360 | They educate and inspire hundreds of thousands

00:02:46.200 | of students in over 110 countries

00:02:48.920 | and have a perfect rating and charity navigator,

00:02:51.680 | which means the donated money is used

00:02:53.560 | to the maximum effectiveness.

00:02:55.920 | When you get Cash App from the App Store or Google Play

00:02:58.600 | and use code LEXPODCAST, you'll get $10,

00:03:02.640 | and Cash App will also donate $10 to FIRST, which again,

00:03:06.480 | is an organization that I've personally

00:03:08.280 | seen inspire girls and boys to dream

00:03:10.600 | of engineering a better world.

00:03:12.680 | And now, here's my conversation with Judea Pearl.

00:03:18.000 | You mentioned in an interview that science is not

00:03:20.760 | a collection of facts, but a constant human struggle

00:03:24.000 | with the mysteries of nature.

00:03:26.720 | What was the first mystery that you can recall

00:03:29.240 | that hooked you, that kept you curious?

00:03:30.760 | - Oh, the first mystery, that's a good one.

00:03:34.480 | Yeah, I remember that.

00:03:37.840 | - What was it? - I had a fever for three days.

00:03:41.480 | And when I learned about Descartes, analytic geometry,

00:03:47.080 | and I found out that you can do all the construction

00:03:49.880 | in geometry using algebra.

00:03:52.920 | And I couldn't get over it.

00:03:54.520 | I simply couldn't get out of bed.

00:03:58.280 | - So what kind of world does analytic geometry unlock?

00:04:02.800 | - Well, it connects algebra with geometry.

00:04:07.360 | Okay, so Descartes had the idea that geometrical construction

00:04:12.960 | and geometrical theorems and assumptions

00:04:16.240 | can be articulated in the language of algebra,

00:04:19.600 | which means that all the proof that we did in high school

00:04:24.880 | and trying to prove that the three bisectors meet

00:04:28.920 | at one point and that, okay, all this can be proven

00:04:35.600 | by just shuffling around notation.

00:04:39.480 | Yeah, that was a traumatic experience.

00:04:43.600 | - Traumatic experience. - For me, it was.

00:04:45.200 | I'm telling you, right? - So it's the connection

00:04:47.000 | between the different mathematical disciplines

00:04:49.160 | that they all-- - Not within two languages.

00:04:53.000 | - Languages. - Yeah.

00:04:54.440 | - So which mathematic discipline is most beautiful?

00:04:57.200 | Is geometry it for you?

00:04:58.600 | - Both are beautiful.

00:04:59.480 | They have almost the same power.

00:05:02.440 | - But there's a visual element to geometry being--

00:05:04.920 | - Visual, it's more transparent.

00:05:08.120 | But once you get over to algebra,

00:05:10.680 | then a linear equation is a straight line.

00:05:14.440 | This translation is easily absorbed.

00:05:18.200 | And to pass a tangent to a circle,

00:05:22.800 | you know, you have the basic theorems

00:05:25.520 | and you can do it with algebra.

00:05:27.520 | So but the transition from one to another was really,

00:05:31.560 | I thought that Descartes was the greatest mathematician

00:05:34.160 | of all times.

00:05:35.200 | - So you have been at the, if you think of engineering

00:05:40.840 | and mathematics as a spectrum.

00:05:43.240 | - Yes.

00:05:44.080 | - You have been, you have walked casually

00:05:47.360 | along this spectrum throughout your life.

00:05:51.520 | You know, a little bit of engineering

00:05:53.000 | and then, you know, done a little bit

00:05:57.320 | of mathematics here and there.

00:05:58.760 | - Not a little bit.

00:05:59.600 | I mean, we got a very solid background in mathematics

00:06:04.080 | because our teachers were geniuses.

00:06:07.120 | Our teachers came from Germany in the 1930s,

00:06:09.760 | running away from Hitler.

00:06:12.320 | They left their careers in Heidelberg and Berlin

00:06:15.080 | and came to teach high school in Israel.

00:06:17.880 | And we were the beneficiary of that experiment.

00:06:20.880 | So I, and they taught us math the good way.

00:06:25.240 | - What's a good way to teach math?

00:06:26.760 | - Chronologically.

00:06:27.760 | - The people.

00:06:29.920 | - The people behind the theorems, yeah.

00:06:33.360 | Their cousins and their nieces and their faces.

00:06:37.800 | And how they jumped from the bathtub

00:06:40.960 | when they scream, "Eureka!"

00:06:42.440 | (laughs)

00:06:43.520 | And ran naked in town.

00:06:45.200 | - So you're almost educated as a historian of math.

00:06:49.280 | - No, we just got a glimpse of that history

00:06:51.960 | together with a theorem.

00:06:53.800 | So every exercise in math was connected with a person.

00:06:58.240 | And the time of the person.

00:07:01.000 | The period.

00:07:03.520 | - The period also mathematically speaking.

00:07:05.560 | - Mathematically speaking, yes.

00:07:06.880 | Not the politics.

00:07:08.040 | - Yeah. - No.

00:07:09.040 | - So, and then in university,

00:07:14.000 | you have gone on to do engineering.

00:07:16.240 | - Yeah.

00:07:17.200 | I get a BS in engineering in Technion.

00:07:19.200 | And then I moved here for graduate work.

00:07:25.600 | And I got to, I did engineering

00:07:27.920 | in addition to physics in Rutgers.

00:07:31.920 | And it combined very nicely with my thesis

00:07:35.840 | which I did in LCA Laboratories in superconductivity.

00:07:39.240 | - And then somehow thought to switch

00:07:43.760 | to almost computer science software.

00:07:46.840 | Not switch, but long to become,

00:07:51.040 | to get into software engineering a little bit.

00:07:53.000 | - Yes. - Programming.

00:07:54.480 | If you can call it that in the '70s.

00:07:56.200 | So there's all these disciplines.

00:07:58.160 | - Yeah.

00:07:59.000 | - If you were to pick a favorite,

00:08:00.360 | in terms of engineering and mathematics,

00:08:03.880 | which path do you think has more beauty?

00:08:07.120 | Which path has more power?

00:08:08.600 | - It's hard to choose, no.

00:08:10.560 | I enjoy doing physics.

00:08:12.600 | I even have a vortex named on my name.

00:08:16.200 | So I have investment in immortality.

00:08:21.200 | (both laughing)

00:08:23.360 | - So what is a vortex?

00:08:25.160 | - Vortex is in superconductivity.

00:08:27.000 | - In the superconductivity, yeah.

00:08:27.840 | - You have permanent current swirling around.

00:08:30.880 | One way or the other, you can have a store

00:08:32.680 | one or zero for computer.

00:08:35.360 | That's what we worked on in the 1960s in RCA.

00:08:39.680 | And I discovered a few nice phenomena with the vortices.

00:08:44.120 | You push current and they move.

00:08:44.960 | - So that's a pearl vortex.

00:08:46.600 | - Pearl vortex, right, you can Google it.

00:08:49.080 | Right?

00:08:50.240 | I didn't know about it, but the physicist

00:08:53.000 | picked up on my thesis, on my PhD thesis,

00:08:57.200 | and it becomes popular.

00:09:01.360 | I mean, thin film superconductors became important

00:09:04.760 | for high temperature superconductors.

00:09:06.920 | So they called it pearl vortex without my knowledge.

00:09:10.840 | I discovered it only about 15 years ago.

00:09:13.800 | - You have footprints in all of the sciences.

00:09:17.560 | So let's talk about the universe a little bit.

00:09:20.960 | Is the universe at the lowest level deterministic

00:09:23.880 | or stochastic in your amateur philosophy view?

00:09:27.400 | Put another way, does God play dice?

00:09:30.120 | - We know it is stochastic, right?

00:09:33.040 | - Today, today we think it is stochastic.

00:09:35.200 | - Yes.

00:09:36.040 | We think because we have the Heisenberg

00:09:38.880 | uncertainty principle and we have some experiments

00:09:42.720 | to confirm that.

00:09:44.680 | - All we have is experiments to confirm it.

00:09:47.960 | We don't understand why.

00:09:49.240 | - Why is already--

00:09:51.440 | - You wrote a book about why.

00:09:53.080 | (both laughing)

00:09:55.080 | - Yeah, it's a puzzle.

00:09:57.240 | It's a puzzle that you have the dice flipping machine,

00:10:02.240 | or God,

00:10:03.200 | and the result of the flipping propagate

00:10:09.160 | with a speed faster than the speed of light.

00:10:12.280 | We can't explain it, okay?

00:10:14.240 | So, but it only governs microscopic phenomena.

00:10:19.240 | - So you don't think of quantum mechanics as useful

00:10:25.680 | for understanding the nature of reality?

00:10:28.240 | - No, it's diversionary.

00:10:30.400 | - So in your thinking,

00:10:33.200 | the world might as well be deterministic.

00:10:36.000 | - The world is deterministic,

00:10:38.480 | and as far as the neuron firing is concerned,

00:10:42.800 | it is deterministic to first approximation.

00:10:47.240 | - What about free will?

00:10:48.920 | - Free will is also a nice exercise.

00:10:52.960 | Free will is an illusion that we AI people are gonna solve.

00:10:57.960 | - So what do you think, once we solve it,

00:11:01.800 | that solution will look like?

00:11:03.360 | Once we put it in the page.

00:11:04.200 | - The solution will look like,

00:11:06.240 | first of all, it will look like a machine.

00:11:08.920 | A machine that act as though it has free will.

00:11:12.560 | It communicates with other machines

00:11:14.760 | as though they have free will,

00:11:17.160 | and you wouldn't be able to tell the difference

00:11:19.480 | between a machine that does

00:11:21.560 | and a machine that doesn't have free will.

00:11:23.720 | - So the illusion, it propagates the illusion of free will

00:11:27.520 | amongst the other machines.

00:11:29.000 | - And faking it is having it.

00:11:31.880 | Okay, that's what Turing test is all about.

00:11:35.200 | Faking intelligence is intelligent

00:11:37.200 | because it's not easy to fake.

00:11:41.120 | It's very hard to fake,

00:11:43.280 | and you can only fake if you have it.

00:11:45.120 | (Lex laughing)

00:11:47.700 | - That's such a beautiful statement.

00:11:51.320 | (Lex laughing)

00:11:54.040 | - Yeah, you can't fake it if you don't have it.

00:11:57.600 | So let's begin at the beginning with probability,

00:12:04.520 | both philosophically and mathematically.

00:12:09.360 | What does it mean to say the probability

00:12:11.540 | of something happening is 50%?

00:12:15.160 | What is probability?

00:12:16.960 | - It's a degree of uncertainty that an agent has

00:12:20.640 | about the world.

00:12:22.400 | - You're still expressing some knowledge in that statement.

00:12:24.720 | - Of course.

00:12:25.560 | If the probability is 90%,

00:12:27.840 | it's absolutely a different kind of knowledge

00:12:29.760 | than if it is 10%.

00:12:32.480 | - But it's still not solid knowledge.

00:12:35.640 | - It is solid knowledge, but hey,

00:12:38.520 | if you tell me that 90% assurance smoking

00:12:43.520 | will give you lung cancer in five years

00:12:47.600 | versus 10%, it's a piece of useful knowledge.

00:12:52.400 | - So the statistical view of the universe,

00:12:56.120 | why is it useful?

00:12:57.620 | So we're swimming in complete uncertainty,

00:13:00.760 | most of everything around us.

00:13:01.600 | - It allows you to predict things with a certain probability

00:13:06.120 | and computing those probabilities are very useful.

00:13:09.240 | That's the whole idea of prediction,

00:13:15.080 | and you need prediction to be able to survive.

00:13:18.160 | If you can't predict the future,

00:13:19.680 | then you're just crossing the street

00:13:22.360 | will be extremely fearful.

00:13:25.060 | - And so you've done a lot of work in causation,

00:13:28.880 | and so let's think about correlation.

00:13:32.160 | - I started with probability.

00:13:34.360 | - You started with probability.

00:13:35.680 | You've invented the Bayesian networks.

00:13:38.800 | - Yeah.

00:13:39.640 | - And so we'll dance back and forth

00:13:43.920 | between these levels of uncertainty.

00:13:47.520 | But what is correlation?

00:13:49.360 | What is it, so probability of something happening

00:13:53.180 | is something, but then there's a bunch of things happening,

00:13:56.500 | and sometimes they happen together, sometimes not.

00:13:59.560 | They're independent or not.

00:14:00.800 | So how do you think about correlation of things?

00:14:03.660 | - Correlation occurs when two things vary together

00:14:06.320 | over a very long time.

00:14:07.840 | There's one way of measuring it.

00:14:09.720 | Or when you have a bunch of variables

00:14:11.840 | that they all vary cohesively,

00:14:14.560 | then we call it, we have a correlation here.

00:14:18.600 | And usually when we think about correlation,

00:14:21.720 | we really think causally.

00:14:24.440 | Things cannot be correlated unless there is a reason

00:14:27.960 | for them to vary together.

00:14:30.280 | Why should they vary together?

00:14:32.080 | If they don't see each other,

00:14:33.380 | why should they vary together?

00:14:35.560 | - So underlying it somewhere is causation.

00:14:38.240 | - Yes.

00:14:39.240 | - But hidden in our intuition,

00:14:40.960 | there is a notion of causation

00:14:43.200 | because we cannot grasp any other logic except causation.

00:14:48.200 | - And how does conditional probability

00:14:52.520 | differ from causation?

00:14:54.540 | So what is conditional probability?

00:14:57.960 | - Conditional probability, how things vary

00:15:00.580 | when one of them stays the same.

00:15:05.040 | Now staying the same means that I have chosen

00:15:09.320 | to look only at those incidents

00:15:11.720 | where the guy has the same value as the previous one.

00:15:16.160 | It's my choice as an experimenter.

00:15:19.300 | So things that are not correlated before

00:15:22.260 | could become correlated.

00:15:24.260 | Like for instance, if I have two coins

00:15:26.860 | which are uncorrelated, okay,

00:15:29.240 | and I choose only those flippings experiments

00:15:33.760 | in which a bell rings,

00:15:35.620 | and the bell rings when at least one of them is a tail,

00:15:40.000 | okay, then suddenly I see correlation

00:15:42.560 | between the two coins

00:15:44.360 | because I only look at the cases where the bell rang.

00:15:48.420 | You see, it's my design,

00:15:51.400 | with my ignorance essentially,

00:15:53.680 | with my audacity to ignore certain incidents,

00:16:01.280 | I suddenly create a correlation

00:16:04.720 | where it doesn't exist physically.

00:16:06.760 | - Right, so that's, you just outlined one of the flaws

00:16:11.400 | of observing the world

00:16:13.040 | and trying to infer something from the math about the world

00:16:16.080 | from looking at the correlation.

00:16:17.520 | - I don't look at it as a flaw,

00:16:18.880 | the world works like that.

00:16:20.380 | But the flaws come if we try to impose

00:16:27.920 | causal logic on correlation,

00:16:32.280 | it doesn't work too well.

00:16:34.720 | - I mean, but that's exactly what we do,

00:16:36.320 | that's what, that has been the majority of science.

00:16:40.040 | - The majority of naive science.

00:16:42.660 | Statisticians know it, statisticians know it,

00:16:46.240 | if you condition on a third variable,

00:16:49.480 | then you can destroy or create correlations

00:16:53.800 | among two other variables.

00:16:55.640 | They know it, it's in the data.

00:16:57.880 | - Right.

00:16:58.720 | - There's nothing surprising,

00:16:59.560 | that's why they all dismiss the Simpson Paradox,

00:17:02.400 | ah, we know it, they don't know anything about it.

00:17:05.720 | - Well, there's disciplines like psychology

00:17:09.680 | where all the variables are hard to account for,

00:17:12.900 | and so oftentimes there's a leap

00:17:15.240 | between correlation to causation.

00:17:17.520 | You're imposing--

00:17:18.720 | - What do you mean, a leap?

00:17:20.080 | Who is trying to get causation from correlation?

00:17:25.640 | - Not, you're not proving causation,

00:17:27.960 | but you're sort of discussing it,

00:17:31.720 | implying, sort of hypothesizing with our ability--

00:17:35.320 | - Which discipline you have in mind?

00:17:37.080 | I'll tell you if they are obsolete,

00:17:40.480 | or if they are outdated, or they're about to get outdated.

00:17:44.240 | - Yes, yes.

00:17:45.480 | - Tell me which one you have in mind.

00:17:46.760 | - Oh, psychology, you know.

00:17:48.200 | - Psychology, what, is it SEM, Structural Equation?

00:17:50.800 | - No, no, I was thinking of applied psychology studying,

00:17:54.320 | for example, we work with human behavior

00:17:57.240 | in semi-autonomous vehicles, how people behave,

00:18:00.360 | and you have to conduct these studies

00:18:02.600 | of people driving cars.

00:18:03.960 | - Everything starts with a question.

00:18:05.520 | What is the research question?

00:18:07.840 | - What is the research question?

00:18:09.480 | The research question, do people fall asleep

00:18:14.340 | when the car is driving itself?

00:18:17.560 | - Do they fall asleep, or do they tend to fall asleep

00:18:22.280 | more frequently--

00:18:23.160 | - More frequently.

00:18:24.000 | - When the car not driving itself.

00:18:25.680 | - Not driving itself.

00:18:26.520 | - That's a good question, okay.

00:18:28.720 | - And so you measure, you put people in the car,

00:18:32.480 | because it's real world, you can't conduct an experiment

00:18:35.200 | where you control everything.

00:18:36.320 | - Why can't you--

00:18:37.840 | - You could.

00:18:38.680 | - Turn the automatic module on and off?

00:18:43.680 | - Because it's on-road public, I mean,

00:18:48.160 | there's aspects to it that's unethical,

00:18:52.680 | because it's testing on public roads.

00:18:54.920 | So you can only use vehicle, they have to,

00:18:57.580 | the people, the drivers themselves

00:19:00.240 | have to make that choice themselves.

00:19:02.840 | And so they regulate that.

00:19:04.420 | So you just observe when they drive it autonomously

00:19:09.040 | and when they don't.

00:19:10.360 | And then--

00:19:11.200 | - But maybe they turn it off when they're very tired.

00:19:13.160 | - Yeah, that kind of thing.

00:19:14.560 | But you don't know those variables.

00:19:16.600 | - Okay, so that you have now uncontrolled experiment.

00:19:19.480 | - Uncontrolled experiment.

00:19:20.720 | We call it observational study.

00:19:23.240 | And we form the correlation, detected,

00:19:27.200 | we have to infer causal relationship.

00:19:30.400 | Whether it was the automatic piece

00:19:33.480 | that caused them to fall asleep, or, okay.

00:19:36.000 | So that is an issue that is about 120 years old.

00:19:41.000 | I should only go 100 years old, okay?

00:19:49.480 | And-- - Let's count.

00:19:51.400 | - Oh, maybe it's not, actually I should say

00:19:53.880 | it's 2,000 years old,

00:19:55.240 | because we have this experiment by Daniel.

00:19:58.520 | But the Babylonian king that wanted the exile,

00:20:03.520 | the people from Israel that were taken in exile

00:20:12.400 | to Babylon to serve the king,

00:20:14.680 | he wanted to serve them king's food,

00:20:17.240 | which was meat, and Daniel, as a good Jew,

00:20:20.360 | couldn't eat non-kosher food,

00:20:22.760 | so he asked them to eat vegetarian food.

00:20:26.640 | But the king overseer says, "I'm sorry,

00:20:29.240 | "but if the king sees that your performance

00:20:33.000 | "falls below that of other kids,

00:20:37.400 | "he's going to kill me."

00:20:39.360 | Daniel said, "Let's make an experiment.

00:20:41.520 | "Let's take four of us from Jerusalem, okay?

00:20:44.240 | "Give us vegetarian food.

00:20:46.340 | "Let's take the other guys to eat the king's food,

00:20:50.200 | "and in about a week's time, we'll test our performance."

00:20:54.080 | And you know the answer.

00:20:55.440 | Of course, he did the experiment,

00:20:57.800 | and they were so much better than the others,

00:21:02.120 | and the king nominated them to super position in his case.

00:21:07.120 | So it was the first experiment, yes.

00:21:10.160 | So there was a very simple,

00:21:12.760 | it's also the same research questions.

00:21:15.520 | We want to know if vegetarian food

00:21:17.440 | assists or obstructs your mental ability.

00:21:22.840 | Okay, so the question is very old.

00:21:29.160 | Even Democritus said,

00:21:32.980 | if I could discover one cause of things,

00:21:39.280 | I would rather discover one cause

00:21:41.480 | than be a king of Persia.

00:21:45.060 | The task of discovering causes

00:21:48.400 | was in the mind of ancient people

00:21:50.880 | from many, many years ago,

00:21:53.480 | but the mathematics of doing that

00:21:57.360 | was only developed in the 1920s.

00:22:00.480 | So science has left us orphaned, okay?

00:22:05.080 | Science has not provided us with the mathematics

00:22:08.320 | to capture the idea of X causes Y,

00:22:12.000 | and Y does not cause X,

00:22:14.320 | 'cause all the questions of physics

00:22:16.540 | are symmetrical, algebraic.

00:22:18.620 | The equality sign goes both ways.

00:22:20.640 | - Okay, let's look at machine learning.

00:22:23.100 | Machine learning today,

00:22:24.980 | if you look at deep neural networks,

00:22:26.860 | you can think of it as a kind of

00:22:30.700 | conditional probability estimators.

00:22:33.660 | - Correct, beautiful.

00:22:35.540 | - So-- - Where did you say that?

00:22:38.100 | Conditional probability estimators.

00:22:41.520 | - None of the machine learning people clobbered you?

00:22:44.420 | Attacked you?

00:22:46.180 | (both laughing)

00:22:48.840 | - Most people, and this is why today's conversation,

00:22:52.380 | I think, is interesting,

00:22:53.220 | is most people would agree with you.

00:22:55.780 | There's certain aspects that are just effective today,

00:22:58.660 | but we're going to hit a wall,

00:23:00.220 | and there's a lot of ideas,

00:23:02.420 | I think you're very right,

00:23:03.540 | that we're gonna have to return to,

00:23:05.420 | about causality.

00:23:06.460 | Let's try to explore it.

00:23:10.940 | - Okay. - Let's even take a step back.

00:23:13.160 | You've invented Bayesian networks

00:23:15.200 | that look awfully a lot like they express

00:23:20.960 | something like causation, but they don't, not necessarily.

00:23:24.000 | So how do we turn Bayesian networks

00:23:28.600 | into expressing causation?

00:23:30.840 | How do we build causal networks?

00:23:33.160 | This A causes B, B causes C,

00:23:36.480 | how do we start to infer that kind of thing?

00:23:38.840 | - We start asking ourselves questions.

00:23:41.480 | What are the factors that would determine

00:23:44.540 | the value of X?

00:23:46.340 | X could be blood pressure, death,

00:23:50.220 | hunger.

00:23:52.260 | - But these are hypotheses that we propose.

00:23:55.980 | - Hypothesis, everything which has to do with causality

00:23:59.060 | comes from a theory.

00:24:00.780 | The difference is only how you interrogate

00:24:06.980 | the theory that you have in your mind.

00:24:09.080 | - So it still needs the human expert to propose.

00:24:13.820 | - Right, you need the human expert to specify

00:24:18.740 | the initial model.

00:24:20.980 | Initial model could be very qualitative.

00:24:24.020 | Just who listens to whom?

00:24:27.020 | By whom listen to, I mean one variable listens to the other.

00:24:31.260 | So I say, okay, the tide is listening to the moon,

00:24:36.100 | and not to the rooster crow.

00:24:40.820 | And so forth.

00:24:43.140 | This is our understanding of the world in which we live.

00:24:46.040 | Scientific understanding of reality.

00:24:50.380 | We have to start there,

00:24:53.500 | because if we don't know how to handle

00:24:56.980 | cause and effect relationship,

00:24:58.580 | when we do have a model,

00:25:01.260 | and we certainly do not know how to handle it

00:25:03.740 | when we don't have a model.

00:25:05.460 | So let's start first.

00:25:07.260 | In AI, slogan is representation first, discovery second.

00:25:12.180 | But if I give you all the information that you need,

00:25:17.260 | can you do anything useful with it?

00:25:19.860 | That is the first, representation.

00:25:21.540 | How do you represent it?

00:25:22.580 | I give you all the knowledge in the world.

00:25:24.620 | How do you represent it?

00:25:25.820 | When you represent it, I ask you,

00:25:30.740 | can you infer X or Y or Z?

00:25:33.260 | Can you answer certain queries?

00:25:35.300 | Is it complex?

00:25:36.940 | Is it polynomial?

00:25:38.060 | All the computer science exercises we do

00:25:42.060 | once you give me a representation for my knowledge.

00:25:47.060 | Then you can ask me, now I understand

00:25:50.060 | how to represent things, how do I discover them?

00:25:52.780 | It's a secondary thing.

00:25:54.780 | - First of all, I should echo the statement

00:25:57.060 | that mathematics and the current,

00:25:59.820 | much of the machine learning world

00:26:02.620 | has not considered causation, that A causes B.

00:26:06.260 | Just in anything, that seems like a non-obvious thing

00:26:11.260 | that you think we would have really acknowledged it,

00:26:18.260 | but we haven't.

00:26:19.200 | So we have to put that on the table.

00:26:21.060 | So knowledge, how hard is it to create a knowledge

00:26:26.060 | from which to work?

00:26:28.460 | - In certain area, it's easy

00:26:31.260 | because we have only four or five major variables.

00:26:36.060 | And an epidemiologist or an economist can put them down.

00:26:41.560 | What, minimum wage, unemployment policy, X, Y, Z,

00:26:48.420 | and start collecting data and quantify the parameter

00:26:57.140 | that were left unquantified with the initial knowledge.

00:27:01.500 | That's the routine work that you find

00:27:06.500 | in experimental psychology, in economics,

00:27:12.020 | everywhere, in the health science, that's a routine thing.

00:27:16.540 | But I should emphasize,

00:27:18.740 | you should start with the research question,

00:27:21.180 | what do you want to estimate?

00:27:24.860 | Once you have that, you have to have a language

00:27:27.460 | of expressing what you want to estimate.

00:27:30.160 | You think it's easy?

00:27:31.620 | No.

00:27:32.740 | - So we can talk about two things, I think.

00:27:35.780 | One is how the science of causation is very useful

00:27:40.780 | for answering certain questions.

00:27:47.380 | And then the other is how do we create intelligence systems

00:27:50.300 | that need to reason with causation?

00:27:53.580 | So if my research question is how do I pick up

00:27:56.180 | this water bottle from the table,

00:27:58.660 | all the knowledge that is required to be able to do that,

00:28:05.280 | how do we construct that knowledge base?

00:28:07.980 | Do we return back to the problem that we didn't solve

00:28:11.980 | in the '80s with expert systems?

00:28:13.580 | Do we have to solve that problem

00:28:15.440 | of automated construction of knowledge?

00:28:19.660 | - You're talking about the task

00:28:23.560 | of eliciting knowledge from an expert.

00:28:26.580 | - Task of eliciting knowledge from an expert,

00:28:28.500 | or the self-discovery of more knowledge,

00:28:31.540 | more and more knowledge.

00:28:34.260 | So automating the building of knowledge as much as possible.

00:28:38.620 | - It's a different game in the causal domain,

00:28:42.420 | because it's essentially the same thing.

00:28:46.460 | You have to start with some knowledge,

00:28:48.700 | and you're trying to enrich it.

00:28:51.500 | But you don't enrich it by asking for more rules.

00:28:56.500 | You enrich it by asking for the data,

00:28:58.980 | to look at the data and quantifying and ask queries

00:29:02.820 | that you couldn't answer when you started.

00:29:05.500 | You couldn't because the question is quite complex,

00:29:11.500 | and it's not within the capability of ordinary cognition.

00:29:16.900 | Of ordinary person, ordinary expert even, to answer.

00:29:21.900 | - So what kind of questions do you think

00:29:24.940 | we can start to answer?

00:29:26.980 | - Even a simple one.

00:29:27.860 | Suppose, yeah, I start with easy one.

00:29:31.260 | - Let's do it.

00:29:32.100 | - Okay, what's the effect of a drug on recovery?

00:29:35.920 | What is the aspirin that caused my headache to be cured?

00:29:42.340 | Or what is the television program?

00:29:44.660 | Or the good news I received?

00:29:46.300 | This is already, you see, it's a difficult question

00:29:49.940 | because it's find the cause from effect.

00:29:52.820 | The easy one is find the effect from cause.

00:29:55.860 | - That's right.

00:29:57.740 | So first you construct a model saying

00:29:59.460 | that this is an important research question.

00:30:01.260 | This is an important question.

00:30:02.820 | Then you--

00:30:03.660 | - No, I didn't construct a model yet.

00:30:05.540 | I just said it's an important question.

00:30:07.140 | - It's an important question.

00:30:07.980 | - And the first exercise is express it mathematically.

00:30:12.300 | What do you want to?

00:30:13.820 | Like, if I tell you what will be the effect

00:30:17.000 | of taking this drug?

00:30:18.740 | Okay, you have to say that in mathematics.

00:30:21.340 | How do you say that?

00:30:22.900 | - Yes.

00:30:23.740 | - Can you write down the question?

00:30:25.420 | Not the answer.

00:30:26.400 | I want to find the effect of the drug on my headache.

00:30:32.420 | - Right.

00:30:33.260 | - Write it down.

00:30:34.100 | Write it down.

00:30:34.920 | - That's where the do calculus comes in.

00:30:35.940 | - Yes.

00:30:36.780 | Do operator, what is do operator?

00:30:38.260 | - Do operator, yeah.

00:30:39.100 | - Yeah.

00:30:39.940 | - Which is nice.

00:30:40.760 | It's the difference between association and intervention.

00:30:43.300 | Very beautifully sort of constructed.

00:30:45.740 | - Yeah, so we have a do operator.

00:30:48.900 | So do calculus connected on the do operator itself

00:30:52.560 | connects the operation of doing

00:30:55.560 | to something that we can see.

00:30:57.560 | - So as opposed to the purely observing,

00:31:01.740 | you're making the choice to change a variable.

00:31:05.900 | - That's what it expresses.

00:31:08.220 | And then the way that we interpret it,

00:31:11.860 | and the mechanism by which we take your query

00:31:15.420 | and we translate it into something that we can work with

00:31:18.620 | is by giving it semantics.

00:31:21.060 | Saying that you have a model of the world

00:31:23.340 | and you cut off all the incoming error into x.

00:31:26.820 | And you're looking now in the modified mutilated model,

00:31:30.700 | you ask for the probability of y.

00:31:33.660 | That is interpretation of doing x.

00:31:36.380 | Because by doing things, you liberate them

00:31:40.220 | from all influences that acted upon them earlier.

00:31:45.220 | And you subject them to the tyranny of your muscles.

00:31:49.180 | - So you remove all the questions about causality

00:31:54.060 | by doing them.

00:31:55.780 | - No, because there's one level of questions.

00:31:59.020 | Answer questions about what will happen if you do things.

00:32:01.940 | - If you do, if you drink the coffee,

00:32:03.320 | if you take the aspirin.

00:32:04.160 | - Right.

00:32:05.340 | - So how do we get the doing data

00:32:10.100 | from the-- - Ah, now the question is,

00:32:12.600 | if we cannot run experiments, right,

00:32:16.540 | then we have to rely on observational study.

00:32:20.980 | - So first we could, sorry to interrupt,

00:32:22.560 | we could run an experiment.

00:32:23.900 | - Yeah.

00:32:24.740 | - Where we do something, where we drink the coffee

00:32:26.940 | and this, the do operator allows you

00:32:29.780 | to sort of be systematic about expressing.

00:32:31.780 | - To imagine how the experiment will look like

00:32:34.580 | even though we cannot physically

00:32:36.860 | and technologically conduct it.

00:32:38.780 | I'll give you an example.

00:32:40.620 | What is the effect of blood pressure on mortality?

00:32:43.680 | I cannot go down into your vein

00:32:47.340 | and change your blood pressure.

00:32:49.400 | But I can ask the question.

00:32:50.800 | Which means I can, if I have a model of your body,

00:32:55.100 | I can imagine the effect of your,

00:32:58.620 | how the blood pressure change will affect your mortality.

00:33:04.700 | How I go into the model and I conduct this surgery

00:33:09.700 | about the blood pressure,

00:33:12.060 | even though physically I can do, I cannot do it.

00:33:15.840 | - Let me ask the quantum mechanics question.

00:33:19.740 | Does the doing change the observation?

00:33:22.200 | Meaning the surgery of changing the blood pressure is,

00:33:28.260 | I mean-- - No, the surgery is,

00:33:31.420 | it's called very delicate.

00:33:35.460 | - It's very delicate, infinitely delicate.

00:33:37.820 | - Incisive and delicate, which means,

00:33:40.780 | do means, do x means I'm gonna touch only x.

00:33:45.780 | - Only x. - Directly into x.

00:33:48.780 | So that means that I change only things

00:33:52.820 | which depends on x by virtue of x changing.

00:33:56.820 | But I don't depend things which are not depends on x.

00:34:00.420 | Like I wouldn't change your sex or your age,

00:34:04.260 | I just change your blood pressure.

00:34:06.080 | - So in the case of blood pressure,

00:34:08.740 | it may be difficult or impossible

00:34:11.180 | to construct such an experiment.

00:34:12.820 | - No, physically, yes.

00:34:14.940 | But hypothetically, no. - Hypothetically, no.

00:34:17.380 | - If we have a model, that is what the model is for.

00:34:20.740 | So you conduct surgeries on a model,

00:34:24.620 | you take it apart, put it back, that's the idea of a model.

00:34:28.860 | It's the idea of thinking counterfactually, imagining,

00:34:31.620 | and that's the idea of creativity.

00:34:35.140 | - So by constructing that model, you can start to infer

00:34:37.940 | if the higher, the blood pressure leads to mortality,

00:34:42.940 | which increases or decreases by--

00:34:47.340 | - I construct a model, I still cannot answer it.

00:34:50.780 | I have to see if I have enough information in the model

00:34:53.820 | that would allow me to find out the effects of intervention

00:34:58.340 | from a non-interventional study,

00:35:00.540 | from observation, hands-off study.

00:35:03.580 | - So what's needed to make that--

00:35:06.340 | - You need to have assumptions about who affects whom.

00:35:11.340 | If the graph had a certain property,

00:35:16.380 | the answer is yes, you can get it from observational study.

00:35:20.540 | If the graph is too meshy, bushy, bushy,

00:35:23.740 | the answer is no, you cannot.

00:35:25.680 | Then you need to find either different kind of observation

00:35:30.680 | that you haven't considered, or one experiment.

00:35:34.060 | - So basically, that puts a lot of pressure on you

00:35:38.860 | to encode wisdom into that graph.

00:35:41.900 | - Correct.

00:35:42.940 | But you don't have to encode more than what you know.

00:35:47.500 | God forbid, if you put, like economists are doing this,

00:35:51.380 | they call it identifying assumptions.

00:35:52.860 | They put assumptions, even if they don't prevail in the world

00:35:56.040 | they put assumptions so they can identify things.

00:35:59.260 | - But the problem is, yes, beautifully put,

00:36:01.500 | but the problem is you don't know what you don't know.

00:36:04.460 | So--

00:36:05.940 | - You know what you don't know,

00:36:07.540 | because if you don't know, you say it's possible,

00:36:10.620 | it's possible that X affect the traffic tomorrow.

00:36:15.620 | It's possible.

00:36:18.660 | You put down an arrow which says it's possible.

00:36:20.900 | Every arrow in the graph says it's possible.

00:36:23.940 | - So there's not a significant cost to adding arrows that--

00:36:28.020 | - The more arrow you add, the less likely you are

00:36:32.220 | to identify things from purely observational data.

00:36:36.420 | So if the whole world is bushy,

00:36:39.520 | and everybody affect everybody else,

00:36:45.420 | the answer is, you can answer it ahead of time.

00:36:49.160 | I cannot answer my query from observational data.

00:36:54.160 | I have to go to experiments.

00:36:55.740 | - So you talk about machine learning

00:36:58.340 | is essentially learning by association,

00:37:01.580 | or reasoning by association,

00:37:03.100 | and this do calculus is allowing for intervention.

00:37:07.140 | I like that word.

00:37:07.980 | Action.

00:37:09.860 | So you also talk about counterfactuals.

00:37:12.380 | - Yeah.

00:37:13.220 | - And trying to sort of understand the difference

00:37:15.860 | between counterfactuals and intervention.

00:37:18.320 | What's the, first of all, what is counterfactuals,

00:37:22.320 | and why are they useful?

00:37:25.100 | Why are they especially useful

00:37:29.680 | as opposed to just reasoning what effect actions have?

00:37:34.680 | - Counterfactual contains what we normally call explanations.

00:37:39.920 | - Can you give an example of a counterfactual?

00:37:41.080 | - If I tell you that acting one way

00:37:44.320 | affects something else, I didn't explain anything yet.

00:37:47.720 | But if I ask you, was it the aspirin

00:37:52.720 | that cured my headache?

00:37:55.400 | I'm asking for explanation, what cured my headache?

00:37:58.640 | And putting a finger on aspirin,

00:38:02.200 | provide an explanation.

00:38:04.640 | It was aspirin that was responsible

00:38:08.160 | for your headache going away.

00:38:11.560 | If you didn't take the aspirin,

00:38:14.400 | you would still have a headache.

00:38:15.960 | - So by saying, if I didn't take aspirin,

00:38:20.260 | I would have a headache, you're thereby saying

00:38:22.760 | that aspirin is the thing that removes the headache.

00:38:25.960 | - Yeah, but you have to have another important information.

00:38:30.440 | I took the aspirin, and my headache is gone.

00:38:33.680 | It's very important information.

00:38:36.380 | Now I'm reasoning backward,

00:38:38.080 | and I said, was it the aspirin?

00:38:40.520 | - Yeah, by considering what would have happened

00:38:44.400 | if everything else is the same, but I didn't take aspirin.

00:38:46.960 | - That's right, so you know that things took place.

00:38:49.520 | Joe killed Schmoe, and Schmoe would be alive

00:38:56.000 | had Joe not used his gun.

00:38:58.640 | So that is the counterfactual.

00:39:01.800 | It had a conflict here, or clash,

00:39:06.640 | between observed fact, that he did shoot,

00:39:11.640 | and the hypothetical predicate,

00:39:16.600 | which says had he not shot, you have a logical clash.

00:39:21.560 | They cannot exist together.

00:39:23.820 | That's the counterfactual, and that is the source

00:39:26.160 | of our explanation of the idea of responsibility,

00:39:31.160 | regret, and free will.

00:39:34.820 | - Yeah, so it certainly seems,

00:39:37.220 | that's the highest level of reasoning, right?

00:39:39.780 | - Yes, and physicists do it all the time.

00:39:41.900 | - Who does it all the time?

00:39:42.740 | - Physicists. - Physicists.

00:39:44.940 | - In every equation of physics,

00:39:47.100 | let's say you have a Hooke's law,

00:39:49.580 | and you put one kilogram on the spring,

00:39:52.220 | and the spring is one meter,

00:39:54.700 | and you say, had this weight been two kilogram,

00:39:58.380 | the spring would have been twice as long.

00:40:02.060 | It's no problem for physicists to say that,

00:40:05.560 | except that mathematics is only in the form of equation,

00:40:09.560 | equating the weight, proportionality constant,

00:40:15.700 | and the length of the string.

00:40:18.540 | So you don't have the asymmetry in the equation of physics,

00:40:23.300 | although every physicist thinks counterfactually.

00:40:26.820 | Ask high school kids, had the weight been three kilograms,

00:40:31.100 | what would be the length of the spring?

00:40:33.380 | They can answer it immediately,

00:40:35.160 | because they do the counterfactual processing in their mind,

00:40:38.900 | and then they put it into equation, algebraic equation,

00:40:42.300 | and they solve it, okay?

00:40:44.260 | But a robot cannot do that.

00:40:46.700 | - How do you make a robot learn these relationships?

00:40:51.700 | - Why you would learn?

00:40:53.220 | Suppose you tell him, can you do it?

00:40:55.580 | So before you go learning, you have to ask yourself,

00:40:59.380 | suppose I give him all the information, okay?

00:41:01.780 | Can the robot perform the task that I ask him to perform?

00:41:07.820 | Can he reason and say, no, it wasn't the aspirin,

00:41:10.980 | it was the good news you received on the phone?

00:41:13.320 | - Right, because, well, unless the robot had a model,

00:41:19.060 | a causal model of the world.

00:41:23.660 | - Right, right.

00:41:24.500 | - I'm sorry I have to linger on this.

00:41:26.180 | - But now we have to linger, and we have to say,

00:41:27.860 | how do we do it?

00:41:29.100 | How do we build it?

00:41:29.940 | - Yes.

00:41:30.760 | - How do we build a causal model

00:41:32.220 | without a team of human experts running around?

00:41:37.220 | - Why don't you go to learning right away?

00:41:39.580 | You're too much involved with learning.

00:41:41.220 | - 'Cause I like babies, babies learn fast,

00:41:43.180 | I'm trying to figure out how they do it.

00:41:45.140 | - Good.

00:41:46.700 | That's another question.

00:41:47.660 | How do the babies come out

00:41:49.140 | with the counterfactual model of the world?

00:41:51.780 | And babies do that.

00:41:53.580 | They know how to play in the crib.

00:41:56.900 | They know which balls hits another one.

00:41:59.500 | And they learn it by playful manipulation of the world.

00:42:04.500 | - Yes.

00:42:07.660 | - The simple world involve only toys and balls and chimes.

00:42:11.860 | But if you think about it, it's a complex world.

00:42:17.260 | - We take for granted how complex.

00:42:20.340 | - And the kids do it by playful manipulation

00:42:23.740 | plus parent's guidance, peer wisdom, and hearsay.

00:42:28.740 | They meet each other and they say,

00:42:34.780 | you shouldn't have taken my toy.

00:42:38.940 | - Right.

00:42:39.780 | And these multiple sources of information

00:42:43.540 | they're able to integrate.

00:42:44.900 | So the challenge is about how to integrate,

00:42:49.260 | how to form these causal relationships

00:42:52.620 | from different sources of data.

00:42:54.260 | - Correct.

00:42:55.580 | - So how much information is it to play,

00:42:59.980 | how much causal information is required

00:43:03.060 | to be able to play in the crib with different objects?

00:43:06.860 | - I don't know.

00:43:08.260 | I haven't experimented with the crib.

00:43:11.340 | - Okay, not a crib.

00:43:12.700 | - I don't know, it's a very interesting--

00:43:14.180 | - Manipulating physical objects on this very,

00:43:16.900 | opening the pages of a book, all the tasks,

00:43:21.700 | physical manipulation tasks.

00:43:23.740 | Do you have a sense?

00:43:25.260 | Because my sense is the world is extremely complicated.

00:43:27.980 | - Extremely complicated.

00:43:29.420 | I agree and I don't know how to organize it

00:43:31.260 | because I've been spoiled by easy problems

00:43:34.620 | such as cancer and death.

00:43:36.580 | (laughs)

00:43:38.660 | - First we have to start trying to--

00:43:41.020 | - No, but it's easy.

00:43:42.620 | It's easy in the sense that you have only 20 variables

00:43:46.980 | and they are just variables, they're not mechanics.

00:43:51.460 | It's easy, you just put them on the graph

00:43:53.580 | and they speak to you.

00:43:56.060 | - Yeah, and you're providing a methodology

00:44:00.500 | for letting them speak.

00:44:02.380 | - I'm working only in the abstract.

00:44:05.140 | The abstract is knowledge in, knowledge out,

00:44:08.980 | data in between.

00:44:10.700 | - Now, can we take a leap to trying to learn

00:44:15.100 | in this very, when it's not 20 variables,

00:44:18.120 | but 20 million variables,

00:44:20.620 | trying to learn causation in this world?

00:44:24.060 | Not learn, but somehow construct models.

00:44:27.180 | I mean, it seems like you would only have to be able

00:44:29.580 | to learn because constructing it manually

00:44:33.900 | would be too difficult.

00:44:35.560 | Do you have ideas of--

00:44:37.900 | - I think it's a matter of combining simple models

00:44:41.220 | from many, many sources, from many, many disciplines

00:44:44.140 | and many metaphors.

00:44:48.220 | Metaphors are the basics of human intelligence, basis.

00:44:51.940 | - Yeah, so how do you think about a metaphor

00:44:53.940 | in terms of its use in human intelligence?

00:44:56.180 | - Metaphors is an expert system.

00:45:00.300 | An expert, it's mapping problem

00:45:05.700 | with which you are not familiar

00:45:09.580 | to a problem with which you are familiar.

00:45:13.800 | Like, I'll give you a good example.

00:45:15.960 | The Greek believed that the sky is an opaque shell.

00:45:20.960 | It's not really infinite space.

00:45:25.940 | It's an opaque shell, and the stars are holes

00:45:29.660 | poked in the shells through which you see the eternal light.

00:45:34.260 | It was a metaphor, why?

00:45:36.980 | Because they understand how you poke holes in shells.

00:45:42.800 | They were not familiar with infinite space.

00:45:45.740 | And we are walking on a shell of a turtle,

00:45:52.300 | and if you get too close to the edge,

00:45:54.540 | you're gonna fall down to Hades or wherever.

00:45:56.880 | That's a metaphor.

00:45:59.880 | It's not true, but this kind of metaphor

00:46:04.960 | enabled Aristoteles to measure the radius of the Earth

00:46:10.680 | because he said, come on, if we are walking

00:46:13.660 | on a turtle shell, then the ray of light

00:46:16.980 | coming to this angle will be different,

00:46:19.800 | this place will be a different angle

00:46:21.640 | than coming to this place.

00:46:23.040 | I know the distance, I'll measure the two angles,

00:46:26.400 | and then I have the radius of the shell of the turtle.

00:46:31.400 | And he did, and he found his measurement

00:46:38.480 | very close to the measurements we have today,

00:46:42.320 | to the 6,700 kilometers of the Earth.

00:46:48.280 | That's something that would not occur

00:46:55.080 | to Babylonian astronomers, even though

00:47:00.320 | the Babylonian experiments were the machine learning people

00:47:03.440 | of the time.

00:47:04.600 | They fit curves, and they could predict

00:47:07.880 | the eclipse of the moon much more accurately

00:47:12.240 | than the Greek, because they fit curve.

00:47:14.780 | That's a different metaphor.

00:47:19.240 | Something that you're familiar with,

00:47:20.560 | a game, a turtle shell.

00:47:21.800 | What does it mean if you are familiar?

00:47:27.520 | Familiar means that answers to certain questions

00:47:31.880 | are explicit, you don't have to derive them.

00:47:35.600 | - And they were made explicit because somewhere

00:47:38.520 | in the past, you've constructed a model of that.

00:47:42.400 | - You're familiar with, so the child is familiar

00:47:46.180 | with billiard balls.

00:47:48.360 | So the child could predict that if you let loose

00:47:51.440 | of one ball, the other one will bounce off.

00:47:54.160 | You obtain that by familiarity.

00:48:00.200 | Familiarity is answering questions,

00:48:02.920 | and you store the answer explicitly.

00:48:05.880 | You don't have to derive them.

00:48:08.000 | So this is the idea of a metaphor.

00:48:09.640 | All our life, all our intelligence

00:48:11.640 | is built around metaphors.

00:48:13.420 | Mapping from the unfamiliar to the familiar,

00:48:16.240 | but the marriage between the two is a tough thing,

00:48:20.560 | which we haven't yet been able to algorithmize.

00:48:24.760 | - So you think of that process of using metaphor

00:48:29.280 | to leap from one place to another,

00:48:31.160 | we can call it reasoning?

00:48:33.540 | Is it a kind of reasoning?

00:48:35.880 | - It is reasoning by metaphor, metaphorical reasoning.

00:48:39.480 | - Do you think of that as learning?

00:48:43.040 | So learning is a popular terminology today

00:48:46.480 | in a narrow sense.

00:48:47.640 | - It is, it is, it is definitely a form of learning.

00:48:49.920 | - So you may not, okay, right.

00:48:51.640 | - One of the most important learning,

00:48:53.800 | taking something which theoretically is derivable

00:48:57.640 | and store it in accessible format.

00:49:01.720 | I'll give you an example, chess, okay?

00:49:05.580 | Finding the winning starting move in chess is hard.

00:49:12.800 | But there is an answer.

00:49:20.200 | Either there is a winning move for white,

00:49:23.760 | or there isn't, or there is a draw.

00:49:26.820 | So it is, the answer to that is available

00:49:31.200 | through the rule of the games.

00:49:33.680 | But we don't know the answer.

00:49:35.440 | So what does a chess master have that we don't have?

00:49:38.480 | He has stored explicitly an evaluation

00:49:41.780 | of certain complex pattern of the board.

00:49:45.360 | We don't have it, ordinary people like me,

00:49:49.000 | I don't know about you, I'm not a chess master.

00:49:52.840 | So for me, I have to derive things

00:49:56.240 | that for him is explicit.

00:49:58.620 | He has seen it before, or he has seen the pattern before,

00:50:02.500 | or similar pattern, you see, metaphor,

00:50:05.080 | and he generalized and said,

00:50:08.600 | "Don't move, it's a dangerous move."

00:50:11.180 | - It's just that, not in the game of chess,

00:50:15.520 | but in the game of billiard balls,

00:50:18.960 | we humans are able to initially derive very effectively

00:50:22.400 | and then reason by metaphor very effectively,

00:50:25.120 | and make it look so easy, that it makes one wonder

00:50:28.720 | how hard is it to build it in a machine?

00:50:31.240 | So in your sense, how far away are we

00:50:37.360 | to be able to construct--

00:50:40.720 | - I don't know, I'm not a futurist.

00:50:42.860 | All I can tell you is that we are making tremendous progress

00:50:48.440 | in the causal reasoning domain.

00:50:52.160 | Something that I even dare to call it revolution,

00:50:57.160 | the causal revolution, because what we have achieved

00:51:03.880 | in the past three decades is something that

00:51:08.760 | dwarfed everything that was derived in the entire history.

00:51:15.420 | - So there's an excitement about

00:51:17.760 | current machine learning methodologies,

00:51:20.600 | and there's really important good work you're doing

00:51:23.920 | in causal inference.

00:51:26.420 | Where do these worlds collide, and what does that look like?

00:51:32.720 | - First, they're gonna work without collisions.

00:51:37.580 | It's gonna work in harmony.

00:51:40.560 | - Harmony, it's not--

00:51:41.760 | - The human is going to jumpstart the exercise

00:51:48.520 | by providing qualitative, non-committing models

00:51:53.520 | of how the universe works.

00:51:56.440 | Universe, how in reality, the domain of discourse works.

00:52:01.440 | The machine is gonna take over from that point of view

00:52:06.800 | and derive whatever the calculus says can be derived.

00:52:11.800 | Namely, quantitative answer to our questions.

00:52:16.720 | These are complex questions.

00:52:18.440 | I'll give you some example of complex questions

00:52:21.200 | that would boggle your mind if you think about it.

00:52:26.200 | You take results of studies in diverse populations

00:52:32.560 | under diverse conditions, and you infer the cause effect

00:52:38.640 | of a new population which doesn't even resemble

00:52:43.120 | any of the ones studied.

00:52:45.160 | And you do that by, do calculus, you do that by generalizing

00:52:50.160 | from one study to another.

00:52:52.640 | See, what's common between the two?

00:52:54.800 | What is different?

00:52:57.000 | Let's ignore the differences and pull out the commonality.

00:53:01.200 | And you do it over maybe 100 hospitals around the world.

00:53:06.160 | From that, you can get really mileage from big data.

00:53:11.160 | It's not only do you have many samples,

00:53:15.040 | you have many sources of data.

00:53:18.720 | - So that's a really powerful thing,

00:53:20.520 | I think, especially for medical applications.

00:53:23.360 | I mean, cure cancer, right?

00:53:25.840 | That's how from data you can cure cancer.

00:53:28.560 | So we're talking about causation,

00:53:30.080 | which is the temporal relationship between things.

00:53:35.080 | - Not only temporal.

00:53:36.800 | It's both structural and temporal.

00:53:38.680 | Temporal enough, temporal presence by itself

00:53:43.000 | cannot replace causation.

00:53:45.240 | - Is temporal precedence, the arrow of time in physics--

00:53:50.920 | - It's important, necessary.

00:53:52.200 | - It's important. - But not sufficient, yes.

00:53:54.280 | - Is it?

00:53:55.800 | - Yes, I never seen cause propagate backward.

00:54:00.200 | - But if we use the word cause,

00:54:04.000 | but there's relationships that are timeless.

00:54:07.080 | I suppose that's still forward in the arrow of time.

00:54:10.400 | But are there relationships, logical relationships,

00:54:14.840 | that fit into the structure?

00:54:17.160 | - Sure, do calculate this logical relationship.

00:54:21.960 | - That doesn't require a temporal.

00:54:23.800 | It has just the condition that

00:54:25.560 | you're not traveling back in time.

00:54:28.600 | - Yes, correct.

00:54:31.200 | - So it's really a generalization of,

00:54:34.060 | a powerful generalization of what--

00:54:39.760 | - Boolean logic.

00:54:40.720 | - Yeah, Boolean logic.

00:54:41.720 | - Yes.

00:54:42.560 | - That is simply put and allows us to

00:54:47.960 | reason about the order of events, the source, the--

00:54:54.920 | - Not about, we're not deriving the order of events.

00:54:58.040 | We are given cause-effect relationship.

00:55:00.280 | There ought to be

00:55:02.360 | obeying the time-presence relationship.

00:55:08.920 | We are given that.

00:55:09.960 | And now that we ask questions about

00:55:12.520 | other causal relationship that could be derived

00:55:15.480 | from the initial ones,

00:55:17.880 | but were not given to us explicitly.

00:55:19.960 | Like the case of the firing squad I gave you

00:55:25.960 | in the first chapter.

00:55:28.240 | And I ask what if a rifleman A declined to shoot?

00:55:33.040 | Would the prisoner still be dead?

00:55:37.960 | If he declined to shoot, it means that he disobeyed order.

00:55:42.040 | And the rule of the game is that he is

00:55:46.280 | obedient and marksman.

00:55:51.000 | That's how you start, that's the initial order.

00:55:53.560 | But now you ask question about breaking the rules.

00:55:56.560 | What if he decided not to pull the trigger?

00:56:00.440 | He just became a pacifist.

00:56:02.140 | And you and I can answer that.

00:56:06.040 | The other rifleman would have killed him.

00:56:08.140 | I want a machine to do that.

00:56:10.640 | Is it so hard to ask a machine to do that?

00:56:15.440 | It's such a simple task.

00:56:16.920 | But you have to have a calculus for that.

00:56:19.360 | - Yes.

00:56:20.200 | But the curiosity, the natural curiosity for me is

00:56:24.320 | that yes, you're absolutely correct and important.

00:56:27.980 | And it's hard to believe that we haven't done this

00:56:31.080 | seriously, extensively, already a long time ago.

00:56:35.360 | So this is really important work.

00:56:37.000 | But I also wanna know, maybe you can philosophize

00:56:41.320 | about how hard is it to learn?

00:56:43.320 | - Okay, let's assume we're learning.

00:56:44.400 | We wanna learn it, okay?

00:56:45.600 | - We wanna learn.

00:56:46.440 | - So what do we do?

00:56:47.260 | We put a learning machine that watches execution trials

00:56:51.520 | in many countries and many locations, okay?

00:56:56.520 | All the machine can learn is to see shot or not shot.

00:57:01.040 | Dead, not dead.

00:57:03.440 | Court issued an order or didn't, okay?

00:57:05.640 | That's the fact.

00:57:07.340 | From the fact you don't know who listens to whom.

00:57:10.120 | You don't know that the condemned person

00:57:13.720 | listened to the bullets, that the bullets are listening

00:57:17.040 | to the captain, okay?

00:57:19.280 | All we hear is one command, two shots, dead, okay?

00:57:24.280 | A triple of variable.

00:57:27.080 | Yes, no, yes, no.

00:57:29.660 | From that you can learn who listens to whom

00:57:32.120 | and you can answer the question, no.

00:57:33.960 | - Definitively no, but don't you think

00:57:36.680 | you can start proposing ideas for humans to review?

00:57:40.480 | - You want machine to learn it, right?

00:57:43.040 | You want a robot.

00:57:44.360 | So robot is watching trials like that, 200 trials,

00:57:49.360 | and then he has to answer the question,

00:57:52.560 | what if rifleman A refrained from shooting?

00:57:56.960 | - Yeah.

00:57:57.800 | How to do that?

00:58:01.600 | That's exactly my point.

00:58:03.640 | It's looking at the facts don't give you the strings

00:58:06.200 | behind the facts.

00:58:07.160 | - Absolutely, but do you think of machine learning

00:58:11.860 | as it's currently defined as only something

00:58:15.220 | that looks at the facts and tries to--

00:58:17.600 | - Right now they only look at the facts, yeah.

00:58:19.160 | - So is there a way to modify, in your sense--

00:58:23.480 | - Playful manipulation.

00:58:25.120 | - Playful manipulation.

00:58:26.120 | - Yes, once in a while--

00:58:26.960 | - Doing the interventionist kind of thing, intervention.

00:58:29.720 | - But it could be at random.

00:58:31.160 | For instance, the rifleman is sick that day,

00:58:34.520 | or he just vomits or whatever.

00:58:37.240 | So machine can observe this unexpected event

00:58:41.240 | which introduced noise.

00:58:43.600 | The noise still have to be random to be able to relate it

00:58:48.120 | to randomized experiment.

00:58:51.640 | And then you have observational studies

00:58:55.480 | from which to infer the strings behind the facts.

00:58:59.940 | It's doable to a certain extent.

00:59:03.000 | But now that we are expert in what you can do

00:59:06.240 | once you have a model, we can reason back and say

00:59:09.080 | what kind of data you need to build a model.

00:59:13.000 | - Got it, so I know you're not a futurist,

00:59:17.280 | but are you excited?

00:59:19.760 | Have you, when you look back at your life,

00:59:22.520 | long for the idea of creating

00:59:24.520 | a human level intelligence system?

00:59:25.960 | - Yeah, I'm driven by that.

00:59:28.320 | All my life I'm driven just by one thing.

00:59:30.480 | (laughs)

00:59:32.560 | But I go slowly, I go from what I know

00:59:36.860 | to the next step incrementally.

00:59:39.380 | - So without imagining what the end goal looks like,

00:59:42.420 | do you imagine what--

00:59:44.540 | - And the end goal is gonna be a machine

00:59:47.780 | that can answer sophisticated questions,

00:59:50.900 | counterfactuals of regret, compassion,

00:59:53.880 | responsibility, and free will.

00:59:58.700 | - So what is a good test?

01:00:01.600 | Is a Turing test a reasonable test?

01:00:04.920 | - A test of free will doesn't exist yet.

01:00:06.920 | - How would you test free will?

01:00:09.960 | - So far we know only one thing.

01:00:12.520 | If robots can communicate with reward and punishment

01:00:19.960 | among themselves, hitting each other on the wrist

01:00:25.440 | and say you shouldn't have done that, okay?

01:00:27.800 | Playing better soccer because they can do that.

01:00:32.280 | - What do you mean because they can do that?

01:00:35.940 | - Because they can communicate among themselves.

01:00:38.100 | - Because of the communication they can do this--

01:00:40.100 | - Because they communicate like us, reward and punishment.

01:00:44.060 | Yes, you didn't pass the ball the right time,

01:00:47.580 | and so therefore you're gonna sit on the bench

01:00:50.060 | for the next two.

01:00:51.560 | If they start communicating like that,

01:00:53.660 | the question is will they play better soccer?

01:00:56.380 | As opposed to what?

01:00:57.660 | As opposed to what they do now?

01:00:59.680 | Without this ability to reason about reward and punishment,

01:01:04.680 | responsibility.

01:01:06.460 | - And counterfactuals.

01:01:08.420 | - So far I can only think about communication.

01:01:11.740 | - Communication is, not necessarily natural language,

01:01:15.380 | but just communication.

01:01:16.220 | - Yeah, just communication.

01:01:17.580 | And that's important to have a quick and effective means

01:01:22.000 | of communicating knowledge.

01:01:24.100 | If the coach tells you you should have passed the ball,

01:01:26.460 | ping, he conveys so much knowledge to you

01:01:28.780 | as opposed to what?

01:01:30.500 | Go down and change your software, right?

01:01:33.460 | That's the alternative.

01:01:35.340 | But the coach doesn't know your software.

01:01:37.740 | So how can a coach tell you you should have passed the ball?

01:01:41.580 | But our language is very effective.

01:01:44.300 | You should have passed the ball.

01:01:45.620 | You know your software, you tweak the right module,

01:01:48.620 | and next time you don't do it.

01:01:51.840 | - Now that's for playing soccer

01:01:53.600 | where the rules are well defined.

01:01:55.240 | - No, no, no, no, no, they're not well defined.

01:01:57.440 | When you should pass the ball--

01:01:58.840 | - Is not well defined.

01:02:00.040 | - No, it's very soft, very noisy.

01:02:04.440 | - Yeah, the mystery.

01:02:05.280 | - You have to do it under pressure.

01:02:06.640 | - It's art.

01:02:07.960 | But in terms of aligning values

01:02:11.360 | between computers and humans,

01:02:14.360 | do you think this cause and effect type of thinking

01:02:20.240 | is important to align the values, morals,

01:02:24.220 | ethics under which the machines make decisions?

01:02:26.420 | Is the cause effect where the two can come together?

01:02:31.420 | - Cause effect is necessary component

01:02:34.760 | to build an ethical machine.

01:02:38.280 | 'Cause the machine has to empathize,

01:02:40.460 | to understand what's good for you,

01:02:42.620 | to build a model of you as a recipient,

01:02:47.160 | which should be very much, what is compassion?

01:02:50.900 | They imagine that you suffer pain as much as me.

01:02:55.900 | - As much as me.

01:02:57.020 | - I do have already a model of myself, right?

01:03:00.300 | So it's very easy for me to map you to mine.

01:03:02.780 | I don't have to rebuild the model.

01:03:04.620 | It's much easier to say, oh, you're like me.

01:03:06.920 | Okay, therefore I will not hate you.

01:03:08.720 | - And the machine has to imagine,

01:03:11.300 | has to try to fake to be human,

01:03:13.980 | essentially so you can imagine

01:03:15.500 | that you're like me, right?

01:03:19.440 | - And moreover, who is me?

01:03:21.680 | That's the first, that's consciousness.

01:03:24.200 | They have a model of yourself.

01:03:25.760 | Where do you get this model?

01:03:28.080 | You look at yourself as if you are a part of the environment.

01:03:32.400 | If you build a model of yourself versus the environment,

01:03:35.440 | then you can say I need to have a model of myself.

01:03:38.240 | I have abilities, I have desires and so forth, okay?

01:03:41.800 | I have a blueprint of myself, though.

01:03:44.360 | Not a full detail because I cannot get the whole thing

01:03:48.020 | problem, but I have a blueprint.

01:03:50.700 | So on that level of a blueprint, I can modify things.

01:03:54.220 | I can look at myself in the mirror and say,

01:03:56.580 | hmm, if I change this, tweak this model,

01:03:59.180 | I'm gonna perform differently.

01:04:01.120 | That is what we mean by free will.

01:04:04.360 | - And consciousness.

01:04:06.940 | What do you think is consciousness?

01:04:10.400 | Is it simply self-awareness,

01:04:11.820 | so including yourself into the model of the world?

01:04:14.640 | - That's right.

01:04:15.480 | Some people tell me, no, this is only part of consciousness,

01:04:19.600 | and then they start telling me what they really mean

01:04:21.440 | by consciousness, and I lose them.

01:04:23.140 | For me, consciousness is having a blueprint

01:04:29.400 | of your software.

01:04:30.240 | - Do you have concerns about the future of AI,

01:04:36.540 | all the different trajectories of all of our research?

01:04:39.640 | - Yes.

01:04:40.680 | - Where's your hope, where the movement heads,

01:04:43.280 | where are your concerns?

01:04:44.360 | - I'm concerned because I know we are building

01:04:48.000 | a new species that has a capability of exceeding us,

01:04:52.400 | exceeding our capabilities, and can breed itself

01:05:01.280 | and take over the world, absolutely.

01:05:03.560 | It's a new species that is uncontrolled.

01:05:07.640 | We don't know the degree to which we control it.

01:05:10.080 | We don't even understand what it means

01:05:12.620 | to be able to control this new species.

01:05:15.180 | So I'm concerned.

01:05:17.720 | I don't have anything to add to that

01:05:21.120 | because it's such a gray area, it's unknown.

01:05:26.120 | It never happened in history.

01:05:27.700 | The only time it happened in history

01:05:34.240 | was evolution with human beings.

01:05:36.000 | It wasn't very successful, was it?

01:05:39.840 | (laughs)

01:05:40.840 | - Some people say it was a great success.

01:05:42.720 | - For us it was, but a few people along the way,

01:05:46.360 | a few creatures along the way would not agree.

01:05:49.360 | So it's just because it's such a gray area,

01:05:53.080 | there's nothing else to say.

01:05:55.000 | - We have a sample of one.

01:05:56.840 | - Sample of one.

01:05:58.120 | - That's us.

01:05:58.960 | - But some people would look at you and say,

01:06:05.840 | yeah, but we were looking to you to help us

01:06:09.800 | make sure that sample two works out okay.

01:06:13.160 | - We have more than a sample of one.

01:06:14.880 | We have theories, and that's good.

01:06:18.680 | We don't need to be statisticians.

01:06:20.760 | So sample of one doesn't mean poverty of knowledge.

01:06:25.440 | It's not.

01:06:26.480 | Sample of one plus theory,

01:06:28.840 | conjectural theory of what could happen.

01:06:31.780 | That we do have.

01:06:34.400 | But I really feel helpless in contributing to this argument

01:06:39.400 | because I know so little,

01:06:42.000 | and my imagination is limited,

01:06:46.640 | and I know how much I don't know,

01:06:49.500 | but I'm concerned.

01:06:54.480 | - You were born and raised in Israel.

01:06:57.800 | - Born and raised in Israel, yes.

01:06:59.240 | - And later served in Israel military, defense forces.

01:07:05.200 | - In the Israel defense force.

01:07:08.000 | - Yeah.

01:07:08.840 | What did you learn from that experience?

01:07:13.840 | - From this experience.

01:07:15.160 | - There's a kibbutz in there as well.

01:07:18.160 | - Yes, because I was in the Nakhal,

01:07:20.520 | which is a combination of agricultural work

01:07:25.520 | and military service.

01:07:27.800 | I was really idealist.

01:07:31.200 | I wanted to be a member of the kibbutz throughout my life

01:07:36.200 | and to live a communal life.

01:07:38.280 | So I prepared myself for that.

01:07:44.520 | Slowly, slowly I went the greater challenge.

01:07:49.960 | - So that's a far world away.

01:07:55.000 | - But I learned from that what I can.

01:07:57.280 | It was a miracle.

01:08:01.320 | It was a miracle that I served in the 1950s.

01:08:05.680 | I don't know how we survived.

01:08:09.400 | The country was under austerity.

01:08:13.680 | It tripled its population from 600,000 to a million point

01:08:20.360 | eight when I finished college.

01:08:23.280 | No one went hungry.

01:08:24.800 | Austerity, yes.

01:08:29.360 | When you wanted to make an omelet in a restaurant,

01:08:34.200 | you had to bring your own egg.

01:08:35.680 | And they imprisoned people from bringing food

01:08:43.120 | from the farming and from the villages to the city.

01:08:47.920 | But no one went hungry.

01:08:50.800 | And I always add to it,

01:08:53.400 | and higher education did not suffer any budget cut.

01:08:59.160 | They still invested in me, in my wife, in our generation

01:09:04.160 | to get the best education that they could.

01:09:08.000 | So I'm really grateful for the opportunity.

01:09:14.680 | And I'm trying to pay back now.

01:09:17.280 | It's a miracle that we survived the war of 1948.

01:09:22.920 | We were so close to a second genocide.

01:09:27.280 | It was all planned.

01:09:29.080 | But we survived it by a miracle.

01:09:32.200 | And then the second miracle

01:09:33.600 | that not many people talk about, the next phase.

01:09:37.800 | How no one went hungry

01:09:40.280 | and the country managed to triple its population.

01:09:43.960 | You know what it means to triple?

01:09:45.280 | Imagine United States going from what, 350 million

01:09:50.240 | to a trillion.

01:09:51.560 | - Yeah, yeah.

01:09:52.760 | - Unbelievable.

01:09:53.880 | - This is a really tense part of the world.

01:09:57.000 | It's a complicated part of the world.

01:09:59.080 | Israel and all around.

01:10:00.760 | Religion is at the core of that complexity.

01:10:06.800 | One of the components.

01:10:09.400 | - Religion is a strong motivating cause

01:10:12.440 | for many, many people in the Middle East.

01:10:15.400 | - In your view, looking back, is religion good for society?

01:10:21.000 | - That's a good question for robotics, you know?

01:10:26.560 | - There's echoes of that question.

01:10:28.200 | - Equip robots with religious beliefs.

01:10:31.320 | Suppose we find out, or we agree,

01:10:34.640 | that religion is good to you, to keep you in line.

01:10:37.920 | Should we give the robot the metaphor of a god?

01:10:42.920 | As a matter of fact, the robot will get it without us also.

01:10:46.480 | Why?

01:10:48.180 | The robot will reason by metaphor.

01:10:51.000 | And what is the most primitive metaphor?

01:10:56.400 | A child grows with mother's smile,

01:11:00.360 | father teaching, father image, and mother image.

01:11:04.480 | That's God.

01:11:05.400 | So, whether you want it or not,

01:11:09.400 | the robot will,

01:11:10.800 | but assuming the robot is gonna have a mother and a father,

01:11:14.800 | it may only have a programmer,

01:11:16.360 | which doesn't supply warmth and discipline.

01:11:20.960 | But discipline it does.

01:11:22.440 | So the robot will have this model of the trainer,

01:11:26.200 | and everything that happens in the world,

01:11:29.320 | cosmology and so on,

01:11:30.760 | is going to be mapped into the programmer.

01:11:34.140 | That's God.

01:11:36.400 | - The thing that represents the origin

01:11:41.380 | of everything for that robot.

01:11:43.960 | - It's the most primitive relationship.

01:11:46.340 | - So it's gonna arrive there by metaphor.

01:11:48.680 | And so the question is if overall that metaphor

01:11:53.120 | has served us well as humans.

01:11:56.200 | - I really don't know.

01:11:58.000 | I think it did.

01:11:59.140 | But as long as you keep in mind it's only a metaphor.

01:12:03.240 | (laughs)

01:12:05.160 | - So, if you think we can,

01:12:09.760 | can we talk about your son?

01:12:11.560 | - Yes, yes.

01:12:13.240 | - Can you tell his story?

01:12:15.080 | - His story?

01:12:15.920 | - Daniel.

01:12:18.000 | - The way he's known is he was abducted in Pakistan

01:12:23.000 | by Al-Qaeda driven sect.

01:12:27.000 | And under various pretenses.

01:12:32.000 | I don't even pay attention to what the pretense was.

01:12:35.180 | Originally they wanted to have the United States

01:12:39.800 | deliver some promised airplanes.

01:12:47.520 | It was all made up.

01:12:49.000 | All these demands were bogus.

01:12:53.260 | I don't know really.

01:12:55.400 | But eventually he was executed in front of a camera.

01:13:00.400 | - At the core of that is hate and intolerance.

01:13:07.400 | - At the core, yes, absolutely, yes.

01:13:10.000 | We don't really appreciate the depth of the hate

01:13:16.320 | at which billions of peoples are educated.

01:13:21.320 | We don't understand it.

01:13:27.520 | I just listened recently to what they teach you

01:13:31.360 | in Mogadishu.

01:13:32.200 | When the water stop in the tap,

01:13:45.680 | we knew exactly who did it, the Jews.

01:13:49.360 | - The Jews.

01:13:50.600 | - We didn't know how, but we knew who did it.

01:13:54.000 | We don't appreciate what it means to us.

01:13:58.080 | The depth is unbelievable profound.

01:14:00.480 | - Do you think all of us are capable of evil?

01:14:04.660 | And the education, the indoctrination

01:14:09.840 | is really what creates evil.

01:14:10.680 | - Absolutely we are capable of evil.

01:14:12.240 | If you're indoctrinated sufficiently long and in depth,

01:14:17.240 | you're capable of ISIS, you're capable of Nazism.

01:14:23.800 | Yes, we are.

01:14:25.820 | But the question is whether we,

01:14:28.400 | after we have gone through some Western education

01:14:32.840 | and we learn that everything is really relative.

01:14:35.680 | There is no absolute God.

01:14:37.680 | There's only a belief in God.

01:14:40.080 | Whether we are capable now of being transformed

01:14:43.520 | under certain circumstances to become brutal.

01:14:48.520 | That is a question, I'm worried about it

01:14:53.000 | because some people say yes, given the right circumstances,

01:14:57.360 | given the bad economical crisis,

01:15:02.360 | you are capable of doing it too.

01:15:06.160 | That worries me.

01:15:08.420 | I want to believe it, I'm not capable.

01:15:10.520 | - This is seven years after Daniel's death.

01:15:14.540 | He wrote an article at the Wall Street Journal

01:15:16.780 | titled Daniel Pearl and the Normalization of Evil.

01:15:19.740 | - Yes.

01:15:20.560 | - What was your message back then

01:15:23.060 | and how did it change today over the years?

01:15:27.540 | - I lost.

01:15:28.820 | - What was the message?

01:15:31.860 | - The message was that we are not treating terrorism.

01:15:36.620 | Terrorism is a taboo.

01:15:40.660 | We are treating it as a bargaining device

01:15:45.860 | that is accepted.

01:15:47.620 | People have grievance and they go and bomb restaurants.

01:15:52.620 | It's normal.

01:15:55.260 | Look, you're even not surprised when I tell you that.

01:15:58.180 | 20 years ago you'd say what?

01:16:01.640 | For grievance you go and blow a restaurant?

01:16:05.020 | Today it's becoming normalized.

01:16:06.940 | The banalization of evil.

01:16:09.820 | And we have created that to ourselves by normalizing,

01:16:16.080 | by making it part of political life.

01:16:21.180 | It's a political debate.

01:16:26.780 | Every terrorist yesterday becomes a freedom fighter today

01:16:34.420 | and tomorrow it becomes a terrorist again.

01:16:36.580 | It's switchable.

01:16:37.660 | - And so we should call out evil when there's evil.

01:16:42.220 | - If we don't want to be part of it.

01:16:46.820 | - Become it.

01:16:47.660 | - Yeah, if we want to separate good from evil.

01:16:52.300 | That's one of the first things that,

01:16:54.100 | what was it, in the Garden of Eden,

01:16:57.540 | remember the first thing that God tells him

01:17:02.580 | was hey, you want some knowledge?

01:17:05.060 | Here's a tree of good and evil.

01:17:07.380 | - So this evil touched your life personally.

01:17:12.260 | Does your heart have anger, sadness, or is it hope?

01:17:17.220 | - Look, I see some beautiful people coming from Pakistan.

01:17:25.300 | I see beautiful people everywhere.

01:17:29.420 | But I see horrible propagation of evil in this country too.

01:17:34.420 | It shows you how populistic slogans

01:17:42.780 | can catch the mind of the best intellectuals.

01:17:47.260 | - Today is Father's Day.

01:17:50.100 | - I didn't know that.

01:17:51.060 | - Yeah.

01:17:51.900 | - I heard it.

01:17:52.740 | - What's a fond memory you have of Daniel?

01:17:58.380 | - Oh, many good memories, immense.

01:18:01.740 | He was my mentor.

01:18:03.900 | He had a sense of balance that I didn't have.

01:18:11.100 | He saw the beauty in every person.

01:18:17.500 | He was not as emotional as I am,

01:18:22.020 | more looking at things in perspective.

01:18:26.220 | He really liked every person.

01:18:29.380 | He really grew up with the idea that a foreigner

01:18:33.420 | is a reason for curiosity, not for fear.

01:18:39.740 | There's one time we went in Berkeley

01:18:45.300 | and a homeless came out from some dark alley

01:18:49.220 | and said hey man, can you spare a dime?

01:18:51.180 | I retreated back, two feet back,

01:18:54.620 | and then I just hugged him and said here's a dime,

01:18:57.540 | enjoy yourself, maybe you want some money

01:19:02.780 | to take a bus or whatever.

01:19:04.380 | Where did he get it?

01:19:06.580 | Not from me.

01:19:07.420 | - Do you have advice for young minds today

01:19:12.400 | dreaming about creating, as you have dreamt,

01:19:16.200 | creating intelligent systems?

01:19:17.880 | What is the best way to arrive at new breakthrough ideas

01:19:21.380 | and carry them through the fire of criticism

01:19:23.820 | and past conventional ideas?

01:19:27.040 | - Ask your questions.

01:19:30.100 | Really, your questions are never dumb.

01:19:36.640 | And solve them your own way.

01:19:39.400 | And don't take no for an answer.

01:19:42.700 | If they are really dumb, you will find out quickly

01:19:48.380 | by trying an arrow to see that they're not leading

01:19:51.140 | any place, but follow them and try to understand

01:19:56.140 | things your way.

01:19:58.400 | That is my advice.

01:20:01.580 | I don't know if it's gonna help anyone.

01:20:03.980 | - No, that's brilliant.

01:20:05.500 | - There is a lot of inertia in science, in academia.

01:20:10.500 | It is slowing down science.

01:20:18.580 | - Yeah, those two words, your way, that's a powerful thing.

01:20:22.580 | It's against inertia, potentially.

01:20:26.100 | Against the flow.

01:20:26.940 | - Against your professor.

01:20:28.580 | I wrote the book of why in order to democratize common sense.

01:20:35.460 | (laughter)

01:20:38.660 | In order to instill rebellious spirit in students

01:20:45.420 | so they wouldn't wait until the professor get things right.

01:20:49.920 | - So you wrote the manifesto of the rebellion

01:20:56.600 | against the professor.

01:20:58.100 | - Against the professor, yes.

01:21:00.340 | - So looking back at your life of research,

01:21:02.780 | what ideas do you hope ripple through the next many decades?

01:21:06.180 | What do you hope your legacy will be?

01:21:09.440 | - I already have a tombstone.

01:21:13.440 | (laughter)

01:21:15.700 | - Carved.

01:21:16.540 | (laughter)

01:21:18.780 | - Oh boy.

01:21:21.540 | - The fundamental law of counterfactuals.

01:21:24.780 | That's what, it's a simple equation.

01:21:29.800 | What is counterfactual in terms of a model surgery?

01:21:34.440 | That's it, because everything follows from that.

01:21:37.940 | If you get that, all the rest.

01:21:43.720 | I can die in peace and my student can derive

01:21:48.040 | all my knowledge by mathematical means.

01:21:51.920 | - The rest follows.

01:21:53.520 | - Yeah.

01:21:54.360 | - Judea, thank you so much for talking today.

01:21:56.440 | I really appreciate it.

01:21:57.280 | - Thank you for being so attentive and instigating.

01:22:02.280 | (laughter)

01:22:03.880 | - We did it.

01:22:05.240 | - We did it.

01:22:06.080 | - The coffee helped.

01:22:07.500 | Thanks for listening to this conversation with Judea Pearl.

01:22:11.280 | And thank you to our presenting sponsor, Cash App.

01:22:14.240 | Download it, use code LEGSPODCAST, you'll get $10,

01:22:18.280 | and $10 will go to FIRST, a STEM education nonprofit

01:22:21.920 | that inspires hundreds of thousands of young minds

01:22:24.840 | to learn and to dream of engineering our future.

01:22:28.280 | If you enjoy this podcast, subscribe on YouTube,

01:22:31.080 | get five stars on Apple Podcasts, support on Patreon,

01:22:34.320 | or simply connect with me on Twitter.

01:22:36.840 | And now, let me leave you with some words of wisdom

01:22:39.280 | from Judea Pearl.

01:22:41.040 | You cannot answer a question that you cannot ask,

01:22:44.000 | and you cannot ask a question you have no words for.

01:22:47.380 | Thank you for listening, and hope to see you next time.

01:22:51.800 | (upbeat music)

01:22:54.380 | (upbeat music)

01:22:56.960 | [BLANK_AUDIO]

Judea Pearl: Causal Reasoning, Counterfactuals, and the Path to AGI | Lex Fridman Podcast #56