Michael I. Jordan: Machine Learning, Recommender Systems, and Future of AI

00:00:00.000 | The following is a conversation with Michael I. Jordan,

00:00:04.040 | a professor at Berkeley

00:00:05.360 | and one of the most influential people

00:00:07.220 | in the history of machine learning,

00:00:08.800 | statistics and artificial intelligence.

00:00:11.560 | He has been cited over 170,000 times

00:00:14.640 | and has mentored many of the world-class researchers

00:00:17.720 | defining the field of AI today,

00:00:20.400 | including Andrew Ng, Zubin Garamani,

00:00:23.800 | Ben Tasker and Yoshua Bengio.

00:00:27.480 | All of this to me is as impressive as the over 32,000 points

00:00:32.480 | in the six NBA championships

00:00:34.600 | of the Michael J. Jordan of basketball fame.

00:00:38.780 | There's a non-zero probability

00:00:40.280 | that I talked to the other Michael Jordan,

00:00:42.320 | given my connection to

00:00:43.640 | and love of the Chicago Bulls in the '90s,

00:00:46.120 | but if I had to pick one,

00:00:47.720 | I'm going with the Michael Jordan

00:00:49.640 | of statistics and computer science,

00:00:52.200 | or as Jan LeCun calls him,

00:00:53.800 | the Miles Davis of machine learning.

00:00:56.880 | In his blog post titled,

00:00:58.160 | "Artificial Intelligence,

00:00:59.320 | "the revolution hasn't happened yet,"

00:01:01.320 | Michael argues for broadening the scope

00:01:03.760 | of the artificial intelligence field.

00:01:06.040 | In many ways, the underlying spirit of this podcast

00:01:09.080 | is the same, to see artificial intelligence

00:01:12.200 | as a deeply human endeavor,

00:01:14.320 | to not only engineer algorithms and robots,

00:01:17.240 | but to understand and empower human beings

00:01:20.240 | at all levels of abstraction,

00:01:22.640 | from the individual to our civilization as a whole.

00:01:26.680 | This is the Artificial Intelligence Podcast.

00:01:29.440 | If you enjoy it, subscribe on YouTube,

00:01:31.840 | give it five stars at Apple Podcast,

00:01:33.720 | support it on Patreon,

00:01:35.080 | or simply connect with me on Twitter,

00:01:37.200 | @LexFriedman, spelled F-R-I-D-M-A-N.

00:01:40.720 | As usual, I'll do one or two minutes of ads now,

00:01:45.080 | and never any ads in the middle

00:01:46.520 | that can break the flow of the conversation.

00:01:48.760 | I hope that works for you

00:01:50.240 | and doesn't hurt the listening experience.

00:01:52.520 | This show is presented by Cash App,

00:01:56.040 | the number one finance app in the App Store.

00:01:58.320 | When you get it, use code LEXPODCAST.

00:02:01.960 | Cash App lets you send money to friends,

00:02:04.040 | buy Bitcoin, and invest in the stock market

00:02:06.480 | with as little as $1.

00:02:07.640 | Since Cash App does fractional share trading,

00:02:11.040 | let me mention that the order execution algorithm

00:02:13.640 | that works behind the scenes

00:02:15.240 | to create the abstraction of the fractional orders

00:02:18.280 | is, to me, an algorithmic marvel.

00:02:20.960 | So big props for the Cash App engineers

00:02:23.320 | for solving a hard problem

00:02:25.080 | that, in the end, provides an easy interface

00:02:27.760 | that takes a step up to the next layer of abstraction

00:02:30.680 | over the stock market,

00:02:32.200 | making trading more accessible for new investors

00:02:35.200 | and diversification much easier.

00:02:38.360 | So once again, if you get Cash App

00:02:40.320 | from the App Store or Google Play

00:02:42.080 | and use the code LEXPODCAST, you'll get $10,

00:02:45.800 | and Cash App will also donate $10 to Thirst,

00:02:48.600 | one of my favorite organizations

00:02:50.240 | that is helping to advance robotics and STEM education

00:02:53.800 | for young people around the world.

00:02:55.800 | And now, here's my conversation with Michael I. Jordan.

00:03:01.380 | Given that you're one of the greats in the field of AI,

00:03:05.200 | machine learning, computer science, and so on,

00:03:07.800 | you're trivially called the Michael Jordan

00:03:10.360 | of machine learning.

00:03:12.120 | Although, as you know, you were born first,

00:03:15.600 | so technically MJ is the Michael I. Jordan of basketball,

00:03:19.040 | but anyway, my favorite is Yan LeCun

00:03:22.160 | calling you the Miles Davis of machine learning,

00:03:25.280 | because as he says, you reinvent yourself periodically

00:03:28.100 | and sometimes leave fans scratching their heads

00:03:31.200 | after you change direction.

00:03:32.400 | So can you put, at first, your historian hat on

00:03:37.120 | and give a history of computer science and AI

00:03:39.520 | as you saw it, as you experienced it,

00:03:42.280 | including the four generations of AI successes

00:03:46.160 | that I've seen you talk about?

00:03:48.520 | - Sure.

00:03:49.480 | Yeah, first of all, I much prefer Yan's metaphor.

00:03:53.880 | Miles Davis was a real explorer in jazz

00:03:57.600 | and he had a coherent story.

00:03:59.860 | So I think I have one, but it's not just the one you lived,

00:04:03.240 | it's the one you think about later,

00:04:04.800 | what a good historian does is they look back

00:04:07.560 | and they revisit.

00:04:08.800 | I think what's happening right now is not AI.

00:04:13.120 | That was an intellectual aspiration.

00:04:15.440 | That's still alive today as an aspiration.

00:04:18.480 | But I think this is akin to the development

00:04:20.440 | of chemical engineering from chemistry

00:04:22.380 | or electrical engineering from electromagnetism.

00:04:25.780 | So if you go back to the '30s or '40s,

00:04:29.200 | there wasn't yet chemical engineering.

00:04:30.920 | There was chemistry, there was fluid flow,

00:04:32.520 | there was mechanics and so on.

00:04:34.180 | But people pretty clearly viewed interesting goals

00:04:39.360 | to try to build factories that make chemicals products

00:04:42.220 | and do it viably, safely, make good ones, do it at scale.

00:04:47.960 | So people started to try to do that, of course,

00:04:49.880 | and some factories worked, some didn't,

00:04:52.000 | some were not viable, some exploded.

00:04:53.980 | But in parallel, developed a whole field

00:04:55.840 | called chemical engineering.

00:04:57.880 | Chemical engineering is a field.

00:04:59.160 | It's no bones about it.

00:05:00.960 | It has theoretical aspects to it.

00:05:02.520 | It has practical aspects.

00:05:04.640 | It's not just engineering, quote unquote.

00:05:06.520 | It's the real thing, real concepts are needed.

00:05:09.520 | Same thing with electrical engineering.

00:05:11.600 | There was Maxwell's equations, which in some sense

00:05:13.960 | were everything you need to know about electromagnetism,

00:05:16.380 | but you needed to figure out how to build circuits,

00:05:17.980 | how to build modules, how to put them together,

00:05:19.680 | how to bring electricity from one point to another

00:05:21.620 | safely and so on and so forth.

00:05:23.220 | So a whole field developed called electrical engineering.

00:05:26.100 | I think that's what's happening right now,

00:05:27.940 | is that we have a proto-field, which is statistics,

00:05:32.500 | compute more of the theoretical side of it,

00:05:34.380 | algorithmic side of it, computer science.

00:05:36.180 | That was enough to start to build things,

00:05:37.860 | but what things?

00:05:38.780 | Systems that bring value to human beings

00:05:41.260 | and use human data and mix in human decisions.

00:05:44.020 | The engineering side of that is all ad hoc.

00:05:47.260 | That's what's emerging.

00:05:48.420 | In fact, if you want to call machine learning a field,

00:05:50.420 | I think that's what it is.

00:05:51.340 | That's a proto-form of engineering based on statistical

00:05:54.260 | and computational ideas of previous generations.

00:05:56.500 | - But do you think there's something deeper about AI

00:05:59.460 | in his dreams and aspirations as compared

00:06:01.740 | to chemical engineering and electrical engineering?

00:06:03.780 | - Well, the dreams and aspirations may be,

00:06:05.460 | but those are 500 years from now.

00:06:08.020 | I think that that's like the Greeks sitting there

00:06:09.620 | and saying it would be neat to get to the moon someday.

00:06:11.820 | - Right.

00:06:12.940 | - I think we have no clue how the brain does computation.

00:06:16.100 | We're just a clueless.

00:06:16.940 | We're even worse than the Greeks

00:06:19.180 | on most anything interesting scientifically of our era.

00:06:23.620 | - Can you linger on that just for a moment

00:06:25.220 | because you stand not completely unique,

00:06:28.500 | but a little bit unique in the clarity of that.

00:06:31.260 | Can you elaborate your intuition of why we,

00:06:34.820 | like where we stand in our understanding of the human brain?

00:06:37.660 | And a lot of people say, scientists say,

00:06:39.660 | we're not very far in understanding the human brain,

00:06:41.740 | but you're saying we're in the dark here.

00:06:44.620 | - Well, I know I'm not unique.

00:06:46.220 | I don't even think in the clarity,

00:06:47.300 | but if you talk to real neuroscientists

00:06:48.940 | that really study real synapses or real neurons,

00:06:51.620 | they agree, they agree.

00:06:53.140 | It's a hundreds of year task

00:06:55.700 | and they're building it up slowly and surely.

00:06:58.260 | What the signal is there is not clear.

00:07:00.840 | We have all of our metaphors.

00:07:02.600 | We think it's electrical, maybe it's chemical.

00:07:05.460 | It's a whole soup.

00:07:06.620 | It's ions and proteins and it's a cell.

00:07:09.420 | And that's even around like a single synapse.

00:07:11.060 | If you look at a electron micrograph of a single synapse,

00:07:13.780 | it's a city of its own.

00:07:15.820 | And that's one little thing on a dendritic tree,

00:07:18.420 | which is extremely complicated,

00:07:19.980 | you know, electrochemical thing.

00:07:21.900 | And it's doing these spikes and voltages

00:07:24.060 | are even flying around and then proteins are taking that

00:07:26.100 | and taking it down into the DNA and who knows what.

00:07:29.380 | So it is the problem of the next few centuries.

00:07:31.700 | It is fantastic.

00:07:33.240 | But we have our metaphors about it.

00:07:34.900 | Is it an economic device?

00:07:36.100 | Is it like the immune system?

00:07:37.420 | Or is it like a layered, you know, set of,

00:07:39.780 | you know, arithmetic computations?

00:07:41.980 | We have all these metaphors and they're fun.

00:07:44.740 | But that's not real science per se.

00:07:48.060 | There is neuroscience.

00:07:48.980 | That's not neuroscience.

00:07:50.500 | All right, that's like the Greeks speculating

00:07:52.540 | about how to get to the moon.

00:07:53.460 | Fun, right?

00:07:55.180 | And I think that I like to say this fairly strongly

00:07:57.900 | 'cause I think a lot of young people

00:07:59.140 | think that we're on the verge.

00:08:00.780 | Because a lot of people who don't talk about it clearly

00:08:03.140 | let it be understood that yes,

00:08:04.860 | we kind of, this is brain inspired.

00:08:06.460 | We're kind of close.

00:08:07.740 | You know, breakthroughs are on the horizon.

00:08:10.100 | And unscrupulous people sometimes

00:08:12.020 | who need money for their labs.

00:08:13.520 | As I'm saying, unscrupulous.

00:08:15.460 | But people will oversell.

00:08:17.180 | I need money for my lab.

00:08:18.140 | I'm studying computational neuroscience.

00:08:22.580 | I'm gonna oversell it.

00:08:23.780 | And so there's been too much of that.

00:08:25.140 | - So let's step into the gray area

00:08:28.300 | between metaphor and engineering with,

00:08:31.060 | I'm not sure if you're familiar

00:08:32.660 | with brain computer interfaces.

00:08:35.360 | So a company like Elon Musk has Neuralink

00:08:38.900 | that's working on putting electrodes into the brain

00:08:42.620 | and trying to be able to read,

00:08:43.940 | both read and send electrical signals.

00:08:46.260 | Just as you said, even the basic mechanism of communication

00:08:51.260 | in the brain is not something we understand.

00:08:55.100 | But do you hope without understanding

00:08:57.980 | the fundamental principles of how the brain works,

00:09:00.780 | we'll be able to do something interesting

00:09:04.200 | at that gray area of metaphor?

00:09:06.460 | - It's not my area.

00:09:07.300 | So I hope in the sense like anybody else hopes

00:09:09.860 | for some interesting things to happen from research.

00:09:12.540 | I would expect more something like Alzheimer's

00:09:14.760 | will get figured out from modern neuroscience.

00:09:17.100 | That, you know, there's a lot of human suffering

00:09:18.900 | based on brain disease.

00:09:20.620 | And we throw things like lithium at the brain,

00:09:23.220 | it kind of works.

00:09:24.060 | No one has a clue why.

00:09:25.700 | That's not quite true, but you know, mostly we don't know.

00:09:28.020 | And that's even just about the biochemistry of the brain

00:09:30.740 | and how it leads to mood swings and so on.

00:09:32.920 | How thought emerges from that.

00:09:34.620 | We just, we were really, really completely dim.

00:09:38.060 | So that you might wanna hook up electrodes

00:09:39.900 | and try to do some signal processing on that

00:09:41.620 | and try to find patterns.

00:09:43.100 | Fine, you know, by all means go for it.

00:09:45.500 | It's just not scientific at this point.

00:09:47.580 | It's just, so it's like kind of sitting in a satellite

00:09:50.500 | and watching the emissions from a city

00:09:53.100 | and trying to infer things about the microeconomy,

00:09:55.140 | even though you don't have microeconomic concepts.

00:09:57.420 | I mean, it's really that kind of thing.

00:09:58.820 | And so yes, can you find some signals

00:10:00.720 | that do something interesting or useful?

00:10:02.380 | Can you control a cursor or mouse with your brain?

00:10:06.400 | Yeah, absolutely.

00:10:07.740 | You know, and I can imagine business models based on that.

00:10:11.180 | And even, you know, medical applications of that.

00:10:14.020 | But from there to understanding algorithms

00:10:16.300 | that allow us to really tie in deeply

00:10:19.060 | to from the brain to the computer,

00:10:20.580 | you know, I just, no, I don't agree with Elon Musk.

00:10:22.580 | I don't think that's even, that's not for our generation.

00:10:24.740 | It's not even for the century.

00:10:26.440 | - So just in hopes of getting you to dream,

00:10:30.780 | you've mentioned Kolmogorov and Turing might pop up.

00:10:34.460 | Do you think that there might be breakthroughs

00:10:38.380 | that will get you to sit back in five, 10 years

00:10:41.340 | and say, wow.

00:10:43.900 | - Oh, I'm sure there will be,

00:10:45.220 | but I don't think that there'll be demos that impress me.

00:10:49.220 | I don't think that having a computer call a restaurant

00:10:52.500 | and pretend to be a human is a breakthrough.

00:10:56.340 | And people, you know, some people present it as such.

00:10:59.660 | It's imitating human intelligence.

00:11:01.540 | It's even putting coughs in the thing

00:11:04.740 | to make a bit of a PR stunt.

00:11:07.020 | And so fine, the world runs on those things too.

00:11:11.060 | And I don't want to diminish all the hard work

00:11:13.780 | and engineering that goes behind things like that

00:11:15.340 | and the ultimate value to the human race.

00:11:17.660 | But that's not scientific understanding.

00:11:19.620 | And I know the people that work on these things,

00:11:22.020 | they are after scientific understanding.

00:11:23.580 | You know, in the meantime, they've got to kind of,

00:11:25.300 | you know, the train's got to run

00:11:26.260 | and they got mouths to feed and they got things to do.

00:11:28.580 | And there's nothing wrong with all that.

00:11:30.500 | I would call that though, just engineering.

00:11:32.500 | And I want to distinguish that

00:11:33.780 | between an engineering field

00:11:34.820 | like electrical engineering,

00:11:35.980 | that originally emerged, that had real principles

00:11:38.940 | and you really know what you're doing

00:11:39.900 | and you have a little scientific understanding,

00:11:41.760 | maybe not even complete.

00:11:43.580 | So it became more predictable

00:11:44.860 | and it was really gave value to human life

00:11:46.940 | because it was understood.

00:11:48.620 | And so we don't want to muddle too much these waters

00:11:52.460 | of what we're able to do versus what we really can do

00:11:56.140 | in a way that's going to impress the next.

00:11:57.980 | So I don't need to be wowed,

00:11:59.420 | but I think that someone comes along in 20 years,

00:12:02.200 | a younger person who's absorbed all the technology

00:12:06.620 | and for them to be wowed,

00:12:07.840 | I think they have to be more deeply impressed.

00:12:09.460 | A young Kolmogorov would not be wowed

00:12:11.100 | by some of the stunts that you see right now

00:12:12.820 | coming from the big companies.

00:12:13.860 | - The demos, but do you think the breakthroughs

00:12:15.980 | from Kolmogorov would be,

00:12:18.180 | and give this question a chance,

00:12:19.540 | do you think there'll be in the scientific

00:12:21.940 | fundamental principles arena

00:12:24.020 | or do you think it's possible

00:12:25.460 | to have fundamental breakthroughs in engineering?

00:12:28.220 | Meaning, I would say some of the things

00:12:31.020 | that Elon Musk is working with SpaceX and then others

00:12:34.620 | sort of trying to revolutionize the fundamentals

00:12:37.220 | of engineering, of manufacturing,

00:12:38.940 | of saying here's a problem we know how to do a demo of

00:12:42.680 | and actually taking it to scale.

00:12:44.580 | - Yeah, so there's gonna be all kinds of breakthroughs.

00:12:46.980 | I just don't like that terminology.

00:12:48.320 | I'm a scientist and I work on things day in and day out

00:12:50.700 | and things move along

00:12:51.540 | and eventually say, wow, something happened,

00:12:53.340 | but I don't like that language very much.

00:12:55.900 | Also, I don't like to prize theoretical breakthroughs

00:12:59.360 | over practical ones.

00:13:01.060 | I tend to be more of a theoretician

00:13:02.620 | and I think there's lots to do in that arena right now.

00:13:05.640 | And so I wouldn't point to the Kolmogorovs,

00:13:08.180 | I might point to the Edisons of the era

00:13:09.700 | and maybe Musk is a bit more like that.

00:13:11.680 | But Musk, God bless him, also will say things about AI

00:13:16.580 | that he knows very little about

00:13:18.020 | and he leads people astray when he talks about things

00:13:22.140 | he doesn't know anything about.

00:13:23.540 | Trying to program a computer to understand natural language,

00:13:26.180 | to be involved in a dialogue like we're having right now,

00:13:28.740 | ain't gonna happen in our lifetime.

00:13:30.380 | You could fake it, you can mimic,

00:13:32.500 | sort of take old sentences that humans use and retread them,

00:13:35.980 | but the deep understanding of language,

00:13:37.580 | no, it's not gonna happen.

00:13:38.500 | And so from that, I hope you can perceive that deeper,

00:13:41.580 | yet deeper kind of aspects of intelligence

00:13:43.580 | are not gonna happen.

00:13:44.420 | Now, will there be breakthroughs?

00:13:45.660 | I think that Google was a breakthrough.

00:13:48.060 | I think Amazon's a breakthrough.

00:13:49.620 | I think Uber is a breakthrough.

00:13:51.220 | Bring value to human beings at scale in brand new ways

00:13:54.660 | based on data flows and so on.

00:13:56.900 | A lot of these things are slightly broken

00:13:58.660 | because there's not a kind of a engineering field

00:14:01.780 | that takes economic value in context of data

00:14:04.180 | and at planetary scale and worries about

00:14:07.480 | all the externalities, the privacy.

00:14:09.460 | You know, we don't have that field,

00:14:10.980 | so we don't think these things through very well.

00:14:12.980 | But I see that as emerging and that will be,

00:14:16.180 | looking back from 100 years,

00:14:17.380 | that will be a breakthrough in this era,

00:14:19.220 | just like electrical engineering was a breakthrough

00:14:21.340 | in the early part of the last century

00:14:22.860 | and chemical engineering was a breakthrough.

00:14:24.500 | - So the scale, the markets that you talk about

00:14:26.500 | and we'll get to, will be seen as sort of breakthrough.

00:14:30.460 | And we're in the very early days

00:14:31.820 | of really doing interesting stuff there.

00:14:33.860 | And we'll get to that,

00:14:34.820 | but it's just taking a quick step back.

00:14:37.300 | Can you give, we kind of threw off the historian hat.

00:14:42.140 | I mean, you briefly said that the history of AI

00:14:46.420 | kind of mimics the history of chemical engineering,

00:14:48.820 | but-- - I keep saying machine learning,

00:14:50.300 | you keep wanting to say AI,

00:14:51.260 | just to let you know, I don't, you know, I resist that.

00:14:54.300 | I don't think this is about,

00:14:55.340 | AI really was John McCarthy as almost a philosopher

00:15:00.340 | saying, "Wouldn't it be cool

00:15:01.740 | "if we could put thought in a computer?

00:15:03.380 | "If we could mimic the human capability to think

00:15:06.180 | "or put intelligence in in some sense into a computer?"

00:15:09.700 | That's an interesting philosophical question

00:15:11.860 | and he wanted to make it more than philosophy.

00:15:13.540 | He wanted to actually write down logical formula

00:15:15.260 | and algorithms that would do that.

00:15:17.220 | And that is a perfectly valid, reasonable thing to do.

00:15:19.860 | That's not what's happening in this era.

00:15:21.860 | - So the reason I keep saying AI actually,

00:15:25.380 | and I'd love to hear what you think about it,

00:15:27.380 | machine learning has a very particular set

00:15:31.420 | of methods and tools.

00:15:33.140 | - Maybe your version of it is that mine doesn't.

00:15:36.220 | - No, it doesn't. - Mine is very, very open.

00:15:37.700 | It does optimization, it does sampling, it does--

00:15:40.060 | - So systems that learn is what machine learning is.

00:15:42.460 | - Systems that learn and make decisions.

00:15:44.420 | - And make decisions.

00:15:45.460 | So it's not just pattern recognition and finding patterns.

00:15:48.620 | It's all about making decisions in real worlds

00:15:50.820 | and having close feedback loops.

00:15:52.420 | - So something like symbolic AI, expert systems,

00:15:55.300 | reasoning systems, knowledge-based representation,

00:15:58.220 | all of those kinds of things, search,

00:16:00.020 | does that neighbor fit into what you think

00:16:03.620 | of as machine learning?

00:16:04.620 | - So I don't even like the word machine learning.

00:16:06.140 | I think that with the field you're talking about

00:16:07.940 | is all about making large collections of decisions

00:16:10.140 | under uncertainty by large collections of entities.

00:16:12.900 | - Yes. - Right?

00:16:13.740 | And there are principles for that at that scale.

00:16:15.980 | You don't have to say the principles are for a single entity

00:16:18.100 | that's making decisions, a single agent or a single human.

00:16:20.480 | It really immediately goes to the network of decisions.

00:16:22.900 | - Is a good word for that or no?

00:16:24.060 | - No, there's no good words for any of this.

00:16:25.280 | That's kind of part of the problem.

00:16:27.140 | So we can continue the conversation to use AI for all that.

00:16:29.940 | I just want to kind of raise a flag here

00:16:32.860 | that this is not about, we don't know what intelligence is

00:16:37.020 | and real intelligence.

00:16:38.060 | We don't know much about abstraction and reasoning

00:16:40.060 | at the level of humans.

00:16:40.940 | We don't have a clue.

00:16:41.780 | We're not trying to build that because we don't have a clue.

00:16:44.300 | Eventually it may emerge.

00:16:46.180 | I don't know if there'll be breakthroughs

00:16:47.380 | but eventually we'll start to get glimmers of that.

00:16:50.020 | It's not what's happening there right now.

00:16:51.860 | We're taking data,

00:16:52.820 | we're trying to make good decisions based on that.

00:16:54.540 | We're trying to do a scale,

00:16:55.360 | we're trying to do it economically viably.

00:16:56.700 | We're trying to build markets.

00:16:58.160 | We're trying to keep value at that scale.

00:17:00.780 | And aspects of this will look intelligent.

00:17:03.700 | They will look, computers were so dumb before.

00:17:06.980 | They will seem more intelligent.

00:17:08.100 | We will use that buzzword of intelligence.

00:17:09.800 | So we can use it in that sense.

00:17:11.200 | But, so machine learning, you can scope it narrowly

00:17:14.860 | as just learning from data and pattern recognition.

00:17:17.820 | But whatever, when I talk about these topics,

00:17:20.440 | maybe data science is another word

00:17:21.820 | you could throw in the mix.

00:17:23.580 | It really is important that the decisions are,

00:17:25.820 | as part of it, it's consequential decisions

00:17:28.100 | in the real world.

00:17:29.020 | Am I gonna have a medical operation?

00:17:30.780 | Am I gonna drive down the street?

00:17:33.100 | Things that, where there's scarcity.

00:17:35.060 | Things that impact other human beings

00:17:36.940 | or other, the environment and so on.

00:17:39.260 | How do I do that based on data?

00:17:40.740 | How do I do that adaptively?

00:17:41.640 | How do I use computers to help

00:17:42.760 | those kind of things go forward?

00:17:43.880 | Whatever you wanna call that.

00:17:45.540 | So let's call it AI.

00:17:46.380 | Let's agree to call it AI.

00:17:47.420 | But it's, let's not say that what the goal of that is

00:17:51.980 | is intelligence.

00:17:52.920 | The goal of that is really good working systems

00:17:54.800 | at planetary scale that we've never seen before.

00:17:56.540 | - So reclaim the word AI from the Dartmouth conference

00:17:59.260 | from many decades ago of the dream of human--

00:18:01.340 | - I don't wanna reclaim it.

00:18:02.160 | I want a new word.

00:18:03.000 | I think it was a bad choice.

00:18:04.180 | I mean, if you read one of my little things,

00:18:07.080 | the history was basically that McCarthy needed a new name

00:18:10.360 | 'cause cybernetics already existed.

00:18:12.160 | And he didn't like, you know,

00:18:13.840 | no one really liked Norbert Wiener.

00:18:15.280 | Norbert Wiener was kind of an island to himself.

00:18:17.320 | And he felt that he had encompassed all this.

00:18:19.640 | And in some sense he did.

00:18:21.120 | You look at the language of cybernetics,

00:18:23.040 | it was everything we're talking about.

00:18:24.320 | It was control theory and signal processing

00:18:26.040 | and some notions of intelligence

00:18:27.620 | and closed feedback loops and data.

00:18:29.720 | It was all there.

00:18:30.800 | It's just not a word that lived on partly

00:18:32.360 | 'cause of the maybe the personalities.

00:18:34.120 | But McCarthy needed a new word to say,

00:18:35.720 | I'm different from you.

00:18:36.780 | I'm not part of your show.

00:18:38.400 | I got my own.

00:18:39.640 | Invented this word.

00:18:41.520 | And again, as a kind of a,

00:18:43.320 | thinking forward about the movies

00:18:45.440 | that would be made about it,

00:18:47.240 | it was a great choice.

00:18:48.600 | But thinking forward about creating a sober academic

00:18:50.840 | and real world discipline, it was a terrible choice

00:18:52.760 | because it led to promises that are not true,

00:18:54.880 | that we understand.

00:18:56.240 | We understand artificial perhaps,

00:18:57.680 | but we don't understand intelligence.

00:18:59.280 | - As a small tangent,

00:19:00.400 | because you're one of the great personalities

00:19:02.320 | of machine learning, whatever the heck you call the field,

00:19:05.160 | do you think science progresses by personalities

00:19:09.520 | or by the fundamental principles and theories

00:19:11.880 | and research that's outside of personalities?

00:19:15.040 | - Yeah, both.

00:19:15.880 | And I wouldn't say there should be one kind of personality.

00:19:17.600 | I have mine and I have my preferences

00:19:19.440 | and I have a kind of network around me that feeds me

00:19:23.520 | and some of them agree with me and some of them disagree,

00:19:25.560 | but all kinds of personalities are needed.

00:19:28.300 | Right now, I think the personality

00:19:29.800 | that it's a little too exuberant,

00:19:31.000 | a little bit too ready to promise the moon

00:19:32.760 | is a little bit too much in ascendance.

00:19:34.720 | And I do think that there's some good to that.

00:19:38.080 | It certainly attracts lots of young people to our field,

00:19:40.600 | but a lot of those people come in with strong misconceptions

00:19:43.480 | and they have to then unlearn those

00:19:45.200 | and then find something to do.

00:19:48.240 | And so I think there's just gotta be some multiple voices

00:19:51.480 | and I wasn't hearing enough of the more sober voice.

00:19:54.760 | - So as a continuation of a fun tangent

00:19:58.200 | and speaking of vibrant personalities,

00:20:00.460 | what would you say is the most interesting disagreement

00:20:05.080 | you have with Jan LeCun?

00:20:07.280 | - So Jan's an old friend and I just say

00:20:09.880 | that I don't think we disagree about very much really.

00:20:13.360 | He and I both kind of have a let's build it kind of mentality

00:20:17.320 | and does it work kind of mentality and kind of concrete.

00:20:21.280 | We both speak French and we speak French more together

00:20:23.720 | and we have a lot in common.

00:20:25.640 | And so if one wanted to highlight a disagreement,

00:20:30.320 | it's not really a fundamental one.

00:20:31.840 | I think it's just kind of where we're emphasizing.

00:20:34.900 | Jan has emphasized pattern recognition

00:20:38.360 | and has emphasized prediction.

00:20:40.920 | And it's interesting to try to take that as far as you can.

00:20:45.320 | If you could do perfect prediction,

00:20:46.600 | what would that give you kind of as a thought experiment?

00:20:49.300 | And I think that's way too limited.

00:20:54.920 | We cannot do perfect prediction.

00:20:56.600 | We will never have the data sets

00:20:57.840 | that allow me to figure out what you're about ready to do,

00:20:59.560 | what question you're gonna ask next.

00:21:00.760 | I have no clue.

00:21:01.600 | I will never know such things.

00:21:02.900 | Moreover, most of us find ourselves during the day

00:21:06.140 | in all kinds of situations we had no anticipation of

00:21:09.220 | that are kind of various, novel in various ways.

00:21:13.240 | And in that moment, we want to think through what we want.

00:21:16.200 | And also there's gonna be market forces acting on us.

00:21:18.940 | I'd like to go down that street, but now it's full

00:21:20.920 | because there's a crane in the street.

00:21:22.320 | I gotta think about that.

00:21:23.780 | I gotta think about what I might really want here.

00:21:26.120 | And I gotta sort of think about how much it costs me

00:21:28.000 | to do this action versus this action.

00:21:30.120 | I gotta think about the risks involved.

00:21:32.860 | A lot of our current pattern recognition

00:21:34.580 | and prediction systems don't do any risk evaluations.

00:21:36.940 | They have no error bars.

00:21:38.980 | I gotta think about other people's decisions around me.

00:21:41.060 | I gotta think about a collection of my decisions.

00:21:43.500 | Even just thinking about like a medical treatment.

00:21:45.740 | I'm not gonna take the prediction of a neural net

00:21:48.740 | about my health, about something consequential.

00:21:51.020 | I might about ready to have a heart attack

00:21:52.180 | because some number is over 0.7.

00:21:54.380 | Even if you had all the data in the world

00:21:55.820 | that's ever been collected about heart attacks,

00:21:58.820 | better than any doctor ever had,

00:22:00.920 | I'm not gonna trust the output of that neural net

00:22:02.620 | to predict my heart attack.

00:22:03.900 | I'm gonna wanna ask what if questions around that.

00:22:06.460 | I'm gonna wanna look at some other possible data

00:22:08.420 | I didn't have, causal things.

00:22:10.420 | I'm gonna wanna have a dialogue with a doctor

00:22:12.340 | about things we didn't think about

00:22:13.780 | when we gathered the data.

00:22:14.980 | I could go on and on.

00:22:16.700 | I hope you can see.

00:22:17.580 | And I think that if you say prediction is everything,

00:22:20.260 | that you're missing all of this stuff.

00:22:23.160 | And so prediction plus decision-making is everything,

00:22:26.340 | but both of them are equally important.

00:22:28.100 | And so the field has emphasized prediction.

00:22:30.020 | Aeon, rightly so, has seen how powerful that is.

00:22:33.460 | But at the cost of people not being aware

00:22:35.700 | that decision-making is where the rubber really

00:22:37.300 | hits the road, where human lives are at stake,

00:22:39.460 | where risks are being taken,

00:22:41.120 | where you gotta gather more data,

00:22:42.300 | you gotta think about the error bars,

00:22:43.580 | you gotta think about the consequences

00:22:45.020 | of your decisions on others,

00:22:45.900 | you gotta think about the economy around your decisions,

00:22:47.460 | blah, blah, blah, blah.

00:22:48.980 | I'm not the only one working on those,

00:22:50.320 | but we're a smaller tribe.

00:22:51.980 | And right now we're not the one

00:22:53.740 | that people talk about the most.

00:22:56.020 | But if you go out in the real world, in industry,

00:22:58.700 | at Amazon, I'd say half the people there

00:23:01.300 | are working on decision-making

00:23:02.140 | and the other half are doing the pattern recognition.

00:23:04.780 | It's important.

00:23:05.620 | - And the words of pattern recognition and prediction,

00:23:07.660 | I think the distinction there, not to linger on words,

00:23:11.020 | but the distinction there is more a constraint

00:23:13.780 | sort of in the lab data set versus decision-making

00:23:17.340 | is talking about consequential decisions in the real world

00:23:20.320 | under the messiness and the uncertainty of the real world.

00:23:23.380 | And just the whole of it,

00:23:25.300 | the whole mess of it that actually touches human beings

00:23:27.140 | and scale, like you said, market forces,

00:23:29.180 | that's the distinction.

00:23:30.700 | - It helps add that perspective, that broader perspective.

00:23:33.580 | You're right, I totally agree.

00:23:35.620 | On the other hand, if you're a real prediction person,

00:23:37.380 | of course you want it to be in the real world,

00:23:38.540 | you wanna predict real world events.

00:23:39.700 | I'm just saying that's not possible with just data sets,

00:23:43.100 | that it has to be in the context of

00:23:44.780 | strategic things that someone's doing,

00:23:47.340 | data they might gather,

00:23:48.260 | things they could have gathered,

00:23:49.100 | the reasoning process around data.

00:23:50.540 | It's not just taking data

00:23:51.860 | and making predictions based on the data.

00:23:53.480 | - So one of the things that you're working on,

00:23:56.760 | I'm sure there's others working on it,

00:23:58.060 | but I don't hear often it talked about,

00:24:02.060 | especially in the clarity that you talk about it,

00:24:04.700 | and I think it's both the most exciting

00:24:06.420 | and the most concerning area of AI

00:24:09.500 | in terms of decision-making.

00:24:11.460 | So you've talked about AI systems

00:24:12.820 | that help make decisions that scale in a distributed way,

00:24:15.500 | millions, billions decisions,

00:24:17.800 | and sort of markets of decisions.

00:24:19.600 | Can you, as a starting point,

00:24:21.380 | sort of give an example of a system

00:24:23.140 | that you think about when you're thinking

00:24:25.660 | about these kinds of systems?

00:24:27.620 | - Yeah, so first of all,

00:24:28.500 | you're absolutely getting into some territory

00:24:30.160 | which I will be beyond my expertise,

00:24:32.240 | and there are lots of things

00:24:33.580 | that are gonna be very not obvious to think about.

00:24:35.580 | Just like, again, I like to think about history a little bit

00:24:38.860 | but think about, put yourself back in the '60s,

00:24:40.740 | there was kind of a banking system

00:24:41.940 | that wasn't computerized really.

00:24:43.340 | There was database theory emerging.

00:24:45.920 | And database people had to think about

00:24:47.500 | how do I actually not just move data around

00:24:49.100 | but actual money and have it be valid

00:24:52.640 | and have transactions at ATMs happen

00:24:54.560 | that are actually all valid and so on and so forth.

00:24:57.760 | So that's the kind of issues you get into

00:25:00.140 | when you start to get serious about things like this.

00:25:02.980 | I like to think about,

00:25:04.020 | as kind of almost a thought experiment,

00:25:05.580 | to help me think something simpler,

00:25:07.740 | which is the music market.

00:25:10.300 | 'Cause there is, to first order,

00:25:13.340 | there is no music market in the world right now,

00:25:16.300 | in our country, for sure.

00:25:18.620 | There are something called,

00:25:20.060 | things called record companies,

00:25:21.700 | and they make money,

00:25:23.140 | and they prop up a few really good musicians

00:25:26.600 | and make them superstars,

00:25:27.920 | and they all make huge amounts of money.

00:25:30.820 | But there's a long tail of huge numbers of people

00:25:32.760 | that make lots and lots of really good music

00:25:34.320 | that is actually listened to by more people

00:25:35.940 | than the famous people.

00:25:37.100 | They are not in a market, they cannot have a career.

00:25:42.680 | They do not make money.

00:25:43.800 | - The creators, the creators, right.

00:25:45.360 | - The creators, the so-called influencers or whatever,

00:25:47.640 | that diminishes who they are, right?

00:25:49.280 | So there are people who make extremely good music,

00:25:52.140 | especially in the hip hop or Latin world these days.

00:25:55.220 | They do it on their laptop,

00:25:56.220 | that's what they do on the weekend,

00:25:58.700 | and they have another job during the week,

00:26:00.940 | and they put it up on SoundCloud or other sites.

00:26:03.420 | Eventually, it gets streamed,

00:26:04.780 | it down gets turned into bits.

00:26:06.020 | It's not economically valuable,

00:26:07.460 | the information is lost.

00:26:08.900 | It gets put up there, people stream it.

00:26:11.260 | You walk around in a big city,

00:26:13.700 | you see people with headphones,

00:26:15.140 | especially young kids listening to music all the time.

00:26:17.140 | If you look at the data,

00:26:18.880 | very little of the music they're listening to

00:26:20.400 | is the famous people's music,

00:26:21.840 | and none of it's old music, it's all the latest stuff.

00:26:24.760 | But the people who made that latest stuff

00:26:26.060 | are like some 16-year-old somewhere

00:26:27.360 | who will never make a career out of this,

00:26:28.740 | who will never make money.

00:26:30.000 | Of course, there'll be a few counter examples.

00:26:31.440 | The record companies incentivize to pick out a few

00:26:33.560 | and highlight them.

00:26:35.160 | Long story short, there's a missing market there.

00:26:37.600 | There is not a consumer-producer relationship

00:26:40.200 | at the level of the actual creative acts.

00:26:42.300 | The pipelines and Spotify's of the world

00:26:45.680 | that take this stuff and stream it along,

00:26:47.960 | they make money off of subscriptions or advertising

00:26:50.620 | and those things, they're making the money.

00:26:52.340 | And then they will offer bits and pieces of it

00:26:54.220 | to a few people, again, to highlight that,

00:26:56.220 | simulate a market.

00:26:58.620 | Anyway, a real market would be,

00:27:00.620 | if you're a creator of music,

00:27:02.180 | that you actually are somebody who's good enough

00:27:04.240 | that people wanna listen to you,

00:27:06.180 | you should have the data available to you.

00:27:07.780 | There should be a dashboard

00:27:09.460 | showing a map of the United States,

00:27:11.340 | showing last week,

00:27:12.220 | here's all the places your songs were listened to.

00:27:14.580 | It should be transparent,

00:27:17.100 | vettable so that if someone down in Providence

00:27:19.580 | sees that you're being listened to 10,000 times

00:27:21.900 | in Providence, that they know that's real data,

00:27:24.060 | you know it's real data,

00:27:25.260 | they will have you come give a show down there.

00:27:27.220 | They will broadcast to the people

00:27:28.460 | who've been listening to you that you're coming.

00:27:30.300 | If you do this right,

00:27:31.140 | you could go down there and make $20,000.

00:27:34.380 | You do that three times a year, you start to have a career.

00:27:37.000 | So in this sense, AI creates jobs.

00:27:39.460 | It's not about taking away human jobs,

00:27:40.660 | it's creating new jobs because it creates a new market.

00:27:43.340 | Once you've created a market,

00:27:44.300 | you've now connected up producers and consumers.

00:27:46.740 | The person who's making the music

00:27:48.180 | can say to someone who comes to their shows a lot,

00:27:49.820 | "Hey, I'll play at your daughter's wedding for $10,000."

00:27:53.060 | You'll say 8,000, they'll say 9,000.

00:27:55.100 | Then again, you can now get an income up to $100,000.

00:27:58.940 | You're not gonna be a millionaire.

00:28:01.100 | All right?

00:28:01.940 | And now even think about really the value of music

00:28:05.020 | is in these personal connections,

00:28:06.660 | even so much so that a young kid wants to wear a T-shirt

00:28:11.380 | with their favorite musician's signature on it, right?

00:28:14.740 | So if they listen to the music on the internet,

00:28:16.820 | the internet should be able to provide them

00:28:18.140 | with a button that they push

00:28:19.420 | and the merchandise arrives the next day.

00:28:21.740 | We can do that, right?

00:28:22.900 | And now why should we do that?

00:28:24.300 | Well, because the kid who bought the shirt will be happy,

00:28:26.340 | but more the person who made the music will get the money.

00:28:29.500 | There's no advertising needed, right?

00:28:32.260 | So you can create markets between producers and consumers,

00:28:35.160 | take 5% cut, your company will be perfectly sound,

00:28:39.060 | it'll go forward into the future

00:28:40.540 | and it will create new markets

00:28:41.820 | and that raises human happiness.

00:28:44.620 | Now, this seems like it was easy,

00:28:46.700 | just create this dashboard,

00:28:47.780 | kind of create some connections and all that.

00:28:49.140 | But if you think about Uber or whatever,

00:28:50.860 | you think about the challenges in the real world

00:28:52.980 | of doing things like this.

00:28:53.900 | And there are actually new principles gonna be needed.

00:28:56.100 | You're trying to create a new kind of two-way market

00:28:57.980 | at a different scale that's ever been done before.

00:28:59.880 | There's gonna be unwanted aspects of the market,

00:29:04.660 | there'll be bad people,

00:29:05.780 | the data will get used in the wrong ways,

00:29:09.500 | it'll fail in some ways, it won't deliver value.

00:29:11.460 | You have to think that through,

00:29:12.520 | just like anyone who ran a big auction

00:29:14.460 | or ran a big matching service in economics,

00:29:17.180 | will think these things through.

00:29:18.780 | And so that maybe doesn't get at all the huge issues

00:29:21.300 | that can arise when you start to create markets,

00:29:22.620 | but it starts, at least for me, solidify my thoughts

00:29:25.740 | and allow me to move forward in my own thinking.

00:29:28.540 | - Yeah, so I talked to, had a research at Spotify,

00:29:31.140 | actually, I think their long-term goal, they've said,

00:29:33.420 | is to have at least one million creators

00:29:36.660 | make a comfortable living putting on Spotify.

00:29:41.020 | So, and I think you articulate a really nice vision

00:29:46.020 | of the world and the digital,

00:29:51.200 | in the cyberspace of markets.

00:29:53.400 | What do you think companies like Spotify or YouTube

00:29:57.500 | or Netflix can do to create such markets?

00:30:02.500 | Is it an AI problem?

00:30:05.360 | Is it an interface problem?

00:30:06.840 | So interface design?

00:30:08.440 | Is it some other kind of, is it an economics problem?

00:30:13.080 | Who should they hire to solve these problems?

00:30:15.600 | - Well, part of it's not just top-down.

00:30:17.400 | So the Silicon Valley has this attitude

00:30:18.940 | that they know how to do it.

00:30:19.960 | They will create the system,

00:30:21.200 | just like Google did with the search box,

00:30:22.720 | that will be so good that they'll just,

00:30:23.920 | everyone will adopt that, right?

00:30:26.000 | It's not, it's everything you said,

00:30:28.680 | but really, I think missing that kind of culture.

00:30:31.200 | All right, so it's literally that 16-year-old

00:30:32.920 | who's able to create the songs.

00:30:34.720 | You don't create that as a Silicon Valley entity.

00:30:36.940 | You don't hire them per se, right?

00:30:39.260 | You have to create an ecosystem in which they are wanted

00:30:42.680 | and that they're belong, right?

00:30:44.340 | And so you have to have some cultural credibility

00:30:46.220 | to do things like this.

00:30:47.500 | You know, Netflix, to their credit,

00:30:49.340 | wanted some of that sort of credibility.

00:30:50.980 | They created shows, you know, content.

00:30:53.020 | They call it content.

00:30:53.900 | It's such a terrible word, but it's culture, right?

00:30:56.700 | And so with movies, you can kind of go give

00:30:59.020 | a large sum of money to somebody

00:31:00.860 | graduating from the USC film school.

00:31:03.340 | It's a whole thing of its own,

00:31:04.400 | but it's kind of like rich white people's thing to do.

00:31:07.200 | You know, and American culture has not been so much

00:31:10.780 | about rich white people.

00:31:11.700 | It's been about all the immigrants,

00:31:13.100 | all the Africans who came and brought that culture

00:31:16.820 | and those rhythms to this world

00:31:20.900 | and created this whole new thing, you know,

00:31:22.660 | American culture.

00:31:23.660 | And so companies can't artificially create that.

00:31:26.780 | They can't just say, "Hey, we're here.

00:31:28.340 | We're gonna buy it up."

00:31:29.780 | You got a partner.

00:31:31.140 | And so, but anyway, you know,

00:31:33.640 | not to denigrate, these companies are all trying

00:31:35.260 | and they should, and I'm sure they're asking these questions

00:31:38.660 | and some of them are even making an effort,

00:31:40.100 | but it is partly a respect the culture

00:31:42.820 | as you are as a technology person.

00:31:44.380 | You got to blend your technology with cultural meaning.

00:31:49.380 | - How much of a role do you think the algorithm,

00:31:52.220 | so machine learning has in connecting the consumer

00:31:55.180 | to the creator, sort of the recommender system

00:31:58.660 | aspect of this?

00:31:59.500 | - Yeah, it's a great question.

00:32:00.780 | I think pretty high.

00:32:03.380 | There's no magic in the algorithms,

00:32:04.820 | but a good recommender system is way better

00:32:07.260 | than a bad recommender system.

00:32:08.860 | And recommender systems was a billion dollar industry

00:32:11.700 | back even 10, 20 years ago.

00:32:13.800 | And it continues to be extremely important going forward.

00:32:17.380 | - What's your favorite recommender system,

00:32:18.760 | just so we can put something?

00:32:19.740 | - Well, just historically, I was one of the,

00:32:21.380 | you know, when I first went to Amazon,

00:32:22.860 | I first didn't like Amazon

00:32:24.460 | 'cause they put the book people are out of business,

00:32:26.300 | the library, you know,

00:32:27.140 | the local booksellers went out of business.

00:32:29.240 | I've come to accept that there, you know,

00:32:32.300 | there probably are more books being sold now

00:32:34.180 | and poor people reading them than ever before.

00:32:36.580 | And then local books stores are coming back.

00:32:39.340 | So, you know, that's how economics sometimes work.

00:32:41.480 | You go up and you go down.

00:32:42.780 | But anyway, when I finally started going there

00:32:46.300 | and I bought a few books,

00:32:47.460 | I was really pleased to see another few books

00:32:49.900 | being recommended to me that I never would have thought of.

00:32:52.100 | And I bought a bunch of them.

00:32:53.300 | So they obviously had a good business model,

00:32:55.220 | but I learned things.

00:32:56.120 | And I still to this day kind of browse using that service.

00:33:00.860 | And I think lots of people get a lot, you know,

00:33:03.780 | that is a good aspect of a recommendation system.

00:33:05.780 | I'm learning from my peers in an indirect way.

00:33:08.960 | And their algorithms are not meant to have them impose

00:33:12.860 | what we learn.

00:33:13.860 | It really is trying to find out what's in the data.

00:33:16.600 | It doesn't work so well for other kinds of entities,

00:33:18.620 | but that's just the complexity of human life like shirts.

00:33:20.880 | You know, I'm not gonna get recommendations on shirts.

00:33:23.780 | But that's interesting.

00:33:26.180 | If you try to recommend restaurants,

00:33:29.900 | it's hard, it's hard to do it at scale.

00:33:33.420 | But a blend of recommendation systems

00:33:37.440 | with other economic ideas, matchings and so on

00:33:42.140 | is really, really still very open research wise.

00:33:45.200 | And there's new companies that are gonna emerge

00:33:46.380 | that do that well.

00:33:47.360 | - What do you think is going to the messy, difficult land

00:33:52.180 | of say politics and things like that,

00:33:54.420 | that YouTube and Twitter have to deal with

00:33:56.220 | in terms of recommendation systems,

00:33:58.340 | being able to suggest, I think Facebook

00:34:01.020 | just launched Facebook News.

00:34:02.980 | So they're having, recommend the kind of news

00:34:06.580 | that are most likely for you to be interesting.

00:34:09.340 | You think this is a AI solvable,

00:34:12.340 | again, whatever term you wanna use.

00:34:13.580 | Do you think it's a solvable problem for machines

00:34:15.860 | or is this a deeply human problem that's unsolvable?

00:34:18.660 | - So I don't even think about it that level.

00:34:20.260 | I think that what's broken with some of these companies,

00:34:22.440 | it's all monetization by advertising.

00:34:25.300 | They're not, at least Facebook, I wanna critique them,

00:34:28.020 | that they didn't really try to connect a producer

00:34:30.340 | and a consumer in an economic way.

00:34:32.660 | No one wants to pay for anything.

00:34:34.620 | And so they all, starting with Google, then Facebook,

00:34:37.260 | they went back to the playbook of the television companies

00:34:40.280 | back in the day.

00:34:41.380 | No one wanted to pay for this signal.

00:34:43.180 | They will pay for the TV box, but not for the signal,

00:34:45.700 | at least back in the day.

00:34:47.100 | And so advertising kind of filled that gap

00:34:48.860 | and advertising was new and interesting

00:34:50.380 | and it somehow didn't take over our lives quite.

00:34:52.740 | Fast forward, Google provides a service

00:34:56.260 | that people don't wanna pay for.

00:34:58.300 | And so somewhat surprisingly in the '90s,

00:35:02.380 | they made, ended up making huge amounts.

00:35:04.260 | They cornered the advertising market.

00:35:05.620 | It didn't seem like that was gonna happen, at least to me.

00:35:08.340 | These little things on the right-hand side of the screen

00:35:10.060 | just did not seem all that economically interesting,

00:35:12.220 | but that companies had maybe no other choice.

00:35:14.340 | The TV market was going away and billboards and so on.

00:35:17.780 | So they got it.

00:35:19.620 | And I think that sadly that Google just has,

00:35:22.940 | it was doing so well with that and making such money,

00:35:24.980 | it didn't think much more about how, wait a minute,

00:35:27.180 | is there a producer-consumer relationship

00:35:28.780 | to be set up here?

00:35:29.620 | Not just between us and the advertisers,

00:35:31.860 | market to be created.

00:35:32.860 | Is there an actual market between the producer and consumer?

00:35:35.060 | They're the producers,

00:35:35.940 | the person who created that video clip,

00:35:37.380 | the person that made that website,

00:35:38.980 | the person who could make more such things,

00:35:40.620 | the person who could adjust it as a function of demand,

00:35:43.620 | the person on the other side

00:35:44.940 | who's asking for different kinds of things.

00:35:47.100 | So you see glimmers of that now,

00:35:48.980 | there's influencers and there's kind of

00:35:50.540 | a little glimmering of a market,

00:35:51.940 | but it should have been done 20 years ago.

00:35:53.580 | It should have been thought about.

00:35:54.420 | It should have been created in parallel

00:35:55.820 | with the advertising ecosystem.

00:35:58.020 | And then Facebook inherited that.

00:35:59.820 | And I think they also didn't think very much about that.

00:36:02.980 | So fast forward and now they are making huge amounts

00:36:06.180 | of money off of advertising.

00:36:07.700 | And the news thing and all these clicks

00:36:09.780 | is just feeding the advertising.

00:36:11.460 | It's all connected up to the advertising.

00:36:13.540 | So you want more people to click on certain things

00:36:15.500 | because that money flows to you, Facebook.

00:36:18.460 | You're very much incentivized to do that.

00:36:19.980 | And when you start to find it's breaking,

00:36:22.220 | so people were telling you,

00:36:23.060 | well, we're getting into some troubles,

00:36:24.060 | you try to adjust it with your smart AI algorithms, right?

00:36:27.500 | And figure out what are bad clicks.

00:36:28.940 | So maybe it shouldn't be click-through rate.

00:36:31.060 | I find that pretty much hopeless.

00:36:33.740 | It does get into all the complexity of human life

00:36:35.940 | and you can try to fix it, you should,

00:36:38.460 | but you could also fix the whole business model.

00:36:40.700 | And the business model is that really,

00:36:42.460 | what are, are there some human producers

00:36:44.140 | and consumers out there?

00:36:45.220 | Is there some economic value to be liberated

00:36:47.020 | by connecting them directly?

00:36:48.580 | Is it such that it's so valuable

00:36:50.700 | that people will be willing to pay for it?

00:36:53.060 | All right.

00:36:53.900 | - Micro payments, like small payments.

00:36:54.740 | - Micro, but even after you micro,

00:36:56.500 | so I like the example, suppose I'm going,

00:36:58.740 | next week I'm going to India,

00:36:59.940 | never been to India before, right?

00:37:02.460 | I have a couple of days in Mumbai.

00:37:05.220 | I have no idea what to do there, right?

00:37:06.740 | And I could go on the web right now and search.

00:37:08.740 | It's gonna be kind of hopeless.

00:37:10.060 | I'm not gonna find, you know,

00:37:12.140 | I'll have lots of advertisers in my face, right?

00:37:14.740 | What I really wanna do is broadcast to the world

00:37:16.900 | that I am going to Mumbai

00:37:18.500 | and have someone on the other side of a market look at me

00:37:21.820 | and there's a recommendation system there.

00:37:23.940 | So they're not looking at all possible people

00:37:25.180 | coming to Mumbai.

00:37:26.020 | They're looking at the people who are relevant to them.

00:37:27.620 | So someone my age group,

00:37:28.740 | someone who kind of knows me in some level.

00:37:31.860 | I give up a little privacy by that,

00:37:33.340 | but I'm happy because what I'm gonna get back

00:37:35.020 | is this person can make a little video for me

00:37:37.140 | or they're gonna write a little two page paper

00:37:38.860 | on here's the cool things that you want to do

00:37:40.740 | in Mumbai this week, especially, right?

00:37:43.180 | I'm gonna look at that.

00:37:44.020 | I'm not gonna pay a micro payment.

00:37:45.140 | I'm gonna pay, you know, $100 or whatever for that.

00:37:47.780 | It's real value.

00:37:48.700 | It's like journalism.

00:37:50.500 | And as a non-subscription,

00:37:52.260 | it's that I'm gonna pay that person in that moment.

00:37:54.620 | Company's gonna take 5% of that.

00:37:56.540 | And that person has now got it.

00:37:57.700 | It's a gig economy, if you will,

00:37:58.900 | but, you know, done for it, you know,

00:38:00.340 | thinking about a little bit behind YouTube,

00:38:02.420 | there was actually people

00:38:03.260 | who could make more of those things.

00:38:04.860 | If they were connected to a market,

00:38:06.140 | they would make more of those things independently.

00:38:07.900 | You don't have to tell them what to do.

00:38:08.900 | You don't have to incentivize them any other way.

00:38:11.400 | And so, yeah, these companies,

00:38:13.580 | I don't think I've thought long and heard about that.

00:38:15.660 | So I do distinguish on, you know,

00:38:18.060 | Facebook on the one side

00:38:19.020 | who's just not thought about these things at all.

00:38:20.460 | I think thinking that AI will fix everything.

00:38:23.620 | And Amazon thinks about them all the time

00:38:25.260 | because they were already out in the real world.

00:38:26.500 | They were delivering packages to people's doors.

00:38:28.020 | They were worried about a market.

00:38:29.380 | They were worried about sellers.

00:38:30.220 | And, you know, they worry

00:38:31.140 | and some things they do are great.

00:38:32.520 | Some things maybe not so great,

00:38:33.500 | but, you know, they're in that business model.

00:38:36.300 | And then I'd say Google sort of hovers somewhere in between.

00:38:38.420 | I don't think for a long, long time they got it.

00:38:41.460 | I think they probably see that YouTube

00:38:43.400 | is more pregnant with possibility

00:38:44.820 | than they might've thought

00:38:47.140 | and that they're probably heading that direction.

00:38:49.740 | But, you know, Silicon Valley has been dominated

00:38:52.380 | by the Google, Facebook kind of mentality

00:38:54.100 | and the subscription and advertising.

00:38:55.660 | And that's the core problem, right?

00:38:58.500 | The fake news actually rides on top of that

00:39:01.620 | 'cause it means that you're monetizing

00:39:03.580 | with clip through rate.

00:39:04.420 | And that is the core problem.

00:39:05.580 | You got to remove that.

00:39:06.780 | - So advertisement, if we're gonna linger on that,

00:39:09.620 | I mean, that's an interesting thesis.

00:39:11.220 | I don't know if everyone really deeply thinks about that.

00:39:14.940 | So you're right.

00:39:16.560 | The thought is the advertising model

00:39:19.220 | is the only thing we have,

00:39:20.260 | the only thing we'll ever have.

00:39:21.460 | So we have to fix,

00:39:23.500 | we have to build algorithms that,

00:39:25.900 | despite that business model,

00:39:28.740 | you know, find the better angels of our nature

00:39:31.740 | and do good by society and by the individual.

00:39:34.640 | But you think we can slowly,

00:39:37.060 | you think, first of all,

00:39:38.660 | there's a difference between should and could.

00:39:40.960 | So you're saying we should slowly move away

00:39:44.740 | from the advertising model

00:39:45.940 | and have a direct connection

00:39:47.180 | between the consumer and the creator.

00:39:49.660 | The question I also have is,

00:39:51.700 | can we, because the advertising model is so successful now

00:39:55.300 | in terms of just making a huge amount of money

00:39:58.460 | and therefore being able to build a big company

00:40:00.540 | that provides, has really smart people working

00:40:02.900 | that create a good service.

00:40:04.140 | Do you think it's possible?

00:40:05.460 | And just to clarify, you think we should move away?

00:40:08.020 | - Well, I think we should, yeah.

00:40:08.980 | But we is, you know, not me.

00:40:10.820 | - Society.

00:40:11.980 | - Yeah, well, the companies.

00:40:13.980 | I mean, so first of all, full disclosure,

00:40:15.340 | I'm doing a day a week at Amazon

00:40:16.940 | 'cause I kind of want to learn more about how they do things.

00:40:18.820 | So, you know, I'm not speaking for Amazon in any way,

00:40:20.980 | but, you know, I did go there

00:40:22.820 | because I actually believe they get a little bit of this

00:40:24.820 | or trying to create these markets.

00:40:26.140 | - And they don't really use,

00:40:27.860 | advertisement is not a crucial part of it.

00:40:29.580 | - That's a good question.

00:40:30.420 | So it has become not crucial,

00:40:32.500 | but it's become more and more present

00:40:33.900 | if you go to Amazon website.

00:40:35.260 | And, you know, without revealing too many deep secrets

00:40:37.980 | about Amazon, I can tell you that, you know,

00:40:39.740 | a lot of people in the company question this

00:40:41.060 | and there's a huge questioning going on.

00:40:43.480 | You do not want a world where there's zero advertising.

00:40:45.580 | That actually is a bad world, okay?

00:40:47.620 | So here's a way to think about it.

00:40:49.180 | You're a company that like Amazon

00:40:51.720 | is trying to bring products to customers, right?

00:40:54.860 | And the customer,

00:40:55.700 | and then you get more, you want to buy a vacuum cleaner,

00:40:57.300 | say, you want to know what's available for me.

00:40:59.420 | And, you know, it's not gonna be that obvious.

00:41:00.820 | You have to do a little bit of work at it.

00:41:02.140 | The recommendation system will sort of help, right?

00:41:04.540 | But now suppose this other person over here

00:41:06.580 | has just made the world, you know,

00:41:07.780 | they spent a huge amount of energy.

00:41:08.900 | They had a great idea.

00:41:09.740 | They made a great vacuum cleaner.

00:41:10.740 | They know, they really did it.

00:41:12.340 | They nailed it.

00:41:13.180 | It's an MIT, you know, whiz kid

00:41:14.460 | that made a great new vacuum cleaner, right?

00:41:16.540 | It's not gonna be in the recommendation system.

00:41:18.140 | No one will know about it.

00:41:19.180 | The algorithms will not find it

00:41:20.740 | and AI will not fix that, okay, at all, right?

00:41:24.180 | How do you allow that vacuum cleaner

00:41:26.280 | to start to get in front of people, be sold?

00:41:29.460 | Well, advertising.

00:41:30.580 | And here what advertising is,

00:41:31.900 | it's a signal that you believe in your product enough

00:41:35.480 | that you're willing to pay some real money for it.

00:41:37.340 | And to me as a consumer, I look at that signal.

00:41:39.500 | I say, well, first of all,

00:41:40.620 | I know these are not just cheap little ads

00:41:42.780 | 'cause we have now right now.

00:41:44.000 | I know that, you know, these are super cheap, you know,

00:41:46.380 | pennies.

00:41:47.660 | If I see an ad where it's actually,

00:41:49.140 | I know the company is only doing a few of these

00:41:51.020 | and they're making, you know, real money is kind of flowing

00:41:53.220 | and I see an ad, I may pay more attention to it.

00:41:55.140 | And I actually might want that because I see,

00:41:57.300 | hey, that guy spent money on his vacuum cleaner.

00:41:59.900 | Oh, maybe there's something good there.

00:42:02.260 | So I will look at it.

00:42:03.180 | And so that's part of the overall information flow

00:42:05.380 | in a good market.

00:42:06.540 | So advertising has a role.

00:42:08.060 | But the problem is, of course,

00:42:10.500 | that that signal is now completely gone

00:42:12.260 | because it just, you know,

00:42:13.720 | dominated by these tiny little things

00:42:15.120 | that add up to big money for the company.

00:42:17.320 | You know, so I think it will change

00:42:20.400 | because the societies just don't, you know,

00:42:22.440 | stick with things that annoy a lot of people.

00:42:24.680 | And advertising currently annoys people

00:42:26.360 | more than it provides information.

00:42:28.360 | So I think that a Google probably is smart enough

00:42:30.720 | to figure out that this is a dead, this is a bad model,

00:42:33.580 | even though it's a huge amount of money

00:42:35.040 | and they'll have to figure out

00:42:35.960 | how to pull it away from it slowly.

00:42:37.920 | And I'm sure the CEO there will figure it out,

00:42:39.800 | but they need to do it.

00:42:42.040 | And they needed it to,

00:42:44.100 | so if you reduce advertising, not to zero,

00:42:46.100 | but you reduce it at the same time you bring up

00:42:48.780 | producer, consumer, actual real value being delivered,

00:42:51.580 | so real money is being paid,

00:42:52.900 | and they take a 5% cut,

00:42:54.580 | that 5% could start to get big enough

00:42:56.340 | to cancel out the lost revenue

00:42:58.020 | from the kind of the poor kind of advertising.

00:43:00.060 | And I think that a good company will do that,

00:43:02.260 | will realize that.

00:43:03.240 | And their company, you know, Facebook,

00:43:06.580 | you know, again, God bless them.

00:43:08.140 | They bring, you know, grandmothers, you know,

00:43:11.800 | they bring children's pictures into grandmothers' lives,

00:43:14.600 | it's fantastic.

00:43:15.680 | But they need to think of a new business model.

00:43:19.240 | And that's the core problem there.

00:43:22.240 | Until they start to connect producer, consumer,

00:43:24.280 | I think they will just continue to make money

00:43:26.560 | and then buy the next social network company

00:43:28.480 | and then buy the next one.

00:43:29.840 | And the innovation level will not be high

00:43:31.640 | and the health issues will not go away.

00:43:34.800 | - So I apologize that we kind of returned to words.

00:43:38.360 | I don't think the exact terms matter,

00:43:40.940 | but in sort of defense of advertisement,

00:43:43.580 | don't you think the kind of direct connection

00:43:49.140 | between consumer and creator, producer,

00:43:53.520 | is the best, like the,

00:43:57.440 | is what advertisement strives to do, right?

00:44:00.840 | So that is best advertisement is literally now,

00:44:04.760 | Facebook is listening to our conversation

00:44:06.740 | and heard that you're going to India

00:44:08.620 | and will be able to actually start automatically

00:44:11.340 | for you making these connections

00:44:13.140 | and start giving this offer.

00:44:14.460 | So like, I apologize if it's just a matter of terms,

00:44:18.420 | but just to draw a distinction,

00:44:19.820 | is it possible to make advertisements

00:44:21.300 | just better and better and better algorithmically

00:44:23.120 | to where it actually becomes a connection,

00:44:25.540 | almost a direct connection? - That's a good question.

00:44:26.780 | So let's put it on the, push it on.

00:44:28.220 | First of all, what we just talked about,

00:44:30.720 | I was defending advertising, okay?

00:44:32.500 | So I was defending it as a way to get signals into a market

00:44:34.980 | that don't come any other way, especially algorithmically.

00:44:37.620 | It's a sign that someone spent money on it.

00:44:39.360 | It's a sign they think it's valuable.

00:44:41.520 | And if I think that if other things,

00:44:43.080 | someone else thinks it's valuable,

00:44:44.520 | then if I trust other people, I might be willing to listen.

00:44:47.400 | I don't trust that Facebook though,

00:44:49.680 | who's an intermediary between this.

00:44:51.880 | I don't think they care about me, okay?

00:44:55.380 | I don't think they do.

00:44:56.560 | And I find it creepy that they know I'm going to India

00:44:59.440 | next week because of our conversation.

00:45:01.080 | - Why do you think that is?

00:45:01.920 | Can we, so what, can you just put your PR hat on?

00:45:05.760 | (laughing)

00:45:07.000 | Why do you think you find Facebook creepy

00:45:10.500 | and not trust them as do majority of the population?

00:45:14.100 | So they're, out of the Silicon Valley companies,

00:45:16.280 | I saw like, not approval rate,

00:45:18.640 | but there's ranking of how much people trust companies

00:45:21.240 | and Facebook is in the gutter.

00:45:23.020 | - In the gutter, including people inside of Facebook.

00:45:25.480 | - So what do you attribute that to?

00:45:27.800 | Because when I--

00:45:28.640 | - Come on, you don't find it creepy

00:45:29.980 | that right now we're talking that I might walk out

00:45:31.840 | on the street right now that some unknown person

00:45:33.720 | who I don't know kind of comes up to me and says,

00:45:35.880 | "I hear you're going to India."

00:45:37.520 | I mean, that's not even Facebook.

00:45:38.800 | That's just a, I want transparency in human society.

00:45:42.600 | I want to have, if you know something about me,

00:45:44.580 | there's actually some reason you know something about me.

00:45:46.980 | That's something that if I look at it later

00:45:48.720 | and audit it kind of, I approve.

00:45:51.460 | You know something about me 'cause you care in some way.

00:45:54.480 | There's a caring relationship even,

00:45:56.080 | or an economic one or something.

00:45:58.000 | Not just that you're someone who could exploit it

00:46:00.040 | in ways I don't know about or care about

00:46:02.120 | or I'm troubled by or whatever.

00:46:05.160 | And we're in a world right now

00:46:06.200 | where that happens way too much.

00:46:08.640 | And that Facebook knows things about a lot of people

00:46:11.560 | and could exploit it and does exploit it at times.

00:46:14.760 | I think most people do find that creepy.

00:46:16.800 | It's not for them.

00:46:17.640 | It's not that Facebook does not do it

00:46:20.040 | 'cause they care about them, right, in any real sense.

00:46:23.320 | And they shouldn't.

00:46:24.160 | They should not be a big brother caring about us.

00:46:26.680 | That is not the role of a company like that.

00:46:28.760 | - Why not?

00:46:29.800 | Not the big brother part, but the caring, the trusting.

00:46:32.200 | I mean, don't those companies,

00:46:34.500 | just to linger on it because a lot of companies

00:46:36.600 | have a lot of information about us.

00:46:38.360 | I would argue that there's companies like Microsoft

00:46:41.120 | that has more information about us than Facebook does.

00:46:44.320 | And yet we trust Microsoft more.

00:46:45.920 | - Well, Microsoft is pivoting.

00:46:47.400 | Microsoft, you know, under Satya Nadella has decided

00:46:49.920 | this is really important.

00:46:51.120 | We don't wanna do creepy things.

00:46:53.080 | Really want people to trust us

00:46:54.400 | to actually only use information in ways

00:46:56.200 | that they really would approve of, that we don't decide.

00:46:59.200 | Right?

00:47:00.120 | And I'm just kind of adding that the health of a market

00:47:04.660 | is that when I connect to someone who, producer, consumer,

00:47:07.620 | it's not just a random producer or consumer.

00:47:09.060 | It's people who see each other.

00:47:10.540 | They don't like each other,

00:47:11.460 | but they sense that if they transact,

00:47:13.620 | some happiness will go up on both sides.

00:47:15.900 | If a company helps me to do that

00:47:18.060 | in moments that I choose, of my choosing, then fine.

00:47:22.700 | So, and also think about the difference between,

00:47:25.180 | you know, browsing versus buying, right?

00:47:28.460 | There are moments in my life

00:47:29.420 | I just wanna buy, you know, a gadget or something.

00:47:31.760 | I need something for that moment.

00:47:33.080 | I need some ammonia for my house or something,

00:47:34.920 | 'cause I got a problem, a spill.

00:47:36.520 | I wanna just go in.

00:47:38.280 | I don't wanna be advertised at that moment.

00:47:40.080 | I don't wanna be led down various, you know,

00:47:42.320 | that's annoying.

00:47:43.160 | I want to just go and have it be extremely easy

00:47:45.280 | to do what I want.

00:47:46.320 | Other moments I might say, no, it's like,

00:47:50.800 | today I'm going to the shopping mall.

00:47:52.440 | I wanna walk around and see things

00:47:53.880 | and see people and be exposed to stuff.

00:47:55.480 | So I want control over that though.

00:47:56.800 | I don't want the company's algorithms to decide for me.

00:48:00.000 | Right, and I think that's the thing.

00:48:01.200 | There's a total loss of control

00:48:02.460 | if Facebook thinks they should take the control from us

00:48:04.880 | of deciding when we want to have

00:48:06.040 | certain kinds of information, when we don't,

00:48:07.520 | what information that is,

00:48:09.060 | how much it relates to what they know about us

00:48:10.960 | that we didn't really want them to know about us.

00:48:13.200 | They're not, I don't want them to be helping me in that way.

00:48:15.880 | I don't want them to be helping them

00:48:17.040 | by they decide, they have control over

00:48:19.640 | what I want and when.

00:48:22.200 | - I totally agree.

00:48:23.040 | So Facebook, by the way, I have this optimistic thing

00:48:26.560 | where I think Facebook has the kind of personal information

00:48:29.520 | about us that could create a beautiful thing.

00:48:32.400 | So I'm really optimistic of what Facebook could do.

00:48:36.600 | It's not what it's doing, but what it could do.

00:48:39.240 | - I don't see that.

00:48:40.080 | I think that optimism is misplaced

00:48:42.040 | because you have to have a business model

00:48:43.720 | behind these things.

00:48:44.560 | - Yes, no, you have to.

00:48:45.380 | - Create a beautiful thing is really, let's be clear.

00:48:48.340 | It's about something that people would value.

00:48:51.000 | And I don't think they have that business model.

00:48:53.720 | And I don't think they will suddenly discover it

00:48:55.680 | by what, you know, a long hot shower.

00:48:58.920 | - I disagree.

00:48:59.760 | I disagree in terms of you can discover

00:49:02.600 | a lot of amazing things in a shower.

00:49:04.680 | So.

00:49:05.520 | - I didn't say that.

00:49:06.360 | I said, they won't come.

00:49:07.200 | - They won't.

00:49:08.020 | - They won't do it.

00:49:08.860 | In the shower.

00:49:09.680 | I think a lot of other people will discover it.

00:49:11.300 | I think that this, so I should also, full disclosure,

00:49:14.400 | there's a company called United Masters,

00:49:15.720 | which I'm on their board,

00:49:16.600 | and they've created this music market.

00:49:18.240 | They have 100,000 artists now signed on.

00:49:20.480 | And they've done things like gone to the NBA

00:49:22.400 | and the NBA, the music you find behind NBA clips right now

00:49:25.520 | is their music, right?

00:49:26.800 | That's a company that had the right business model

00:49:29.240 | in mind from the get go, right?

00:49:31.360 | Executed on that.

00:49:32.620 | And from day one, there was value brought to,

00:49:35.320 | so here you have a kid who made some songs

00:49:37.280 | who suddenly their songs are on the NBA website, right?

00:49:40.720 | That's real economic value to people.

00:49:43.120 | And so, you know.

00:49:45.400 | - So you and I differ on the optimism of being able to

00:49:49.200 | sort of change the direction of the Titanic, right?

00:49:54.200 | So I.

00:49:55.180 | - Yeah, I'm older than you,

00:49:56.920 | so I've seen some of the Titanic's crash.

00:49:58.960 | - Got it.

00:50:01.080 | But, and just to elaborate,

00:50:03.240 | 'cause I totally agree with you,

00:50:04.520 | and I just want to know how difficult

00:50:06.040 | you think this problem is of,

00:50:07.560 | so for example, I want to read some news.

00:50:11.440 | And I would, there's a lot of times in the day

00:50:14.320 | where something makes me either smile or think in a way

00:50:17.640 | where I like consciously think this really gave me value.

00:50:21.040 | Like I sometimes listen to the Daily Podcast

00:50:24.440 | and the New York Times,

00:50:25.720 | way better than the New York Times themselves, by the way,

00:50:28.240 | for people listening.

00:50:29.140 | That's like real journalism is happening for some reason

00:50:31.660 | in the podcast space, it doesn't make sense to me.

00:50:33.640 | But often I listen to it 20 minutes,

00:50:36.400 | and I would be willing to pay for that,

00:50:38.720 | like $5, $10 for that experience.

00:50:41.680 | And how difficult, that's kind of what you're getting at,

00:50:45.920 | is that little transaction.

00:50:47.800 | How difficult is it to create a frictionless system

00:50:50.200 | like Uber has, for example, for other things?

00:50:53.360 | What's your intuition there?

00:50:55.240 | - So first of all, I pay little bits of money to,

00:50:57.560 | you know, there's something called Quartz

00:50:59.240 | that does financial things.

00:51:00.360 | I like Medium as a site, I don't pay there, but I would.

00:51:04.400 | - You had a great post on Medium,

00:51:06.440 | I would have loved to pay you a dollar and not others.

00:51:10.360 | - I wouldn't have wanted it per se,

00:51:11.720 | because there should be also sites

00:51:14.680 | where that's not actually the goal.

00:51:16.120 | The goal is to actually have a broadcast channel

00:51:18.520 | that I monetize in some other way if I chose to.

00:51:20.720 | I mean, I could now, people know about it,

00:51:23.080 | I could, I'm not doing it, but that's fine with me.

00:51:26.160 | Also, the musicians who are making all this music,

00:51:28.360 | I don't think the right model is that you pay

00:51:30.120 | a little subscription fee to them, all right?

00:51:32.360 | Because people can copy the bits too easily,

00:51:34.600 | and it's just not that, that's not where the value is.

00:51:36.120 | The value is that a connection was made

00:51:37.840 | between real human beings, then you can follow up on that,

00:51:41.000 | all right, and create yet more value.

00:51:42.840 | So no, I think--

00:51:44.080 | - There's a lot of open questions here.

00:51:46.520 | - A lot of open questions, but also, yeah,

00:51:47.960 | I do want good recommendation systems

00:51:49.600 | that recommend cool stuff to me, but it's pretty hard,

00:51:52.040 | right, I don't like them to recommend stuff

00:51:54.160 | just based on my browsing history.

00:51:55.840 | I don't like them to base it on stuff

00:51:57.120 | they know about me, quote, unquote.

00:51:58.600 | What's unknown about me is the most interesting.

00:52:00.760 | - So this is the really interesting question.

00:52:03.520 | We may disagree, maybe not.

00:52:05.880 | I think that I love recommender systems,

00:52:09.120 | and I wanna give them everything about me

00:52:12.000 | in a way that I trust.

00:52:13.480 | - Yeah, but you don't, because, so for example,

00:52:15.640 | this morning, I clicked on, I was pretty sleepy

00:52:18.360 | this morning, I clicked on a story

00:52:21.920 | about the Queen of England, right?

00:52:24.400 | I do not give a damn about the Queen of England.

00:52:26.480 | I really do not, but it was clickbait.

00:52:28.480 | It kinda looked funny, and I had to say,

00:52:30.320 | "What the heck are they talking about there?"

00:52:31.560 | I don't wanna have my life heading that direction.

00:52:33.960 | Now that's in my browsing history.

00:52:36.080 | The system, and any reasonable system,

00:52:38.400 | will think that I care about the Queen of England.

00:52:39.240 | - That's browsing history.

00:52:40.560 | - Right, but you're saying all the trace,

00:52:42.840 | all the digital exhaust, or whatever,

00:52:44.320 | that's been kind of the model, is if you collect

00:52:45.920 | all this stuff, you're gonna figure all of us out.

00:52:48.880 | Well, if you're trying to figure out one person,

00:52:50.420 | like Trump or something, maybe you could figure him out,

00:52:52.600 | but if you're trying to figure out 500 million people,

00:52:56.200 | no way, no way.

00:52:57.920 | - Do you think so?

00:52:58.760 | - No, I think so.

00:52:59.600 | I think we are, humans are just amazingly rich

00:53:01.800 | and complicated.

00:53:02.640 | Every one of us has our little quirks.

00:53:03.880 | Every one of us has our little things that could intrigue us

00:53:05.880 | that we don't even know will intrigue us,

00:53:07.840 | and there's no sign of it in our past,

00:53:09.880 | but by God, there it comes, and you fall in love with it,

00:53:12.860 | and I don't want a company trying to figure that out

00:53:14.480 | for me and anticipate that.

00:53:15.760 | - Okay, well let me--

00:53:16.600 | - I want them to provide a forum, a market,

00:53:18.480 | a place that I kind of go, and by hook or by crook,

00:53:22.080 | this happens.

00:53:22.920 | I'm walking down the street, and I hear some Chilean music

00:53:26.040 | being played, and I never knew I liked Chilean music,

00:53:27.640 | but wow, so there is that side, and I want them

00:53:30.360 | to provide a limited, but interesting place to go, right?

00:53:34.760 | And so don't try to use your AI to kind of figure me out

00:53:38.920 | and then put me in a world where you figured me out.

00:53:42.280 | No, create spaces for human beings

00:53:45.040 | where our creativity and our style will be enriched

00:53:48.440 | and come forward, and it'll be a lot of more transparency.

00:53:50.880 | I won't have people randomly, anonymously

00:53:53.360 | putting comments up, and I'll special,

00:53:55.240 | based on stuff they know about me, facts that you know.

00:53:58.320 | We are so broken right now, especially if you're a celebrity,

00:54:01.800 | but it's about anybody that, anonymous people

00:54:04.920 | are hurting lots and lots of people right now,

00:54:06.560 | and that's part of this thing that Silicon Valley

00:54:08.560 | is thinking that just collect all this information

00:54:11.000 | and use it in a great way.

00:54:12.400 | So no, I'm not a pessimist, I'm very much an optimist

00:54:15.520 | by nature, but I think that's just been the wrong path

00:54:17.680 | for the whole technology to take.

00:54:19.840 | Be more limited, create, let humans rise up.

00:54:23.480 | Don't try to replace them, that's the AI mantra.

00:54:26.600 | Don't try to anticipate them, don't try to predict them,

00:54:30.320 | 'cause you're not gonna be able to do those things,

00:54:33.280 | you're gonna make things worse.

00:54:34.600 | - Okay, so right now, just give this a chance,

00:54:38.520 | right now the recommender systems are the creepy people

00:54:42.040 | in the shadow watching your every move.

00:54:45.360 | So they're looking at traces of you,

00:54:47.680 | they're not directly interacting with you.

00:54:50.240 | Sort of your close friends and family,

00:54:52.920 | the way they know you is by having conversation,

00:54:55.000 | by actually having interactions back and forth.

00:54:57.320 | Do you think there's a place for recommender systems,

00:55:00.680 | sort of to step, 'cause you just emphasized the value

00:55:03.200 | of human to human connection, but just give it a chance,

00:55:05.800 | AI human connection, is there a role for an AI system

00:55:09.360 | to have conversations with you, in terms of,

00:55:12.880 | to try to figure out what kind of music you like,

00:55:14.720 | not by just watching what you listen to,

00:55:16.720 | but actually having a conversation,

00:55:18.200 | natural language or otherwise.

00:55:19.600 | - Yeah, so I'm not against it,

00:55:21.840 | I just wanted to push back against,

00:55:23.040 | maybe you're saying you have autism for Facebook,

00:55:25.120 | so there I think it's misplaced, but I think that--

00:55:28.720 | - I'm the one guy defending Facebook.

00:55:30.280 | - Yeah, no, so good for you, go for it.

00:55:34.000 | - That's a hard spot to be.

00:55:34.840 | - Yeah, no, good, human interaction on our daily,

00:55:37.440 | the context around me in my own home is something

00:55:39.660 | that I don't want some big company to know about at all,

00:55:41.320 | but I would be more than happy

00:55:42.360 | to have technology help me with it.

00:55:44.080 | - Which kind of technology?

00:55:45.240 | - Well, you know, just--

00:55:46.320 | - Alexa, Amazon.

00:55:47.640 | - Well, Alexa's done right,

00:55:49.240 | I think Alexa's a research platform right now,

00:55:50.800 | more than anything else, but Alexa done right,

00:55:53.640 | could do things like, I leave the water running in my garden

00:55:56.600 | and I say, "Hey, Alexa, the water's running in my garden,"

00:55:59.120 | and even have Alexa figure out that that means

00:56:00.640 | when my wife comes home that she should be told about that.

00:56:03.520 | That's a little bit of a reasoning, I would call that AI,

00:56:05.920 | and by any kind of stretch, it's a little bit of reasoning,

00:56:08.280 | and it actually kind of would make my life

00:56:09.820 | a little easier and better,

00:56:10.820 | and I wouldn't call this a wow moment,

00:56:13.400 | but I kind of think that overall rises human happiness up

00:56:16.440 | to have that kind of thing.

00:56:18.280 | - But not when you're lonely, Alexa knowing loneliness--

00:56:21.460 | - No, no, I don't want Alexa to feel intrusive,

00:56:24.840 | and I don't want just the designer of the system

00:56:27.420 | to kind of work all this out,

00:56:28.480 | I really wanna have a lot of control,

00:56:30.080 | and I want transparency and control,

00:56:32.340 | and if a company can stand up and give me that

00:56:34.480 | in the context of new technology,

00:56:36.520 | I think they're gonna, first of all,

00:56:37.360 | be way more successful than our current generation,

00:56:39.760 | and like I said, I was mentioning Microsoft earlier,

00:56:41.640 | I really think they're pivoting

00:56:43.300 | to kind of be the trusted old uncle,

00:56:45.060 | but I think that they get that this is the way to go,

00:56:47.680 | that if you let people find technology

00:56:49.840 | empowers them to have more control

00:56:51.320 | and have control not just over privacy,

00:56:53.840 | but over this rich set of interactions,

00:56:56.440 | that people are gonna like that a lot more,

00:56:58.120 | and that's the right business model going forward.

00:57:00.360 | - What does control over privacy look like?

00:57:02.240 | Do you think you should be able to just view

00:57:03.600 | all the data that--

00:57:04.960 | - No, it's much more than that.

00:57:05.960 | I mean, first of all, it should be an individual decision.

00:57:07.760 | Some people don't want privacy,

00:57:09.200 | they want their whole life out there,

00:57:10.600 | other people's want it.

00:57:11.760 | Privacy is not a zero one, it's not a legal thing,

00:57:16.880 | it's not just about which data is available, which is not.

00:57:20.320 | I like to recall to people that a couple hundred years ago,

00:57:24.160 | everyone, there was not really big cities,

00:57:25.960 | everyone lived in on the countryside and villages,

00:57:28.720 | and in villages, everybody knew everything about you,

00:57:30.840 | very, you didn't have any privacy, is that bad?

00:57:33.400 | Are we better off now?

00:57:34.880 | Well, arguably no, because what did you get for that loss

00:57:37.880 | of at least certain kinds of privacy?

00:57:40.520 | Well, people helped each other,

00:57:43.160 | because they know everything about you,

00:57:44.080 | they know something bad's happening,

00:57:45.400 | they will help you with that, right?

00:57:46.640 | And now you live in a big city,

00:57:47.680 | no one knows about you, you get no help.

00:57:49.640 | So it kind of depends, the answer.

00:57:52.720 | I want certain people who I trust,

00:57:55.360 | and there should be relationships,

00:57:56.360 | I should kind of manage all those,

00:57:57.680 | but who knows what about me,

00:57:59.040 | I should have some agency there.

00:58:01.200 | I shouldn't just be adrift in a sea of technology

00:58:03.720 | where I have no agency.

00:58:04.720 | I don't wanna go reading things and checking boxes.

00:58:08.480 | So I don't know how to do that,

00:58:09.960 | and I'm not a privacy researcher per se,

00:58:11.880 | I recognize the vast complexity of this,

00:58:14.320 | it's not just technology,

00:58:15.280 | it's not just legal scholars meeting technologists,

00:58:18.680 | there's gotta be kind of a whole layers around it.

00:58:20.840 | And so when I alluded to this emerging engineering field,

00:58:24.120 | this is a big part of it.

00:58:25.360 | When electrical engineering came,

00:58:28.400 | I'm not went around at the time,

00:58:29.640 | but you just didn't plug electricity into walls

00:58:33.280 | and all kind of worked,

00:58:34.120 | you don't have to have like underwriters laboratory

00:58:36.160 | that reassured you that that plugs

00:58:37.600 | not gonna burn up your house,

00:58:39.560 | and that that machine will do this and that and everything,

00:58:41.680 | there'll be whole people who can install things,

00:58:44.480 | there'll be people who can watch the installers,

00:58:46.280 | there'll be a whole layers,

00:58:47.800 | an onion of these kinds of things.

00:58:49.760 | And for things as deeply interesting as privacy,

00:58:52.960 | which is as least as interesting as electricity,

00:58:55.840 | that's gonna take decades to kind of work out,

00:58:57.520 | but it's gonna require a lot of new structures

00:58:59.360 | that we don't have right now,

00:59:00.240 | so it's getting hard to talk about it.

00:59:02.160 | - And you're saying there's a lot of money to be made

00:59:03.960 | if you get it right, so--

00:59:05.000 | - Absolutely. - It's something

00:59:05.840 | you should look at. - A lot of money to be made

00:59:07.000 | and all these things that provide human services

00:59:08.800 | and people recognize them as useful parts of their lives.

00:59:12.280 | So yeah, the dialect sometimes goes

00:59:15.760 | from the exuberant technologists

00:59:18.480 | to the no technology is good kind of,

00:59:20.680 | and that's in our public discourse,

00:59:23.040 | and newsrooms you see too much of this kind of thing.

00:59:25.680 | And the sober discussions in the middle,

00:59:27.680 | which are the challenging ones to have

00:59:29.080 | are where we need to be having our conversations.

00:59:31.280 | And actually there's not many forum for those.

00:59:35.080 | That's kind of what I would look for.

00:59:39.000 | Maybe I could go and I could read a comment section

00:59:41.120 | of something and it would actually be

00:59:42.560 | this kind of dialogue going back and forth.

00:59:44.480 | You don't see much of this, right?

00:59:45.680 | - Which is why actually there's a resurgence

00:59:47.640 | of podcasts out of all,

00:59:49.120 | because people are really hungry for conversation.

00:59:51.760 | But technology is not helping much,

00:59:55.680 | so comment sections of anything, including YouTube,

00:59:58.680 | - Yeah.

00:59:59.520 | - Is not hurting. - Or hurting.

01:00:00.920 | - And not helping. - Or hurting, yeah.

01:00:03.280 | - And you think technically speaking,

01:00:05.920 | it's possible to help?

01:00:07.840 | - I don't know the answers,

01:00:08.920 | but it's a less anonymity, a little more locality,

01:00:13.080 | worlds that you kind of enter in

01:00:15.520 | and you trust the people there in those worlds

01:00:17.400 | so that when you start having a discussion,

01:00:19.280 | not only is that people not gonna hurt you,

01:00:20.920 | but it's not gonna be a total waste of your time,

01:00:22.880 | 'cause there's a lot of wasting of time.

01:00:24.800 | A lot of us, I pulled out of Facebook early on

01:00:26.760 | 'cause it was clearly gonna waste a lot of my time,

01:00:28.720 | even though there was some value.

01:00:31.000 | And so yeah, worlds that are somehow you enter in

01:00:33.840 | and you know what you're getting

01:00:34.720 | and it kind of appeals to you,

01:00:36.720 | new things might happen,

01:00:37.680 | but you kind of have some trust in that world.

01:00:40.680 | - And there's some deep, interesting, complex,

01:00:43.600 | psychological aspects around anonymity,

01:00:46.280 | how that changes human behavior.

01:00:47.880 | - Indeed. - That's quite dark.

01:00:49.760 | - Quite dark, yeah.

01:00:50.640 | I think a lot of us are,

01:00:52.280 | especially those of us who really love

01:00:54.240 | the advent of technology.

01:00:55.440 | I loved social networks when they came out.

01:00:56.760 | I was just, I didn't see any negatives there at all.

01:00:59.400 | But then I started seeing comment sections,

01:01:01.720 | I think it was maybe CNN or something,

01:01:04.640 | and I started to go, wow, this darkness,

01:01:06.840 | I just did not know about,

01:01:08.760 | and our technology is now amplifying it.

01:01:11.840 | - So sorry for the big philosophical question,

01:01:13.760 | but on that topic, do you think human beings,

01:01:15.880 | 'cause you've also, out of all things,

01:01:17.320 | had a foot in psychology too,

01:01:19.140 | do you think human beings are fundamentally good?

01:01:23.680 | Like all of us have good intent that could be mind,

01:01:28.200 | or is it, depending on context and environment,

01:01:33.000 | everybody could be evil?

01:01:34.840 | - So my answer is fundamentally good,

01:01:37.560 | but fundamentally limited.

01:01:39.040 | All of us have very, you know, blinkers on.

01:01:41.240 | We don't see the other person's pain that easily.

01:01:43.840 | We don't see the other person's point of view that easily.

01:01:46.600 | We're very much in our own head, in our own world.

01:01:49.680 | And on my good days, I think that technology

01:01:51.960 | could open us up to more perspectives,

01:01:54.000 | and more less blinkered, and more understanding.

01:01:56.540 | You know, a lot of wars in human history

01:01:58.280 | happened because of just ignorance.

01:01:59.840 | They didn't, they thought the other person was doing this,

01:02:01.960 | well, the other person wasn't doing this,

01:02:03.080 | and we have huge amounts of that.

01:02:05.200 | But in my lifetime, I've not seen technology

01:02:07.220 | really help in that way yet.

01:02:09.080 | And I do believe in that, but you know,

01:02:12.240 | no, I think fundamentally humans are good.

01:02:14.200 | People suffer, people have grievances,

01:02:16.320 | people have grudges, and those things

01:02:17.520 | cause them to do things they probably wouldn't want.

01:02:19.880 | They regret it often.

01:02:21.360 | So no, I think it's, you know,

01:02:25.760 | part of the progress of technology is to indeed allow it

01:02:28.520 | to be a little easier to be the real good person

01:02:30.200 | you actually are.

01:02:31.760 | - Well, but do you think individual human life,

01:02:35.740 | or society, could be modeled as an optimization problem?

01:02:40.140 | - Not the way I think, typically.

01:02:43.460 | I mean, that's, you're talking about

01:02:44.300 | one of the most complex phenomena in the whole,

01:02:46.320 | you know, in all of the universe.

01:02:47.520 | - Which the individual human life, or society,

01:02:49.720 | as a whole. - Both, both.

01:02:51.160 | I mean, individual human life is amazingly complex.

01:02:54.280 | And so, you know, optimization is kind of

01:02:57.240 | just one branch of mathematics

01:02:58.520 | that talks about certain kind of things.

01:02:59.880 | And it just feels way too limited

01:03:02.120 | for the complexity of such things.

01:03:04.280 | - What properties of optimization problems,

01:03:06.680 | do you think most interesting problems

01:03:09.960 | that could be solved through optimization,

01:03:12.200 | what kind of properties does that surface have?

01:03:14.320 | Non-convexity, convexity, linearity,

01:03:17.520 | all those kinds of things, saddle points.

01:03:19.960 | - Well, so optimization's just one piece of mathematics.

01:03:22.080 | You know, there's like, just even in our era,

01:03:24.600 | we're aware that, say, sampling is coming up,

01:03:28.040 | examples of something, coming up with a distribution.

01:03:30.720 | - What's optimization, what's sampling?

01:03:32.720 | - Well, you can, if you're a kind of,

01:03:34.840 | a certain kind of mathematician,

01:03:35.680 | you can try to blend them and make them

01:03:37.320 | seem to be sort of the same thing.

01:03:38.560 | But optimization is, roughly speaking,

01:03:40.180 | trying to find a point that, a single point,

01:03:43.920 | that is the optimum of a criterion function of some kind.

01:03:47.340 | And sampling is trying to, from that same surface,

01:03:51.640 | treat that as a distribution or a density

01:03:54.080 | and find points that have high density.

01:03:56.760 | So I want the entire distribution in a sampling paradigm

01:04:01.360 | and I want the single point that's the best point

01:04:04.360 | in the optimization paradigm.

01:04:07.520 | Now, if you were optimizing

01:04:08.840 | in the space of probability measures,

01:04:11.100 | the output of that could be a whole probability distribution.

01:04:13.280 | So you can start to make these things the same.

01:04:15.480 | But in mathematics, if you go too high up

01:04:17.160 | that kind of abstraction, you start to lose

01:04:19.520 | the ability to do the interesting theorems.

01:04:22.800 | So you kind of don't try to,

01:04:23.760 | you don't try to overly, over-abstract.

01:04:25.920 | - So, as a small tangent, what kind of world view

01:04:30.120 | do you find more appealing?

01:04:31.360 | One that is deterministic or stochastic?

01:04:35.320 | - Well, that's easy.

01:04:37.160 | I mean, I'm a statistician.

01:04:38.400 | The world is highly stochastic.

01:04:40.400 | I don't know what's gonna happen

01:04:41.240 | in the next five minutes, right?

01:04:42.480 | 'Cause what you're gonna ask, what we're gonna do,

01:04:43.920 | what I'll say. - Due to the uncertainty.

01:04:45.240 | Due to the-- - Massive uncertainty.

01:04:47.280 | You know, massive uncertainty.

01:04:48.760 | And so the best I can do is have kind of rough sense

01:04:51.400 | or probability distribution on things

01:04:53.160 | and somehow use that in my reasoning about what to do now.

01:04:56.560 | - So how does the distributed at scale,

01:05:02.620 | when you have multi-agent systems,

01:05:04.780 | look like, so optimization can optimize sort of,

01:05:10.440 | it makes a lot more sense.

01:05:12.440 | Sort of, at least from my, from a robotics perspective,

01:05:15.160 | for a single robot, for a single agent,

01:05:17.080 | trying to optimize some objective function.

01:05:19.340 | When you start to enter the real world,

01:05:22.640 | this game-theoretic concept starts popping up.

01:05:25.240 | That, how do you see optimization in this?

01:05:30.520 | 'Cause you've talked about markets and the scale.

01:05:32.360 | What does that look like?

01:05:33.840 | Do you see it as optimization?

01:05:34.960 | Do you see it as sampling?

01:05:36.080 | Do you see, like how should you--

01:05:37.880 | - Yeah, these all blend together.

01:05:39.320 | And a system designer thinking about

01:05:41.200 | how to build an incentivized system

01:05:43.520 | will have a blend of all these things.

01:05:44.800 | So, a particle in a potential well

01:05:47.700 | is optimizing a functional called a Lagrangian.

01:05:50.560 | The particle doesn't know that.

01:05:51.840 | There's no algorithm running that does that.

01:05:54.600 | It just happens.

01:05:55.640 | So it's a description mathematically of something

01:05:57.440 | that helps us understand as analysts what's happening.

01:06:00.760 | And so the same thing will happen when we talk about

01:06:02.760 | mixtures of humans and computers

01:06:04.120 | and markets and so on and so forth.

01:06:05.880 | There'll be certain principles

01:06:06.880 | that allow us to understand what's happening

01:06:08.160 | and whether or not the actual algorithms

01:06:09.620 | are being used by any sense is not clear.

01:06:12.860 | Now, at some point I may have set up a multi-agent

01:06:16.760 | or market kind of system.

01:06:18.960 | And I'm now thinking about an individual agent

01:06:21.000 | in that system.

01:06:22.320 | And they're asked to do some task

01:06:23.840 | and they're incentivized in some way.

01:06:24.800 | They get certain signals and they have some utility.

01:06:27.800 | Maybe what they will do at that point

01:06:29.400 | is they just won't know the answer.

01:06:30.800 | They may have to optimize to find an answer.

01:06:33.240 | So an optus could be embedded inside of an overall market.

01:06:37.080 | And game theory is very, very broad.

01:06:39.760 | It is often studied very narrowly

01:06:41.860 | for certain kinds of problems.

01:06:43.940 | But it's roughly speaking,

01:06:45.100 | there's just the, I don't know what you're gonna do.

01:06:47.780 | So I kind of anticipate that a little bit

01:06:49.620 | and you anticipate what I'm anticipating.

01:06:51.460 | And we kind of go back and forth in our own minds.

01:06:53.260 | We run kind of thought experiments.

01:06:55.260 | - You've talked about this interesting point

01:06:56.900 | in terms of game theory.

01:06:58.700 | You know, most optimization problems

01:07:01.220 | really hate saddle points.

01:07:02.700 | Maybe you can describe what saddle points are.

01:07:04.660 | But I've heard you kind of mentioned

01:07:07.080 | that there's a branch of optimization

01:07:09.600 | that you could try to explicitly look for saddle points

01:07:13.840 | as a good thing.

01:07:14.920 | - Oh, not optimization.

01:07:15.760 | That's just game theory.

01:07:16.760 | So there's all kinds of different equilibrium game theory.

01:07:20.320 | And some of them are highly explanatory behavior.

01:07:22.960 | They're not attempting to be algorithmic.

01:07:24.680 | They're just trying to say,

01:07:26.080 | if you happen to be at this equilibrium,

01:07:28.480 | you would see certain kind of behavior.

01:07:29.720 | And we see that in real life.

01:07:30.840 | That's what an economist wants to do,

01:07:32.400 | especially a behavioral economist.

01:07:34.940 | In continuous differential game theory,

01:07:39.940 | you're in continuous spaces.

01:07:42.380 | Some of the simplest equilibria are saddle points.

01:07:44.340 | A Nash equilibrium is a saddle point.

01:07:46.300 | It's a special kind of saddle point.

01:07:48.340 | So classically in game theory,

01:07:50.540 | you are trying to find Nash equilibrium.

01:07:52.540 | And in algorithmic game theory,

01:07:53.980 | you're trying to find algorithms that would find them.

01:07:56.100 | And so you're trying to find saddle points.

01:07:57.740 | I mean, so that's literally what you're trying to do.

01:08:00.580 | But, you know, any economist knows

01:08:01.800 | that Nash equilibria have their limitations.

01:08:04.100 | They are definitely not that explanatory

01:08:06.460 | in many situations.

01:08:08.100 | They're not what you really want.

01:08:10.220 | There's other kind of equilibria.

01:08:11.980 | And there's names associated with these

01:08:13.680 | 'cause they came from history

01:08:14.900 | with certain people working on them,

01:08:15.940 | but there'll be new ones emerging.

01:08:17.940 | So, you know, one example is a Stackelberg equilibrium.

01:08:21.060 | So, you know, Nash, you and I are both playing this game

01:08:24.060 | against each other or for each other,

01:08:25.760 | maybe it's cooperative.

01:08:27.120 | And we're both gonna think it through,

01:08:28.540 | and then we're gonna decide

01:08:29.380 | and we're gonna do our thing simultaneously.

01:08:32.460 | You know, and a Stackelberg,

01:08:33.500 | no, I'm gonna be the first mover.

01:08:34.640 | I'm gonna make a move.

01:08:35.780 | You're gonna look at my move,

01:08:36.920 | and then you're gonna make yours.

01:08:38.300 | Now, since I know you're gonna look at my move,

01:08:40.640 | I anticipate what you're gonna do.

01:08:42.100 | And so I don't do something stupid.

01:08:43.660 | But then I know that you are also anticipating me.

01:08:46.860 | So we're kind of going back and forth in line.

01:08:48.440 | But there is then a first mover thing.

01:08:51.440 | And so those are different equilibria, all right?

01:08:54.580 | And so just mathematically, yeah,

01:08:57.140 | these things have certain topologies

01:08:58.620 | and certain shapes that are like salivating,

01:09:00.140 | algorithmically or dynamically,

01:09:01.540 | how do you move towards them?

01:09:02.780 | How do you move away from things?

01:09:04.460 | You know, so some of these questions have answers,

01:09:07.540 | they've been studied, others do not.

01:09:09.380 | And especially if it becomes stochastic,

01:09:11.780 | especially if there's large numbers

01:09:13.220 | of decentralized things,

01:09:14.420 | there's just, you know, young people getting in this field

01:09:16.780 | who kind of think it's all done

01:09:17.780 | because we have, you know, TensorFlow.

01:09:19.700 | Well, no, these are all open problems,

01:09:21.920 | and they're really important and interesting.

01:09:23.540 | And it's about strategic settings.

01:09:25.060 | How do I collect data?

01:09:26.380 | Suppose I don't know what you're gonna do

01:09:27.980 | 'cause I don't know you very well, right?

01:09:29.860 | Well, I gotta collect data about you.

01:09:31.060 | So maybe I wanna push you in a part of the space

01:09:33.200 | where I don't know much about you so I can get data.

01:09:35.580 | And then later I'll realize that you'll never go there

01:09:38.540 | 'cause of the way the game is set up.

01:09:39.740 | But you know, that's part of the overall,

01:09:41.580 | you know, data analysis context.

01:09:43.780 | - Even the game of poker is a fascinating space.

01:09:46.340 | Whenever there's any uncertainty,

01:09:47.540 | a lack of information, it's a super exciting space.

01:09:50.820 | - Yeah.

01:09:52.300 | - Just lingering on optimization for a second.

01:09:55.340 | So if we look at deep learning,

01:09:56.860 | it's essentially minimization

01:09:58.500 | of a complicated loss function.

01:10:01.460 | So is there something insightful or hopeful

01:10:04.100 | that you see in the kinds of function surface

01:10:07.500 | that loss functions, that deep learning

01:10:09.580 | in the real world is trying to optimize over?

01:10:13.780 | Is there something interesting?

01:10:15.380 | Is it just the usual kind of problems of optimization?

01:10:18.860 | - I think from an optimization point of view,

01:10:21.940 | that surface virtual, it's pretty smooth.

01:10:25.380 | And secondly, if it's over parameterized,

01:10:28.060 | there's kind of lots of paths down to reasonable optima.

01:10:31.340 | And so kind of the getting downhill to an optima

01:10:34.500 | is viewed as not as hard as you might have expected

01:10:37.180 | in high dimensions.

01:10:38.380 | The fact that some optima tend to be really good ones

01:10:42.300 | and others not so good, and you tend to,

01:10:44.380 | sometimes you find the good ones

01:10:45.660 | is sort of still needs explanation.

01:10:48.320 | - Yes, that's a total mystery.

01:10:50.380 | - But the particular surface is coming

01:10:51.940 | from the particular generation of neural nets,

01:10:53.580 | I kind of suspect those will change.

01:10:56.180 | In 10 years, it will not be exactly those surfaces,

01:10:58.420 | there'll be some others that are,

01:10:59.780 | and optimization theory will help contribute

01:11:01.380 | to why other surfaces or why other algorithms.

01:11:03.880 | Layers of arithmetic operations

01:11:07.220 | with a little bit of non-linearity,

01:11:08.820 | that's not, that didn't come from neuroscience per se.

01:11:10.980 | I mean, maybe in the minds of some of the people

01:11:12.260 | working on it, they were thinking,

01:11:13.580 | you know, about brains, but they were arithmetic circuits

01:11:16.700 | in all kinds of fields, you know,

01:11:18.260 | computer science control theory and so on.

01:11:20.340 | And that layers of these could transform things

01:11:22.620 | in certain ways, and that if it's smooth,

01:11:24.580 | maybe you could find parameter values,

01:11:27.660 | you know, is a sort of big discovery

01:11:32.060 | that it's working, it's able to work at this scale.

01:11:34.860 | But I don't think that we're stuck with that,

01:11:38.600 | and we're certainly not stuck with that

01:11:40.060 | 'cause we're understanding the brain.

01:11:42.020 | - So in terms of, on the algorithm side,

01:11:44.020 | sort of gradient descent, do you think we're stuck

01:11:46.420 | with gradient descent, is variance of it,

01:11:49.180 | what variance do you find interesting,

01:11:50.900 | or do you think there'll be something else

01:11:52.380 | invented that is able to walk all over

01:11:56.940 | these optimization spaces in more interesting ways?

01:11:59.580 | - So there's a co-design of the surface,

01:12:01.820 | and there are the architecture and the algorithm.

01:12:04.580 | So if you just ask if we stay with the kind

01:12:06.860 | of architectures that we have now,

01:12:07.940 | not just neural nets, but, you know,

01:12:10.100 | phase retrieval architectures,

01:12:11.500 | or matrix completion architectures and so on,

01:12:13.740 | you know, I think we've kind of come to a place

01:12:16.540 | where, yeah, a stochastic gradient algorithms

01:12:19.660 | are dominant, and there are versions,

01:12:23.220 | you know, that are a little better than others,

01:12:25.780 | they, you know, have more guarantees,

01:12:27.700 | they're more robust, and so on,

01:12:29.140 | and there's ongoing research to kind of figure out

01:12:31.100 | which is the best algorithm for which situation.

01:12:34.140 | But I think that that'll start to co-evolve,

01:12:35.740 | that that'll put pressure on the actual architecture,

01:12:37.780 | and so we shouldn't do it in this particular way,

01:12:39.540 | we should do it in a different way,

01:12:40.700 | 'cause this other algorithm is now available

01:12:42.100 | if you do it in a different way.

01:12:43.700 | So that I can't really anticipate,

01:12:48.660 | that co-evolution process, but, you know,

01:12:51.100 | gradients are amazing mathematical objects,

01:12:54.340 | they have a lot of people who sort of study them

01:12:58.820 | more deeply mathematically, are kind of shocked

01:13:01.500 | about what they are and what they can do.

01:13:03.620 | I mean, think about it this way,

01:13:05.780 | if, suppose that I tell you,

01:13:07.140 | if you move along the x-axis, you get, you know,

01:13:10.500 | you go uphill in some objective by, you know, three units,

01:13:13.340 | whereas if you move along the y-axis,

01:13:15.380 | you go uphill by seven units, right?

01:13:17.940 | Now I'm gonna only allow you to move a certain,

01:13:19.820 | you know, unit distance, all right?

01:13:22.260 | What are you gonna do?

01:13:23.300 | Well, the most people will say,

01:13:25.540 | I'm gonna go along the y-axis,

01:13:26.620 | I'm getting the biggest bang for my buck,

01:13:28.580 | you know, and my buck is only one unit,

01:13:30.140 | so I'm gonna put all of it in the y-axis, right?

01:13:33.420 | And why should I even take any of my strength,

01:13:37.140 | my step size, and put any of it in the x-axis,

01:13:39.420 | 'cause I'm getting less bang for my buck?

01:13:41.380 | That seems like a completely, you know, clear argument,

01:13:45.100 | and it's wrong, 'cause the gradient direction

01:13:47.460 | is not to go along the y-axis,

01:13:49.180 | it's to take a little bit of the x-axis.

01:13:51.660 | And that, to understand that, you have to know some math,

01:13:55.140 | and so even a, you know, a trivial,

01:13:58.340 | so-called operator-like gradient is not trivial,

01:14:00.300 | and so, you know, exploiting its properties

01:14:02.220 | is still very, very important.

01:14:03.900 | Now we know that just periodic descent

01:14:05.260 | has got all kinds of problems.

01:14:06.300 | It gets stuck in many ways,

01:14:07.500 | and it doesn't have, you know,

01:14:08.820 | good dimension dependence and so on.

01:14:10.820 | So my own line of work recently

01:14:13.060 | has been about what kinds of stochasticity,

01:14:15.460 | how can we get dimension dependence,

01:14:16.740 | how can we do the theory of that?

01:14:19.100 | And we've come up with pretty favorable results

01:14:20.660 | with certain kinds of stochasticity.

01:14:22.580 | We have sufficient conditions, generally.

01:14:24.740 | We know if you do this, we will give you a good guarantee.

01:14:28.700 | We don't have necessary conditions

01:14:29.980 | that it must be done a certain way in general.

01:14:32.140 | - So stochasticity, how much randomness to inject

01:14:35.020 | into the walking along the gradient?

01:14:38.220 | - And what kind of randomness?

01:14:39.860 | - Why is randomness good in this process?

01:14:42.140 | Why is stochasticity good?

01:14:44.220 | - Yeah, so I can give you simple answers,

01:14:46.700 | but in some sense, again, it's kind of amazing.

01:14:48.700 | Stochasticity just, you know,

01:14:52.660 | particular features of a surface that could have hurt you

01:14:55.420 | if you were doing one thing deterministically

01:14:58.020 | won't hurt you because, you know, by chance,

01:15:01.380 | you know, there's very little chance that you would get hurt.

01:15:03.260 | And, you know, so here stochasticity, you know,

01:15:08.260 | it just kind of saves you

01:15:10.820 | from some of the particular features of surfaces that,

01:15:14.260 | you know, in fact, if you think about, you know,

01:15:17.220 | surfaces that are discontinuous in a first derivative,

01:15:19.300 | like, you know, absolute value function,

01:15:21.940 | you will go down and hit that point

01:15:23.500 | where there's non-differentiability, right?

01:15:25.260 | And if you're running a deterministic algorithm,

01:15:27.220 | at that point, you can really do something bad, right?

01:15:30.060 | Whereas stochasticity just means it's pretty unlikely

01:15:32.140 | that's gonna happen, that you're gonna hit that point.

01:15:35.620 | So, you know, it's again, non-trivial to analyze,

01:15:37.940 | but especially in higher dimensions, also stochasticity,

01:15:41.740 | our intuition isn't very good about it,

01:15:43.260 | but it has properties that kind of are very appealing

01:15:45.540 | in high dimensions for kind of law of large number reasons.

01:15:49.140 | So it's all part of the mathematics,

01:15:51.340 | that's what's fun to work in the field

01:15:52.600 | is that you get to try to understand this mathematics.

01:15:55.300 | But long story short, you know,

01:15:58.580 | partly empirically it was discovered stochastic gradient

01:16:00.920 | is very effective in theory,

01:16:02.500 | kind of followed, I'd say, that,

01:16:05.060 | but I don't see that we're getting clearly out of that.

01:16:07.860 | - What's the most beautiful, mysterious,

01:16:11.580 | or profound idea to you in optimization?

01:16:15.580 | - I don't know the most, but let me just say that,

01:16:18.580 | you know, Nesterov's work on Nesterov acceleration to me

01:16:21.620 | is pretty surprising and pretty deep.

01:16:24.660 | - Can you elaborate?

01:16:27.260 | - Well, Nesterov acceleration is just that,

01:16:29.980 | suppose that we are gonna use gradients

01:16:32.300 | to move around into space, for the reasons I've alluded to,

01:16:34.260 | there are nice directions to move.

01:16:37.020 | And suppose that I tell you

01:16:38.580 | that you're only allowed to use gradients,

01:16:40.140 | you're not gonna be allowed to,

01:16:41.740 | you see this local person that can only sense

01:16:43.820 | kind of a change in the surface.

01:16:46.300 | But I'm gonna give you kind of a computer

01:16:48.940 | that's able to store all your previous gradients,

01:16:50.860 | and so you start to learn something about the surface.

01:16:53.860 | And I'm gonna restrict you to maybe move in the direction

01:16:57.180 | of like a linear span of all the gradients,

01:16:59.300 | so you can't kind of just move

01:17:00.260 | in some arbitrary direction, right?

01:17:02.740 | So now we have a well-defined mathematical complexity model,

01:17:05.620 | there's a certain classes of algorithms that can do that,

01:17:07.980 | and others that can't.

01:17:09.180 | And we can ask for certain kinds of surfaces,

01:17:11.340 | how fast can you get down to the optimum?

01:17:13.700 | So there's answers to these,

01:17:14.900 | so for a smooth convex function, there's an answer,

01:17:19.460 | which is one over the number of steps squared,

01:17:22.020 | is that you will be within a ball of that size

01:17:24.380 | after K steps.

01:17:27.140 | Gradient descent in particular has a slower rate,

01:17:31.420 | it's one over K, okay?

01:17:33.780 | So you could ask, is gradient descent actually,

01:17:37.380 | even though we know it's a good algorithm,

01:17:38.900 | is it the best algorithm?

01:17:40.220 | And the answer is no.

01:17:42.140 | Well, not clear yet,

01:17:43.820 | because one over K squared is a lower bound,

01:17:47.180 | that's provably the best you can do,

01:17:49.580 | gradient is one over K, but is there something better?

01:17:52.620 | And so I think it's a surprise to most,

01:17:55.420 | though Nesterov discovered a new algorithm

01:17:58.100 | that has got two pieces to it, it uses two gradients,

01:18:01.260 | and puts those together in a certain kind of obscure way,

01:18:06.260 | and the thing doesn't even move downhill all the time,

01:18:09.260 | it sometimes goes back uphill.

01:18:11.460 | And if you're a physicist, that kind of makes some sense,

01:18:13.100 | you're building up some momentum,

01:18:14.540 | and that is kind of the right intuition,

01:18:16.380 | but that intuition is not enough to understand

01:18:19.060 | kind of how to do it and why it works.

01:18:21.020 | But it does, it achieves one over K squared,

01:18:24.580 | and it has a mathematical structure,

01:18:26.940 | and it's still kind of, to this day,

01:18:28.620 | a lot of us are writing papers

01:18:29.660 | and trying to explore that and understand it.

01:18:32.460 | So there are lots of cool ideas in optimization,

01:18:35.100 | but just kind of using gradients, I think, is number one,

01:18:37.380 | that goes back 150 years, and then Nesterov, I think,

01:18:41.500 | has made a major contribution with this idea.

01:18:43.460 | - So like you said, gradients themselves

01:18:45.220 | are in some sense mysterious.

01:18:47.460 | - Yeah.

01:18:48.300 | - They're not as trivial as--

01:18:50.220 | - Not as trivial.

01:18:51.060 | - Mathematically speaking.

01:18:51.900 | - Coordinate descent is more of a trivial one,

01:18:54.020 | you just pick one of the coordinates--

01:18:55.300 | - That's how we think, that's how our human minds--

01:18:56.900 | - That's how our human minds think,

01:18:58.020 | and gradients are not that easy

01:18:59.980 | for our human mind to grapple with.

01:19:03.180 | - An absurd question, but what is statistics?

01:19:07.140 | - So here it's a little bit,

01:19:09.780 | it's somewhere between math and science and technology,

01:19:12.100 | it's somewhere in that convex hole.

01:19:13.340 | So it's a set of principles that allow you

01:19:15.900 | to make inferences that have got some reason to be believed,

01:19:18.820 | and also principles that allow you to make decisions

01:19:21.340 | where you can have some reason to believe

01:19:23.300 | you're not gonna make errors.

01:19:25.020 | So all of that requires some assumptions

01:19:26.460 | about what do you mean by an error,

01:19:27.540 | what do you mean by the probabilities,

01:19:31.180 | but after you start making some of those assumptions,

01:19:34.260 | you're led to conclusions that yes,

01:19:37.900 | I can guarantee that if you do this in this way,

01:19:40.700 | your probability of making an error will be small.

01:19:43.460 | Your probability of continuing to not make errors

01:19:46.100 | over time will be small,

01:19:47.580 | and probability you found something that's real

01:19:50.420 | will be small, will be high.

01:19:52.580 | - So decision making is a big part of that.

01:19:54.460 | - Decision making is a big part, yeah.

01:19:55.660 | So the original, so statistics,

01:19:58.460 | short history was that it goes back,

01:20:01.220 | as a formal discipline, 250 years or so.

01:20:04.860 | It was called inverse probability,

01:20:06.420 | because around that era, probability was developed

01:20:09.740 | especially to explain gambling situations.

01:20:13.300 | - Of course, interesting.

01:20:15.340 | - So you would say, well, given the state of nature is this,

01:20:18.180 | there's a certain roulette board

01:20:19.140 | that has a certain mechanism in it,

01:20:20.860 | what kind of outcomes do I expect to see?

01:20:23.420 | And especially if I do things long amounts of time,

01:20:26.820 | what outcomes will I see?

01:20:27.660 | And the physicists started to pay attention to this.

01:20:30.500 | And then people said, well, given,

01:20:31.940 | let's turn the problem around.

01:20:33.380 | What if I saw certain outcomes,

01:20:35.260 | could I infer what the underlying mechanism was?

01:20:37.340 | That's an inverse problem.

01:20:38.420 | And in fact, for quite a while,

01:20:39.860 | statistics was called inverse probability.

01:20:41.900 | That was the name of the field.

01:20:43.820 | And I believe that it was Laplace

01:20:47.340 | who was working in Napoleon's government,

01:20:49.420 | who needed to do a census of France,

01:20:53.020 | learn about the people there.

01:20:54.220 | So he went and got and gathered data,

01:20:55.900 | and he analyzed that data to determine policy,

01:20:59.940 | and said, well, let's call this field

01:21:02.820 | that does this kind of thing statistics,

01:21:04.540 | 'cause the word state is in there.

01:21:07.340 | In French, that's état,

01:21:08.620 | but it's the study of data for the state.

01:21:11.920 | So anyway, that caught on,

01:21:14.580 | and it's been called statistics ever since.

01:21:17.100 | But by the time it got formalized,

01:21:20.420 | it was sort of in the 30s.

01:21:21.860 | And around that time,

01:21:24.340 | there was game theory and decision theory developed nearby.

01:21:27.340 | People in that era didn't think of themselves

01:21:29.780 | as either computer science or statistics

01:21:31.620 | or control or econ.

01:21:32.620 | They were all of the above.

01:21:34.380 | And so, von Neumann is developing game theory,

01:21:36.540 | but also thinking of that as decision theory.

01:21:39.180 | Wald is an econometrician developing decision theory,

01:21:42.420 | and then turning that into statistics.

01:21:44.820 | And so it's all about,

01:21:45.980 | here's not just data and you analyze it.

01:21:48.740 | Here's a loss function.

01:21:50.100 | Here's what you care about.

01:21:50.940 | Here's the question you're trying to ask.

01:21:52.860 | Here is a probability model,

01:21:54.780 | and here is the risk you will face

01:21:56.340 | if you make certain decisions.

01:21:57.900 | And to this day, in most advanced statistical curricula,

01:22:02.960 | you teach decision theory as the starting point.

01:22:05.300 | And then it branches out into the two branches

01:22:07.300 | of Bayesian and Frequentist.

01:22:08.380 | But it's all about decisions.

01:22:10.500 | - In statistics, what is the most beautiful,

01:22:16.180 | mysterious, maybe surprising idea that you've come across?

01:22:22.140 | - Yeah, good question.

01:22:23.280 | I mean, there's a bunch of surprising ones.

01:22:27.500 | There's something that's way too technical for this thing,

01:22:29.540 | but something called James Stein estimation,

01:22:31.380 | which is kind of surprising

01:22:33.500 | and really takes time to wrap your head around.

01:22:35.900 | - Can you try to maybe--

01:22:37.100 | - Nah, I think I don't even wanna try.

01:22:39.900 | Let me just say a colleague,

01:22:41.980 | Stephen Stickler at University of Chicago

01:22:43.460 | wrote a really beautiful paper on James Stein estimation,

01:22:45.940 | which just helps to, it's viewed as a paradox.

01:22:48.380 | It kind of defeats the mind's attempts to understand it,

01:22:50.600 | but you can, and Steve has a nice perspective on that.

01:22:53.560 | So one of the troubles with statistics

01:22:58.360 | is that it's like in physics, or in quantum physics,

01:23:00.700 | you have multiple interpretations.

01:23:02.420 | There's a wave and particle duality in physics.

01:23:04.700 | And you get used to that over time,

01:23:07.220 | but it still kind of haunts you

01:23:08.500 | that you don't really quite understand the relationship.

01:23:11.660 | The electron's a wave and electron's a particle.

01:23:14.020 | Well, the same thing happens here.

01:23:16.660 | There's Bayesian ways of thinking in Frequentist,

01:23:19.060 | and they are different.

01:23:20.420 | They sometimes become sort of the same in practice,

01:23:23.780 | but they are physically different.

01:23:24.880 | And then in some practice, they are not the same at all.

01:23:27.600 | They give you rather different answers.

01:23:30.360 | And so it is very much like wave and particle duality,

01:23:32.920 | and that is something you have to kind of

01:23:34.280 | get used to in the field.

01:23:35.720 | - Can you define Bayesian in Frequentist?

01:23:37.600 | - Yeah, in decision theory, you can make,

01:23:38.860 | I have a video that people could see.

01:23:41.240 | It's called Are You a Bayesian or a Frequentist?

01:23:43.200 | And kind of help try to make it really clear.

01:23:45.980 | It comes from decision theory.

01:23:47.060 | So, decision theory, you're talking about loss functions,

01:23:51.300 | which are a function of data X and parameter theta.

01:23:55.180 | They're a function of two arguments.

01:23:56.980 | Neither one of those arguments is known.

01:23:59.820 | You don't know the data a priori, it's random,

01:24:02.820 | and the parameter's unknown.

01:24:04.380 | So you have this function of two things you don't know,

01:24:06.220 | and you're trying to say, I want that function to be small.

01:24:08.180 | I want small loss.

01:24:09.240 | Well, what are you gonna do?

01:24:13.340 | So you sort of say, well, I'm gonna average

01:24:15.100 | over these quantities or maximize over them or something

01:24:17.900 | so that I turn that uncertainty into something certain.

01:24:21.940 | So you could look at the first argument and average over it,

01:24:25.380 | or you could look at the second argument, average over it.

01:24:26.900 | That's Bayesian Frequentist.

01:24:27.900 | So the Frequentist says, I'm gonna look at the X, the data,

01:24:32.140 | and I'm gonna take that as random,

01:24:33.660 | and I'm gonna average over the distribution.

01:24:35.220 | So I take the expectation of loss under X.

01:24:38.540 | Theta's held fixed, right?

01:24:40.540 | That's called the risk.

01:24:42.020 | And so it's looking at all the datasets you could get,

01:24:44.980 | and saying how well will a certain procedure do

01:24:48.140 | under all those datasets?

01:24:50.020 | That's called a Frequentist guarantee.

01:24:52.500 | So I think it is very appropriate

01:24:54.080 | when you're building a piece of software,

01:24:55.900 | and you're shipping it out there,

01:24:57.000 | and people are using it on all kinds of datasets.

01:24:59.200 | You wanna have a stamp, a guarantee on it

01:25:00.860 | that has people running on many, many datasets

01:25:02.660 | that you never even thought about,

01:25:03.700 | that 95% of the time it will do the right thing.

01:25:06.180 | Perfectly reasonable.

01:25:08.440 | The Bayesian perspective says, well, no,

01:25:11.720 | I'm gonna look at the other argument of the loss function,

01:25:13.700 | the theta part, okay?

01:25:14.980 | That's unknown, and I'm uncertain about it.

01:25:17.500 | So I could have my own personal probability for what it is.

01:25:20.700 | How many tall people are there out there?

01:25:22.140 | I'm trying to infer the average height of the population.

01:25:24.020 | Well, I have an idea of roughly what the height is.

01:25:27.320 | So I'm gonna average over the theta.

01:25:32.060 | So now that loss function has only now, again,

01:25:35.420 | one argument's gone.

01:25:37.100 | Now it's a function of X.

01:25:38.780 | And that's what a Bayesian does, is they say,

01:25:40.340 | well, let's just focus on a particular X we got,

01:25:42.340 | the dataset we got, we condition on that.

01:25:44.940 | Condition on the X, I say something about my loss.

01:25:48.100 | That's a Bayesian approach to things.

01:25:50.300 | And the Bayesian will argue that it's not relevant

01:25:53.220 | to look at all the other datasets you could have gotten

01:25:55.840 | and average over them, the frequentist approach.

01:25:58.640 | It's really only the datasets you got, all right?

01:26:01.940 | And I do agree with that,

01:26:03.500 | especially in situations where you're working

01:26:05.140 | with a scientist, you can learn a lot about the domain,

01:26:07.540 | and you're really only focused on certain kinds of data,

01:26:09.540 | and you've gathered your data, and you make inferences.

01:26:12.280 | I don't agree with it though, in the sense that

01:26:15.940 | there are needs for frequentist guarantees.

01:26:18.100 | You're writing software, people are using it out there,

01:26:19.940 | you wanna say something.

01:26:20.820 | So these two things have got to fight each other a little bit

01:26:23.220 | but they have to blend.

01:26:24.780 | So long story short, there's a set of ideas

01:26:26.700 | that are right in the middle,

01:26:27.540 | that are called empirical Bayes.

01:26:29.540 | And empirical Bayes sort of starts

01:26:31.500 | with the Bayesian framework.

01:26:33.540 | It's kind of arguably philosophically more,

01:26:38.300 | reasonable and kosher.

01:26:40.420 | Write down a bunch of the math that kind of flows from that,

01:26:43.380 | and then realize there's a bunch of things you don't know,

01:26:45.340 | because it's the real world,

01:26:46.660 | and you don't know everything,

01:26:47.940 | so you're uncertain about certain quantities.

01:26:50.060 | At that point ask, is there a reasonable way

01:26:52.100 | to plug in an estimate for those things?

01:26:54.740 | Okay, and in some cases,

01:26:56.500 | there's quite a reasonable thing to do, to plug in.

01:26:59.780 | There's a natural thing you can observe in the world

01:27:01.740 | that you can plug in,

01:27:03.140 | and then do a little bit more mathematics

01:27:04.620 | and assure yourself it's really good.

01:27:06.340 | - So based on math or based on human expertise,

01:27:08.700 | what are good--

01:27:09.980 | - They're both going in.

01:27:10.820 | The Bayesian framework allows you

01:27:11.860 | to put a lot of human expertise in.

01:27:13.660 | But the math kind of guides you along that path,

01:27:17.820 | and then kind of reassures you at the end,

01:27:18.980 | you could put that stamp of approval.

01:27:20.460 | Under certain assumptions, this thing will work.

01:27:22.660 | So you asked the question, what's my favorite,

01:27:24.780 | what's the most surprising nice idea?

01:27:26.020 | So one that is more accessible

01:27:27.620 | is something called false discovery rate,

01:27:29.620 | which is you're making not just one hypothesis test,

01:27:34.380 | you're making one decision,

01:27:35.300 | you're making a whole bag of them.

01:27:37.020 | And in that bag of decisions,

01:27:39.300 | you look at the ones where you made a discovery,

01:27:41.100 | you announced that something interesting had happened.

01:27:43.740 | All right, that's gonna be some subset of your big bag.

01:27:46.980 | In the ones you made a discovery,

01:27:48.420 | which subset of those are bad,

01:27:50.660 | there are false, false discoveries.

01:27:53.180 | You'd like the fraction of your false discoveries

01:27:55.080 | among your discoveries to be small.

01:27:57.460 | That's a different criterion than accuracy or precision

01:28:00.740 | or recall or sensitivity and specificity.

01:28:02.780 | It's a different quantity.

01:28:04.700 | Those latter ones are almost all of them

01:28:06.740 | have more of a frequentist flavor.

01:28:09.900 | They say, given the truth is that the null hypothesis is true

01:28:13.700 | here's what accuracy I would get.

01:28:15.700 | Or given that the alternative is true,

01:28:17.060 | here's what I would get.

01:28:18.500 | So it's kind of going forward

01:28:19.580 | from the state of nature to the data.

01:28:22.300 | The Bayesian goes the other direction

01:28:23.760 | from the data back to the state of nature.

01:28:25.780 | And that's actually what false discovery rate is.

01:28:28.100 | It says, given you made a discovery,

01:28:30.540 | okay, that's conditioned on your data,

01:28:32.500 | what's the probability of the hypothesis?

01:28:34.940 | It's going the other direction.

01:28:36.500 | And so the classical frequency look at that.

01:28:39.500 | So I can't know that there's some priors needed in that.

01:28:42.440 | And the empirical Bayesian goes ahead and plows forward

01:28:45.820 | and starts writing down these formulas

01:28:47.580 | and realizes at some point,

01:28:49.300 | some of those things can actually be estimated

01:28:50.820 | in a reasonable way.

01:28:52.500 | And so it's a beautiful set of ideas.

01:28:54.180 | So this kind of line of argument has come out.

01:28:56.580 | It's not certainly mine,

01:28:57.860 | but it sort of came out from Robbins around 1960.

01:29:02.260 | Brad Efron has written beautifully about this

01:29:04.980 | in various papers and books.

01:29:06.260 | And the FDR is, you know, Ben Yamini in Israel,

01:29:11.260 | John Story did this Bayesian interpretation and so on.

01:29:14.700 | So I've just absorbed these things over the years

01:29:16.940 | and find it a very healthy way to think about statistics.

01:29:19.840 | - Let me ask you about intelligence

01:29:23.220 | to jump slightly back out into philosophy, perhaps.

01:29:28.180 | You said that, maybe you can elaborate,

01:29:31.080 | but you said that defining just even the question

01:29:33.980 | of what is intelligence is a very difficult question.

01:29:38.720 | Is it a useful question?

01:29:40.020 | Do you think we'll one day understand

01:29:41.900 | the fundamentals of human intelligence and what it means?

01:29:44.800 | You know, have good benchmarks for general intelligence

01:29:50.800 | that we put before our machines.

01:29:53.420 | - So I don't work on these topics so much.

01:29:55.380 | You're really asking a question for a psychologist, really.

01:29:58.500 | And I studied some, but I don't consider myself

01:30:01.760 | at least an expert at this point.

01:30:03.400 | You know, a psychologist aims to understand

01:30:06.120 | human intelligence, right?

01:30:07.560 | And I think many psychologists I know

01:30:09.560 | are fairly humble about this.

01:30:10.800 | They might try to understand how a baby understands,

01:30:14.040 | you know, whether something's a solid or liquid

01:30:16.000 | or whether something's hidden or not.

01:30:18.400 | And maybe how a child starts to learn the meaning

01:30:22.880 | of certain words, what's a verb, what's a noun,

01:30:25.040 | and also, you know, slowly but surely

01:30:27.700 | trying to figure out things.

01:30:30.480 | But humans' ability to take a really complicated

01:30:33.300 | environment, reason about it, abstract about it,

01:30:35.860 | find the right abstractions, communicate about it,

01:30:39.200 | interact, and so on, is just, you know,

01:30:41.840 | really staggeringly rich and complicated.

01:30:45.300 | And so, you know, I think in all humility,

01:30:49.060 | we don't think we're kind of aiming for that

01:30:51.380 | in the near future.

01:30:52.300 | And certainly a psychologist doing experiments

01:30:54.040 | with babies in the lab or with people talking

01:30:56.660 | has a much more limited aspiration.

01:30:58.680 | And, you know, Kahneman-Dvorsky would look

01:31:00.300 | at our reasoning patterns, and they're not deeply

01:31:02.600 | understanding all the how we do our reasoning,

01:31:04.660 | but they're sort of saying, "Here's some oddities

01:31:06.500 | "about the reasoning and some things you need

01:31:08.580 | "to think about it."

01:31:09.420 | But also, as I emphasize in some things I've been writing

01:31:12.180 | about, you know, AI, the revolution hasn't happened yet.

01:31:15.500 | - Yeah, great blog post.

01:31:17.240 | - I've been emphasizing that, you know,

01:31:19.860 | if you step back and look at intelligent systems

01:31:22.660 | of any kind, whatever you mean by intelligence,

01:31:24.900 | it's not just the humans or the animals

01:31:26.420 | or, you know, the plants or whatever, you know.

01:31:29.400 | So a market that brings goods into a city, you know,

01:31:31.880 | food to restaurants or something every day is a system.

01:31:35.580 | It's a decentralized set of decisions.

01:31:37.560 | Looking at it from far enough away,

01:31:38.840 | it's just like a collection of neurons.

01:31:40.200 | Every neuron is making its own little decisions,

01:31:42.740 | presumably in some way.

01:31:44.340 | And if you step back enough, every little part

01:31:46.460 | of an economic system is making all of its decisions.

01:31:49.220 | And just like with the brain, who knows what,

01:31:51.180 | an individual neuron doesn't know what the overall goal is,

01:31:54.320 | right, but something happens at some aggregate

01:31:56.360 | level, same thing with the economy.

01:31:58.320 | People eat in a city and it's robust.

01:32:01.240 | It works at all scales, small villages to big cities.

01:32:04.740 | It's been working for thousands of years.

01:32:07.040 | It works rain or shine, so it's adaptive.

01:32:09.160 | So all the kind of, you know, those are adjectives

01:32:12.160 | one tends to apply to intelligent systems, robust,

01:32:15.240 | adaptive, you know, you don't need to keep adjusting it,

01:32:17.840 | self-healing, whatever, plus not perfect.

01:32:20.840 | You know, intelligences are never perfect

01:32:22.440 | and markets are not perfect.

01:32:24.560 | But I do not believe in this era

01:32:25.920 | that you can say, well, our computers,

01:32:28.240 | our humans are smart, but no markets are not.

01:32:30.560 | Well, markets are, so they are intelligent.

01:32:33.000 | Now, we humans didn't evolve to be markets.

01:32:37.960 | We've been participating in them, right,

01:32:40.160 | but we are not ourselves a market per se.

01:32:43.280 | - The neurons could be viewed as the market.

01:32:45.200 | - You can, there's economic, you know,

01:32:46.840 | neuroscience kind of perspectives.

01:32:48.080 | That's interesting to pursue all that.

01:32:50.280 | The point, though, is that if you were to study humans

01:32:52.720 | and really be the world's best psychologist

01:32:54.800 | and study for thousands of years

01:32:55.840 | and come up with the theory of human intelligence,

01:32:57.440 | you might have never discovered principles of markets,

01:33:00.440 | you know, supply-demand curves and, you know,

01:33:02.640 | matching and auctions and all that.

01:33:04.840 | Those are real principles and they lead

01:33:06.280 | to a form of intelligence

01:33:07.680 | that's not maybe human intelligence.

01:33:09.360 | It's arguably another kind of intelligence.

01:33:11.400 | There probably are third kinds of intelligence or fourth

01:33:13.840 | that none of us are really thinking

01:33:14.960 | too much about right now.

01:33:16.400 | So if you really, and all those are relevant

01:33:18.440 | to computer systems in the future.

01:33:20.440 | Certainly the market one is relevant right now,

01:33:23.760 | whereas understanding human intelligence is not so clear

01:33:26.160 | that it's relevant right now, probably not.

01:33:28.360 | So if you want general intelligence,

01:33:30.920 | whatever one means by that,

01:33:31.960 | or understanding intelligence in a deep sense and all that,

01:33:34.640 | it definitely has to be not just human intelligence.

01:33:37.560 | It's gotta be this broader thing.

01:33:39.080 | And that's not a mystery.

01:33:40.160 | Markets are intelligent.

01:33:41.240 | So, you know, it's definitely not just a philosophical stance

01:33:44.360 | to say we gotta move beyond human intelligence.

01:33:45.960 | That sounds ridiculous, but it's not.

01:33:48.240 | - And in that block,

01:33:49.200 | well, as you define different kinds

01:33:50.520 | of like intelligent infrastructure, AI,

01:33:52.760 | which I really like,

01:33:54.000 | it's some of the concepts you've just been describing.

01:33:57.920 | Do you see ourselves,

01:33:59.000 | if we see Earth, human civilization as a single organism,

01:34:02.600 | do you think the intelligence of that organism,

01:34:05.160 | when you think from the perspective of markets

01:34:07.040 | and intelligence infrastructure is increasing?

01:34:10.760 | Is it increasing linearly?

01:34:12.320 | Is it increasing exponentially?

01:34:14.080 | What do you think the future of that intelligence?

01:34:16.040 | - I don't know.

01:34:16.880 | I don't tend to think,

01:34:17.760 | I don't tend to answer questions like that

01:34:19.760 | 'cause, you know, that's science fiction.

01:34:21.320 | - I was hoping to catch you off guard.

01:34:23.240 | (laughing)

01:34:25.000 | Well, again, because you said it's so far in the future,

01:34:28.120 | it's fun to ask and you'll probably, you know,

01:34:32.320 | like you said, predicting the future

01:34:33.920 | is really nearly impossible.

01:34:36.200 | But say as an axiom,

01:34:38.960 | one day we create a human level,

01:34:41.760 | a superhuman level intelligent,

01:34:43.440 | not the scale of markets,

01:34:44.720 | but the scale of an individual.

01:34:47.180 | What do you think is,

01:34:49.920 | what do you think it would take to do that?

01:34:51.680 | Or maybe to ask another question

01:34:53.760 | is how would that system be different

01:34:56.680 | than the biological human beings

01:34:59.720 | that we see around us today?

01:35:01.360 | Is it possible to say anything interesting to that question

01:35:04.200 | or is it just a stupid question?

01:35:06.080 | - It's not a stupid question, but it's science fiction.

01:35:08.200 | - Science fiction.

01:35:09.040 | - And so I'm totally happy to read science fiction

01:35:11.320 | and think about it from time of my own life.

01:35:13.400 | I love the, there was this like brain in a vat kind of,

01:35:16.200 | you know, little thing that people were talking about

01:35:18.200 | when I was a student.

01:35:19.040 | I remember, you know, imagine that,

01:35:21.000 | you know, between your brain and your body,

01:35:24.240 | there's a bunch of wires, right?

01:35:26.400 | And suppose that every one of them

01:35:28.040 | was replaced with a literal wire.

01:35:31.400 | And then suppose that wire was turned

01:35:32.640 | into actually a little wireless, you know,

01:35:34.200 | there's a receiver and sender.

01:35:35.920 | So the brain has got all the senders and receiver,

01:35:38.400 | you know, on all of its exiting, you know, axons

01:35:42.120 | and all the dendrites down in the body

01:35:43.880 | are replaced with senders and receivers.

01:35:45.840 | Now you could move the body off somewhere

01:35:47.560 | and put the brain in a vat, right?

01:35:49.680 | And then you could do things like start killing off

01:35:52.440 | those senders and receivers one by one.

01:35:54.440 | And after you've killed off all of them,

01:35:55.720 | where is that person?

01:35:56.720 | You know, they thought they were out in the body

01:35:58.280 | walking around the world and they moved on.

01:35:59.760 | So those are science fiction things.

01:36:00.880 | Those are fun to think about.

01:36:02.020 | It's just intriguing about what is thought,

01:36:03.960 | where is it and all that.

01:36:05.440 | And I think every 18 year old should take philosophy classes

01:36:09.240 | and think about these things.

01:36:10.560 | And I think that everyone should think about

01:36:11.800 | what could happen in society that's kind of bad

01:36:13.680 | and all that.

01:36:14.520 | But I really don't think that's the right thing

01:36:15.800 | for most of us that are my age group to be doing

01:36:18.160 | and thinking about.

01:36:19.500 | I really think that we have so many more present,

01:36:22.840 | you know, first challenges and dangers

01:36:26.360 | and real things to build and all that,

01:36:28.520 | such that, you know, spending too much time

01:36:32.040 | on science fiction, at least in public for like this,

01:36:33.920 | I think is not what we should be doing.

01:36:35.840 | - Maybe over beers in private.

01:36:37.440 | - That's right.

01:36:38.280 | I'm well,

01:36:39.100 | (laughing)

01:36:40.840 | I'm not gonna broadcast where I have beers

01:36:42.440 | because this is gonna go on Facebook

01:36:43.720 | and I don't want a lot of people showing up there.

01:36:45.320 | But yeah.

01:36:47.000 | - I love Facebook, Twitter, Amazon, YouTube.

01:36:51.640 | I have, I'm optimistic and hopeful,

01:36:54.160 | but maybe I don't have grounds for such optimism and hope.

01:36:59.000 | Let me ask, you've mentored some of the brightest,

01:37:04.960 | sort of some of the seminal figures in the field.

01:37:08.240 | Can you give advice to people who are undergraduates today?

01:37:13.880 | What does it take to take, you know, advice on their journey

01:37:16.760 | if they're interested in machine learning and AI

01:37:19.080 | and in the ideas of markets from economics to psychology

01:37:23.720 | and all the kinds of things that you're exploring,

01:37:25.520 | what steps should they take on that journey?

01:37:28.160 | - Well, yeah, first of all, the door's open

01:37:29.400 | and second, it's a journey.

01:37:30.360 | I like your language there.

01:37:31.760 | It is not that you're so brilliant

01:37:35.560 | and you have great, brilliant ideas

01:37:36.760 | and therefore that's just, you know,

01:37:38.760 | that's how you have success

01:37:39.880 | or that's how you enter into the field.

01:37:42.280 | It's that you apprentice yourself,

01:37:44.000 | you spend a lot of time, you work on hard things,

01:37:46.880 | you try and pull back and you be as broad as you can,

01:37:51.520 | you talk to lots of people.

01:37:53.640 | And it's like entering any kind of a creative community.

01:37:56.760 | There's years that are needed

01:37:59.080 | and human connections are critical to it.

01:38:01.520 | So, you know, I think about, you know,

01:38:03.140 | being a musician or being an artist or something,

01:38:05.960 | you don't just, you know, immediately from day one,

01:38:08.480 | you know, you're a genius and therefore you do it.

01:38:11.120 | No, you, you know, practice really, really hard on basics

01:38:16.120 | and you be humble about where you are

01:38:19.520 | and then you realize you'll never be an expert

01:38:21.600 | on everything, so you kind of pick

01:38:23.240 | and there's a lot of randomness and a lot of kind of luck,

01:38:27.400 | but luck just kind of picks out which branch

01:38:30.360 | of the tree you go down, but you'll go down some branch.

01:38:33.800 | So yeah, it's a community.

01:38:35.400 | So the graduate school is, I still think,

01:38:37.400 | is one of the wonderful phenomena

01:38:38.640 | that we have in our world.

01:38:40.640 | It's very much about apprenticeship with an advisor.

01:38:43.080 | It's very much about a group of people you belong to.

01:38:45.600 | It's a four or five year process.

01:38:46.960 | So it's plenty of time to start from kind of nothing

01:38:50.120 | to come up to something, you know, more expertise

01:38:52.520 | and then start to have your own creativity start to flower,

01:38:54.480 | even surprising your own self.

01:38:56.340 | And it's a very cooperative endeavor.

01:38:59.600 | It's, I think a lot of people think

01:39:02.000 | of science as highly competitive

01:39:04.880 | and I think in some other fields, it might be more so.

01:39:07.960 | Here it's way more cooperative than you might imagine.

01:39:10.560 | And people are always teaching each other something

01:39:13.480 | and people are always more than happy to be clear.

01:39:16.760 | So I feel I'm an expert on certain kinds of things,

01:39:19.300 | but I'm very much not expert on lots of other things.

01:39:21.320 | And a lot of them are relevant and a lot of them are,

01:39:23.520 | I should know, but should in some sense, you know, you don't.

01:39:26.280 | So I'm always willing to reveal my ignorance

01:39:29.620 | to people around me so they can teach me things.

01:39:31.840 | And I think a lot of us feel that way about our field.

01:39:34.200 | So it's very cooperative.

01:39:35.960 | I might add it's also very international

01:39:37.920 | 'cause it's so cooperative.

01:39:39.040 | We see no barriers.

01:39:40.560 | And so the nationalism that you see,

01:39:43.200 | especially in the current era and everything,

01:39:44.520 | is just at odds with the way that most of us think

01:39:46.840 | about what we're doing here,

01:39:47.960 | where this is a human endeavor and we cooperate

01:39:51.200 | and are very much trying to do it together

01:39:53.560 | for the benefit of everybody.

01:39:56.520 | - So last question, where and how and why

01:39:59.800 | did you learn French?

01:40:01.360 | And which language is more beautiful, English or French?

01:40:05.480 | - Great question.

01:40:06.360 | So first of all, I think Italian's actually more beautiful

01:40:09.240 | than French and English.

01:40:10.080 | And I also speak that.

01:40:11.080 | So I'm married to an Italian and I have kids

01:40:14.200 | and we speak Italian.

01:40:15.240 | Anyway, all kidding aside,

01:40:19.720 | every language allows you to express things

01:40:21.480 | a bit differently.

01:40:22.960 | And it is one of the great fun things to do in life

01:40:25.760 | is to explore those things.

01:40:26.760 | So in fact, when I kids or teens or college students

01:40:31.760 | ask me what they should study,

01:40:33.520 | I say, well, do what your heart, where your heart is,

01:40:35.800 | certainly do a lot of math.

01:40:36.840 | Math is good for everybody,

01:40:38.480 | but do some poetry and do some history

01:40:40.360 | and do some language too.

01:40:41.680 | Throughout your life, you'll wanna be a thinking person.

01:40:44.560 | You'll wanna have done that.

01:40:46.040 | For me, yeah, French I learned when I was,

01:40:50.400 | I'd say a late teen.

01:40:53.480 | I was living in the middle of the country in Kansas

01:40:56.120 | and not much was going on in Kansas

01:40:58.440 | with all due respect to Kansas.

01:40:59.880 | And so my parents happened to have some French books

01:41:02.860 | on the shelf and just in my boredom,

01:41:04.400 | I pulled them down and I found this is fun.

01:41:07.040 | And I kind of learned the language by reading.

01:41:08.920 | And when I first heard it spoken,

01:41:11.280 | I had no idea what was being spoken,

01:41:13.040 | but I realized I had somehow knew it

01:41:14.460 | from some previous life.

01:41:15.520 | And so I made the connection.

01:41:16.960 | But then I traveled and just,

01:41:20.080 | I love to go beyond my own barriers

01:41:22.160 | and my own comfort or whatever.

01:41:23.960 | And I found myself on trains in France

01:41:26.520 | next to say older people who had lived

01:41:29.360 | a whole life of their own.

01:41:30.280 | And the ability to communicate with them

01:41:32.640 | was special and ability to also see myself

01:41:37.640 | in other people's shoes and have empathy

01:41:39.700 | and kind of work on that language as part of that.

01:41:42.200 | So after that kind of experience

01:41:46.960 | and also embedding myself in French culture,

01:41:49.060 | which is quite amazing, languages are rich,

01:41:52.280 | not just 'cause there's something

01:41:53.400 | inherently beautiful about it,

01:41:54.520 | but it's all the creativity that went into it.

01:41:55.920 | So I learned a lot of songs, read poems, read books.

01:41:59.420 | And then I was here actually at MIT

01:42:01.760 | where we're doing the podcast today.

01:42:04.120 | And young professor, not yet married

01:42:08.520 | and not having a lot of friends in the area.

01:42:11.920 | So I just didn't have, I was kind of a bored person.

01:42:14.000 | I said, I heard a lot of Italians around.

01:42:15.960 | There's happened to be a lot of Italians at MIT,

01:42:17.760 | a lot of Italian professors for some reason.

01:42:19.760 | And so I was kind of vaguely understanding

01:42:21.600 | what they were talking about.

01:42:22.440 | I said, well, I should learn this language too.

01:42:23.620 | So I did.

01:42:25.280 | And then later met my spouse and Italian

01:42:28.560 | became a more important part of my life.

01:42:30.760 | But I go to China a lot these days.

01:42:32.200 | I go to Asia, I go to Europe.

01:42:33.960 | And every time I go, I kind of amazed

01:42:37.720 | by the richness of human experience.

01:42:39.240 | And the people don't have any idea if you haven't traveled

01:42:42.640 | kind of how amazingly rich and I love the diversity.

01:42:46.840 | It's not just a buzzword to me, it really means something.

01:42:49.120 | I love the, embed myself with other people's experiences.

01:42:53.060 | And so yeah, learning language is a big part of that.

01:42:56.440 | I think I've said in some interview at some point

01:42:58.400 | that if I had millions of dollars

01:43:00.360 | and infinite time or whatever,

01:43:01.380 | what would you really work on if you really wanted to do AI?

01:43:03.720 | And for me, that is natural language and really done right.

01:43:07.180 | Deep understanding of language.

01:43:09.640 | That's to me, amazingly interesting scientific challenge.

01:43:13.400 | - One we're very far away on.

01:43:14.960 | - One we're very far away, but good natural language people

01:43:17.840 | are kind of really invested in that.

01:43:19.160 | I think a lot of them see that's where the core of AI is.

01:43:21.580 | If you understand that,

01:43:22.520 | you really help human communication.

01:43:24.480 | You understand something about the human mind,

01:43:26.080 | the semantics that come out of the human mind.

01:43:28.440 | And I agree, I think that will be such a long time.

01:43:30.960 | So I didn't do that in my career

01:43:32.320 | just 'cause I kind of, I was behind in the early days.

01:43:34.760 | I didn't kind of know enough of that stuff.

01:43:36.520 | I was at MIT, I didn't learn much language

01:43:39.560 | and it was too late at some point

01:43:40.960 | to kind of spend a whole career doing that.

01:43:42.900 | But I admire that field.

01:43:44.400 | And so my little way by learning language,

01:43:48.440 | that part of my brain has been trained up.

01:43:54.280 | - Jan was right, you truly are the Miles Davis

01:43:56.720 | of machine learning.

01:43:57.560 | I don't think there's a better place than it.

01:43:59.440 | Mike, it was a huge honor talking to you today.

01:44:01.400 | Merci beaucoup.

01:44:02.400 | - All right, it's been my pleasure.

01:44:03.480 | Thank you.

01:44:04.880 | - Thanks for listening to this conversation

01:44:06.760 | with Michael I. Jordan.

01:44:08.480 | And thank you to our presenting sponsor, Cash App.

01:44:11.480 | Download it, use code LexPodcast, you'll get $10

01:44:15.680 | and $10 will go to FIRST, an organization that inspires

01:44:18.960 | and educates young minds to become science

01:44:21.360 | and technology innovators of tomorrow.

01:44:23.800 | If you enjoy this podcast, subscribe on YouTube,

01:44:26.640 | give it five stars on Apple Podcast, support on Patreon

01:44:29.880 | or simply connect with me on Twitter @LexFriedman.

01:44:33.480 | And now let me leave you with some words of wisdom

01:44:37.120 | from Michael I. Jordan from his blog post titled,

01:44:40.400 | "Artificial Intelligence, the revolution hasn't happened yet

01:44:44.240 | calling for broadening the scope of the AI field."

01:44:48.500 | We should embrace the fact that what we are witnessing

01:44:51.480 | is the creation of a new branch of engineering.

01:44:54.280 | The term engineering is often invoked in a narrow sense

01:44:57.640 | in academia and beyond, with overtones of cold,

01:45:01.720 | effectless machinery and negative connotations

01:45:04.720 | of loss of control by humans.

01:45:06.540 | But an engineering discipline can be what we want it to be.

01:45:10.960 | In the current era, we have a real opportunity

01:45:13.520 | to conceive of something historically new,

01:45:16.400 | a human centric engineering discipline.

01:45:19.760 | I'll resist giving this emerging discipline a name,

01:45:22.520 | but if the acronym AI continues to be used,

01:45:25.600 | let's be aware of the very real limitations

01:45:28.120 | of this placeholder.

01:45:29.720 | Let's broaden our scope, tone down the hype

01:45:33.000 | and recognize the serious challenges ahead.

01:45:35.780 | Thank you for listening and hope to see you next time.

01:45:40.180 | (upbeat music)

01:45:42.760 | (upbeat music)

01:45:45.340 | [BLANK_AUDIO]

Michael I. Jordan: Machine Learning, Recommender Systems, and Future of AI | Lex Fridman Podcast #74

Chapters