Dmitri Dolgov: Waymo and the Future of Self-Driving Cars

00:00:00.000 | The following is a conversation with Dmitry Dolgov,

00:00:03.240 | the CTO of Waymo,

00:00:05.240 | which is an autonomous driving company

00:00:07.100 | that started as Google's self-driving car project in 2009

00:00:11.000 | and became Waymo in 2016.

00:00:13.960 | Dmitry was there all along.

00:00:16.200 | Waymo is currently leading

00:00:17.260 | in the fully autonomous vehicle space

00:00:19.480 | in that they actually have an at-scale deployment

00:00:23.000 | of publicly accessible autonomous vehicles

00:00:25.820 | driving passengers around with no safety driver,

00:00:30.280 | with nobody in the driver's seat.

00:00:32.940 | This, to me, is an incredible accomplishment of engineering

00:00:37.360 | on one of the most difficult

00:00:38.900 | and exciting artificial intelligence challenges

00:00:41.540 | of the 21st century.

00:00:43.560 | Quick mention of a sponsor

00:00:45.080 | followed by some thoughts related to the episode.

00:00:47.800 | Thank you to Trial Labs,

00:00:49.640 | a company that helps businesses apply machine learning

00:00:52.760 | to solve real-world problems.

00:00:55.360 | Blinkist, an app I use for reading

00:00:57.480 | through summaries of books.

00:00:59.240 | BetterHelp, online therapy

00:01:01.040 | with a licensed professional.

00:01:02.620 | And Cash App, the app I use to send money to friends.

00:01:05.920 | Please check out the sponsors in the description

00:01:08.400 | to get a discount at the support of this podcast.

00:01:11.160 | As a side note, let me say that autonomous

00:01:13.780 | and semi-autonomous driving was the focus

00:01:16.320 | of my work at MIT and is a problem space

00:01:19.120 | that I find fascinating and full of open questions

00:01:22.240 | from both a robotics and a human psychology perspective.

00:01:26.360 | There's quite a bit that I could say here

00:01:28.760 | about my experiences in academia on this topic

00:01:32.040 | that revealed to me, let's say,

00:01:34.360 | the less admirable sides of human beings.

00:01:38.000 | But I choose to focus on the positive, on solutions,

00:01:41.160 | on brilliant engineers like Dmitry and the team at Waymo

00:01:44.760 | who work tirelessly to innovate

00:01:46.880 | and to build amazing technology that will define our future.

00:01:50.800 | Because of Dmitry and others like him,

00:01:53.400 | I'm excited for this future.

00:01:55.640 | And who knows, perhaps I too will help contribute

00:01:59.320 | something of value to it.

00:02:01.760 | If you enjoy this thing, subscribe on YouTube,

00:02:04.000 | review it with Five Stars and Up, a podcast,

00:02:06.160 | follow on Spotify, support on Patreon,

00:02:08.680 | or connect with me on Twitter @LexFriedman.

00:02:11.960 | And now, here's my conversation with Dmitry Dolgov.

00:02:15.700 | When did you first fall in love with robotics

00:02:19.640 | or even computer science more in general?

00:02:21.720 | - Computer science first, at a fairly young age.

00:02:25.360 | Then robotics happened much later.

00:02:27.060 | I think my first interesting introduction to computers

00:02:33.200 | was in the late '80s when we got our first computer.

00:02:38.200 | I think it was an IBM, I think IBM AT.

00:02:44.320 | Remember those things that had like a turbo button

00:02:46.640 | in the front? - Turbo button, yeah.

00:02:47.480 | - That you would press it and make the thing

00:02:49.440 | go faster.

00:02:50.520 | - Did that already have floppy disks?

00:02:52.480 | - Yeah, yeah, yeah, yeah.

00:02:53.320 | Like the 5.4 inch ones.

00:02:57.160 | - I think there was a bigger inch.

00:02:58.800 | So go on something, then five inches and three inches.

00:03:02.120 | - Yeah, I think that was the five.

00:03:03.600 | Maybe that was before that was the giant plates

00:03:05.680 | and I didn't get that.

00:03:07.240 | But it was definitely not the three inch ones.

00:03:10.080 | Anyway, so we got that computer.

00:03:13.400 | I spent the first few months just playing video games,

00:03:18.640 | as you would expect.

00:03:19.720 | I got bored of that.

00:03:21.600 | So I started messing around and trying to figure out

00:03:25.640 | how to make the thing do other stuff.

00:03:28.880 | Got into exploring programming.

00:03:32.120 | And a couple of years later, it got to a point

00:03:36.640 | where I actually wrote a game, a little game.

00:03:40.400 | And a game developer, a Japanese game developer

00:03:43.120 | actually offered to buy it for me for a few hundred bucks.

00:03:45.880 | But for a kid in Russia.

00:03:48.840 | - That's a big deal.

00:03:49.680 | - That's a big deal, yeah.

00:03:50.520 | I did not take the deal.

00:03:51.880 | - Wow, integrity.

00:03:53.280 | - Yeah, I instead--

00:03:55.000 | - Stupidity.

00:03:55.840 | - Yes, that was not the most acute financial move

00:03:59.200 | that I made in my life.

00:04:00.120 | Looking back at it now, I instead put it,

00:04:02.480 | well, I had a reason.

00:04:03.560 | I put it online.

00:04:04.920 | It was, what'd you call it back in the day?

00:04:07.680 | So it was a freeware thing, right?

00:04:09.040 | It was not open source, but you could upload the binaries,

00:04:11.520 | you would put the game online.

00:04:12.480 | And the idea was that people like it

00:04:14.320 | and then they contribute

00:04:15.800 | and they send you little donations, right?

00:04:17.120 | So I did my quick math of like,

00:04:18.800 | of course, thousands and millions of people

00:04:21.200 | are gonna play my game,

00:04:22.040 | send me a couple of bucks a piece,

00:04:23.560 | should definitely do that.

00:04:25.080 | As I said, not the best financial decision of my life.

00:04:27.280 | - You're already playing with business models

00:04:28.840 | at that young age.

00:04:29.880 | Remember what language it was?

00:04:31.360 | What programming it was?

00:04:32.200 | - Pascal.

00:04:33.720 | - Which, what?

00:04:34.560 | - Pascal.

00:04:35.400 | - Pascal.

00:04:36.240 | And it had a graphical component, so it's not text-based?

00:04:38.080 | - Yeah, it was like, I think 320 by 200,

00:04:43.160 | whatever it was, I think that kind of the earlier version.

00:04:45.120 | - That's the resolution.

00:04:45.960 | - VGA resolution, right?

00:04:47.200 | And I actually think the reason why this company

00:04:49.120 | wanted to buy it is not like the fancy graphics

00:04:51.040 | or the implementation,

00:04:52.120 | it was maybe the idea of my actual game.

00:04:55.280 | - The idea of the game, okay.

00:04:57.880 | Well, one of the things, it's so funny,

00:04:59.720 | I used to play this game called Golden Axe

00:05:02.320 | and the simplicity of the graphics

00:05:05.720 | and something about the simplicity of the music,

00:05:08.560 | like, it still haunts me.

00:05:11.440 | I don't know if that's a childhood thing,

00:05:12.760 | I don't know if that's the same thing

00:05:13.800 | for Call of Duty these days for young kids,

00:05:16.040 | but I still think that the,

00:05:19.880 | when the games are simple,

00:05:21.920 | that simple purity makes for,

00:05:26.160 | like, allows your imagination to take over

00:05:28.680 | and thereby creating a more magical experience.

00:05:31.280 | Like now with better and better graphics,

00:05:33.280 | it feels like your imagination doesn't get to create worlds,

00:05:38.280 | which is kind of interesting.

00:05:40.360 | It could be just an old man on a porch,

00:05:42.280 | like waving at kids these days that have no respect,

00:05:45.320 | but I still think that graphics

00:05:47.640 | almost get in the way of the experience.

00:05:50.440 | I don't know.

00:05:51.320 | - Flippant Bird.

00:05:52.480 | - Yeah, that's true.

00:05:54.160 | - Well, I don't know if the imagination-

00:05:56.320 | - It's closed.

00:05:57.160 | - Okay, it's closed.

00:05:58.200 | I don't, yeah, but that's more about games that,

00:06:01.560 | like, that's more like Tetris World

00:06:03.160 | where they optimally, masterfully,

00:06:06.720 | like, create a fun, short-term dopamine experience

00:06:12.600 | versus, I'm more referring to, like, role-playing games

00:06:15.680 | where there's like a story,

00:06:16.760 | you can live in it for months or years.

00:06:19.720 | Like, there's an Elder Scrolls series,

00:06:23.520 | which is probably my favorite set of games.

00:06:25.800 | That was a magical experience.

00:06:27.240 | And the graphics are terrible.

00:06:29.160 | The characters were all randomly generated,

00:06:31.120 | but they're, I don't know, it pulls you in.

00:06:34.400 | There's a story.

00:06:35.320 | It's like an interactive version

00:06:38.240 | of an Elder Scrolls Tolkien world.

00:06:41.280 | And you get to live in it.

00:06:42.840 | I don't know.

00:06:44.160 | I miss it.

00:06:45.280 | It's one of the things that suck about being an adult

00:06:48.560 | is there's no, you have to live in the real world

00:06:51.240 | as opposed to the Elder Scrolls world.

00:06:53.240 | - You know, whatever brings you joy, right?

00:06:56.240 | Minecraft, right?

00:06:57.080 | Minecraft's a great example.

00:06:57.920 | You create, like, it's not the fancy graphics,

00:06:59.600 | but it's the creation of your own worlds.

00:07:02.480 | - Yeah, that one is crazy.

00:07:03.720 | You know, one of the pitches for being a parent

00:07:05.760 | that people tell me is that you can, like,

00:07:07.960 | use the excuse of parenting

00:07:09.960 | to go back into the video game world.

00:07:13.080 | And, like, that's like, you know,

00:07:15.840 | father, son, father, daughter time,

00:07:18.640 | but really you just get to play video games with your kids.

00:07:20.840 | So anyway, at that time,

00:07:23.040 | did you have any ridiculous, ambitious dreams

00:07:26.440 | of where as a creator you might go as an engineer?

00:07:30.640 | What did you think of yourself as an engineer,

00:07:33.960 | as a tinker, or did you want to be like an astronaut

00:07:36.280 | or something like that?

00:07:38.160 | - You know, I'm tempted to make something up about,

00:07:40.440 | you know, robots, engineering,

00:07:43.040 | or, you know, mysteries of the universe,

00:07:44.600 | but that's not the actual memory that pops into my mind

00:07:48.080 | when you ask me about childhood dreams,

00:07:49.640 | so I'll actually share the real thing.

00:07:51.640 | When I was maybe four or five years old,

00:07:57.280 | I, as we all do, I thought about,

00:08:00.480 | you know, what I wanted to do when I grow up,

00:08:01.960 | and I had this dream of being a trainer

00:08:07.000 | and a traffic control cop.

00:08:08.720 | You know, they don't have those today, I think,

00:08:11.320 | but, you know, back in the '80s and, you know, in Russia,

00:08:14.600 | you probably are familiar with that, Lex.

00:08:16.000 | They had these, you know, police officers

00:08:19.080 | that would stand in the middle of an intersection all day,

00:08:20.920 | and they would have their, like,

00:08:21.760 | striped, black and white batons

00:08:23.600 | that they would use to, you know,

00:08:24.680 | control the flow of traffic.

00:08:26.160 | And, you know, for whatever reasons,

00:08:27.760 | I was strangely infatuated with this whole process,

00:08:31.080 | and like that, that was my dream.

00:08:33.080 | That's what I wanted to do when I grew up.

00:08:35.080 | And, you know, my parents, both physics profs, by the way,

00:08:39.960 | I think were, you know, a little concerned

00:08:42.480 | with that level of ambition coming from their child,

00:08:45.040 | at, you know, that age.

00:08:47.280 | - Well, that, it's an interesting,

00:08:48.560 | I don't know if you can relate,

00:08:50.560 | but I very much love that idea.

00:08:52.840 | I have a OCD nature that I think lends itself

00:08:56.720 | very close to the engineering mindset,

00:08:59.960 | which is you want to kind of optimize,

00:09:04.520 | you know, solve a problem

00:09:05.960 | by creating an automated solution,

00:09:09.280 | like a set of rules, a set of rules you can follow,

00:09:13.080 | and then thereby make it ultra efficient.

00:09:15.360 | I don't know if that's, it was of that nature.

00:09:17.840 | I certainly have that.

00:09:19.160 | There's like SimCity and factory building games,

00:09:22.640 | all those kinds of things,

00:09:24.000 | kind of speak to that engineering mindset.

00:09:26.360 | Or did you just like the uniform?

00:09:28.040 | - I think it was more of the latter.

00:09:29.440 | I think it was the uniform and the, you know,

00:09:31.520 | the stripe baton that made cars go in the right directions.

00:09:36.520 | But I guess, you know, it is, I did end up,

00:09:40.400 | I guess, you know, working on the transportation industry

00:09:43.440 | one way or another.

00:09:44.280 | - No uniform, no.

00:09:45.600 | - That's right.

00:09:46.440 | - Maybe it was my, you know, deep inner infatuation

00:09:50.600 | with the traffic control batons that led to this career.

00:09:55.520 | - Okay, what, when did you,

00:09:58.120 | when was the leap from programming to robotics?

00:10:01.000 | - That happened later.

00:10:01.840 | That was after grad school.

00:10:03.800 | After, and actually, then it was self-driving cars

00:10:05.840 | was I think my first real hands-on introduction

00:10:09.600 | to robotics.

00:10:10.600 | But I never really had that much hands-on experience

00:10:14.040 | in school and training.

00:10:15.120 | I worked on applied math and physics.

00:10:17.920 | Then in college, I did more kind of abstract

00:10:22.720 | computer science.

00:10:24.120 | And it was after grad school that I really got involved

00:10:28.160 | in robotics, which was actually self-driving cars.

00:10:30.560 | And, you know, that was a big, big flip.

00:10:33.120 | - What grad school?

00:10:34.800 | - So I went to grad school in Michigan,

00:10:36.240 | and then I did a postdoc at Stanford,

00:10:37.840 | which is, that was the postdoc where I got to play

00:10:41.280 | with self-driving cars.

00:10:42.960 | - Yeah, so we'll return there.

00:10:44.440 | Let's go back to Moscow.

00:10:46.720 | So, you know, for episode 100, I talked to my dad.

00:10:50.440 | And also I grew up with my dad, I guess.

00:10:52.640 | (laughing)

00:10:55.360 | So I had to put up with him for many years.

00:10:58.320 | And he went to the FISTIEH, or MIPT.

00:11:03.320 | It's weird to say in English,

00:11:05.920 | 'cause I've heard all of this in Russian.

00:11:08.200 | Moscow Institute of Physics and Technology.

00:11:10.440 | And to me, that was like, I met some super interesting,

00:11:15.320 | as a child, I met some super interesting characters.

00:11:18.280 | It felt to me like the greatest university in the world,

00:11:21.080 | the most elite university in the world.

00:11:23.480 | And just the people that I met that came out of there

00:11:26.880 | were like, not only brilliant, but also special humans.

00:11:31.880 | It seems like that place really tested the soul.

00:11:35.840 | (laughing)

00:11:37.240 | Both in terms of technically and spiritually.

00:11:41.520 | So that could be just the romanticization of that place,

00:11:44.600 | I'm not sure, so maybe you can speak to it.

00:11:46.840 | But is it correct to say that you spent some time

00:11:50.280 | at FISTIEH?

00:11:51.120 | - Yeah, that's right, six years.

00:11:52.600 | I got my bachelor's and master's in physics and math there.

00:11:56.160 | And it's actually interesting, 'cause my dad,

00:11:59.200 | and actually both my parents went there,

00:12:01.040 | and I think all the stories that I heard,

00:12:04.680 | just like you, Alex, growing up about the place

00:12:07.920 | and how interesting and special and magical it was,

00:12:10.400 | I think that was a significant, maybe the main reason

00:12:13.600 | I wanted to go there for college.

00:12:16.680 | Enough so that I actually went back to Russia from the US.

00:12:20.680 | I graduated high school in the US.

00:12:22.560 | - You went back there?

00:12:24.880 | - I went back there, yeah.

00:12:25.880 | - Wow.

00:12:26.720 | - Exactly the reaction most of my peers in college had,

00:12:29.840 | but perhaps a little bit stronger,

00:12:31.960 | that point me out as this crazy kid.

00:12:35.080 | - Were your parents supportive of that?

00:12:36.280 | - Yeah, yeah, I gave them your previous question.

00:12:38.200 | They supported me in letting me pursue my passions

00:12:43.120 | and the things that I was interested in.

00:12:44.480 | - That's a bold move, wow.

00:12:45.840 | What was it like there?

00:12:46.760 | - It was interesting.

00:12:47.640 | Definitely fairly hardcore on the fundamentals

00:12:50.640 | of math and physics, and lots of good memories

00:12:54.960 | from those times.

00:12:56.960 | - So, okay, so Stanford,

00:12:58.480 | how'd you get into autonomous vehicles?

00:13:00.520 | - I had the great fortune and great honor

00:13:04.960 | to join Stanford's DARPA Urban Challenge Team in 2006.

00:13:09.960 | This was a third in the sequence of the DARPA challenges.

00:13:13.600 | There were two grand challenges prior to that,

00:13:16.320 | and then in 2007, they held the DARPA Urban Challenge.

00:13:20.720 | So, you know, I was doing my postdoc.

00:13:22.280 | I joined the team and worked on motion planning

00:13:27.280 | for that competition.

00:13:31.680 | - So, okay, so for people who might not know,

00:13:33.880 | I know from a certain,

00:13:36.320 | autonomous vehicles is a funny world.

00:13:38.320 | In a certain circle of people, everybody knows everything,

00:13:41.200 | and in a certain circle, nobody knows anything

00:13:45.440 | in terms of general public.

00:13:46.600 | So, it's interesting.

00:13:47.640 | It's a good question what to talk about,

00:13:50.560 | but I do think that the Urban Challenge is worth revisiting.

00:13:55.560 | It's a fun little challenge,

00:13:57.600 | one that first sparked so many incredible minds

00:14:02.600 | to focus on one of the hardest problems of our time

00:14:07.080 | in artificial intelligence.

00:14:08.200 | So, that's a success from a perspective

00:14:10.720 | of a single little challenge.

00:14:12.640 | But can you talk about what did the challenge involve?

00:14:15.680 | So, were there pedestrians?

00:14:17.400 | Were there other cars?

00:14:19.040 | What was the goal?

00:14:20.680 | Who was on the team?

00:14:22.360 | How long did it take?

00:14:23.920 | Any fun sort of specs?

00:14:26.640 | - Sure, sure, sure.

00:14:28.360 | So, the way the challenge was constructed

00:14:30.680 | and just a little bit of backgrounding,

00:14:31.960 | as I mentioned, this was the third competition

00:14:35.520 | in that series.

00:14:36.360 | The first two were the Grand Challenge,

00:14:38.120 | called the Grand Challenge.

00:14:38.960 | The goal there was to just drive

00:14:40.680 | in a completely static environment.

00:14:42.320 | You know, you had to drive in a desert.

00:14:44.120 | That was very successful.

00:14:47.760 | So, then DARPA followed with what they called

00:14:50.240 | the Urban Challenge, where the goal was to have,

00:14:53.640 | you know, build vehicles that could operate

00:14:55.480 | in more dynamic environments

00:14:57.080 | and, you know, share them with other vehicles.

00:14:58.680 | There were no pedestrians there.

00:15:00.280 | But what DARPA did is they took over

00:15:02.400 | an abandoned Air Force Base.

00:15:04.800 | And it was kind of like a little fake city

00:15:07.080 | that they built out there.

00:15:08.400 | And they had a bunch of robots, you know, cars,

00:15:13.320 | that were autonomous in there, all at the same time,

00:15:16.280 | mixed in with other vehicles driven by professional drivers.

00:15:21.040 | And each car had a mission.

00:15:24.040 | And so, there's a crude map that they received

00:15:27.040 | at the beginning.

00:15:27.880 | And they had a mission, you know, go here and then there

00:15:29.960 | and over here.

00:15:31.480 | And they kind of all were sharing this environment

00:15:34.800 | at the same time they had to interact with each other,

00:15:36.760 | they had to interact with the human drivers.

00:15:38.560 | So, it's this very first, very rudimentary version

00:15:43.080 | of a self-driving car that, you know, could operate

00:15:46.760 | in an environment, you know,

00:15:49.960 | shared with other dynamic actors.

00:15:51.600 | That, as you said, you know, really, in many ways,

00:15:54.960 | you know, kick-started this whole industry.

00:15:57.120 | - Okay, so who was on the team and how'd you do?

00:16:00.440 | I forget.

00:16:01.400 | (laughing)

00:16:03.400 | - I came in second.

00:16:04.360 | Perhaps that was my contribution to the team.

00:16:07.200 | I think the Stanford team came in first

00:16:08.640 | in the DARPA challenge, but then I joined the team.

00:16:10.880 | And, you know, we came-

00:16:11.720 | - You were the only one with a bug in the code.

00:16:13.960 | I mean, do you have sort of memories

00:16:15.360 | of some particularly challenging things or,

00:16:18.200 | you know, one of the cool things, it's not, you know,

00:16:21.840 | this isn't a product, this isn't a thing that, you know,

00:16:26.240 | it, there's, you have a little bit more freedom

00:16:27.800 | to experiment, so you can take risks,

00:16:29.600 | and there's, so you can make mistakes.

00:16:32.920 | Is there interesting mistakes?

00:16:35.040 | Is there interesting challenges that stand out to you

00:16:37.120 | as something that taught you a good technical lesson

00:16:41.200 | or a good philosophical lesson from that time?

00:16:44.320 | - Yeah, you know, definitely, definitely

00:16:46.000 | a very memorable time.

00:16:47.680 | Not really a challenge, but like one of the most vivid

00:16:51.480 | memories that I have from the time,

00:16:53.960 | and I think that was actually one of the days

00:16:57.680 | that really got me hooked on this whole field was

00:17:01.520 | the first time I got to run my software on the car.

00:17:06.800 | And I was working on a part of our planning algorithm

00:17:11.800 | that had to navigate in parking lots.

00:17:13.880 | So it was something that, you know,

00:17:15.080 | called free space motion planning.

00:17:16.800 | So the very first version of that was, you know,

00:17:19.600 | we tried on the car, it was on Stanford's campus

00:17:22.560 | in the middle of the night, and you had this little,

00:17:25.040 | you know, course constructed with cones

00:17:26.920 | in the middle of a parking lot.

00:17:28.360 | So we're there in like 3 a.m., you know,

00:17:29.840 | by the time we got the code to, you know,

00:17:31.960 | compile and turn over, and, you know, it drove.

00:17:36.160 | I could actually did something quite reasonable.

00:17:38.120 | And, you know, it was of course very buggy at the time

00:17:42.160 | and had all kinds of problems,

00:17:43.840 | but it was pretty darn magical.

00:17:48.200 | I remember going back and, you know,

00:17:50.400 | late at night and trying to fall asleep

00:17:52.680 | and just, you know, being unable to fall asleep

00:17:54.200 | for the rest of the night, just my mind was blown.

00:17:58.480 | And that's what I've been doing ever since

00:18:01.640 | for more than a decade.

00:18:03.520 | - In terms of challenges and, you know,

00:18:06.280 | interesting memories, like on the day of the competition,

00:18:08.520 | it was pretty nerve wracking.

00:18:10.760 | I remember standing there with Mike Montemarolo,

00:18:12.640 | who was the software lead and wrote most of the code.

00:18:15.920 | I think I did one little part of the planner,

00:18:17.800 | Mike, you know, incredibly did pretty much the rest of it

00:18:21.680 | with a bunch of other incredible people.

00:18:24.320 | But I remember standing on the day of the competition,

00:18:27.360 | you know, watching the car, you know, with Mike,

00:18:29.080 | and, you know, cars are completely empty, right?

00:18:32.320 | They're all there lined up in the beginning of the race.

00:18:34.760 | And then, you know, DARPA sends them, you know,

00:18:37.000 | on their mission one by one.

00:18:38.640 | So then leave, and Mike, you just,

00:18:40.600 | they had these sirens,

00:18:41.480 | (imitates siren)

00:18:42.320 | they all had their different silence, right?

00:18:43.680 | Each siren had its own personality, if you will.

00:18:46.400 | So, you know, off they go and you don't see them.

00:18:48.640 | You just kind of, and then every once in a while,

00:18:50.040 | they, you know, come a little bit closer

00:18:51.240 | to where the audience is,

00:18:53.720 | and you can kind of hear, you know,

00:18:55.080 | the sound of your car and, you know,

00:18:56.440 | it seems to be moving along.

00:18:57.480 | So that, you know, it gives you hope.

00:18:58.600 | And then, you know, it goes away

00:19:00.440 | and you can't hear it for too long.

00:19:01.640 | You start getting anxious, right?

00:19:02.520 | So it's a little bit like, you know,

00:19:03.360 | sending your kids to college and like, you know,

00:19:04.720 | kind of you invested in them.

00:19:05.800 | You hope you build it properly,

00:19:08.480 | but like it's still anxiety inducing.

00:19:11.960 | So that was an incredibly fun few days.

00:19:15.800 | In terms of, you know, bugs,

00:19:17.600 | as you mentioned, you know, one,

00:19:19.040 | that was my bug that caused us the loss of the first place,

00:19:22.800 | is there still a debate that, you know,

00:19:24.640 | occasionally have with people on the CMU team.

00:19:26.200 | CMU came first, I should mention.

00:19:28.280 | - CMU, haven't heard of them, but yeah.

00:19:31.600 | It's a small school.

00:19:33.520 | It's really a glitch that, you know,

00:19:36.120 | they happen to succeed at something robotics related.

00:19:38.280 | - Very scenic though.

00:19:39.160 | So most people go there for the scenery.

00:19:42.240 | Yeah, it's a beautiful campus.

00:19:44.040 | (laughing)

00:19:45.520 | I apologize. - Unlike Stanford.

00:19:46.960 | - So for people, yeah, that's true, unlike Stanford.

00:19:49.240 | For people who don't know,

00:19:50.080 | CMU is one of the great robotics

00:19:51.680 | and sort of artificial intelligence universities

00:19:53.880 | in the world.

00:19:54.720 | CMU, Carnegie Mellon University.

00:19:56.880 | Okay, sorry, go ahead.

00:19:58.520 | - Good PSA.

00:19:59.560 | So in the part that I contributed to,

00:20:04.280 | which was navigating parking lots,

00:20:06.120 | and the way that part of the mission worked is,

00:20:09.800 | in a parking lot, you would get from DARPA

00:20:14.280 | an outline of the map.

00:20:15.760 | You basically get this giant polygon

00:20:17.840 | that defined the perimeter of the parking lot.

00:20:20.240 | And there would be an entrance,

00:20:21.400 | and maybe multiple entrances or access to it.

00:20:23.880 | And then you would get a goal within that open space.

00:20:29.880 | XY heading, where the car had to park.

00:20:33.000 | It had no information about the obstacles

00:20:35.400 | that the car might encounter there.

00:20:36.920 | So it had to navigate a completely free space

00:20:39.680 | from the entrance to the parking lot

00:20:42.040 | into that parking space.

00:20:44.120 | And then once you're parked there,

00:20:47.320 | it had to exit the parking lot.

00:20:50.480 | While of course, encountering and reasoning

00:20:51.840 | about all the obstacles that it encounters in real time.

00:20:55.240 | So our interpretation,

00:20:59.280 | or at least my interpretation of the rules

00:21:00.800 | was that you had to reverse out of the parking spot.

00:21:03.800 | And that's what our cars did,

00:21:05.280 | even if there's no obstacle in front.

00:21:07.120 | That's not what CMU's car did.

00:21:08.960 | And it just kind of drove right through.

00:21:11.000 | So there's still a debate.

00:21:12.720 | And of course, you know,

00:21:13.560 | as you stop and then reverse out

00:21:14.560 | and go out the different way,

00:21:16.000 | that costs you some time, right?

00:21:17.240 | So there's still a debate,

00:21:18.920 | whether it was my poor implementation

00:21:20.760 | that cost us extra time,

00:21:22.040 | or whether it was CMU violating

00:21:26.400 | an important rule of the competition.

00:21:27.800 | And I have my own opinion here.

00:21:30.200 | In terms of other bugs,

00:21:31.120 | and I have to apologize to Mike Montemariella

00:21:34.240 | for sharing this on air,

00:21:35.880 | but it is actually one of the more memorable ones.

00:21:39.320 | And it's something that's kind of become a bit of a metaphor,

00:21:43.480 | I don't know, a label in the industry since then.

00:21:45.920 | I think at least in some circles,

00:21:47.440 | it's called the victory circle or victory lap.

00:21:50.080 | And our cars did that.

00:21:53.800 | So in one of the missions in the urban challenge,

00:21:56.880 | in one of the courses,

00:21:58.280 | there was this big oval right by the start

00:22:01.880 | and finish of the race.

00:22:02.720 | So the ARPA had a lot of the missions would finish

00:22:05.240 | kind of in that same location.

00:22:07.200 | And it was pretty cool

00:22:08.040 | because you could see the cars come by,

00:22:09.760 | you know, kind of finish that part,

00:22:10.960 | like over the trip,

00:22:11.800 | or that like over the mission,

00:22:12.640 | and then, you know, go on and finish the rest of it.

00:22:17.440 | And other vehicles would, you know,

00:22:20.320 | come hit their waypoint and, you know,

00:22:23.600 | exit the oval and off they would go.

00:22:25.720 | Our car in the hand, which hit the checkpoint,

00:22:28.040 | and then it would do an extra lap around the oval

00:22:30.360 | and only then, you know, leave and go on its merry way.

00:22:32.920 | So over the course of, you know, the full day,

00:22:34.440 | it accumulated some extra time.

00:22:36.520 | And the problem was that we had a bug

00:22:37.760 | where it wouldn't, you know,

00:22:39.120 | start reasoning about the next waypoint

00:22:40.920 | and plan a route to get to that next point

00:22:42.640 | until it hit a previous one.

00:22:43.840 | And in that particular case,

00:22:44.960 | by the time you hit that one,

00:22:46.880 | it was too late for us to consider the next one

00:22:49.240 | and kind of make a lane change.

00:22:50.240 | So every time it would do like an extra lap.

00:22:52.080 | So, you know, that's the Stanford victory lap.

00:22:56.160 | - The victory lap.

00:22:57.000 | (laughing)

00:22:58.040 | Oh, that's, I feel like there's something

00:23:00.040 | philosophically profound in there somehow,

00:23:01.840 | but I mean, ultimately,

00:23:04.000 | everybody is a winner in that kind of competition.

00:23:07.240 | And it led to sort of famously to the creation

00:23:11.880 | of Google self-driving car project.

00:23:14.920 | And now Waymo.

00:23:16.720 | So can we give an overview of how is Waymo born?

00:23:21.280 | How's the Google self-driving car project born?

00:23:24.200 | What is the mission?

00:23:25.760 | What is the hope?

00:23:27.320 | What is it is the engineering kind of set of milestones

00:23:32.320 | that it seeks to accomplish?

00:23:35.640 | There's a lot of questions in there.

00:23:37.120 | - Yeah.

00:23:37.960 | (laughing)

00:23:38.800 | I mean, you're right.

00:23:39.640 | Kind of the DARPA Urban Challenge

00:23:40.760 | and the previous DARPA Grand Challenges

00:23:43.080 | kind of led, I think, to a very large degree

00:23:46.040 | to that next step.

00:23:46.880 | And then Larry and Sergey, Larry Page and Sergey Brin,

00:23:51.040 | Google Founders Group, saw that competition

00:23:53.960 | and believed in the technology.

00:23:56.040 | So the Google self-driving car project was born.

00:23:59.240 | You know, at that time, and we started in 2009,

00:24:03.120 | it was a pretty small group of us,

00:24:04.600 | about a dozen people who came together

00:24:07.640 | to work on this project at Google.

00:24:10.760 | At that time, we saw that incredible early result

00:24:15.760 | in the DARPA Urban Challenge.

00:24:19.280 | I think we're all incredibly excited about where we got to.

00:24:24.200 | And we believed in the future of the technology,

00:24:26.360 | but we still had a very rudimentary understanding

00:24:30.120 | of the problem space.

00:24:31.600 | So the first goal of this project in 2009

00:24:35.680 | was to really better understand what we're up against.

00:24:39.840 | And with that goal in mind, when we started the project,

00:24:43.800 | we created a few milestones for ourselves

00:24:46.120 | that maximized learnings, if you will.

00:24:49.960 | The two milestones were,

00:24:52.160 | one was to drive 100,000 miles in autonomous mode,

00:24:55.880 | which was at that time, orders of magnitude

00:24:57.880 | that more than anybody has ever done.

00:25:01.040 | And the second milestone was to drive 10 routes.

00:25:05.360 | Each one was 100 miles long.

00:25:08.520 | They were specifically chosen to be kind of extra spicy,

00:25:12.400 | extra complicated, and sampled the full complexity

00:25:14.920 | of that domain.

00:25:18.560 | And you had to drive each one from beginning to end

00:25:23.280 | with no intervention, no human intervention.

00:25:25.200 | So you would get to the beginning of the course,

00:25:26.880 | you would press the button that would engage in autonomy,

00:25:30.200 | and you had to go for 100 miles,

00:25:33.200 | beginning to end with no interventions.

00:25:36.120 | And it sampled, again,

00:25:38.960 | the full complexity of driving conditions.

00:25:40.800 | Some were on freeways.

00:25:43.280 | We had one route that went all through all the freeways

00:25:45.240 | and all the bridges in the Bay Area.

00:25:47.160 | You know, we had some that went around Lake Tahoe

00:25:50.040 | and kind of mountains roads.

00:25:52.440 | We had some that drove through dense urban environments

00:25:56.360 | like in downtown Palo Alto and through San Francisco.

00:25:59.840 | So it was incredibly interesting to work on.

00:26:05.280 | And it took us just under two years,

00:26:10.280 | about a year and a half, a little bit more,

00:26:12.160 | to finish both of these milestones.

00:26:14.560 | And in that process,

00:26:16.240 | A, it was an incredible amount of fun,

00:26:20.440 | probably the most fun I had in my professional career.

00:26:23.120 | And you're just learning so much.

00:26:25.720 | The goal here is to learn and prototype.

00:26:27.160 | You're not yet starting to build a production system.

00:26:29.520 | So you just, you were,

00:26:31.280 | this is when you're kind of working 24/7

00:26:33.720 | and you're hacking things together.

00:26:35.080 | And you also don't know how hard this is.

00:26:37.960 | I mean, that's the point.

00:26:39.200 | Like, so, I mean, that's an ambitious,

00:26:41.440 | if I put myself in that mindset, even still,

00:26:44.480 | that's a really ambitious set of goals.

00:26:47.120 | Like just those two, just picking,

00:26:49.600 | just picking 10 different difficult, spicy challenges

00:26:54.600 | and then having zero interventions.

00:26:59.960 | So like not saying gradually we're going to like,

00:27:04.080 | you know, over a period of 10 years,

00:27:06.880 | we're going to have a bunch of roots

00:27:08.440 | and gradually reduce the number of interventions.

00:27:11.000 | You know, that literally says like,

00:27:12.600 | by as soon as possible, we want to have zero

00:27:16.600 | and on hard roads.

00:27:18.600 | So like, to me, if I was facing that,

00:27:20.840 | it's unclear that whether that takes two years

00:27:24.120 | or whether that takes 20 years.

00:27:25.760 | I mean, it may be-- - It took us under two.

00:27:28.160 | - And I guess that speaks to a really big difference

00:27:31.200 | between doing something once and having a prototype

00:27:35.520 | where you're going after, you know,

00:27:37.800 | learning about the problem

00:27:39.320 | versus how you go about engineering a product that,

00:27:43.400 | where you look at, you know,

00:27:45.800 | do you properly do evaluation?

00:27:47.280 | You look at metrics, you know, drive down,

00:27:49.080 | and you're confident that you can do that.

00:27:50.720 | And I guess that's the, you know,

00:27:51.560 | why it took a dozen people, you know,

00:27:55.840 | 16 months or a little bit more than that,

00:27:58.280 | back in 2009 and 2010,

00:28:00.880 | and with the technology of, you know,

00:28:02.960 | more than a decade ago,

00:28:04.640 | that amount of time to achieve that milestone

00:28:08.120 | of 10 routes, 100 miles each and no interventions.

00:28:12.360 | And, you know, it took us a little bit longer

00:28:16.880 | to get to, you know, a full driverless product

00:28:19.680 | that customers use.

00:28:20.960 | - That's another really important moment.

00:28:22.320 | Is there some memories of technical lessons

00:28:27.720 | or just one, like, what did you learn

00:28:30.400 | about the problem of driving from that experience?

00:28:33.480 | I mean, we can now talk about like what you learned

00:28:36.080 | from modern day Waymo,

00:28:38.720 | but I feel like you may have learned some profound things

00:28:42.160 | in those early days, even more so,

00:28:45.760 | because it feels like what Waymo is now

00:28:48.080 | is the trying to, you know, how to do scale,

00:28:50.680 | how to make sure you create a product,

00:28:52.160 | how to make sure it's like safety and all those things,

00:28:54.240 | which is all fascinating challenges.

00:28:56.600 | But like you were facing the more fundamental,

00:28:59.840 | philosophical problem of driving in those early days.

00:29:03.800 | Like what the hell is driving as an autonomous vehicle?

00:29:08.200 | Maybe I'm again, romanticizing it,

00:29:10.000 | but is there some valuable lessons

00:29:15.000 | you picked up over there at those two years?

00:29:18.200 | - A ton.

00:29:20.400 | The most important one is probably

00:29:23.280 | that we believe that it's doable

00:29:26.240 | and we've gotten far enough into the problem that,

00:29:30.760 | you know, we had a, I think only a glimpse

00:29:33.280 | of the true complexity of the domain.

00:29:38.080 | You know, it's a little bit like, you know,

00:29:39.240 | climbing a mountain where you kind of, you know,

00:29:40.680 | see the next peak and you think that's kind of the summit,

00:29:43.120 | but then you get to that and you kind of see

00:29:44.400 | that this is just the start of the journey.

00:29:47.560 | But we've tried, we've sampled enough of the problem space

00:29:50.960 | and we've made enough rapid success,

00:29:54.200 | even, you know, with technology of 2009, 2010,

00:29:57.240 | that it gave us confidence to then, you know,

00:30:00.840 | pursue this as a real product.

00:30:04.440 | - So, okay.

00:30:05.640 | So the next step, you mentioned the milestones

00:30:09.080 | that you had in those two years.

00:30:11.560 | What are the next milestones that then led

00:30:13.920 | to the creation of Waymo and beyond?

00:30:16.160 | - Yeah, it was a really interesting journey.

00:30:18.080 | And, you know, Waymo came a little bit later.

00:30:22.280 | Then, you know, we completed those milestones in 2010.

00:30:26.440 | That was the pivot when we decided to focus

00:30:30.000 | on actually building a product, you know,

00:30:31.920 | using this technology.

00:30:33.040 | The initial couple of years after that,

00:30:36.400 | we were focused on a freeway, you know,

00:30:39.280 | what you would call a driver assist,

00:30:41.200 | maybe, you know, an L3 driver assist program.

00:30:44.600 | Then around 2013, we've learned enough about the space

00:30:49.600 | and have thought more deeply about, you know,

00:30:52.880 | the product that we wanted to build that we pivoted.

00:30:56.720 | We pivoted towards this vision of, you know,

00:31:00.880 | building a driver and deploying it fully driverless vehicles

00:31:04.280 | without a person.

00:31:05.120 | And that's the path that we've been on since then.

00:31:07.240 | And it was exactly the right decision for us.

00:31:10.680 | - So there was a moment where you also considered like,

00:31:13.200 | what is the right trajectory here?

00:31:15.040 | What is the right role of automation

00:31:16.840 | in the task of driving?

00:31:18.360 | Or is still, it wasn't from the early days,

00:31:21.920 | obviously you want to go fully autonomous.

00:31:24.240 | - From the early days, it was not.

00:31:25.240 | I think it was in 20, around 2013, maybe,

00:31:27.520 | that we've, that became very clear and we made that pivot.

00:31:32.640 | And it also became very clear

00:31:34.760 | and that it's, even the way you go building

00:31:37.840 | a driver assist system is, you know,

00:31:40.760 | fundamentally different from how you go building

00:31:42.600 | a fully driverless vehicle.

00:31:43.600 | So, you know, we've pivoted towards the latter

00:31:47.800 | and that's what we've been working on ever since.

00:31:50.720 | And so that was around 2013.

00:31:53.120 | Then there's a sequence of really meaningful for us,

00:31:58.040 | really important, defining milestones since then.

00:32:01.080 | In 2015, we had our first,

00:32:06.080 | actually the world's first fully driverless ride

00:32:13.840 | on public roads.

00:32:15.560 | It was in a custom built vehicle that we had.

00:32:18.080 | I must've seen those.

00:32:18.960 | We called them the Firefly, that, you know,

00:32:20.560 | funny looking, marshmallow looking thing.

00:32:23.240 | And we put a passenger, his name was Steve Mann,

00:32:28.240 | a great friend of our project from the early days.

00:32:32.560 | The man happens to be blind.

00:32:35.000 | So we put him in that vehicle.

00:32:36.520 | The car had no steering wheel, no pedals.

00:32:38.600 | It was an uncontrolled environment, you know,

00:32:41.200 | no lead or chase cars, no police escorts.

00:32:44.960 | And, you know, we did that trip a few times in Austin, Texas.

00:32:48.360 | So that was a really big milestone.

00:32:50.000 | - That was in Austin.

00:32:51.080 | - Yeah. - Cool, okay.

00:32:52.200 | - And, you know, we only, but at that time we're only,

00:32:55.760 | it took a tremendous amount of engineering.

00:32:57.760 | It took a tremendous amount of validation

00:32:59.720 | to get to that point.

00:33:01.440 | But, you know, we only did it a few times.

00:33:04.200 | Maybe we only did that.

00:33:05.040 | It was a fixed route.

00:33:05.920 | It was not kind of a controlled environment,

00:33:07.280 | but it was a fixed route and we only did a few times.

00:33:10.520 | Then in 2016, end of 2016, beginning of 2017,

00:33:15.520 | is when we founded Waymo, the company.

00:33:20.640 | That's when we, kind of, that was the next phase

00:33:24.000 | of the project where I wanted,

00:33:26.720 | we believed in kind of the commercial vision

00:33:29.560 | of this technology.

00:33:30.880 | And it made sense to create an independent entity,

00:33:33.400 | you know, within that alphabet umbrella

00:33:35.840 | to pursue this product at scale.

00:33:39.800 | Beyond that in 2017, later in 2017,

00:33:43.160 | was another really huge step for us,

00:33:46.960 | really big milestone where we started,

00:33:49.480 | I think it was October of 2017,

00:33:52.040 | where when we started regular driverless operations

00:33:57.040 | on public roads, that first day of operations,

00:34:00.960 | we drove in one day, in that first day,

00:34:03.920 | a hundred miles in driverless fashion.

00:34:06.440 | And then we've, the most important thing

00:34:08.280 | about that milestone was not that, you know,

00:34:09.880 | a hundred miles in one day,

00:34:11.080 | but that it was the start of kind of regular,

00:34:13.800 | ongoing driverless operations.

00:34:15.480 | - And when you say driverless, it means no driver.

00:34:18.760 | - That's exactly right.

00:34:20.480 | So on that first day, we actually had a mix

00:34:22.280 | and we didn't want to like, you know,

00:34:26.040 | be on YouTube and Twitter that same day.

00:34:27.720 | So in many of the rides,

00:34:31.000 | we had somebody in the driver's seat,

00:34:33.000 | but they could not disengage, like the car,

00:34:34.920 | - I gotcha. - not disengaged.

00:34:36.480 | But actually on that first day,

00:34:38.440 | some of the miles were driven

00:34:40.720 | and just completely empty driver's seat.

00:34:43.240 | - And this is the key distinction

00:34:45.280 | that I think people don't realize,

00:34:47.160 | you know, that oftentimes when you talk

00:34:49.840 | about autonomous vehicles,

00:34:51.440 | there's often a driver in the seat

00:34:54.960 | that's ready to take over,

00:34:58.600 | what's called a safety driver.

00:35:01.040 | And then Waymo is really one of the only companies,

00:35:05.480 | at least that I'm aware of,

00:35:06.480 | or at least as like boldly and carefully

00:35:09.720 | and all that is actually has cases.

00:35:13.160 | And now we'll talk about more and more

00:35:15.320 | where there's literally no driver.

00:35:17.800 | So that's another, the interesting case

00:35:20.880 | of where the driver's not supposed to disengage.

00:35:22.680 | That's like a nice middle ground.

00:35:24.480 | They're still there,

00:35:25.320 | but they're not supposed to disengage.

00:35:27.080 | But really there's the case when there's no,

00:35:30.440 | okay, there's something magical

00:35:32.400 | about there being nobody in the driver's seat.

00:35:35.280 | Like, just like to me,

00:35:38.520 | you mentioned the first time you wrote some code

00:35:42.840 | for free space navigation of the parking lot,

00:35:45.360 | that was like a magical moment.

00:35:47.200 | To me, just sort of as an observer of robots,

00:35:52.000 | the first magical moment is seeing

00:35:55.800 | an autonomous vehicle turn,

00:35:57.840 | like make a left turn,

00:35:59.800 | like apply sufficient torque to the steering wheel

00:36:04.800 | to where like there's a lot of rotation.

00:36:07.240 | And for some reason,

00:36:08.320 | and there's nobody in the driver's seat,

00:36:10.720 | for some reason that communicates

00:36:14.080 | that here's a being with power that makes a decision.

00:36:18.200 | There's something about like the steering wheel,

00:36:20.080 | 'cause we perhaps romanticize the notion

00:36:22.240 | of the steering wheel.

00:36:23.160 | It's so essential to our conception,

00:36:25.760 | our 20th century conception of a car.

00:36:28.120 | And it turning the steering wheel

00:36:30.000 | with nobody in driver's seat,

00:36:32.160 | that to me, I think maybe to others,

00:36:35.320 | it's really powerful.

00:36:36.640 | Like this thing is in control.

00:36:38.600 | And then there's this leap of trust that you give,

00:36:41.280 | like I'm gonna put my life

00:36:42.760 | in the hands of this thing that's in control.

00:36:44.720 | So in that sense,

00:36:45.800 | when there's no driver in the driver's seat,

00:36:49.160 | that's a magical moment for robots.

00:36:51.840 | So I got a chance to last year to take a ride

00:36:55.840 | in a Waymo vehicle.

00:36:57.920 | And that was the magical moment.

00:36:59.480 | There's like nobody in the driver's seat.

00:37:01.800 | It's like the little details.

00:37:04.680 | You would think it doesn't matter

00:37:05.920 | whether there's a driver or not,

00:37:07.640 | but like if there's no driver

00:37:09.720 | and the steering wheel is turning on its own,

00:37:12.640 | I don't know, that's magical.

00:37:15.320 | - It's absolutely magical.

00:37:16.600 | I have taken many of these rides

00:37:18.600 | in a completely empty car.

00:37:20.320 | No human in the car pulls up.

00:37:23.120 | You call it on your cell phone, it pulls up.

00:37:25.000 | You get in, it takes you on its way.

00:37:27.200 | There's nobody in the car but you, right?

00:37:30.600 | That's something called fully driverless,

00:37:33.080 | our rider only mode of operation.

00:37:36.200 | Yeah, it is magical.

00:37:40.840 | It is transformative.

00:37:43.000 | This is what we hear from our riders.

00:37:45.840 | It kind of really changes your experience.

00:37:47.720 | And that really is what unlocks

00:37:50.760 | the real potential of this technology.

00:37:52.760 | But coming back to our journey,

00:37:56.200 | that was 2017 when we started

00:37:58.120 | truly driverless operations.

00:38:00.920 | Then in 2018, we've launched

00:38:04.520 | our public commercial service

00:38:07.440 | that we called Waymo One in Phoenix.

00:38:10.080 | In 2019, we started offering

00:38:14.960 | truly driverless rider only rides

00:38:17.400 | to our early rider population of users.

00:38:22.000 | And then 2020 has also been

00:38:25.400 | a pretty interesting year.

00:38:27.560 | One of the first ones, less about technology,

00:38:29.400 | but more about the maturing

00:38:30.800 | and the growth of Waymo as a company.

00:38:34.120 | We raised our first round of external financing this year.

00:38:39.120 | We were part of Alphabet,

00:38:40.440 | so obviously we have access to significant resources.

00:38:44.560 | But as kind of on the journey of Waymo maturing as a company,

00:38:47.640 | it made sense for us to partially go externally

00:38:51.080 | in this round.

00:38:52.200 | So we raised about $3.2 billion from that round.

00:38:57.200 | We've also started putting our fifth generation

00:39:03.160 | of our driver, our hardware,

00:39:05.440 | that is on the new vehicle,

00:39:07.520 | but it's also a qualitatively different set

00:39:10.440 | of self-driving hardware

00:39:12.360 | that is now on the JLR pace.

00:39:16.480 | So that was a very important step for us.

00:39:19.320 | - The hardware specs, fifth generation,

00:39:22.680 | I think it'd be fun to maybe,

00:39:25.040 | I apologize if I'm interrupting,

00:39:26.600 | but maybe talk about maybe the generations

00:39:31.160 | with a focus on what we're talking about

00:39:33.440 | on the fifth generation in terms of hardware specs,

00:39:35.880 | like what's on this car?

00:39:37.520 | - Sure.

00:39:38.360 | So we separated out the actual car

00:39:40.640 | that we are driving from the self-driving hardware

00:39:43.120 | we put on it.

00:39:44.680 | Right now we have,

00:39:45.880 | so this is, as I mentioned, the fifth generation.

00:39:47.800 | We've gone through,

00:39:49.880 | we started building our own hardware many, many years ago.

00:39:54.880 | And that Firefly vehicle also had the hardware suite

00:40:00.080 | that was mostly designed, engineered, and built in-house.

00:40:04.320 | Lighters are one of the more important components

00:40:08.800 | that we design and build from the ground up.

00:40:11.680 | So on the fifth generation of our drivers,

00:40:16.560 | of our self-driving hardware

00:40:18.080 | that we're switching to right now,

00:40:20.920 | we have, as with previous generations,

00:40:24.160 | in terms of sensing, we have lighters, cameras, and radars.

00:40:27.680 | And we have a pretty beefy computer

00:40:30.240 | that processes all that information

00:40:31.720 | and makes decisions in real time on board the car.

00:40:35.640 | So in all of the,

00:40:37.760 | and it's really a qualitative jump forward

00:40:41.800 | in terms of the capabilities and the various parameters

00:40:45.760 | and the specs of the hardware

00:40:47.000 | compared to what we had before

00:40:48.240 | and compared to what you can kind of get off the shelf

00:40:51.280 | in the market today.

00:40:52.280 | - Meaning from fifth to fourth or from fifth to first?

00:40:55.360 | - Definitely from first to fifth, but also from the fourth.

00:40:57.920 | - That was the world's dumbest question.

00:40:59.440 | - Definitely, definitely from fourth to fifth.

00:41:02.200 | As well as this, the last step is a big step forward.

00:41:07.200 | - So everything's in-house.

00:41:09.160 | So like LIDAR is built in-house

00:41:11.560 | and cameras are built in-house?

00:41:15.680 | - It's different.

00:41:17.200 | We work with partners.

00:41:18.080 | There's some components that we get from our manufacturing

00:41:23.080 | and supply chain partners.

00:41:24.760 | What exactly is in-house is a bit different.

00:41:28.880 | We do a lot of custom design on all of our sensing models.

00:41:33.880 | There's lighters, radars, cameras.

00:41:36.800 | Exactly, there's, lighters are almost exclusively in-house

00:41:41.800 | and some of the technologies that we have,

00:41:44.480 | some of the fundamental technologies there

00:41:46.080 | are completely unique to Waymo.

00:41:49.200 | That is also largely true about radars and cameras.

00:41:52.120 | It's a little bit more of a mix

00:41:54.400 | in terms of what we do ourselves

00:41:55.720 | versus what we get from partners.

00:41:58.080 | - Is there something super sexy about the computer

00:42:00.960 | that you can mention that's not top secret?

00:42:03.520 | Like for people who enjoy computers for, I mean,

00:42:08.520 | see, there's a lot of machine learning involved,

00:42:12.280 | but there's a lot of just basic compute.

00:42:13.880 | There's, you have to probably do a lot of signal processing

00:42:17.880 | on all the different sensors.

00:42:19.960 | You have to integrate everything.

00:42:20.800 | It has to be in real time.

00:42:22.040 | There's probably some kind of redundancy type of situation.

00:42:25.560 | Is there something interesting you could say

00:42:27.400 | about the computer for the people who love hardware?

00:42:31.200 | - It does have all of the characteristics,

00:42:32.720 | all the properties that you just mentioned.

00:42:34.560 | Redundancy, very beefy compute for general processing

00:42:39.560 | as well as inference and ML models.

00:42:43.360 | It is some of the more sensitive stuff

00:42:45.040 | that I don't wanna get into for IP reasons,

00:42:47.000 | but yeah, it can, we've shared a little bit

00:42:50.760 | in terms of the specs of the sensors

00:42:54.160 | that we have on the car.

00:42:55.480 | We've actually shared some videos

00:42:57.280 | of what our lidars see in the world.

00:43:02.280 | We have 29 cameras, we have five lidars,

00:43:05.280 | we have six radars on these vehicles,

00:43:08.000 | and you can kind of get a feel for the amount of data

00:43:11.040 | that they're producing.

00:43:11.880 | That all has to be processed in real time

00:43:14.440 | to do perception, to do complex reasoning.

00:43:17.640 | So it kind of gives you some idea

00:43:18.640 | of how beefy those computers are,

00:43:20.080 | but I don't wanna get into specifics

00:43:21.680 | of exactly how we build them.

00:43:23.240 | - Okay, well, let me try some more questions

00:43:24.960 | that you can't get into the specifics of,

00:43:26.760 | like GPU wise, is that something you can get into?

00:43:29.960 | I know that Google works with GPUs and so on.

00:43:32.680 | I mean, for machine learning folks,

00:43:34.520 | it's kind of interesting, or is there no,

00:43:36.560 | how do I ask it?

00:43:40.680 | I've been talking to people in the government about UFOs

00:43:44.120 | and they don't answer any questions.

00:43:45.640 | So this is how I feel right now asking about GPUs.

00:43:48.560 | (laughs)

00:43:50.520 | But is there something interesting that you could reveal

00:43:53.840 | or is it just, you know,

00:43:55.140 | or leave it up to our imagination, some of the compute?

00:43:59.680 | Is there any, I guess, is there any fun trickery?

00:44:02.880 | Like I talked to Chris Latner for a second time

00:44:05.800 | and he was a key person about TPUs

00:44:08.240 | and there's a lot of fun stuff going on in Google

00:44:11.320 | in terms of hardware that optimizes for machine learning.

00:44:16.160 | Is there something you can reveal

00:44:18.080 | in terms of how much, you mentioned customization,

00:44:20.840 | how much customization there is for hardware

00:44:23.840 | for machine learning purposes?

00:44:25.640 | - I'm gonna be like that government, you know,

00:44:27.000 | you've got a guy, a person who bought UFOs.

00:44:29.920 | But I, you know, I guess I will say that it's really,

00:44:34.920 | compute is really important.

00:44:38.400 | We have very data hungry and compute hungry ML models

00:44:43.400 | all over our stack.

00:44:45.000 | And this is where, you know,

00:44:46.960 | both being part of Alphabet as well as designing

00:44:51.320 | our own sensors and the entire hardware suite together

00:44:54.440 | where on one hand you get access to like really rich,

00:44:59.440 | raw sensor data that you can pipe from your sensors

00:45:02.840 | into your compute platform.

00:45:06.400 | Yeah, and build like build a whole pipe

00:45:09.120 | from sensor, raw sensor data to the big compute

00:45:11.560 | as then have the massive compute to process all that data.

00:45:15.160 | This is where we're finding that having a lot of control

00:45:18.080 | of that hardware part of the stack is really advantageous.

00:45:22.560 | - One of the fascinating magical places to me,

00:45:25.440 | again, might not be able to speak to the details,

00:45:28.360 | but it is the other compute, which is like, you know,

00:45:32.080 | this we're just talking about a single car,

00:45:34.360 | but the, you know, the driving experience

00:45:38.640 | is a source of a lot of fascinating data.

00:45:40.760 | And you have a huge amount of data coming in on the car

00:45:44.400 | and, you know, the infrastructure of storing

00:45:47.760 | some of that data to then train or to analyze or so on.

00:45:52.280 | That's a fascinating like piece of it

00:45:55.800 | that I understand a single car,

00:45:58.320 | I don't understand how you pull it all together

00:46:00.160 | in a nice way.

00:46:01.120 | Is that something that you could speak to

00:46:02.720 | in terms of the challenges of seeing the network of cars

00:46:07.720 | and then bringing the data back and analyzing things

00:46:10.920 | that like edge cases of driving,

00:46:13.960 | be able to learn on them to improve the system,

00:46:15.880 | to see where things went wrong, where things went right

00:46:20.240 | and analyze all that kind of stuff.

00:46:21.720 | Is there something interesting there

00:46:24.000 | from an engineering perspective?

00:46:26.120 | - Oh, there's an incredible amount

00:46:30.360 | of really interesting work that's happening there,

00:46:32.600 | both in the real time operation of the fleet of cars

00:46:36.480 | and the information that they exchange with each other

00:46:39.440 | in real time to make better decisions,

00:46:41.960 | as well as on the kind of the off-board component

00:46:45.720 | where you have to deal with massive amounts of data

00:46:48.520 | for training your ML models, evaluating the ML models,

00:46:52.600 | for simulating the entire system

00:46:55.440 | and for evaluating your entire system.

00:46:57.800 | And this is where being part of Alphabet

00:47:00.720 | has once again been tremendously advantageous.

00:47:04.280 | I think we consume an incredible amount of compute

00:47:07.080 | for ML infrastructure.

00:47:09.280 | We build a lot of custom frameworks to get good

00:47:11.840 | on data mining, finding the interesting edge cases

00:47:17.640 | for training and for evaluation of the system

00:47:20.400 | for both training and evaluating sub-components

00:47:24.040 | and sub parts of the system and various ML models,

00:47:27.080 | as well as evaluating the entire system and simulation.

00:47:31.200 | - Okay, is that first piece that you mentioned

00:47:32.880 | that cars communicating to each other, essentially,

00:47:36.240 | I mean, through perhaps through a centralized point,

00:47:38.800 | but what, that's fascinating too.

00:47:41.640 | How much does that help you?

00:47:43.080 | Like, if you imagine, you know, right now,

00:47:45.520 | the number of Waymo vehicles is whatever, X.

00:47:48.800 | I don't know if you can talk to what that number is,

00:47:51.160 | but it's not in the hundreds of millions yet.

00:47:54.240 | And imagine if the whole world is Waymo vehicles,

00:47:57.840 | like that changes potentially the power of connectivity.

00:48:03.280 | Like the more cars you have, I guess, actually,

00:48:05.840 | if you look at Phoenix, 'cause there's enough vehicles,

00:48:08.680 | there's enough, when there's like some level of density,

00:48:12.520 | you can start to probably do some really interesting stuff

00:48:14.800 | with the fact that cars can negotiate,

00:48:17.800 | can be, can communicate with each other

00:48:21.760 | and thereby make decisions.

00:48:23.560 | Is there something interesting there that you can talk to

00:48:27.240 | about like, how does that help with the driving problem

00:48:29.640 | from as compared to just a single car

00:48:32.280 | solving the driving problem by itself?

00:48:34.560 | - Yeah, it's a spectrum.

00:48:37.280 | I, first I'll say that, you know, it helps

00:48:41.800 | and it helps in various ways, but it's not required.

00:48:44.880 | Right now, the way we build our system,

00:48:46.400 | like each cars can operate independently,

00:48:48.080 | they can operate with no connectivity.

00:48:50.280 | So I think it is important that, you know,

00:48:52.280 | you have a fully autonomous, you know,

00:48:55.120 | fully capable driver that, you know,

00:48:59.760 | computerized driver that each car has.

00:49:02.480 | Then, you know, they do share information

00:49:05.200 | and they share information in real time.

00:49:06.640 | It really, really helps.

00:49:07.720 | All right, so the way we do this today is, you know,

00:49:12.440 | whenever one car encounters something interesting

00:49:15.640 | in the world, whether it might be an accident

00:49:17.640 | or a new construction zone,

00:49:19.040 | that information immediately gets, you know,

00:49:21.800 | uploaded over the air and it's propagated

00:49:23.960 | to the rest of the fleet.

00:49:25.120 | So, and that's kind of how we think about maps

00:49:27.200 | as priors in terms of the knowledge of our drivers,

00:49:32.200 | of our fleet of drivers that is distributed

00:49:37.040 | across the fleet and it's updated in real time.

00:49:40.000 | So that's one use case.

00:49:42.200 | You know, you can imagine as the, you know,

00:49:45.000 | the density of these vehicles go up

00:49:48.120 | that they can exchange more information

00:49:50.160 | in terms of what they're planning to do

00:49:52.360 | and start influencing how they interact with each other,

00:49:56.120 | as well as, you know,

00:49:56.960 | potentially sharing some observations, right?

00:49:59.600 | To help with, you know,

00:50:00.680 | if you have enough density of these vehicles where,

00:50:02.640 | you know, one car might be seeing something

00:50:04.200 | that another is relevant to another car

00:50:06.440 | that is very dynamic.

00:50:07.440 | You know, it's not part of kind of your updating

00:50:09.080 | your static prior of the map of the world,

00:50:11.040 | but it's more of a dynamic information

00:50:12.440 | that could be relevant to the decisions

00:50:14.360 | that another car is making real time.

00:50:15.720 | So you can see them exchanging that information

00:50:17.840 | and you can build on that.

00:50:18.840 | But again, I see that as an advantage,

00:50:22.080 | but it's, you know, not a requirement.

00:50:25.120 | - So what about the human in the loop?

00:50:27.520 | So when I got a chance to drive with a ride in a Waymo,

00:50:32.520 | you know, there's customer service.

00:50:36.440 | So like there is somebody that's able to dynamically

00:50:42.720 | like tune in and help you out.

00:50:46.960 | What role does the human play in that picture?

00:50:50.720 | That's a fascinating, like, you know,

00:50:52.400 | the idea of teleoperation,

00:50:53.720 | be able to remotely control a vehicle.

00:50:56.320 | So here, what we're talking about is like,

00:50:58.440 | like frictionless, like a human being able to,

00:51:04.560 | in a frictionless way, sort of help you out.

00:51:08.520 | I don't know if they're able to actually control the vehicle.

00:51:10.960 | Is that something you could talk to?

00:51:12.680 | - Yes. - Okay.

00:51:13.560 | - To be clear, we don't do teleoperation.

00:51:15.920 | I got to believe in teleoperation for a reason

00:51:18.120 | is that's not what we have in our cars.

00:51:20.760 | We do, as you mentioned, have, you know,

00:51:22.400 | version of customer support, you know,

00:51:24.560 | we call it life health.

00:51:25.400 | In fact, we find it that it's very important

00:51:28.600 | for our rider experience, especially if it's your first trip,

00:51:32.320 | you've never been in a fully driverless ride

00:51:34.360 | or only Waymo vehicle, you get in, there's nobody there.

00:51:37.200 | And so you can imagine having all kinds of questions

00:51:40.120 | in your head, like how this thing works.

00:51:42.040 | So we've put a lot of thought into kind of guiding

00:51:44.080 | our riders, our customers through that experience,

00:51:48.040 | especially for the first time,

00:51:49.240 | they get some information on the phone.

00:51:51.480 | If the fully driverless vehicle is used

00:51:54.840 | to service their trip, when you get into the car,

00:51:58.000 | we have an in-car screen and audio that kind of guides them

00:52:01.680 | and explains what to expect.

00:52:04.520 | They also have a button that they can push

00:52:06.960 | that will connect them to, you know,

00:52:09.600 | a real life human being that they can talk to, right,

00:52:12.800 | about this whole process.

00:52:14.200 | So that's one aspect of it.

00:52:15.800 | There is, you know, I should mention that there is

00:52:19.040 | another function that humans provide to our cars,

00:52:23.320 | but it's not teleoperation.

00:52:24.640 | You can think of it a little bit more like, you know,

00:52:26.400 | fleet assistance, kind of like, you know,

00:52:28.000 | traffic control that you have, where our cars,

00:52:32.840 | again, they're responsible on their own

00:52:34.960 | for making all of the decisions,

00:52:36.960 | all of the driving decisions

00:52:37.920 | that don't require connectivity.

00:52:39.560 | They, you know, anything that is safety or latency critical

00:52:43.520 | is done, you know, purely autonomously by onboard,

00:52:46.800 | our onboard system.

00:52:49.200 | But there are situations where, you know,

00:52:50.720 | if connectivity is available,

00:52:52.560 | and a car encounters a particularly challenging situation,

00:52:54.600 | you can imagine like a super hairy scene of an accident,

00:52:58.320 | the cars will do their best.

00:52:59.920 | They will recognize that it's an off-nominal situation.

00:53:02.400 | They will, you know, do their best to come up, you know,

00:53:06.240 | with the right interpretation,

00:53:07.200 | the best course of action in that scenario.

00:53:08.840 | But if connectivity is available,

00:53:10.160 | they can ask for confirmation from, you know,

00:53:12.720 | a human-mode human assistant

00:53:16.440 | to kind of confirm those actions,

00:53:18.320 | and, you know, perhaps provide a little bit

00:53:20.720 | of kind of contextual information and guidance.

00:53:23.040 | - So October 8th was when you're talking about the,

00:53:26.560 | was Waymo launched the fully self,

00:53:31.560 | the public version of its fully driverless,

00:53:36.320 | that's the right term, I think, service in Phoenix.

00:53:39.720 | Is that October 8th?

00:53:40.640 | - That's right.

00:53:41.480 | It was the introduction of fully driverless

00:53:43.480 | rider-only vehicles into our public Waymo One service.

00:53:46.680 | - Okay, so that's amazing.

00:53:48.640 | So it's like anybody can get into Waymo in Phoenix?

00:53:52.560 | - That's right.

00:53:53.400 | So we previously had early people

00:53:56.680 | in our early rider program

00:53:58.480 | taking fully driverless rides in Phoenix.

00:54:01.080 | And just this, a little while ago,

00:54:05.280 | we opened on October 8th,

00:54:06.320 | we opened that mode of operation to the public.

00:54:09.280 | So I can download the app and go on the ride.

00:54:12.280 | There is a lot more demand right now for that service

00:54:16.760 | than we have capacity.

00:54:18.240 | So we're kind of managing that,

00:54:20.040 | but that's exactly the way you described it.

00:54:21.600 | - Yeah, well, that's interesting.

00:54:22.520 | So there's more demand than you can handle.

00:54:25.720 | Like what has been the reception so far?

00:54:29.920 | Like what, I mean, okay, so, you know,

00:54:32.360 | that's, this is a product, right?

00:54:36.120 | That's a whole nother discussion of like

00:54:38.080 | how compelling of a product it is.

00:54:40.160 | Great, but it's also like one of the most

00:54:42.160 | kind of transformational technologies of the 21st century.

00:54:45.480 | So it's also like a tourist attraction.

00:54:48.440 | Like it's fun to, you know, to be a part of it.

00:54:51.400 | So it'd be interesting to see like, what do people say?

00:54:54.320 | What do people, what have been the feedback so far?

00:54:58.960 | - You know, still early days,

00:54:59.880 | but so far the feedback has been incredible,

00:55:03.840 | incredibly positive.

00:55:05.520 | They, you know, we asked them for feedback during the ride.

00:55:07.760 | We asked them for feedback after the ride

00:55:10.560 | as part of their trip.

00:55:11.440 | You know, we asked them some questions.

00:55:12.520 | We asked them to, you know,

00:55:13.360 | rate the performance of our driver.

00:55:15.840 | Most by far, you know, most of our drivers

00:55:17.960 | give us five stars in our app,

00:55:20.800 | which is absolutely great to see.

00:55:23.800 | And you know, that's, and we're,

00:55:24.840 | they're also giving us feedback on, you know,

00:55:26.240 | things we can improve.

00:55:27.760 | And, you know, that's one of the main reasons

00:55:29.400 | we're doing this with Phoenix.

00:55:30.280 | And, you know, over the last couple of years

00:55:32.240 | and every day today,

00:55:33.960 | we are just learning a tremendous amount of new stuff

00:55:37.240 | from our users.

00:55:38.160 | There's no substitute for actually doing the real thing,

00:55:41.760 | actually having a fully driverless product

00:55:43.760 | out there in the field with, you know,

00:55:45.480 | users that are actually, you know,

00:55:47.640 | paying us money to get from point A to point B.

00:55:49.920 | - So this is a legitimate, like,

00:55:51.960 | this is a paid service.

00:55:53.000 | - That's right.

00:55:53.840 | - And the idea is you use the app to go from point A

00:55:57.600 | to point B, and then what are the A's?

00:56:00.160 | What are the, what's the freedom of the,

00:56:02.600 | of the starting and ending places?

00:56:05.440 | - It's an area of geography where that service is enabled.

00:56:09.040 | It's a decent size of geography of territory.

00:56:12.800 | It's actually larger than, you know,

00:56:14.240 | the size of San Francisco.

00:56:16.400 | And, you know, within that, you have full freedom

00:56:19.720 | of, you know, selecting where you want to go.

00:56:21.840 | You know, of course there are some,

00:56:23.080 | and you, on your app, you get a map,

00:56:25.760 | you tell the car where you want to be picked up,

00:56:29.440 | you know, where you want the car to pull over

00:56:31.640 | and pick you up, and then you tell it

00:56:32.720 | where you want to be dropped off, right?

00:56:34.240 | And of course there are some exclusions, right?

00:56:35.760 | You don't want to be, you know,

00:56:37.080 | where in terms of where the car is allowed to pull over,

00:56:39.920 | right, so, you know, that you can't do,

00:56:41.280 | but, you know, besides that, it's-

00:56:43.120 | - Amazing.

00:56:43.960 | - It's not like a fixed, just would be very,

00:56:45.400 | I guess, I don't know,

00:56:46.240 | maybe that's what's the question behind your question,

00:56:47.640 | but it's not a, you know, preset set of-

00:56:49.840 | - Yes, I guess, so within the geographic constraints

00:56:52.760 | with that, within that area, anywhere else,

00:56:54.720 | it can be picked up and dropped off anywhere.

00:56:57.440 | - That's right, and, you know, people use them

00:56:59.160 | on like all kinds of trips.

00:57:00.960 | They, we have, and we have an incredible spectrum of riders.

00:57:03.720 | I think the youngest, actually have car seats in them,

00:57:05.840 | and we have, you know, people taking their kids on rides.

00:57:07.880 | I think the youngest riders we had on cars

00:57:09.800 | are, you know, one or two years old, you know,

00:57:11.440 | and the full spectrum of use cases.

00:57:12.800 | People, you can take them to, you know, schools,

00:57:16.240 | to, you know, go grocery shopping,

00:57:19.060 | to restaurants, to bars, you know, run errands,

00:57:22.760 | you know, go shopping, et cetera, et cetera.

00:57:24.040 | You can go to your office, right?

00:57:25.800 | Like the full spectrum of use cases,

00:57:27.240 | and people, you're gonna use them in their daily lives

00:57:31.480 | to get around, and we see all kinds of, you know,

00:57:35.240 | really interesting use cases,

00:57:37.160 | and that's what's providing us

00:57:38.920 | incredibly valuable experience

00:57:41.880 | that we then, you know, use to improve our product.

00:57:44.520 | - So as somebody who's been on, done a few long rants

00:57:49.520 | with Joe Rogan and others about the toxicity

00:57:53.160 | of the internet and the comments,

00:57:55.040 | and the negativity in the comments,

00:57:56.800 | I'm fascinated by feedback.

00:57:58.080 | I believe that most people are good and kind and intelligent

00:58:03.080 | and can provide, like, even in disagreement,

00:58:08.280 | really fascinating ideas.

00:58:09.520 | So on a product side, it's fascinating to me,

00:58:12.600 | like, how do you get the richest possible user feedback,

00:58:16.960 | like, to improve?

00:58:18.680 | What are the channels that you use

00:58:22.280 | to measure, 'cause like, you're no longer,

00:58:25.020 | that's one of the magical things about autonomous vehicles,

00:58:29.160 | is it's not, like, it's frictionless interaction

00:58:32.280 | with the human, so like, you don't get to,

00:58:35.120 | you know, it's just giving a ride.

00:58:36.920 | So like, how do you get feedback from people

00:58:39.280 | in order to improve?

00:58:40.760 | - Oh, yeah, great question.

00:58:42.160 | Various mechanisms.

00:58:43.800 | So as part of the normal flow, we ask people for feedback.

00:58:47.160 | They, as the car is driving around,

00:58:49.040 | you know, we have on the phone and in the car,

00:58:51.120 | and we have a touchscreen in the car,

00:58:53.280 | you can actually click some buttons

00:58:54.480 | and provide real-time feedback on how the car is doing

00:58:58.560 | and how the car is handling a particular situation,

00:59:00.640 | you know, both positive and negative.

00:59:02.160 | So that's one channel.

00:59:03.680 | We have, as we discussed, customer support or life help,

00:59:06.640 | where, you know, if a customer wants to,

00:59:08.800 | has a question or he has some sort of concern,

00:59:12.200 | they can talk to a person in real time.

00:59:14.960 | So that is another mechanism that gives us feedback.

00:59:19.080 | At the end of a trip, you know,

00:59:20.680 | we also ask them how things went.

00:59:22.760 | They give us comments and, you know, a star rating.

00:59:25.840 | And, you know, if it's, we also, you know,

00:59:28.280 | ask them to explain what, you know,

00:59:31.800 | went well and, you know, what could be improved.

00:59:34.160 | And we have, our riders are providing,

00:59:39.000 | you know, very rich feedback there.

00:59:41.520 | A lot, a large fraction is very passionate

00:59:44.480 | and very excited about this technology.

00:59:45.720 | So we get really good feedback.

00:59:47.320 | We also run UXR studies, right?

00:59:49.640 | You know, specific that are kind of more, you know,

00:59:53.000 | go more in depth and we will run both kind of lateral

00:59:55.760 | and longitudinal studies where we have deeper engagement

01:00:00.440 | with our customers.

01:00:01.440 | You know, we have our user experience research team

01:00:04.360 | tracking over time.

01:00:05.680 | That's when you say about longitudinal, it's cool.

01:00:07.600 | - That's exactly right.

01:00:08.720 | And, you know, that's another really valuable feedback,

01:00:12.000 | source of feedback.

01:00:12.840 | And we're just covering a tremendous amount, right?

01:00:16.440 | People go grocery shopping and they like want to load,

01:00:19.480 | you know, 20 bags of groceries in our cars.

01:00:21.480 | And like that's one workflow that you maybe don't,

01:00:24.040 | you know, think about, you know, getting just right

01:00:27.680 | when you're building the driverless product.

01:00:30.280 | I have people like, you know, who bike as part

01:00:35.000 | of their trip.

01:00:35.840 | So they, you know, bike somewhere,

01:00:36.920 | then they get in our cars, they take apart their bike,

01:00:39.240 | they load into our vehicle, then they go,

01:00:40.520 | and that's, you know, how they, you know,

01:00:42.240 | where we want to pull over and how that, you know,

01:00:44.600 | get in and get out process works,

01:00:48.480 | provides us very useful feedback.

01:00:50.760 | In terms of, you know, what makes a good pickup

01:00:53.520 | and drop off location, we get really valuable feedback.

01:00:57.320 | And in fact, we had to do some really interesting work

01:01:01.720 | with high definition maps and thinking about

01:01:06.000 | walking directions.

01:01:07.040 | And if you imagine you're in a store, right?

01:01:09.120 | In some giant space, and then, you know,

01:01:10.560 | you want to be picked up somewhere.

01:01:12.400 | If you just drop a pin at a current location,

01:01:14.920 | which is maybe in the middle of a shopping mall,

01:01:16.760 | like what's the best location for the car

01:01:18.960 | to come pick you up?

01:01:20.000 | And you can have simple heuristics where you just kind of

01:01:22.160 | take your, you know, you clean in distance

01:01:24.040 | and find the nearest spot where the car can pull over

01:01:27.480 | that's closest to you.

01:01:28.320 | But oftentimes that's not the most convenient one.

01:01:30.000 | You know, I have many anecdotes where that heuristic

01:01:32.200 | breaks in horrible ways.

01:01:33.680 | One example that I often mention is somebody wanted to be,

01:01:39.000 | you know, dropped off in Phoenix and, you know,

01:01:43.960 | we got car picked a location that was close,

01:01:47.960 | I think the closest to their, you know,

01:01:49.600 | where the pin was dropped on the map in terms of,

01:01:51.960 | you know, latitude and longitude.

01:01:53.640 | But it happened to be on the other side of a parking lot

01:01:57.680 | that had this row of cacti and the poor person had to like

01:02:00.640 | walk all around the parking lot to get to where they wanted

01:02:02.960 | to be in 110 degree heat.

01:02:04.680 | So that, you know, that was the bottom.

01:02:06.160 | So then, you know, we took all, take all of these,

01:02:08.680 | all of that feedback from our users and incorporate it

01:02:11.800 | into our system and improve it.

01:02:14.000 | - Yeah, I feel like that's like requires AGI

01:02:16.600 | to solve the problem of like, when you're,

01:02:19.320 | which is a very common case,

01:02:20.920 | when you're in a big space of some kind,

01:02:23.640 | like apartment building, it doesn't matter.

01:02:25.120 | It's just not some large space.

01:02:27.040 | And then you call the, like the Waymo from there, right?

01:02:31.000 | It's like, whatever, it doesn't matter,

01:02:32.800 | a ride share vehicle.

01:02:34.960 | And like, where's the pin supposed to drop?

01:02:39.200 | I feel like that's, you don't think,

01:02:41.200 | I think that requires AGI.

01:02:42.720 | I'm gonna, in order to solve, okay, the alternative,

01:02:47.320 | which I think the Google search engine has taught

01:02:50.680 | is like, there's something really valuable

01:02:53.640 | about the perhaps slightly dumb answer,

01:02:57.440 | but a really powerful one, which is like,

01:02:59.680 | what was done in the past by others?

01:03:02.360 | Like, what was the choice made by others?

01:03:04.600 | That seems to be like, in terms of Google search,

01:03:07.640 | when you have like billions of searches,

01:03:09.720 | that you could see which, like when they recommend

01:03:13.520 | what you might possibly mean,

01:03:15.120 | they suggest based on not some machine learning thing,

01:03:18.360 | which they also do, but like,

01:03:19.880 | on what was successful for others in the past

01:03:22.720 | and finding a thing that they were happy with.

01:03:24.800 | Is that integrated at all with Waymo?

01:03:27.280 | Like, what pickups worked for others?

01:03:30.320 | - It is.

01:03:31.240 | I think you're exactly right.

01:03:32.200 | So there's a real, it's an interesting problem.

01:03:35.560 | Naive solutions have interesting failure modes.

01:03:40.560 | So there's definitely lots of things

01:03:46.680 | that can be done to improve

01:03:49.440 | and both learning from what works,

01:03:54.840 | what doesn't work in actual hail

01:03:56.400 | from getting richer data and getting more information

01:03:59.000 | about the environment and richer maps.

01:04:02.280 | But you're absolutely right that there's something,

01:04:04.280 | I think there's some properties of solutions

01:04:05.880 | that in terms of the effect that they have on users,

01:04:09.000 | some are much, much, much better than others, right?

01:04:10.680 | And predictability and understandability is important.

01:04:13.760 | So you can have maybe something

01:04:14.880 | that is not quite as optimal,

01:04:16.200 | but is very natural and predictable to the user

01:04:19.040 | and kind of works the same way all the time.

01:04:22.400 | And that matters.

01:04:23.640 | That matters a lot for the user experience.

01:04:26.680 | - But to get to the basics,

01:04:28.640 | the pretty fundamental property

01:04:31.840 | is that the car actually arrives where you told it to ride.

01:04:36.200 | Like you can always change it, see it on the map

01:04:37.960 | and you can move it around if you don't like it.

01:04:40.360 | But like that property that the car actually shows up

01:04:43.280 | on the pin is critical,

01:04:45.760 | which where compared to some of the human driven analogs,

01:04:50.760 | I think you can have more predictability.

01:04:54.360 | It's actually the fact,

01:04:55.920 | if I do a little bit of a detour here,

01:04:59.360 | I think the fact that it's your phone and the car,

01:05:02.560 | it's two computers talking to each other

01:05:04.280 | can lead to some really interesting things we can do

01:05:07.120 | in terms of the user interfaces,

01:05:09.160 | both in terms of function,

01:05:10.880 | like the car actually shows up exactly where you told it

01:05:14.240 | you want it to be,

01:05:15.200 | but also some really interesting things

01:05:17.000 | on the user interface,

01:05:17.880 | like as the car is driving,

01:05:18.880 | as you call it and it's on the way to come and pick you up.

01:05:21.920 | And of course you get the position of the car

01:05:23.400 | and the route on the map,

01:05:25.280 | but, and they actually follow that route, of course,

01:05:27.960 | but it can also share some really interesting information

01:05:30.120 | about what it's doing.

01:05:31.120 | So, you know, our cars, as they are coming to pick you up,

01:05:35.800 | if it's come, if a car is coming up to a stop sign,

01:05:38.280 | it will actually show you that like it's there sitting

01:05:40.440 | because it's at a stop sign or a traffic light

01:05:42.040 | will show you that it's got, you know,

01:05:43.000 | sitting at a red light.

01:05:44.000 | So, you know, they're like little things, right?

01:05:46.440 | But it's, I find those little touch,

01:05:49.760 | touches really interesting, really magical.

01:05:53.760 | And it's just, you know,

01:05:54.600 | little things like that that you can do

01:05:56.160 | to kind of delight your users.

01:05:57.840 | - You know, this makes me think of,

01:06:00.600 | there's some products that I just love.

01:06:04.800 | Like there's a company called Rev,

01:06:09.000 | Rev.com where I like for this podcast, for example,

01:06:13.560 | I can just drag and drop a video

01:06:16.160 | and then they do all the captioning.

01:06:19.480 | It's humans doing the captioning,

01:06:21.240 | but they connect, they automate everything

01:06:24.040 | of connecting you to the humans

01:06:25.360 | and they do the captioning and transcription.

01:06:27.360 | It's all effortless.

01:06:28.840 | And it like, I remember when I first started using them,

01:06:30.880 | it was like, life's good.

01:06:34.560 | Like, because it was so painful to figure that out earlier.

01:06:38.600 | The same thing with something called iZotope RX,

01:06:42.400 | this company I use for cleaning up audio,

01:06:44.400 | like the sound cleanup they do,

01:06:46.400 | it's like drag and drop

01:06:47.520 | and it just cleans everything up very nicely.

01:06:51.040 | Another experience like that I had

01:06:52.440 | with Amazon OneClick purchase first time,

01:06:55.760 | I mean, other places do that now,

01:06:57.560 | but just the effortlessness of purchasing,

01:07:00.280 | making it frictionless,

01:07:02.000 | it kind of communicates to me,

01:07:04.200 | like I'm a fan of design, I'm a fan of products,

01:07:08.000 | that you can just create a really pleasant experience.

01:07:11.840 | The simplicity of it, the elegance

01:07:13.560 | just makes you fall in love with it.

01:07:15.560 | So, do you think about this kind of stuff?

01:07:18.880 | I mean, it's exactly what we've been talking about.

01:07:21.600 | It's like the little details

01:07:23.280 | that somehow make you fall in love with the product.

01:07:25.640 | Is that, we went from like urban challenge days

01:07:29.560 | where love was not part of the conversation probably,

01:07:33.720 | and to this point where there's human beings

01:07:38.720 | and you want them to fall in love with the experience.

01:07:42.600 | Is that something you're trying to optimize for,

01:07:44.600 | trying to think about,

01:07:45.440 | like how do you create an experience that people love?

01:07:48.560 | - Oh, absolutely.

01:07:49.760 | I think that's the vision is removing any friction

01:07:54.760 | or complexity from getting our users,

01:08:00.200 | our writers to where they wanna go.

01:08:03.400 | And making that as simple as possible.

01:08:06.560 | And then, beyond that, just transportation,

01:08:09.480 | making things and goods get to their destination

01:08:12.720 | as seamlessly as possible.

01:08:14.080 | I talked about a drag and drop experience

01:08:16.880 | where I kind of express your intent

01:08:18.120 | and then it just magically happens.

01:08:20.560 | And for our writers, that's what we're trying to get to

01:08:23.040 | is you download an app and you click and car shows up.

01:08:28.040 | It's the same car.

01:08:29.560 | It's very predictable.

01:08:31.000 | It's a safe and high quality experience.

01:08:34.960 | And then it gets you in a very reliable,

01:08:38.160 | very convenient, frictionless way to where you wanna be.

01:08:43.160 | And along the journey,

01:08:46.560 | I think we also want to do little things

01:08:48.880 | to delight our users.

01:08:51.360 | - Like the ride sharing companies,

01:08:54.360 | because they don't control the experience, I think,

01:08:57.920 | they can't make people fall in love necessarily

01:09:00.440 | with the experience.

01:09:02.160 | Or maybe they haven't put in the effort.

01:09:04.360 | But I think if I were to speak

01:09:07.200 | to the ride sharing experience I currently have,

01:09:09.840 | it's just very convenient.

01:09:13.320 | But there's a lot of room for falling in love with it.

01:09:18.000 | We can speak to sort of car companies.

01:09:20.120 | Car companies do this well.

01:09:21.200 | You can fall in love with a car, right?

01:09:22.840 | And be like a loyal car person, like whatever.

01:09:26.520 | Like I like badass hot rods, I guess '69 Corvette.

01:09:30.320 | And at this point, you can't really...

01:09:33.480 | Cars are so, owning a car is so 20th century, man.

01:09:37.680 | But is there something about the Waymo experience

01:09:41.800 | where you hope that people will fall in love with it?

01:09:44.000 | Is that part of it?

01:09:46.320 | Or is it just about making a convenient ride,

01:09:51.400 | not ride sharing, I don't know what the right term is,

01:09:53.400 | but just a convenient A to B autonomous transport?

01:09:58.400 | Or like, do you want them to fall in love with Waymo?

01:10:03.680 | So maybe elaborate a little bit.

01:10:05.400 | I mean, almost like from a business perspective,

01:10:07.640 | I'm curious, like how...

01:10:11.880 | Do you wanna be in the background invisible?

01:10:14.560 | Or do you want to be like a source of joy

01:10:19.560 | that's in very much in the foreground?

01:10:22.400 | - I want to provide the best,

01:10:26.040 | most enjoyable transportation solution.

01:10:28.920 | And that means building it, building our product

01:10:34.920 | and building our service in a way that people do.

01:10:37.480 | Kind of use in a very seamless, frictionless way

01:10:43.400 | in their day-to-day lives.

01:10:46.920 | And I think that does mean,

01:10:48.960 | in some way falling in love in that product, right?

01:10:51.440 | It just kind of becomes part of your routine.

01:10:54.000 | It comes down in my mind to safety,

01:10:58.480 | predictability of the experience and privacy, I think.

01:11:03.920 | Privacy, I think, aspects of it, right?

01:11:06.560 | Our cars, you get the same car,

01:11:09.920 | you get very predictable behavior.

01:11:12.320 | And that is important.

01:11:14.320 | I think if you're gonna use it in your daily life.

01:11:17.160 | Privacy, I mean, when you're in a car,

01:11:18.800 | you can do other things.

01:11:19.640 | You're spending a bunch, just another space

01:11:21.760 | where you're spending a significant part of your life.

01:11:24.600 | And so not having to share it with other people

01:11:27.640 | who you don't wanna share it with,

01:11:28.880 | I think is a very nice property.

01:11:32.360 | Maybe you wanna take a phone call

01:11:33.640 | or do something else in the vehicle.

01:11:35.440 | And safety on the quality of the driving,

01:11:40.760 | as well as the physical safety of not having to share

01:11:45.240 | that ride is important to a lot of people.

01:11:49.380 | - What about the idea that when there's somebody,

01:11:54.380 | like a human driving and they do a rolling stop

01:11:58.040 | on a stop sign, like sometimes,

01:12:01.400 | you get an Uber or Lyft or whatever, like human driver,

01:12:04.400 | and they can be a little bit aggressive as drivers.

01:12:09.120 | It feels like there is, not all aggression is bad.

01:12:12.660 | Now that may be a wrong, again,

01:12:16.240 | 20th century conception of driving.

01:12:18.480 | Maybe it's possible to create a driving experience.

01:12:21.320 | Like if you're in the back, busy doing something,

01:12:24.520 | maybe aggression is not a good thing.

01:12:26.640 | It's a very different kind of experience perhaps.

01:12:29.320 | But it feels like in order to navigate this world,

01:12:33.560 | you need to, how do I phrase this?

01:12:38.200 | You need to kind of bend the rules a little bit,

01:12:39.960 | or at least like test the rules.

01:12:42.120 | I don't know what language politicians use to discuss this,

01:12:44.680 | but whatever language they use,

01:12:47.600 | you like flirt with the rules, I don't know.

01:12:49.600 | But like you sort of have a bit of an aggressive way

01:12:56.000 | of driving that asserts your presence in this world,

01:13:00.640 | thereby making other vehicles

01:13:02.480 | and people respect your presence,

01:13:04.760 | and thereby allowing you to sort of navigate

01:13:06.960 | through intersections in a timely fashion.

01:13:09.800 | I don't know if any of that made sense,

01:13:11.200 | but like how does that fit into the experience

01:13:15.080 | of driving autonomously?

01:13:17.300 | Is that-- - It's a lot of stuff.

01:13:19.560 | This is, you're hitting on a very important point

01:13:21.360 | of a number of behavioral components

01:13:26.360 | and parameters that make your driving feel assertive

01:13:31.360 | and natural and comfortable and predictable.

01:13:36.720 | Now, our cars will follow rules, right?

01:13:38.400 | They will do the safest thing possible in all situations,

01:13:40.440 | let me be clear on that.

01:13:42.160 | But if you think of really, really good drivers,

01:13:46.800 | just think about professional lemon drivers, right?

01:13:49.320 | They will follow the rules.

01:13:50.480 | They're very, very smooth, and yet they're very efficient.

01:13:55.480 | But they're assertive.

01:13:57.200 | They're comfortable for the people in the vehicle.

01:14:00.640 | They're predictable for the other people outside the vehicle

01:14:03.360 | that they share the environment with.

01:14:04.640 | And that's the kind of driver that we want to build.

01:14:07.360 | And you think if maybe there's a sport analogy there,

01:14:11.920 | you can do in very many sports,

01:14:15.560 | the true professionals are very efficient

01:14:19.920 | in their movements, right?

01:14:21.160 | So they don't do like hectic flailing, right?

01:14:25.200 | They're smooth and precise, right?

01:14:28.760 | And they get the best results.

01:14:29.680 | So that's the kind of driver that we want to build.

01:14:31.480 | In terms of aggressiveness, yeah,

01:14:33.000 | you can like roll through the stop signs.

01:14:34.920 | You can do crazy lane changes.

01:14:37.160 | Typically doesn't get you to your destination faster.

01:14:39.280 | Typically not the safest or most predictable,

01:14:41.760 | or most comfortable thing to do.

01:14:44.880 | But there is a way to do both.

01:14:48.280 | And that's what we're trying to build,

01:14:50.760 | the driver that is safe, comfortable,

01:14:54.880 | smooth, and predictable.

01:14:57.480 | - Yeah, that's a really interesting distinction.

01:14:59.640 | I think in the early days of autonomous vehicles,

01:15:02.280 | the vehicles felt cautious as opposed to efficient.

01:15:05.720 | And still probably, but when I rode in the Waymo,

01:15:10.720 | I mean, it was quite assertive.

01:15:14.480 | It moved pretty quickly.

01:15:17.520 | And like, yeah, then he's one of the surprising feelings

01:15:21.240 | was that it actually, it went fast

01:15:23.760 | and it didn't feel like awkwardly cautious

01:15:28.440 | than autonomous vehicle.

01:15:29.640 | So I've also programmed autonomous vehicles

01:15:32.960 | and everything I've ever built was felt awkwardly,

01:15:36.480 | either overly aggressive, okay?

01:15:39.160 | Especially when it was my code,

01:15:41.240 | or like awkwardly cautious is the way I would put it.

01:15:46.320 | And Waymo's vehicle felt like assertive

01:15:51.320 | and I think efficient is like the right terminology here.

01:15:57.680 | They wasn't, and I also like the professional limo driver.

01:16:01.640 | 'Cause we often think like, you know,

01:16:04.120 | an Uber driver or a bus driver or a taxi.

01:16:08.240 | This is the funny thing is people think

01:16:10.240 | like taxi drivers are professionals.

01:16:15.080 | I mean, it's like, that's like saying

01:16:18.400 | I'm a professional walker

01:16:19.760 | just because I've been walking all my life.

01:16:22.080 | I think there's an art to it, right?

01:16:25.160 | And if you take it seriously as an art form,

01:16:28.160 | then there's a certain way that mastery looks like.

01:16:32.680 | And it's interesting to think about

01:16:33.960 | what does mastery look like in driving?

01:16:37.000 | And perhaps what we associate with like aggressiveness

01:16:41.680 | is unnecessary, like it's not part

01:16:44.720 | of the experience of driving.

01:16:46.960 | It's like unnecessary fluff that efficiency,

01:16:51.960 | you can be, you can create a good driving experience

01:16:57.040 | within the rules.

01:16:59.060 | That's, I mean, you're the first person to tell me this.

01:17:03.240 | So it's kind of interesting.

01:17:04.520 | I need to think about this,

01:17:05.800 | but that's exactly what it felt like with Waymo.

01:17:08.120 | I kind of had this intuition,

01:17:09.200 | maybe it's the Russian thing, I don't know,

01:17:11.420 | that you have to break the rules in life to get anywhere.

01:17:14.480 | But maybe, maybe it's possible that that's not the case

01:17:19.640 | in driving.

01:17:22.120 | I have to think about that.

01:17:23.280 | But it certainly felt that way on the streets of Phoenix

01:17:25.720 | when I was there in Waymo,

01:17:27.440 | that that was a very pleasant experience

01:17:29.960 | and it wasn't frustrating in that like,

01:17:32.280 | come on, move already kind of feeling.

01:17:34.280 | It wasn't, that wasn't there.

01:17:36.240 | - Yeah, I mean, that's what we're going after.

01:17:38.940 | I don't think you have to pick one.

01:17:40.000 | I think truly good driving,

01:17:42.240 | it gives you both efficiency, a certainness,

01:17:44.920 | but also comfort and predictability and safety.

01:17:47.360 | And that's what fundamental improvements

01:17:53.360 | in the core capabilities truly unlock.

01:17:57.480 | And you can kind of think of it as,

01:17:59.640 | precision and recall trade-off.

01:18:01.520 | You have certain capabilities of your model,

01:18:03.200 | and then it's very easy when you have some curve

01:18:05.620 | of precision and recall, you can move things around

01:18:07.280 | and can choose your operating point

01:18:08.560 | in your training of precision versus recall,

01:18:10.240 | false positives versus false negatives.

01:18:12.680 | But then, and you can tune things on that curve

01:18:15.720 | and be kind of more cautious or more aggressive,

01:18:17.520 | but then aggressive is bad or cautious is bad.

01:18:19.920 | But true capabilities come from actually moving

01:18:22.360 | the whole curve up.

01:18:24.120 | And then you are on a very different plane

01:18:28.220 | of those trade-offs.

01:18:29.480 | And that's what we're trying to do here

01:18:31.640 | is to move the whole curve up.

01:18:33.460 | - Before I forget, let's talk about trucks a little bit.

01:18:37.880 | So I also got a chance to check out

01:18:39.560 | some of the Waymo trucks.

01:18:42.000 | I'm not sure if we wanna go too much into that space,

01:18:45.860 | but it's a fascinating one.

01:18:46.840 | So maybe we can mention at least briefly,

01:18:48.840 | Waymo is also now doing autonomous trucking.

01:18:53.040 | And how different, like philosophically and technically

01:18:57.960 | is that whole space of problems?

01:19:00.240 | - It's one of our two big products

01:19:04.520 | and commercial applications of our driver,

01:19:08.280 | ride hailing and deliveries.

01:19:10.920 | We have Waymo One and Waymo Via,

01:19:12.800 | moving people and moving goods.

01:19:14.920 | Trucking is an example of moving goods.

01:19:18.800 | We've been working on trucking since 2017.

01:19:22.360 | It is a very interesting space.

01:19:28.280 | And your question of how different is it?

01:19:32.560 | It has this really nice property

01:19:34.200 | that the first order challenges,

01:19:37.120 | like the science, the hard engineering,

01:19:40.120 | whether it's hardware or onboard software

01:19:43.120 | or off-board software,

01:19:44.480 | all of the systems that you build

01:19:46.520 | for training your ML models,

01:19:48.720 | for evaluating your entire system,

01:19:51.400 | those fundamentals carry over.

01:19:53.760 | The true challenges of driving, perception,

01:19:58.760 | semantic understanding, prediction,

01:20:01.400 | decision making, planning, evaluation,

01:20:04.640 | the simulator, ML infrastructure, those carry over.

01:20:08.000 | Like the data and the application

01:20:09.520 | and kind of the domains might be different,

01:20:12.880 | but the most difficult problems,

01:20:15.440 | all of that carries over between the domains.

01:20:18.160 | So that's very nice.

01:20:19.360 | So that's how we approach it.

01:20:20.400 | We're kind of build investing in the core,

01:20:23.320 | the technical core.

01:20:25.160 | And then there is specialization of that core technology

01:20:29.920 | to different product lines,

01:20:31.160 | to different commercial applications.

01:20:33.040 | So on, just to tease it apart a little bit,

01:20:36.440 | on trucks, so starting with the hardware,

01:20:39.160 | the configuration of the sensors is different.

01:20:42.120 | They're different physically, geometrically,

01:20:45.880 | different vehicles.

01:20:47.640 | So for example, we have two of our main laser

01:20:50.880 | on the trucks on both sides,

01:20:53.200 | so that we have, not have the blind spots.

01:20:55.680 | Whereas on the JLR I-PACE, we have one of it,

01:20:59.880 | sitting at the very top,

01:21:01.160 | but the actual sensors are almost the same

01:21:04.440 | or largely the same.

01:21:05.680 | So all of the investment that over the years

01:21:09.680 | we've put into building our custom lighters,

01:21:12.040 | custom radars, pulling the whole system together,

01:21:14.200 | that carries over very nicely.

01:21:16.280 | Then on the perception side,

01:21:18.200 | the fundamental challenges of seeing,

01:21:21.440 | understanding the world,

01:21:22.320 | whether it's object detection, classification,

01:21:24.720 | tracking, semantic understanding, all that carries over.

01:21:27.960 | Yes, there's some specialization

01:21:29.360 | when you're driving on freeways,

01:21:31.200 | range becomes more important,

01:21:33.240 | the domain is a little bit different,

01:21:34.480 | but again, the fundamentals carry over very, very nicely.

01:21:38.200 | Same, and I guess you get into prediction

01:21:41.120 | or decision-making, right?

01:21:42.480 | The fundamentals of what it takes to predict

01:21:45.920 | what other people are going to do,

01:21:48.280 | to find the long tail, to improve your system

01:21:51.440 | in that long tail of behavior prediction and response,

01:21:54.760 | that carries over, right?

01:21:55.720 | And so on and so on.

01:21:56.720 | So, I mean, that's pretty exciting.

01:21:59.080 | By the way, does Waymo VIA include

01:22:01.920 | using the smaller vehicles for transportation goods?

01:22:06.200 | That's an interesting distinction.

01:22:07.600 | So, I would say there's three interesting

01:22:11.160 | modes of operation.

01:22:13.080 | So, one is moving humans, one is moving goods,

01:22:15.720 | and one is like moving nothing, zero occupancy,

01:22:19.680 | meaning like you're going to the destination,

01:22:22.960 | you're empty vehicle.

01:22:25.440 | I mean, it's-

01:22:27.280 | - The third is the less of it,

01:22:28.680 | that's the entirety of it,

01:22:29.520 | it's the less exciting from the commercial perspective.

01:22:32.120 | (both laughing)

01:22:34.400 | - Well, I mean, in terms of like,

01:22:36.920 | if you think about what's inside a vehicle as it's moving,

01:22:40.120 | because it does, some significant fraction

01:22:44.240 | of the vehicle's movement has to be empty.

01:22:48.760 | I mean, it's kind of fascinating,

01:22:50.120 | maybe just on that small point,

01:22:51.680 | is there different control and like,

01:22:57.640 | policies that are applied for zero occupancy vehicle?

01:23:01.480 | So, a vehicle with nothing in it,

01:23:03.400 | or is it just move as if there is a person inside,

01:23:06.320 | what was with some subtle differences?

01:23:09.680 | - As a first order approximation, there are no differences.

01:23:13.000 | And if you think about, you know, safety and, you know,

01:23:17.000 | comfort and quality of driving, only part of it,

01:23:23.440 | has to do with the people or the goods

01:23:27.400 | inside of the vehicle, right?

01:23:28.800 | But you don't wanna be, you know,

01:23:30.120 | you wanna drive smoothly, as we discussed,

01:23:32.080 | not for the purely for the benefit of, you know,

01:23:34.600 | whatever you have inside the car, right?

01:23:36.280 | It's also for the benefit of the people outside

01:23:39.200 | kind of feeding, fitting naturally and predictably

01:23:41.720 | into that whole environment, right?

01:23:43.240 | So, you know, yes, there are some second order things

01:23:45.520 | you can do, you can change your route,

01:23:47.680 | and, you know, optimize maybe kind of your fleet,

01:23:50.920 | things at the fleet scale,

01:23:52.480 | and you would take into account whether, you know,

01:23:54.560 | some of your cars are actually, you know,

01:23:58.320 | serving a useful trip, whether with people or with goods,

01:24:01.040 | whereas, you know, other cars are, you know,

01:24:03.440 | driving completely empty, you know,

01:24:04.760 | to that next valuable trip that they're going to provide,

01:24:08.860 | but that those are mostly second order effects.

01:24:11.560 | - Okay, cool.

01:24:12.520 | So Phoenix is an incredible place,

01:24:16.760 | and what you've announced in Phoenix is,

01:24:21.120 | it's kind of amazing, but, you know,

01:24:23.320 | that's just like one city.

01:24:24.920 | How do you take over the world?

01:24:27.560 | I mean, I'm asking for a friend.

01:24:32.000 | - One step at a time.

01:24:33.740 | - One city at a time.

01:24:36.160 | - Is that the cartoon pinky in the brain?

01:24:38.600 | Okay.

01:24:39.440 | - But, you know, gradually is a true answer.

01:24:42.920 | - So I think the heart of your question is, you know, what--

01:24:47.360 | - Can you ask a better question than I asked?

01:24:49.720 | - You're asking a great question.

01:24:50.560 | - Answer that one.

01:24:51.760 | - I'm, you know, just gonna, you know,

01:24:54.200 | phrase it in the terms that I want to answer.

01:24:56.760 | - Answer it, perfect.

01:24:58.000 | That's exactly right, brilliant.

01:24:59.840 | Please.

01:25:00.880 | - No, you know, where are we today?

01:25:03.000 | And, you know, what happens next?

01:25:04.720 | And what does it take to go beyond Phoenix?

01:25:07.440 | And what does it take to get this technology

01:25:11.840 | to more places and more people around the world, right?

01:25:15.960 | - So our next big area of focus

01:25:20.960 | is exactly that, larger scale commercialization

01:25:26.640 | and just, you know, scaling up.

01:25:28.200 | If I think about, you know, the main,

01:25:35.200 | and, you know, Phoenix gives us that platform

01:25:39.120 | and gives us that foundation upon which we can build.

01:25:44.200 | And it's, there are a few really challenging aspects

01:25:49.200 | of this whole problem that you have to pull together

01:25:54.200 | in order to, you know, build the technology,

01:25:57.520 | in order to deploy it into the field,

01:26:02.440 | to go from a driverless car to a fleet of cars

01:26:07.440 | that are providing a service,

01:26:10.440 | and then all the way to, you know, commercialization.

01:26:13.160 | So, and, you know, this is what we have in Phoenix.

01:26:15.720 | We've taken the technology from a proof point

01:26:20.080 | to an actual deployment,

01:26:21.120 | and have taken our driver, you know, from, you know,

01:26:24.840 | one car to a fleet that can provide a service.

01:26:27.140 | Beyond that, if I think about what it will take

01:26:31.040 | to scale up and, you know, deploy in, you know,

01:26:36.040 | more places with more customers,

01:26:38.520 | I tend to think about three main dimensions,

01:26:43.520 | three main axes of scale.

01:26:46.600 | One is the core technology, you know,

01:26:49.640 | the hardware and software core capabilities of our driver.

01:26:53.640 | The second dimension is evaluation and deployment.

01:26:58.640 | And the third one is just the, you know,

01:27:01.960 | product, commercial, and operational excellence.

01:27:05.960 | So you can talk a bit about where we are along, you know,

01:27:09.360 | each one of those three dimensions,

01:27:10.440 | about where we are today and, you know,

01:27:12.120 | what has, what will happen next.

01:27:14.120 | On, you know, the core technology,

01:27:17.880 | on, you know, the hardware and software, you know,

01:27:19.800 | together, comprised of driver,

01:27:22.280 | we, you know, obviously have that foundation

01:27:26.920 | that is providing fully driverless trips to our customers

01:27:31.560 | as we speak, in fact.

01:27:34.320 | And we've learned a tremendous amount from that.

01:27:38.480 | So now what we're doing is we are incorporating

01:27:42.120 | all those lessons into some pretty fundamental improvements

01:27:45.920 | in our core technology, both on the hardware side

01:27:48.040 | and on the software side,

01:27:50.160 | to build a more general, more robust solution

01:27:53.360 | that then will enable us to massively scale,

01:27:55.960 | you know, beyond Phoenix.

01:27:57.080 | So on the hardware side,

01:28:00.120 | all of those lessons are now incorporated

01:28:04.440 | into this fifth generation hardware platform

01:28:07.160 | that is, you know, being deployed right now.

01:28:10.960 | And that's the platform, the fourth generation,

01:28:13.720 | the thing that we have right now driving in Phoenix,

01:28:15.560 | it's good enough to operate fully driverless,

01:28:18.840 | you know, night and day, you know,

01:28:20.680 | various speeds and various conditions.

01:28:22.600 | But the fifth generation is the platform

01:28:25.120 | upon which we want to go to massive scale.

01:28:28.360 | We've really made qualitative improvements

01:28:31.920 | in terms of the capability of the system,

01:28:34.360 | the simplicity of the architecture,

01:28:36.120 | the reliability of the redundancy.

01:28:38.200 | It is designed to be manufacturable at very large scale

01:28:41.840 | and, you know, provides the right unit economics.

01:28:43.440 | So that's the next big step for us on the hardware side.

01:28:48.080 | - That's already there for scale, the version five.

01:28:50.800 | - That's right.

01:28:51.640 | - And is that a coincidence

01:28:53.560 | or should we look into a conspiracy theory

01:28:55.360 | that it's the same version as the Pixel phone?

01:28:58.080 | (laughs)

01:28:59.040 | Is that what's the hardware?

01:28:59.880 | - I can neither confirm nor deny, Lux.

01:29:03.000 | - All right, cool.

01:29:03.840 | So, sorry.

01:29:04.680 | So that's the, okay, that's that axis.

01:29:07.040 | What else?

01:29:08.280 | - So similarly, you know,

01:29:09.120 | the hardware is a very discreet jump,

01:29:11.520 | but similar to how we're making that change

01:29:15.040 | from the fourth generation hardware to the fifth,

01:29:16.880 | we're making similar improvements on the software side

01:29:19.040 | to make it more robust and more general

01:29:21.080 | and allow us to kind of quickly scale beyond Phoenix.

01:29:24.640 | So that's the first dimension of core technology.

01:29:26.560 | The second dimension is evaluation and deployment.

01:29:28.880 | Now, how do you measure your system?

01:29:33.880 | How do you evaluate it?

01:29:35.040 | How do you build a release and deployment process

01:29:37.520 | where, you know, with confidence,

01:29:39.960 | you can regularly release new versions

01:29:42.840 | of your driver into a fleet?

01:29:45.480 | How do you get good at it so that it is not, you know,

01:29:49.040 | a huge tax on your researchers and engineers that, you know,

01:29:52.480 | so you can, how do you build all of these, you know,

01:29:54.240 | processes, the frameworks, the simulation, the evaluation,

01:29:57.960 | the data science, the validation,

01:29:59.760 | so that, you know, people can focus on improving the system

01:30:02.440 | and kind of the releases just go out the door

01:30:04.720 | and get deployed across the fleet.

01:30:06.640 | So we've gotten really good at that in Phoenix.

01:30:09.640 | That's been a tremendously difficult problem,

01:30:13.080 | but that's what we have in Phoenix right now

01:30:14.920 | that gives us that foundation.

01:30:16.240 | And now we're working on kind of incorporating

01:30:18.320 | all the lessons that we've learned

01:30:19.880 | to make it more efficient, to go to new places,

01:30:22.320 | you know, and scale up and just kind of, you know,

01:30:23.560 | stamp things out.

01:30:25.200 | So that's that second dimension of evaluation and deployment.

01:30:28.080 | And the third dimension is product, commercial,

01:30:32.440 | and operational excellence, right?

01:30:34.320 | And again, Phoenix there is providing

01:30:37.720 | an incredibly valuable platform.

01:30:40.160 | You know, that's why we're doing things end-to-end

01:30:42.320 | in Phoenix, we're learning as, you know,

01:30:43.680 | we discussed a little earlier today,

01:30:46.200 | tremendous amount of really valuable lessons

01:30:49.240 | from our users getting really incredible feedback.

01:30:52.200 | And we'll continue to iterate on that

01:30:54.840 | and incorporate all those lessons

01:30:57.400 | into making our product, you know,

01:31:00.080 | even better and more convenient for our users.

01:31:01.760 | - So you're converting this whole process of Phoenix

01:31:04.800 | in Phoenix into something that could be copied

01:31:08.040 | and pasted elsewhere.

01:31:09.440 | So like, perhaps you didn't think of it that way

01:31:12.640 | when you were doing the experimentation Phoenix,

01:31:14.520 | but so how long did, basically,

01:31:18.960 | you can correct me, but you've,

01:31:21.920 | I mean, it's still early days,

01:31:23.080 | but you've taken the full journey in Phoenix, right?

01:31:26.120 | As you were saying of like what it takes

01:31:28.920 | to basically automate, I mean,

01:31:30.720 | it's not the entirety of Phoenix, right?

01:31:32.360 | But I imagine it can encompass the entirety of Phoenix

01:31:37.360 | at some near-term date,

01:31:40.640 | but that's not even perhaps important,

01:31:42.400 | like as long as it's a large enough geographic area.

01:31:45.160 | So what, how copy-pastable is that process currently?

01:31:50.800 | Is that process currently?

01:31:53.000 | And how, like, you know,

01:31:57.600 | like when you copy and paste in Google Docs, I think,

01:32:02.600 | no, or in Word, you can like apply source formatting

01:32:06.920 | or apply destination formatting.

01:32:09.440 | So when you copy and paste the Phoenix into like,

01:32:14.440 | say, Boston, how do you apply the destination formatting?

01:32:19.840 | Like how much of the core of the entire process

01:32:24.160 | of bringing an actual public transportation,

01:32:29.160 | autonomous transportation service to a city

01:32:32.800 | is there in Phoenix that you understand enough

01:32:35.800 | to copy and paste into Boston or wherever?

01:32:38.740 | - So we're not quite there yet.

01:32:40.600 | We're not at a point where we're kind of massively

01:32:42.720 | copy and pasting all over the place,

01:32:44.520 | but Phoenix, what we did in Phoenix,

01:32:48.840 | and we very intentionally have chosen Phoenix

01:32:51.520 | as our first full deployment area,

01:32:56.520 | you know, exactly for that reason,

01:32:57.640 | to kind of tease the problem apart,

01:32:59.560 | look at each dimension,

01:33:01.640 | focus on the fundamentals of complexity

01:33:03.840 | and de-risking those dimensions,

01:33:05.920 | and then bringing the entire thing together

01:33:07.600 | to get all the way,

01:33:09.200 | force ourselves to learn all those hard lessons

01:33:11.560 | on technology, hardware and software,

01:33:13.540 | on the evaluation deployment,

01:33:15.000 | on operating a service, operating a business,

01:33:17.500 | using actually serving our customers,

01:33:21.480 | all the way so that we're fully informed

01:33:23.800 | about the most difficult, most important challenges

01:33:28.500 | to get us to that next step of massive copy and pasting,

01:33:32.240 | as you said.

01:33:34.640 | And that's what we're doing right now.

01:33:39.080 | We're incorporating all those things that we learned

01:33:41.120 | into that next system that then will allow us

01:33:44.040 | to kind of copy and paste all over the place

01:33:46.040 | and to massively scale to more users and more locations.

01:33:49.700 | I mean, you know, I just talked a little bit about,

01:33:51.100 | what does that mean along those different dimensions?

01:33:53.600 | So on the hardware side, for example,

01:33:55.140 | again, it's that switch from the fourth

01:33:57.740 | to the fifth generation.

01:33:58.740 | And the fifth generation is designed

01:34:00.300 | to kind of have that property.

01:34:01.800 | - Can you say what other cities you're thinking about?

01:34:05.300 | Like I'm thinking about,

01:34:07.140 | sorry, we're in San Francisco now.

01:34:10.020 | I thought I want to move to San Francisco,

01:34:12.580 | but I'm thinking about moving to Austin.

01:34:15.520 | I don't know why.

01:34:16.680 | People are not being very nice about San Francisco currently.

01:34:19.680 | Maybe it's in vogue right now.

01:34:23.440 | But Austin seems, I visited there and it was,

01:34:26.500 | I was in a Walmart.

01:34:28.960 | It's funny, these moments like turn your life.

01:34:33.000 | There's this very nice woman with kind eyes,

01:34:37.000 | just like stopped and said,

01:34:39.800 | "You look so handsome in that tie, honey," to me.

01:34:44.440 | This has never happened to me in my life,

01:34:46.180 | but just the sweetness of this woman

01:34:48.540 | is something I've never experienced,

01:34:49.780 | certainly in the streets of Boston,

01:34:51.540 | but even in San Francisco where people wouldn't,

01:34:53.840 | that's just not how they speak or think.

01:34:57.040 | I don't know.

01:34:57.880 | There's a warmth to Austin that love.

01:35:00.260 | And since Waymo does have a little bit of a history there,

01:35:03.940 | is that a possibility?

01:35:05.160 | - Is this your version of asking the question of like,

01:35:08.580 | you know, Dimitri, I know you can't share

01:35:10.020 | your commercial and deployment roadmap,

01:35:12.020 | but I'm thinking about moving to San Francisco, Austin,

01:35:15.180 | like in a blink twice, if you think I should move to.

01:35:18.540 | - Yeah, that's true.

01:35:19.380 | That's true.

01:35:20.200 | You got me.

01:35:21.040 | We've been testing all over the place.

01:35:24.060 | I think we've been testing more than 25 cities.

01:35:26.700 | We drive in San Francisco, we drive in Michigan for snow.

01:35:31.100 | We are doing significant amount of testing in the Bay Area,

01:35:34.380 | including San Francisco.

01:35:35.540 | - But just not like,

01:35:37.100 | 'cause we're talking about the various different thing,

01:35:39.020 | which is like a full on large geographic area,

01:35:43.340 | public service.

01:35:45.340 | You can't share.

01:35:46.980 | Okay.

01:35:47.820 | What about Moscow?

01:35:51.980 | Is that, when is that happening?

01:35:54.300 | Take on Yandex.

01:35:56.580 | I'm not paying attention to those folks.

01:35:58.860 | They're doing, you know, there's a lot of fun.

01:36:02.300 | I mean, maybe as a way of a question,

01:36:06.960 | you didn't speak to sort of like policy

01:36:11.560 | or like, is there tricky things with government and so on?

01:36:15.680 | Is there other friction that you've encountered

01:36:22.160 | except sort of technological friction

01:36:24.880 | of solving this very difficult problem?

01:36:27.480 | Is there other stuff that you have to overcome

01:36:29.840 | when deploying a public service in a city?

01:36:34.060 | That's interesting.

01:36:36.560 | - It's very important.

01:36:37.800 | So we put significant effort in creating those partnerships

01:36:42.800 | and those relationships with governments at all levels,

01:36:48.640 | local governments, municipalities,

01:36:50.280 | state level, federal level.

01:36:52.400 | We've been engaged in very deep conversations

01:36:54.560 | from the earliest days of our projects

01:36:57.520 | whenever at all of these levels,

01:36:59.840 | whenever we go to test or operate in a new area,

01:37:06.200 | we always lead with a conversation

01:37:09.360 | with the local officials.

01:37:11.240 | But the result of that investment is that,

01:37:13.400 | no, it's not challenges we have to overcome,

01:37:15.960 | but it is a very important

01:37:17.480 | that we continue to have this conversation.

01:37:20.360 | - Yeah, I love politicians too.

01:37:22.200 | Okay, so Mr. Elon Musk said that LIDAR is a crutch.

01:37:27.200 | What are your thoughts?

01:37:30.760 | - I wouldn't characterize it exactly that way.

01:37:35.200 | I know, I think LIDAR is very important.

01:37:37.920 | It is a key sensor that we use just like other modalities.

01:37:42.920 | As we discussed, our cars use cameras, LIDARs and radars.

01:37:47.580 | They are all very important.

01:37:50.920 | They are at the kind of the physical level.

01:37:54.920 | They are very different.

01:37:57.520 | They have very different physical characteristics.

01:38:00.440 | Cameras are passive, LIDARs and radars are active,

01:38:03.160 | and use different wavelengths.

01:38:05.540 | So that means they complement each other very nicely.

01:38:08.980 | And together, combined, they can be used

01:38:13.980 | to build a much safer and much more capable system.

01:38:19.540 | So to me, it's more of a question,

01:38:25.420 | why the heck would you handicap yourself

01:38:27.540 | and not use one or more of those sensing modalities

01:38:30.540 | when they undoubtedly just make your system

01:38:34.300 | more capable and safer?

01:38:36.980 | Now, what might make sense for one product

01:38:41.980 | or one business might not make sense for another one.

01:38:48.580 | So if you're talking about driver assist technologies,

01:38:51.100 | you make certain design decisions

01:38:52.580 | and you make certain trade-offs,

01:38:53.780 | and you make different ones if you're building a driver

01:38:56.820 | that you deploy in fully driverless vehicles.

01:39:00.640 | And LIDAR specifically, when this question comes up,

01:39:04.940 | typically the criticisms that I hear

01:39:10.700 | are the counterpoints that cost and aesthetics.

01:39:15.120 | And I don't find either of those, honestly, very compelling.

01:39:20.620 | So on the cost side,

01:39:22.420 | there's nothing fundamentally prohibitive

01:39:24.480 | about the cost of LIDARs.

01:39:27.020 | Radars used to be very expensive

01:39:28.620 | before people made certain advances in technology

01:39:32.700 | and started to manufacture them at massive scale

01:39:36.020 | and deploy them in vehicles, similar with LIDARs.

01:39:39.060 | And this is where the LIDARs that we have on our cars,

01:39:41.860 | especially the fifth generation,

01:39:44.220 | we've been able to make some pretty qualitative

01:39:47.700 | discontinuous jumps in terms of the fundamental technology

01:39:50.480 | that allow us to manufacture those things

01:39:52.780 | at very significant scale and at a fraction of the cost

01:39:57.400 | of both our previous generation,

01:40:00.440 | as well as a fraction of the cost

01:40:02.380 | of what might be available on the market

01:40:05.060 | off the shelf right now.

01:40:06.460 | And that improvement will continue.

01:40:07.940 | So I think cost is not a real issue.

01:40:11.740 | Second one is aesthetics.

01:40:13.600 | I don't think that's a real issue either.

01:40:17.580 | - Beauty is in the eye of the beholder.

01:40:21.380 | You can make LIDAR sexy again.

01:40:23.020 | - I think you're exactly right.

01:40:23.860 | I think it is sexy.

01:40:24.680 | Like honestly, I think form is the function.

01:40:26.940 | - Well, okay.

01:40:27.780 | You know, I was actually,

01:40:29.220 | somebody brought this up to me.

01:40:31.220 | I mean, all forms of LIDAR,

01:40:33.180 | even like the ones that are like big,

01:40:36.940 | you can make look, I mean, you can make look beautiful.

01:40:40.460 | Like there's no sense in which

01:40:42.060 | you can't integrate it into design.

01:40:44.200 | Like there's all kinds of awesome designs.

01:40:46.300 | I don't think small and humble is beautiful.

01:40:50.540 | It could be like, you know, brutalism

01:40:53.260 | or like it could be like harsh corners.

01:40:56.700 | I mean, like I said, like hot rods.

01:40:58.180 | Like I don't necessarily like,

01:41:00.860 | like, oh man, I'm gonna start so much controversy with this.

01:41:03.860 | I don't like Porsches.

01:41:06.980 | Okay.

01:41:07.820 | The Porsche 911, like everyone says,

01:41:09.500 | oh, it's the most beautiful.

01:41:10.660 | No, no.

01:41:11.900 | It's like a baby car.

01:41:13.740 | It doesn't make any sense.

01:41:15.500 | But everyone, it's beauty is in the eye of the beholder.

01:41:18.340 | You're already looking at me like,

01:41:19.180 | what is this kid talking about?

01:41:21.100 | - I'm happy to talk about-

01:41:23.540 | - You're digging your own hole.

01:41:24.900 | - The form and function and my take on the beauty

01:41:28.660 | of the hardware that we put on our vehicles.

01:41:31.060 | You know, I will not comment on your Porsche monologue.

01:41:34.780 | - Okay.

01:41:35.660 | All right.

01:41:36.500 | So, but aesthetics, fine.

01:41:37.940 | But there's an underlying like philosophical question

01:41:41.540 | behind the kind of LIDAR question is like,

01:41:44.260 | how much of the problem can be solved

01:41:48.020 | with computer vision, with machine learning?

01:41:51.700 | So I think without sort of disagreements and so on,

01:41:56.700 | it's nice to put it on the spectrum

01:42:01.940 | because Waymo's doing a lot of machine learning as well.

01:42:04.900 | It's interesting to think how much of driving,

01:42:06.980 | if we look at five years, 10 years, 50 years down the road,

01:42:11.260 | what can be learned in almost more and more

01:42:14.900 | and more end to end way.

01:42:17.340 | If we look at what Tesla is doing

01:42:19.260 | as a machine learning problem,

01:42:22.020 | they're doing a multitask learning thing

01:42:24.780 | where it's just, they break up driving

01:42:26.180 | into a bunch of learning tasks

01:42:27.940 | and they have one single neural network

01:42:29.420 | and they're just collecting huge amounts of data

01:42:30.980 | that's training that.

01:42:32.180 | I've recently hung out with George Hotz.

01:42:33.860 | I don't know if you know George.

01:42:35.460 | (laughs)

01:42:37.540 | I love him so much.

01:42:40.180 | He's just an entertaining human being.

01:42:41.940 | We were off mic talking about Hunter S. Thompson.

01:42:43.860 | He's the Hunter S. Thompson of the time I was driving.

01:42:46.260 | Okay.

01:42:47.100 | So he, I didn't realize this with Kama AI,

01:42:49.980 | but they're like really trying to do end to end.

01:42:52.660 | Like looking at the machine learning problem,

01:42:56.460 | they're really not doing multitask learning,

01:43:00.380 | but it's computing the drivable area

01:43:04.540 | as a machine learning task

01:43:06.100 | and hoping that like down the line,

01:43:09.380 | this level two system, this driver assistance

01:43:12.580 | will eventually lead to allowing you

01:43:15.940 | to have a fully autonomous vehicle.

01:43:17.860 | Okay.

01:43:18.700 | There's an underlying deep philosophical question there,

01:43:20.780 | technical question of how much of driving can be learned.

01:43:25.780 | So LiDAR is an effective tool today

01:43:29.420 | for actually deploying a successful service in Phoenix,

01:43:33.140 | right, that's safe, that's reliable, et cetera, et cetera.

01:43:36.060 | But the question,

01:43:39.380 | and I'm not saying you can't do machine learning on LiDAR,

01:43:41.700 | but the question is that like how much of driving

01:43:45.340 | can be learned eventually?

01:43:47.420 | Can we do fully autonomous that's learned?

01:43:50.140 | - Yeah.

01:43:51.580 | You know, learning is all over the place

01:43:53.420 | and plays a key role in every part of our system.

01:43:56.740 | I, as you said, I would decouple the sensing modalities

01:44:01.140 | from the ML and the software parts of it.

01:44:05.860 | LiDAR, radar, cameras, it's all machine learning.

01:44:09.780 | All of the object detection classification, of course,

01:44:11.860 | like that's what these modern deep nets

01:44:14.220 | and con nets are very good at.

01:44:15.700 | You feed them raw data, massive amounts of raw data.

01:44:18.900 | And that's actually what our custom build LiDARs

01:44:22.780 | and radars are really good at.

01:44:24.220 | And radars, they don't just give you point estimates

01:44:25.980 | of objects in space,

01:44:26.980 | they give you raw, like physical observations.

01:44:29.620 | And then you take all of that raw information,

01:44:31.780 | you know, there's colors of the pixels,

01:44:33.420 | whether it's LiDARs returns,

01:44:34.860 | it's some auxiliary information,

01:44:35.780 | it's not just distance, right?

01:44:37.140 | And angle and distance is much richer information

01:44:39.220 | that you get from those returns,

01:44:40.500 | plus really rich information from the radars.

01:44:43.060 | You fuse it all together and you feed it

01:44:44.620 | into those massive ML models that then, you know,

01:44:48.460 | lead to the best results in terms of, you know,

01:44:51.540 | object detection, classification, you know, state estimation.

01:44:55.940 | - So there's a, sorry to interrupt, but there is a fusion.

01:44:58.540 | I mean, that's something that people didn't do

01:44:59.820 | for a very long time,

01:45:01.140 | which is like at the sensor fusion level, I guess,

01:45:04.620 | like early on fusing the information together,

01:45:07.100 | whether so that the sensory information

01:45:10.780 | that the vehicle receives from the different modalities

01:45:12.940 | or even from different cameras is combined

01:45:15.900 | before it is fed into the machine learning models.

01:45:18.460 | - Yeah, so I think this is one of the trends.

01:45:20.860 | You're seeing more of that, you mentioned end-to-end,

01:45:22.700 | there's different interpretation of end-to-end.

01:45:24.580 | There is kind of the purest interpretation

01:45:27.620 | of I'm gonna like have one model

01:45:29.660 | that goes from raw sensor data to like, you know,

01:45:32.580 | steering torque and, you know, gas brakes.

01:45:34.860 | That, you know, that's too much.

01:45:36.060 | I don't think that's the right way to do it.

01:45:37.580 | There's more, you know, smaller versions of end-to-end

01:45:40.700 | where you're kind of doing more end-to-end learning

01:45:44.580 | or core training or deep propagation

01:45:46.540 | of kind of signals back and forth

01:45:48.540 | across the different stages of your system.

01:45:50.500 | There's, you know, really good ways.

01:45:51.900 | It gets into some fairly complex design choices

01:45:54.900 | where on one hand you want modularity

01:45:56.420 | and the composability of your system.

01:46:00.220 | But on the other hand,

01:46:01.740 | you don't wanna create interfaces that are too narrow

01:46:04.220 | or too brittle, too engineered,

01:46:05.900 | where you're giving up on the generality of a solution

01:46:08.060 | or you're unable to properly propagate signal,

01:46:10.460 | you know, reach signal forward and losses and, you know,

01:46:14.220 | back so you can optimize the whole system jointly.

01:46:17.580 | So I would decouple.

01:46:18.820 | And I guess what you're seeing in terms of the fusion

01:46:21.220 | of the sensing data from different modalities,

01:46:24.100 | as well as kind of fusion in the temporal level,

01:46:26.940 | going more from, you know, frame by frame,

01:46:29.260 | where, you know, you would have one net

01:46:31.580 | that would do frame by frame detection in camera.

01:46:33.260 | And then, you know, something that does frame by frame

01:46:35.140 | and lighter and then radar,

01:46:36.580 | and then you fuse it, you know,

01:46:38.100 | in a weaker engineered way later.

01:46:39.740 | Like the field over the last decade has been evolving

01:46:42.460 | in more kind of joint fusion, more end-to-end models

01:46:45.220 | that are solving some of these tasks, you know, jointly.

01:46:47.740 | And there's tremendous power in that.

01:46:49.340 | And, you know, that's the progression

01:46:51.540 | that kind of our stack has been on as well.

01:46:54.740 | Now, to your, you know,

01:46:55.940 | so I would decouple the sensing

01:46:57.820 | and how that information is used

01:46:58.980 | from the role of ML in the entire stack.

01:47:01.420 | And, you know, I guess it's,

01:47:02.860 | there's trade-offs in modularity

01:47:06.300 | and how do you inject inductive bias into your system?

01:47:11.300 | Right, this is, there's tremendous power

01:47:15.260 | in being able to do that.

01:47:16.700 | So, you know, we have, there's no part of our system

01:47:20.740 | that is not heavily, that does not heavily, you know,

01:47:24.300 | leverage data-driven development or, you know,

01:47:28.260 | state-of-the-art ML.

01:47:29.660 | But there's mapping, there's a simulator,

01:47:32.020 | or there's perception, you know, object level,

01:47:34.180 | you know, perception, whether it's semantic understanding,

01:47:35.980 | prediction, decision-making, you know, so forth and so on.

01:47:38.980 | It's, and of course, object detection and classification,

01:47:44.940 | like you're finding pedestrians and cars and cyclists

01:47:47.100 | and, you know, cones and signs and vegetation

01:47:49.780 | and being very good at estimating

01:47:51.900 | kind of detection classification and state estimation,

01:47:54.020 | there's just stable stakes.

01:47:55.100 | Like that's step zero of this whole stack.

01:47:57.500 | You can be incredibly good at that,

01:47:59.420 | whether you use cameras or light as a radar,

01:48:00.980 | but that's just, you know, that's stable stakes.

01:48:02.540 | That's just step zero.

01:48:03.740 | Beyond that, you get into the really interesting challenges

01:48:05.780 | of semantic understanding, the perception level.

01:48:08.220 | You get into scene level reasoning.

01:48:10.140 | You get into very deep problems

01:48:12.180 | that have to do with prediction and joint prediction

01:48:14.420 | and interaction, so on and so on,

01:48:16.220 | between all of the actors in the environment,

01:48:18.420 | pedestrians, cyclists, other cars,

01:48:19.780 | and you get into decision-making, right?

01:48:21.540 | So how do you build a lot of systems?

01:48:22.980 | So we leverage ML very heavily in all of these components.

01:48:27.340 | I do believe that the best results you achieve

01:48:30.700 | by kind of using a hybrid approach

01:48:33.140 | and having different types of ML,

01:48:35.140 | having different models with different degrees

01:48:40.500 | of inductive bias that you can have,

01:48:43.020 | and combining kind of model-free approaches

01:48:46.300 | with some model-based approaches

01:48:48.140 | and some rule-based, physics-based systems.

01:48:53.140 | So, you know, one example I can give you is traffic lights.

01:48:56.180 | There's a problem of the detection of traffic light state,

01:48:59.780 | and obviously that's a great problem

01:49:01.740 | for computer vision confidence.

01:49:04.580 | That's their bread and butter, right?

01:49:06.740 | That's how you build that.

01:49:07.860 | But then the interpretation of a traffic light,

01:49:11.540 | that you're gonna need to learn that, right?

01:49:14.260 | Red, you don't need to build some complex ML model

01:49:17.140 | that infers with some precision and recall

01:49:21.300 | that red means stop.

01:49:22.900 | It's a very clear engineered signal

01:49:25.620 | with very clear semantics, right?

01:49:27.740 | So you wanna induce that bias.

01:49:29.020 | Like how you induce that bias and that,

01:49:31.100 | whether it's a constraint or a cost function in your stack,

01:49:35.820 | but it is important to be able to inject

01:49:38.700 | that clear semantic signal into your stack.

01:49:42.380 | And that's what we do.

01:49:43.540 | But then the question of like,

01:49:46.260 | and that's when you apply it to yourself,

01:49:49.060 | when you are making decisions,

01:49:50.340 | whether you wanna stop for a red light or not.

01:49:52.580 | But if you think about how other people treat traffic lights,

01:49:57.980 | we're back to the ML version of that.

01:49:59.940 | As you know, they're supposed to stop for a red light,

01:50:01.940 | but that doesn't mean they will.

01:50:03.020 | So then you're back in the like very heavy ML domain

01:50:06.820 | where you're picking up on like very subtle keys about,

01:50:11.540 | you know, that have to do with the behavior of objects

01:50:13.780 | and pedestrians, cyclists, cars,

01:50:16.140 | and the whole thing, you know,

01:50:18.060 | entire configuration of the scene

01:50:19.580 | that allow you to make accurate predictions

01:50:21.380 | on whether they will in fact stop or run a red light.

01:50:24.260 | - So it sounds like already for Waymo,

01:50:27.140 | like machine learning is a huge part of the stack.

01:50:29.900 | So it's a huge part of like, not just,

01:50:33.260 | so obviously the first level zero or whatever you said,

01:50:37.620 | which is like just the object detection

01:50:39.940 | and things that, you know,

01:50:40.780 | with know that machine learning can do,

01:50:42.700 | but also starting to do prediction behavior and so on

01:50:46.260 | to model what other parties in the scene,

01:50:50.020 | entities in the scene are gonna do.

01:50:51.700 | So machine learning is more and more

01:50:53.980 | playing a role in that as well.

01:50:56.220 | - Of course, absolutely.

01:50:57.900 | I think we've been going back to the earliest days,

01:51:01.220 | like, you know, the DARPA Urban,

01:51:03.100 | the DARPA Grand Challenge,

01:51:04.340 | and team was leveraging, you know, machine learning.

01:51:06.380 | I was like pre, you know, ImageNet,

01:51:08.140 | and it was a very different type of ML,

01:51:09.820 | but, and I think actually it was before my time,

01:51:12.220 | but the Stanford team on during the Grand Challenge

01:51:14.980 | had a very interesting machine learned system

01:51:17.460 | that would, you know, use lighter and camera,

01:51:20.780 | we've been driving in the desert,

01:51:21.820 | and it, we had built the model

01:51:24.620 | where it would kind of extend the range

01:51:28.020 | of free space reasoning.

01:51:29.700 | So we get a clear signal from lighter,

01:51:31.660 | and then it had a model that said,

01:51:32.620 | "Hey, like this stuff in camera

01:51:33.820 | kind of sort of looks like this stuff in lighter.

01:51:35.820 | And I know this stuff that I've seen in lighter,

01:51:37.660 | I'm very confident that it's free space.

01:51:38.980 | So let me extend that free space zone

01:51:41.660 | into the camera range

01:51:43.100 | that would allow the vehicle to drive faster."

01:51:44.700 | And then we've been building on top of that

01:51:46.260 | and kind of staying and pushing the state of the art in ML,

01:51:49.020 | in all kinds of different ML over the years.

01:51:51.740 | And in fact, from the earliest days,

01:51:53.500 | I think, you know, 2010 is probably the year

01:51:56.900 | where Google, maybe 2011 probably,

01:52:00.060 | got pretty heavily involved in machine learning,

01:52:04.900 | kind of deep nuts.

01:52:06.420 | And at that time, it was probably the only company

01:52:08.140 | that was very heavily investing in kind of state of the art ML

01:52:11.940 | and self-driving cars, right?

01:52:13.300 | And they go hand in hand.

01:52:16.340 | And we've been on that journey ever since.

01:52:18.540 | We're doing, pushing a lot of these areas

01:52:22.220 | in terms of research at Waymo,

01:52:23.980 | and we collaborate very heavily

01:52:25.500 | with the researchers in Alphabet.

01:52:27.980 | And I call all kinds of ML,

01:52:29.460 | supervised ML, unsupervised ML,

01:52:31.540 | published some interesting research papers in the space,

01:52:37.860 | especially recently.

01:52:39.420 | It's just super active. - Super active learning as well.

01:52:41.300 | - Yeah, so super, super active.

01:52:43.020 | And of course there's kind of the more mature stuff

01:52:46.260 | like, you know, ConvNets for object detection,

01:52:49.100 | but there's some really interesting,

01:52:50.380 | really active work that's happening

01:52:52.860 | and kind of more, you know, and bigger models

01:52:57.580 | and models that have more structure to them,

01:53:02.580 | you know, not just large bitmaps

01:53:04.420 | and reason about temporal sequences.

01:53:06.780 | And some of the interesting breakthroughs

01:53:09.340 | that we've seen in language models, right?

01:53:12.860 | You know, transformers, you know, GPT-3 inference.

01:53:16.300 | There's some really interesting applications

01:53:18.980 | of some of the core breakthroughs

01:53:20.140 | to those problems of, you know, behavior prediction,

01:53:23.020 | as well as, you know, decision-making and planning, right?

01:53:25.180 | You can think about it, kind of the behavior,

01:53:27.780 | how, you know, the path, the trajectories,

01:53:29.700 | the how people drive,

01:53:31.500 | and they have kind of a share

01:53:33.020 | a lot of the fundamental structure, you know, this problem.

01:53:35.900 | There's, you know, sequential nature.

01:53:38.860 | There's a lot of structure in this representation.

01:53:41.980 | There is a strong locality, kind of like in sentences,

01:53:45.140 | you know, words that follow each other,

01:53:46.540 | they're strongly connected,

01:53:47.780 | but there are also kind of larger contexts

01:53:49.940 | that doesn't have that locality.

01:53:51.100 | And you also see that in driving, right?

01:53:52.340 | What, you know, is happening in the scene as a whole

01:53:54.700 | has very strong implications on, you know,

01:53:58.540 | the kind of the next step in that sequence

01:54:00.620 | where whether you're predicting

01:54:02.460 | what other people are going to do,

01:54:03.820 | whether you're making your own decisions,

01:54:05.540 | or whether in the simulator,

01:54:06.820 | you're building generative models of, you know,

01:54:09.580 | humans walking, cyclists riding, and other cars driving.

01:54:11.860 | - Oh, that's all really fascinating.

01:54:13.420 | Like how it's fascinating to think that

01:54:15.660 | transformer models and all the breakthroughs in language

01:54:19.500 | and NLP that might be applicable to like driving

01:54:22.380 | at the higher level, at the behavior level.

01:54:24.260 | That's kind of fascinating.

01:54:25.980 | Let me ask about pesky little creatures

01:54:27.940 | called pedestrians and cyclists.

01:54:30.260 | They seem, so humans are a problem.

01:54:32.540 | If we can get rid of them, I would.

01:54:34.300 | But unfortunately, they're also a source of joy

01:54:37.700 | and love and beauty, so let's keep them around.

01:54:40.060 | - They're also our customers.

01:54:41.540 | - Oh, for your perspective, yes, yes, for sure.

01:54:44.060 | They're a source of money, very good.

01:54:48.300 | But I don't even know where I was going.

01:54:52.060 | Oh yes, pedestrians and cyclists.

01:54:54.100 | I, you know, they're a fascinating injection

01:54:59.380 | into the system of uncertainty,

01:55:01.500 | of like a game theoretic dance of what to do.

01:55:06.500 | And also they have perceptions of their own

01:55:11.740 | and they can tweet about your product.

01:55:14.980 | So you don't want to run them over.

01:55:17.620 | - From that perspective, I mean, I don't know,

01:55:20.420 | I'm joking a lot, but I think in seriousness,

01:55:24.180 | like, you know, pedestrians are a complicated,

01:55:26.580 | computer vision problem, a complicated behavioral problem.

01:55:30.860 | Is there something interesting you could say

01:55:32.380 | about what you've learned

01:55:34.300 | from a machine learning perspective,

01:55:36.220 | from also an autonomous vehicle and a product perspective

01:55:40.060 | about just interacting with the humans in this world?

01:55:43.060 | - Yeah, just, you know, to state on record,

01:55:44.980 | we care deeply about the safety of pedestrians,

01:55:47.140 | you know, even the ones that don't have Twitter accounts.

01:55:50.300 | - Thank you.

01:55:51.140 | (laughs)

01:55:51.980 | All right, all right, cool.

01:55:53.020 | Not me.

01:55:53.860 | But yes, I'm glad somebody does.

01:55:57.620 | Okay.

01:55:58.460 | - But you know, in all seriousness,

01:56:00.420 | safety of vulnerable road users,

01:56:04.740 | pedestrians or cyclists is one of our highest priorities.

01:56:08.780 | We do a tremendous amount of testing and validation

01:56:13.780 | and put a very significant emphasis

01:56:16.260 | on the capabilities of our systems

01:56:18.700 | that have to do with safety

01:56:20.620 | around those unprotected vulnerable road users.

01:56:22.980 | You know, cars, as we discussed earlier in Phoenix,

01:56:27.180 | we have completely empty cars,

01:56:28.300 | completely driverless cars,

01:56:29.740 | driving in this very large area.

01:56:32.220 | And you know, some people use them to go to school,

01:56:34.780 | so they will drive through school zones, right?

01:56:36.540 | So kids are kind of the very special class

01:56:40.020 | of those vulnerable road users, right?

01:56:41.740 | You wanna be super, super safe

01:56:44.420 | and super, super cautious around those.

01:56:45.860 | So we take it very, very, very seriously.

01:56:48.380 | And you know, what does it take to be good at it?

01:56:53.060 | An incredible amount of performance

01:56:59.460 | across your whole stack.

01:57:01.700 | You know, it starts with hardware.

01:57:04.220 | And again, you wanna use all sensing modalities

01:57:06.860 | available to you.

01:57:07.700 | Imagine driving on a residential road at night

01:57:10.220 | and kind of making a turn

01:57:11.380 | and you don't have headlights covering some part

01:57:14.180 | of the space and like, you know, a kid might run out.

01:57:17.020 | And you know, lighters are amazing at that.

01:57:19.980 | They see just as well in complete darkness

01:57:22.980 | as they do during the day, right?

01:57:24.140 | So just again, it gives you that extra,

01:57:26.220 | you know, margin in terms of capability

01:57:31.700 | and performance and safety and quality.

01:57:33.700 | And in fact, we oftentimes, in these kinds of situations,

01:57:36.340 | we have our system detect something,

01:57:38.620 | in some cases even earlier than our trained operators

01:57:41.380 | in the car might do, right?

01:57:42.420 | Especially in conditions like very dark nights.

01:57:45.660 | So it starts with sensing.

01:57:48.220 | Then, you know, perception has to be incredibly good.

01:57:52.940 | And you have to be very, very good

01:57:54.300 | at kind of detecting pedestrians

01:57:57.700 | in all kinds of situations

01:58:00.500 | and all kinds of environments,

01:58:01.500 | including people in weird poses,

01:58:03.700 | people kind of running around

01:58:06.060 | and being partially occluded.

01:58:09.940 | So, you know, that's step number one.

01:58:13.140 | Then you have to have very high accuracy

01:58:17.620 | and very low latency in terms of your reactions

01:58:21.260 | to what these actors might do, right?

01:58:26.060 | And we've put a tremendous amount of engineering

01:58:28.940 | and tremendous amount of validation

01:58:30.620 | in to make sure our system performs properly.

01:58:33.780 | And oftentimes it does require a very strong reaction

01:58:36.820 | to do the safe thing.

01:58:37.860 | And we actually see a lot of cases like that.

01:58:39.540 | It's the long tail of really rare,

01:58:41.980 | you know, really crazy events

01:58:45.500 | that contribute to the safety around pedestrians.

01:58:49.580 | Like one example that comes to mind

01:58:51.540 | that we actually got happened in Phoenix,

01:58:54.140 | where we were driving along

01:58:57.700 | and I think it was a 45 mile per hour road.

01:58:59.780 | So you have pretty high speed traffic

01:59:01.220 | and there was a sidewalk next to it.

01:59:03.460 | And there was a cyclist on the sidewalk.

01:59:05.580 | And as we were in the right lane,

01:59:09.180 | right next to the sidewalk, it was a multi-lane road.

01:59:11.780 | So as we got close to the cyclist on the sidewalk,

01:59:15.460 | it was a woman, she tripped and fell,

01:59:17.300 | just fell right into the path of our vehicle.

01:59:19.780 | And our car, you know, this was actually with a test driver.

01:59:26.180 | Our test drivers did exactly the right thing.

01:59:29.900 | They kind of reacted and came to a stop.

01:59:31.500 | It requires both very strong steering

01:59:33.140 | and strong application of the brake.

01:59:35.900 | And then we simulated what our system would have done

01:59:37.860 | in that situation and it did exactly the same thing.

01:59:40.420 | And that speaks to all of those components

01:59:44.060 | of really good state estimation and tracking.

01:59:47.060 | And like imagine a person on a bike

01:59:49.700 | and they're falling over

01:59:50.900 | and they're doing that right in front of you.

01:59:52.340 | So you have to be really like, things are changing.

01:59:53.700 | The appearance of that whole thing is changing.

01:59:56.100 | And a person goes one way, they're falling on the road,

01:59:58.220 | they're being flat on the ground in front of you.

02:00:01.300 | The bike goes flying the other direction.

02:00:03.460 | Like the two objects that used to be one

02:00:04.860 | are now are splitting apart.

02:00:06.940 | And the car has to like detect all of that.

02:00:09.100 | Like milliseconds matter.

02:00:10.380 | And it's not good enough to just brake.

02:00:12.900 | You have to like steer and brake

02:00:14.380 | and there's traffic around you.

02:00:15.700 | So like it all has to come together.

02:00:17.580 | And it was really great to see in this case

02:00:20.020 | and other cases like that,

02:00:21.260 | that we're actually seeing in the wild

02:00:22.740 | that our system is performing exactly the way

02:00:26.100 | that we would have liked

02:00:27.860 | and is able to avoid collisions like this.

02:00:30.820 | - It's such an exciting space for robotics.

02:00:32.820 | Like in that split second

02:00:35.340 | to make decisions of life and death.

02:00:37.620 | I don't know.

02:00:38.620 | The stakes are high in a sense,

02:00:40.180 | but it's also beautiful that

02:00:42.140 | for somebody who loves artificial intelligence,

02:00:45.700 | the possibility that an AI system

02:00:47.460 | might be able to save a human life.

02:00:49.220 | That's kind of exciting as a problem, like to wake up.

02:00:54.180 | It's terrifying probably for an engineer to wake up

02:00:57.540 | and to think about, but it's also exciting

02:01:00.380 | 'cause it's in your hands.

02:01:02.940 | Let me try to ask a question

02:01:04.220 | that's often brought up about autonomous vehicles.

02:01:07.500 | And it might be fun to see

02:01:09.100 | if you have anything interesting to say,

02:01:11.500 | which is about the trolley problem.

02:01:13.300 | So a trolley problem is a interesting philosophical construct

02:01:19.220 | that highlights, and there's many others like it,

02:01:23.380 | of the difficult ethical decisions

02:01:25.940 | that we humans have before us

02:01:29.740 | in this complicated world.

02:01:31.540 | So specifically is the choice between

02:01:35.300 | if you were forced to choose to kill a group X of people

02:01:40.300 | versus a group Y of people, like one person.

02:01:44.060 | If you did nothing, you would kill one person,

02:01:46.860 | but if you would kill five people,

02:01:49.740 | and if you decide to swerve out of the way,

02:01:51.340 | you would only kill one person.

02:01:53.180 | Do you do nothing or you choose to do something?

02:01:55.660 | And you can construct all kinds of sort of

02:01:58.100 | ethical experiments of this kind that,

02:02:01.100 | I think at least on a positive note,

02:02:05.460 | inspire you to think about, like introspect

02:02:09.580 | what are the physics of our morality?

02:02:14.580 | And there's usually not good answers there.

02:02:18.220 | I think people love it

02:02:19.620 | 'cause it's just an exciting thing to think about.

02:02:22.060 | I think people who build autonomous vehicles

02:02:24.620 | usually roll their eyes

02:02:27.180 | because this is not,

02:02:29.940 | this one as constructed,

02:02:32.060 | this like literally never comes up in reality.

02:02:35.140 | You never have to choose between killing

02:02:37.180 | (laughs)

02:02:38.260 | one or like one of two groups of people.

02:02:41.260 | But I wonder if you can speak to,

02:02:45.300 | is there something interesting

02:02:48.820 | to you as an engineer of autonomous vehicles

02:02:51.460 | that's within the trolley problem,

02:02:53.500 | or maybe more generally,

02:02:54.940 | are there difficult ethical decisions

02:02:58.020 | that you find that a algorithm must make?

02:03:00.940 | - On the specific version of the trolley problem,

02:03:03.460 | which one would you do?

02:03:05.260 | If you're driving?

02:03:06.780 | - The question itself is a profound question

02:03:09.580 | because we humans ourselves cannot answer it.

02:03:12.020 | And that's the very point.

02:03:13.860 | I would kill both.

02:03:16.020 | (laughs)

02:03:18.100 | - Humans, I think you're exactly right

02:03:20.420 | in that humans are not particularly good.

02:03:22.180 | I think the kind of phrase is like,

02:03:23.580 | what would a computer do?

02:03:24.500 | But humans are not very good.

02:03:27.220 | And actually oftentimes I think that freezing

02:03:30.820 | and kind of not doing anything

02:03:32.060 | because like you've taken a few extra milliseconds

02:03:34.580 | to just process,

02:03:35.420 | and then you end up like doing the worst

02:03:37.180 | of possible outcomes, right?

02:03:38.260 | So I do think that as you've pointed out,

02:03:41.260 | it can be a bit of a distraction

02:03:43.420 | and it can be a bit of a kind of a red herring.

02:03:45.620 | I think it's an interesting discussion

02:03:47.940 | in the realm of philosophy, right?

02:03:50.740 | But in terms of what,

02:03:52.540 | how that affects the actual engineering

02:03:55.900 | and deployment of self-driving vehicles,

02:03:57.900 | it's not how you go about building a system, right?

02:04:02.380 | We've talked about how you engineer a system,

02:04:04.580 | how you go about evaluating the different components

02:04:07.580 | and the safety of the entire thing.

02:04:09.860 | How do you kind of inject the various model-based,

02:04:14.540 | safety-based,

02:04:15.620 | or I'm gonna say like,

02:04:16.460 | yes, you reason at parts of the system,

02:04:18.660 | you reason about the probability of a collision,

02:04:21.940 | the severity of that collision, right?

02:04:24.260 | And that is incorporated,

02:04:26.020 | and you have to properly reason about the uncertainty

02:04:28.180 | that flows through the system, right?

02:04:29.300 | So those factors definitely play a role

02:04:34.140 | in how the cars then behave,

02:04:35.820 | but they tend to be more of like the emergent behavior.

02:04:38.300 | And what you see, like you're absolutely right,

02:04:39.900 | that these clear theoretical problems that they,

02:04:43.780 | you don't have a car that in system,

02:04:46.060 | and really kind of being back to our previous discussion,

02:04:48.500 | of like, which one do you choose?

02:04:51.460 | Well, oftentimes you made a mistake earlier,

02:04:55.660 | like you shouldn't be in that situation

02:04:57.500 | in the first place, right?

02:04:58.700 | And in reality, the system comes up.

02:05:00.740 | If you build a very good, safe and capable driver,

02:05:03.900 | you have enough clues in the environment

02:05:07.500 | that you drive defensively,

02:05:09.460 | so you don't put yourself in that situation, right?

02:05:11.340 | And again, if you go back to that analogy

02:05:14.100 | of precision and recall,

02:05:15.180 | like, okay, you can make a very hard trade-off,

02:05:17.740 | but neither answer is really good.

02:05:19.540 | But what instead you focus on

02:05:21.500 | is kind of moving the whole curve up,

02:05:24.020 | and then you focus on building the right capability

02:05:26.020 | and the right defensive driving

02:05:27.420 | so that you don't put yourself in a situation like this.

02:05:30.180 | - I don't know if you have a good answer for this,

02:05:33.140 | but people love it when I ask this question about books.

02:05:36.580 | Are there books in your life

02:05:41.100 | that you've enjoyed, philosophical, fiction, technical,

02:05:44.700 | that had a big impact on you as an engineer

02:05:47.100 | or as a human being?

02:05:48.940 | You know, everything from science fiction

02:05:50.460 | to a favorite textbook.

02:05:51.980 | Is there three books that stand out that you can think of?

02:05:55.620 | - Three books.

02:05:56.460 | So I would, you know, that impacted me,

02:05:59.460 | I would say,

02:06:00.820 | this one is, you probably know it well,

02:06:06.500 | but not generally well known,

02:06:09.660 | I think in the US or kind of internationally,

02:06:12.300 | "The Master and Margarita."

02:06:14.380 | It's one of, actually, my favorite books.

02:06:19.320 | It is by a Russian,

02:06:21.520 | it's a novel by Russian author Mikhail Bulgakov.

02:06:25.240 | And it's just, it's a great book.

02:06:26.840 | And it's one of those books

02:06:27.680 | that you can like reread your entire life,

02:06:29.680 | and it's very accessible.

02:06:31.040 | You can read it as a kid.

02:06:32.400 | And like, it's, you know, the plot is interesting.

02:06:35.280 | It's, you know, the devil, you know,

02:06:36.800 | visiting the Soviet Union.

02:06:38.280 | But it, like, you read it,

02:06:41.040 | reread it at different stages of your life,

02:06:43.760 | and you enjoy it for different, very different reasons.

02:06:47.960 | And you keep finding like deeper and deeper meaning.

02:06:50.600 | And, you know, it kind of affected,

02:06:52.000 | you know, it had a,

02:06:52.840 | definitely had an imprint on me,

02:06:55.080 | mostly from the,

02:06:57.720 | probably kind of the cultural stylistic aspect.

02:07:00.240 | Like it makes you,

02:07:01.080 | it's one of those books that, you know,

02:07:02.640 | is good and makes you think,

02:07:03.480 | but also has like this really, you know,

02:07:05.640 | silly, quirky, dark sense of, you know, humor.

02:07:08.160 | - It captures the Russian soul

02:07:09.640 | more than perhaps many other books.

02:07:11.640 | On that like slight note, just out of curiosity,

02:07:14.440 | one of the saddest things is I've read that book

02:07:17.160 | in English.

02:07:18.320 | Did you by chance read it in English or in Russian?

02:07:22.600 | - In Russian, only in Russian.

02:07:24.000 | And I actually, that is a question I had.

02:07:26.160 | Kind of posed to myself every once in a while.

02:07:29.960 | Like I wonder how well it translates,

02:07:31.680 | if it translates at all.

02:07:33.040 | And there's the language aspect of it,

02:07:34.360 | and then there's the cultural aspect.

02:07:35.680 | So I, and actually I'm not sure if, you know,

02:07:37.760 | either of those would work well in English.

02:07:40.840 | - Now I forget their names,

02:07:42.200 | but so when the COVID lifts a little bit,

02:07:44.160 | I'm traveling to Paris

02:07:46.680 | for several reasons.

02:07:48.280 | One is just, I've never been to Paris.

02:07:49.600 | I want to go to Paris,

02:07:50.440 | but there's the most famous translators

02:07:54.400 | of Dostoevsky, Tolstoy, of most of Russian literature

02:07:58.440 | live there.

02:07:59.280 | There's a couple, they're famous, a man and a woman.

02:08:01.680 | And I'm going to sort of have a series of conversations

02:08:03.960 | with them.

02:08:04.800 | And in preparation for that,

02:08:05.920 | I'm starting to read Dostoevsky in Russian.

02:08:08.200 | So I'm really embarrassed to say that I've read this,

02:08:10.720 | everything I've read in Russian literature

02:08:12.800 | of like serious depth has been in English,

02:08:16.760 | even though I can also read, I mean, obviously in Russian,

02:08:21.520 | but for some reason it seemed,

02:08:24.800 | in the optimization of life,

02:08:29.040 | it seemed the improper decision to do it,

02:08:31.040 | to read in Russian.

02:08:32.200 | Like, you know, like I don't need to opt,

02:08:35.080 | I need to think in English, not in Russian,

02:08:37.200 | but now I'm changing my mind on that.

02:08:38.960 | And so the question of how well it translate

02:08:41.160 | is a really fundamental one.

02:08:42.360 | Like it, even with Dostoevsky.

02:08:44.120 | So from what I understand, Dostoevsky translates easier.

02:08:47.000 | Others don't as much.

02:08:49.680 | Obviously the poetry doesn't translate as well.

02:08:52.520 | I'm also the music, big fan of Vladimir Vysotsky.

02:08:57.520 | He doesn't obviously translate well.

02:09:00.320 | People have tried.

02:09:01.320 | But mastermind, I don't know.

02:09:04.480 | I don't know about that one.

02:09:05.640 | I just know it in English, you know,

02:09:06.960 | it's fun as hell in English.

02:09:08.520 | So, but it's a curious question

02:09:11.320 | and I wanna study it rigorously

02:09:13.000 | from both the machine learning aspect

02:09:15.480 | and also because I want to do a couple of interviews

02:09:17.880 | in Russia that I'm still unsure

02:09:22.880 | of how to properly conduct an interview

02:09:27.160 | across a language barrier.

02:09:28.760 | It's a fascinating question that ultimately communicates

02:09:31.880 | to an American audience.

02:09:32.880 | There's a few Russian people that I think

02:09:36.880 | are truly special human beings.

02:09:39.240 | And I feel like I sometimes encounter this

02:09:44.160 | with some incredible scientists

02:09:45.800 | and maybe you encounter this as well

02:09:48.960 | at some point in your life that it feels like

02:09:52.000 | because of the language barrier,

02:09:53.280 | their ideas are lost to history.

02:09:55.480 | It's a sad thing.

02:09:56.400 | I think about like Chinese scientists or even authors

02:10:00.040 | that like, that we don't in English speaking world

02:10:04.600 | don't get to appreciate some like the depth of the culture

02:10:07.440 | because it's lost in translation.

02:10:09.040 | And I feel like I would love to show that to the world.

02:10:13.240 | Like I'm just some idiot, but because I have this,

02:10:17.000 | like at least some semblance of skill in speaking Russian,

02:10:20.920 | I feel like, and I know how to record stuff

02:10:23.080 | on a video camera.

02:10:25.080 | I feel like I wanna catch like Grigori Pearlman,

02:10:27.480 | who's a mathematician.

02:10:28.480 | I'm not sure if you're familiar with him.

02:10:30.320 | I wanna talk to him, like he's a fascinating mind

02:10:32.720 | and to bring him to a wider audience in English speaking,

02:10:36.080 | it'll be fascinating.

02:10:36.960 | But that requires to be rigorous about this question

02:10:40.120 | of how well Bulgakov translates.

02:10:44.160 | I mean, I know it's a silly concept,

02:10:46.840 | but it's a fundamental one because how do you translate?

02:10:50.440 | And that's the thing that Google Translate is also facing

02:10:54.080 | as a more machine learning problem.

02:10:57.000 | But I wonder as a more bigger problem for AI,

02:11:00.600 | how do we capture the magic that's there in the language?

02:11:05.800 | - I think that's really interesting,

02:11:08.920 | really challenging problem.

02:11:10.600 | If you do read it, Master and Margarita in English,

02:11:14.160 | sorry, in Russian, I'd be curious to get your opinion.

02:11:18.440 | And I think part of it is language,

02:11:19.880 | but part of it's just centuries of culture

02:11:22.000 | that the cultures are different.

02:11:23.600 | So it's hard to connect that.

02:11:26.360 | - Okay, so that was my first one, right?

02:11:29.360 | You had two more.

02:11:30.880 | The second one, I would probably pick the science fiction

02:11:34.960 | by the Strogowski brothers.

02:11:37.280 | I know it's up there with Isaac Asimov

02:11:39.320 | and Ray Bradbury and company.

02:11:42.600 | The Strogowski brothers kind of appealed more to me.

02:11:46.680 | I think it made more of an impression on me growing up.

02:11:49.920 | - Can you, I apologize if I'm showing my complete ignorance.

02:11:55.080 | I'm so weak on sci-fi.

02:11:56.680 | What did they write?

02:11:57.920 | - Oh, "Roadside Picnic."

02:12:04.120 | "Hard to Be a God."

02:12:06.080 | "Beetle in an Ant Hill."

02:12:11.200 | "Monday Starts on Saturday."

02:12:13.920 | Like it's not just science fiction.

02:12:15.600 | It's also like has very interesting,

02:12:17.560 | interpersonal and societal questions.

02:12:20.280 | And some of the language is just completely hilarious.

02:12:25.280 | - (speaking in foreign language)

02:12:27.960 | - That's the one. - That's right.

02:12:29.080 | - Oh, interesting.

02:12:29.920 | "Monday Starts on Saturday."

02:12:31.160 | So I need to read, okay.

02:12:32.880 | - Oh boy.

02:12:34.040 | You put that in the category of science fiction?

02:12:36.680 | - That one is, I mean, this was more of a silly,

02:12:40.200 | humorous work.

02:12:41.760 | I mean, there is kind of--

02:12:43.240 | - But it's profound too, right?

02:12:44.640 | - Science fiction, right, is about this research institute.

02:12:47.600 | It has deep parallels to like serious research,

02:12:52.160 | but the setting of course is that they're working on magic.

02:12:56.120 | And there's a lot of, so I, and that's their style, right?

02:12:59.560 | They go, and other books are very different, right?

02:13:03.280 | "Hard to Be a God," right?

02:13:04.360 | It's about kind of this higher society being injected

02:13:07.120 | into this primitive world and how they operate there.

02:13:10.040 | Some of the very deep ethical questions there, right?

02:13:13.840 | And like they've got this full spectrum.

02:13:15.520 | Some is more about kind of more adventure style.

02:13:18.480 | But like I enjoy all of their books.

02:13:20.200 | There's probably a couple, actually one I think

02:13:22.560 | that they considered their most important work.

02:13:24.960 | I think it's "The Snail on a Hill."

02:13:28.560 | I don't know exactly how it translates.

02:13:30.200 | I tried reading a couple of times.

02:13:31.480 | I still don't get it, but everything else I fully enjoyed.

02:13:35.000 | And like for one of my birthdays as a kid,

02:13:36.520 | I got like their entire collection,

02:13:38.360 | like occupied a giant shelf in my room.

02:13:40.200 | And then I'll like over the holidays, I just like,

02:13:42.680 | my parents couldn't drag me out of the room

02:13:44.040 | and I read the whole thing cover to cover

02:13:45.840 | and I really enjoyed it.

02:13:48.880 | - And that's the one more, for the third one,

02:13:52.080 | maybe a little bit darker, but you know,

02:13:54.840 | comes to mind is Orwell's "1984."

02:13:58.600 | And you asked what made an impression on me

02:14:01.840 | and the books that people should read.

02:14:03.480 | That one I think falls in the category of both.

02:14:06.080 | Definitely it's one of those books that you read

02:14:07.960 | and you just kind of put it down

02:14:11.440 | and you stay in space for a while.

02:14:13.160 | That kind of work.

02:14:15.560 | I think there's lessons there people should not ignore.

02:14:20.560 | And nowadays with everything that's happening in the world,

02:14:25.800 | I can't help it, but have my mind jump to some parallels

02:14:30.800 | with what Orwell described.

02:14:32.520 | And like there's this whole concept of double think

02:14:36.520 | and ignoring logic and holding completely contradictory

02:14:39.640 | opinions in your mind and not have that not bother you

02:14:42.120 | and sticking to the party line at all costs.

02:14:45.000 | Like there's something there.

02:14:48.360 | - If anything, 2020 has taught me,

02:14:50.920 | and I'm a huge fan of Animal Farm,

02:14:52.640 | which is a kind of friendly, is a friend of "1984"

02:14:56.880 | by Orwell.

02:14:57.720 | It's kind of another thought experiment

02:15:01.600 | of how our society may go in directions

02:15:05.120 | that we wouldn't like it to go.

02:15:07.480 | But if anything that's been kind of heartbreaking

02:15:12.480 | to an optimist about 2020 is that society's kind of fragile.

02:15:21.000 | Like we have this, this is a special little experiment

02:15:24.600 | we have going on and it's not unbreakable.

02:15:28.960 | Like we should be careful to like preserve

02:15:32.360 | whatever the special thing we have going on.

02:15:34.680 | I mean, I think "1984" and these books, "Brave New World",

02:15:38.800 | they're helpful in thinking like stuff can go wrong

02:15:43.680 | in non-obvious ways.

02:15:45.360 | And it's like, it's up to us to preserve it.

02:15:48.480 | And it's like, it's a responsibility.

02:15:50.120 | It's been weighing heavy on me.

02:15:51.280 | 'Cause like, for some reason,

02:15:52.880 | like more than my mom follows me on Twitter.

02:15:57.480 | And I feel like I have like now somehow responsibility

02:16:01.000 | to this world.

02:16:05.440 | And it dawned on me that like me and millions of others

02:16:10.440 | had like the little ants that maintain this little colony.

02:16:14.760 | Right?

02:16:15.600 | So we have a responsibility not to be,

02:16:18.120 | I don't know what the right analogy is,

02:16:19.320 | but put a flamethrower to the place.

02:16:22.760 | We wanna not do that.

02:16:24.960 | And there's interesting, complicated ways of doing that

02:16:27.120 | as "1984" shows.

02:16:28.600 | It could be through bureaucracy,

02:16:29.800 | it could be through incompetence,

02:16:31.440 | it could be through misinformation,

02:16:33.320 | it could be through division and toxicity.

02:16:35.760 | I'm a huge believer in like that love

02:16:38.760 | will be the somehow the solution.

02:16:42.480 | So love and robots.

02:16:45.080 | (laughing)

02:16:45.920 | - Love and robots, yeah.

02:16:46.740 | I think you're exactly right.

02:16:47.580 | Unfortunately, I think it's less of a flamethrower

02:16:50.400 | type of an answer.

02:16:51.240 | I think it's more of a,

02:16:52.080 | in many cases can be more of a slow boil.

02:16:54.480 | And that's the danger.

02:16:55.680 | - Let me ask, it's a fun thing to make

02:17:00.320 | a world-class roboticist, engineer and leader

02:17:04.280 | uncomfortable with a ridiculous question about life.

02:17:07.680 | What is the meaning of life, Dmitry,

02:17:10.840 | from a robotics and a human perspective?

02:17:13.540 | - You only have a couple minutes,

02:17:16.120 | or one minute to answer.

02:17:17.360 | So.

02:17:18.200 | (laughing)

02:17:20.040 | - I don't know if that makes it more difficult

02:17:21.040 | or easier, actually.

02:17:22.040 | Yeah, you know, I'm very tempted to quote

02:17:26.040 | one of the stories by Isaac Asimov, actually.

02:17:33.360 | Actually titled, appropriately titled,

02:17:37.040 | "The Last Question," a short story,

02:17:39.360 | where the plot is that humans build this supercomputer,

02:17:43.000 | you know, this AI intelligence,

02:17:44.920 | and once it gets powerful enough,

02:17:48.000 | they pose this question to it.

02:17:49.720 | How can the entropy in the universe be reduced?

02:17:52.920 | Right, so the computer replies,

02:17:54.560 | hang on, as of yet, insufficient information

02:17:57.080 | to give a meaningful answer, right?

02:17:58.560 | And then, you know, thousands of years go by

02:18:00.400 | and they keep posing the same question.

02:18:01.640 | The computer gets more and more powerful

02:18:03.780 | and keeps giving the same answer.

02:18:05.560 | As of yet, insufficient information

02:18:07.280 | to give a meaningful answer,

02:18:08.400 | or something along those lines, right?

02:18:09.980 | And then, you know, it keeps happening

02:18:12.680 | and happening, you fast forward,

02:18:14.200 | like millions of years into the future

02:18:16.040 | and billions of years, and like at some point,

02:18:18.160 | it's just the only entity in the universe.

02:18:19.920 | It's like it's absorbed all humanity

02:18:21.680 | and all knowledge in the universe,

02:18:22.920 | and it keeps posing the same question to itself.

02:18:25.480 | And, you know, finally, it gets to the point

02:18:29.120 | where it is able to answer that question.

02:18:30.920 | But of course, at that point, you know,

02:18:32.280 | there's the heat death of the universe has occurred,

02:18:34.760 | and that's the only entity,

02:18:35.720 | and there's nobody else to provide that answer to.

02:18:38.360 | So the only thing it can do is to, you know,

02:18:40.520 | answer it by demonstration.

02:18:41.600 | So it like, you know, recreates the Big Bang, right?

02:18:44.320 | And resets the clock, right?

02:18:45.720 | (both laughing)

02:18:47.120 | But I can try to give kind of a different version

02:18:51.480 | of the answer, you know, maybe not on the behalf

02:18:54.880 | of all humanity.

02:18:55.720 | I think that might be a little presumptuous

02:18:57.480 | for me to speak about the meaning of life

02:18:59.220 | on the behalf of all humans,

02:19:01.480 | but at least, you know, personally, it changes, right?

02:19:05.040 | I think if you think about kind of what gives,

02:19:10.360 | you know, you and your life meaning and purpose

02:19:13.480 | and kind of what drives you,

02:19:14.880 | it seems to change over time, right?

02:19:19.840 | And the lifespan of, you know, kind of your existence,

02:19:24.840 | you know, when you just enter this world, right?

02:19:27.240 | It's all about kind of new experiences, right?

02:19:28.920 | You get like new smells, new sounds, new emotions, right?

02:19:33.240 | And like, that's what's driving you, right?

02:19:35.600 | You're experiencing new, amazing things, right?

02:19:38.280 | And that's magical, right?

02:19:39.560 | That's pretty awesome, right?

02:19:41.480 | That gives you kind of a meaning.

02:19:43.120 | Then, you know, you get a little bit older,

02:19:44.480 | you start more intentionally learning about things, right?

02:19:49.480 | I guess actually before you start intentionally learning,

02:19:51.480 | probably fun.

02:19:52.300 | Fun is a thing that gives you kind of meaning and purpose

02:19:54.680 | and the thing you optimize for, right?

02:19:56.920 | And like fun is good.

02:19:58.120 | Then you get, you know, start learning.

02:20:00.640 | And I guess that this joy of comprehension

02:20:05.720 | and discovery is another thing that, you know,

02:20:08.800 | gives you meaning and purpose and drives you, right?

02:20:11.440 | Then, you know, you learn enough stuff

02:20:14.120 | and you wanna give some of it back, right?

02:20:17.080 | And so impact and contributions back to technology

02:20:19.880 | or society, people, you know, local or more globally

02:20:24.880 | becomes a new thing that drives a lot of kind

02:20:27.720 | of your behavior and is something that gives you purpose

02:20:31.540 | and that you derive positive feedback from, right?

02:20:35.360 | You know, then you go and so on and so forth.

02:20:37.440 | You go through various stages of life.

02:20:39.400 | If you have kids, like that definitely changes

02:20:44.400 | your perspective on things.

02:20:45.880 | You know, I have three that definitely flips some bits

02:20:48.800 | in your head in terms of kind of what you care about

02:20:51.640 | and what you optimize for and, you know, what matters,

02:20:53.840 | what doesn't matter, right?

02:20:55.040 | So, you know, and so on and so forth, right?

02:20:57.240 | And it seems to me that, you know, it's all of those things.

02:21:01.840 | And as you kind of go through life, you know,

02:21:06.560 | you want these to be additive, right?

02:21:08.640 | New experiences, fun, learning, impact.

02:21:12.240 | Like you wanna, you know, be accumulating.

02:21:14.200 | I don't wanna, you know, stop having fun

02:21:16.320 | or experiencing new things.

02:21:17.760 | And I think it's important that, you know,

02:21:19.120 | just kind of becomes additive as opposed

02:21:22.080 | to a replacement or subtraction.

02:21:23.760 | But, you know, those few as far as I got,

02:21:27.000 | but, you know, ask me in a few years,

02:21:28.040 | I might have one or two more to add to the list.

02:21:30.320 | - And before you know it, time is up,

02:21:32.400 | just like it is for this conversation.

02:21:34.960 | But hopefully it was a fun ride.

02:21:37.000 | It was a huge honor to meet you.

02:21:38.200 | As you know, I've been a fan of yours

02:21:41.360 | and a fan of Google self-driving car

02:21:43.880 | and Waymo for a long time.

02:21:45.800 | I can't wait.

02:21:46.680 | I mean, it's one of the most exciting.

02:21:48.560 | If we look back in the 21st century,

02:21:50.240 | I truly believe it will be one of the most exciting things

02:21:52.520 | we descendants of apes have created on this earth.

02:21:56.240 | So I'm a huge fan and I can't wait to see what you do next.

02:22:01.040 | Thanks so much for talking to me.

02:22:02.440 | - Thanks.

02:22:03.280 | Thanks for having me.

02:22:04.120 | And it's also a huge fan.

02:22:05.800 | Doing work on this and I really enjoy this.

02:22:09.280 | Thank you.

02:22:10.600 | - Thanks for listening to this conversation

02:22:12.080 | with Dmitry Dolgov.

02:22:13.200 | And thank you to our sponsors.

02:22:14.880 | Trial Labs, a company that helps businesses

02:22:18.440 | apply machine learning to solve real world problems.

02:22:21.480 | Blinkist, an app I use for reading

02:22:23.280 | through summaries of books.

02:22:25.080 | BetterHelp, online therapy with a licensed professional

02:22:28.600 | and Cash App, the app I use to send money to friends.

02:22:31.840 | Please check out these sponsors in the description

02:22:34.000 | to get a discount and to support this podcast.

02:22:37.120 | If you enjoy this thing, subscribe on YouTube,

02:22:39.560 | review it with 5 Stars on Apple Podcasts,

02:22:41.720 | follow on Spotify, support on Patreon

02:22:44.280 | or connect with me on Twitter @LexFriedman.

02:22:47.160 | And now let me leave you with some words from Isaac Asimov.

02:22:51.480 | Science can amuse and fascinate us all,

02:22:54.360 | but it is engineering that changes the world.

02:22:57.080 | Thank you for listening and hope to see you next time.

02:23:00.800 | (upbeat music)

02:23:03.380 | (upbeat music)

02:23:05.960 | [BLANK_AUDIO]

Dmitri Dolgov: Waymo and the Future of Self-Driving Cars | Lex Fridman Podcast #147

Chapters