Oliver Cameron (CEO, Voyage) - MIT Self-Driving Cars

00:00:00.000 | All right, welcome back to 6.094,

00:00:02.520 | Deep Learning for Self-Driving Cars.

00:00:04.860 | Today we have Oliver Cameron.

00:00:06.560 | He's the co-founder and the CEO of Voyage.

00:00:10.560 | Before that, he was the lead of the

00:00:12.560 | Udacity Self-Driving Car Program

00:00:14.540 | that made ideas in autonomous vehicle research

00:00:17.120 | and development accessible to the entire world.

00:00:19.840 | He has a passion for the topic and a genuine open nature

00:00:23.960 | that makes him one of my favorite people in general

00:00:27.160 | and one of my favorite people working in this space.

00:00:29.300 | And I think thousands of people agree with that.

00:00:32.020 | So please give Oliver a warm welcome.

00:00:34.920 | (audience applauding)

00:00:38.080 | - Thank you very much, Lex.

00:00:40.240 | And thank you all for having me here today.

00:00:42.960 | Super excited to speak all about Voyage.

00:00:46.400 | But in reality, the kind of thing I wanna share today

00:00:50.000 | is kind of like this title says,

00:00:52.240 | how to start a self-driving car startup.

00:00:55.280 | Rarely do you kind of get an inside scoop

00:00:58.580 | of how a startup is formed.

00:01:00.480 | You kind of hear all the PR,

00:01:02.640 | all the kind of very lovey-dovey press releases out there.

00:01:07.640 | I wanna share kind of the inside of how

00:01:10.340 | at least Voyage came to be,

00:01:12.060 | which was a little unconventional

00:01:13.840 | compared to your average self-driving car startup.

00:01:17.040 | They always tell you that the path to a startup,

00:01:19.680 | getting to the goal you want, is kind of a zigzag.

00:01:22.240 | Ours was kind of a insane zigzag.

00:01:25.600 | So we'll go through all of that stuff.

00:01:28.340 | Let's talk about my background.

00:01:30.500 | Also a little unconventional.

00:01:32.500 | I'm not very good at learning in a classroom.

00:01:36.260 | For me, I found learning by doing, by building,

00:01:39.700 | has always been the thing that's worked best for me.

00:01:42.940 | So going all the way back to when I was a teenager,

00:01:46.620 | software just in general was my passion.

00:01:48.820 | This idea that you can make something

00:01:50.540 | out of absolutely nothing,

00:01:52.200 | and then all of a sudden, millions,

00:01:54.140 | and in Facebook's case, billions of people

00:01:56.580 | can be using that thing.

00:01:57.940 | And after building lots of crazy stuff,

00:02:02.820 | and perhaps not being too popular in high school

00:02:04.740 | because that's all I did,

00:02:06.820 | I started a company.

00:02:08.620 | I won't bore you with all the details,

00:02:10.140 | but learned a lot during that experience,

00:02:12.260 | and joined, went through Y Combinator,

00:02:15.860 | which I believe started right here in Cambridge,

00:02:17.660 | which is very cool.

00:02:19.300 | And then this very pivotal moment happened to me.

00:02:22.660 | I heard about this online class

00:02:24.800 | which was generating a whole bunch of scandal

00:02:27.620 | and lots of controversy,

00:02:30.900 | and it was from this guy called Sebastian Thrun.

00:02:34.740 | He'd taken this Stanford class he taught

00:02:37.740 | in artificial intelligence and just said,

00:02:39.500 | "Screw it, we're gonna put the whole thing online."

00:02:41.660 | And back then, and this was around 2011,

00:02:44.740 | this was a very controversial thing to do.

00:02:47.460 | Today, MIT and many others do this all the time,

00:02:50.820 | but back then, there was a hell of a lot of controversy

00:02:53.580 | around doing something like this.

00:02:55.740 | But this learning format really just appealed to me.

00:02:58.780 | Being able to sit in front of my laptop,

00:03:01.100 | learn at my own pace, build, build, build,

00:03:04.140 | was something that really resonated with me.

00:03:06.420 | And I took this class in 2013,

00:03:09.540 | artificial intelligence for robotics.

00:03:11.660 | And this again was just this pivotal moment.

00:03:13.540 | My head exploded, all the enthusiasm.

00:03:16.620 | I'd had the software kind of transferred

00:03:18.580 | to artificial intelligence and robotics,

00:03:22.520 | and just became addicted to the format

00:03:24.380 | of what are now called MOOCs,

00:03:26.340 | massively open online courses.

00:03:28.580 | And I loved them so much that I decided,

00:03:30.260 | hey, I wanna go do this and help others learn this stuff.

00:03:34.300 | So, hey, let's go join Udacity

00:03:35.980 | and build more classes like this.

00:03:39.380 | So I did that for four years,

00:03:41.540 | led our machine learning robotics,

00:03:43.080 | and eventually our self-driving car curriculum,

00:03:45.380 | which was a lot of fun.

00:03:47.860 | And I got to learn directly from two great company builders,

00:03:51.140 | like truly great company builders.

00:03:53.060 | One was Vishal Makhijani.

00:03:55.020 | He was the operator extraordinaire at Udacity,

00:03:58.940 | understood how to build a company,

00:04:00.900 | how to build a culture, how to incentivize,

00:04:03.620 | and how to do all those things

00:04:05.900 | that we don't often talk about, and Sebastian Thrun.

00:04:09.340 | He, of course, founded the Google Self-Driving Car Project

00:04:13.780 | in its early days,

00:04:15.540 | and right now I believe he's building flying cars.

00:04:19.460 | Just in general, I learned so much from him,

00:04:21.780 | but this idea that you are literally in control

00:04:24.380 | of your destiny, you can build absolutely anything

00:04:26.060 | if you put your mind to it,

00:04:27.580 | was always pretty inspirational.

00:04:30.060 | Today, of course, I build self-driving cars at Voyage,

00:04:33.700 | and we'll talk more about what makes us special

00:04:35.840 | compared to the other self-driving car companies

00:04:37.620 | you may have heard of in this class and beyond.

00:04:40.700 | Let's talk about Udacity.

00:04:42.100 | Can you raise your hand if you've heard of Udacity?

00:04:44.220 | Very curious.

00:04:45.060 | There you go, that's most of the room.

00:04:48.340 | Udacity, like I said, was founded by Sebastian Thrun.

00:04:50.900 | He took this class online and it all just exploded,

00:04:53.500 | and he built a company around it.

00:04:55.620 | Udacity's real focus is on increasing the world's GDP,

00:04:59.740 | this idea that talent is everywhere,

00:05:03.460 | that it isn't now just constrained

00:05:06.300 | to the best schools in the world,

00:05:07.860 | that because of this proliferation of content,

00:05:10.920 | there are talented students all over the world,

00:05:13.700 | and all they need is the content

00:05:15.420 | in which to be able to build crazy cool,

00:05:17.660 | world-changing things.

00:05:18.980 | And what I see as my job today is to go out into the world

00:05:24.860 | and find these ridiculously talented people,

00:05:27.460 | and then put them to work on the hardest problems

00:05:30.460 | that exist, and Udacity, to me,

00:05:32.140 | felt like the perfect place to do this.

00:05:34.580 | As a kind of prelude to this,

00:05:36.260 | about three years into Udacity,

00:05:39.540 | we had had this real focus, like I said,

00:05:42.900 | on machine learning and robotics,

00:05:45.340 | but we really wanted to take it to the next step.

00:05:47.540 | And we came up with this kind of concept internally

00:05:50.060 | that we called Only at Udacity.

00:05:52.020 | What if we taught the things

00:05:53.980 | that other places weren't teaching?

00:05:55.660 | What if people all around the world could come learn

00:05:59.220 | from what may appear to be niche topics,

00:06:01.940 | but were just being taught at the right time,

00:06:04.920 | because that industry's about to blow up?

00:06:07.860 | And the first one we did of this,

00:06:09.140 | and we've done some after, including Flying Cars,

00:06:11.820 | a much more in-depth curriculum on artificial intelligence,

00:06:16.180 | was self-driving cars.

00:06:17.380 | So this is a quick video that introduces it,

00:06:19.460 | and this is, of course, Sebastian Thrun,

00:06:21.780 | robotics legend.

00:06:23.220 | Let's see if this plays.

00:06:24.420 | (soft music)

00:06:26.840 | (speaking in foreign language)

00:06:30.760 | (speaking in foreign language)

00:06:34.680 | (speaking in foreign language)

00:06:38.600 | (speaking in foreign language)

00:06:42.520 | (speaking in foreign language)

00:06:46.440 | (soft music)

00:07:11.240 | (speaking in foreign language)

00:07:15.160 | And why did we want to do this?

00:07:18.400 | What was our goal?

00:07:20.600 | It was to accelerate the deployment of self-driving cars.

00:07:24.280 | Like Sebastian says in that video,

00:07:26.400 | there's a number of reasons

00:07:27.520 | why self-driving cars are transformational.

00:07:30.080 | And at the time, this was around 2016,

00:07:33.920 | it felt like self-driving cars

00:07:35.340 | were just taking a little bit too long.

00:07:38.200 | We rewind to that particular spot in time.

00:07:41.040 | Google was the really, the only main effort going on.

00:07:44.240 | And what we believed is that it needed to happen faster,

00:07:47.560 | and that one of the reasons it wasn't happening fast enough

00:07:49.640 | is because there wasn't enough talent in the space.

00:07:52.340 | So what we decided to do is, like I said,

00:07:54.880 | build something quite special.

00:07:56.760 | We wanted to pair up a world-class curriculum,

00:07:58.880 | an actual self-driving car, which we'll talk about more,

00:08:01.560 | and what we called our open-source challenges.

00:08:04.420 | And all of that would come together

00:08:06.160 | to build this quite special curriculum.

00:08:10.480 | So let's start with the curriculum.

00:08:13.200 | One of our beliefs was that partnering with industry

00:08:17.720 | was the right way to go.

00:08:19.300 | That was because it felt, and I believe this,

00:08:22.240 | that the knowledge of how to build a self-driving car

00:08:25.280 | was not necessarily trapped in academia,

00:08:27.060 | it was trapped in industry.

00:08:28.600 | So we had to go straight to industry,

00:08:30.620 | work with engineers that were already

00:08:33.980 | challenging themselves with these problems,

00:08:35.880 | and get them on camera.

00:08:37.200 | Have them teach the concepts that they know

00:08:39.220 | and build day in, day out, and have

00:08:40.960 | that be transplanted to thousands

00:08:42.520 | of minds around the world.

00:08:43.800 | So these are just some of those partners.

00:08:45.520 | There was many, many more.

00:08:46.880 | But we had a real focus on finding these engineers

00:08:49.200 | wherever they may be and getting those folks on camera.

00:08:52.960 | We also built an incredibly talented team.

00:08:54.700 | This is just a small snippet of the curriculum team.

00:08:57.720 | But of course, Sebastian Thrun was

00:08:59.600 | a big part of this curriculum.

00:09:02.160 | When I told folks that I'd gotten the chance

00:09:04.640 | to work with him on specifically self-driving cars,

00:09:07.960 | he likened it to getting basketball lessons

00:09:10.300 | from Michael Jordan, which I thought was pretty fun.

00:09:13.100 | And they were probably just as entertaining.

00:09:15.420 | But some really, truly great folks

00:09:17.140 | working on this curriculum and still doing that to this day,

00:09:20.380 | who deserve all of the credit, frankly.

00:09:22.740 | Here's a quick photo of our first lecture recordings

00:09:26.660 | with eventual Voyage co-founders, Eric and Mac.

00:09:30.780 | Eric, who's on the left, hates this picture.

00:09:33.060 | And here's why.

00:09:34.980 | There you go.

00:09:36.460 | He still isn't at Mac's height, but he still

00:09:39.540 | has that box on his desk.

00:09:42.260 | And we built a whole 12-month curriculum

00:09:46.940 | to take an intermediate software engineer who

00:09:49.500 | may be in consumer software or just

00:09:51.500 | some other part of the software world

00:09:54.660 | and take them into self-driving cars.

00:09:57.340 | We wanted to cover perception, prediction, planning,

00:10:01.020 | localization, controls even, just

00:10:04.180 | the whole breadth of the industry.

00:10:07.620 | And the reason we wanted to do that is because we saw

00:10:10.300 | the best fit for a Udacity student

00:10:12.620 | not necessarily being a specialist in a niche--

00:10:16.740 | for example, just perception, although there's

00:10:20.860 | been a whole bunch of folks doing that as well--

00:10:22.860 | but that the skills of a Udacity student

00:10:25.260 | tend to pair themselves well with being a generalist,

00:10:27.700 | someone that can contribute all across the stack.

00:10:30.100 | So we tried to give these folks that breadth of knowledge.

00:10:33.380 | So here's another quick video of just the curriculum

00:10:36.540 | that we built with some previews.

00:10:39.780 | In the first term, you'll build projects

00:10:42.420 | on deep learning and computer vision.

00:10:44.540 | For example, you'll build a behavioral cloning project

00:10:47.500 | where you drive a car yourself in a simulator,

00:10:49.860 | kind of like in a video game.

00:10:51.660 | And then you use data from your own driving in the simulator

00:10:54.860 | to train a neural network to drive that car for you.

00:10:57.980 | This is the type of project that cutting-edge Silicon Valley

00:11:00.580 | startups are working on right now.

00:11:02.460 | And it puts you at the forefront of the deep learning

00:11:04.620 | and autonomous vehicle industry.

00:11:06.980 | You'll also build a project to detect and track

00:11:09.660 | vehicles in a video stream, just like real autonomous vehicles

00:11:13.220 | have to do out in the highway.

00:11:15.260 | In term two, you'll learn about sensor fusion, localization,

00:11:19.180 | and control.

00:11:20.460 | This is hardcore robotics that every self-driving car

00:11:23.140 | engineer needs to know in order to actuate and move

00:11:26.500 | the vehicle through space.

00:11:27.940 | In the localization module, you'll

00:11:30.260 | build a kidnapped vehicle project, which

00:11:32.500 | takes a vehicle that's lost and figures out

00:11:34.900 | where it is in the world with the help of sensor readings

00:11:37.380 | and a map.

00:11:38.700 | This is exactly what real self-driving cars

00:11:40.860 | have to do every time they turn on in order to figure out

00:11:43.380 | where they are in the world.

00:11:45.500 | In the control module, you'll build

00:11:47.300 | a model-predictive controller, which

00:11:49.220 | is a really advanced type of controller that's actually

00:11:52.100 | how most self-driving cars move through the world

00:11:54.900 | and use the steering wheel, throttle, and brake

00:11:57.620 | to follow a set of waypoints or a trajectory

00:11:59.940 | to get from one point to the next.

00:12:02.020 | In term three, you'll learn about path planning.

00:12:05.060 | You'll have an elective month.

00:12:07.060 | And you'll learn about system integration.

00:12:09.580 | Path planning is really the brains of a self-driving car.

00:12:12.460 | It's how the car figures out how to get from one point

00:12:14.780 | to another, as well as how to react when you

00:12:17.020 | meet obstacles in terms of seating.

00:12:19.060 | I'm going to give you a sneak preview of how path planning

00:12:21.540 | works.

00:12:22.220 | And this is something that nobody's ever seen before.

00:12:24.540 | So get ready.

00:12:26.060 | Path planning involves three parts.

00:12:28.380 | There is prediction, which is figuring out

00:12:31.020 | what the other vehicles are going to do around us.

00:12:34.020 | There is behavior, which is figuring out

00:12:36.380 | what we want to do.

00:12:37.540 | This goes on for a while.

00:12:38.580 | So we'll pause it there.

00:12:40.380 | The impact of this curriculum was bigger

00:12:43.740 | than we thought it would be.

00:12:45.940 | When we pitched, as a small team, this idea to Sebastian

00:12:49.420 | and to Vish at Udacity, there was a lot of skepticism

00:12:53.620 | that something like this was going to be successful.

00:12:57.540 | And the reason that there was that skepticism

00:12:59.580 | is that one of the formulas that Udacity looked at

00:13:02.500 | to determine the impact of building

00:13:04.000 | a certain type of content was the number of open jobs

00:13:06.300 | available.

00:13:07.700 | If there was, for example, in web development,

00:13:10.900 | mobile development, all that good stuff,

00:13:12.900 | there was millions of jobs open.

00:13:14.340 | So it felt like there was a massive opportunity

00:13:16.420 | to impact the area.

00:13:18.060 | But if you were to, in 2016, search

00:13:19.740 | for self-driving car engineers or the different disciplines

00:13:22.540 | that exist within, it was kind of just Google.

00:13:26.020 | So it was very interesting just to see

00:13:28.100 | the instantaneous reaction that we had

00:13:31.060 | to launching this curriculum.

00:13:32.980 | Today, over 14,000 successful students

00:13:35.780 | from all around the world, as you can see.

00:13:37.620 | Probably the most exciting thing is

00:13:39.080 | to see what students have done with this sort of curriculum.

00:13:42.900 | For example, I learned recently that a set of our students

00:13:46.540 | here are building a self-driving truck startup in India.

00:13:50.340 | Another set of students in South Korea

00:13:52.780 | are building a perception engine for self-driving cars.

00:13:56.500 | Just a whole bunch of folks building truly amazing things.

00:14:00.540 | And not only that, they've gotten jobs at Cruise, Zoogs,

00:14:02.860 | Waymo, Argo, all the big names, and are actively

00:14:06.860 | impacting those companies today.

00:14:09.220 | Now for the fun stuff.

00:14:10.420 | So we also decided to make that curriculum extra special.

00:14:15.420 | And we decided to do that by building

00:14:17.180 | an actual self-driving car.

00:14:19.100 | And whenever I talked about this internally at Udacity,

00:14:21.980 | people asked me why.

00:14:23.180 | Like, why do we need to do this, right?

00:14:24.460 | Isn't the curriculum just enough?

00:14:27.020 | Why go to the length of building an actual self-driving car?

00:14:30.500 | And selfishly, some of it was just a personal want

00:14:32.660 | to build a self-driving car.

00:14:34.980 | But the reasoning that I use is that what better way

00:14:39.260 | to prove to these students that are putting their faith in us

00:14:42.200 | that we know what we're doing,

00:14:43.420 | than to build our own self-driving car?

00:14:45.820 | And also, what better way to collaborate with these students

00:14:47.940 | on an area that is really infantile,

00:14:50.420 | than again, by having this platform

00:14:52.220 | that students could actually run code on a car.

00:14:55.580 | So we decided to buy a car,

00:14:58.860 | and we'll talk more about that in a second,

00:15:00.080 | but we set ourselves a milestone for our self-driving car.

00:15:04.660 | It was to drive from Mountain View to San Francisco,

00:15:07.460 | 32 miles of driving with zero disengagements.

00:15:10.680 | It should be repeatable.

00:15:11.820 | It won't be zero disengagements every single time,

00:15:13.860 | 'cause otherwise we've got an actual self-driving car.

00:15:16.180 | But in a short period of time,

00:15:17.900 | how much progress can we make towards this stated goal?

00:15:21.740 | Raise your hand if you've been on El Camino Real

00:15:23.380 | in that sort of region, okay?

00:15:26.260 | So you probably understand it's got a lot of traffic lights.

00:15:29.060 | In fact, in our route, about 130 traffic lights.

00:15:32.580 | It's a multi-lane, three lanes,

00:15:35.380 | speed limit of about 40, 45, something like that.

00:15:38.980 | So it's fairly complex,

00:15:42.240 | but it's also got some constraining factors,

00:15:44.220 | which is what we're looking for too.

00:15:46.240 | So it focused our tech efforts.

00:15:48.700 | This is the car we bought.

00:15:50.300 | You're probably very familiar

00:15:51.380 | if you follow self-driving cars

00:15:52.500 | with the Lincoln MKZs of the world.

00:15:54.140 | They're everywhere, and there's a reason for that

00:15:56.140 | in terms of the drive-by-wire nature of the vehicle

00:15:58.820 | and other stuff.

00:15:59.980 | And we outfitted a whole bunch of sensors,

00:16:01.780 | some cameras, some LIDARs, all that good stuff.

00:16:04.340 | We also tried to build our own mount.

00:16:07.580 | We affectionately called this the Periscope.

00:16:10.300 | I don't know why it's in slow motion,

00:16:11.580 | but this was not our final design.

00:16:14.780 | Built all this from parts at Home Depot.

00:16:17.520 | Truly a MVP.

00:16:19.720 | And then we got to work.

00:16:21.000 | The goal was to accomplish that milestone

00:16:24.060 | within six months.

00:16:25.380 | So we, of course, had to work fast,

00:16:27.080 | assembled a dream team of folks

00:16:29.120 | that I'd worked with on different projects at Udacity

00:16:31.480 | that also wanted to come dabble in this,

00:16:34.520 | folks that worked on the machine learning curriculum,

00:16:36.240 | robotics curriculum, et cetera.

00:16:38.160 | So this was one of our first days testing.

00:16:40.640 | And we did this at the Shoreline Amphitheater parking lot,

00:16:43.760 | which actually now is a very popular place

00:16:45.120 | to test self-driving cars in the Bay Area

00:16:47.060 | because Google used to do it in the past.

00:16:49.740 | We saw a lot of weird stuff.

00:16:51.060 | For example, you'll see here.

00:16:52.560 | (car beeping)

00:16:59.540 | (audience laughing)

00:17:02.620 | We saw what I believe to be a motorcycle gang.

00:17:05.320 | And we made progress.

00:17:10.600 | We kept iterating, kept building.

00:17:13.720 | And it started to come together.

00:17:15.200 | In fact, some stuff that we thought wouldn't work

00:17:17.640 | surprisingly just started to work.

00:17:19.840 | This is on El Camino Real.

00:17:21.600 | I'm in the back seat here.

00:17:25.140 | So Mac discovered that we shouldn't have stopped

00:17:37.200 | at that traffic light.

00:17:38.680 | But we did.

00:17:39.640 | We resolved the mystery later.

00:17:42.480 | Let's go to the next video.

00:17:43.520 | And of course, we learned a lot by going in this route,

00:17:47.480 | the different behaviors of drivers.

00:17:49.400 | One of the things that we were worried about

00:17:51.200 | is vehicles cutting us off.

00:17:52.740 | And when we say cutting us off,

00:17:55.020 | it means a vehicle pulling out in front of us,

00:17:57.240 | even a few hundred feet in front.

00:17:59.680 | You'll see here.

00:18:00.520 | (audience member speaking indistinctly)

00:18:09.900 | (audience member speaking indistinctly)

00:18:14.480 | We drove a little slow, 25.

00:18:24.420 | (audience member speaking indistinctly)

00:18:31.920 | Turned out it was fine.

00:18:38.380 | And pretty soon it got quite boring.

00:18:41.940 | Car was doing very well driving itself.

00:18:45.460 | We built some cool algorithms to change lanes

00:18:48.340 | when necessary, similar to what you see

00:18:51.340 | with Tesla Autopilot these days.

00:18:53.020 | We collaborated with some students

00:19:03.800 | on a traffic light classifier,

00:19:05.220 | which was integrated into Roster.

00:19:08.660 | (audience member speaking indistinctly)

00:19:10.900 | And yeah, pretty boring stuff.

00:19:12.400 | So you can tell Eric was surprised that it was just fine.

00:19:17.800 | And we also had a penchant for building,

00:19:21.580 | or for recording themed videos,

00:19:24.140 | like you saw maybe from Elon Musk and the Tesla team

00:19:26.740 | with Paint It Black.

00:19:28.020 | We've got our own version of that.

00:19:30.440 | Eventually we became pretty confident,

00:19:32.700 | but we always wanted to test most of the day

00:19:34.980 | just to get the most learnings out of everything.

00:19:36.880 | This video was made at 2.30 a.m.,

00:19:38.940 | driving from Mountain View to San Francisco,

00:19:41.780 | all 32 miles.

00:19:43.720 | Of course there's a backing track.

00:19:46.320 | (gentle music)

00:19:48.900 | (gentle music)

00:19:51.480 | (gentle music)

00:19:54.060 | (gentle music)

00:19:56.640 | (gentle music)

00:19:59.720 | (gentle music)

00:20:02.300 | (gentle music)

00:20:04.880 | (gentle music)

00:20:07.460 | (gentle music)

00:20:10.040 | (gentle music)

00:20:12.620 | (gentle music)

00:20:15.200 | (gentle music)

00:20:21.780 | (gentle music)

00:20:25.700 | (gentle music)

00:20:29.100 | (gentle music)

00:20:32.560 | (gentle music)

00:20:42.440 | Maybe I want to turn it down.

00:20:43.880 | (laughs)

00:20:46.400 | So it's easier because there's less traffic, right?

00:20:49.640 | This is kind of cheating,

00:20:50.600 | and didn't count as the milestone, just to be clear.

00:20:53.520 | You'll see that we eventually hit the 32 miles,

00:20:56.400 | and Mac, who's in the driver's seat,

00:20:58.040 | is pretty excited about that.

00:20:59.480 | (gentle music)

00:21:02.060 | (gentle music)

00:21:07.140 | (gentle music)

00:21:12.460 | (gentle music)

00:21:15.900 | (laughs)

00:21:17.980 | (gentle music)

00:21:20.560 | (gentle music)

00:21:23.140 | (laughs)

00:21:47.220 | (gentle music)

00:21:49.800 | (gentle music)

00:21:52.380 | (laughs)

00:22:02.460 | And they hit it.

00:22:05.020 | But of course, that didn't count

00:22:10.280 | because it's in the middle of the night,

00:22:11.840 | and that's not gonna be a very useful route,

00:22:13.760 | but it was an awesome accomplishment

00:22:15.440 | just to even make it 32 miles with no disengagements

00:22:19.040 | when there's traffic lights, lane changes,

00:22:20.600 | all that good stuff.

00:22:21.600 | But after four months, this is in the daytime,

00:22:24.660 | this began, I believe, at like six, sorry, 7 a.m.,

00:22:28.720 | we accomplished it.

00:22:29.660 | That small team had come together

00:22:32.320 | and built something pretty cool

00:22:35.200 | that could handle, again, multi-lane roadways,

00:22:38.660 | varying speed limits, traffic lights, objects,

00:22:41.940 | all that good stuff.

00:22:43.220 | And the thing that really brought this home to me

00:22:47.240 | is that the industry was now ready, right?

00:22:50.160 | It felt like this feeling I had in software

00:22:53.960 | where someone in their bedroom can go and build something

00:22:56.440 | and launch it, almost feeling overnight,

00:22:58.960 | could now, not quite the same, but close to the same,

00:23:02.120 | happen in self-driving cars.

00:23:03.620 | But we'll talk more about what this led to

00:23:06.960 | in a little bit.

00:23:08.440 | Let's talk about open-source challenges.

00:23:11.120 | We also got the same question, why do this?

00:23:14.560 | And it was clear to me that for something

00:23:16.800 | like self-driving cars, which was so formative,

00:23:20.160 | we had to collaborate with students

00:23:22.640 | to figure out the best stuff

00:23:23.560 | because even the folks that were at Udacity

00:23:25.680 | were not necessarily the world's leading experts

00:23:28.540 | in these topics, so we wanted to use

00:23:30.120 | this hive mind of activity from around the world

00:23:33.040 | to teach the best stuff.

00:23:35.340 | So just through a period of a year,

00:23:39.800 | these are all the different challenges we launched.

00:23:42.260 | There was prizes and leaderboards

00:23:45.160 | and all this sort of fun stuff.

00:23:47.360 | The one that I'll focus most on today

00:23:49.480 | is using deep learning to predict steering angles.

00:23:52.440 | And the challenge was clear.

00:23:56.940 | It was that given a single camera frame,

00:23:59.500 | you have to predict the appropriate steering angle

00:24:03.240 | of the vehicle.

00:24:04.880 | If anyone had read NVIDIA's end-to-end papers in 2016,

00:24:07.840 | this stuff was all the rage,

00:24:09.920 | and it felt like one of those areas

00:24:12.540 | that was just begging for more exploration.

00:24:15.160 | And again, let's use this,

00:24:17.320 | all these students from around the world to do it.

00:24:20.000 | And we did have students from all around the world.

00:24:21.680 | There was over 100 teams,

00:24:23.660 | people self-organized into these little groups

00:24:26.680 | to go and build this.

00:24:28.760 | And over the course of about four months,

00:24:31.680 | we had a whole bunch of submissions,

00:24:33.800 | all taking incredibly different approaches to the problem.

00:24:37.800 | We released data sets, validation sets,

00:24:40.080 | all that good stuff.

00:24:42.260 | Here you'll see our V-winning model.

00:24:45.960 | And I later found out that the author of this model

00:24:49.800 | actually went on to lead the self-driving car team

00:24:53.280 | at Yandex, which if you've been following CES

00:24:55.840 | is doing some pretty cool stuff in self-driving cars today.

00:24:59.640 | But you'll see this is on a route

00:25:01.200 | from the Bay Area to Half Moon Bay, a very windy road.

00:25:06.200 | And you'll see that the prediction

00:25:10.200 | matches pretty closely to the actual, which is nice.

00:25:14.140 | And if you read his description of his solution,

00:25:16.860 | it's a pretty cool solution.

00:25:18.700 | And I think the most exciting thing

00:25:20.700 | was just the number of different approaches to the problem,

00:25:25.100 | all resulting in some awesome stuff.

00:25:28.420 | And again, in true Voyage fashion,

00:25:30.660 | we recorded a video of what this model

00:25:32.860 | performed like on our car.

00:25:34.340 | (video playing)

00:25:37.000 | It wasn't perfect as any first model

00:25:51.960 | and just the general approach of camera only driving

00:25:56.800 | had its faults.

00:25:59.080 | One of the main ones that we realized

00:26:02.240 | after trying all this stuff out is that, of course,

00:26:05.360 | a car, when steered by such an input,

00:26:10.100 | performs differently in a car than it does on your desk

00:26:13.760 | in a simulator or through prerecorded camera frames.

00:26:17.940 | So adjusting for those corrections

00:26:20.300 | that might need to be made is something

00:26:23.100 | that students after the fact added, which was pretty cool.

00:26:26.000 | So after all of these things, building that curriculum,

00:26:29.640 | building a self-driving car, launching these challenges,

00:26:33.020 | it felt like it was time for something new.

00:26:35.320 | It was awesome to go and collaborate

00:26:36.780 | with all these students.

00:26:38.460 | And it felt like I had to go build something.

00:26:42.180 | So gathered that same team that had built this curriculum

00:26:45.440 | and we said, we're gonna go build a self-driving car.

00:26:48.120 | This is from my pitch at Coastal Ventures.

00:26:50.860 | You can kind of see the pitch deck there a little bit.

00:26:53.680 | Voyage is a new kind of taxi service.

00:26:56.360 | Our pitch has changed somewhat through time,

00:26:57.840 | but that's still pretty accurate.

00:26:59.840 | And we started what is now called Voyage.

00:27:04.580 | And our goal really was that we wanted to, again,

00:27:07.960 | build a self-driving car,

00:27:09.320 | but we wanted to do it differently.

00:27:10.700 | We didn't wanna follow the same formula

00:27:12.200 | that we felt we'd seen from some of the other folks

00:27:15.000 | in the field.

00:27:15.920 | And the reason is that those folks have real advantages.

00:27:18.520 | When you think about Google's project,

00:27:20.320 | of which I'm a big fan,

00:27:21.640 | they have this massive engineering pipeline

00:27:23.820 | of folks that wanna go build a self-driving car

00:27:26.040 | at today Waymo.

00:27:27.720 | But they also have a cash bank balance

00:27:30.240 | of billions of dollars that is hard to match.

00:27:34.000 | They also have the brand recognition

00:27:35.240 | of getting to work with Google and all that good stuff.

00:27:37.400 | So we just knew we had to think about this problem

00:27:39.600 | quite differently.

00:27:40.520 | And what motivated me is that today, as we all know,

00:27:45.600 | we have this incredibly broken transportation system.

00:27:49.040 | You step outside onto the roads today,

00:27:52.160 | and I don't know about you guys,

00:27:53.480 | but I don't feel particularly safe when I jump into my car.

00:27:57.400 | Over, we all know the stats,

00:27:58.560 | over one million people have,

00:28:01.880 | suffer fatalities on the roads today.

00:28:03.920 | Doesn't include folks that break necks,

00:28:07.520 | that injure, break bones, all that horrific stuff.

00:28:11.400 | It's also incredibly inefficient.

00:28:12.980 | We've, again, all observed this as we go about our day.

00:28:16.360 | Just the number of lanes that exist on a road today

00:28:18.760 | to account for peak traffic,

00:28:21.280 | the number of vehicles which have enough room

00:28:23.280 | for eight people have usually one person in that front seat.

00:28:27.720 | I read a stat recently that only 7% of the average vehicles'

00:28:32.160 | energy usage is going towards moving the things

00:28:34.720 | that are actually in the car.

00:28:36.400 | The rest is waste.

00:28:38.040 | So an incredibly inefficient system.

00:28:40.360 | It's also expensive.

00:28:41.860 | The reason we see a lot of old cars on the road today

00:28:44.420 | is because that's, at least today,

00:28:46.220 | the most optimal and affordable way

00:28:48.840 | to for lots of folks to get around,

00:28:51.120 | and inaccessible.

00:28:52.200 | And you'll see why this matters to us in particular.

00:28:54.800 | Our goal is to introduce a new way

00:28:57.280 | to explore our communities.

00:28:58.800 | This is a video of one of our cars

00:29:01.080 | at a particularly cool place, which we'll talk more about.

00:29:04.740 | And this is kind of our mission.

00:29:07.200 | And why now?

00:29:09.480 | Why is it possible to build a self-driving car now?

00:29:12.080 | A number of factors that we learned

00:29:14.360 | during that Udacity experience, but some new as well.

00:29:17.720 | It feels, from everything we see,

00:29:19.800 | that sensors are now in this position,

00:29:22.180 | which these sensors are now capable

00:29:25.360 | of level four self-driving cars.

00:29:26.920 | The resolution, the range, the reliability,

00:29:30.020 | all those things that were necessary

00:29:32.380 | for an L4 self-driving car are today ready.

00:29:35.000 | That didn't used to be the case.

00:29:36.200 | If you rewind to 2007 and look at the cars

00:29:40.760 | that were participating in the DARPA challenges,

00:29:44.000 | you'll see a lot of single channel lasers.

00:29:45.980 | You'll see the relic of the Velodyne HTL64,

00:29:50.260 | the spinning bucket, as it's called today.

00:29:53.180 | And no one would have claimed those sensors already.

00:29:55.220 | But today, you've got this enormous breadth of sensors

00:29:58.380 | that can take you that way.

00:30:00.560 | Compute is there.

00:30:01.740 | When we think about the recent rise in GPUs and whatnot,

00:30:05.240 | finally being able to have enough performance

00:30:09.060 | in the back of a car with the power constraints

00:30:11.780 | that you have, it's there.

00:30:14.140 | And talent.

00:30:15.420 | Again, this is not just Google today.

00:30:18.100 | You've got all of these great minds

00:30:19.500 | from all around the world building this technology.

00:30:22.420 | So you're able to recruit those folks,

00:30:24.160 | put them to work on the problems they've solved

00:30:26.660 | in many cases beforehand.

00:30:28.540 | The reason I have yellow for computer vision,

00:30:31.260 | which is not a knock against computer vision,

00:30:32.700 | is because it's not quite there yet

00:30:35.220 | for a fully driverless self-driving car.

00:30:39.340 | If you, again, rewound three, four, five years,

00:30:43.880 | this would have been a red.

00:30:45.420 | But today, with all the community

00:30:48.820 | and whatnot around computer vision,

00:30:50.980 | this is steadily getting to a green state.

00:30:53.540 | So pretty soon, that'll be green.

00:30:54.900 | And of course, then you'll have that perfect formula

00:30:57.400 | for level four driving.

00:30:58.980 | What we run after is ride sharing.

00:31:02.380 | We believe that the optimal way for people to move around

00:31:06.100 | is to be able to summon a car.

00:31:08.380 | But the thing that's suboptimal today

00:31:09.940 | is that you have to have a human driving you

00:31:12.300 | whenever you wanna move around.

00:31:14.540 | Prevents the cost from being lower,

00:31:16.060 | prevents some safety issues,

00:31:17.940 | prevents some quality issues.

00:31:19.700 | We think solving that will mean

00:31:22.280 | these next generation way of moving around

00:31:25.180 | will come to fruition.

00:31:27.020 | But what we also see is that if you,

00:31:29.380 | let's say we never remove the driver from the car,

00:31:31.300 | that a ride-hailing network always had a human driver,

00:31:34.580 | you are inherently limited by the number of miles

00:31:37.700 | you can drive, which means that

00:31:41.140 | it'll never replace personal car ownership,

00:31:43.260 | will never fix that fatality number I talked about,

00:31:47.260 | all of those things we must solve.

00:31:49.020 | So we think by having a self-driving car

00:31:51.540 | that these next generation transportation networks

00:31:56.020 | will come to fruition.

00:31:57.460 | Our lead VC is a guy called Vinod Khosla,

00:32:01.740 | the founder of Khosla Ventures,

00:32:04.020 | an awesome guy who's done some

00:32:05.700 | truly world-changing things.

00:32:07.340 | He has this quote, which I'm a big fan of.

00:32:09.540 | "Your market entry strategy is often different

00:32:11.620 | "from your market disruption.

00:32:12.860 | "Start where you find a gap in the market

00:32:14.700 | "and push your way through."

00:32:16.300 | And this better communicated

00:32:18.620 | what I mentioned at the very beginning,

00:32:19.940 | which is that we should build a self-driving car,

00:32:21.980 | but do it in a different way.

00:32:23.900 | Because if we don't do that,

00:32:25.180 | we're gonna fall into the same traps

00:32:26.660 | as many of the others that have died along the way.

00:32:28.900 | We have to find a way to do something different

00:32:31.060 | that we own and that we are really, really good at.

00:32:34.540 | And for us, that was retirement communities.

00:32:38.100 | Hands up if you've ever visited a retirement community.

00:32:41.340 | Let's see, way less, there you go.

00:32:43.380 | Surprise, Lex, I've gotta get you out to one.

00:32:45.580 | But these are just amazing places.

00:32:50.060 | And the reasons we choose retirement communities first

00:32:54.340 | to deploy our self-driving technology in

00:32:56.460 | is for these four reasons.

00:32:58.980 | They are slower, the speed limits in these communities

00:33:02.900 | tend to be far slower than you'd see on public road.

00:33:05.820 | Much calmer roadway.

00:33:07.660 | When you visit these locations,

00:33:09.660 | I liken it to listening to a podcast at 0.75x.

00:33:13.140 | Just very constrained, very slow,

00:33:15.760 | and a little boring from time to time.

00:33:19.460 | But you've also got these

00:33:20.340 | heartfelt transportation challenges.

00:33:22.940 | We hear from these residents all the time

00:33:25.700 | about how transportation is a pain point

00:33:28.740 | and that their only option is a personally owned vehicle.

00:33:31.460 | These folks know in many cases they shouldn't be driving,

00:33:34.060 | but because they don't have an alternative,

00:33:35.580 | they still drive.

00:33:37.060 | We hear from folks that put off much needed surgeries,

00:33:40.060 | hip replacements, things like that,

00:33:41.560 | because they don't have a friend in town

00:33:43.020 | who's gonna be able to move them around.

00:33:45.020 | We hear from folks with vision degeneration

00:33:49.540 | that they just don't see a way

00:33:50.720 | that they'll be able to move around

00:33:51.900 | and keep that quality of life

00:33:53.860 | that they've been able to have.

00:33:55.420 | Folks gripping steering wheels for extended period of times,

00:33:59.300 | all these challenges that felt like the best first place

00:34:02.440 | for a self-driving car to begin.

00:34:04.180 | And a clear path to customers.

00:34:05.620 | We see that on the roads today,

00:34:09.020 | ride sharing on public cities and whatnot

00:34:11.780 | is a particularly brutal battle,

00:34:13.620 | a race to the bottom in terms of cost.

00:34:16.020 | If we owned every retirement community in the country,

00:34:18.700 | meaning the transportation networks there,

00:34:20.580 | that would in and itself be a very valuable business.

00:34:24.980 | One of my favorite passengers is Anahid.

00:34:27.640 | She came to visit us recently

00:34:30.340 | and gave this quick speech

00:34:32.100 | about why self-driving cars matter to her

00:34:34.700 | and her community.

00:34:36.220 | - Not only that, but we're concerned about safety.

00:34:39.620 | I was on the road and it was one of the drivers.

00:34:42.780 | A car turned and went the wrong way right at us.

00:34:46.980 | A four-hundred meter spine just caught up with us.

00:34:51.620 | An older person who doesn't have the same reflexes

00:34:55.540 | strapped up to their door in an accident.

00:34:59.740 | - Let's talk about our first community.

00:35:02.140 | This is the villagers.

00:35:04.420 | Whenever I show this slide,

00:35:05.620 | people are astounded by the number of residents

00:35:08.460 | in a community like this.

00:35:10.020 | Over 125,000 and growing.

00:35:12.780 | Over 750 miles of road.

00:35:15.300 | And what we have in this location

00:35:16.420 | is an exclusive license to operate

00:35:18.220 | an autonomous vehicle service.

00:35:20.300 | This is one of our other beliefs,

00:35:22.140 | which is that by partnering very deeply with the community,

00:35:25.840 | it means that we're able to deliver a better service

00:35:29.340 | and that we're able to grow a more reliable business.

00:35:32.180 | We won't have entrants and competitors

00:35:35.020 | from all of the other self-driving car companies

00:35:36.880 | in our communities.

00:35:38.260 | What we actually do in exchange for that exclusive license

00:35:41.180 | is grant these communities equity.

00:35:44.020 | Because if we win, it's probably, in fact,

00:35:46.540 | highly likely as a result of those communities.

00:35:49.420 | And the addressable market of transportation

00:35:51.300 | in these regions is massive.

00:35:53.300 | These residents tend to be,

00:35:55.500 | as a lot of seniors tend to be, quite affluent,

00:35:58.620 | which means that they have some disposable income

00:36:00.500 | when it comes to being able to pay for ride-sharing services

00:36:05.180 | and other things like that.

00:36:06.460 | So we find that that recipe is absolutely perfect here.

00:36:10.980 | And we're launching and have launched passenger services

00:36:13.660 | to these residents.

00:36:15.680 | Gotten a lot of awesome feedback.

00:36:18.660 | Learned a lot about the needs of providing ride-sharing

00:36:22.220 | for senior citizens.

00:36:23.980 | Just some quick stats.

00:36:26.180 | This is from my Series X fundraising deck,

00:36:31.180 | just about the size of the senior market.

00:36:33.460 | Again, this is the first place we go,

00:36:35.220 | but you can get a feel for just how large

00:36:37.500 | this transportation market is.

00:36:39.120 | Today, there are 47 million seniors.

00:36:40.720 | That's growing by 2060 to over 100 million seniors

00:36:43.700 | in the US.

00:36:44.900 | The total addressable market for just seniors

00:36:48.200 | is incredibly large.

00:36:49.860 | 2,500 plus communities, all that good stuff.

00:36:52.420 | And this is how we see the world,

00:36:55.140 | the landscape of potential deployments.

00:36:58.700 | You've kind of got a lot of the big guys

00:37:00.700 | focusing on that bottom left quadrant.

00:37:03.460 | They're focusing on large cities.

00:37:05.140 | And it makes sense because it's playing

00:37:07.040 | to their unique strengths.

00:37:08.060 | It's playing to their ability to deploy thousands of cars,

00:37:11.300 | tens of thousands of cars.

00:37:13.060 | It plays to the strength that they have,

00:37:15.360 | at least some patience or ability

00:37:17.700 | to have more extended timelines

00:37:19.880 | when it comes to building this technology.

00:37:22.420 | But for a startup like us,

00:37:24.380 | that fights for survival every single day,

00:37:27.900 | it means that we have to do things differently.

00:37:29.500 | So we focus on that top right quadrant there,

00:37:33.420 | what we've kind of coined as self-contained communities.

00:37:36.960 | These places are simpler, slower,

00:37:39.340 | but they also have this ability

00:37:41.460 | for us to have that exclusivity that I talked about.

00:37:44.180 | And there's some others, of course,

00:37:46.660 | that we play in, whether it's the senior market

00:37:48.980 | or maybe even small cities and things like that.

00:37:51.700 | Let's talk about autonomous technology.

00:37:54.340 | So just to reiterate,

00:37:56.540 | why do we deploy in retirement communities?

00:37:58.660 | Slower speed, simpler roadway.

00:38:01.380 | There is a central authority.

00:38:03.620 | These places tend to be run by private companies,

00:38:06.260 | which makes for a quite unique relationship

00:38:09.540 | in a very positive way.

00:38:11.100 | It means we can deploy faster.

00:38:12.680 | It means we have the potential

00:38:14.300 | to have more impact in these regions.

00:38:16.460 | It also turns out that retirement communities

00:38:18.740 | tend to be located where there's ideal weather

00:38:20.380 | for self-driving cars.

00:38:21.580 | Think about Arizona, Florida, et cetera.

00:38:24.580 | We have a world-class team building this at Voyage

00:38:28.580 | from all the major programs out there,

00:38:32.220 | and that makes our lives infinitely easier.

00:38:34.340 | One thing that also makes our lives easier

00:38:36.860 | is the sensor configuration of our car.

00:38:39.260 | We've intentionally made this decision

00:38:41.840 | that we're not gonna focus on optimizing for cost today,

00:38:44.860 | but to optimize for performance.

00:38:46.820 | We wanna get to truly driverless sooner than most,

00:38:49.500 | and one of the easiest ways you can, again,

00:38:51.780 | make your life easier

00:38:52.620 | is by optimizing for high-resolution sensors.

00:38:56.860 | At the very top of the vehicle, we have the VLS-128,

00:38:59.800 | which is a 128-channel LiDAR

00:39:01.600 | that's capable of seeing 300 meters in 360 degrees.

00:39:05.200 | Many of the different LiDARs on the vehicle

00:39:08.220 | to cover different certain blind spots.

00:39:10.700 | Altogether, we process 12.6 million points per second,

00:39:14.520 | and that just looks incredibly high-resolution.

00:39:19.380 | You'll see our car at the bottom there,

00:39:21.340 | and that's the raw point cloud output

00:39:25.740 | that we see in the world.

00:39:27.700 | We run towards level four,

00:39:29.040 | and for us, what that means is that

00:39:31.820 | if you're building a demo self-driving car,

00:39:33.820 | kinda like we did at the Udacity project,

00:39:35.920 | you may focus on just the top four items, that top row.

00:39:39.620 | You may focus on perception, prediction,

00:39:41.400 | planning, and controls,

00:39:42.980 | and it turns out you can build a very impressive demo

00:39:46.180 | quite quickly by just focusing on those things,

00:39:49.340 | but of course, those things fall apart

00:39:51.140 | whenever edge cases are introduced,

00:39:53.740 | which happen all the time,

00:39:55.400 | so we've spent a ton of time on all the items here

00:39:59.140 | because, again, our goal is to build not a demo

00:40:01.780 | but a truly driverless vehicle.

00:40:03.900 | We also have an emphasis on partnerships

00:40:10.860 | because what we've noticed in the self-driving ecosystem

00:40:13.900 | is that there's not just more self-driving car companies

00:40:17.320 | building the full stack.

00:40:18.580 | There's now folks getting into simulation,

00:40:21.060 | to mapping, to middlewares, to teleoperations,

00:40:23.580 | to routing, to sensors, of course,

00:40:25.760 | and a ton more, so we make our lives, again, easier

00:40:30.820 | by partnering with companies like this

00:40:32.820 | so that we don't have to spin up a simulation team

00:40:34.980 | or we don't have to spin up an operations team

00:40:36.640 | to go map the world.

00:40:37.860 | We can just work with these very cool companies.

00:40:40.580 | Let's talk about one unsolved problem which fascinates me.

00:40:46.300 | It's to do with perception,

00:40:48.100 | and you probably won't be able to notice

00:40:49.760 | this unsolved problem from just this picture,

00:40:52.140 | but maybe if I add some annotations, you might.

00:40:55.940 | Foliage, trees, bushes, whatever you wanna call them.

00:41:00.740 | You may have seen some quotes in the media

00:41:04.380 | about some popular AV programs struggling

00:41:08.020 | with such foliage.

00:41:10.440 | For example, cruise cars sometimes slow down or stop

00:41:12.560 | if they see a bush on the side of a street

00:41:14.380 | or a lane-dividing pole.

00:41:16.380 | That was in the information.

00:41:18.180 | Oop, wrong way.

00:41:19.220 | This one, Uber's self-driving car software

00:41:23.180 | has routinely been fooled by the shadows of tree branches,

00:41:26.140 | which it would sometimes mistake for real objects,

00:41:28.300 | insiders say, that's Business Insider, and even Voyage.

00:41:32.780 | There's only one hard stop on the way.

00:41:34.420 | The culprit is a bush two feet high

00:41:36.180 | that protrudes into a lane from a street median,

00:41:38.760 | which Voyage considers a possible threat.

00:41:41.320 | Voyage may trim it, and we did,

00:41:44.660 | but we don't think that's scalable.

00:41:46.740 | And, or maybe it is, I don't know.

00:41:48.980 | But we, at the beginning of 2018,

00:41:52.540 | decided to solve this problem.

00:41:54.340 | So, of course, all of this resides

00:41:56.900 | in the world of perception,

00:41:58.300 | area of particular fascination for me.

00:42:00.680 | We're sharing these slides,

00:42:02.740 | but these are just some of the papers

00:42:05.180 | and research that we see going on

00:42:07.660 | that intends to solve those sorts of issues.

00:42:10.580 | One of the reasons you've seen those programs,

00:42:13.560 | including ours, be particularly sensitive to foliage

00:42:16.500 | is because, from a perception perspective,

00:42:19.780 | one of the most well-known way to detect objects

00:42:23.980 | is to utilize the map.

00:42:26.300 | So if you have this map, and you effectively,

00:42:29.460 | it's simplifying to a certain extent,

00:42:30.660 | but subtract objects that aren't in the map,

00:42:34.720 | and then use that as a way to understand

00:42:38.460 | what's in and around you that's dynamic,

00:42:40.380 | then, of course, you'll end up with

00:42:42.140 | decent representations of cars and pedestrians and whatnot.

00:42:45.140 | But if foliage grows, which it does, trees,

00:42:48.920 | then that's gonna extend out from the map

00:42:51.780 | and mean that that particular bush

00:42:54.020 | is now an object in your path.

00:42:56.280 | These networks here, which these are all neural networks,

00:43:00.380 | don't use that same technique.

00:43:03.300 | They don't use the map as a prior.

00:43:05.220 | Instead, what they do is take, of course,

00:43:09.120 | the 3D scan of the world,

00:43:10.820 | and then take a more learned approach to the problem.

00:43:14.300 | You'll have tens of thousands, hundreds of thousands

00:43:17.100 | of labels of cars, humans, et cetera,

00:43:20.300 | and then these next networks will be able

00:43:21.900 | to pick these ones out.

00:43:23.940 | We're particularly fascinated by PIXOR,

00:43:25.900 | which came from some great researchers at Uber ATG.

00:43:28.900 | VoxelNet came from Apple SPG.

00:43:31.080 | I've heard our engineers talking a lot

00:43:33.700 | about Fast and Furious recently, which merges together

00:43:36.340 | perception, prediction, and tracking into a single network,

00:43:40.580 | which is pretty cool, and PointPillars,

00:43:43.020 | which I think came from the Neutronomy team recently.

00:43:45.220 | I think Carl is speaking soon, right?

00:43:47.180 | So just in general, we see a whole bunch

00:43:50.740 | of work going out there to solve these issues.

00:43:53.080 | The other one that these sorts of networks solve,

00:43:55.700 | which I also find particularly fascinating,

00:43:57.340 | is that if you use traditional clustering algorithms,

00:44:01.540 | what you might see is that if two people

00:44:03.220 | are stood next to each other,

00:44:04.660 | a traditional algorithm will cluster as one object,

00:44:07.900 | which when you're trying to move away

00:44:09.880 | from those edge cases and build a truly self-driving car,

00:44:12.620 | that's a non-starter, right?

00:44:14.180 | Because pedestrians are the most important thing

00:44:15.820 | you can probably detect, and detecting two things

00:44:18.260 | as one thing is not gonna cut it.

00:44:20.080 | And of course, it does that because it's a dumb algorithm.

00:44:23.620 | It's not trained on any sort of information.

00:44:26.300 | But these networks, again, are very, very good

00:44:28.860 | at understanding the features and perspectives of humans,

00:44:31.680 | even if they are in crowds and whatnot.

00:44:34.180 | And that then helps all your stack downstream,

00:44:36.340 | because if you have accurate perception information

00:44:38.540 | about objects in and around you,

00:44:40.340 | your predictions are much better,

00:44:42.180 | your tracking is much better,

00:44:43.700 | and ultimately how you navigate the world is much safer.

00:44:46.580 | I'm also particularly fascinated by reinforcement learning,

00:44:50.260 | which I know Lex is as well.

00:44:51.660 | If you've read Waymo's recent work on imitation learning,

00:44:55.860 | I think that's particularly cool.

00:44:57.300 | Another company we track quite closely,

00:44:59.480 | just 'cause they do amazing stuff, is Wave,

00:45:02.160 | trying to build an entirely self-driving car

00:45:05.100 | powered by reinforcement learning.

00:45:07.260 | Think about disengagements as rewards and things like that,

00:45:10.740 | to be able to tune that to better performance.

00:45:14.820 | Also just areas of learned behavior planning,

00:45:17.500 | ultimately fusing rules of the road

00:45:20.100 | with more learned behaviors.

00:45:22.180 | The ecosystem, I think it's this area

00:45:24.500 | that is thriving today, seeing just how many folks

00:45:27.140 | are diving into not just the full stack,

00:45:29.420 | but building tools and building other

00:45:31.840 | really important parts of the stack.

00:45:35.180 | The maturation of sensors, not just higher resolution LiDAR,

00:45:39.780 | but things like 3D radar.

00:45:41.620 | We get pitched all the time from these companies,

00:45:43.780 | and it's clear to see there's been a rise in volume

00:45:46.140 | from all these great efforts.

00:45:50.860 | Lessons learned, now that I've been building Voyage

00:45:53.940 | for two years, and prior to that, four years at Udacity,

00:45:56.520 | what things have I personally learned

00:45:58.620 | that are not technical in nature?

00:46:00.780 | So many things.

00:46:03.220 | So these all may look like cliches,

00:46:05.540 | but I promise you they all came from lessons

00:46:07.140 | which were really, really painful in the moment.

00:46:10.340 | Don't be intimidated.

00:46:11.180 | So the thing that I feel happens a lot in self-driving cars

00:46:16.020 | is that because it started in this very academic sense,

00:46:21.020 | meaning Stanford, Carnegie Mellon, and whatnot,

00:46:24.660 | that it felt like to break into the industry,

00:46:28.340 | you had to also go through that same path.

00:46:30.140 | You had to get a PhD in something,

00:46:32.140 | and really go the path that was well-trodden.

00:46:37.100 | But I think that only takes the industry so far.

00:46:40.780 | And I think it's really important that we get folks

00:46:42.540 | from all different backgrounds, all different industries,

00:46:45.920 | to come contribute to this field.

00:46:47.660 | 'Cause if we don't, there is no driverless.

00:46:50.240 | It can't happen in that isolated bubble.

00:46:52.020 | It needs to be extended out.

00:46:53.140 | So don't be intimidated by those things.

00:46:56.420 | Understand your limitations.

00:46:57.460 | This is perhaps more of a kind of CEO lesson for myself,

00:47:01.800 | but I think when you're building out a company

00:47:04.700 | from one person or five people to,

00:47:07.940 | today we're 44 folks, you cannot do everything.

00:47:11.540 | And it's really important that you build a team around you

00:47:14.460 | that is able to do what you used to do,

00:47:17.080 | but do it 10 times better.

00:47:19.160 | I probably didn't spend enough time building out that team

00:47:21.080 | until we had some challenges our way

00:47:24.840 | when it comes to that stuff.

00:47:26.180 | Be proactive versus reactive.

00:47:28.020 | I think it's really crucial, again,

00:47:31.020 | when you're building a company,

00:47:32.020 | to try and predict what's gonna happen next.

00:47:34.600 | Because if you're reactive, you're constantly

00:47:36.640 | two steps behind what other folks are doing.

00:47:39.500 | Get out of the way.

00:47:41.160 | I think a lot of folks, again,

00:47:44.000 | perhaps overstay their welcome

00:47:45.160 | in certain areas of the company

00:47:47.220 | when they should just say, okay, I've got experts now.

00:47:49.720 | I can just step aside and let those folks

00:47:51.880 | do what they do best.

00:47:52.980 | And speaking of which, hire the best.

00:47:56.160 | It's really easy when all this pressure's on,

00:47:58.320 | when you're building a company,

00:47:59.320 | to kind of sacrifice when it comes to your culture,

00:48:02.200 | when it comes to hiring.

00:48:04.040 | It's really crucial that you find folks

00:48:05.840 | that are not just the best in their field,

00:48:07.860 | but are the best match for your company.

00:48:10.400 | And always be curious.

00:48:11.360 | I think it's always one of the things

00:48:13.660 | we believe in at Voyage is that

00:48:15.200 | it's important that knowledge is not isolated

00:48:19.800 | to just one person, that that knowledge

00:48:21.840 | should be spread throughout the company.

00:48:23.240 | Because even though it may feel like oversharing

00:48:26.560 | or overcommunicating, what that knowledge may mean

00:48:29.920 | for someone that has a particularly unique background

00:48:32.960 | is they may do something incredibly cool with it.

00:48:34.720 | They may build something that

00:48:36.120 | totally transforms our company.

00:48:37.700 | So that's about it.

00:48:39.960 | Can jump to questions if that's helpful.

00:48:43.280 | That was great.

00:48:44.120 | Please give a big hand.

00:48:44.940 | (audience applauding)

00:48:46.960 | How did you identify retired communities

00:48:49.200 | as the target market to prioritize?

00:48:51.840 | Yes, so retirement communities for us

00:48:55.240 | was actually, there's a really long story,

00:48:57.520 | but I'll trim it down a little bit.

00:48:59.340 | So when we were starting Voyage,

00:49:02.160 | Sebastian Thrum was very helpful

00:49:03.600 | in helping us start this company.

00:49:06.640 | And of course, as kind of naive founders of a company,

00:49:10.960 | we were like, oh, let's just take this El Camino thing

00:49:12.760 | and put it on everywhere else that looks like El Camino

00:49:14.960 | and just do that over and over again.

00:49:17.400 | But he cautioned against that.

00:49:18.680 | And very wisely so, because again,

00:49:21.440 | you're nothing special compared to the other

00:49:23.240 | self-driving car companies out there by doing so.

00:49:26.360 | And in 2009, he had really advocated to Google leadership,

00:49:31.360 | et cetera, Larry Page, that retirement communities

00:49:36.640 | for self-driving cars might just be the best way

00:49:38.540 | for Google to go about deploying their self-driving cars.

00:49:42.040 | But, and I can understand why,

00:49:43.440 | I think the Google folks were Google, right?

00:49:46.360 | We're not just about retirement communities,

00:49:48.680 | we're about the world, like level five or nothing, right?

00:49:51.880 | So he got some pushback,

00:49:52.760 | but he did some research in that process, met some folks.

00:49:55.840 | So when we were starting, he was like,

00:49:57.360 | you got to check out these retirement communities.

00:50:00.000 | So we did, we went to visit and eventually we got there.

00:50:03.680 | So we wouldn't have got to that point

00:50:05.320 | without Sebastian pushing for that.

00:50:07.680 | - Just to follow up on the question

00:50:08.920 | of retirement communities, the question is,

00:50:11.120 | do you ever think about the other collateral issues,

00:50:15.640 | especially the retirement community would have

00:50:17.920 | to get into a car?

00:50:19.520 | - Yep.

00:50:21.280 | - And how exactly would they interface,

00:50:24.200 | like somebody wants to make a call to have a car

00:50:27.800 | come to their, wherever they are,

00:50:29.400 | and they have to move from A.A to point B.

00:50:33.560 | So how did you ever think about all these issues

00:50:38.240 | that are very germane?

00:50:41.640 | It's not just a vehicle moving on its own.

00:50:44.200 | - Yep.

00:50:45.040 | - These are all collateral issues.

00:50:46.840 | How do you plan to address this?

00:50:48.920 | - It's a good question.

00:50:49.760 | So the way we think about this is that today

00:50:52.840 | we've intentionally focused it on a segment of the market,

00:50:55.480 | which is called the active adult communities.

00:50:59.200 | These folks tend to be able to go into their own cars

00:51:03.560 | or into a taxi, open the door, sit down

00:51:06.680 | without the need for any assistance when it comes to that.

00:51:10.720 | But they may have vision issues,

00:51:12.040 | they may have other issues that prevent them

00:51:15.000 | from driving perhaps.

00:51:16.080 | For example, we hear a lot that folks feel

00:51:17.720 | really uncomfortable driving in the evenings.

00:51:19.800 | They feel comfortable driving in the daytime

00:51:21.320 | 'cause their vision supports it,

00:51:22.320 | but when it comes to the evening time,

00:51:23.360 | they have this mad rush to get home.

00:51:26.120 | But there is that other market

00:51:27.080 | which you're talking about, right,

00:51:28.000 | which is folks that just need that helping hand

00:51:31.120 | towards getting to the car.

00:51:32.960 | And one of our beliefs as a company

00:51:34.560 | is that the senior market, like I had in that slide,

00:51:38.920 | is surprisingly large.

00:51:40.400 | And what that means to us is that we think we can own it.

00:51:43.040 | We think we can be that company

00:51:44.960 | that any senior citizen in that situation thinks,

00:51:46.840 | oh, I should call Voyage because I need to get

00:51:49.480 | from point A to point B.

00:51:51.200 | Instead of thinking I should call Waymo or Cruise

00:51:54.920 | or any of the folks that are gonna go

00:51:55.880 | after the general big market, they'll think about Voyage.

00:51:57.760 | And the reason they'll think about it

00:51:58.600 | is because we'll deliver a product to them

00:52:00.760 | that is meant for those folks,

00:52:03.480 | that is designed for their use cases.

00:52:05.640 | It may be that actually if they're going on a long trip,

00:52:08.000 | let's say they're traveling 50 miles,

00:52:10.280 | the first mile of that trip and the last mile of that trip

00:52:12.440 | may involve a human, like helping them into the car

00:52:15.240 | and then dropping that human off somewhere else

00:52:16.800 | to go do that all over again.

00:52:18.520 | It may involve crazy robots

00:52:19.920 | that help people from their cars.

00:52:21.160 | We've heard from folks at Toyota

00:52:23.920 | that are building these bag-carrying robots

00:52:25.960 | and other things that may assist seniors

00:52:27.840 | from getting to the cars and whatnot.

00:52:29.680 | So I think that's why that market for us

00:52:33.680 | is particularly exciting because it feels like

00:52:35.760 | you can deliver these tailored products

00:52:37.240 | that would enable us to be the market leader.

00:52:40.000 | But today we focus on active adult,

00:52:42.520 | but who knows where you go next.

00:52:44.800 | - Can you talk a little bit about

00:52:46.240 | how you determined your final sensor suite?

00:52:48.840 | - Yeah, so the truth is it's never final.

00:52:53.360 | So we think about generations of vehicles.

00:52:56.200 | So we have our first generation vehicle,

00:52:58.560 | which was a Ford Fusion, had a single Validyne HDL64 in it,

00:53:03.560 | bunch of cameras, radar,

00:53:05.680 | and we set some milestones based on that vehicle

00:53:09.360 | and we accomplished those milestones.

00:53:11.440 | And then once we reached the max

00:53:14.960 | in which we're able to take that vehicle,

00:53:16.160 | we then say, "Oh, we need to bring on our G2 vehicle,

00:53:19.880 | "our second generation vehicle."

00:53:21.520 | So we did that and we said,

00:53:22.680 | "Okay, we have these certain goals in mind,

00:53:24.480 | "which are pretty lofty and pretty ambitious.

00:53:26.360 | "We need incredible range,

00:53:27.720 | "incredible resolution for these things."

00:53:30.560 | And actually what we've discovered

00:53:31.560 | is that in our particular communities

00:53:33.200 | going at the speeds that we're going at,

00:53:34.800 | radar isn't particularly useful.

00:53:36.600 | So we don't have radar

00:53:37.520 | on our second generation vehicle, for example.

00:53:40.040 | But I'm sure that when we go to that third generation vehicle

00:53:42.240 | there'll be other driving factors that, you know,

00:53:44.960 | we work backwards from the milestone to say,

00:53:47.160 | "What do we need on this vehicle?"

00:53:49.160 | Maybe cost in the third generation vehicle, right?

00:53:51.040 | We may say that,

00:53:51.880 | "Hey, we need a more affordable sensor suite

00:53:54.560 | "than what exists in our second generation vehicle."

00:53:57.760 | But they're driven by technical requirements

00:53:59.520 | and that means that we are able

00:54:02.640 | to really marry the two with the vehicle.

00:54:05.840 | - I was curious, when you showed the student-led content,

00:54:08.640 | or when you showed one of the students

00:54:10.080 | in your first practice car

00:54:12.880 | had developed a traffic light sensor,

00:54:16.040 | and then you showed later on that, you know,

00:54:19.120 | you were getting student input for deep learning models

00:54:21.400 | for steering wheel turns.

00:54:23.840 | I was wondering how,

00:54:25.840 | what your system architecture kind of looks like

00:54:27.960 | in terms of the kinds of perception that you take in,

00:54:30.480 | how modular it is,

00:54:31.480 | and to what extent deep learning algorithms

00:54:33.480 | have played a part in those different parts of that system?

00:54:38.000 | - Yeah, that's a good question.

00:54:38.840 | So I really encourage folks to get familiar with ROS.

00:54:41.440 | So ROS has always been this kind of playground

00:54:45.080 | for roboticists of all different types of robots

00:54:48.280 | to be able to try things out on robots.

00:54:51.920 | And ROS 1 is particularly notorious

00:54:56.920 | for kind of hacky and hobbyist types of projects,

00:55:00.240 | but it's not meant for production.

00:55:02.440 | ROS 2 though, which is in kind of an alpha release state,

00:55:05.560 | is definitely meant for more production-oriented things.

00:55:08.520 | And the reason I mentioned ROS

00:55:09.640 | is because it has this awesome architecture

00:55:13.040 | which lets you plug and play what they call nodes

00:55:16.240 | and be able to experiment

00:55:18.880 | with different approaches to the problem.

00:55:21.320 | So for example, what, you know,

00:55:22.560 | was running that deep learning model,

00:55:24.040 | predicting steering angles,

00:55:25.560 | effectively replaced our more rules-based planner

00:55:30.560 | and perception engine.

00:55:32.760 | And we just plugged the output of that to,

00:55:35.120 | of the steering angle straight to our controller

00:55:38.760 | to just actuate the vehicle.

00:55:40.440 | And ROS is particularly good at those sorts of architectures

00:55:44.320 | and it's all open source.

00:55:46.640 | So you can do some cool stuff with it.

00:55:48.960 | - Can you tell like how you handle the liability

00:55:52.240 | and insurance for passengers for your vehicles also?

00:55:56.080 | - How we handle insurance?

00:55:57.320 | Is that a question?

00:55:58.880 | So we have a pretty cool deal

00:56:01.440 | with a company called Intact Insurance.

00:56:04.520 | And the idea is that insurance in the autonomous age

00:56:09.080 | is gonna be very different than insurance,

00:56:10.920 | you know, today, right, for human drivers

00:56:12.520 | because there's different risk assessments and whatnot.

00:56:15.040 | And one of the ways that we're able to

00:56:17.840 | prove to these insurers that, you know,

00:56:21.840 | we're good at what we do is actually sending them data,

00:56:23.920 | right, we send them data from our cars as we drive

00:56:27.640 | showing that as we move through the world,

00:56:30.600 | we accurately detected things and planned around things

00:56:33.240 | and all that good stuff.

00:56:34.440 | And then they use that data

00:56:35.360 | to inform our rates of insurance.

00:56:37.360 | I think that the future actually of insurance

00:56:39.080 | will be on a similar lines, but perhaps more extreme

00:56:41.960 | where, for example, the rates will change

00:56:44.720 | depending on the complexity of the environment.

00:56:47.520 | If we're just driving down a straight road,

00:56:49.360 | completely straight and there's zero vehicles around us,

00:56:51.480 | our insurance rate should be super low, right?

00:56:53.720 | But if we enter a city center

00:56:55.240 | and there's thousands of people and cars

00:56:57.040 | and all that crazy stuff,

00:56:58.240 | our insurance rate should just rise almost instantaneously.

00:57:02.440 | So we're partnering with someone today

00:57:04.640 | that ensures the passenger, the car, sensors,

00:57:08.640 | all that stuff, but I think there's a lot of room

00:57:10.640 | for innovation there too.

00:57:11.880 | - Did you have any problems

00:57:13.000 | like onboarding the retired people initially?

00:57:15.600 | Were they like, you know, skeptical, scared?

00:57:17.520 | And then the other question is,

00:57:19.080 | what are the like major missing pieces

00:57:21.040 | in terms of computer vision to achieve L4?

00:57:23.240 | - What was that last, missing pieces between computer vision?

00:57:27.240 | - In computer vision to achieve like L4 self-driving.

00:57:29.840 | - Gotcha.

00:57:31.280 | So one of the more interesting insights

00:57:34.720 | I think we had about retirees is that, again,

00:57:38.000 | in my kind of naive state back in 2016,

00:57:41.680 | my general feeling was retirement communities

00:57:45.240 | might not be the first to adopt this technology, right?

00:57:47.560 | Because they may be slower to adopt new technology,

00:57:51.160 | might be scared of the technology,

00:57:52.720 | all those sorts of things.

00:57:54.320 | And to kind of validate that,

00:57:57.400 | I went to talk to some senior citizens

00:57:59.280 | 'cause I talked to my own grandma.

00:58:00.440 | She hates self-driving cars.

00:58:01.720 | Sounds like that's not a good sign.

00:58:03.240 | But went to talk to these folks in these sorts of locations.

00:58:06.240 | And the really interesting thing we learned

00:58:07.440 | is that traditional consumer software or devices,

00:58:12.120 | yes, there is definitely a lag in adoption

00:58:14.160 | with senior citizens.

00:58:15.000 | And that's proven in many studies, many stats,

00:58:16.880 | that senior citizens are slower to adopt

00:58:18.600 | the Facebooks of the world or the Instagrams

00:58:21.120 | or the WhatsApps, all those sorts of things.

00:58:23.840 | Cryptocurrency, I don't know.

00:58:25.320 | But that's because they have these very well-defined

00:58:30.160 | processes that they've had for most of their lives, right?

00:58:32.600 | Instead of using Facebook, they call someone up

00:58:35.000 | and they have a chat, a conversation with someone

00:58:37.400 | about their day or stuff that's going on.

00:58:39.840 | Or they don't share a picture on Instagram,

00:58:43.840 | they physically mail a picture or something like that.

00:58:46.200 | So to change that behavior is tough, right?

00:58:48.440 | Because that's a behavior that is fundamentally different

00:58:51.360 | than what they're used to.

00:58:52.440 | They have to log onto a computer,

00:58:53.800 | go to this weird Facebook thing

00:58:55.280 | and share pictures with thousands of people.

00:58:57.360 | That's weird.

00:58:59.440 | But the difference between that and a self-driving car

00:59:02.400 | is that our experience is no different

00:59:04.480 | than the car they're used to.

00:59:06.040 | It just turns out it's being driven differently, right?

00:59:07.920 | Like they see a car, it's the same,

00:59:09.720 | similar form factor to what they're used to.

00:59:11.840 | They open the door, they sit in the back seat.

00:59:13.600 | Okay, there is a button that I have to press to say go,

00:59:15.920 | but it's pretty similar to what I'm used to in my past.

00:59:17.760 | I don't have to learn a new behavior.

00:59:18.840 | I don't have to change something that I'm used to.

00:59:22.040 | So that was our first learning.

00:59:23.000 | And then also, they actually really don't care too much

00:59:25.720 | that it's autonomous.

00:59:26.920 | They are very, when I'm in the car,

00:59:29.200 | I'm quite curious and enthusiastic about the technology

00:59:32.360 | and wanna tell them about, I don't know,

00:59:33.880 | LIDAR and deep learning and perception.

00:59:36.160 | And they just don't wanna hear any of that stuff.

00:59:39.200 | And it kind of dawned on me that the reason that is

00:59:41.560 | is because what they, senior citizens,

00:59:44.480 | have witnessed over their lifetimes

00:59:46.240 | is far more dramatic than I have, right?

00:59:47.920 | Like our oldest passenger was 93.

00:59:50.760 | And she told me a story about how when she was very young,

00:59:53.920 | she remembers literally moving on an almost daily basis

00:59:56.720 | in a horse and cart.

00:59:57.720 | So when you talk about self-driving cars to those folks,

01:00:01.240 | like they just, they couldn't care less

01:00:04.040 | because between that period and today,

01:00:05.320 | they've seen the birth of flight planes everywhere.

01:00:08.080 | They've seen car proliferation.

01:00:09.640 | They've seen scooters now.

01:00:11.560 | They've seen all of this crazy subway systems.

01:00:14.200 | So a self-driving car to them is like, oh, that's cool.

01:00:17.800 | I just want it to move me.

01:00:19.100 | So that's our biggest learning there.

01:00:21.840 | The question was computer vision,

01:00:23.360 | what needs to happen between now and level four?

01:00:26.440 | Yeah, so I think the holy grail, right?

01:00:31.320 | So if you had perfect perception,

01:00:33.360 | self-driving cars are solved.

01:00:34.640 | If we knew every object that was on the road,

01:00:36.560 | in and around us within a reasonable distance,

01:00:38.240 | self-driving cars are solved.

01:00:39.680 | False positives are accepted today, which I think is good,

01:00:45.720 | but you really wanna minimize false negatives, right?

01:00:48.400 | You want zero false negatives in the world.

01:00:51.200 | And I think that's why we still have

01:00:52.480 | a tiny bit of work to do

01:00:54.520 | because when you think about

01:00:57.040 | the reason for a test driver being in the vehicle,

01:00:59.440 | well, perception feeds everything downstream, right?

01:01:02.200 | So if you miss an object, misidentify an object,

01:01:06.360 | any of that sort of stuff,

01:01:07.840 | then that effect causes the whole stack downstream

01:01:10.800 | to become quite chaotic.

01:01:13.280 | That's why I'm excited about all those networks

01:01:15.160 | that I talked about.

01:01:16.800 | One of the other things we believe that helps us

01:01:19.560 | minimize false negatives to non-existent kind of status

01:01:23.160 | for us is that we band together multiple networks.

01:01:26.960 | So we don't just rely on a single layer of perception.

01:01:30.240 | We say different networks have different strengths.

01:01:32.720 | For example, VoxelNet is particularly good at pedestrians,

01:01:36.840 | but Pixar is not so great at pedestrians

01:01:39.240 | 'cause it's from a bird's eye view

01:01:42.040 | where pedestrians are quite thin and whatnot.

01:01:44.760 | So let's band those two networks together

01:01:46.480 | and let's also band together

01:01:47.680 | some more traditional computer vision algorithms

01:01:50.120 | that may not be processed on the entire 360 scan,

01:01:54.120 | but may be processed on a small sample,

01:01:55.760 | maybe at the front of the vehicle, for example.

01:01:57.880 | So there's just lots of little bits and pieces like that

01:02:01.080 | to go through to minimize the worst case scenario,

01:02:03.120 | which is a false negative.

01:02:05.160 | But it's clear that when you see Waymo and whatnot,

01:02:07.320 | that they feel very, very, very close to that sort of state.

01:02:11.120 | - You mentioned that weather was one of the main reasons

01:02:14.360 | this was a great place to start.

01:02:16.520 | Can you talk about hurricanes?

01:02:19.000 | - Yes, it was funny.

01:02:20.400 | I got a question recently from Alex Roy.

01:02:23.520 | Me and Lex were just talking about,

01:02:26.200 | okay, in the event of a hurricane, right?

01:02:28.560 | Let's not talk about the technology second,

01:02:29.840 | but in the event of a hurricane,

01:02:31.280 | we've all seen those pictures of people

01:02:33.160 | getting on the freeways

01:02:34.000 | and trying to get out of the path of the hurricane, right?

01:02:36.200 | How is that gonna work in a world

01:02:37.520 | where self-driving cars are everywhere

01:02:39.880 | and personally driven vehicles

01:02:41.080 | are maybe more of the smaller size?

01:02:44.600 | I don't quite have an answer to that yet,

01:02:47.040 | but I think it's an interesting kind of thought problem.

01:02:49.280 | From a technology perspective,

01:02:50.840 | the really important part of weather for us

01:02:56.360 | is remote operation.

01:02:57.840 | So inside every one of our, sorry,

01:03:00.320 | all of our vehicles have a cellular connection, right?

01:03:04.160 | And each of those vehicles is connected

01:03:06.280 | to a remote operator that's sat

01:03:08.840 | in somewhat close proximity to that vehicle.

01:03:11.560 | And that remote operator has a few jobs.

01:03:13.840 | One is to just ensure the safe operation of the vehicle,

01:03:16.520 | make sure that that vehicle is doing

01:03:17.680 | as it's intended to do, all those good things.

01:03:20.520 | But another is to make sure that the operational domain

01:03:22.840 | that we are currently operating in

01:03:24.280 | is the one that it's designed for.

01:03:26.200 | So all these different camera feeds

01:03:27.560 | are being live streamed to this remote operator.

01:03:29.720 | And if there is sudden downpour of rain,

01:03:32.160 | that remote operator has the ability

01:03:34.620 | to bring that vehicle to a safe stop

01:03:38.120 | until that rain shower disappears or whatever,

01:03:41.400 | or hurricane, whatever it may be.

01:03:45.120 | But there are companies,

01:03:46.480 | I was pitched recently by a company

01:03:47.720 | that's building weather forecasting

01:03:49.960 | on a scale that is not really used today,

01:03:54.480 | but really microclimate.

01:03:55.920 | So thinking about just like this small subsection

01:03:58.880 | of the villages, predicting and understanding

01:04:01.320 | exact weather within those regions,

01:04:03.160 | and then having webhooks to tell you or us, Voyage,

01:04:06.000 | that that's about to happen.

01:04:07.540 | So there's a lot of cool stuff happening there,

01:04:09.180 | but remote operators currently kind of the eyes

01:04:11.240 | and ears of our cars to prevent that sort of issue.

01:04:14.720 | So please give Oliver a big hand.

01:04:16.840 | Thank you very much.

01:04:17.680 | Love you guys.

01:04:18.520 | (audience applauding)

01:04:21.680 | (upbeat music)

01:04:24.260 | (upbeat music)

01:04:26.840 | (upbeat music)

01:04:29.420 | (upbeat music)

01:04:32.000 | (upbeat music)

01:04:34.580 | [BLANK_AUDIO]

Oliver Cameron (CEO, Voyage) - MIT Self-Driving Cars

Chapters