Sertac Karaman: Robots That Fly and Robots That Drive

00:00:00.000 | The following is a conversation with Sertac Karaman,

00:00:03.120 | a professor at MIT,

00:00:04.680 | co-founder of the autonomous vehicle company Optimus Ride,

00:00:07.960 | and is one of the top roboticists in the world,

00:00:10.880 | including robots that drive and robots that fly.

00:00:14.620 | To me, personally, he has been a mentor,

00:00:17.520 | a colleague, and a friend.

00:00:19.560 | He's one of the smartest, most generous people I know.

00:00:22.600 | So it was a pleasure and honor to finally sit down with him

00:00:25.280 | for this recorded conversation.

00:00:27.640 | This is the Artificial Intelligence Podcast.

00:00:30.160 | If you enjoy it, subscribe on YouTube,

00:00:32.360 | review it with five stars on Apple Podcasts,

00:00:34.600 | support on Patreon, or simply connect with me on Twitter

00:00:37.800 | at Lex Friedman, spelled F-R-I-D-M-A-N.

00:00:41.400 | As usual, I'll do a few minutes of ads now

00:00:43.800 | and never any ads in the middle

00:00:45.280 | that can break the flow of the conversation.

00:00:47.360 | I hope that works for you

00:00:48.680 | and doesn't hurt the listening experience.

00:00:51.640 | This show is presented by Cash App,

00:00:53.680 | the number one finance app in the App Store.

00:00:56.040 | When you get it, use the code LEXPODCAST.

00:00:59.440 | Cash App lets you send money to friends,

00:01:01.520 | buy Bitcoin, and invest in the stock market

00:01:03.720 | with as little as $1.

00:01:05.640 | Since Cash App allows you to send

00:01:07.280 | and receive money digitally,

00:01:09.040 | let me mention a surprising fact about physical money.

00:01:12.460 | It costs 2.4 cents to produce a single penny.

00:01:16.780 | In fact, I think it costs $85 million annually

00:01:19.960 | to produce them.

00:01:20.880 | That's a crazy little fact about physical money.

00:01:25.160 | So again, if you get Cash App

00:01:26.840 | from the App Store or Google Play

00:01:28.400 | and use the code LEXPODCAST, you get $10,

00:01:31.400 | and Cash App will also donate $10 to Thirst,

00:01:34.080 | an organization that is helping to advance robotics

00:01:36.600 | and STEM education for young people around the world.

00:01:40.200 | And now, here's my conversation with Sertac Karaman.

00:01:43.700 | Since you have worked extensively on both,

00:01:46.560 | what is the more difficult task,

00:01:49.000 | autonomous flying or autonomous driving?

00:01:51.640 | - That's a good question.

00:01:52.880 | I think that autonomous flying,

00:01:56.040 | just kind of doing it for consumer drones and so on,

00:01:58.560 | the kinds of applications that we're looking at right now

00:02:01.480 | is probably easier.

00:02:02.480 | And so I think that that's maybe one of the reasons

00:02:04.280 | why it took off, like literally,

00:02:06.720 | a little earlier than the autonomous cars.

00:02:09.120 | But I think if you look ahead,

00:02:10.520 | I would think that the real benefits of autonomous flying,

00:02:14.480 | unleashing them in like transportation, logistics,

00:02:16.600 | and so on, I think it's a lot harder than autonomous driving.

00:02:19.440 | So I think my guess is that we've seen

00:02:22.040 | a few kind of machines fly here and there,

00:02:24.480 | but we really haven't yet seen any kind of,

00:02:27.160 | machine, like at massive scale,

00:02:30.520 | large scale being deployed and flown and so on.

00:02:33.120 | And I think that's gonna be after we kind of resolve

00:02:36.400 | some of the large scale deployments of autonomous driving.

00:02:39.480 | - So what's the hard part?

00:02:40.440 | What's your intuition behind why at scale,

00:02:44.400 | when consumer facing drones are tough?

00:02:47.560 | - So I think in general, at scale is tough.

00:02:51.680 | Like for example, when you think about it,

00:02:54.440 | we have actually deployed a lot of robots

00:02:57.120 | in the, let's say the past 50 years.

00:02:59.880 | - We as academics or we business entrepreneur?

00:03:02.720 | - I think we as humanity.

00:03:04.160 | - Humanity?

00:03:05.000 | - A lot of people working on it.

00:03:06.360 | (laughing)

00:03:07.480 | So we humans deployed a lot of robots.

00:03:09.400 | And I think that, but when you think about it,

00:03:12.320 | robots, they're autonomous, they work,

00:03:15.520 | they work on their own,

00:03:17.000 | but they are either like in isolated environments

00:03:19.960 | or they are in sort of,

00:03:22.400 | they may be at scale,

00:03:26.320 | but they're really confined to a certain environment

00:03:28.480 | that they don't interact so much with humans.

00:03:30.640 | And so, they work in, I don't know,

00:03:32.280 | factory floors, warehouses, they work on Mars,

00:03:35.760 | they are fully autonomous over there.

00:03:38.240 | But I think that the real challenge of our time

00:03:41.440 | is to take these vehicles

00:03:43.680 | and put them into places where humans are present.

00:03:47.080 | So now I know that there's a lot of like human robot

00:03:49.400 | interaction type of things that need to be done.

00:03:52.040 | And so, that's one thing,

00:03:53.520 | but even just from the fundamental algorithms and systems

00:03:57.360 | and the business cases, or maybe the business models,

00:04:01.200 | even like architecture, planning, societal issues,

00:04:03.720 | legal issues, there's a whole bunch of pack of things

00:04:06.720 | that are related to us putting robotic vehicles

00:04:10.240 | into human present environments.

00:04:12.360 | And these humans, they will not potentially be

00:04:16.160 | even trained to interact with them.

00:04:18.320 | They may not even be using the services

00:04:20.320 | that are provided by these vehicles.

00:04:21.840 | They may not even know that they're autonomous.

00:04:23.680 | They're just doing their thing,

00:04:25.360 | living in environments that are designed for humans,

00:04:27.600 | not for robots.

00:04:28.800 | And that I think is one of the biggest challenges,

00:04:32.120 | I think, of our time, to put vehicles there.

00:04:35.000 | And to go back to your question,

00:04:36.680 | I think doing that at scale,

00:04:40.040 | meaning you go out in a city

00:04:42.320 | and you have thousands or tens of thousands

00:04:46.600 | of autonomous vehicles that are going around.

00:04:48.800 | It is so dense to the point where if you see one of them,

00:04:52.280 | you look around, you see another one.

00:04:54.280 | It is that dense.

00:04:55.360 | And that density,

00:04:57.040 | we've never done anything like that before.

00:04:59.360 | And I would bet that that kind of density

00:05:01.920 | will first happen with autonomous cars,

00:05:04.960 | because I think we can bend the environment a little bit.

00:05:08.520 | Especially kind of making them safe is a lot easier

00:05:13.240 | when they're on the ground.

00:05:15.800 | When they're in the air,

00:05:16.920 | it's a little bit more complicated.

00:05:19.440 | But I don't see that there's gonna be a big separation.

00:05:21.600 | I think that there will come a time

00:05:23.240 | that we're gonna quickly see these things unfold.

00:05:25.720 | - Do you think there will be a time

00:05:27.560 | where there's tens of thousands of delivery drones

00:05:30.440 | that fill the sky?

00:05:31.960 | - You know, I think it's possible, to be honest.

00:05:33.880 | Delivery drones is one thing,

00:05:35.080 | but you can imagine for transportation,

00:05:38.160 | like an important use case is,

00:05:40.600 | you know, we're in Boston,

00:05:41.480 | you wanna go from Boston to New York.

00:05:43.480 | And you wanna do it from the top of this building

00:05:46.280 | to the top of another building in Manhattan.

00:05:48.480 | And you're gonna do it in one and a half hours.

00:05:50.840 | And that's a big opportunity, I think.

00:05:53.440 | - Personal transport, so like you and me be a friend.

00:05:56.000 | Like almost like Uber. - Yeah, or almost like an Uber.

00:05:58.600 | So like four people, six people, eight people.

00:06:01.680 | In our work in autonomous vehicles, I see that.

00:06:03.960 | So there's kind of like a bit of a need

00:06:05.240 | for one person transport, but also like a few people.

00:06:09.080 | So you and I could take the trip together,

00:06:10.880 | we could have lunch.

00:06:13.400 | You know, I think kind of sounds crazy,

00:06:15.200 | maybe even sounds a bit cheesy,

00:06:16.720 | but I think that those kinds of things

00:06:19.000 | are some of the real opportunities.

00:06:20.440 | And I think, you know, it's not like the typical airplane

00:06:24.360 | and the airport would disappear very quickly.

00:06:26.600 | But I would think that, you know,

00:06:28.920 | many people would feel like they would spend

00:06:31.600 | an extra $100 on doing that,

00:06:33.760 | and cutting that four hour travel

00:06:36.240 | down to one and a half hours.

00:06:37.880 | - So how feasible are flying cars?

00:06:39.800 | It's been the dream.

00:06:41.320 | It's like when people imagine the future for 50 plus years,

00:06:44.400 | they think flying cars.

00:06:45.680 | It's like all technologies, it's cheesy to think about now

00:06:50.840 | because it seems so far away, but overnight it can change.

00:06:54.720 | But just technically speaking, in your view,

00:06:56.920 | how feasible is it to make that happen?

00:06:59.280 | - I'll get to that question.

00:07:00.440 | But just one thing is that I think, you know,

00:07:02.840 | sometimes we think about what's gonna happen

00:07:05.960 | in the next 50 years.

00:07:07.160 | It's just really hard to guess, right?

00:07:08.640 | Next 50 years, I don't know.

00:07:09.800 | I mean, we could get what's gonna happen

00:07:11.520 | in transportation in the next 50 years.

00:07:12.920 | We could get flying saucers.

00:07:14.600 | I could bet on that.

00:07:16.080 | I think there's a 50/50 chance that, you know,

00:07:17.960 | like you can build machines that can ionize the air

00:07:20.120 | around them and push it down with magnets

00:07:22.280 | and they would fly like a flying saucer.

00:07:24.360 | That is possible.

00:07:25.920 | And it might happen in the next 50 years.

00:07:27.920 | So it's a bit hard to guess,

00:07:29.480 | like when you think about 50 years before.

00:07:32.040 | But I would think that, you know,

00:07:33.920 | there's this kind of notion where

00:07:37.880 | there's a certain type of airspace

00:07:39.400 | that we call the agile airspace.

00:07:41.440 | And there's good amount of opportunities in that airspace.

00:07:44.080 | So that would be the space that is kind of

00:07:47.000 | a little bit higher than the place

00:07:49.240 | where you can throw a stone.

00:07:51.440 | Because that's a tough thing when you think about it.

00:07:52.960 | You know, it takes a kid and a stone

00:07:54.360 | to take an aircraft down and then what happens?

00:07:58.360 | But, you know, imagine the airspace that's high enough

00:08:02.920 | so that you cannot throw a stone,

00:08:05.480 | but it is low enough that you're not interacting

00:08:07.560 | with the very large aircraft

00:08:10.200 | that are, you know, flying several thousand feet above.

00:08:15.440 | And that airspace is underutilized.

00:08:18.440 | Or it's actually kind of not utilized at all.

00:08:20.280 | - Yeah, that's right.

00:08:21.120 | - So there's, you know, there's like recreational people

00:08:23.120 | kind of fly every now and then, but it's very few.

00:08:25.240 | Like if you look up in the sky,

00:08:26.800 | you may not see any of them at any given time.

00:08:29.960 | Every now and then you'll see one airplane

00:08:31.640 | kind of utilizing that space and you'll be surprised.

00:08:34.360 | And the moment you're outside of an airport a little bit,

00:08:36.840 | like it just kind of flies off and then it goes out.

00:08:39.760 | And I think utilizing that airspace,

00:08:42.000 | the technical challenges there is, you know,

00:08:44.920 | building an autonomy and ensuring

00:08:49.080 | that that kind of autonomy is safe.

00:08:51.640 | Ultimately, I think it is going to be building

00:08:55.800 | in complex software or complicated

00:08:58.480 | so that it's maybe a few orders of magnitude

00:09:01.760 | more complicated than what we have on aircraft today.

00:09:05.240 | And at the same time, ensuring just like we ensure

00:09:08.200 | on aircraft, ensuring that it's safe.

00:09:10.440 | And so that becomes like building that kind

00:09:13.080 | of complicated hardware and software becomes a challenge.

00:09:16.240 | Especially when, you know, you build that hardware,

00:09:19.800 | I mean, you build that software with data.

00:09:21.920 | And so, you know, it's, of course,

00:09:26.680 | there's some rule-based software in there

00:09:28.320 | that kind of do a certain set of things,

00:09:29.840 | but then, you know, there's a lot of training there.

00:09:32.360 | - Do you think machine learning will be key

00:09:34.680 | to delivering safe vehicles in the future,

00:09:39.560 | especially flight?

00:09:40.680 | - Not maybe the safe part,

00:09:42.200 | but I think the intelligent part.

00:09:44.560 | I mean, there are certain things

00:09:45.640 | that we do it with machine learning

00:09:47.440 | and it's just, there's like right now no other way.

00:09:50.280 | And I don't know how else they could be done.

00:09:53.520 | And, you know, there's always this conundrum.

00:09:56.240 | I mean, we could maybe gather billions of programmers,

00:10:04.120 | humans who program perception algorithms

00:10:08.960 | that detect things in the sky and whatever,

00:10:11.000 | or, you know, we, I don't know,

00:10:12.920 | we maybe even have robots like learning

00:10:14.760 | a simulation environment and transfer.

00:10:17.080 | And they might be learning a lot better

00:10:19.200 | in a simulation environment than a billion humans

00:10:22.960 | put their brains together and try to program.

00:10:25.400 | Humans, pretty limited.

00:10:26.760 | - So what's the role of simulations with drones?

00:10:30.280 | You've done quite a bit of work there.

00:10:32.200 | How promising, just the very thing you said just now,

00:10:34.920 | how promising is the possibility of training

00:10:38.760 | and developing a safe flying robot in simulation

00:10:43.760 | and deploying it and having that work pretty well

00:10:46.960 | in the real world?

00:10:48.360 | - I think that, you know, a lot of people,

00:10:49.920 | when they hear simulation,

00:10:51.760 | they will focus on training immediately.

00:10:53.680 | But I think one thing that you said,

00:10:55.320 | which was interesting, it's developing.

00:10:57.520 | I think simulation environments actually could be key

00:11:00.120 | and great for development.

00:11:01.800 | And that's not new.

00:11:03.240 | Like for example, you know, there's people

00:11:06.840 | in the automotive industry have been using

00:11:09.120 | dynamic simulation for like decades now.

00:11:11.720 | And it's pretty standard that, you know,

00:11:14.120 | you would build and you would simulate.

00:11:16.120 | If you want to build an embedded controller,

00:11:18.080 | you plug that kind of embedded computer

00:11:20.560 | into another computer,

00:11:22.040 | that other computer would simulate and so on.

00:11:24.560 | And I think, you know, fast forward these things,

00:11:26.320 | you can create pretty crazy simulation environments.

00:11:29.680 | Like for instance, one of the things

00:11:32.080 | that has happened recently and that, you know,

00:11:34.840 | we can do now is that we can simulate cameras

00:11:37.680 | a lot better than we used to simulate them.

00:11:39.640 | We were able to simulate them before.

00:11:41.000 | And that's, I think we just hit the elbow

00:11:43.600 | on that kind of improvement.

00:11:45.360 | I would imagine that with improvements in hardware,

00:11:48.560 | especially, and with improvements in machine learning,

00:11:52.400 | I think that we would get to a point

00:11:54.160 | where we can simulate cameras very, very well.

00:11:57.280 | - Simulate cameras means simulate how a real camera

00:12:01.640 | would see the real world.

00:12:03.200 | Therefore you can explore the limitations of that.

00:12:07.540 | You can train perception algorithms on that in simulation,

00:12:11.600 | all that kind of stuff.

00:12:12.440 | - Exactly.

00:12:13.280 | So, you know, it has been easier to simulate

00:12:17.300 | what we would call interceptive sensors,

00:12:19.340 | like internal sensors.

00:12:20.520 | So for example, inertial sensing has been easy to simulate.

00:12:23.960 | It has also been easy to simulate dynamics,

00:12:26.560 | like physics that are governed

00:12:28.520 | by ordinary differential equations.

00:12:30.080 | I mean, like how a car goes around,

00:12:32.320 | maybe how it rolls on the road,

00:12:33.840 | how it interacts with the road,

00:12:36.560 | or even an aircraft flying around,

00:12:38.440 | like the dynamic physics of that.

00:12:40.580 | What has been really hard

00:12:42.320 | has been to simulate extraceptive sensors,

00:12:44.960 | sensors that kind of like look out from the vehicle.

00:12:48.540 | And that's a new thing that's coming,

00:12:49.980 | like laser range finders that are a little bit easier.

00:12:53.480 | Cameras, radars are a little bit tougher.

00:12:56.240 | I think once we nail that down,

00:12:58.480 | the next challenge I think in simulation

00:13:01.360 | will be to simulate human behavior.

00:13:03.380 | That's also extremely hard.

00:13:05.280 | Even when you imagine like how a human driven car

00:13:09.880 | would act around, even that is hard.

00:13:11.980 | But imagine trying to simulate, you know,

00:13:14.320 | a model of a human just doing a bunch of gestures

00:13:17.780 | and so on, and you know, it's actually simulated.

00:13:20.320 | It's not captured like with motion capture,

00:13:22.840 | but it is simulated.

00:13:23.680 | That's very hard.

00:13:24.500 | In fact, today I get involved a lot

00:13:27.480 | with like sort of this kind of

00:13:28.600 | very high end rendering projects.

00:13:30.900 | And I have like this test that I pass it to my friends

00:13:33.320 | or my mom, you know, I send like two photos,

00:13:35.440 | two kind of pictures and I say,

00:13:37.480 | rendered, which one is rendered, which one is real?

00:13:40.080 | And it's pretty hard to distinguish,

00:13:41.600 | except I realized, except when we put humans in there.

00:13:45.200 | It's possible that our brains are trained in a way

00:13:48.120 | that we recognize humans extremely well.

00:13:50.720 | But we don't so much recognize the built environments

00:13:53.120 | because built environments sort of came after per se,

00:13:55.680 | we evolved into sort of being humans,

00:13:57.960 | but humans were always there.

00:14:00.040 | Same thing happens, for example,

00:14:01.340 | you look at like monkeys and you can't distinguish

00:14:04.240 | one from another, but they sort of do.

00:14:06.860 | And it's very possible that they look at humans,

00:14:08.600 | it's kind of pretty hard to distinguish one from another,

00:14:10.760 | but we do.

00:14:11.920 | And so our eyes are pretty well trained to look at humans

00:14:14.640 | and understand if something is off, we will get it.

00:14:18.260 | We may not be able to pinpoint it.

00:14:19.620 | So in my typical friend test or mom test,

00:14:21.920 | what would happen is that we'd put like a human walking

00:14:24.680 | in a thing and they say, this is not right.

00:14:29.320 | Something is off in this video.

00:14:31.360 | I don't know what, but I can tell you it's the human.

00:14:34.200 | I can take the human and I can show you like inside

00:14:36.560 | of a building or like an apartment,

00:14:38.800 | and it will look like if we had time to render it,

00:14:41.640 | it will look great.

00:14:42.560 | And this should be no surprise,

00:14:43.800 | a lot of movies that people are watching,

00:14:45.460 | it's all computer generated.

00:14:47.720 | Nowadays, even you watch a drama movie

00:14:50.440 | and like there's nothing going on action wise,

00:14:52.520 | but it turns out it's kind of like cheaper,

00:14:54.160 | I guess to render the background.

00:14:55.720 | And so they would.

00:14:57.360 | - But how do we get there?

00:14:59.680 | How do we get a human that's would pass the mom/friend test,

00:15:04.680 | a simulation of a human walking?

00:15:10.080 | So do you think that's something we can creep up to

00:15:13.440 | by just doing kind of a comparison learning

00:15:17.220 | where you have humans annotate what's more realistic

00:15:21.020 | and not just by watching?

00:15:23.060 | Like what's the path?

00:15:24.220 | 'Cause it seems totally mysterious

00:15:26.460 | how we simulate human behavior.

00:15:29.820 | - It's hard because a lot of the other things

00:15:32.180 | that I mentioned to you, including simulating cameras,

00:15:35.140 | it is, the thing there is that we know the physics,

00:15:40.140 | we know how it works like in the real world,

00:15:43.780 | and we can write some rules and we can do that.

00:15:46.560 | Like for example, simulating cameras,

00:15:48.180 | there's this thing called ray tracing.

00:15:50.060 | I mean, you literally just kind of imagine,

00:15:52.780 | it's very similar to, it's not exactly the same,

00:15:54.920 | but it's very similar to tracing photon by photon.

00:15:57.780 | They're going around, bouncing on things

00:15:59.600 | and coming to your eye.

00:16:01.260 | But human behavior, developing a dynamic,

00:16:04.700 | like a model of that, that is mathematical

00:16:08.440 | so that you can put it into a processor

00:16:11.580 | that would go through that, that's gonna be hard.

00:16:13.860 | And so what else do you got?

00:16:15.660 | You can collect data, right?

00:16:17.900 | And you can try to match the data.

00:16:19.980 | Or another thing that you can do is that,

00:16:21.580 | you can show the front test, you can say this or that

00:16:24.740 | and this or that, and that will be labeling.

00:16:27.000 | Anything that requires human labeling,

00:16:28.740 | ultimately we're limited by the number of humans

00:16:31.080 | that we have available at our disposal

00:16:33.740 | and the things that they can do.

00:16:35.200 | They have to do a lot of other things

00:16:36.400 | than also labeling this data.

00:16:38.000 | So that modeling human behavior part is, I think,

00:16:43.480 | we're gonna realize it's very tough.

00:16:45.640 | And I think that also affects

00:16:47.360 | our development of autonomous vehicles.

00:16:50.760 | I see that in self-driving as well.

00:16:52.480 | Like you wanna use, so you're building self-driving,

00:16:55.720 | at the first time, like right after Urban Challenge,

00:16:59.240 | I think everybody focused on localization,

00:17:02.140 | mapping and localization.

00:17:03.560 | Slam algorithms came in, Google was just doing that.

00:17:06.840 | And so building these HD maps,

00:17:08.580 | basically that's about knowing where you are.

00:17:11.600 | And then five years later in 2012, 2013,

00:17:14.000 | came the kind of coding code AI revolution

00:17:16.160 | and that started telling us where everybody else is.

00:17:19.160 | But we're still missing

00:17:20.720 | what everybody else is gonna do next.

00:17:23.160 | And so you wanna know where you are,

00:17:24.520 | you wanna know what everybody else is,

00:17:26.200 | hopefully you know what you're gonna do next

00:17:28.580 | and then you wanna predict what other people

00:17:30.120 | are going to do and that last bit

00:17:32.520 | has been a real challenge.

00:17:35.520 | - What do you think is the role, your own,

00:17:38.160 | of your, the ego vehicle, the robot,

00:17:41.880 | the you, the robotic you in controlling

00:17:47.120 | and having some control of how the future unrolls,

00:17:49.720 | of what's gonna happen in the future?

00:17:51.640 | That seems to be a little bit ignored

00:17:53.760 | in trying to predict the future

00:17:55.280 | is how you yourself can affect that future

00:17:59.220 | by being either aggressive or less aggressive

00:18:03.440 | or signaling in some kind of way,

00:18:06.380 | sort of this kind of game theoretic dance.

00:18:08.760 | Seems to be ignored for the moment.

00:18:10.680 | - It's, yeah, it's totally ignored.

00:18:12.640 | I mean, it's quite interesting actually,

00:18:15.160 | like how we interact with things

00:18:19.360 | versus we interact with humans.

00:18:21.620 | Like so if you see a vehicle that's completely empty

00:18:25.660 | and it's trying to do something,

00:18:27.480 | all of a sudden it becomes a thing.

00:18:29.520 | So interact it with, like you interact with this table

00:18:32.600 | and so you can throw your backpack

00:18:34.280 | or you can kick it, put your feet on it

00:18:37.140 | and things like that.

00:18:38.500 | But when it's a human,

00:18:39.740 | there's all kinds of ways of interacting with a human.

00:18:42.100 | So if, like you and I are face to face, we're very civil,

00:18:45.780 | we talk and we understand each other for the most part.

00:18:48.540 | We'll see, you just, that's done.

00:18:50.180 | You never know what's gonna happen.

00:18:52.260 | But the thing is that, like for example,

00:18:54.740 | you and I might interact through YouTube comments

00:18:57.580 | and the conversation may go at a totally different angle.

00:19:01.060 | And so I think people kind of abusing these autonomous

00:19:05.360 | vehicles is a real issue in some sense.

00:19:08.280 | And so when you're an ego vehicle,

00:19:09.940 | you're trying to coordinate your way, make your way,

00:19:13.120 | it's actually kind of harder than being a human.

00:19:15.520 | It's like, you not only need to be as smart

00:19:19.680 | as kind of humans are, but you also, you're a thing.

00:19:22.120 | So they're gonna abuse you a little bit.

00:19:23.880 | So you need to make sure that you can get around

00:19:27.400 | and do something.

00:19:28.360 | So I, in general, believe in that sort of

00:19:33.360 | game theoretic aspects.

00:19:34.620 | I've actually personally have done quite a few papers,

00:19:37.740 | both on that kind of game theory and also like this kind of

00:19:42.020 | understanding people's social value orientation,

00:19:44.300 | for example.

00:19:45.140 | Some people are aggressive, some people not so much.

00:19:48.620 | And a robot could understand that by just looking

00:19:52.980 | at how people drive.

00:19:54.560 | And as they kind of come and approach,

00:19:56.280 | you can actually understand, like if someone is gonna

00:19:58.340 | be aggressive or not as a robot,

00:20:01.280 | and you can make certain decisions.

00:20:03.100 | - Well, in terms of predicting what they're going to do,

00:20:05.760 | the hard question is you as a robot,

00:20:08.780 | should you be aggressive or not?

00:20:11.140 | When faced with an aggressive robot.

00:20:12.960 | Right now, it seems like aggressive is a very dangerous

00:20:15.500 | thing to do because it's costly from a societal perspective,

00:20:20.500 | how you're perceived.

00:20:22.700 | People are not very accepting of aggressive robots

00:20:25.380 | in modern society.

00:20:27.120 | - I think that's accurate.

00:20:28.320 | So, it really is.

00:20:30.960 | And so I'm not entirely sure how to go about,

00:20:34.740 | but I know for a fact that how these robots interact

00:20:37.940 | with other people in there is going to be,

00:20:40.340 | and that interaction is always gonna be there.

00:20:42.420 | I mean, you could be interacting with other vehicles

00:20:44.860 | or other just people kind of like walking around.

00:20:48.020 | And like I said, the moment there's nobody in the seat,

00:20:51.740 | it's like an empty thing just rolling off the street.

00:20:54.220 | It becomes like no different than any other thing

00:20:59.020 | that's not human.

00:20:59.860 | And so people, and maybe abuse is the wrong word,

00:21:03.080 | but people, maybe rightfully even, they feel like,

00:21:06.720 | this is a human present environment,

00:21:08.360 | it's designed for humans to be,

00:21:10.200 | and they kind of, they want to own it.

00:21:13.000 | And then the robots, they would need to understand it

00:21:16.000 | and they would need to respond in a certain way.

00:21:18.520 | And I think that this actually opens up

00:21:21.000 | quite a few interesting societal questions for us

00:21:23.440 | as we deploy, like we talk, robots at large scale.

00:21:26.880 | So what would happen when we try to deploy robots

00:21:29.320 | at large scale, I think is that we can design systems

00:21:32.040 | in a way that they're very efficient,

00:21:34.460 | or we can design them that they're very sustainable.

00:21:37.120 | But ultimately the sustainability efficiency trade-offs,

00:21:39.840 | like they're gonna be right in there,

00:21:42.520 | and we're gonna have to make some choices.

00:21:44.280 | Like we're not gonna be able to just kind of put it aside.

00:21:47.400 | So for example, we can be very aggressive

00:21:49.960 | and we can reduce transportation delays,

00:21:52.360 | increase capacity of transportation,

00:21:54.760 | or we can be a lot nicer and allow other people

00:21:57.920 | to kind of quote unquote, own the environment

00:21:59.880 | and live in a nice place, and then efficiency will drop.

00:22:04.240 | So when you think about it,

00:22:05.240 | I think sustainability gets attached to energy consumption

00:22:09.440 | or environmental impact immediately.

00:22:11.160 | And those are there, but like livability

00:22:13.940 | is another sustainability impact.

00:22:15.680 | So you create an environment that people wanna live in.

00:22:19.240 | And if robots are going around being aggressive,

00:22:22.080 | you don't wanna live in that environment, maybe.

00:22:24.680 | However, you should note that if you're not being aggressive

00:22:27.000 | then you're probably taking up some delays

00:22:29.640 | in transportation and this and that.

00:22:31.600 | So you're always balancing that.

00:22:34.760 | And I think this choice has always been there

00:22:37.120 | in transportation, but I think the more autonomy comes in,

00:22:40.000 | the more explicit the choice becomes.

00:22:42.540 | - Yeah, and when it becomes explicit,

00:22:45.280 | then we can start to optimize it.

00:22:47.160 | And then we'll get to ask the very difficult societal

00:22:49.980 | questions of what do we value more, efficiency

00:22:52.240 | or sustainability?

00:22:53.880 | It's kind of interesting.

00:22:54.720 | - I think that will happen.

00:22:56.160 | I think we're gonna have to like,

00:22:58.080 | I think that the interesting thing about like

00:23:00.280 | the whole autonomous vehicles question,

00:23:01.840 | I think is also kind of, I think a lot of times,

00:23:06.280 | you know, we have focused on technology development,

00:23:09.080 | like hundreds of years and, you know,

00:23:12.380 | the products somehow followed and then, you know,

00:23:14.560 | we got to make these choices and things like that.

00:23:16.560 | But this is a good time that, you know,

00:23:18.680 | we even think about, you know, autonomous taxi type

00:23:21.800 | of deployments and the systems that would evolve from there.

00:23:25.400 | And you realize the business models are different,

00:23:28.280 | the impact on architecture is different, urban planning,

00:23:31.660 | you get into like regulations,

00:23:34.160 | and then you get into like these issues that you didn't

00:23:37.440 | think about before, but like sustainability and ethics

00:23:39.820 | is like right in the middle of it.

00:23:42.080 | I mean, even testing autonomous vehicles,

00:23:43.800 | like think about it, you're testing autonomous vehicles

00:23:45.840 | in human present environments.

00:23:47.120 | I mean, the risk may be very small, but still, you know,

00:23:50.400 | it's a strictly greater than zero risk

00:23:54.360 | that you're putting people into.

00:23:56.180 | And so then you have that innovation, you know,

00:23:59.720 | risk trade off that you're in that somewhere.

00:24:03.080 | And we understand that pretty well now is that

00:24:07.580 | if we don't test, at least the development will be slower.

00:24:12.360 | I mean, it doesn't mean that we're not gonna be able

00:24:14.560 | to develop, I think it's gonna be pretty hard actually,

00:24:16.840 | maybe we can, I don't know, but the thing is that

00:24:20.240 | those kinds of trade offs we already are making.

00:24:23.080 | And as these systems become more ubiquitous,

00:24:25.480 | I think those trade offs will just really hit.

00:24:28.860 | - So you are one of the founders of Optimus Ride,

00:24:33.040 | an autonomous vehicle company, we'll talk about it.

00:24:35.040 | But let me, on that point, ask maybe good examples,

00:24:40.040 | keeping Optimus Ride out of this question,

00:24:46.200 | sort of exemplars of different strategies

00:24:49.760 | on the spectrum of innovation and safety or caution.

00:24:54.760 | So like Waymo, Google self-driving car,

00:24:59.040 | Waymo represents maybe a more cautious approach.

00:25:03.680 | And then you have Tesla on the other side,

00:25:06.760 | headed by Elon Musk that represents a more,

00:25:10.200 | however, which adjective you wanna use,

00:25:12.120 | aggressive, innovative, I don't know.

00:25:14.520 | But what do you think about the difference

00:25:18.280 | between the two strategies in your view?

00:25:21.360 | What's more likely, what's needed

00:25:24.120 | and is more likely to succeed in the short term

00:25:27.080 | and the long term?

00:25:28.800 | - Definitely some sort of a balance

00:25:31.320 | is kind of the right way to go.

00:25:33.120 | But I do think that the thing that is the most important

00:25:36.520 | is actually like an informed public.

00:25:39.120 | So I don't mind, I personally,

00:25:43.400 | like if I were in some place, I wouldn't mind so much,

00:25:46.920 | like taking a certain amount of risk.

00:25:48.880 | Some other people might.

00:25:51.980 | And so I think the key is for people to be informed

00:25:55.720 | and so that they can, ideally, they can make a choice.

00:25:58.980 | In some cases, that kind of choice,

00:26:01.920 | making that unanimously is of course very hard.

00:26:06.360 | But I don't think it's actually that hard to inform people.

00:26:10.440 | So I think in one case, like for example,

00:26:14.400 | even the Tesla approach, I don't know,

00:26:17.400 | it's hard to judge how informed it is,

00:26:19.160 | but it is somewhat informed.

00:26:20.400 | I mean, things kind of come out,

00:26:22.000 | I think people know what they're taking

00:26:23.560 | and things like that and so on.

00:26:25.780 | But I think the underlying,

00:26:28.380 | I do think that these two companies

00:26:30.140 | are a little bit kind of representing like,

00:26:32.280 | of course, one of them seems a bit safer,

00:26:36.360 | the other one or whatever the objective for that is

00:26:40.040 | and the other one seems more aggressive

00:26:41.640 | or whatever the objective for that is.

00:26:43.000 | But I think when you turn the tables,

00:26:45.960 | there are actually two other orthogonal dimensions

00:26:48.280 | that these two are focusing on.

00:26:50.220 | On the one hand, for Waymo, I can see that they're,

00:26:53.160 | I mean, I think they a little bit see it as research as well.

00:26:57.200 | So they kind of, I'm not sure if they're like

00:26:59.100 | really interested in like an immediate product.

00:27:02.620 | They talk about it.

00:27:06.160 | Sometimes there's some pressure to talk about it.

00:27:08.200 | So they kind of go for it.

00:27:09.620 | But I think that they're thinking,

00:27:13.600 | maybe in the back of their minds,

00:27:14.740 | maybe they don't put it this way,

00:27:15.840 | but I think they realize

00:27:17.800 | that we're building like a new engine.

00:27:20.080 | It's kind of like call it the AI engine

00:27:21.880 | or whatever that is.

00:27:22.800 | And autonomous vehicles is a very interesting embodiment

00:27:26.800 | of that engine that allows you to understand

00:27:29.120 | where the ego vehicle is, the ego thing is,

00:27:31.760 | where everything else is, what everything else is gonna do

00:27:34.360 | and how do you react?

00:27:35.520 | How do you actually interact with humans the right way?

00:27:38.640 | How do you build these systems?

00:27:39.820 | And I think they wanna know that.

00:27:41.900 | They wanna understand that.

00:27:43.100 | And so they keep going and doing that.

00:27:45.500 | And so on the other dimension,

00:27:47.020 | Tesla is doing something interesting.

00:27:48.380 | I mean, I think that they have a good product.

00:27:50.200 | People use it.

00:27:51.040 | I think that, you know, like it's not for me,

00:27:53.320 | but I can totally see people like it.

00:27:55.940 | And people, I think they have a good product

00:27:58.380 | outside of automation,

00:27:59.260 | but I was just referring to the automation itself.

00:28:02.300 | I mean, you know, like it kind of drives itself.

00:28:05.540 | You still have to be kind of,

00:28:07.500 | you still have to pay attention to it, right?

00:28:10.100 | But you know, people seem to use it.

00:28:12.480 | So it works for something.

00:28:14.060 | And so people, I think people are willing to pay for it.

00:28:16.480 | People are willing to buy it.

00:28:17.660 | I think it's one of the other reasons

00:28:20.560 | why people buy a Tesla car.

00:28:22.560 | Maybe one of those reasons is Elon Musk is the CEO.

00:28:25.180 | And you know, he seems like a visionary person.

00:28:27.060 | That's what people think.

00:28:27.900 | He seems like a visionary person.

00:28:29.140 | And so that adds like 5K to the value of the car.

00:28:32.100 | And then maybe another 5K is the autopilot.

00:28:34.220 | And you know, it's useful.

00:28:35.740 | I mean, it's useful in the sense

00:28:39.360 | that like people are using it.

00:28:40.860 | And so I can see Tesla, sure,

00:28:44.140 | of course they want to be visionary.

00:28:45.460 | They want to kind of put out a certain approach

00:28:47.060 | and they may actually get there.

00:28:49.360 | But I think that there's also a primary benefit

00:28:53.100 | of doing all these updates and rolling it out

00:28:55.340 | because people pay for it.

00:28:57.540 | And it's basic, you know, demand, supply, market

00:29:02.500 | and people like it.

00:29:03.580 | They're happy to pay another 5K, 10K for that novelty

00:29:07.380 | or whatever that is.

00:29:08.420 | And they use it.

00:29:10.580 | It's not like they get it and they try it a couple of times.

00:29:12.740 | It's a novelty, but they use it a lot of the time.

00:29:15.260 | And so I think that's what Tesla is doing.

00:29:17.620 | It's actually pretty different.

00:29:18.540 | Like they are on pretty orthogonal dimensions

00:29:20.480 | of what kind of things that they're building.

00:29:23.040 | They are using the same AI engine.

00:29:25.100 | So it's very possible that, you know,

00:29:26.500 | they're both going to be sort of one day

00:29:31.420 | kind of using a similar, almost like an internal

00:29:34.060 | combustion engine.

00:29:35.140 | It's a very bad metaphor,

00:29:36.380 | but similar internal combustion engine.

00:29:38.900 | And maybe one of them is building like a car.

00:29:41.180 | The other one is building a truck or something.

00:29:42.940 | So ultimately the use case is very different.

00:29:45.340 | - So you, like I said, are one of the founders

00:29:47.780 | of Optimus Ride.

00:29:48.620 | Let's take a step back.

00:29:49.540 | It's one of the success stories

00:29:52.160 | in the autonomous vehicle space.

00:29:54.140 | It's a great autonomous vehicle company.

00:29:56.420 | Let's go from the very beginning.

00:29:58.420 | What does it take to start an autonomous vehicle company?

00:30:02.300 | How do you go from idea to deploying vehicles

00:30:04.600 | like you are in a bunch of places, including New York?

00:30:08.080 | - I would say that, I think that, you know,

00:30:10.100 | what happened to us is it was the following.

00:30:12.320 | I think we've realized a lot of kind of talk

00:30:16.580 | in the autonomous vehicle industry back in like 2014, even,

00:30:20.320 | when we wanted to kind of get started.

00:30:22.320 | And I don't know, like I kind of,

00:30:27.220 | I would hear things like fully autonomous vehicles

00:30:29.660 | two years from now, three years from now.

00:30:31.260 | I kind of never bought it.

00:30:32.720 | You know, I was a part of MIT's Urban Challenge Entry.

00:30:36.880 | It kind of like, it has an interesting history.

00:30:39.960 | So I did in college and in high school,

00:30:44.100 | sort of a lot of mathematically oriented work.

00:30:46.820 | And I think I kind of, you know, at some point

00:30:50.060 | it kind of hit me.

00:30:50.940 | I wanted to build something.

00:30:52.660 | And so I came to MIT's mechanical engineering program.

00:30:55.640 | And I now realize, I think my advisor hired me

00:30:58.420 | because I could do like really good math.

00:31:00.500 | But I told him that, no, no, no,

00:31:01.620 | I want to work on that urban challenge car.

00:31:04.380 | I want to build the autonomous car.

00:31:06.580 | And I think that was kind of like a process

00:31:08.500 | where we really learned, I mean, what the challenges are

00:31:11.540 | and what kind of limitations are we up against?

00:31:14.460 | You know, like having the limitations of computers

00:31:18.040 | or understanding human behavior.

00:31:19.660 | There's so many of these things.

00:31:21.820 | And I think it just kind of didn't.

00:31:23.780 | And so we said, hey, you know, like,

00:31:26.400 | why don't we take a more like a market-based approach?

00:31:29.480 | So we focus on a certain kind of market

00:31:31.440 | and we build a system for that.

00:31:34.880 | What we're building is not so much

00:31:36.280 | of like an autonomous vehicle only, I would say.

00:31:38.920 | So we build full autonomy into the vehicles.

00:31:41.960 | But you know, the way we kind of see it is that

00:31:44.600 | we think that the approach should actually involve humans

00:31:49.160 | operating them, not just not sitting in the vehicle.

00:31:52.880 | And I think today, what we have is today,

00:31:55.700 | we have one person operate one vehicle,

00:31:58.520 | no matter what that vehicle.

00:32:00.000 | It could be a forklift, it could be a truck,

00:32:02.320 | it could be a car, whatever that is.

00:32:04.540 | And we want to go from that

00:32:06.680 | to 10 people operate 50 vehicles.

00:32:09.340 | How do we do that?

00:32:10.940 | - You're referring to a world of maybe perhaps

00:32:13.740 | teleoperation.

00:32:15.140 | So can you just say what it means for 10?

00:32:17.980 | Might be confusing for people listening.

00:32:19.660 | What does it mean for 10 people to control 50 vehicles?

00:32:23.000 | - That's a good point.

00:32:23.840 | So I think it's, I very deliberately

00:32:26.740 | didn't call it teleoperation.

00:32:28.000 | 'Cause what people think then is that people think

00:32:30.500 | away from the vehicle sits a person,

00:32:34.420 | sees like maybe puts on goggles or something,

00:32:36.820 | VR and drives the car.

00:32:38.260 | So that's not at all what we mean.

00:32:40.440 | But we mean the kind of intelligence whereby

00:32:43.300 | humans are in control, except in certain places,

00:32:47.040 | the vehicles can execute on their own.

00:32:49.380 | And so imagine like a room where people can see

00:32:53.260 | what the other vehicles are doing and everything.

00:32:56.220 | And there will be some people who are more like

00:33:00.220 | air traffic controllers, call them like AV controllers.

00:33:04.300 | And so these AV controllers would actually see

00:33:07.060 | kind of like a whole map.

00:33:08.880 | And they would understand why vehicles are really confident

00:33:12.420 | and where they kind of need a little bit more help.

00:33:16.460 | And the help shouldn't be for safety.

00:33:18.980 | Help should be for efficiency.

00:33:20.660 | Vehicles should be safe no matter what.

00:33:22.840 | If you had zero people, they could be very safe,

00:33:25.380 | but they'd be going five miles an hour.

00:33:27.660 | And so if you want them to go around 25 miles an hour,

00:33:30.200 | then you need people to come in.

00:33:32.020 | And for example, the vehicle come to an intersection

00:33:36.100 | and the vehicle can say, I can wait,

00:33:39.960 | I can inch forward a little bit,

00:33:41.540 | show my intent or I can turn left.

00:33:43.660 | And right now it's clear, I can turn, I know that,

00:33:47.700 | but before you give me the go, I won't.

00:33:50.260 | And so that's one example.

00:33:51.580 | This doesn't mean necessarily we're doing that actually.

00:33:53.900 | I think if you go down all that much detail

00:33:58.260 | that every intersection,

00:33:59.380 | you're kind of expecting a person to press a button,

00:34:02.520 | then I don't think you'll get

00:34:03.420 | the efficiency benefits you want.

00:34:04.780 | You need to be able to kind of go around

00:34:06.780 | and be able to do these things.

00:34:07.740 | But I think you need people to be able to set

00:34:10.180 | high level behavior to vehicles.

00:34:12.460 | That's the other thing with autonomous vehicles.

00:34:14.060 | I think a lot of people kind of think about it as follows.

00:34:16.140 | I mean, this happens with technology a lot.

00:34:18.260 | You think, all right, so I know about cars

00:34:21.860 | and I heard robots.

00:34:23.300 | So I think how this is gonna work out is that

00:34:26.100 | I'm gonna buy a car, press a button,

00:34:28.500 | and it's gonna drive itself.

00:34:29.740 | And when is that gonna happen?

00:34:31.660 | And people kind of tend to think about it that way.

00:34:33.580 | But when you think about what really happens

00:34:35.260 | is that something comes in

00:34:37.380 | in a way that you didn't even expect.

00:34:39.980 | If asked, you might have said, I don't think I need that,

00:34:43.060 | or I don't think it should be that and so on.

00:34:45.020 | And then that becomes the next big thing, coding code.

00:34:49.340 | And so I think that this kind of different ways

00:34:52.500 | of humans operating vehicles could be really powerful.

00:34:55.700 | I think that sooner than later,

00:34:58.260 | we might open our eyes up to a world in which

00:35:01.300 | you go around walking a mall

00:35:03.220 | and there's a bunch of security robots

00:35:04.660 | that are exactly operated in this way.

00:35:06.620 | You go into a factory or a warehouse,

00:35:08.380 | there's a whole bunch of robots

00:35:09.500 | that are operated exactly in this way.

00:35:11.500 | You go to the Brooklyn Navy Yard,

00:35:15.380 | you see a whole bunch of autonomous vehicles, Optimus ride,

00:35:18.380 | and they're operated maybe in this way.

00:35:21.020 | But I think people kind of don't see that.

00:35:22.460 | I sincerely think that there's a possibility

00:35:25.860 | that we may almost see like a whole mushrooming

00:35:28.700 | of this technology in all kinds of places

00:35:31.460 | that we didn't expect before.

00:35:33.380 | And that may be the real surprise.

00:35:35.420 | And then one day when your car actually drives itself,

00:35:39.060 | it may not be all that much of a surprise at all

00:35:41.020 | because you see it all the time, you interact with them,

00:35:43.220 | you take the Optimus ride, hopefully that's your choice.

00:35:46.740 | And then you hear a bunch of things,

00:35:50.380 | you go around, you interact with them.

00:35:52.060 | I don't know, like you have a little delivery vehicle

00:35:54.260 | that goes around the sidewalks and delivers you things

00:35:56.900 | and then you take it, it says, "Thank you."

00:35:59.340 | And then you get used to that.

00:36:01.100 | And one day your car actually drives itself

00:36:03.740 | and the regulation goes by

00:36:05.060 | and you can hit the button and sleep.

00:36:07.460 | And it wouldn't be a surprise at all.

00:36:08.700 | I think that may be the real reality.

00:36:10.740 | - So there's gonna be a bunch of applications

00:36:13.300 | that pop up around autonomous vehicles.

00:36:17.220 | Some of which, maybe many of which we don't expect at all.

00:36:20.620 | So if we look at Optimus ride, what do you think,

00:36:23.620 | the viral application,

00:36:27.100 | the one that like really works for people in mobility,

00:36:31.060 | what do you think Optimus ride will connect with

00:36:33.860 | in the near future first?

00:36:36.260 | - I think that the first places that I like to target,

00:36:39.140 | honestly, is like these places where transportation

00:36:43.140 | is required within an environment,

00:36:44.900 | like people typically call it geofence.

00:36:46.700 | So you can imagine like roughly two mile by two mile,

00:36:49.860 | could be bigger, could be smaller type of an environment.

00:36:53.220 | And there's a lot of these kinds of environments

00:36:54.780 | that are typically transportation deprived.

00:36:57.340 | The Brooklyn Navy Yard that we're in today,

00:36:59.620 | we're in a few different places,

00:37:01.140 | but that was the one that was last publicized.

00:37:05.020 | And that's a good example.

00:37:06.180 | So there's not a lot of transportation there.

00:37:08.660 | And you wouldn't expect, like, I don't know,

00:37:11.140 | I think maybe operating an Uber there

00:37:13.660 | ends up being sort of a little too expensive.

00:37:15.900 | Or when you compare it with operating Uber elsewhere,

00:37:19.820 | that becomes the elsewhere becomes the priority.

00:37:22.180 | And these people,

00:37:23.020 | those places become totally transportation deprived.

00:37:26.180 | And then what happens is that,

00:37:27.340 | people drive into these places

00:37:29.060 | and to go from point A to point B inside this place,

00:37:32.660 | within that day, they use their cars.

00:37:35.380 | And so we end up building more parking

00:37:38.180 | for them to, for example,

00:37:39.260 | take their cars and go to a lunch place.

00:37:41.300 | And I think that one of the things that can be done

00:37:45.580 | is that you can put in efficient, safe,

00:37:49.660 | sustainable transportation systems

00:37:51.420 | into these types of places first.

00:37:53.860 | And I think that you could deliver mobility

00:37:56.500 | in an affordable way, affordable, accessible,

00:38:00.780 | sustainable way.

00:38:03.300 | But I think what also enables is that this kind of effort,

00:38:07.340 | money, area, land that we spend on parking,

00:38:10.940 | we could reclaim some of that.

00:38:12.780 | And that is on the order of like,

00:38:14.540 | even for a small environment, like two mile by two mile,

00:38:17.540 | it doesn't have to be smack in the middle of New York.

00:38:19.580 | I mean, anywhere else,

00:38:21.940 | you're talking tens of millions of dollars.

00:38:23.620 | If you're smack in the middle of New York,

00:38:25.020 | you're looking at billions of dollars of savings

00:38:26.900 | just by doing that.

00:38:28.580 | And that's the economic part of it.

00:38:29.860 | And there's a societal part, right?

00:38:31.300 | I mean, just look around.

00:38:32.420 | I mean, the places that we live are like built for cars.

00:38:37.420 | It didn't look like this just like a hundred years ago.

00:38:41.420 | Like today, no one walks in the middle of the street.

00:38:44.020 | It's for cars.

00:38:45.780 | No one tells you that growing up,

00:38:47.580 | but you grow into that reality.

00:38:49.620 | And so sometimes they close the road, it happens here.

00:38:52.220 | You know, like the celebration, they close the road,

00:38:54.420 | still people don't walk in the middle of the road,

00:38:56.260 | like just walk in the middle and people don't.

00:38:58.540 | But I think it has so much impact,

00:39:01.580 | the car in the space that we have.

00:39:03.900 | And I think we talked about sustainability, livability.

00:39:07.460 | I mean, ultimately these kinds of places

00:39:09.780 | that parking spots at the very least

00:39:11.700 | could change into something more useful

00:39:13.420 | or maybe just like park areas, recreational.

00:39:16.300 | And so I think that's the first thing that we're targeting.

00:39:19.220 | And I think that we're getting like a really good response,

00:39:21.980 | both from an economic societal point of view,

00:39:24.660 | especially places that are a little bit forward looking.

00:39:27.540 | And like, for example, Brooklyn Navy Yard,

00:39:29.700 | they have tenants, there's distinct,

00:39:32.060 | they're called like New Lab.

00:39:33.660 | It's kind of like an innovation center.

00:39:35.300 | There's a bunch of startups there.

00:39:36.460 | And so, you know, you get those kinds of people

00:39:38.620 | and you know, they're really interested in

00:39:41.140 | sort of making that environment more livable.

00:39:44.020 | And these kinds of solutions that Optimist Ride provides

00:39:47.260 | almost kind of comes in and becomes that.

00:39:50.460 | And many of these places that are transportation deprived,

00:39:53.620 | you know, they actually rent shuttles.

00:39:57.740 | And so, you know, you can ask anybody,

00:40:00.980 | the shuttle experience is like terrible.

00:40:03.140 | People hate shuttles.

00:40:04.620 | And I can tell you why.

00:40:05.780 | It's because, you know, like the driver is very expensive

00:40:09.940 | in a shuttle business.

00:40:11.020 | So what makes sense is to attach 20, 30 seats to a driver.

00:40:15.500 | And a lot of people have this misconception.

00:40:17.220 | They think that shuttles should be big.

00:40:18.860 | Sometimes we get that at Optimist Ride.

00:40:20.300 | We tell them we're going to give you like four-seaters,

00:40:22.100 | six-seaters, and we get asked like,

00:40:23.780 | "How about like 20-seaters?"

00:40:25.140 | I'm like, you know, you don't need 20-seaters.

00:40:27.340 | You want to split up those seats

00:40:29.140 | so that they can travel faster

00:40:30.860 | and the transportation delays would go down.

00:40:32.860 | That's what you want.

00:40:34.260 | If you make it big,

00:40:35.580 | not only you will get delays in transportation,

00:40:38.100 | but you won't have an agile vehicle.

00:40:40.300 | It will take a long time to speed up, slow down, and so on.

00:40:44.180 | You need to climb up to the thing.

00:40:45.780 | So it's kind of like really hard to interact with.

00:40:48.740 | - And scheduling too,

00:40:49.580 | perhaps when you have more smaller vehicles,

00:40:52.060 | it becomes closer to Uber

00:40:53.460 | where you can actually get a personal,

00:40:56.020 | I mean, just the logistics of getting the vehicle to you

00:40:59.500 | becomes easier when you have a giant shuttle.

00:41:02.780 | There's fewer of them,

00:41:04.220 | and it probably goes on a route,

00:41:06.140 | a specific route that it's supposed to hit.

00:41:08.340 | - And when you go on a specific route

00:41:10.140 | and all seats travel together

00:41:11.780 | versus, you know, you have a whole bunch of them,

00:41:14.860 | you can imagine the route you can still have,

00:41:17.420 | but you can imagine you split up the seats

00:41:19.580 | and instead of, you know, them traveling,

00:41:21.200 | like, I don't know, a mile apart,

00:41:23.880 | they could be like, you know, half a mile apart

00:41:26.460 | if you split them into two.

00:41:28.180 | That basically would mean that your delays,

00:41:31.220 | when you go out, you won't wait for them for a long time.

00:41:34.500 | And that's one of the main reasons,

00:41:35.580 | or you don't have to climb up.

00:41:37.060 | The other thing is that I think

00:41:38.140 | if you split them up in a nice way,

00:41:40.340 | and if you can actually know

00:41:42.180 | where people are going to be somehow,

00:41:44.660 | you don't even need the app.

00:41:46.020 | A lot of people ask us the app.

00:41:47.740 | We say, "Why don't you just walk into the vehicle?

00:41:50.660 | How about you just walk into the vehicle,

00:41:52.200 | it recognizes who you are,

00:41:53.540 | and it gives you a bunch of options of places that you go,

00:41:55.900 | and you just kind of go there."

00:41:57.520 | I mean, people kind of also internalize the apps.

00:42:00.780 | Everybody needs an app.

00:42:01.960 | It's like, you don't need an app,

00:42:03.100 | you just walk into the thing.

00:42:04.300 | - You just walk up.

00:42:05.460 | - But I think one of the things that, you know,

00:42:07.340 | we really try to do is to take that shuttle experience

00:42:10.700 | that no one likes and tilt it into something

00:42:13.060 | that everybody loves.

00:42:14.580 | And so I think that's another important thing.

00:42:17.560 | I would like to say that carefully,

00:42:18.840 | just like teleoperation, we don't do shuttles.

00:42:21.820 | You know, we're really kind of thinking of this as a system

00:42:24.500 | or a network that we're designing.

00:42:26.300 | But ultimately, we go to places

00:42:30.140 | that would normally rent a shuttle service

00:42:32.620 | that people wouldn't like as much,

00:42:34.220 | and we want to tilt it into something that people love.

00:42:37.340 | - So you've mentioned this actually earlier,

00:42:39.500 | but how many Optimus ride vehicles

00:42:42.420 | do you think would be needed

00:42:43.980 | for any person in Boston or New York?

00:42:47.100 | If they step outside, there will be,

00:42:49.260 | this is like a mathematical question,

00:42:52.640 | there'll be two Optimus ride vehicles within line of sight.

00:42:56.660 | Is that the right number?

00:42:57.880 | Two, well, at least one.

00:42:58.720 | - Yeah, like for example, that's the density.

00:43:01.740 | So meaning that if you see one vehicle,

00:43:04.540 | you look around, you see another one too.

00:43:06.880 | Imagine like, you know, Tesla will tell you

00:43:10.320 | they collect a lot of data.

00:43:11.740 | Do you see that with Tesla?

00:43:12.840 | Like you just walk around and you look around,

00:43:14.920 | you see Tesla?

00:43:15.800 | Probably not.

00:43:16.640 | - Very specific areas of California, maybe.

00:43:19.200 | - Maybe.

00:43:20.760 | You're right.

00:43:21.600 | Like there's a couple zip codes that, you know.

00:43:24.000 | But I think that's kind of important

00:43:25.640 | because you know, like maybe the couple zip codes.

00:43:28.100 | The one thing that we kind of depend on,

00:43:30.000 | and I'll get to your question in a second,

00:43:31.480 | but now like we're taking a lot of tangents today.

00:43:34.400 | - Hell yes.

00:43:35.240 | - And so I think that this is actually important.

00:43:38.260 | People call this data density or data velocity.

00:43:40.960 | So it's very good to collect data in a way that,

00:43:44.280 | you know, you see the same place so many times.

00:43:47.120 | Like you can drive 10,000 miles around the country,

00:43:50.720 | or you drive 10,000 miles in a confined environment.

00:43:54.240 | You'll see the same intersection hundreds of times.

00:43:56.600 | And when it comes to predicting what people are gonna do

00:43:59.000 | in that specific intersection, you become really good at it.

00:44:02.720 | Versus if you draw on like 10,000 miles around the country,

00:44:05.320 | you've seen that only once.

00:44:06.800 | And so trying to predict what people do becomes hard.

00:44:09.440 | And I think that, you know, you said what is needed.

00:44:12.720 | It's tens of thousands of vehicles.

00:44:14.360 | You know, you really need to be

00:44:15.680 | like a specific fraction of vehicle.

00:44:17.840 | Like for example, in good times in Singapore,

00:44:20.560 | you can go and you can just grab a cab.

00:44:23.280 | And they are like, you know, 10%, 20% of traffic,

00:44:26.720 | those taxis.

00:44:27.700 | Ultimately, that's where you need to get to.

00:44:31.880 | So that, you know, you get to a certain place

00:44:34.000 | where you really, the benefits really kick off

00:44:37.040 | in like orders of magnitude type of a point.

00:44:40.760 | But once you get there, you actually get the benefits.

00:44:43.480 | And you can certainly carry people.

00:44:44.880 | I think that's one of the things.

00:44:45.880 | People really don't like to wait for themselves.

00:44:50.880 | But for example, they can wait a lot more for the goods

00:44:54.520 | if they order something.

00:44:55.720 | Like if you're sitting at home

00:44:56.840 | and you wanna wait half an hour, that sounds great.

00:44:58.680 | People will say, it's great.

00:45:00.000 | You're gonna take a cab, you're waiting half an hour.

00:45:02.520 | Like that's crazy.

00:45:04.460 | You don't wanna wait that much.

00:45:06.080 | But I think, you know, you can, I think,

00:45:08.200 | really get to a point where the system,

00:45:10.640 | at peak times, really focuses on

00:45:12.600 | kind of transporting humans around.

00:45:14.320 | And then it's really, it's a good fraction of traffic

00:45:17.880 | to the point where, you know, you go, you look around

00:45:20.120 | and there's something there

00:45:20.960 | and you just kind of basically get in there.

00:45:24.160 | And it's already waiting for you or something like that.

00:45:27.160 | And then you take it.

00:45:28.480 | If you do it at that scale, like today, for instance, Uber,

00:45:33.280 | if you talk to a driver, right?

00:45:35.820 | I mean, Uber takes a certain cut.

00:45:37.280 | It's a small cut.

00:45:39.360 | Or drivers would argue that it's a large cut.

00:45:41.520 | But, you know, it's, when you look at

00:45:44.200 | the grand scheme of things,

00:45:46.000 | most of that money that you pay Uber

00:45:49.260 | kind of goes to the driver.

00:45:50.320 | And if you talk to the driver,

00:45:51.300 | the driver will claim that most of it is their time.

00:45:54.600 | You know, it's not spent on gas, they think.

00:45:57.100 | It's not spent on the car per se as much.

00:46:01.220 | It's like their time.

00:46:02.920 | And if you didn't have a person driving,

00:46:05.440 | or if you're in a scenario where, you know,

00:46:07.080 | like 0.1 person is driving the car,

00:46:10.640 | a fraction of a person is kind of operating the car,

00:46:14.280 | because, you know, one operates several.

00:46:16.280 | If you're in that situation,

00:46:19.040 | you realize that the internal combustion engine

00:46:21.080 | type of cars are very inefficient.

00:46:23.200 | You know, we build them to go on highways,

00:46:25.100 | they pass crash tests, they're like really heavy.

00:46:27.800 | They really don't need to be like 25 times

00:46:30.080 | the weight of its passengers, or, you know,

00:46:32.620 | like area-wise and so on.

00:46:35.880 | But if you get through those inefficiencies,

00:46:38.080 | and if you really build like urban cars

00:46:39.880 | and things like that,

00:46:40.720 | I think the economics really starts to check out,

00:46:43.320 | like to the point where, I mean, I don't know,

00:46:45.840 | you may be able to get into a car,

00:46:47.160 | and it may be less than a dollar to go from A to B.

00:46:50.300 | As long as you don't change your destination,

00:46:52.600 | you just pay 99 cents and go there.

00:46:55.700 | If you share it, if you take another stop somewhere,

00:46:58.080 | it becomes a lot better.

00:46:59.560 | You know, these kinds of things, at least for models,

00:47:03.560 | at least for mathematics and theory,

00:47:05.100 | they start to really check out.

00:47:07.360 | - So I think it's really exciting

00:47:09.080 | what Optimus Ride is doing in terms of,

00:47:11.400 | it feels the most reachable,

00:47:12.960 | like it'll actually be here and have an impact.

00:47:15.840 | - Yeah, that is the idea.

00:47:17.280 | - And if we contrast that, again,

00:47:20.000 | we'll go back to our old friends, Waymo and Tesla.

00:47:23.600 | So Waymo seems to have sort of technically similar approaches

00:47:32.440 | as Optimus Ride, but a different,

00:47:36.140 | they're not as interested as having impact today.

00:47:39.300 | They have a longer-term sort of investment.

00:47:43.980 | It's almost more of a research project still,

00:47:47.460 | meaning they're trying to solve, as far as I understand,

00:47:49.980 | maybe you can differentiate,

00:47:52.740 | but they seem to want to do more unrestricted movement,

00:47:57.740 | meaning move from A to B, where A to B is all over the place,

00:48:01.460 | versus Optimus Ride is really nicely geo-fenced

00:48:04.740 | and really sort of establish mobility

00:48:08.380 | in a particular environment before you expand it.

00:48:11.460 | And then Tesla is like the complete opposite,

00:48:14.220 | which is the entirety of the world, actually,

00:48:18.980 | is going to be automated.

00:48:21.100 | Highway driving, urban driving, every kind of driving,

00:48:24.720 | you kind of creep up to it by incrementally improving

00:48:30.100 | the capabilities of the autopilot system.

00:48:33.260 | So when you contrast all of these,

00:48:35.940 | and on top of that, let me throw a question

00:48:37.820 | that nobody likes, but is timeline.

00:48:41.820 | When do you think each of these approaches,

00:48:44.580 | loosely speaking, nobody can predict the future,

00:48:47.580 | will see mass deployment?

00:48:49.820 | So Musk predicts the craziest approach is,

00:48:54.140 | I've heard figures like at the end of this year, right?

00:48:58.580 | So that's probably wildly inaccurate,

00:49:03.580 | but how wildly inaccurate is it?

00:49:06.040 | - I mean, first thing to lay out, like everybody else,

00:49:09.100 | it's really hard to guess.

00:49:11.660 | I mean, I don't know where Tesla can look at,

00:49:15.980 | or Elon Musk can look at and say,

00:49:18.260 | hey, it's the end of this year.

00:49:19.860 | I mean, I don't know what you can look at.

00:49:21.960 | You know, even the data that, I mean,

00:49:25.380 | if you look at the data, even kind of trying

00:49:29.740 | to extrapolate the end state without knowing

00:49:32.700 | what exactly is gonna go, especially for like

00:49:34.620 | a machine learning approach.

00:49:35.700 | I mean, it's just kind of very hard to predict,

00:49:38.460 | but I do think the following does happen.

00:49:41.560 | I think a lot of people, you know what they do

00:49:44.220 | is that there's something that I called a couple times

00:49:47.420 | time dilation in technology prediction happens.

00:49:50.940 | Let me try to describe a little bit.

00:49:53.100 | There's a lot of things that are so far ahead,

00:49:55.940 | people think they're close.

00:49:57.700 | And there's a lot of things that are actually close,

00:49:59.900 | people think it's far ahead.

00:50:01.780 | People try to kind of look at a whole landscape

00:50:04.660 | of technology development.

00:50:06.380 | Admittedly, it's chaos.

00:50:08.260 | Anything can happen in any order at any time.

00:50:10.660 | And there's a whole bunch of things in there.

00:50:12.040 | People take it, clamp it,

00:50:14.240 | and put it into the next three years.

00:50:16.620 | And so then what happens is that there's some things

00:50:19.980 | that maybe can happen by the end of the year

00:50:21.820 | or next year and so on.

00:50:23.460 | And they push that into like few years ahead

00:50:25.460 | because it's just hard to explain.

00:50:27.900 | And there are things that are like,

00:50:29.300 | we're looking at 20 years more maybe,

00:50:31.560 | hopefully in my lifetime type of things.

00:50:35.300 | And 'cause we don't know.

00:50:37.660 | I mean, we don't know how hard it is even.

00:50:40.560 | Like that's a problem.

00:50:41.500 | We don't know like if some of these problems

00:50:43.380 | are actually AI complete.

00:50:44.700 | Like we have no idea what's going on.

00:50:47.660 | And we take all of that and then we clamp it

00:50:51.700 | and then we say three years from now.

00:50:54.180 | And then some of us are more optimistic.

00:50:57.100 | So they're shooting at the end of the year.

00:50:59.340 | And some of us are more realistic.

00:51:00.820 | They say like five years, but we all,

00:51:03.060 | I think it's just hard to know.

00:51:05.700 | And I think trying to predict like products ahead,

00:51:10.700 | two, three years, it's hard to know in the following sense.

00:51:13.980 | You know, like we typically say,

00:51:15.700 | okay, this is a technology company,

00:51:17.460 | but sometimes really you're trying to build something

00:51:20.580 | where the technology does,

00:51:21.620 | like there's a technology gap.

00:51:23.220 | And Tesla had that with electric vehicles.

00:51:27.740 | You know, like when they first started,

00:51:30.500 | they would look at a chart,

00:51:31.820 | much like a Moore's law type of chart.

00:51:33.460 | And they would just kind of extrapolate that out.

00:51:35.220 | And they'd say, we want to be here.

00:51:37.220 | What's the technology to get that?

00:51:38.860 | We don't know.

00:51:39.700 | It goes like this.

00:51:40.540 | So it's probably just going to keep going.

00:51:43.000 | With AI that goes into the cars, we don't even have that.

00:51:47.500 | Like we can't, I mean, what can you quantify?

00:51:50.460 | Like what kind of chart are you looking at?

00:51:52.660 | You know?

00:51:53.480 | But so I think when there's that technology gap,

00:51:56.780 | it's just kind of really hard to predict.

00:51:58.340 | So now I realize I talked like five minutes

00:52:01.020 | and I avoided your question.

00:52:01.900 | I didn't tell you anything about that.

00:52:04.380 | It was very skillfully done.

00:52:06.020 | - That was very well done.

00:52:07.180 | And I don't think you,

00:52:08.180 | I think you've actually argued that it's not a use,

00:52:10.100 | even any answer you provide now is not that useful.

00:52:12.580 | - It's going to be very hard.

00:52:13.860 | There's one thing that I really believe in

00:52:15.780 | and you know, this is not my idea

00:52:18.100 | and it's been discussed several times,

00:52:19.900 | but this kind of like something like a startup

00:52:24.420 | or a kind of an innovative company,

00:52:28.420 | including definitely Waymo, Tesla,

00:52:31.260 | maybe even some of the other big companies

00:52:33.100 | that are kind of trying things.

00:52:34.760 | This kind of like iterated learning is very important.

00:52:38.440 | The fact that we're over there

00:52:39.820 | and we're trying things and so on,

00:52:41.540 | I think that's important.

00:52:44.300 | We try to understand.

00:52:45.300 | And I think that, you know,

00:52:47.380 | the code in code Silicon Valley has done that

00:52:49.420 | with business models pretty well.

00:52:52.180 | And now I think we're trying to get to do it

00:52:54.380 | where there's a literal technology gap.

00:52:57.020 | I mean, before, like, you know, you're trying to build,

00:52:59.360 | I'm not trying to, you know,

00:53:00.780 | I think these companies are building great technology

00:53:03.220 | to, for example, enable internet search,

00:53:06.460 | to do it so quickly.

00:53:07.580 | And that kind of didn't, wasn't there so much,

00:53:10.760 | but at least like it was a kind of a technology

00:53:12.660 | that you could predict to some degree and so on.

00:53:14.580 | And now we're just kind of trying to build,

00:53:16.700 | you know, things that it's kind of hard to quantify

00:53:18.900 | what kind of a metric are we looking at.

00:53:21.660 | - So psychologically as a sort of,

00:53:24.740 | as a leader of graduate students and at Optimus Ride,

00:53:28.820 | a bunch of brilliant engineers, just curiosity,

00:53:33.060 | psychologically, do you think it's good to think that,

00:53:37.640 | you know, whatever technology gap we're talking about

00:53:41.100 | can be closed by the end of the year?

00:53:43.680 | Or do you, you know, 'cause we don't know.

00:53:46.180 | So the way, do you want to say

00:53:49.980 | that everything is going to improve exponentially

00:53:53.940 | to yourself and to others around you as a leader?

00:53:57.340 | Or do you want to be more sort of maybe not cynical,

00:54:01.400 | but I don't want to use realistic

00:54:03.660 | 'cause it's hard to predict, but yeah,

00:54:05.860 | maybe more cynical, pessimistic

00:54:08.240 | about the ability to close that gap?

00:54:11.280 | - Yeah, I think that, you know, going back,

00:54:13.220 | I think that iterated learning is like key,

00:54:15.940 | that, you know, you're out there,

00:54:17.060 | you're running experiments to learn.

00:54:19.300 | And that doesn't mean sort of like, you know,

00:54:21.420 | like you're Optimus Ride, you're kind of doing something,

00:54:23.500 | but like in an environment,

00:54:26.040 | but like what Tesla is doing,

00:54:27.420 | I think is also kind of like this kind of notion.

00:54:30.260 | And, you know, people can go around and say like,

00:54:32.740 | you know, this year, next year, the other year and so on.

00:54:35.340 | But I think that the nice thing about it

00:54:38.140 | is that they're out there,

00:54:38.980 | they're pushing this technology in.

00:54:41.060 | I think what they should do more of,

00:54:43.080 | I think that kind of inform the people

00:54:45.380 | about what kind of technology that they're providing,

00:54:47.300 | you know, the good and the bad,

00:54:48.540 | and not just sort of, you know, if it works very well.

00:54:52.180 | But I think, and I'm not saying

00:54:53.620 | they're not doing bad on informing.

00:54:55.500 | I think they're kind of trying,

00:54:57.060 | they put up certain things,

00:54:58.500 | or at the very least, YouTube videos comes out

00:55:00.980 | on how the summon function works every now and then,

00:55:03.780 | and, you know, people get informed.

00:55:05.540 | And so that kind of cycle continues,

00:55:07.400 | but, you know, I admire it.

00:55:10.200 | I think they're kind of go out there

00:55:11.580 | and they do great things.

00:55:13.060 | They do their own kind of experiment.

00:55:14.640 | I think we do our own.

00:55:16.460 | And I think we're closing some similar technology gaps,

00:55:20.020 | but some also, some are orthogonal as well.

00:55:22.460 | You know, I think like we talked about, you know,

00:55:24.700 | people being remote, like it's something,

00:55:26.900 | or in the kind of environments that we're in,

00:55:28.520 | or, you know, think about a Tesla car,

00:55:30.740 | maybe you can enable it one day,

00:55:32.640 | like there's, you know, low traffic,

00:55:34.500 | like you're kind of, the stop and go motion,

00:55:36.460 | you just hit the button, and you can really,

00:55:38.860 | or maybe there's another, you know, lane

00:55:40.620 | that you can pass into, you go in that.

00:55:42.340 | I think they can enable these kinds of,

00:55:44.100 | I believe it.

00:55:44.940 | And so I think that that part,

00:55:48.340 | that is really important, and that is really key.

00:55:51.220 | And beyond that, I think, you know,

00:55:54.440 | when is it exactly gonna happen and so on?

00:55:57.360 | I mean, it's, like I said, it's very hard to predict.

00:56:01.080 | And I would imagine that it would be good to do

00:56:06.640 | some sort of like a one or two year plan,

00:56:08.680 | when it's a little bit more predictable,

00:56:10.820 | that, you know, the technology gaps you close,

00:56:13.020 | and the kind of sort of product that would ensue.

00:56:17.980 | So I know that from Optimus Ride,

00:56:20.620 | or, you know, other companies that I get involved in,

00:56:22.900 | I mean, at some point, you find yourself in a situation

00:56:27.100 | where you're trying to build a product,

00:56:28.700 | and people are investing in that, you know, building effort.

00:56:33.500 | And those investors, they do wanna know,

00:56:37.500 | as they compare the investments they wanna make,

00:56:39.900 | they do wanna know what happens

00:56:41.100 | in the next one or two years.

00:56:42.220 | And I think that's good to communicate that.

00:56:44.660 | But I think beyond that, it becomes a vision

00:56:47.300 | that we wanna get to someday,

00:56:48.740 | and saying five years, 10 years,

00:56:50.620 | I don't think it means anything.

00:56:52.340 | - But iterative learning is key, though,

00:56:54.420 | to do and learn.

00:56:56.180 | - I think that is key.

00:56:57.340 | - You know, I gotta sort of throw back right at you,

00:57:00.060 | criticism, in terms of, you know, like Tesla,

00:57:04.380 | or somebody communicating, you know,

00:57:06.220 | how someone works and so on.

00:57:07.860 | I got a chance to visit Optimus Ride,

00:57:10.220 | and you guys are doing some awesome stuff,

00:57:12.460 | and yet the internet doesn't know about it.

00:57:15.020 | So you should also communicate more,

00:57:17.060 | showing off, you know, showing off some of the awesome stuff,

00:57:20.500 | the stuff that works and stuff that doesn't work.

00:57:22.900 | I mean, it's just, the stuff I saw

00:57:25.180 | with the tracking of different objects and pedestrians,

00:57:27.260 | so I mean, incredible stuff going on there.

00:57:29.700 | It's just, maybe it's just the nerd in me,

00:57:31.500 | but I think the world would love to see that kind of stuff.

00:57:34.780 | - Yeah, that's well taken.

00:57:36.180 | I think, you know, I should say that it's not like,

00:57:39.380 | you know, we weren't able to,

00:57:41.580 | I think we made a decision at some point.

00:57:43.980 | That decision did involve me quite a bit

00:57:46.900 | on kind of sort of doing this

00:57:50.460 | in kind of coding code stealth mode for a bit.

00:57:52.860 | But I think that, you know,

00:57:54.620 | we'll open it up quite a lot more.

00:57:56.660 | And I think that we are also at Optimus Ride

00:57:59.740 | kind of hitting a new era.

00:58:02.380 | You know, we're big now,

00:58:04.980 | we're doing a lot of interesting things.

00:58:06.780 | And I think, you know, some of the deployments

00:58:08.900 | that we kind of announced were some of the first bits

00:58:12.820 | of information that we kind of put out into the world.

00:58:16.140 | We'll also put out our technology.

00:58:17.980 | A lot of the things that we've been developing

00:58:19.780 | is really amazing.

00:58:20.740 | And then, you know, we're gonna start putting that out.

00:58:24.940 | We're especially interested in sort of like

00:58:26.980 | being able to work with the best people.

00:58:28.540 | And I think it's good to not just kind of show them

00:58:32.420 | when they come to our office for an interview,

00:58:34.020 | but just put it out there in terms of like, you know,

00:58:36.340 | get people excited about what we're doing.

00:58:39.180 | - So on the autonomous vehicle space,

00:58:40.900 | let me ask one last question.

00:58:43.660 | So Elon Musk famously said that LIDAR is a crutch.

00:58:47.340 | So I've talked to a bunch of people about it,

00:58:50.380 | gotta ask you.

00:58:51.780 | You use that crutch quite a bit in the DARPA days.

00:58:55.140 | So, you know, and his idea in general,

00:59:00.140 | sort of, you know, more provocative and fun, I think,

00:59:02.780 | than a technical discussion.

00:59:04.420 | But the idea is that camera-based,

00:59:07.540 | primarily camera-based systems is going to be

00:59:11.620 | what defines the future of autonomous vehicles.

00:59:14.060 | So what do you think of this idea?

00:59:15.800 | LIDAR is a crutch versus primarily camera-based systems?

00:59:20.800 | - First things first.

00:59:22.020 | I think, you know, I'm a big believer

00:59:24.340 | in just camera-based autonomous vehicle systems.

00:59:27.920 | Like I think that, you know, you can put in

00:59:30.380 | a lot of autonomy and you can do great things.

00:59:33.380 | And it's very possible that at the timescales,

00:59:36.740 | like we said, we can't predict 20 years from now,

00:59:40.700 | like you may be able to do things that we're doing today

00:59:44.100 | only with LIDAR and then you may be able

00:59:45.740 | to do them just with cameras.

00:59:47.780 | And I think that, you know, you can just,

00:59:51.080 | I think that I will put my name on it too.

00:59:54.480 | Like, you know, there will be a time

00:59:56.660 | when you can only use cameras and you'll be fine.

00:59:59.840 | At that time though, it's very possible that, you know,

01:00:03.980 | you find the LIDAR system as another robustifier

01:00:08.580 | or it's so affordable that it's stupid not to, you know,

01:00:13.020 | just kind of put it there.

01:00:14.280 | And I think we may be looking at a future like that.

01:00:19.940 | - Do you think we're over-relying on LIDAR right now

01:00:23.500 | because we understand the better, it's more reliable

01:00:26.260 | in many ways in terms from a safety perspective?

01:00:28.460 | - It's easier to build with, that's the other thing.

01:00:31.220 | I think to be very frank with you,

01:00:33.620 | I mean, you know, we've seen a lot of sort of

01:00:36.820 | autonomous vehicles companies come and go

01:00:39.620 | and the approach has been, you know,

01:00:40.980 | you slap a LIDAR on a car and it's kind of easy to build

01:00:44.740 | with and you have a LIDAR, you know,

01:00:46.820 | just kind of coat it up and you hit the button

01:00:49.460 | and you do a demo.

01:00:50.740 | So I think there's admittedly, there's a lot of people

01:00:54.300 | they focus on the LIDAR 'cause it's easier to build with.

01:00:57.860 | That doesn't mean that, you know, without the camera,

01:01:00.380 | just cameras, you cannot do what they're doing,

01:01:03.460 | but it's just kind of a lot harder.

01:01:05.060 | And so you need to have certain kind of expertise

01:01:06.820 | to exploit that.

01:01:08.540 | What we believe in and you know,

01:01:09.860 | you've maybe seen some of it is that

01:01:11.780 | we believe in computer vision.

01:01:14.220 | We certainly work on computer vision and OptumSprite

01:01:17.260 | by a lot, like, and we've been doing that from day one.

01:01:21.380 | And we also believe in sensor fusion.

01:01:23.060 | So, you know, we have a relatively minimal use of LIDARs,

01:01:27.180 | but we do use them.

01:01:29.300 | And I think, you know, in the future,

01:01:30.780 | I really believe that the following sequence

01:01:33.100 | of events may happen.

01:01:34.300 | First things first, number one,

01:01:37.340 | there may be a future in which, you know,

01:01:39.140 | there's like cars with LIDARs and everything

01:01:41.300 | and the cameras, but you know,

01:01:43.140 | this in this 50 year ahead future,

01:01:45.220 | they can just drive with cameras as well,

01:01:47.540 | especially in some isolated environments and cameras,

01:01:50.100 | they go and they do the thing.

01:01:51.940 | In the same future, it's very possible that, you know,

01:01:54.980 | the LIDARs are so cheap and frankly make the software

01:01:57.980 | maybe a little less compute intensive at the very least,

01:02:02.540 | or maybe less complicated so that they can be certified

01:02:05.820 | or ensure their safety and things like that,

01:02:09.020 | that it's kind of stupid not to put the LIDAR.

01:02:11.980 | Like, imagine this, you either pay money for the LIDAR

01:02:16.180 | or you pay money for the compute.

01:02:18.220 | And if you don't put the LIDAR,

01:02:20.220 | it's a more expensive system

01:02:21.500 | because you have to put in a lot of compute.

01:02:23.700 | Like this is another possibility.

01:02:26.180 | I do think that a lot of the sort of initial deployments

01:02:29.620 | of self-driving vehicles, I think they will enroll LIDARs

01:02:33.020 | and especially either low range or short,

01:02:36.620 | either short range or low resolution LIDARs

01:02:39.180 | are actually not that hard to build in solid state.

01:02:42.900 | They're still scanning,

01:02:44.220 | but like MEMS type of scanning LIDARs and things like that,

01:02:46.900 | they're like, they're actually not that hard.

01:02:48.620 | I think they will, maybe kind of playing with the spectrum

01:02:51.460 | and the phase arrays, they're a little bit harder,

01:02:53.260 | but I think like putting a MEMS mirror in there

01:02:57.580 | that kind of scans the environment, it's not hard.

01:03:00.340 | The only thing is that, you know, you,

01:03:02.620 | just like with a lot of the things that we do nowadays

01:03:04.660 | in developing technology,

01:03:05.820 | you hit fundamental limits of the universe.

01:03:08.580 | The speed of light becomes a problem

01:03:10.820 | in when you're trying to scan the environment.

01:03:12.580 | So you don't get either good resolution

01:03:14.260 | or you don't get range,

01:03:15.500 | but you know, it's still,

01:03:18.180 | it's something that you can put in there affordably.

01:03:21.340 | - So let me jump back to drones.

01:03:24.180 | You have a role in the Lockheed Martin

01:03:27.660 | Alpha Pilot Innovation Challenge,

01:03:29.840 | where teams compete in drone racing.

01:03:33.820 | It's super cool, super intense,

01:03:36.100 | interesting application of AI.

01:03:38.620 | So can you tell me about the very basics of the challenge

01:03:42.340 | and where you fit in,

01:03:43.820 | what your thoughts are on this problem?

01:03:45.940 | And it's sort of echoes of the early DARPA challenge

01:03:49.860 | in the, through the desert that we're seeing now,

01:03:52.340 | now with drone racing.

01:03:54.340 | - Yeah, I mean, one interesting thing about it is that,

01:03:56.860 | you know, people, drone racing exists as an e-sport.

01:04:01.180 | And so it's much like you're playing a game,

01:04:03.300 | but there's a real drone going in an environment.

01:04:06.140 | - A human being is controlling it with goggles on.

01:04:08.780 | So there's no, it is a robot, but there's no AI.

01:04:13.240 | - There's no AI, yeah.

01:04:14.380 | Human being is controlling it.

01:04:15.700 | And so that's already there.

01:04:17.700 | And I've been interested in this problem

01:04:19.660 | for quite a while, actually,

01:04:21.900 | from a roboticist point of view.

01:04:23.460 | And that's what's happening in Alphapilot.

01:04:25.060 | - Which problem, of aggressive flight?

01:04:27.340 | - Of aggressive flight, fully autonomous, aggressive flight.

01:04:31.500 | The problem that I'm interested in,

01:04:32.820 | I mean, you asked about Alphapilot,

01:04:33.940 | and I'll get there in a second,

01:04:35.220 | but the problem that I'm interested in,

01:04:36.980 | I'd love to build autonomous vehicles like drones

01:04:41.020 | that can go far faster than any human possibly can.

01:04:45.260 | I think we should recognize that we as humans have,

01:04:48.140 | you know, limitations in how fast

01:04:50.300 | we can process information.

01:04:52.420 | And those are some biological limitations.

01:04:54.980 | Like we think about this AI this way too.

01:04:56.860 | I mean, this has been discussed a lot,

01:04:58.700 | and this is not sort of my idea per se,

01:05:00.780 | but a lot of people kind of think about human level AI.

01:05:03.940 | And they think that, you know, AI is not human level.

01:05:06.180 | One day it'll be human level,

01:05:07.500 | and humans and AIs, they kind of interact.

01:05:09.600 | Versus I think that the situation really is that

01:05:12.620 | humans are at a certain place,

01:05:14.700 | and AI keeps improving,

01:05:16.100 | and at some point just crosses off,

01:05:17.820 | and you know, it gets smarter and smarter and smarter.

01:05:21.100 | And so drone racing, the same issue.

01:05:23.500 | Humans play this game,

01:05:26.060 | and you know, you have to like react in milliseconds.

01:05:29.260 | And there's really, you know,

01:05:30.780 | you see something with your eyes,

01:05:32.940 | and then that information just flows through your brain,

01:05:35.220 | into your hands so that you can command it.

01:05:37.900 | And there's some also delays on, you know,

01:05:39.300 | getting information back and forth.

01:05:40.620 | But suppose those delays didn't exist.

01:05:41.940 | You just, just a delay between your eye and your fingers

01:05:46.660 | is a delay that a robot doesn't have to have.

01:05:50.700 | So we end up building in my research group,

01:05:54.660 | like systems that, you know,

01:05:56.660 | see things at a kilohertz,

01:05:58.340 | like a human eye would barely hit a hundred hertz.

01:06:00.900 | So imagine things that see stuff in slow motion,

01:06:05.440 | like 10X slow motion.

01:06:07.720 | It will be very useful.

01:06:08.660 | Like we talked a lot about autonomous cars,

01:06:10.260 | so, you know, we don't get to see it,

01:06:13.420 | but a hundred lives are lost every day,

01:06:16.780 | just in the United States on traffic accidents.

01:06:19.460 | And many of them are like known cases, you know,

01:06:22.020 | like the, you're coming through like a ramp,

01:06:25.460 | going into a highway, you hit somebody and you're off,

01:06:28.820 | or, you know, like you kind of get confused.

01:06:30.940 | You try to like swerve into the next lane,

01:06:32.900 | you go off the road and you crash, whatever.

01:06:35.820 | And I think if you had enough compute in a car

01:06:38.780 | and a very fast camera, right at the time of an accident,

01:06:43.140 | you could use all compute you have,

01:06:44.860 | like you could shut down the infotainment system

01:06:47.260 | and use that kind of computing resources,

01:06:50.580 | instead of rendering,

01:06:51.620 | you use it for the kind of artificial intelligence

01:06:54.420 | that goes in there, the autonomy.

01:06:56.340 | And you can either take control of the car

01:06:59.060 | and bring it to a full stop,

01:07:00.140 | but even if you can't do that,

01:07:01.900 | you can deliver what the human is trying to do.

01:07:04.260 | Human is trying to change the lane,

01:07:06.280 | but goes off the road,

01:07:07.900 | not being able to do that with motor skills and the eyes,

01:07:10.740 | and you know, you can get in there.

01:07:12.620 | And I was, there's so many other things

01:07:14.260 | that you can enable

01:07:15.100 | with what I would call high throughput computing.

01:07:17.340 | You know, data is coming in extremely fast,

01:07:21.280 | and in real time, you have to process it.

01:07:23.900 | And the current CPUs,

01:07:26.740 | however fast you clock it, are typically not enough.

01:07:30.700 | You need to build those computers from the ground up

01:07:32.820 | so that they can ingest all that data.

01:07:35.100 | That I'm really interested in.

01:07:36.540 | - Just on that point, just really quick,

01:07:38.720 | is the, currently what's the bottom?

01:07:41.260 | Like you mentioned the delays in humans.

01:07:44.020 | Is it the hardware?

01:07:45.620 | So you work a lot with Nvidia hardware.

01:07:47.660 | Is it the hardware or is it the software?

01:07:50.180 | - I think it's both.

01:07:51.540 | I think it's both.

01:07:52.380 | In fact, they need to be co-developed,

01:07:54.100 | I think, in the future.

01:07:55.020 | I mean, that's a little bit what Nvidia does.

01:07:57.180 | Sort of like they almost like build the hardware,

01:07:59.660 | and then they build the neural networks,

01:08:01.220 | and then they build the hardware back,

01:08:02.460 | and the neural networks back,

01:08:03.520 | and it goes back and forth, but it's that co-design.

01:08:06.420 | And I think that, you know, like,

01:08:08.460 | we tried to, way back,

01:08:09.820 | we tried to build a fast drone

01:08:11.620 | that could use a camera image to like track what's moving

01:08:14.900 | in order to find where it is in the world.

01:08:17.340 | This typical sort of, you know,

01:08:18.900 | visual inertial state estimation problems

01:08:20.820 | that we would solve.

01:08:22.200 | And, you know, we just kind of realized

01:08:23.540 | that we're at the limit sometimes of, you know,

01:08:25.820 | doing simple tasks.

01:08:26.780 | We're at the limit of the camera frame rate.

01:08:29.500 | Because, you know, if you really want to track things,

01:08:31.640 | you want the camera image to be 90% kind of like,

01:08:35.100 | or somewhat the same from one frame to the next.

01:08:38.020 | - That's right.

01:08:38.940 | - And why are we at the limit of the camera frame rate?

01:08:41.940 | It's because camera captures data.

01:08:44.660 | It puts it into some serial connection.

01:08:46.980 | It could be USB,

01:08:48.660 | or like there's something called camera serial interface

01:08:50.980 | that we use a lot.

01:08:52.460 | It puts into some serial connection,

01:08:54.860 | and copper wires can only transmit so much data.

01:08:58.260 | And you hit the Shannon limit on copper wires.

01:09:01.140 | And, you know, you hit yet another kind of universal limit

01:09:05.580 | that you can transfer the data.

01:09:06.900 | So you have to be much more intelligent

01:09:09.740 | on how you capture those pixels.

01:09:11.260 | You can take compute and put it right next to the pixels.

01:09:14.640 | People are building those-

01:09:16.940 | - How hard is it to do?

01:09:17.780 | How hard is it to get past the bottleneck

01:09:21.860 | of the copper wire?

01:09:23.660 | - Yeah, you need to do a lot of parallel processing,

01:09:26.180 | as you can imagine.

01:09:27.060 | The same thing happens in the GPUs.

01:09:28.660 | You know, like the data is transferred in parallel somehow.

01:09:31.660 | It gets into some parallel processing.

01:09:34.100 | I think that, you know, like,

01:09:35.900 | now we're really kind of diverted off

01:09:37.460 | into so many different dimensions, but-

01:09:39.100 | - Great, so it's aggressive flight.

01:09:40.540 | How do we make drones see many more frames a second,

01:09:45.100 | you know, to enable aggressive flight?

01:09:46.820 | That's a super interesting problem.

01:09:48.100 | - That's an interesting problem.

01:09:49.140 | So, but like, think about it.

01:09:50.220 | You have CPUs.

01:09:52.820 | You clock them at, you know, several gigahertz.

01:09:55.460 | We don't clock them faster,

01:09:58.940 | largely because, you know,

01:09:59.860 | we run into some heating issues and things like that.

01:10:01.660 | But another thing is that three gigahertz clock,

01:10:04.940 | light travels kind of like on the order of a few inches

01:10:08.620 | or an inch.

01:10:09.820 | That's the size of a chip.

01:10:11.540 | And so you pass a clock cycle,

01:10:14.420 | and as the clock signal is going around in the chip,

01:10:17.580 | you pass another one.

01:10:19.180 | And so trying to coordinate that,

01:10:21.380 | the design of the complexity of the chip becomes so hard.

01:10:23.820 | I mean, we have hit the fundamental limits of the universe

01:10:27.700 | in so many things that we're designing.

01:10:29.420 | I don't know if people realize that.

01:10:30.620 | It's great, but like, we can't make transistors smaller

01:10:33.620 | because like quantum effects,

01:10:34.940 | electrons start to tunnel around.

01:10:36.740 | We can't clock it faster.

01:10:38.260 | One of the reasons why is because like,

01:10:41.700 | information doesn't travel faster in the universe.

01:10:44.900 | And we're limited by that.

01:10:45.980 | Same thing with the laser scanner.

01:10:47.900 | But so then it becomes clear that, you know,

01:10:52.140 | the way you organize the chip into a CPU or even a GPU,

01:10:56.660 | you now need to look at how to redesign that

01:10:59.500 | if you're gonna stick with silicon.

01:11:01.820 | You could go do other things too.

01:11:02.980 | I mean, there's that too,

01:11:03.820 | but you really almost need to take those transistors,

01:11:06.580 | put them in a different way

01:11:07.620 | so that the information travels on those transistors

01:11:10.820 | in a different way,

01:11:11.980 | in a much more way that is specific

01:11:15.060 | to the high speed cameras coming in.

01:11:16.860 | And so that's one of the things

01:11:18.500 | that we talk about quite a bit.

01:11:20.540 | - So drone racing kind of really makes that-

01:11:23.980 | - Embodies that.

01:11:24.820 | - It embodies that.

01:11:25.820 | And that's why it's really exciting.

01:11:26.660 | - And it's exciting.

01:11:27.500 | It's exciting for people, you know, students like it.

01:11:30.100 | It embodies all those problems.

01:11:32.020 | But going back, we're building,

01:11:34.220 | code and code another engine.

01:11:36.100 | And that engine, I hope one day will be

01:11:39.300 | just like how impactful seatbelts were in driving.

01:11:43.940 | I hope so.

01:11:45.420 | Or it could enable, you know,

01:11:46.900 | next generation autonomous air taxis and things like that.

01:11:49.580 | I mean, it sounds crazy,

01:11:51.060 | but one day we may need to purge these things.

01:11:53.740 | If you really wanna go from Boston to New York

01:11:56.620 | in one and a half hours,

01:11:57.980 | you may wanna fix big aircraft.

01:11:59.860 | Most of these companies that are kind of doing

01:12:01.940 | code and code flying cars, they're focusing on that.

01:12:04.060 | But then how do you land it on top of a building?

01:12:06.540 | You may need to pull off like kind of

01:12:08.340 | fast maneuvers for a robot,

01:12:10.100 | like perch land, it's just gonna go perch into a building.

01:12:13.980 | If you wanna do that, like you need these kinds of systems.

01:12:16.980 | And so drone racing, you know,

01:12:20.260 | it's being able to go way faster

01:12:23.220 | than any human can comprehend.

01:12:24.820 | Take an aircraft, forget the quadcopter,

01:12:28.380 | you take a fixed wing.

01:12:29.780 | While you're at it,

01:12:30.620 | you might as well put some like rocket engines in the back

01:12:32.700 | and just light it.

01:12:33.980 | You go through the gate and a human looks at it

01:12:35.940 | and just said, "What just happened?"

01:12:38.100 | And they would say, "It's impossible for me to do that."

01:12:41.380 | And that's closing the same technology gap

01:12:44.060 | that would, you know, one day steer cars out of accidents.

01:12:48.860 | - So, but then let's get back to the practical,

01:12:51.780 | which is sort of just getting the thing

01:12:55.260 | to work in a race environment,

01:12:57.500 | which is kind of what the,

01:12:59.060 | it's another kind of exciting thing,

01:13:01.340 | which the DARPA Challenge for the desert did.

01:13:03.500 | You know, theoretically we had autonomous vehicles,

01:13:05.980 | but making them successfully finish a race,

01:13:09.540 | first of all, which nobody finished the first year.

01:13:12.300 | And then the second year just to get, you know,

01:13:15.060 | to finish and go at a reasonable time

01:13:17.460 | is really difficult engineering,

01:13:19.460 | practically speaking challenge.

01:13:21.100 | So that, let me ask about the Alpha Pilot Challenge.

01:13:25.940 | There's a, I guess, a big prize

01:13:27.860 | potentially associated with it.

01:13:29.540 | But let me ask, reminiscent of the DARPA days,

01:13:32.980 | predictions, you think anybody will finish?

01:13:35.780 | (laughing)

01:13:37.740 | - Well, not soon.

01:13:39.820 | I think that depends on how you set up the race course.

01:13:42.380 | And so if the race course is a slalom course,

01:13:44.300 | I think people will kind of do it.

01:13:46.220 | But can you set up some course, like literally some course,

01:13:50.540 | you get to design it, as the algorithm developer,

01:13:54.100 | can you set up some course

01:13:55.700 | so that you can beat the best human?

01:13:57.540 | When is that gonna happen?

01:14:00.500 | Like, that's not very easy.

01:14:01.820 | Even just setting up some course.

01:14:03.700 | If you let the human that you're competing with

01:14:05.940 | set up the course, it becomes a lot harder.

01:14:08.820 | - So how many in the space of all possible courses

01:14:13.860 | are, would humans win and would machines win?

01:14:18.700 | - Great question.

01:14:19.540 | Let's get to that.

01:14:20.460 | I wanna answer your other question,

01:14:21.860 | which is like, the DARPA challenge days, right?

01:14:24.580 | What was really hard?

01:14:25.660 | I think we understand, we understood what we wanted to build

01:14:29.820 | but still building things, that experimentation,

01:14:32.180 | that iterated learning,

01:14:33.180 | that takes up a lot of time actually.

01:14:35.980 | And so in my group, for example,

01:14:38.500 | in order for us to be able to develop fast,

01:14:41.380 | we build like VR environments.

01:14:43.100 | We'll take an aircraft,

01:14:44.580 | we'll put it in a motion capture room,

01:14:46.420 | big, huge motion capture room,

01:14:48.780 | and we'll fly it in real time,

01:14:51.180 | we'll render other images and beam it back to the drone.

01:14:54.300 | That sounds kind of notionally simple,

01:14:57.340 | but it's actually hard

01:14:58.180 | because now you're trying to fit all that data

01:15:00.500 | through the air into the drone.

01:15:02.620 | And so you need to do a few crazy things to make that happen

01:15:05.660 | but once you do that, then at least you can try things.

01:15:09.260 | If you crash into something, you didn't actually crash.

01:15:12.220 | So it's like the whole drone is in VR.

01:15:14.060 | We can do augmented reality and so on.

01:15:16.060 | And so I think at some point,

01:15:18.900 | testing becomes very important.

01:15:20.540 | One of the nice things about AlphaPilot

01:15:22.060 | is that they built the drone

01:15:24.380 | and they build a lot of drones and it's okay to crash.

01:15:28.220 | In fact, I think maybe the viewers

01:15:31.540 | may kind of like to see things that crash.

01:15:34.540 | - That potentially could be the most exciting part.

01:15:36.860 | - It could be the exciting part.

01:15:38.180 | And I think as an engineer,

01:15:40.020 | it's a very different situation to be in.

01:15:42.580 | Like in academia, a lot of my colleagues

01:15:45.100 | who are actually in this race

01:15:46.220 | and they're really great researchers,

01:15:47.860 | but I've seen them trying to do similar things

01:15:50.660 | whereby they built this one drone

01:15:52.220 | and somebody with like a face mask

01:15:54.980 | and a glows are going right behind the drone,

01:15:58.380 | trying to hold it if it falls down.

01:16:00.580 | Imagine you don't have to do that.

01:16:02.500 | I think that's one of the nice things

01:16:03.580 | about AlphaPilot Challenge where we have these drones

01:16:06.860 | and we're going to design the courses

01:16:09.140 | in a way that will keep pushing people

01:16:11.420 | up until the crashes start to happen.

01:16:13.260 | And we'll hopefully sort of,

01:16:16.940 | I don't think you want to tell people crashing is okay.

01:16:19.660 | Like we want to be careful here,

01:16:20.980 | but because we don't want people to crash a lot,

01:16:23.180 | but certainly we want them to push it

01:16:25.460 | so that everybody crashes once or twice

01:16:28.740 | and they're really pushing it to their limits.

01:16:32.140 | - That's where iterated learning comes in.

01:16:34.220 | 'Cause every crash is a lesson.

01:16:36.220 | - Is a lesson, exactly.

01:16:37.540 | - So in terms of the space of possible courses,

01:16:40.060 | how do you think about it?

01:16:41.340 | In the war of human versus machines,

01:16:46.580 | where do machines win?

01:16:47.780 | - We look at that quite a bit.

01:16:48.940 | I mean, I think that you will see quickly

01:16:50.820 | that you can design a course

01:16:54.060 | and in certain courses, like in the middle somewhere,

01:16:57.940 | if you kind of run through the course once,

01:17:02.660 | the machine gets beaten pretty much consistently by slightly.

01:17:07.660 | But if you go through the course like 10 times,

01:17:10.140 | humans get beaten very slightly, but consistently.

01:17:13.180 | So humans, at some point, you get confused,

01:17:15.580 | you get tired and things like that versus this machine

01:17:18.660 | is just executing the same line of code,

01:17:21.260 | tirelessly just going back to the beginning

01:17:23.820 | and doing the same thing exactly.

01:17:25.540 | I think that kind of thing happens.

01:17:28.740 | And I realized sort of as humans,

01:17:30.740 | there's the classical things that everybody has realized.

01:17:35.860 | If you put in some sort of like strategic thinking,

01:17:39.460 | that's a little bit harder for machines

01:17:41.060 | that I think sort of comprehend.

01:17:44.620 | Precision is easy to do, so that's what they excel in.

01:17:48.620 | And also sort of repeatability is easy to do,

01:17:52.980 | that's what they excel in.

01:17:55.020 | You can build machines that excel in strategy as well

01:17:57.780 | and beat humans that way too,

01:17:59.140 | but that's a lot harder to build.

01:18:00.780 | - I have a million more questions,

01:18:03.020 | but in the interest of time, last question.

01:18:05.380 | - Yeah.

01:18:06.500 | - What is the most beautiful idea

01:18:08.140 | you've come across in robotics?

01:18:10.100 | Whether a simple equation, experiment,

01:18:12.260 | a demo, simulation, piece of software,

01:18:14.940 | what just gives you pause?

01:18:17.700 | - That's an interesting question.

01:18:21.020 | I have done a lot of work myself in decision-making,

01:18:25.180 | so I've been interested in that area.

01:18:26.700 | So in robotics, you have,

01:18:28.740 | somehow the field has split into,

01:18:31.540 | like there's people who would work on like perception,

01:18:34.380 | how robots perceive the environment,

01:18:36.700 | then how do you actually make like decisions?

01:18:38.820 | And there's people also like how do you interact,

01:18:40.580 | people interact with robots.

01:18:41.580 | There's a whole bunch of different fields.

01:18:43.660 | And I have admittedly worked a lot on the more control

01:18:48.260 | and decision-making than the others.

01:18:50.900 | And I think that the one equation

01:18:54.780 | that has always kind of baffled me is Bellman's equation.

01:18:58.980 | And so it's this person who have realized like way back,

01:19:03.980 | more than half a century ago,

01:19:06.500 | on like how do you actually sit down

01:19:10.620 | and if you have several variables

01:19:12.300 | that you're kind of jointly trying to determine,

01:19:15.540 | how do you determine that?

01:19:17.220 | And there's one beautiful equation that,

01:19:20.820 | like today people do reinforcement learning,

01:19:22.500 | we still use it.

01:19:23.900 | And it's baffling to me because it both kind of

01:19:28.900 | tells you the simplicity,

01:19:30.780 | 'cause it's a single equation that anyone can write down.

01:19:33.900 | You can teach it in the first course on decision-making.

01:19:37.340 | At the same time, it tells you how computation,

01:19:39.780 | we have hard the problem is.

01:19:41.500 | I feel like a lot of the things that I've done at MIT

01:19:44.260 | for research has been kind of just this fight

01:19:46.580 | against computational efficiency things.

01:19:48.740 | Like how can we get it faster to the point

01:19:50.820 | where we now got to like, let's just redesign this chip.

01:19:54.700 | Like maybe that's the way.

01:19:56.820 | But I think it talks about how computationally hard

01:20:01.540 | certain problems can be by nowadays

01:20:04.900 | what people call curse of dimensionality.

01:20:07.620 | And so as the number of variables kind of grow,

01:20:11.100 | the number of decisions you can make grows rapidly.

01:20:15.940 | Like if you have 100 variables,

01:20:19.100 | each one of them take 10 values,

01:20:21.300 | all possible assignments is more than the number of atoms

01:20:23.980 | in the universe, it's just crazy.

01:20:25.980 | And that kind of thinking is just embodied

01:20:28.420 | in that one equation that I really like.

01:20:31.300 | - And the beautiful balance between it being

01:20:33.500 | theoretically optimal and somehow, practically speaking,

01:20:38.500 | given the curse of dimensionality,

01:20:41.220 | nevertheless in practice works,

01:20:43.500 | despite all those challenges, which is quite incredible.

01:20:48.020 | - Which is quite incredible.

01:20:49.100 | So I would say that it's kind of like quite baffling,

01:20:52.540 | actually, in a lot of fields that we think about

01:20:55.900 | how little we know.

01:20:57.140 | And so I think here too, we know that in the worst case,

01:21:02.820 | things are pretty hard,

01:21:04.540 | but in practice, generally things work.

01:21:07.300 | So it's just kind of baffling in decision-making

01:21:10.780 | how little we know.

01:21:12.660 | Just like how little we know about the beginning of time,

01:21:15.220 | how little we know about our own future.

01:21:18.600 | Like if you actually go into like from Balman's equation

01:21:22.260 | all the way down, I mean, there's also how little we know

01:21:24.900 | about like mathematics.

01:21:26.180 | I mean, we don't even know if the axioms are like consistent

01:21:28.780 | it's just crazy.

01:21:29.620 | - Yeah, I think a good lesson there

01:21:32.580 | just like as you said, we tend to focus on the worst case

01:21:35.780 | or the boundaries of everything we're studying.

01:21:38.340 | And then the average case seems to somehow work out.

01:21:41.340 | If you think about life in general,

01:21:43.400 | we mess it up a bunch,

01:21:45.220 | we freak out about a bunch of the traumatic stuff,

01:21:47.620 | but in the end it seems to work out okay.

01:21:50.100 | - Yeah, it seems like a good metaphor.

01:21:51.940 | (laughs)

01:21:52.980 | - So Tasha, thank you so much for being a friend,

01:21:56.060 | a colleague, a mentor.

01:21:57.300 | I really appreciate it.

01:21:58.140 | It's an honor to talk to you.

01:21:59.040 | - Likewise, thank you, Lex.

01:22:01.680 | - Thanks for listening to this conversation

01:22:03.260 | with Sertash Karaman

01:22:04.540 | and thank you to our presenting sponsor Cash App.

01:22:07.300 | Please consider supporting the podcast

01:22:08.980 | by downloading Cash App and using code LEXPODCAST.

01:22:12.940 | If you enjoy this podcast, subscribe on YouTube,

01:22:15.580 | review it with Five Stars and Apple Podcast,

01:22:17.820 | support it on Patreon,

01:22:19.140 | or simply connect with me on Twitter @LexFriedman.

01:22:23.140 | And now let me leave you with some words from Hal 9000

01:22:27.260 | from the movie "2001, A Space Odyssey."

01:22:31.100 | "I'm putting myself to the fullest possible use,

01:22:34.220 | which is all I think that any conscious entity

01:22:36.980 | can ever hope to do."

01:22:38.960 | Thank you for listening and hope to see you next time.

01:22:41.780 | (upbeat music)

01:22:44.360 | (upbeat music)

01:22:46.940 | [BLANK_AUDIO]

Sertac Karaman: Robots That Fly and Robots That Drive | Lex Fridman Podcast #97

Chapters