So the way you leapfrog, right, is you come up with an idea or you take a direction, perhaps secretly, that the other people aren't taking. And so the Cruz, Waymo, even Aurora. - I don't know Aurora, Zooks is the same stack as well. They're all the same code base even.

They're all the same DARPA Urban Challenge code base. - So the question is, do you think there's a room for brilliance and innovation there that will change everything? Like say, okay, so I'll give you examples. It could be if revolution and mapping, for example, that allow you to map things, do HD maps of the whole world, all weather conditions somehow really well, or revolution and simulation to where all the way you said before becomes incorrect, that kind of thing.

Any room for breakthrough innovation? - What I said before about, oh, they actually get the whole thing. I'll say this about, we divide driving into three problems and I actually haven't solved the third yet, but I have an idea how to do it. So there's the static. The static driving problem is assuming you are the only car on the road.

And this problem can be solved 100% with mapping and localization. This is why farms work the way they do. If all you have to deal with is the static problem and you can statically schedule your machines, it's the same as like statically scheduling processes. You can statically schedule your tractors to never hit each other on their paths 'cause you know the speed they go at.

So that's the static driving problem. Maps only helps you with the static driving problem. - Yeah, the question about static driving. You've just made it sound like it's really easy. - Static driving is really easy. - How easy? 'Cause the whole drifting out of lane, when Tesla drifts out of lane, it's failing on the fundamental static driving problem.

- Tesla is drifting out of lane? The static driving problem is not easy for the world. The static driving problem is easy for one route. - One route and one weather condition with one state of lane markings and like no deterioration, no cracks in the road. - I'm assuming you have a perfect localizer.

So that's solved for the weather condition and the lane marking condition. - But that's the problem is how do you have a perfect-- - You can build, perfect localizers are not that hard to build. - Okay, come on now. With LIDAR? - With LIDAR, yeah. - Oh, with LIDAR, okay.

- With LIDAR, yeah, but you use LIDAR, right? Like use LIDAR to build a perfect localizer. Building a perfect localizer without LIDAR, (sighs) it's gonna be hard. You can get 10 centimeters without LIDAR, you can get one centimeter with LIDAR. - I'm not even concerned about the one or 10 centimeters.

I'm concerned if every once in a while you're just way off. - Yeah, so this is why you have to carefully make sure you're always tracking your position. You wanna use LIDAR camera fusion, but you can get the reliability of that system up to 100,000 miles and then you write some fallback condition where it's not that bad if you're way off, right?

I think that you can get it to the point, it's like as will be, that you're never in a case where you're way off and you don't know it. - Yeah, okay, so this is brilliant. So that's the static. - Static. - We can, especially with LIDAR and good HD maps, you can solve that problem.

- Easy. - No, I just disagree with your word easy. - The static problem's so easy. - Very typical for you to say something's easy. I got it. It's not as challenging as the other ones, okay. - Well, it's, okay, maybe it's obvious how to solve it. The third one's the hardest.

So where do we get, and a lot of people don't even think about the third one and even see it as different from the second one. So the second one is dynamic. The second one is like, say there's an obvious examples, like a car stopped at a red light, right?

You can't have that car in your map because you don't know whether that car is gonna be there or not. So you have to detect that car in real time and then you have to do the appropriate action, right? Also, that car is not a fixed object. That car may move and you have to predict what that car will do, right?

So this is the dynamic problem. - Yeah. - So you have to deal with this. This involves, again, like you're gonna need models of other people's behavior. - Do you, are you including in that, I don't wanna step on the third one. - Oh. - But are you including in that your influence on people?

- Ah, that's the third one. - Okay. - That's the third one. We call it the counterfactual. - Yeah, brilliant. - And that. - I just talked to Judea Pearl who's obsessed with counterfactuals. - The counterfactual, oh yeah, yeah, I read his books. - So the static and the dynamic.

- Yeah. - Our approach right now for lateral will scale completely to the static and dynamic. The counterfactual, the only way I have to do it yet, the thing that I wanna do once we have all of these cars is I wanna do reinforcement learning on the world. I'm always gonna turn the exploiter up to max.

I'm not gonna have them explore. But the only real way to get at the counterfactual is to do reinforcement learning because the other agents are humans. (whooshing) (upbeat music) (whooshing) (whooshing) (whooshing) (whooshing) (whooshing)