Uber Self-Driving Car Glance Classification
00:00:00.000 | As we design and build autonomous vehicle systems here at MIT,
00:00:03.600 | as we study real-world natural realistic driving,
00:00:06.600 | we begin to understand that it may be one, two, three, four decades
00:00:10.700 | before we're able to build autonomous vehicles
00:00:13.800 | that can be fully autonomous without integrating the human being.
00:00:17.500 | Before then, we have to integrate the human being,
00:00:20.400 | whether as a driver, as a safety driver, or a teleoperator.
00:00:24.600 | For that, at the very beginning, at the very least,
00:00:27.500 | the autonomous vehicle needs to be able to perceive the state of the driver.
00:00:30.800 | As we look at the recent case, the tragic case of the pedestrian fatality in Arizona,
00:00:36.000 | we can see that the perception of what the driver is doing,
00:00:39.800 | whether they're looking on the road or not,
00:00:41.500 | is of critical importance for this environment.
00:00:43.700 | So we'd like to show to you the GLANCE region classification algorithm
00:00:47.900 | running on the video of the driver's face.
00:00:50.300 | And also, in the near future, we're going to make the code open source,
00:00:54.400 | available to everybody, together with an archive submission,
00:00:57.800 | in hopes that companies and universities testing autonomous vehicles
00:01:02.400 | can integrate it into their testing procedures
00:01:04.800 | and make sure they're doing everything they can
00:01:07.100 | to make the testing process as safe as possible.
00:01:10.000 | Let's take a look.
00:01:11.400 | To the left is the original video.
00:01:13.100 | In the middle is the detection of the face region
00:01:15.700 | that is then fed to the GLANCE classification algorithm
00:01:18.200 | as a sequence of images to the neural network.
00:01:20.700 | And then the neural network produces an output,
00:01:22.700 | a prediction of where the driver is looking.
00:01:25.000 | That's shown to the right, the current GLANCE region that's predicted,
00:01:28.800 | whether it's road, left, right, rear view mirror, center stack, instrument cluster.
00:01:33.800 | Then, off-road GLANCE duration, whenever the driver is looking off-road,
00:01:38.500 | that number increases.
00:01:39.800 | We run this GLANCE classification algorithm on this particular video
00:01:43.400 | to show that it is a powerful signal for an autonomous vehicle to have,
00:01:47.800 | especially in the case when a safety driver is tasked with
00:01:51.100 | monitoring the safe operation of the autonomous vehicle.
00:01:55.400 | [ Silence ]