As we design and build autonomous vehicle systems here at MIT, as we study real-world natural realistic driving, we begin to understand that it may be one, two, three, four decades before we're able to build autonomous vehicles that can be fully autonomous without integrating the human being. Before then, we have to integrate the human being, whether as a driver, as a safety driver, or a teleoperator.

For that, at the very beginning, at the very least, the autonomous vehicle needs to be able to perceive the state of the driver. As we look at the recent case, the tragic case of the pedestrian fatality in Arizona, we can see that the perception of what the driver is doing, whether they're looking on the road or not, is of critical importance for this environment.

So we'd like to show to you the GLANCE region classification algorithm running on the video of the driver's face. And also, in the near future, we're going to make the code open source, available to everybody, together with an archive submission, in hopes that companies and universities testing autonomous vehicles can integrate it into their testing procedures and make sure they're doing everything they can to make the testing process as safe as possible.

Let's take a look. To the left is the original video. In the middle is the detection of the face region that is then fed to the GLANCE classification algorithm as a sequence of images to the neural network. And then the neural network produces an output, a prediction of where the driver is looking.

That's shown to the right, the current GLANCE region that's predicted, whether it's road, left, right, rear view mirror, center stack, instrument cluster. Then, off-road GLANCE duration, whenever the driver is looking off-road, that number increases. We run this GLANCE classification algorithm on this particular video to show that it is a powerful signal for an autonomous vehicle to have, especially in the case when a safety driver is tasked with monitoring the safe operation of the autonomous vehicle.