MIT Human-Centered Autonomous Vehicle

00:00:00.000 | This is the Human-Centered Autonomous Vehicle.

00:00:02.880 | One of the main ideas underlying our work

00:00:05.240 | is that solving the task of autonomous driving

00:00:08.560 | is more complicated and more fascinating

00:00:11.640 | than the strictly robotics challenges of localization,

00:00:16.160 | mapping, perception, control, and planning.

00:00:19.700 | You also have to enable the vehicle to perceive,

00:00:23.100 | predict, communicate, and collaborate with human beings.

00:00:27.020 | The humans inside the car, like the driver

00:00:29.520 | and the passengers, and the humans outside the car,

00:00:32.400 | like the pedestrians, cyclists,

00:00:34.960 | the drivers of other vehicles, and even teleoperators.

00:00:38.560 | The studies, the code, the data, and the demos we release

00:00:42.980 | all consider autonomous driving

00:00:44.520 | in this kind of human-centered way,

00:00:46.920 | where the control is transferred from human to machine

00:00:51.000 | and back to human based on the state

00:00:54.560 | of the external driving environment

00:00:56.700 | and the state of the driver.

00:00:58.360 | What we'd like to demonstrate today is the basics,

00:01:01.300 | voice-based transfer of control from human to machine

00:01:05.160 | based on whether the driver is paying attention

00:01:07.960 | to the road or not.

00:01:08.960 | Inside, we have two cameras on the driver,

00:01:12.300 | one on the driver's face, one on the driver's body.

00:01:14.920 | We have two cameras looking at the external roadway,

00:01:17.180 | and we have a few other cameras for filming purposes.

00:01:20.660 | There's a center stack display

00:01:22.440 | showing who's in control of the vehicle, human or machine.

00:01:25.520 | So currently, the human is in control of the vehicle.

00:01:29.080 | Let's drive.

00:01:30.040 | Split the car and drive.

00:01:32.080 | On the center stack display, it shows the gear as drive.

00:01:34.960 | The perception, control, and driver state sensing algorithms

00:01:42.400 | you see today are running in real time,

00:01:44.900 | but the visualizations you're seeing in video

00:01:47.720 | are done in offline post-processing.

00:01:52.880 | Our perception system today is vision-based

00:01:56.000 | using two neural networks.

00:01:58.560 | One is doing road segmentation,

00:02:00.680 | the other is doing object detection of vehicles,

00:02:04.000 | cyclists, pedestrians, traffic signs, traffic lights.

00:02:07.620 | The acceleration, braking, and steering of the car

00:02:12.000 | is performed by PID controllers.

00:02:14.020 | The driver state sensing that we're showing today

00:02:20.480 | is glance region classification,

00:02:23.840 | and that's performed using 3D convolutional neural networks.

00:02:28.120 | High-level planning decisions to transfer control

00:02:32.240 | or to stop the vehicle are performed

00:02:33.960 | by a decision fusion algorithm

00:02:36.200 | that combines risk factors in the external environment

00:02:40.120 | and driver state,

00:02:41.720 | whether the driver's paying attention to the road or not.

00:02:44.560 | Safety for us is the number one priority, always.

00:02:49.600 | We are on a test track.

00:02:51.760 | The vehicles and pedestrians here today

00:02:55.120 | are all part of our team, all part of the demonstration.

00:02:58.240 | There's another safety driver in the car

00:03:00.680 | that can stop the vehicle at any moment

00:03:03.120 | by pressing a single button.

00:03:05.360 | Okay, let's engage in a distracting activity, Twitter,

00:03:14.480 | and let's send a tweet.

00:03:17.740 | (silence)

00:03:19.900 | I'm typing this tweet

00:03:34.340 | while driving in the MIT city center.

00:03:42.940 | In the MIT semi-autonomous vehicle

00:03:47.940 | on a test track.

00:04:01.340 | (silence)

00:04:03.500 | A test track.

00:04:13.380 | Lex, you appear distracted.

00:04:18.500 | Would you like me to take over?

00:04:20.740 | Yes, please.

00:04:21.780 | Great.

00:04:24.580 | I am taking control of steering and braking.

00:04:27.140 | (silence)

00:04:29.300 | The car is now in control

00:04:37.220 | as the center stack display shows.

00:04:40.140 | So I will continue with the tweet.

00:04:43.020 | The car knows that I'm not paying attention

00:04:48.020 | and has taken control after asking me nicely for it.

00:05:16.220 | (silence)

00:05:18.380 | Video out tomorrow.

00:05:31.380 | Okay, here goes nothing.

00:05:36.140 | It's posted.

00:05:39.180 | Very well might be the first tweet ever sent

00:05:45.140 | from an autonomous vehicle while it's driving itself.

00:05:47.780 | (silence)

00:05:52.860 | (car engine roaring)

00:06:20.100 | Elevated driving risk detected.

00:06:22.860 | I am stopping for a pedestrian.

00:06:24.860 | Lex, pedestrian is blocking our lane of travel.

00:06:40.780 | Should I honk?

00:06:41.740 | No, please shift gear to park.

00:06:45.100 | Great.

00:06:47.660 | We are now in park.

00:06:49.540 | (silence)

00:06:51.700 | That was a demo of the basics.

00:06:56.180 | Perception, motion planning, driver state sensing,

00:06:59.500 | transfer control and tweeting.

00:07:02.220 | And we have a lot more to explore together.

00:07:04.580 | Our team is working on various aspects

00:07:07.300 | of human centered artificial intelligence

00:07:10.340 | toward our mission to save lives

00:07:12.860 | through effective human robot collaboration.

00:07:18.340 | (silence)

00:07:20.500 | (silence)

00:07:22.660 | (silence)

00:07:24.820 | (silence)

00:07:26.980 | (silence)

00:07:29.140 | (silence)

00:07:31.300 | (silence)

00:07:33.460 | (silence)

00:07:35.620 | [BLANK_AUDIO]

MIT Human-Centered Autonomous Vehicle

Chapters