Yann LeCun: Was HAL 9000 Good or Evil? - Space Odyssey 2001 | AI Podcast Clips
00:00:00.000 | [Music]
00:00:07.760 | You said that 2001 Space Odyssey is one of your favorite movies.
00:00:12.320 | Hal 9000 decides to get rid of the astronauts for people haven't seen the
00:00:17.440 | movie, spoiler alert, because he, it, she
00:00:22.640 | believes that the astronauts, they will interfere with the mission.
00:00:27.600 | Do you see Hal as flawed in some fundamental way or even
00:00:31.280 | evil or did he do the right thing?
00:00:34.560 | Neither, there's no notion of evil in that in that context
00:00:39.280 | other than the fact that people die but it was an example of what people call
00:00:44.160 | value misalignment, right, you give an objective to a machine
00:00:48.080 | and the machine strives to achieve this objective
00:00:51.680 | and if you don't put any constraints on this objective like don't kill people
00:00:55.200 | and don't do things like this, the
00:00:59.840 | machine given the power will do stupid things just to achieve this
00:01:03.520 | objective or damaging things to achieve this objective. It's a little bit like,
00:01:07.120 | I mean, we are used to this in the context of human society.
00:01:11.760 | We put in place laws to prevent people from doing bad
00:01:17.600 | things because spontaneously they would do those bad things,
00:01:20.560 | right, so we have to shape their cost function, the objective function if you
00:01:25.360 | want through laws to kind of correct and education obviously to sort of
00:01:29.760 | correct for those. So maybe just
00:01:34.080 | pushing a little further on that point, Hal, you know, there's a mission,
00:01:40.400 | there's a, this fuzziness around the ambiguity around what the actual
00:01:44.640 | mission is but, you know, do you think that there
00:01:49.120 | will be a time from a utilitarian perspective
00:01:52.800 | where an AI system, where it is not misalignment, where it is
00:01:56.160 | alignment for the greater good of society, that an AI system will make
00:02:00.080 | decisions that are difficult? Well, that's the trick. I mean, eventually
00:02:04.080 | we'll have to figure out how to do this and again,
00:02:07.600 | we're not starting from scratch because we've been doing this with humans for
00:02:10.960 | millennia. So designing objective functions for people
00:02:15.200 | is something that we know how to do and we don't do it by
00:02:18.480 | programming things, although the legal code is called code.
00:02:24.800 | So that tells you something and it's actually the design of an objective
00:02:28.800 | function that's really what legal code is, right, it tells you
00:02:31.520 | here's what you can do, here's what you can't do, if you do it you pay that much,
00:02:35.040 | that's an objective function. So there is this idea somehow that
00:02:40.000 | it's a new thing for people to try to design objective functions that are
00:02:42.800 | aligned with the common good but no, we've been writing laws for millennia
00:02:45.920 | and that's exactly what it is. So that's where the science of
00:02:52.480 | lawmaking and computer science will come together.
00:02:58.880 | So there's nothing special about HAL or AI systems,
00:03:02.880 | it's just the continuation of tools used to make some of these difficult ethical
00:03:07.280 | judgments that laws make. Yeah, and we have systems like this
00:03:10.800 | already that make many decisions for
00:03:15.040 | ourselves in society that need to be designed in a way that they
00:03:18.640 | like rules about things that sometimes have bad side effects
00:03:23.440 | and we have to be flexible enough about those rules so that they can be broken
00:03:26.720 | when it's obvious that they shouldn't be applied.
00:03:30.080 | So you don't see this on the camera here but all the decoration in this room is
00:03:33.200 | all pictures from 2001 A Space Odyssey.
00:03:37.440 | Wow, is that by accident or is there a lot? It's not by accident, it's by design.
00:03:43.440 | Oh wow, so if you were to build
00:03:47.440 | HAL 10,000, so an improvement of HAL 9,000,
00:03:51.600 | what would you improve? Well, first of all, I wouldn't
00:03:54.800 | ask it to hold secrets and tell lies because that's really what breaks it in
00:03:59.600 | the end, that's the fact that it's asking itself questions about the
00:04:03.840 | purpose of the mission and it's, you know, pieces things together
00:04:06.880 | that it's heard, you know, all the secrecy of the preparation of the mission and
00:04:10.400 | the fact that it was the discovery on the lunar surface that really was kept
00:04:14.480 | secret and one part of HAL's memory knows this
00:04:18.400 | and the other part does not know it and is supposed to not tell
00:04:22.080 | anyone and that creates internal conflict. So you think there never
00:04:25.680 | should be a set of things that an AI system
00:04:30.000 | should not be allowed, like a set of facts that should not be
00:04:35.600 | shared with the human operators? Well, I think, no, I think that it
00:04:40.480 | should be a bit like in the design of autonomous AI systems,
00:04:48.000 | there should be the equivalent of, you know, the
00:04:51.200 | oath that Hippocrates, that doctors
00:04:57.520 | signed up to, right? So there's certain things, certain rules that
00:05:00.880 | you have to abide by and we can sort of hardwire this into
00:05:03.920 | our machines to kind of make sure they don't go.
00:05:07.040 | So I'm not, you know, an advocate of the three laws of robotics, you know,
00:05:11.600 | the Asimov kind of thing because I don't think it's practical but
00:05:16.080 | you know, some level of limits. But to be clear, this is not,
00:05:22.960 | these are not questions that are kind of really worth
00:05:27.280 | asking today because we just don't have the technology to do this. We don't
00:05:30.960 | have autonomous intelligent machines, we have intelligent machines, some are
00:05:33.760 | intelligent machines that are very specialized
00:05:36.800 | but they don't really sort of satisfy an objective, they're just,
00:05:39.760 | you know, kind of trained to do one thing. So until we have some idea for a design
00:05:45.840 | of a full-fledged autonomous intelligent system, asking the question of how we
00:05:50.640 | design this objective, I think is a little too abstract.
00:05:54.400 | It's a little too abstract, there's useful elements to it
00:05:57.440 | in that it helps us understand our own ethical codes, humans.
00:06:04.000 | So even just as a thought experiment, if you imagine
00:06:07.360 | that an AGI system is here today, how would we program it as a kind of
00:06:12.720 | nice thought experiment of constructing how should we
00:06:16.720 | have a law, have a system of laws for us humans.
00:06:20.400 | It's just a nice practical tool. And I think there's echoes of that idea too
00:06:25.840 | in the AI systems we have today that don't have to be that intelligent.
00:06:30.000 | Like autonomous vehicles, these things start creeping in
00:06:33.840 | that are worth thinking about but certainly they shouldn't be framed as
00:06:37.040 | as hell.
00:06:41.920 | you
00:06:44.000 | you
00:06:46.640 | you
00:06:48.720 | you
00:06:51.760 | you
00:06:53.840 | you
00:06:55.920 | [BLANK_AUDIO]