Garry Kasparov: IBM Deep Blue, AlphaZero, and the Limits of AI in Open Systems

00:00:00.000 | You're lost to IBM Deep Blue in 1997.

00:00:05.640 | In my eyes, that is one of the most seminal moments in the history.

00:00:09.040 | Again, I apologize for being romanticizing the notion, but in the history of our civilization,

00:00:15.480 | because humans as a civilization for centuries saw chess as the peak of what man can accomplish,

00:00:24.000 | of intellectual mastery.

00:00:26.840 | And that moment when a machine could beat a human being was inspiring to just an entire

00:00:34.840 | anyone who cares about science, innovation, an entire generation of AI researchers.

00:00:41.080 | And yet, to you that loss, at least if reading your face, seemed like a tragedy, extremely

00:00:47.760 | painful, like you said, physically painful.

00:00:50.400 | Why?

00:00:51.400 | When you look back at your psychology of that loss, why was it so painful?

00:00:56.040 | Were you not able to see the seminal nature of that moment?

00:01:02.040 | Or was that exactly why it was that painful?

00:01:05.960 | As I already said, losing was painful, physically painful.

00:01:12.040 | And the match I lost in 1997 was not the first match I lost to a machine.

00:01:16.280 | It was the first match I lost, period.

00:01:19.400 | That's...

00:01:20.400 | Oh, wow.

00:01:21.400 | Oh, wow.

00:01:22.400 | Yeah, it's...

00:01:26.400 | Right.

00:01:28.400 | That makes all the difference to me.

00:01:31.480 | First time I lost, it's just...

00:01:33.200 | Now I lost, and the reason I was so angry that I just, I had suspicions that my loss

00:01:41.760 | was not just the result of my bad play.

00:01:44.480 | So though I played quite poorly, just when you started looking at the games today, I

00:01:48.120 | made tons of mistakes.

00:01:49.760 | But I had all reasons to believe that there were other factors that had nothing to do

00:01:55.680 | with the game of chess.

00:01:56.680 | And that's why I was angry.

00:01:57.720 | But look, it was 22 years ago.

00:02:00.200 | It's more than the bridge.

00:02:02.000 | We can analyze this match and this is with everything you said.

00:02:05.320 | I agree with probably one exception, is that considering chess as the sort of, as a pinnacle

00:02:13.120 | of intellectual activities was our mistake.

00:02:16.320 | Because we just thought, "Oh, it's a game of the highest intellect and it's just you

00:02:21.080 | have to be so intelligent and you could see things that the ordinary mortals could not

00:02:29.320 | see."

00:02:31.040 | It's a game.

00:02:32.280 | And all machines had to do in this game is just to make fewer mistakes, not to solve

00:02:38.000 | the game.

00:02:39.120 | Because the game cannot be solved.

00:02:40.120 | I mean, according to Koval Shanin, the number of legal moves is 10 to the 46th power.

00:02:44.800 | Too many zeros, just for any computer to finish the job in next few billion years.

00:02:53.360 | But it doesn't have to.

00:02:55.000 | It's all about making fewer mistakes.

00:02:57.480 | And I think that's this match actually.

00:02:59.680 | And what's happened afterwards with other games, with Go, with Shogi, with video games,

00:03:07.480 | it's a demonstration that the machines will always beat humans in what I call closed systems.

00:03:14.200 | The moment you build a closed system, no matter how the system is called, chess, Go, Shogi,

00:03:22.520 | Dota, machines will prevail simply because they will bring down number of mistakes.

00:03:30.920 | Machines don't have to solve it.

00:03:32.320 | They just have to...

00:03:34.040 | The way they outplay us, it's not by just being more intelligent.

00:03:38.240 | It's just by doing something else, but eventually it's capitalizing on our mistakes.

00:03:44.600 | When you look at the chess machines ratings today, and compare this to Magnus Carlsen,

00:03:50.000 | is the same as comparing Ferrari to Usain Bolt.

00:03:54.960 | The gap is, I mean, by chess standards is insane.

00:03:58.800 | 34, 3,500 to 2,800, 2,850 on Magnus.

00:04:03.520 | It's like difference between Magnus and an ordinary player from an open international

00:04:08.040 | tournament.

00:04:10.800 | It's not because machine understanding is better than Magnus Carlsen, but simply because

00:04:15.040 | it's steady.

00:04:16.840 | Machine has steady hand.

00:04:18.760 | And I think that is what we have to learn from 1997 experience and from further encounters

00:04:27.220 | with computers and sort of the current state of affairs was AlphaZero, you were beating

00:04:33.400 | other machines.

00:04:34.860 | The idea that we can compete with computers in so-called intellectual fields, it was wrong

00:04:42.620 | from the very beginning.

00:04:44.960 | By the way, the 1997 match was not the first victory of machines over...

00:04:50.600 | Over grandmasters.

00:04:51.600 | Over grandmasters.

00:04:52.600 | Yeah.

00:04:53.600 | And I played against first decent chess computers from late '80s.

00:04:58.960 | So I played with the prototype of Deep Blue called Deep Thought in 1989, two rapid chess

00:05:04.560 | games in New York.

00:05:05.560 | I won handily to both games.

00:05:07.940 | We played against new chess engines like Fritz and other programs.

00:05:13.720 | And then it was Israeli program Junior that appeared in 1995.

00:05:17.200 | Right, right, I remember.

00:05:18.440 | So there were several programs.

00:05:20.800 | I lost few games in Blitz.

00:05:22.920 | I lost one match against the computer chess engine in 1994, rapid chess.

00:05:27.920 | So I lost one game to Deep Blue in 1996 match, the match I won.

00:05:33.040 | Some people tend to forget about it that I won the first match.

00:05:37.120 | But we made a very important psychological mistake thinking that the reason we lost Blitz

00:05:45.000 | matches, five minutes games, the reason we lost some of the rapid chess matches, 25 minutes

00:05:50.880 | games, because we didn't have enough time.

00:05:53.000 | If you play a longer match, we will not make the same mistakes.

00:05:56.760 | Nonsense.

00:05:57.760 | So yeah, we had more time, but we still make mistakes.

00:06:00.920 | And machine also has more time.

00:06:02.420 | And machine will always be steady and consistent compared to humans' instabilities and inconsistencies.

00:06:13.040 | And today we are at the point where nobody talks about humans playing against machines.

00:06:19.640 | Humans can offer handicap to top players, still will be favored.

00:06:25.840 | I think we're just learning that it's no longer human versus machines.

00:06:30.200 | It's about human working with machines.

00:06:32.720 | That's what I recognized in 1998, just after leaking my wounds and spending one year and

00:06:38.720 | just ruminating so what's happened in this match.

00:06:43.180 | And I knew that we still could play against the machines.

00:06:46.440 | I had two more matches in 2003 playing both deep free and deep junior.

00:06:51.480 | Both matches ended as a tie.

00:06:54.400 | Though these machines were not weaker, at least probably stronger than deep blue.

00:06:59.760 | And by the way, today, chess app on your mobile phone is probably stronger than deep blue.

00:07:05.160 | I'm not speaking about chess engines that are so much superior.

00:07:08.720 | And by the way, when you analyze games we played against deep blue in 1997 on your chess

00:07:13.100 | engine, they'll be laughing.

00:07:15.400 | And it also shows us how chess changed because chess commentators, they'll look at some of

00:07:20.880 | our games like game four, game five, brilliant idea.

00:07:24.120 | Now you ask Stockfish, you ask Houdini, you ask Commodore, all the leading chess engines.

00:07:31.880 | Within 30 seconds, they will show you how many mistakes both Gary and deep blue made

00:07:37.160 | in the game that was trumpeted as a great chess match in 1997.

00:07:45.400 | - Well, okay, so you've made an interesting, if you can untangle that comment.

00:07:51.200 | So now in retrospect, it was a mistake to see chess as the peak of human intellect.

00:07:58.160 | Nevertheless, that was done for centuries.

00:08:01.400 | So in Europe, because you move to the Far East, they will go, they're showing you-

00:08:09.640 | - Games, games.

00:08:10.640 | - Again, some of the games like board games.

00:08:14.840 | Yeah, I agree.

00:08:16.760 | - So if I push back a little bit, so now you say that, okay, but it was a mistake to see

00:08:22.480 | chess as the epitome.

00:08:25.320 | And then now there's other things maybe like language, like conversation, like some of

00:08:30.000 | the things that in your view is still way out of reach of computers, but inside humans.

00:08:35.560 | Do you think, can you talk about what those things might be?

00:08:39.160 | And do you think just like chess that might fall soon with the same set of approaches,

00:08:45.720 | if you look at alpha zero, the same kind of learning approaches as the machines grow in

00:08:51.240 | size?

00:08:52.240 | - No, it's not about growing in size.

00:08:54.120 | It's about, again, it's about understanding the difference between closed system and open

00:08:58.680 | ended system.

00:08:59.680 | - So you think that key difference, so the board games are closed in terms of the rules

00:09:05.440 | that they actions, the state space, everything is just constrained.

00:09:10.760 | You think once you open it, the machines are lost?

00:09:14.760 | - Not lost, but again, the effectiveness is very different because machine does not understand

00:09:19.800 | the moment it's reaching territory of diminishing returns.

00:09:23.400 | It's the, to put it in a different way, machine doesn't know how to ask right questions.

00:09:30.640 | It can ask questions, but it will never tell you which questions are relevant.

00:09:34.180 | So it's like about the, it's a direction.

00:09:37.120 | So I think it's in human machine relations, we have to consider so our role and many people

00:09:43.400 | feel uncomfortable that the territory that belongs to us is shrinking.

00:09:50.440 | I'm saying so what, this is eventually will belong to the last few decimal points, but

00:09:55.800 | it's like having so very powerful gun and all you can do there is slightly alter direction

00:10:05.640 | of the bullet, maybe 0.1 degree of this angle, but that means a mile away, 10 meters of target.

00:10:16.440 | So that's, we have to recognize that is a certain unique human qualities that machines

00:10:22.680 | in a foreseeable future will not be able to reproduce.

00:10:28.400 | And the effectiveness of this cooperation, collaboration depends on our understanding

00:10:32.840 | what exactly we can bring into the game.

00:10:35.160 | So the greatest danger is when we try to interfere with machine superior knowledge.

00:10:40.320 | So that's why I always say that sometimes you'd rather have, by reading this picture

00:10:44.680 | is in radiology, you may probably prefer an experienced nurse than rather than having

00:10:51.280 | top professor, because she will not try to interfere with machines understanding.

00:10:56.880 | So it's very important to know that if machines knows how to do better things in 95%, 96%

00:11:02.720 | of territory, we should not touch it because it's happened.

00:11:06.160 | It's like in chess, recognize, they do it better.

00:11:10.240 | See where we can make the difference.

00:11:12.120 | You mentioned AlphaZero, I mean, AlphaZero, it's actually a first step into what you may

00:11:18.160 | call AI, because everything that's being called AI today, it's one or another variation of

00:11:25.840 | what Claude Shannon characterized as a brute force, is a type A machine.

00:11:30.440 | Whether it's Deep Blue, whether it's Watson, and all these things, the modern technologies

00:11:36.040 | that are being trumpeted as AI, it's still brute force.

00:11:40.120 | It's the, all they do, it's they do optimization.

00:11:43.560 | It's this, they are, they keep improving the way to process human generated data.

00:11:51.640 | Now AlphaZero is the first step towards machine produced knowledge, which is by the way, it's

00:12:00.360 | quite ironic that the first company that championed that was IBM.

00:12:05.480 | Oh, it's in backgammon.

00:12:08.640 | Yes, you should look at IBM, it's a new gammon, it's the scientist, he's still working at

00:12:17.840 | IBM.

00:12:18.840 | They had in the early 90s.

00:12:20.720 | It's the program that played in all the AlphaZero types, so just trying to come up with own

00:12:26.000 | strategies.

00:12:27.000 | But because of success of Deep Blue, this project had been not abandoned, but just it

00:12:32.600 | was put on cold.

00:12:34.760 | And now it's, everybody talks about the machines generated knowledge, so as revolutionary,

00:12:43.240 | and it is, but there's still many open-ended questions.

00:12:48.640 | Yes, AlphaZero generates its own data.

00:12:53.040 | Many ideas that AlphaZero generated in chess were quite intriguing.

00:12:56.920 | So I looked at these games with, not just with interest, but it was quite exciting to

00:13:04.720 | learn how machine could actually juggle all the pieces and just play positions with a

00:13:10.840 | broken material balance, sacrificing material, always being ahead of other programs, one

00:13:15.640 | or two moves ahead by foreseeing the consequences, not over calculating, because other machines

00:13:22.480 | were at least as powerful in calculating, but it's having this unique knowledge based

00:13:28.200 | on discovered patterns.

00:13:30.060 | After playing 60 million games-

00:13:31.840 | Almost something that feels like intuition.

00:13:34.040 | Exactly.

00:13:35.040 | But there's one problem.

00:13:36.040 | Yeah.

00:13:37.040 | Now, the simple question, if AlphaZero faces superior point, let's say another powerful

00:13:44.360 | computer accompanied by a human who could help just to discover certain problems, because

00:13:51.000 | I already, I looked at many AlphaZero games, I visited their lab, spoke to Demis Kasabis

00:13:55.920 | and his team, and I know there's certain weaknesses there.

00:13:58.960 | Now if these weaknesses are exposed, then the question is, how many games will it take

00:14:02.240 | for AlphaZero to correct it?

00:14:04.280 | The answer is hundreds of thousands.

00:14:06.400 | Even if it keeps losing, it's just because the whole system is based.

00:14:11.160 | So it's now, imagine, so you can have a human by just making a few tweaks.

00:14:16.440 | So humans are still more flexible, and as long as we recognize what is our role, where

00:14:23.200 | we can play sort of the most valuable part in this collaboration, so it will help us

00:14:30.600 | to understand what are the next steps in human-machine collaboration.

00:14:33.760 | Thank you.

00:14:34.760 | Thank you.

00:14:35.760 | Thank you.

00:14:36.760 | Thank you.

00:14:41.760 | Thank you.

00:14:46.760 | Thank you.

00:14:51.760 | you

Garry Kasparov: IBM Deep Blue, AlphaZero, and the Limits of AI in Open Systems | AI Podcast Clips

Chapters