Guido van Rossum: Python and the Future of Programming

00:00:00.000 | Can you imagine possible features

00:00:03.080 | that Python 4.0 might have

00:00:07.080 | that would necessitate the creation of the new 4.0?

00:00:12.080 | Given the amount of pain and joy, suffering and triumph

00:00:18.160 | that was involved in the move

00:00:20.320 | between version two and version three.

00:00:23.020 | The following is a conversation with Guido van Rossum,

00:00:28.040 | his second time on this podcast.

00:00:29.880 | He is the creator of the Python programming language

00:00:33.000 | and is Python's emeritus BDFL, Benevolent Dictator for Life.

00:00:38.000 | This is the Lex Friedman Podcast.

00:00:41.080 | To support it,

00:00:41.920 | please check out our sponsors in the description.

00:00:44.320 | And now, dear friends, here's Guido van Rossum.

00:00:47.660 | Python 3.11 is coming out very soon.

00:00:52.480 | In it, CPython claimed to be 10 to 60% faster.

00:00:57.480 | How'd you pull that off?

00:00:59.320 | And what's CPython?

00:01:00.560 | CPython is the last Python implementation standing,

00:01:04.080 | also the first one that was ever created.

00:01:06.360 | The original Python implementation

00:01:08.320 | that I started over 30 years ago.

00:01:10.280 | So what does it mean that Python,

00:01:12.080 | the programming language,

00:01:13.200 | is implemented in another programming language called C?

00:01:16.920 | What kind of audience do you have in mind here?

00:01:19.760 | People who know programming?

00:01:21.320 | No, there's somebody on a boat that's into fishing

00:01:23.920 | and they've never heard about programming,

00:01:25.800 | but also some world-class programmers.

00:01:27.960 | So you're gonna have to speak to both.

00:01:29.400 | Imagine a boat with two people.

00:01:31.360 | One of them has not heard about programming,

00:01:33.320 | is really into fishing.

00:01:34.800 | And the other one is like an incredible

00:01:38.360 | Silicon Valley programmer that's programmed in everything.

00:01:41.480 | C, C++, Python, Rust, Java,

00:01:45.080 | and knows the entire history of programming languages.

00:01:47.240 | So you're gonna have to speak to both.

00:01:49.160 | - I imagine that boat in the middle of the ocean.

00:01:51.400 | - Yes.

00:01:52.240 | - I'm gonna please the guy who knows how to fish first.

00:01:55.080 | - Yes, please.

00:01:55.920 | (both laugh)

00:01:57.240 | - He seems like the most useful in the middle of the ocean.

00:01:59.800 | You gotta make him happy.

00:02:01.520 | - I'm sure he has a cell phone.

00:02:03.040 | So he's probably very suspicious

00:02:06.000 | about what goes on in that cell phone,

00:02:07.880 | but he must have heard that inside his cell phone

00:02:10.520 | is a tiny computer.

00:02:12.200 | And a programming language is computer code

00:02:15.360 | that tells the computer what to do.

00:02:17.400 | - It's a very low level language.

00:02:20.400 | It's zeros and ones, and then there's assembly, and then--

00:02:24.120 | - Oh yeah, we don't talk about these really low levels

00:02:27.880 | because those just confuse people.

00:02:30.120 | I mean, when we're talking about human language,

00:02:32.880 | we're not usually talking about vocal tracts

00:02:35.480 | and how you position your tongue.

00:02:37.440 | I was talking yesterday about how

00:02:39.640 | when you have a Chinese person and they speak English,

00:02:44.000 | this is a bit of a stereotype they often don't know,

00:02:48.520 | or they can't seem to make the difference well

00:02:51.480 | between an L and an R.

00:02:54.120 | And I have a theory about that,

00:02:55.640 | and I've never checked this with linguists,

00:02:58.040 | that it probably has to do with the fact

00:03:02.440 | that in Chinese there is not really a difference.

00:03:05.680 | And it could be that there are regional variations

00:03:08.680 | in how native Chinese speakers pronounce that one sound

00:03:13.680 | that sounds like L to some of them, like R to others.

00:03:19.280 | - So it's both the sounds you produce with your mouth

00:03:23.320 | throughout the history of your life

00:03:25.200 | and what you're used to listening to.

00:03:27.160 | I mean, every language has that.

00:03:28.300 | Russian has-- - Exactly.

00:03:29.600 | - The Slavic languages have sounds like zh,

00:03:31.960 | the letter zh, like Americans or English speakers

00:03:36.960 | don't seem to know the sound zh.

00:03:39.080 | They seem uncomfortable with that sound.

00:03:43.640 | Yeah, so I'm, oh yes, okay, so we're not going

00:03:47.160 | to the shapes of tongues and the sounds

00:03:50.040 | that the mouth can make, fine.

00:03:51.400 | Words-- - And similarly,

00:03:52.360 | we're not going into the ones and zeros or machine language.

00:03:56.280 | I would say a programming language is a list

00:03:58.880 | of instructions like a cookbook recipe

00:04:02.240 | that sort of tells you how to do a certain thing,

00:04:06.920 | like make a sandwich.

00:04:08.040 | Well, acquire a loaf of bread, cut it in slices,

00:04:12.260 | take two slices, put mustard on one,

00:04:16.360 | put jelly on the other or something,

00:04:20.240 | then add the meat, then add the cheese.

00:04:22.480 | I've heard that science teachers can actually

00:04:26.680 | do great stuff with recipes like that

00:04:29.680 | and trying to interpret their students' instructions

00:04:32.440 | incorrectly until the students

00:04:34.600 | are completely unambiguous about it.

00:04:37.440 | - With language, see, that's the difference

00:04:39.120 | between natural languages and programming languages.

00:04:42.880 | I think ambiguity is a feature, not a bug

00:04:46.560 | in human spoken languages.

00:04:48.880 | That's the dance of communication between humans.

00:04:54.280 | - Well, for lawyers, ambiguity certainly is a feature.

00:04:58.120 | For plenty of other cases, the ambiguity

00:05:03.880 | is not much of a feature, but we work around it, of course.

00:05:08.200 | What's more important is context.

00:05:11.200 | So with context, the precision of the statement

00:05:14.240 | becomes more and more concrete.

00:05:16.080 | Right, but when you say I love you

00:05:19.160 | to a person that matters a lot to you,

00:05:22.440 | the person doesn't try to compile that statement

00:05:24.720 | and return an error saying, please define love.

00:05:27.520 | Right? - No, but I imagine

00:05:30.000 | that my wife and my son interpret it very differently.

00:05:34.400 | - Yes. - Even though it's

00:05:36.560 | the same three words. - But imprecisely still.

00:05:40.040 | - Oh, for sure. - Lawyers have a lot

00:05:43.120 | of follow-up questions for you.

00:05:44.560 | - Nevertheless, the context is already different

00:05:47.360 | in that case. - Yes, fair enough.

00:05:49.000 | So that's a programming language,

00:05:52.200 | is ability to unambiguously state a recipe.

00:05:57.160 | Actually, let's go back.

00:05:59.800 | Let's go to PEP8.

00:06:01.400 | You go through in PEP8 the style guide for Python code,

00:06:05.080 | some ideas of what this language should look like,

00:06:09.320 | feel like, read like.

00:06:11.400 | And the big idea there is that code readability counts.

00:06:14.680 | What does that mean to you?

00:06:16.040 | And how do we achieve it?

00:06:17.320 | So this recipe should be readable.

00:06:19.040 | - That's a thing between programmers.

00:06:21.360 | Because on the one hand, we always explain the concept

00:06:26.240 | of programming language as computers need instructions

00:06:31.200 | and computers are very dumb

00:06:32.920 | and they need very precise instructions

00:06:35.040 | because they don't have much context.

00:06:37.560 | In fact, they have lots of context,

00:06:39.200 | but their context is very different.

00:06:42.280 | But what we've seen emerge during the development

00:06:46.680 | of software starting in the, probably in the late 40s,

00:06:50.600 | is that software is a very social activity.

00:06:56.520 | A software developer is not a mad scientist

00:06:59.520 | who sits alone in his lab writing brilliant code.

00:07:02.680 | A software is developed by teams of people.

00:07:08.880 | Even the mad scientist sitting alone in his lab

00:07:11.360 | can type fast enough to produce enough code

00:07:14.800 | so that by the time he's done with his coding,

00:07:17.680 | he still remembers what the first few lines he wrote mean.

00:07:22.080 | So even the mad scientist coding alone in his lab

00:07:26.840 | would be sort of wise to adopt conventions

00:07:31.320 | on how to format the instructions

00:07:35.560 | that he gives to the computer.

00:07:37.040 | So that the thing is, there is a difference

00:07:40.040 | between a cookbook recipe and a computer program.

00:07:44.360 | The cookbook recipe, the author of the cookbook

00:07:47.760 | writes it once and then it's printed in 100,000 copies.

00:07:52.760 | And then lots of people in their kitchens

00:07:55.680 | try to recreate that recipe,

00:07:57.880 | that particular pie or dish from the recipe.

00:08:03.840 | And so there, the goal of the cookbook author

00:08:08.840 | is to make it clear to the human reader of the recipe,

00:08:14.680 | the human amateur chef in most cases.

00:08:18.760 | When you're writing a computer program,

00:08:23.960 | you have two audiences at once.

00:08:26.240 | It needs to tell the computer what to do,

00:08:32.280 | but it also is useful if that program

00:08:37.120 | is readable by other programmers.

00:08:40.040 | Because computer software,

00:08:41.560 | unlike the typical recipe for a cherry pie,

00:08:45.800 | is so complex that you don't get all of it right at once.

00:08:50.800 | You end up with the activity of debugging

00:08:55.960 | and you end up with the activity of...

00:08:58.160 | So debugging is trying to figure out

00:09:03.160 | why your code doesn't run the way

00:09:04.800 | you thought it should run.

00:09:06.560 | - That means broadly, it could be stupid little errors

00:09:08.960 | or it could be big logical errors.

00:09:11.400 | - It could be anything--

00:09:12.960 | - Spiritual.

00:09:14.000 | - Yeah, it could be anything from a typo

00:09:16.640 | to a wrong choice of algorithm

00:09:19.680 | to building something that does what you tell it to do,

00:09:24.280 | but that's not useful.

00:09:26.480 | - Yeah, it seems to work really well 99% of the time,

00:09:30.720 | but does weird things 1% of the time on some edge cases.

00:09:35.160 | - That's pretty much all software nowadays.

00:09:37.560 | - All good software, right?

00:09:38.920 | - Well, yeah, for bad software.

00:09:41.080 | - That 99 goes down a lot.

00:09:44.120 | But it's not just about the complexity of the program.

00:09:47.000 | Like you said, it is a social endeavor

00:09:50.680 | in that you're constantly improving

00:09:53.120 | that recipe for the cherry pie.

00:09:55.320 | But you're in a group of people improving that recipe.

00:10:00.320 | Or the mad scientist is improving the recipe

00:10:04.200 | that he created a year ago and making it better.

00:10:08.720 | Or adding something.

00:10:10.960 | He decides that he wants, I don't know,

00:10:13.880 | he wants some decoration on his pie or icing.

00:10:16.600 | - So there's broad philosophical things

00:10:19.880 | and there's specific advice on style.

00:10:22.680 | So first of all, the thing that people first experience

00:10:24.920 | when they look at Python,

00:10:26.160 | there is a, it is very readable,

00:10:30.120 | but there's also like a spatial structure to it.

00:10:34.200 | Can you explain the indentation style of Python

00:10:37.040 | and what is the magic to it?

00:10:39.400 | - Spaces are important for readability of any kind of text.

00:10:44.360 | If you take a cookbook recipe

00:10:47.600 | and you remove all the sort of,

00:10:52.000 | all the bullets and other markup,

00:10:55.760 | and you just crunch all the text together,

00:10:57.880 | maybe you leave the spaces between the words,

00:10:59.920 | but that's all you leave.

00:11:01.320 | When you're in the kitchen trying to figure out,

00:11:05.600 | oh, what are the ingredients and what are the steps?

00:11:09.400 | And where does this step end and the next step begin?

00:11:12.760 | You're gonna have a hard time

00:11:14.040 | if it's just one solid block of text.

00:11:16.640 | On the other hand, what a typical cookbook does

00:11:20.520 | if the paper is not too expensive,

00:11:23.960 | each recipe starts on its own page.

00:11:26.360 | Maybe there's a picture next to it.

00:11:28.640 | The list of ingredients comes first.

00:11:31.160 | There's a standard notation.

00:11:34.920 | There's shortcuts so that you don't have to

00:11:38.720 | sort of write two sentences on how you have to cut the onion

00:11:43.240 | because there are only three ways

00:11:44.640 | that people ever cut onions in a kitchen,

00:11:46.960 | small, medium, and in slices, or something like that.

00:11:50.520 | - Right.

00:11:52.320 | - None of my examples make any sense

00:11:53.880 | to real cooks, of course, but.

00:11:55.400 | - Yeah.

00:11:56.240 | (laughing)

00:11:58.040 | We're talking to programmers with a metaphor of cooking.

00:12:00.440 | I love it.

00:12:01.680 | But there is a strictness to the spacing that Python defines.

00:12:06.040 | So there's some looser things, some stricter things,

00:12:10.080 | but the four spaces for the indentation

00:12:13.800 | is really interesting.

00:12:14.840 | It really defines what the language looks and feels like.

00:12:19.840 | - Because indentation sort of taking a block of text

00:12:23.840 | and then having inside that block of text

00:12:27.400 | a smaller block of text that is indented further

00:12:31.000 | as sort of a group, it's like you have a bulleted list

00:12:36.000 | in a complex business document

00:12:39.760 | and inside some of the bullets are other bulleted lists.

00:12:43.760 | You will indent those too.

00:12:45.440 | If each bulleted list is indented several inches,

00:12:50.280 | then at two levels deep, there's no space left on the page

00:12:54.280 | to put any of the words of the text.

00:12:56.440 | So you can't indent too far.

00:12:58.160 | On the other hand, if you don't indent at all,

00:13:01.440 | you can't tell whether something is a top-level bullet

00:13:04.440 | or a second-level bullet or a third-level bullet.

00:13:06.880 | So you have to have some compromise.

00:13:10.160 | And based on ancient conventions

00:13:15.160 | and the sort of the typical width of a computer screen

00:13:19.240 | in the '80s and all sorts of things,

00:13:23.120 | we came up with sort of four spaces as a compromise.

00:13:30.360 | I mean, there are large groups of people

00:13:34.320 | who code with two spaces per indent level.

00:13:38.640 | For example, the Google Style Guide,

00:13:40.560 | all the Google Python code,

00:13:43.240 | and I think also all the Google C++ code

00:13:45.560 | is indented with only two spaces per block.

00:13:48.480 | If you're not used to that,

00:13:50.240 | it's harder to, at a glance, understand the code

00:13:55.240 | because the sort of the high-level structure

00:13:58.840 | is determined by the indentation.

00:14:01.200 | On the other hand, there are other programming languages

00:14:04.200 | where the indentation is eight spaces or a whole tab stop

00:14:09.000 | in sort of classic Unix.

00:14:11.240 | And to me, that looks weird

00:14:12.600 | because you sort of, after three indent levels,

00:14:15.000 | you've got no room left.

00:14:17.040 | - Well, there's some languages

00:14:19.360 | where the indentation is a recommendation,

00:14:22.560 | it's a stylistic one.

00:14:23.800 | The code compiles even without any indentation.

00:14:27.240 | And then Python, really,

00:14:29.000 | indentation is a fundamental part of the language, right?

00:14:32.960 | - It doesn't have to be four spaces.

00:14:34.840 | So you can code Python with two spaces per block

00:14:39.000 | or six spaces or 12 if you really want to go wild.

00:14:44.000 | But sort of everything that belongs to the same block

00:14:49.400 | needs to be indented the same way.

00:14:52.960 | In practice, in most other languages,

00:14:55.080 | people recommend doing that anyway.

00:14:57.000 | If you look at C or Rust or C++,

00:15:02.480 | all those languages, Java,

00:15:05.240 | don't have a requirement of indentation,

00:15:08.760 | but except in extreme cases,

00:15:11.720 | they're just as anal about

00:15:14.440 | having their code properly indented.

00:15:17.000 | - So any IDE that the syntax highlighting

00:15:22.000 | that works with Java or C++,

00:15:24.080 | they will yell at you aggressively

00:15:26.680 | if you don't do proper indentation.

00:15:28.640 | - They'd suggest the proper indentation for you,

00:15:32.400 | like in C you type a few words

00:15:36.480 | and then you type a curly brace,

00:15:38.240 | which is their notion of sort of begin an indented block.

00:15:43.240 | Then you hit return

00:15:46.240 | and then it automatically indents four or eight spaces

00:15:49.240 | depending on your style preferences

00:15:52.040 | or how your editor is configured.

00:15:53.880 | - Was there a possible universe

00:15:55.880 | in which you considered having braces in Python?

00:15:59.240 | - Absolutely, yeah.

00:16:00.720 | - What is it, 60/40, 70/30 in your head?

00:16:04.640 | What was the trade-off?

00:16:07.120 | - For a long time, I was actually convinced

00:16:10.960 | that the indentation was just better.

00:16:14.080 | Without context, I would still claim

00:16:20.200 | that indentation is better.

00:16:22.160 | It reduces clutter.

00:16:25.920 | However, as I started to say earlier,

00:16:29.200 | context is almost everything.

00:16:31.040 | In the context of coding,

00:16:34.440 | most programmers are familiar with multiple languages,

00:16:38.800 | even if they're only good at one or two.

00:16:41.600 | Apart from Python and maybe Fortran,

00:16:46.120 | I don't know how that's written these days anymore,

00:16:48.160 | but all the other languages, Java, Rust, C, C++,

00:16:52.080 | JavaScript, TypeScript, Perl,

00:16:55.440 | are all using curly braces to sort of indicate blocks.

00:17:00.440 | And so Python is the odd one out.

00:17:04.720 | - So it's a radical idea.

00:17:06.600 | Do you still, as a radical renegade revolutionary,

00:17:10.000 | do you still stand behind this idea

00:17:12.000 | of indentation versus braces?

00:17:15.960 | Like, what, can you dig into it a little bit more,

00:17:19.000 | why you still stand behind indentation?

00:17:22.640 | - Because context is not the whole story.

00:17:25.640 | History, in a sense, provides more context.

00:17:29.440 | So for Python, there's no chance that we can switch.

00:17:34.440 | Python is using curly braces for something else,

00:17:39.800 | dictionaries mostly.

00:17:41.160 | We would get in trouble if we wanted to switch.

00:17:44.640 | Just like you couldn't redefine C to use indentation,

00:17:49.960 | even if you agree that indentation

00:17:53.640 | sort of in a greenfield environment would be better,

00:17:57.960 | you can't change that kind of thing in a language.

00:18:02.800 | It's hard enough to reach agreement

00:18:05.440 | over much more minor details.

00:18:08.160 | Maybe, I mean, in the past in Python,

00:18:10.320 | we did have a big debate about tabs versus spaces

00:18:13.680 | and four spaces versus fewer or more.

00:18:17.160 | And we sort of came up with a recommended standard

00:18:21.520 | and sort of options for people who want to be different.

00:18:25.520 | - But yes, I guess the thought experiment

00:18:30.080 | I'd like you to consider is if you could travel back

00:18:33.080 | through time when the compatibility is not an issue

00:18:38.080 | and you started Python all over again,

00:18:40.840 | can you make the case for indentation still?

00:18:46.160 | - Well, it frees up a pair of matched brackets

00:18:49.880 | of which there are never enough in the world

00:18:52.160 | for other purposes.

00:18:55.160 | It really makes the language slightly

00:18:57.960 | sort of easier to grasp for people who don't already know

00:19:04.080 | another programming language.

00:19:07.680 | Because the sort of one of the things,

00:19:11.720 | and I mostly got this from my mentors

00:19:14.560 | who taught me programming language design

00:19:18.360 | in the earlier 80s.

00:19:20.680 | When you're teaching programming,

00:19:22.440 | for the total newbie who has not coded before,

00:19:28.880 | in not in any other language,

00:19:32.000 | a whole bunch of concepts in programming are very alien

00:19:38.040 | or sort of new and maybe very interesting,

00:19:44.240 | but also distracting and confusing.

00:19:46.680 | And there are many different things you have to learn.

00:19:48.960 | You have to sort of,

00:19:50.840 | in a typical 13 week programming course,

00:19:55.520 | you have to, if it's like really learning to program

00:20:00.520 | from scratch, you have to cover algorithms,

00:20:03.680 | you have to cover data structures,

00:20:05.240 | you have to cover syntax, you have to cover variables,

00:20:09.000 | loops, functions, recursion, classes.

00:20:13.920 | Expressions, operators.

00:20:16.640 | There are so many concepts.

00:20:18.560 | If you can spend a little less time

00:20:23.440 | having to worry about the syntax.

00:20:27.720 | The classic example was often,

00:20:30.240 | oh, the compiler complains every time I put a semicolon

00:20:36.920 | in the wrong place, or I forget to put a semicolon.

00:20:42.200 | Python doesn't have semicolons in that sense.

00:20:45.280 | So you can't forget them.

00:20:47.200 | And you are also not sort of misled

00:20:51.160 | into putting them where they don't belong

00:20:53.120 | because you don't learn about them in the first place.

00:20:56.960 | - The flip side of that is forcing the strictness

00:21:00.280 | onto the beginning programmer to teach them

00:21:03.480 | that programming values attention to details.

00:21:08.040 | You don't get to just write the way you write

00:21:10.040 | in English paper. - Plenty of other details

00:21:11.680 | that they have to pay attention to.

00:21:13.840 | So I think they'll still get the message

00:21:16.400 | about paying attention to details.

00:21:19.760 | - The interesting design choice,

00:21:21.080 | I still program quite a bit in PHP,

00:21:24.000 | and I'm sure there's other languages like this,

00:21:26.080 | but the dollar sign before a variable,

00:21:29.200 | that was always an annoying thing for me.

00:21:33.080 | It didn't quite fit into my understanding

00:21:36.000 | of why this is good for a programming language.

00:21:38.480 | I'm not sure if you ever thought about that one.

00:21:41.680 | - That is a historical thing.

00:21:44.400 | There is a whole lineage of programming languages.

00:21:47.840 | PHP is one, Perl was one,

00:21:52.360 | and the Unix shell is one of the oldest

00:21:56.520 | or all the different shells.

00:21:58.640 | The dollar was invented for that purpose

00:22:03.360 | because the very earliest shells had a notion of scripting,

00:22:07.640 | but they did not have a notion

00:22:09.240 | of parameterizing the scripting.

00:22:11.320 | And so a script is just a few lines of text

00:22:17.000 | where each line of text is a command

00:22:19.640 | that is read by a very primitive command processor

00:22:23.480 | that then sort of takes the first word on the line

00:22:27.040 | as the name of a program and passes all the rest

00:22:31.480 | of the line as text into the program

00:22:35.000 | for the program to figure out what to do with as arguments.

00:22:39.640 | And so by the time scripting was slightly more mature

00:22:44.320 | than the very first script,

00:22:46.600 | there was a convention that just like the first word

00:22:51.120 | of the line is the name of the program,

00:22:55.160 | the following words could be names of files.

00:23:00.640 | Input.text, output.html, things like that.

00:23:05.640 | The next thing that happens is,

00:23:08.720 | oh, it would actually be really nice

00:23:11.080 | if we could have variables

00:23:12.920 | and especially parameters for scripts.

00:23:15.480 | Parameters are usually what starts this process.

00:23:18.720 | But now you have a problem

00:23:21.440 | because you can't just say the parameters are X, Y, and Z.

00:23:27.480 | And so now we call, say, let's say X is the input file

00:23:31.920 | and Y is the output file, and let's forget about Z for now.

00:23:35.760 | I have my program and I write program X, Y.

00:23:39.680 | Well, that already has a meaning

00:23:41.120 | because that presumably means X itself is the file.

00:23:46.120 | It's a file name.

00:23:48.480 | It's not a variable name.

00:23:50.120 | And so the inventors of things like the Unix shell

00:23:57.440 | and I'm sure job command language at IBM before that

00:24:01.680 | had to use something that made it clear

00:24:07.920 | to the script processor,

00:24:09.760 | here is an X that is not actually the name of a file

00:24:14.560 | which you just pass through to the program you're running.

00:24:19.560 | Here is an X that is the name of a variable.

00:24:23.040 | And when you're writing a script processor,

00:24:27.600 | you try to keep it as simple as possible

00:24:30.840 | because certainly in the '50s and '60s,

00:24:34.480 | the thing that interprets the script

00:24:37.960 | was itself had to be a very small program

00:24:41.000 | because it had to fit in a very small part of memory.

00:24:44.440 | And so saying, oh, just look at each character

00:24:48.680 | and if you see a dollar sign,

00:24:51.000 | you jump to another section of the code

00:24:53.160 | and then you gobble up characters

00:24:54.760 | or say until the next space or something

00:24:57.680 | and you say, that's the variable name.

00:24:59.580 | And so it was sort of invented as a clever way

00:25:05.480 | to make parsing of things that contain both variable

00:25:12.200 | and fixed parts very easy in a very simple script processor.

00:25:18.080 | - It also helps, even then, it also helps the human author

00:25:23.080 | and the human reader of the script to quickly see,

00:25:28.160 | oh, 20 lines down in the script, I see a reference to XYZ.

00:25:34.080 | Oh, it has a dollar in front of it.

00:25:35.620 | So now we know that XYZ must be

00:25:37.840 | one of the parameters of the script.

00:25:39.600 | - Well, this is fascinating.

00:25:41.000 | Several things to say, which is the leftovers

00:25:44.600 | from the simple script processor languages

00:25:47.960 | are now in code bases like behind Facebook

00:25:51.000 | or behind most of the backend.

00:25:52.600 | I think PHP is probably still runs

00:25:54.320 | most of the backend of the internet.

00:25:56.240 | - Oh yeah, yeah, I think there's a lot of it

00:25:57.880 | in Wikipedia too, for example.

00:26:00.400 | - It's funny that those decisions, or not funny,

00:26:02.640 | it's fascinating that those decisions permeate through time.

00:26:07.040 | - Just like biological systems, right?

00:26:10.000 | I mean, the sort of, the inner workings of DNA

00:26:13.880 | have been stable for, well, I don't know how long it was,

00:26:17.720 | like 300 million years, half a billion years.

00:26:21.520 | - Yeah.

00:26:22.400 | - And there are all sorts of weird quirks there

00:26:26.040 | that don't make a lot of sense if you were to design

00:26:29.440 | a system like self-replicating molecules from scratch.

00:26:33.240 | - Well, that system has a lot of interesting resilience.

00:26:36.960 | It has redundancy that results,

00:26:40.880 | like it messes up in interesting ways

00:26:42.880 | that still is resilient when you look

00:26:45.080 | at the system level of the organism.

00:26:47.400 | Code doesn't necessarily have that,

00:26:50.760 | a computer programming code.

00:26:52.640 | - You'd be surprised how much resilience modern code has.

00:26:57.640 | I mean, if you look at the number of bugs per line of code,

00:27:02.900 | even in very well-tested code

00:27:08.440 | that in practice works just fine,

00:27:10.760 | there are actually lots of things that don't work fine.

00:27:16.140 | And there are error-correcting

00:27:18.720 | or self-correcting mechanisms at many levels.

00:27:22.560 | - Including probably the user of the code?

00:27:25.000 | - Well, in the end, the user who sort of is told,

00:27:28.600 | well, you got to reboot your PC, is part of that system.

00:27:33.600 | And a slightly less drastic thing is reload the page,

00:27:38.640 | which we all know how to do without thinking about it

00:27:43.600 | when something weird happens.

00:27:45.560 | You try to reload a few times before you say,

00:27:48.240 | oh, there's something really weird.

00:27:50.040 | - Okay, or try to click the button again

00:27:51.920 | if the first time didn't work.

00:27:53.600 | (both laughing)

00:27:54.560 | - Well, yeah, we should all have learned not to do that

00:27:57.880 | because that's probably just gonna turn the light back off.

00:28:01.520 | - Yeah, true, so do it three times.

00:28:03.360 | That's the right lesson.

00:28:05.040 | And I wonder how many people actually like the dollar sign.

00:28:11.920 | Like you said, it is documentation.

00:28:14.120 | So to me, it's whatever the opposite of syntactic sugar is,

00:28:18.680 | syntactic poison.

00:28:20.760 | To me, it is such a pain in the ass

00:28:23.160 | that I have to type in a dollar sign.

00:28:24.840 | Also super error-prone.

00:28:27.080 | So it's not self-documenting.

00:28:28.880 | It's like a bug-generating thing.

00:28:31.960 | It is a kind of documentation, that's the pro,

00:28:34.080 | and the con is it's a source of a lot of bugs.

00:28:37.280 | But actually, I have to ask you,

00:28:39.640 | this is a really interesting idea of bugs per line of code.

00:28:43.640 | If you look at all the computer systems out there,

00:28:46.520 | from the code that runs nuclear weapons

00:28:49.600 | to the code that runs all the amazing companies

00:28:52.400 | that you've been involved with and not,

00:28:55.440 | the code that runs Twitter and Facebook and Dropbox

00:28:58.120 | and Google and Microsoft, Windows, and so on,

00:29:02.000 | and we like laid out,

00:29:03.680 | wouldn't that be a cool table, bugs per line of code?

00:29:09.720 | And let's put actual companies aside.

00:29:13.520 | Do you think we'd be surprised by the number we see there

00:29:16.320 | for all these companies?

00:29:17.520 | - That depends on whether you've ever read about research

00:29:23.040 | that's been done in this area before.

00:29:26.400 | And I don't know, the last time I saw some research like that

00:29:31.400 | that was probably in the '90s,

00:29:35.120 | and the research might have been done in the '80s,

00:29:38.120 | but the conclusion was across a wide variety of companies

00:29:43.120 | a wide range of different software, different languages,

00:29:48.120 | different companies, different development styles.

00:29:52.720 | The number of bugs is always,

00:29:56.840 | I think it's in the order of about one bug per thousand lines

00:30:00.640 | in sort of mature software that is considered

00:30:05.360 | - Interesting. - as good as it gets.

00:30:08.280 | - Can I give you some facts here?

00:30:09.640 | There's a lot of good papers. - Oh yeah.

00:30:11.640 | - So you said mature software, right?

00:30:13.960 | So here's a report from a programming analytics company.

00:30:18.960 | Now this is from a developer perspective.

00:30:24.920 | Let me just say what it says

00:30:26.240 | 'cause this is very weird and surprising.

00:30:28.720 | On average, a developer creates 70 bugs

00:30:31.600 | per 1,000 lines of code.

00:30:33.440 | 15 bugs per 1,000 lines of code

00:30:36.680 | find their way to the customers.

00:30:38.740 | This is in software they've analyzed.

00:30:41.240 | - Oh, I was wrong by an order of magnitude there.

00:30:44.920 | - Fixing a bug takes 30 times longer

00:30:47.120 | than writing a line of code.

00:30:49.160 | That I can believe. - Yeah, totally.

00:30:51.240 | - 75% of a developer's time is spent on debugging.

00:30:54.480 | That's for an average developer.

00:30:57.240 | They analyze this 1,500 hours a year.

00:31:02.240 | In US alone, $113 billion is spent annually

00:31:07.240 | on identifying and fixing bugs.

00:31:10.440 | And I imagine this is marketing literature

00:31:13.080 | for someone who claims to have a golden bullet

00:31:15.800 | or a silver bullet that makes all that investment

00:31:19.960 | in fixing bugs go away.

00:31:21.680 | But that is usually not going to,

00:31:25.760 | that's not gonna happen.

00:31:27.040 | - Well, they're, I mean, they're referencing

00:31:29.040 | a lot of stuff, of course, but it is a page

00:31:31.200 | that is, you know, there's a contact us button

00:31:34.600 | at the bottom.

00:31:36.200 | Presumably, if you just spend a little bit less

00:31:39.400 | than $100 billion, we're willing to solve

00:31:41.520 | the problem for you.

00:31:42.520 | Right, and there's also a report on Stack Exchange,

00:31:46.760 | Stack Overflow, on the exact same topic,

00:31:48.480 | but when I open it up at the moment,

00:31:50.480 | the page says Stack Overflow is currently offline

00:31:52.960 | for maintenance.

00:31:54.160 | - Oh, that is ironic. - Yes.

00:31:56.600 | By the way, their error page is awesome.

00:31:58.360 | Anyway, I mean, can you believe that number of bugs?

00:32:02.760 | - Oh, absolutely.

00:32:04.080 | - Isn't that scary that 70 bugs per 1,000 lines of code,

00:32:07.920 | so even 10 bugs per 1,000 lines of code?

00:32:09.640 | - Well, that's about one bug every 15 lines,

00:32:13.040 | and that's when you're first typing it in.

00:32:15.160 | - Yeah, from a developer, but like,

00:32:18.240 | how many bugs are gonna be found if you're typing it in?

00:32:22.200 | - Well, the development process is extremely iterative.

00:32:26.280 | Typically, you don't make a plan for what software

00:32:30.400 | you're going to release a year from now,

00:32:32.600 | and work out all the details,

00:32:35.880 | because actually all the details themselves consist,

00:32:40.640 | they sort of compose a program,

00:32:42.760 | and that being a program,

00:32:47.080 | all your plans will have bugs in them too,

00:32:49.760 | and inaccuracies,

00:32:51.240 | but what you actually do is,

00:32:54.920 | you do a bunch of typing,

00:32:58.080 | and I'm actually really, I'm a really bad typist,

00:33:04.040 | I've never learned to type with 10 fingers.

00:33:06.240 | - How many do you use?

00:33:08.880 | - Well, I use all 10 of them, but not very well,

00:33:12.640 | but I never took a typing class,

00:33:15.160 | and I never sort of corrected that,

00:33:17.040 | so the first time I seriously learned,

00:33:20.400 | I had to learn the layout of a QWERTY keyboard,

00:33:24.960 | was actually in college, in my first programming classes,

00:33:28.840 | where we used punch cards,

00:33:31.640 | and so with my two fingers,

00:33:34.400 | I sort of pecked out my code.

00:33:37.920 | Watch anyone give you a little coding demonstration,

00:33:43.840 | they'll have to produce like four lines of code,

00:33:47.440 | and now see how many times they use the backspace key,

00:33:52.680 | yeah, because they made a mistake,

00:33:54.400 | and some people, especially when someone else is looking,

00:34:00.240 | will backspace over 20, 30, 40 characters

00:34:05.240 | to fix a typo earlier in a line,

00:34:08.760 | if you're slightly more experienced,

00:34:12.400 | of course you use your arrow buttons to go,

00:34:15.080 | or your mouse to, but the mouse is usually slower

00:34:17.560 | than the arrows, but a lot of people,

00:34:22.360 | when they type a 20 character word,

00:34:24.920 | which is not unusual,

00:34:27.040 | and they realize they made a mistake

00:34:29.680 | at the start of the word,

00:34:30.760 | they backspace over the whole thing,

00:34:33.120 | and then retype it,

00:34:34.280 | and sometimes it takes three, four times to get it right,

00:34:37.080 | so I don't know what your definition of bug is,

00:34:41.680 | arguably mistyping a word,

00:34:44.000 | and then correcting it immediately is not a bug,

00:34:47.520 | on the other hand, you already do sort of lose time,

00:34:52.520 | and every once in a while,

00:34:54.520 | there's sort of a typo that you don't get in that process,

00:34:59.520 | and now you've typed like 10 lines of code,

00:35:02.880 | and somewhere in the middle of it,

00:35:05.320 | you don't know where yet is a typo,

00:35:07.720 | or maybe a thinko where you forgot

00:35:10.920 | that you had to initialize a variable or something.

00:35:14.600 | - But those are two different things,

00:35:15.760 | and I would say yes,

00:35:16.880 | you have to actually run the code to discover that typo,

00:35:20.000 | but forgetting to initialize a variable

00:35:22.400 | is a fundamentally different thing,

00:35:25.080 | because that thing can go undiscovered.

00:35:27.760 | - That depends on the language,

00:35:29.200 | in Python it will not.

00:35:30.840 | And sort of modern compilers are usually pretty good

00:35:33.520 | at catching that, even for C.

00:35:36.720 | - So for that specific thing,

00:35:38.160 | but actually deeper,

00:35:40.040 | there might be another variable that is initialized,

00:35:45.080 | but logically speaking,

00:35:47.040 | the one you meant-- - Related, yep.

00:35:51.080 | - It's like name the same, but it's a different thing,

00:35:53.120 | and you forgot to initialize whatever,

00:35:55.760 | some counter or some basic variable

00:35:58.280 | that you're using for-- - I can tell

00:35:59.520 | that you've coded.

00:36:00.520 | - By the way, I should mention

00:36:02.920 | that I use a Kinesis keyboard,

00:36:05.280 | which has the backspace under the thumb,

00:36:08.000 | and one of the biggest reasons I use that keyboard

00:36:12.120 | is because you realize in order to use the backspace

00:36:15.600 | on a usual keyboard, you have to stretch your pinky out.

00:36:20.560 | And like, for most normal keyboards,

00:36:24.160 | the backspace is under the pinky,

00:36:26.680 | and so I don't know if people realize

00:36:29.440 | the pain they go through in their life

00:36:31.960 | because of the backspace key being so far away.

00:36:35.000 | So with the Kinesis, it's right under the thumb,

00:36:37.480 | so you don't have to actually move your hands,

00:36:39.160 | the backspace and the delete--

00:36:40.560 | - What do you do if you're ever not with your own keyboard

00:36:45.200 | and you have to use someone else's PC keyboard

00:36:48.520 | that has that standard layout?

00:36:50.480 | - So first of all, it turns out

00:36:52.480 | that you can actually go your whole life

00:36:54.400 | always having the keyboard with you.

00:36:56.960 | So this-- - Well, except for that

00:36:59.000 | little tablet that you're using

00:37:00.840 | for note-taking right now, right?

00:37:02.880 | - Yeah, so it's very inefficient note-taking,

00:37:05.040 | but I'm not, I'm just looking stuff up.

00:37:07.600 | But in most cases, I would be actually using

00:37:10.440 | the keyboard here right now.

00:37:13.040 | I just don't anticipate, you have to calculate

00:37:14.920 | how much typing do you anticipate.

00:37:17.120 | If I anticipate quite a bit, then I'll just,

00:37:19.440 | I have a keyboard with me. - You pull it out.

00:37:21.320 | - And same with, I mean, the embarrassing,

00:37:25.840 | I've accepted being the weirdo that I am,

00:37:28.880 | but when I go on an airplane

00:37:31.880 | and I anticipate to do programming or a lot of typing,

00:37:35.440 | I will have a laptop that will pull out a Kinesis keyboard

00:37:39.920 | in addition to the laptop, and it's just who I am.

00:37:43.000 | You have to accept who you are.

00:37:45.880 | But also, it's, for a lot of people,

00:37:49.320 | for me certainly, there's a comfort space

00:37:52.080 | where there's a certain kind of setups

00:37:54.320 | that are maximized productivity.

00:37:57.160 | And it's like some people have a warm blanket

00:38:01.200 | that they like when they watch a movie.

00:38:04.200 | I like the Kinesis keyboard.

00:38:05.480 | It takes me to a place of focus.

00:38:07.920 | And I still mostly, I'm trying to make sure

00:38:12.000 | I use the state-of-the-art IDEs for everything,

00:38:14.760 | but my comfort place, just like the Kinesis keyboard,

00:38:18.240 | is still Emacs.

00:38:19.440 | So I still use, I still, I mean,

00:38:25.920 | that's one of some of the debates I have with myself

00:38:28.920 | about everything from a technology perspective

00:38:31.600 | is how much to hold on to the tools you're comfortable with

00:38:36.120 | versus how much to invest in using modern tools.

00:38:40.280 | And the signal that the communities provide you with

00:38:43.320 | is the noisy one, because a lot of people year to year

00:38:46.360 | get excited about new tools.

00:38:48.320 | And you have to make a prediction.

00:38:50.120 | Are these tools defining a new generation

00:38:53.200 | or something that will transform programming?

00:38:55.480 | Or is this just a fad that will pass?

00:38:58.120 | Certainly with JavaScript frameworks

00:39:00.800 | and front-end and back-end of the web,

00:39:04.160 | there's a lot of different styles that came and went.

00:39:06.820 | I remember learning, what was it called, ActionScript?

00:39:11.440 | I remember for Flash,

00:39:14.640 | learning how to program in Flash,

00:39:16.960 | learning how to design, do graphic animation,

00:39:19.840 | all that kind of stuff in Flash.

00:39:21.360 | Same with Java applets.

00:39:22.600 | I remember creating quite a lot of Java applets,

00:39:25.360 | thinking that this potentially defines the future of the web.

00:39:28.600 | And it did not.

00:39:29.440 | - Well, you know, in most cases like that,

00:39:32.520 | the particular technology eventually gets replaced,

00:39:37.520 | but many of the concepts that the technology introduced

00:39:44.440 | or made accessible first are preserved, of course,

00:39:49.440 | because yeah, we're not using Java applets anymore,

00:39:54.280 | but the notion of reactive webpages

00:39:57.040 | that sort of contain little bits of code

00:40:01.360 | that respond directly to something you do,

00:40:05.880 | like pressing a button or a link or hovering even,

00:40:09.020 | it has certainly not gone away.

00:40:12.600 | And that those animations that were made painfully

00:40:17.600 | complicated with Flash,

00:40:20.920 | I mean, Flash was an innovation when it first came up.

00:40:25.360 | And when it was replaced by JavaScript equivalence stuff,

00:40:30.360 | it was a somewhat better way to do animations,

00:40:35.380 | but those animations are still there.

00:40:37.740 | Not all of them, but sort of,

00:40:42.080 | again, there is an evolution.

00:40:43.800 | And so often with technology,

00:40:47.400 | that the sort of the technology

00:40:49.160 | that was eventually thrown away or replaced

00:40:52.920 | was still essential to sort of get started.

00:40:57.920 | There wouldn't be jet planes without propeller planes.

00:41:01.920 | I bet you.

00:41:03.080 | - But from a user perspective, yes.

00:41:06.000 | From the feature set, yes.

00:41:07.760 | But from a programmer perspective,

00:41:11.200 | it feels like all the time I've spent with ActionScript,

00:41:16.200 | all the time I've spent with Java on the Applet side

00:41:20.580 | for the GUI development,

00:41:22.080 | well, no, Java I have to push back.

00:41:24.000 | That was useful, 'cause it transfers.

00:41:26.760 | But the Flash doesn't transfer.

00:41:28.560 | So some things you learn and invest time in.

00:41:31.320 | - Yeah, what you learned,

00:41:34.160 | the skill you picked up learning ActionScript,

00:41:38.560 | was sort of, it was perhaps a super valuable skill

00:41:43.560 | at the time you picked it up,

00:41:46.320 | if you learned ActionScript early enough.

00:41:50.820 | But that skill is no longer in demand.

00:41:57.140 | - Well, that's the calculation you have to make

00:41:59.380 | when you're learning new things.

00:42:00.460 | Like today people start learning programming.

00:42:02.640 | Today I'm trying to see what are the new languages to try?

00:42:07.520 | What are the new systems to try?

00:42:10.200 | What are the new IDs to try to keep improving?

00:42:14.080 | - That's why we start when we're young, right?

00:42:16.800 | But that seems very true to me,

00:42:21.120 | that when you're young,

00:42:22.400 | you have your whole life ahead of you

00:42:24.040 | and you're allowed to make mistakes.

00:42:27.480 | In fact, you should feel encouraged

00:42:30.280 | to do a bit of stupid stuff.

00:42:33.500 | Try not to get yourself killed or seriously maimed,

00:42:36.720 | but try stuff that deviates

00:42:40.760 | from what everybody else is doing.

00:42:43.280 | And like nine out of 10 times,

00:42:47.200 | you'll just learn why everybody else is not doing that,

00:42:50.680 | or why everybody else is doing it some other way.

00:42:53.600 | And one out of 10 times,

00:42:55.520 | you discover something that's better or that somehow works.

00:43:01.640 | I mean, there are all sorts of crazy things

00:43:03.600 | that were invented by accident,

00:43:07.720 | by people trying stuff together.

00:43:11.160 | - That's great advice to try random stuff,

00:43:13.520 | make a lot of mistakes.

00:43:14.760 | - Once you're married with kids,

00:43:16.300 | you're probably going to be a little more risk averse

00:43:19.960 | because now there's more at stake

00:43:21.600 | and you've already hopefully had some time

00:43:24.480 | where you were experimenting with crazy shit.

00:43:27.640 | - I like how marriage and kids

00:43:29.320 | solidifies your choice of programming language.

00:43:31.880 | How does that, the Robert Frost poem

00:43:33.840 | with the road less taken,

00:43:36.200 | which I think is misinterpreted by most people.

00:43:38.300 | But anyway, I feel like the choices you make early on,

00:43:42.860 | especially if you go all in,

00:43:44.640 | they're going to define the rest of your life's trajectory

00:43:47.400 | in a way that, like you basically are picking a camp.

00:43:51.200 | So, you know, there's, if you invest a lot in PHP,

00:43:55.740 | if you invest a lot in .NET,

00:43:57.640 | if you invest a lot in JavaScript,

00:44:00.640 | you're going to stick there.

00:44:03.620 | That's your life journey.

00:44:07.040 | It's very hard to tell. - Well, only as far

00:44:09.680 | as that technology remains relevant.

00:44:13.920 | - Yes, yes.

00:44:14.760 | - I mean, if at age 16, you learn coding in C

00:44:19.760 | and by the time you're 26, C is like a dead language,

00:44:28.240 | then there's still time to switch.

00:44:31.800 | There's probably some kind of survivor bias

00:44:34.560 | or whatever it's called in sort of your observation

00:44:38.360 | that you pick a camp because there are many

00:44:42.200 | different camps to pick.

00:44:43.640 | And if you pick .NET,

00:44:45.800 | then you can coast for the rest of your life

00:44:49.800 | because that technology is now so ubiquitous, of course,

00:44:53.720 | that it's, even if it's bound to die,

00:44:56.960 | it's going to take a very long time.

00:44:59.040 | - Well, for me personally, I had a very difficult

00:45:04.040 | and in my own head, brave leap that I had to take

00:45:07.480 | relevant to our discussion,

00:45:09.240 | which is most of my life I programmed in C and C++.

00:45:12.960 | And so having that hammer, everything looked like a nail.

00:45:17.960 | So I would literally even do scripting in C++.

00:45:21.880 | Like I would create programs that do script like things.

00:45:25.000 | And when I first came to Google and before then,

00:45:29.120 | it became already, before TensorFlow, before all of that,

00:45:32.640 | there was a growing realization that C++

00:45:35.760 | is not the right tool for machine learning.

00:45:38.480 | We could talk about why that is.

00:45:40.080 | It's unclear why that is.

00:45:41.240 | A lot of things has to do with community and culture

00:45:45.400 | and how it emerges and stuff like that.

00:45:46.920 | But for me to decide to take the leap to Python,

00:45:50.600 | like all out, basically switch completely from C++

00:45:54.080 | except for a highly performant robotics applications.

00:45:58.640 | There was still a culture of C++ in the space of robotics.

00:46:03.640 | That was a big leap.

00:46:05.880 | Like I had to, you know, like people have like

00:46:09.600 | existential crises or midlife crises or whatever.

00:46:13.400 | You have to realize almost like walking away

00:46:15.840 | from a person you love.

00:46:17.660 | 'Cause I was sure that C++ would have to be

00:46:21.400 | a lifelong companion.

00:46:23.080 | For a lot of problems I would wanna solve,

00:46:24.680 | C++ would be there.

00:46:26.200 | And it was a question to say,

00:46:27.440 | well, that might not be the case.

00:46:29.440 | 'Cause C++ is still one of the most popular languages

00:46:32.280 | in the world, one of the most used,

00:46:33.920 | one of the most dependent on.

00:46:35.160 | - It's also still evolving quite a bit.

00:46:38.920 | I mean, that is not a sort of a fossilizing community.

00:46:43.920 | - Yes.

00:46:47.120 | - They are doing great innovative work actually.

00:46:49.680 | - A lot.

00:46:50.520 | - But that sort of their innovations are hard to follow

00:46:53.320 | if you're not already a hardcore C++ user.

00:46:56.560 | - Well, this was the thing.

00:46:57.920 | It pulls you in, it's a rabbit hole.

00:46:59.520 | I was a hardcore.

00:47:00.800 | The old meta programming, template programming.

00:47:03.000 | Like I would start using the modern C++ as it developed.

00:47:07.720 | Right, not just the shared pointer

00:47:10.920 | and the garbage collection.

00:47:12.400 | That makes it easier for you to work with some of the flaws.

00:47:15.560 | But the detail, like the meta programming,

00:47:17.680 | the crazy stuff that's coming out there.

00:47:20.760 | But then you have to just empirically look and step back

00:47:24.600 | and say, what language am I more productive in?

00:47:28.640 | Sorry to say, what language do I enjoy my life with more?

00:47:34.840 | And readability and able to think through

00:47:37.840 | and all that kind of stuff.

00:47:39.080 | Those questions are harder to ask

00:47:41.120 | when you already have a loved one,

00:47:44.240 | which in my case was C++.

00:47:46.400 | And then there's Python, like that meme.

00:47:49.960 | Is the grass is greener on the other side.

00:47:52.520 | Am I just infatuated with a new fad, new cool thing?

00:47:56.800 | Or is this actually going to make my life better?

00:47:59.080 | And I think a lot of people face that kind of decision.

00:48:01.840 | It was a difficult decision for me when I made it.

00:48:06.280 | At this time, it's an obvious switch

00:48:08.000 | if you're into machine learning.

00:48:09.560 | But at that time, it wasn't quite yet so obvious.

00:48:13.400 | So it was a risk.

00:48:14.640 | And you have the same kind of stuff

00:48:16.200 | with I still, because of my connection to WordPress,

00:48:20.920 | I still do a lot of backend programming in PHP.

00:48:24.240 | And the question is, you know, Node.js, Python,

00:48:29.440 | do you switch backend to any of those programmings?

00:48:34.120 | There's the case for Node.js for me.

00:48:36.240 | Well, more and more and more of the front end,

00:48:39.160 | it runs in JavaScript.

00:48:40.560 | And fascinating cool stuff is done in JavaScript.

00:48:45.440 | Maybe use the same programming language

00:48:46.920 | for the backend as well.

00:48:48.240 | The case for Python for the backend is,

00:48:51.400 | well, you're doing so much programming

00:48:54.320 | outside of the web in Python.

00:48:56.440 | So maybe use Python for the backend.

00:48:58.760 | And then the case for PHP,

00:49:00.440 | well, most of the web still runs in PHP.

00:49:04.120 | You have a lot of experience with PHP.

00:49:06.120 | Why fix something that's not broken?

00:49:10.000 | Those are my own personal struggles,

00:49:12.000 | but I think they reflect the struggles of a lot of people

00:49:14.760 | with different programming languages,

00:49:16.120 | with different problems they're trying to solve.

00:49:18.560 | It's a weird one.

00:49:19.640 | - And there's not a single answer, right?

00:49:21.920 | Because depending on how much time

00:49:24.920 | you have to learn new stuff,

00:49:27.240 | where you are in your life,

00:49:28.840 | what you're currently working on,

00:49:31.320 | who you want to work with, what communities you like,

00:49:35.280 | there's not one right choice.

00:49:37.720 | Maybe if you sort of, if you can look back 20 years,

00:49:42.520 | you can say, well, that whole detour through ActionScript

00:49:45.840 | was a waste of time, but nobody could know that.

00:49:50.840 | So you can't beat yourself up over that.

00:49:54.880 | You just need to accept that not every choice you make

00:50:00.800 | is going to be perfect.

00:50:02.200 | Maybe sort of keep a plan B in the back of your mind,

00:50:07.040 | but don't overthink it.

00:50:11.600 | Don't try to sort of,

00:50:13.480 | don't create a spreadsheet with like,

00:50:17.520 | where you're trying to estimate,

00:50:19.840 | well, if I learn this language,

00:50:22.720 | I expect to make X million dollars in a lifetime.

00:50:26.120 | And if I learn that language,

00:50:28.080 | I expect to make Y million dollars in a lifetime.

00:50:31.760 | And which is higher and which has more risk

00:50:35.760 | and where's the chance that,

00:50:37.440 | it's like picking a stock.

00:50:40.600 | - Kind of, kind of, but I think with stocks,

00:50:45.600 | you can do, diversifying your investment is good.

00:50:51.720 | With productivity in life,

00:50:54.280 | boy, that spreadsheet is possible to construct.

00:50:57.880 | Like if you actually carefully analyze

00:51:01.440 | what your interests in life are,

00:51:02.800 | where you think you can maximally impact the world,

00:51:06.780 | there really is better and worse choices

00:51:09.120 | for a programming language.

00:51:10.480 | They're not just about the syntax,

00:51:12.200 | but about the community,

00:51:14.000 | about where you predict the community's headed,

00:51:16.800 | what large systems are programmed in that.

00:51:19.720 | - But can you create that spreadsheet?

00:51:21.760 | Because that's sort of,

00:51:23.280 | you're mentioning a whole bunch of inputs

00:51:25.440 | that go into that spreadsheet

00:51:27.200 | where you have to estimate things

00:51:28.920 | that are very hard to measure and even harder.

00:51:32.360 | I mean, they're hard to measure retroactively

00:51:36.240 | and they're even harder to predict.

00:51:37.920 | Like, what is the better community?

00:51:41.160 | Well, better is one of those incredibly difficult words.

00:51:46.160 | What's better for you is not better for someone else.

00:51:49.080 | - No, but we're not doing a public speech

00:51:50.840 | about what's better.

00:51:51.680 | We're doing a personal spiritual journey.

00:51:54.800 | I can determine a circle of friends,

00:51:57.280 | circle one and circle two,

00:51:59.680 | and I can have a bunch of parties with one

00:52:01.840 | and a bunch of parties with two,

00:52:03.680 | and then write down or take a mental note

00:52:06.680 | of what made me happier, right?

00:52:09.200 | And that, you know, you have,

00:52:11.000 | if you're a machine learning person,

00:52:12.520 | you wanna say, okay, I want to build a large company

00:52:15.400 | that is grounded in machine learning,

00:52:19.080 | but also has a sexy interface

00:52:21.520 | that has a large impact on the world.

00:52:23.320 | What languages do I use?

00:52:25.280 | You look at what Facebook is using,

00:52:26.720 | you look at what Twitter is using.

00:52:28.400 | Then you look at performant, more newer languages like Rust,

00:52:32.840 | or you look at languages that have taken,

00:52:35.960 | that most of the community uses

00:52:37.280 | in machine learning space, that's Python.

00:52:39.600 | And you can like think through,

00:52:40.880 | you can hang out and think through it.

00:52:42.240 | And it's always a invest,

00:52:44.480 | and the level of activity of the community

00:52:47.480 | is also really interesting, like you said,

00:52:48.960 | C++ and Python are super active

00:52:51.760 | in terms of the development of the language itself.

00:52:54.580 | - But do you think that you can make

00:52:57.640 | objective choices there?

00:52:59.520 | - No, no. - No.

00:53:00.960 | - But there's a gut you build up.

00:53:02.520 | Like, don't you believe in that gut feeling about--

00:53:05.000 | - No, everything is very subjective,

00:53:07.160 | and yes, you most certainly can have a gut feeling,

00:53:10.000 | and your gut can also be wrong.

00:53:11.960 | That's why there are billions of people,

00:53:14.120 | because they're not all right.

00:53:16.080 | I mean, clearly there are more people

00:53:18.720 | living in the Bay Area who have plans

00:53:21.200 | to sort of create a Google-sized company

00:53:25.080 | than there's room in the world for Google-sized companies.

00:53:28.560 | And they're gonna have to duke it out in the market space.

00:53:33.200 | And there's many more choices

00:53:35.160 | than just the programming language.

00:53:37.120 | Speaking of which, let's go back to the boat

00:53:40.080 | with the fisherman who's tuned out long ago.

00:53:42.640 | (laughing)

00:53:43.480 | Let's talk to the programmer.

00:53:44.680 | Let's jump around and go back to CPython

00:53:47.360 | that we tried to define as the reference implementation.

00:53:50.440 | And one of the big things that's coming out in 3.11,

00:53:53.880 | what's the right way to--

00:53:54.720 | - We tend to say 3.11, because it really was like,

00:53:58.160 | we went 3.8, 3.9, 3.10, 3.11,

00:54:01.880 | and we're planning to go up to 3.99.

00:54:05.160 | - 99?

00:54:06.160 | What happens after 99?

00:54:07.560 | - Probably just 3.100, if I make it there.

00:54:10.680 | - Okay.

00:54:11.840 | And go all the way to 420.

00:54:13.360 | I got it.

00:54:14.200 | Forever Python v3.

00:54:15.400 | We'll talk about 4, but more for fun.

00:54:18.340 | So, 3.11's coming out.

00:54:22.400 | One of the big, sexy things in it

00:54:24.120 | is it'll be much faster.

00:54:25.640 | So how did you, beyond hiring a great team

00:54:29.840 | or working with a great team, make it faster?

00:54:32.440 | What are some ideas that makes it faster?

00:54:36.200 | - It has to do with simplicity of software versus performance.

00:54:42.240 | And so, even though C is known to be a low-level language,

00:54:47.440 | which is great for writing

00:54:50.400 | sort of a high-performance language interpreter,

00:54:55.200 | when I originally started Python or CPython,

00:55:00.200 | I didn't expect there would be

00:55:03.240 | great success and fame in my future.

00:55:08.120 | So I tried to get something working

00:55:14.720 | and useful in about three months.

00:55:20.280 | And so I sort of, I cut corners.

00:55:25.220 | I borrowed ideas left and right

00:55:28.660 | when it comes to language design, as well as implementation.

00:55:32.260 | I also wrote much of the code as simple as it could be.

00:55:37.740 | And there are many things that you can code

00:55:43.660 | more efficiently by adding more code.

00:55:49.960 | It's a bit of a sort of a time-space trade-off

00:55:54.240 | where you can compute a certain thing

00:55:58.920 | from a small number of inputs.

00:56:01.200 | And every time you get presented with a new input,

00:56:05.880 | you do the whole computation from the top.

00:56:09.320 | That can be simple-looking code.

00:56:12.400 | It's easy to understand.

00:56:13.920 | It's easy to reason about that.

00:56:15.760 | You can tell quickly that it's correct,

00:56:19.000 | at least in the sort of mathematical sense of correct.

00:56:22.360 | Because it's implemented in C,

00:56:26.520 | maybe it performs relatively well.

00:56:29.660 | But over time, as sort of,

00:56:33.740 | as the requirements for that code

00:56:37.520 | and the need for performance go up,

00:56:42.120 | you might be able to rewrite that same algorithm

00:56:46.120 | using more memory, maybe remember previous results

00:56:51.120 | so you don't have to recompute everything from scratch.

00:56:54.800 | Like the classic example is computing prime numbers.

00:56:59.440 | Like, is 10 a prime number?

00:57:03.400 | Well, you sort of, is it divisible by two?

00:57:06.260 | Is it divisible by three?

00:57:07.640 | Is it divisible by four?

00:57:09.600 | And we go all the way to, is it divisible by nine?

00:57:13.520 | And it is not.

00:57:14.600 | Well, actually 10 is divisible by two,

00:57:17.000 | so there we stop, but say 11.

00:57:19.240 | Is it divisible by 10?

00:57:20.720 | The answer is no, 10 times in a row.

00:57:23.400 | So now we know 11 is a prime number.

00:57:25.860 | On the other hand, if we already know

00:57:28.840 | that two, three, five, and seven are prime numbers,

00:57:32.280 | and you know a little bit about the mathematics

00:57:34.480 | of how prime numbers work,

00:57:37.320 | you know that if you have a rough estimate

00:57:39.880 | for the square root of 11,

00:57:42.200 | you don't actually have to check is it divisible by four

00:57:46.160 | or is it divisible by five?

00:57:47.960 | All you have to check in the case of 11

00:57:50.040 | is is it divisible by two?

00:57:51.280 | Is it divisible by three?

00:57:52.800 | Because take 12.

00:57:54.980 | If it's divisible by four,

00:57:58.080 | well, 12 divided by four is three,

00:58:00.040 | so you should have come across the question,

00:58:02.880 | is it divisible by three first?

00:58:04.760 | So if you know basically nothing about prime numbers

00:58:09.160 | except the definition,

00:58:10.420 | maybe you go for x from two through n minus one

00:58:15.420 | is n divisible by x.

00:58:19.360 | And then at the end,

00:58:20.800 | if you got all nos for every single one of those questions,

00:58:25.800 | you know, oh, it must be a prime number.

00:58:29.620 | Well, the first thing is you can stop iterating

00:58:32.780 | when you find a yes answer.

00:58:35.040 | And the second is you can also stop iterating

00:58:37.640 | when you have reached the square root of n,

00:58:42.640 | because you know that if it has a divisor

00:58:45.360 | larger than the square root,

00:58:47.440 | it must also have a divisor smaller than the square root.

00:58:50.640 | Then you say, oh, except for two,

00:58:54.120 | we don't need to bother with checking for even numbers

00:58:56.880 | because all even numbers are divisible by two.

00:58:59.400 | So if it's divisible by four,

00:59:02.240 | we would already have come across the question,

00:59:04.640 | is it divisible by two?

00:59:06.000 | And so now you go special case,

00:59:08.240 | check is it divisible by two?

00:59:09.980 | And then you just check three, five, seven, 11.

00:59:12.840 | And so now you've sort of reduced your search space

00:59:17.320 | by 50% again, by skipping all the even numbers

00:59:20.920 | except for two.

00:59:22.560 | If you think a bit more about it,

00:59:24.920 | or you just read in your book about the history of math,

00:59:29.280 | one of the first algorithms ever written down,

00:59:33.020 | all you have to do is check,

00:59:34.900 | is it divisible by any of the previous prime numbers

00:59:38.260 | that are smaller than the square root?

00:59:41.100 | And before you get to a better algorithm than that,

00:59:45.540 | you have to have several PhDs in discrete math.

00:59:51.800 | So that's as much as I know.

00:59:54.300 | - So of course that same story applies

00:59:56.460 | to a lot of other algorithms.

00:59:57.860 | String matching is a good example

01:00:00.420 | of how to come up with an efficient algorithm.

01:00:03.060 | And sometimes the more efficient algorithm

01:00:05.780 | is not so much more complex than the inefficient one.

01:00:08.820 | But that's an art, and it's not always the case.

01:00:12.580 | In the general cases, the more performant the algorithm,

01:00:16.780 | the more complex it's gonna be.

01:00:18.700 | There's a kind of trade-off.

01:00:20.740 | - The simpler algorithms are also the ones

01:00:23.540 | that people invent first.

01:00:26.820 | Because when you're looking for a solution,

01:00:29.820 | you look at the simplest way to get there first.

01:00:33.300 | And so if there is a simple solution,

01:00:37.220 | even if it's not the best solution,

01:00:39.220 | not the fastest or the most memory efficient or whatever,

01:00:43.340 | a simple solution, and simple is fairly subjective,

01:00:49.500 | but mathematicians have also thought about

01:00:52.820 | sort of what is a good definition for simple

01:00:55.180 | in the case of algorithms.

01:00:58.300 | But the simpler solutions tend to be easier to follow

01:01:03.300 | for other programmers who haven't made a study

01:01:08.180 | of a particular field.

01:01:09.460 | And when I started with Python,

01:01:11.620 | I was a good programmer in general.

01:01:14.540 | I knew sort of basic data structures.

01:01:16.660 | I knew the C language pretty well.

01:01:19.300 | But there were many areas where I was only

01:01:25.460 | somewhat familiar with the state of the art.

01:01:28.660 | And so I picked in many cases,

01:01:34.340 | the simplest way I could solve a particular sub-problem.

01:01:37.540 | Because when you're designing and implementing a language,

01:01:40.580 | you have to like,

01:01:42.460 | you have many hundreds of little problems to solve.

01:01:45.940 | And you have to have solutions for every one of them

01:01:50.140 | before you can sort of say,

01:01:52.540 | I've invented a programming language.

01:01:55.700 | - First of all, so CPython,

01:01:58.420 | what kind of things does it do?

01:02:00.980 | It's an interpreter.

01:02:02.500 | It takes in this readable language that we talked about,

01:02:05.620 | that is Python.

01:02:06.980 | What is it supposed to do?

01:02:08.220 | - The interpreter basically,

01:02:09.820 | it's sort of a recipe for understanding recipes.

01:02:14.820 | So instead of a recipe that says, bake me a cake,

01:02:21.060 | we have a recipe for,

01:02:23.500 | well, given the text of a program,

01:02:27.700 | how do we run that program?

01:02:30.820 | And that is sort of the recipe for building a computer.

01:02:34.580 | - The recipe for the baker and the chef.

01:02:36.500 | - Yeah.

01:02:37.340 | - What are the algorithmically tricky things

01:02:41.380 | that happen to be low-hanging fruit

01:02:44.580 | that could be improved on?

01:02:45.940 | Maybe throw out the history of Python,

01:02:47.620 | but also now, how is it possible that 3.11 in year 2022,

01:02:52.780 | it's possible to get such a big performance improvement?

01:02:55.580 | - We focused on a few areas

01:03:02.100 | where we still felt there was low-hanging fruit.

01:03:06.780 | The biggest one is actually the interpreter itself.

01:03:11.740 | And this has to do with details of how Python is defined.

01:03:16.140 | So I didn't know if the fisherman

01:03:18.540 | is going to follow this story.

01:03:20.740 | - He already jumped off the boat.

01:03:22.660 | He's-

01:03:23.500 | - He's bored.

01:03:25.580 | - Yeah.

01:03:26.420 | - Stupid.

01:03:27.260 | - Python is actually,

01:03:28.340 | even though it's always called an interpreted language,

01:03:31.980 | there's also a compiler in there.

01:03:34.020 | It just doesn't compile to machine code.

01:03:36.060 | It compiles to bytecode,

01:03:38.740 | which is sort of code for an imaginary computer

01:03:43.700 | that is called the Python interpreter.

01:03:45.540 | - So it's compiling code that is more easily digestible

01:03:49.180 | by the interpreter or is digestible at all.

01:03:51.220 | - It is the code that is digested by the interpreter.

01:03:54.300 | That's the compiler.

01:03:55.260 | We tweaked very minor bits of the compiler.

01:03:57.940 | Almost all the work was done in the interpreter

01:04:00.820 | because when you have a program,

01:04:05.660 | you compile it once

01:04:07.060 | and then you run the code a whole bunch of times.

01:04:10.420 | Or maybe there's one function in the code

01:04:13.260 | that gets run many times.

01:04:15.980 | Now I know that sort of people

01:04:19.460 | who know this field are expecting me to,

01:04:22.900 | at some point, say,

01:04:24.660 | we built a just-in-time compiler.

01:04:26.460 | Actually, we didn't.

01:04:27.900 | We just made the interpreter a little more efficient.

01:04:31.980 | - What's a just-in-time compiler?

01:04:34.460 | - That is a thing from the Java world,

01:04:37.740 | although it's now applied to almost all programming languages,

01:04:42.420 | especially interpreted ones.

01:04:44.580 | - So you see the compiler inside Python

01:04:46.660 | not like a just-in-time compiler,

01:04:49.140 | but it's a compiler that creates bytecode

01:04:51.180 | that is then fed to the interpreter.

01:04:54.500 | And the compiler,

01:04:56.660 | was there something interesting to say about the compiler?

01:04:58.660 | It's interesting that you haven't changed that,

01:05:00.060 | tweaked that at all, or much.

01:05:01.980 | - We changed some parts of the bytecode,

01:05:06.100 | but not very much.

01:05:08.100 | And so we only had to change the parts of the compiler

01:05:11.420 | where we decided that the breakdown of a Python program

01:05:15.260 | in bytecode instructions had to be slightly different.

01:05:19.220 | But that didn't gain us the performance improvements.

01:05:24.220 | The performance improvements were

01:05:30.780 | like making the interpreter faster in part

01:05:33.820 | by sort of removing the fat

01:05:38.260 | from some internal data structures used by the interpreter.

01:05:41.660 | But the key idea is an adaptive specializing interface

01:05:46.700 | for an adaptive specializing interpreter.

01:05:49.820 | - Let's go.

01:05:50.660 | What is adaptive about it?

01:05:52.140 | What is specialized about it?

01:05:53.580 | - Well, let me first talk about the specializing part

01:05:56.740 | because the adaptive part is the sort of

01:05:59.340 | the second order effect, but they're both important.

01:06:03.540 | So bytecode is a bunch of machine instructions,

01:06:08.340 | but it's an imaginary machine.

01:06:10.740 | But the machine can do things like call a function,

01:06:14.940 | add two numbers, print a value.

01:06:18.060 | Those are sort of typical instructions in Python.

01:06:21.660 | And if we take the example of adding two numbers,

01:06:28.700 | actually in Python, the language,

01:06:31.460 | there's no such thing as adding two numbers.

01:06:33.980 | There's just, the compiler doesn't know

01:06:38.540 | that you're adding two numbers.

01:06:39.860 | You might as well be adding two strings or two lists

01:06:44.580 | or two instances of some user defined class

01:06:47.900 | that happened to implement this operator called add.

01:06:52.420 | That's a very interesting

01:06:54.300 | and fairly powerful mathematical concept.

01:06:57.620 | It's mostly a user interface trick

01:06:59.860 | because it means that a certain category of functions

01:07:04.860 | can be written using a single symbol, the plus sign,

01:07:10.460 | and sort of a bunch of other functions can be written

01:07:13.620 | using another single symbol, the multiply sign.

01:07:16.460 | So if we take addition, the way traditionally in Python,

01:07:23.020 | the add bytecode was executed is pointers,

01:07:26.940 | pointers, and more pointers.

01:07:31.540 | So first we have two objects.

01:07:34.540 | An object is basically a pointer to a bunch of memory

01:07:37.820 | that contains more pointers.

01:07:39.340 | - Pointers all the way down.

01:07:41.100 | - Well, not quite, but there are a lot of them.

01:07:43.660 | So to simplify a bit, we look up in one of the objects,

01:07:48.660 | what is the type of that object?

01:07:53.260 | And does that object type define an add operation?

01:07:58.260 | And so you can imagine that there is a sort of

01:08:02.780 | a type integer that knows how to add itself

01:08:05.780 | to another integer.

01:08:07.540 | And there is a type floating point number

01:08:09.580 | that knows how to add itself

01:08:11.860 | to another floating point number.

01:08:14.180 | And the integers and floating point numbers

01:08:18.300 | are sort of important, I think, mostly historically,

01:08:22.140 | because in the first computers,

01:08:23.940 | you used the sort of, the same bit pattern

01:08:29.220 | when interpreted as a floating point number

01:08:31.460 | had a very different value

01:08:32.820 | than when interpreted as an integer.

01:08:34.900 | - Can I ask a dumb question here?

01:08:36.460 | - Please do.

01:08:37.300 | - If you take the basics of int and float and add,

01:08:39.940 | who carries the knowledge of how to add two integers?

01:08:44.020 | Is it the integer?

01:08:45.700 | It's the type integer versus?

01:08:47.780 | - It's the type integer and the type float.

01:08:50.060 | - What about the operator?

01:08:51.460 | Does the operator just exist as a platonic form

01:08:56.460 | possessed by the integer?

01:08:59.500 | - The operator is more like,

01:09:04.620 | it's an index in a list of functions

01:09:08.940 | that the integer type defines.

01:09:12.020 | And so the integer type

01:09:13.940 | is really a collection of functions.

01:09:18.580 | And there is an add function

01:09:20.100 | and there's a multiply function

01:09:21.980 | and there are like 30 other functions for other operations.

01:09:25.300 | There's a power function, for example.

01:09:28.060 | And you can imagine that in memory,

01:09:32.900 | there is a distinct slot for the add operations.

01:09:37.100 | Let's say the add operation is the first operation of a type

01:09:40.860 | and the multiply is the second operation of a type.

01:09:44.300 | So now we take the integer type

01:09:46.020 | and we take the floating point type.

01:09:47.980 | In both cases, the add operation is the first slot

01:09:54.140 | and multiply is the second slot.

01:09:56.500 | But each slot contains a function

01:10:00.220 | and the functions are different

01:10:02.540 | because the add to integers function

01:10:07.020 | interprets the bit patterns as integers.

01:10:09.860 | The add to float function

01:10:13.340 | interprets the same bit pattern

01:10:16.340 | as a floating point number.

01:10:19.180 | And then there is the string data type,

01:10:22.340 | which again, interprets the bit pattern

01:10:26.540 | as the address of a sequence of characters.

01:10:31.820 | There are lots of lies in that story,

01:10:33.540 | but that's sort of a basic idea.

01:10:37.060 | - I can tell the fake news

01:10:39.660 | and the fabrication going on here at the table.

01:10:42.180 | But where's the optimization?

01:10:44.380 | Is it on the operators?

01:10:45.420 | Is it different inside the integer?

01:10:47.980 | - The optimization is the observation

01:10:51.300 | that in a particular line of code,

01:10:56.180 | so now you write your little Python program

01:11:01.220 | and you write a function

01:11:02.580 | and that function sort of takes a bunch of inputs

01:11:05.540 | and at some point it adds two of the inputs together.

01:11:09.260 | Now I bet you, even if you call your function a thousand times

01:11:15.020 | that all those calls are likely

01:11:18.180 | all going to be about integers

01:11:21.140 | because maybe your program is all about integers

01:11:24.260 | or maybe on that particular line of code

01:11:28.500 | where there's that plus operator,

01:11:30.660 | every time the program hits that line,

01:11:35.340 | the variables A and B that are being added together

01:11:38.620 | happen to be strings.

01:11:39.980 | And so what we do is instead of having this single byte code

01:11:45.700 | that says, here's an add operation

01:11:48.260 | and the implementation of add is fully generic.

01:11:50.900 | It looks at the object from the object,

01:11:53.300 | it looks at the type, then it takes the type

01:11:56.100 | and it looks up the function pointer,

01:11:59.020 | then it calls the function.

01:12:00.740 | Now the function has to look at the other argument

01:12:04.180 | and it has to double check

01:12:05.180 | that the other argument has the right type.

01:12:07.900 | And then there's a bunch of error checking

01:12:10.060 | before it can actually just go ahead

01:12:13.700 | and add the two bit patterns in the right way.

01:12:16.940 | What we do is every time we execute

01:12:21.820 | an add instruction like that,

01:12:25.020 | we keep a little note of,

01:12:28.420 | in the end, after we hit the code that did the addition

01:12:34.900 | for a particular type, what type was it?

01:12:38.780 | And then after a few times through that code,

01:12:44.340 | if it's the same type all the time,

01:12:47.900 | we say, oh, so this add operation,

01:12:54.380 | even though it's the generic add operation,

01:12:57.460 | it might as well be the add integer operation.

01:13:01.140 | And add integer operation is much more efficient

01:13:05.260 | because it just says, assume that A and B are integers,

01:13:10.260 | do the addition operation, do it right there in line

01:13:13.940 | and produce the result.

01:13:16.020 | And the big lie here is that in Python,

01:13:21.180 | even if you have great evidence that in the past

01:13:25.220 | it was always two integers that you were adding,

01:13:28.420 | at some point in the future, that same line of code

01:13:31.020 | could still be hit with two floating points or two strings,

01:13:33.780 | or maybe a string and an integer.

01:13:35.980 | - It's not a great lie, that's just the fact of life.

01:13:39.300 | - I didn't account for what should happen in that case

01:13:43.780 | in the way I told the story.

01:13:45.980 | - There is some accounting for that.

01:13:48.220 | And so what we actually have to do is

01:13:52.300 | when we have the add integer operation,

01:13:55.700 | we still have to check,

01:13:58.140 | are the two arguments in fact integers?

01:14:01.780 | We applied some tricks to make those checks efficient.

01:14:06.220 | And we know statistically that the outcome is almost always,

01:14:11.220 | yes, they are both integers.

01:14:14.020 | And so we quickly make that check

01:14:17.500 | and then we proceed with the sort of add integer operation.

01:14:21.420 | And then there is a fallback mechanism where we say,

01:14:25.180 | oops, one of them wasn't an integer.

01:14:27.860 | Now we're gonna pretend that it was just

01:14:29.740 | the fully generic add operation.

01:14:32.460 | We wasted a few cycles,

01:14:34.660 | believing it was going to be two integers

01:14:38.700 | and then we had to back up,

01:14:40.780 | but we didn't waste that much time

01:14:42.860 | and statistically most of the time.

01:14:47.060 | Basically we're sort of hoping that most of the time

01:14:51.500 | we guess right,

01:14:52.340 | because if it turns out that we guessed wrong too often

01:14:57.100 | or we didn't have a good guess at all,

01:15:00.260 | things might actually end up running a little slower.

01:15:04.900 | So someone armed with this knowledge

01:15:08.780 | and a copy of the implementation,

01:15:10.740 | someone could easily construct a counter example

01:15:13.780 | where they say, oh, I have a program

01:15:16.220 | and then now it runs five times as slow in Python 3.11

01:15:19.660 | than it did in Python 3.10.

01:15:21.340 | But that's a very unrealistic program.

01:15:24.340 | That's just like an extreme fluke.

01:15:28.340 | - It's a fun reverse engineering task though.

01:15:31.620 | - Oh yeah.

01:15:32.460 | - So there's a...

01:15:33.300 | People like fun, yes.

01:15:38.020 | So there's some presumably heuristic

01:15:42.100 | of what defines a momentum

01:15:44.620 | of saying, you seem to be working adding two integers,

01:15:48.660 | not two generic types.

01:15:50.980 | So how do you figure out that heuristic?

01:15:54.220 | - I think that the heuristic is actually,

01:15:57.180 | we assume that the weather tomorrow

01:15:59.100 | is gonna be the same as the weather today.

01:16:01.260 | - So you don't need two days of the weather?

01:16:03.020 | - No.

01:16:03.860 | (laughing)

01:16:05.100 | That is already so much better than guessing randomly.

01:16:10.100 | - So how do you find this idea?

01:16:13.860 | Hey, I wonder if instead of adding two generic types,

01:16:18.860 | we start assuming that the weather tomorrow

01:16:22.340 | is the same as the weather today.

01:16:24.300 | Where do you find the idea for that?

01:16:27.060 | Because that ultimately, for you to do that,

01:16:30.420 | you have to kind of understand

01:16:31.980 | how people are using the language, right?

01:16:34.660 | - Python is not the first language

01:16:36.420 | to do a thing like this.

01:16:38.180 | This is a fairly well-known trick,

01:16:40.180 | especially from other interpreted languages

01:16:43.820 | that had reason to be sped up.

01:16:47.100 | We occasionally look at papers about HHVM,

01:16:49.940 | which is Facebook's efficient compiler for PHP.

01:16:54.940 | There are tricks known from the JVM,

01:17:00.260 | and sometimes it just comes from academia.

01:17:03.980 | - And so the trick here is that the type itself doesn't,

01:17:06.980 | the variable doesn't know what type it is.

01:17:09.940 | So this is not a statically typed language

01:17:12.300 | where you can afford to have a shortcut to saying it's ints.

01:17:17.300 | - This is a trick that is especially important

01:17:20.420 | for interpreted languages with dynamic typing,

01:17:24.660 | because if the compiler could read in the source

01:17:29.660 | these X and Y that we're adding are integers,

01:17:34.100 | the compiler can just insert a single add machine code

01:17:38.020 | that hardware machine instruction that exists

01:17:42.660 | on every CPU and ditto for floats.

01:17:46.580 | But because in Python,

01:17:48.940 | you don't generally declare the types of your variables.

01:17:53.620 | You don't even declare the existence of your variables.

01:17:57.140 | They just spring into existence when you first assign them,

01:18:01.180 | which is really cool and sort of helps those beginners

01:18:05.140 | because there is less bookkeeping.

01:18:06.820 | They have to learn how to do

01:18:08.980 | before they can start playing around with code,

01:18:12.380 | but it makes the interpretation of the code less efficient.

01:18:17.380 | And so we're sort of trying to make the interpretation

01:18:22.580 | more efficient without losing

01:18:27.100 | the super dynamic nature of the language.

01:18:30.460 | That's always the challenge.

01:18:31.980 | - 3.5 got the PEP 44 type hints.

01:18:36.660 | What is type hinting and is it used by the interpreter,

01:18:41.660 | the hints, or is it just syntactic sugar?

01:18:44.500 | - So the type hints is an optional mechanism

01:18:48.380 | that people can use.

01:18:50.460 | And it's especially popular with sort of larger companies

01:18:55.180 | that have very large code bases written in Python.

01:18:58.620 | - Do you think of it as almost like documentation

01:19:00.660 | saying these two variables are this type?

01:19:02.580 | - More than documentation.

01:19:04.380 | I mean, so it is a sub-language of Python

01:19:09.380 | where you can express the types of variables.

01:19:13.420 | So here is a variable and it's an integer.

01:19:16.180 | And here's an argument to this function and it's a string.

01:19:18.940 | And here is a function that returns a list of strings.

01:19:22.580 | - But that's not checked when you run the code.

01:19:24.260 | - But exactly.

01:19:26.220 | There is a separate piece of software

01:19:28.940 | called a static type checker that reads all your source code

01:19:32.700 | without executing it and thinks long and hard

01:19:36.660 | about what it looks from just reading the code

01:19:41.420 | that code might be doing and double checks

01:19:46.140 | if that makes sense if you take the types

01:19:49.620 | as annotated into account.

01:19:51.620 | - So this is something you're supposed to run

01:19:53.220 | as you develop.

01:19:54.100 | - It's like a linter, yeah.

01:19:56.260 | That's definitely a development tool,

01:19:58.420 | but the type annotations currently are not used

01:20:02.380 | for speeding up the interpreter.

01:20:05.820 | And there are a number of reasons.

01:20:08.380 | Many people don't use them.

01:20:11.780 | Even when they do use them, they sometimes contain lies

01:20:16.780 | where the static type checker says, everything's fine.

01:20:22.220 | I cannot prove that this integer is ever not an integer,

01:20:26.740 | but at runtime, somehow someone manages

01:20:29.940 | to violate that assumption.

01:20:32.980 | And the interpreter ends up doing just fine.

01:20:36.700 | If we started enforcing type annotations in Python,

01:20:41.500 | many Python programs would no longer work.

01:20:45.180 | And some Python programs wouldn't even be possible

01:20:47.980 | because they're too dynamic.

01:20:50.140 | And so we made a choice of not using the annotations.

01:20:54.460 | There is a possible future where eventually

01:20:58.980 | three, four, five releases in the future,

01:21:03.220 | we could start using those annotations

01:21:05.740 | to sort of provide hints because we can still say,

01:21:10.740 | well, the source code leads us to believe

01:21:15.780 | that these X and Y are both integers.

01:21:18.340 | And so we can generate an add integer instruction,

01:21:23.220 | but we can still have a fallback that says,

01:21:26.860 | oh, if somehow the code at runtime provided something else,

01:21:31.860 | maybe it provided two decimal numbers,

01:21:35.980 | we can still use that generic add operation as a fallback,

01:21:40.540 | but we're not there.

01:21:41.740 | - Is there currently a mechanism

01:21:43.780 | or do you see something like that

01:21:46.300 | where you can almost add like an assert

01:21:48.300 | inside a function that says,

01:21:51.900 | please check that my type hints

01:21:54.660 | are actually mapping to reality?

01:21:56.940 | Sort of like insert manual static typing.

01:22:00.460 | - There are third-party libraries that are in that business.

01:22:05.460 | - Is it possible to do that kind of thing?

01:22:06.860 | Is it possible for a third-party library to take a hint

01:22:10.380 | and enforce it?

01:22:12.580 | It seems like a tricky thing.

01:22:14.100 | - Well, what we actually do is,

01:22:16.500 | I think this is a fairly unique feature in Python.

01:22:20.020 | The type hints can be introspected at runtime.

01:22:24.780 | So while the program is running,

01:22:27.420 | they mean Python is a very introspectable language.

01:22:32.220 | You can look at a variable and ask yourself,

01:22:34.620 | what is the type of this variable?

01:22:37.620 | And if that variable happens to refer to a function,

01:22:41.900 | you can ask, what are the arguments to the function?

01:22:45.700 | And nowadays you can also ask,

01:22:48.220 | what are the type annotations for the function?

01:22:50.820 | - So the type annotations are there inside the variable

01:22:53.700 | as it's at runtime.

01:22:55.660 | - They're mostly associated with the function object,

01:22:58.500 | not with each individual variable,

01:23:00.460 | but you can sort of map from the arguments to the variables.

01:23:05.460 | - And that's what a third-party library can help with.

01:23:07.700 | - Exactly.

01:23:08.540 | And the problem with that is that

01:23:10.540 | all that extra runtime type checking

01:23:12.940 | is going to slow your code down instead of speed it up.

01:23:17.660 | - I think to reference this sales pitchy blog post

01:23:22.660 | that says 75% of developers' time is spent on debugging,

01:23:27.140 | I would say that in some cases that might be okay.

01:23:29.900 | It might be okay to pay the cost of performance

01:23:32.940 | for the catching of the type errors.

01:23:36.140 | - And in most cases,

01:23:38.420 | doing it statically before you ship your code to production

01:23:45.060 | is more efficient than doing it at runtime piecemeal.

01:23:49.140 | - Yeah.

01:23:50.140 | Can you tell me about MYPY project?

01:23:55.140 | What is it?

01:23:57.460 | What's the mission?

01:23:59.260 | And in general,

01:24:00.100 | what is the future of static typing in Python?

01:24:04.040 | - Well, so MYPY was started by a Finnish developer,

01:24:09.040 | Jukka Lätusalo.

01:24:11.700 | - So many cool things out of Finland, I gotta say.

01:24:14.420 | - Just that part of the world.

01:24:15.660 | - I guess people have nothing better to do

01:24:17.420 | in those long, cold winters.

01:24:19.540 | I don't know, I think Jukka lived in England

01:24:22.780 | when he invented that stuff, actually.

01:24:25.380 | But MYPY is the original static type checker for Python.

01:24:30.380 | And the type annotations that were introduced

01:24:34.660 | with PEP484 were sort of developed

01:24:39.180 | together with the static type checker.

01:24:43.540 | And in fact, Jukka had first invented a different syntax

01:24:47.300 | that wasn't quite compatible with Python.

01:24:50.600 | And Jukka and I sort of met at a Python conference

01:24:55.600 | in, I think in 2013.

01:24:58.140 | And we sort of came up with a compromise syntax

01:25:04.140 | that would not require any changes to Python.

01:25:09.900 | And that would let MYPY sort of be an add-on

01:25:13.940 | static type checker for Python.

01:25:15.860 | - Just out of curiosity, was it like double colon

01:25:17.940 | or something, what was he proposing that would break Python?

01:25:21.340 | - I think he was using angular brackets for types

01:25:25.440 | like in C++ or Java generics.

01:25:29.020 | - Yeah, you can't use angular brackets in Python.

01:25:31.900 | It would be too tricky for template type stuff.

01:25:34.340 | - Well, the key thing is that we already had

01:25:38.540 | a syntax for annotations.

01:25:41.780 | We just didn't know what to use them for yet.

01:25:45.260 | So type annotations were just the sort of most logical thing

01:25:50.100 | to use that existing dummy syntax for.

01:25:54.280 | So there was no syntax for defining generics

01:25:59.280 | directly syntactically in the language.

01:26:04.420 | MYPY literally meant my version of Python,

01:26:08.100 | where my refers to Yuka.

01:26:10.020 | He had a parser that translated MYPY into Python

01:26:15.940 | by like doing the type checks

01:26:20.500 | and then removing the annotations

01:26:24.040 | and all the angular brackets from the positions

01:26:27.580 | where he was using them.

01:26:29.340 | But a pre-processor model doesn't work very well

01:26:33.420 | with the typical workflow of Python development projects.

01:26:37.980 | - That's funny.

01:26:38.820 | I mean, that could have been another major split

01:26:41.140 | if it became successful.

01:26:42.980 | Like if you watch TypeScript versus JavaScript,

01:26:46.900 | it's like a split in the community over types, right?

01:26:51.020 | That seems to be stabilizing now.

01:26:53.140 | - It's not necessarily a split.

01:26:54.900 | There are certainly plenty of people

01:26:56.780 | who don't use TypeScript,

01:26:59.740 | but just use the original JavaScript notation,

01:27:04.580 | just like there are many people in the Python world

01:27:07.260 | who don't use type annotations

01:27:08.980 | and don't use static type checkers.

01:27:11.220 | - No, I know, but there is a bit of a split

01:27:12.940 | between TypeScript and old school JavaScript,

01:27:15.980 | ES, whatever.

01:27:17.300 | - Well, in the JavaScript world,

01:27:19.420 | transpilers are sort of the standard way of working anyway,

01:27:23.700 | which is why TypeScript being a transpiler itself

01:27:27.100 | is not a big deal.

01:27:28.940 | - And transpilers for people who don't know,

01:27:30.500 | it's exactly the thing you said with MYPY,

01:27:33.300 | it's the code, I guess you call it pre-processing code

01:27:36.860 | that translates from one language to the other.

01:27:38.700 | And that's part of the culture,

01:27:40.020 | part of the workflow of the JavaScript community.

01:27:43.820 | - That's right.

01:27:44.660 | At the same time, an interesting development

01:27:47.940 | in the JavaScript/TypeScript world at the moment

01:27:51.380 | is that there is a proposal under consideration,

01:27:55.580 | it's only a stage one proposal,

01:27:58.220 | that proposes to add a feature to JavaScript

01:28:01.580 | where just like Python,

01:28:03.900 | it will ignore certain syntax

01:28:07.140 | when running the JavaScript code.

01:28:12.100 | And what it ignores is more or less a superset

01:28:16.220 | of the TypeScript annotation syntax.

01:28:19.420 | - Interesting.

01:28:21.660 | - So that would mean that eventually, if you wanted to,

01:28:25.900 | you could take TypeScript

01:28:27.500 | and you could shove it directly

01:28:31.180 | into a JavaScript interpreter without transpilation.

01:28:36.020 | The interesting thing in the JavaScript world,

01:28:38.500 | at least the web browser world,

01:28:40.620 | the web browsers have changed how they deploy

01:28:43.660 | and they sort of update their JavaScript engines

01:28:48.660 | much more quickly than they used to in the early days.

01:28:53.340 | And so there's much less of a need

01:28:55.380 | for transpilation in JavaScript itself,

01:28:59.700 | because most browsers just support

01:29:02.060 | the most recent version of ECMAScript.

01:29:05.420 | - Just on a tangent of a tangent,

01:29:07.380 | do you see, if you were to recommend somebody use a thing,

01:29:11.180 | would you recommend TypeScript or JavaScript?

01:29:14.300 | - I would recommend TypeScript.

01:29:16.940 | - Just because of the strictness of the typing?

01:29:19.420 | - It's an enormously helpful extra tool

01:29:23.260 | that helps you sort of keep your head straight

01:29:28.100 | about what your code is actually doing.

01:29:31.100 | I mean, it helps with editing your code.

01:29:36.500 | It helps with ensuring that your code is not too incorrect.

01:29:41.500 | And it's actually quite compatible with JavaScript,

01:29:47.140 | nevermind this syntactic sort of hack

01:29:50.620 | that is still years in the future.

01:29:52.980 | But any library that is written in pure JavaScript

01:29:56.700 | can still be used from TypeScript programs.

01:30:00.100 | And also the other way around,

01:30:01.780 | you can write a library in TypeScript

01:30:05.020 | and then export it in a form

01:30:06.900 | that is totally consumable by JavaScript.

01:30:10.580 | That sort of compatibility is sort of the key

01:30:14.420 | to the success of TypeScript.

01:30:17.460 | - Yeah, just to look at it,

01:30:19.140 | it's almost like a biological system that's evolving.

01:30:21.540 | It's fascinating to see JavaScript evolve the way it does.

01:30:24.540 | - Well, maybe we should consider that biological systems

01:30:27.340 | are just engineering systems too, right?

01:30:30.260 | - Yes.

01:30:31.100 | - Just very advanced with more history.

01:30:35.380 | - But it's almost like the most visceral

01:30:38.580 | in the JavaScript world

01:30:39.900 | because there's just so much code written in JavaScript

01:30:44.500 | that for its history was messy.

01:30:48.300 | If you're talking about bugs per line of code,

01:30:50.420 | I just feel like JavaScript eats the cake

01:30:53.700 | or whatever the terminology is.

01:30:55.340 | It beats Python by a lot in terms of number of bugs,

01:30:58.380 | meaning like way more bugs in JavaScript.

01:31:00.940 | And then obviously the browsers are developed.

01:31:05.500 | I mean, just there's so much active development.

01:31:07.420 | It feels a lot more like evolution

01:31:10.100 | where a bunch of stuff is born and dies

01:31:12.100 | and there's experimentation and debates

01:31:15.060 | versus Python is more, all that stuff is happening,

01:31:19.940 | but there's just a longer history

01:31:21.700 | of stable working giant software systems written in Python

01:31:26.100 | versus JavaScript is just a giant, beautiful,

01:31:29.580 | I would say, mess of code.

01:31:31.420 | - It's a very different culture.

01:31:33.140 | And to some extent, differences in culture are random,

01:31:37.380 | but to some extent,

01:31:39.100 | the differences have to do with the environment.

01:31:41.820 | And the fact that JavaScript is primarily

01:31:48.620 | the language for developing web applications,

01:31:53.420 | especially the client side,

01:31:55.700 | and the fact that it's basically the only language

01:31:59.460 | for developing web applications

01:32:02.260 | makes that community sort of just have a different nature

01:32:06.380 | than the community of other languages.

01:32:08.940 | - Plus the graphical component

01:32:12.540 | and the fact that they're deploying it

01:32:16.260 | on all kinds of shapes of screens and devices

01:32:19.580 | and all that kind of stuff,

01:32:20.420 | it just creates a beautiful chaos.

01:32:22.820 | Anyway, back to MyPy.

01:32:24.620 | So what, okay, you met,

01:32:26.260 | you talked about a syntax that could work.

01:32:29.500 | Where does it currently stand?

01:32:31.620 | What's the future of static typing in Python?

01:32:33.900 | - It is still controversial,

01:32:36.780 | but it is much more accepted

01:32:39.180 | than when MyPy and PEP484 were young.

01:32:43.380 | - What's the connection between PEP484 type hints and MyPy?

01:32:48.380 | - MyPy was the original static type checker.

01:32:53.140 | So MyPy quickly evolved from Yuka's own variant of Python

01:32:58.140 | to a static type checker for Python

01:33:02.460 | and sort of PEP484, that was it like

01:33:06.380 | a very productive year where like many hundreds of messages

01:33:12.380 | were exchanged debating the merits

01:33:15.780 | of every aspect of that PEP.

01:33:18.380 | And so MyPy is a static type checker for Python.

01:33:23.420 | It is itself written in Python.

01:33:27.100 | Most additional static typing features

01:33:31.860 | that we introduced in the time since 3.6

01:33:35.420 | were also prototyped through MyPy.

01:33:41.300 | MyPy being an open source project

01:33:44.380 | with a very small number of maintainers

01:33:46.860 | was successful enough that people said

01:33:50.820 | this static type checking stuff for Python

01:33:53.380 | is actually worth an investment for our company.

01:33:57.180 | - Nice.

01:33:58.020 | - But somehow they chose not to support

01:34:02.140 | making MyPy faster say, or adding new features to MyPy,

01:34:09.540 | but both Google and Facebook and later Microsoft

01:34:14.540 | developed their own static type checker.

01:34:17.100 | I think Facebook was one of the first,

01:34:20.300 | they decided that they wanted to use the same technology

01:34:25.100 | that they had successfully used for HHVM

01:34:28.900 | because they sort of, they had a bunch of compiler writers

01:34:35.180 | and sort of static type checking experts

01:34:38.740 | who had written the HHVM compiler

01:34:42.020 | and it was big success within the company.

01:34:44.980 | And they had done it in a certain way, sort of.

01:34:47.580 | They wrote a big, highly parallel application

01:34:53.220 | in an obscure language named OCaml,

01:34:56.340 | which is apparently mostly very good

01:34:58.140 | for writing static type checkers.

01:35:01.100 | - Interesting.

01:35:01.940 | I have a lot of questions about

01:35:04.540 | how to write a static type checker then.

01:35:06.900 | That's very confusing.

01:35:07.940 | - Facebook wrote their version

01:35:10.340 | and they worked on it in secret for about a year

01:35:13.740 | and then they came clean and went open source.

01:35:16.780 | Google in the meantime was developing

01:35:20.380 | something called PyType, which was mostly interesting

01:35:23.860 | because it, as you may have heard,

01:35:28.020 | they have one gigantic mono repo.

01:35:31.140 | So all the code is checked into a single repository.

01:35:35.820 | Facebook has a different approach.

01:35:37.340 | So Facebook developed Pyre, which was written in OCaml,

01:35:42.220 | which worked well with Facebook's development workflow.

01:35:46.380 | Google developed something they called PyType,

01:35:51.060 | which was actually itself written in Python.

01:35:53.500 | And it was meant to sort of fit well

01:35:58.220 | in their static type checking needs

01:36:01.860 | in Google's gigantic mono repo.

01:36:05.300 | - So Google has one giant, got it.

01:36:07.820 | So just to clarify,

01:36:10.460 | this static type checker philosophically

01:36:14.020 | is a thing that's supposed to exist

01:36:15.420 | outside of the language itself.

01:36:17.260 | And it's just a workflow, like a debugger for the programmers.

01:36:20.220 | - It's a linter.

01:36:21.140 | - For people who don't know, a linter,

01:36:22.940 | maybe you can correct me.

01:36:24.540 | But it's a thing that runs through the code continuously,

01:36:28.380 | pre-processing to find issues based on style, documentation.

01:36:34.100 | I mean, there's all kinds of linters, right?

01:36:36.140 | It can check that, what usual things does a linter do?

01:36:39.660 | Maybe check that you haven't too many characters

01:36:44.180 | in a single line.

01:36:45.740 | - Linters often do static analysis

01:36:48.660 | where they try to point out things that are likely mistakes,

01:36:52.740 | but not incorrect according to the language specification.

01:36:57.260 | Like maybe you have a variable that you never use.

01:37:01.500 | For the compiler, that is valid.

01:37:04.300 | You might be planning to use it

01:37:07.500 | in a future version of the code

01:37:10.100 | and the compiler might just optimize it out,

01:37:12.740 | but the compiler is not gonna tell you,

01:37:14.140 | "Hey, you're never using this variable."

01:37:16.460 | A linter will tell you that variable is not used.

01:37:20.220 | Maybe there's a typo somewhere else

01:37:22.980 | where you meant to use it,

01:37:24.340 | but you accidentally use something else,

01:37:26.460 | or there are a number of sort of common scenarios.

01:37:29.900 | And a linter is often a big collection of little heuristics

01:37:34.900 | where by looking at the combination

01:37:39.980 | of how your code is laid out, maybe how it's indented,

01:37:42.940 | maybe the comment structure,

01:37:44.980 | but also just things like definition of names, use of names,

01:37:51.260 | it'll tell you likely things that are wrong.

01:37:56.460 | And in some cases, linters are really style checkers.

01:38:00.940 | For Python, there are a number of linters

01:38:03.900 | that check things like,

01:38:06.020 | do you use the PEP-8 recommended naming scheme

01:38:11.020 | for your functions and classes and variables?

01:38:14.340 | Because like classes start with an uppercase

01:38:16.660 | and the rest starts with a lowercase

01:38:18.420 | and there's like differences there.

01:38:21.420 | And so the linter can tell you,

01:38:22.780 | "Hey, you have a class that,

01:38:25.020 | "whose first letter is not an uppercase letter."

01:38:29.260 | And that's just, I just find it annoying

01:38:31.580 | if I wanted that to be an uppercase letter,

01:38:33.780 | I would have typed an uppercase letter,

01:38:36.260 | but other people find it very comforting

01:38:38.940 | that if the linter is no longer complaining about their code

01:38:43.140 | that they have followed all the style rules.

01:38:46.020 | - Maybe it's a fast way for a new developer

01:38:48.060 | joining a team to learn the style rules, right?

01:38:50.100 | - Yeah, there's definitely that.

01:38:51.700 | But the best use of a linter is probably

01:38:55.580 | not so much to sort of enforce team uniformity,

01:39:00.580 | but to actually help developers catch bugs

01:39:05.780 | that the compilers for whatever reason don't catch.

01:39:09.660 | And there's lots of that in Python.

01:39:12.100 | And so, but a static type checker

01:39:15.340 | focuses on a particular aspect of the linting,

01:39:19.940 | which, I mean, MyPy doesn't care

01:39:23.140 | how you name your classes and variables,

01:39:25.460 | but it is meticulous about when you say

01:39:30.620 | that there was an integer here

01:39:32.700 | and you're passing a string there,

01:39:34.740 | it will tell you, "Hey, that string is not an integer."

01:39:37.500 | So something's wrong.

01:39:38.540 | Either you were incorrect when you said it was an integer

01:39:42.980 | or you're incorrect when you're passing it a string.

01:39:45.780 | - If this is a race of static type checkers,

01:39:48.540 | is somebody winning?

01:39:49.820 | As you said, it's interesting that the companies

01:39:51.660 | didn't choose to invest in this centralized development

01:39:56.660 | of MyPy.

01:39:58.060 | Is there a future for MyPy?

01:40:01.860 | What do you see as the,

01:40:03.580 | will one of the companies win out

01:40:05.180 | and everybody uses like PyType,

01:40:09.140 | whatever Google's is called?

01:40:10.980 | - Well, Microsoft is hoping that Microsoft's horse

01:40:15.020 | in that race called PyWrite is going to win.

01:40:18.660 | - PyWrite, like R-I-G-H-T?

01:40:22.020 | - Correct.

01:40:22.940 | Yeah, all my word processors tend to typo correct

01:40:27.940 | that as PyWrite, the name of the,

01:40:30.740 | I don't know what it is,

01:40:31.900 | some kind of semi-precious metal.

01:40:35.420 | - Oh, right.

01:40:36.260 | (laughing)

01:40:37.540 | I love it.

01:40:38.380 | Okay, so, okay, that's the Microsoft hope,

01:40:41.140 | but, okay, so let me ask the question a different way.

01:40:44.540 | Is there going to be ever a future

01:40:46.540 | where the static type checker gets integrated

01:40:48.860 | into the language?

01:40:49.860 | - Nobody is currently excited about

01:40:57.180 | doing any work towards that.

01:40:59.180 | That doesn't mean that five or 10 years from now,

01:41:02.780 | the situation isn't different.

01:41:06.180 | At the moment, all the static type checkers

01:41:14.060 | still evolve at a much higher speed

01:41:18.260 | than Python and its annotation syntax evolve.

01:41:22.940 | You get a new release of Python once a year.

01:41:26.580 | Those are the only times that you can introduce

01:41:29.100 | new annotation syntax.

01:41:32.220 | And there are always people who invent new annotation syntax

01:41:36.540 | that they're trying to push.

01:41:39.860 | And worse, once we've all agreed

01:41:43.140 | that we are going to put some new syntax in,

01:41:46.460 | we can never take it back.

01:41:48.700 | At least the sort of deprecating an existing feature

01:41:51.860 | takes many releases,

01:41:53.180 | because you have to assume that people started using it

01:41:56.340 | as soon as we announced it.

01:41:58.580 | And then you can't take it away from them right away.

01:42:01.500 | You have to start telling them,

01:42:03.500 | "Well, this will go away,

01:42:05.140 | "but we're not gonna tell you that it's an error yet."

01:42:09.140 | And then later it's gonna be a warning,

01:42:11.300 | and then eventually three releases in the future,

01:42:13.780 | maybe we remove it.

01:42:15.180 | On the other hand, the typical static type checker

01:42:19.500 | still has a release like

01:42:22.820 | every month, every two months,

01:42:27.620 | certainly many times a year.

01:42:29.660 | Some type checkers also include a bunch of

01:42:35.380 | experimental ideas that aren't official

01:42:38.340 | standard Python syntax yet.

01:42:41.060 | The static type checkers also just get better

01:42:45.860 | at discovering things that sort of are unspecified

01:42:50.860 | by the language, but that sort of could make sense.

01:42:53.780 | And so each static type checker actually has

01:42:57.540 | its sort of strong and weak points.

01:43:00.100 | - So it's cool, it's like a laboratory of experiments.

01:43:02.340 | - Yeah.

01:43:03.180 | - Microsoft, Google, and all, and you get to see.

01:43:05.060 | - And you see that everywhere, right?

01:43:06.620 | Because there's not one single JavaScript engine either.

01:43:11.620 | There is one in Chrome, there is one in Safari,

01:43:14.220 | there is one in Firefox.

01:43:15.940 | - But that said, you said there's not interest,

01:43:19.220 | I think there is a lot of interest in type hinting, right?

01:43:22.820 | In the PEP 484.

01:43:26.340 | Actually, how many people use that?

01:43:28.020 | Do you have a sense?

01:43:29.260 | How many people use, 'cause it's optional, it's a sugar.

01:43:32.940 | - I can't put a number on it,

01:43:35.460 | but from the number of packages

01:43:38.740 | that do interesting things with it at runtime,

01:43:41.740 | and the fact that there are like now three or four

01:43:46.180 | very mature type checkers that each have

01:43:50.020 | their segment of the market,

01:43:52.380 | and oh, and then there is PyCharm,

01:43:54.540 | which has a sort of more heuristic-based type checker

01:43:57.420 | that also supports the same syntax.

01:44:00.580 | My assumption is that many, many people

01:44:05.060 | developing Python software professionally

01:44:08.660 | for some kind of production situation

01:44:12.500 | are using a static type checker,

01:44:15.100 | especially anybody who has a continuous integration cycle

01:44:20.100 | probably has one of the steps in their testing routine

01:44:27.140 | that happens for basically every commit

01:44:30.340 | is run a static type checker.

01:44:34.300 | And in most cases, that will be MyPy.

01:44:37.260 | So I think it's pretty popular topic.

01:44:41.780 | - According to this webpage,

01:44:44.580 | 20 to 30% of Python 3 codebases are using type hints.

01:44:50.620 | - Wow, I wonder how they measured that.

01:44:53.140 | Did they just scan all of GitHub?

01:44:55.660 | - Yeah, that's what it looks like.

01:44:57.540 | They did a quick, not all of, but like a random sampling.

01:45:01.420 | So you mentioned PyCharm.

01:45:04.940 | Let me ask you the big subjective question.

01:45:07.740 | What's the best IDE for Python?

01:45:13.220 | And you're extremely biased now that you're with Microsoft.

01:45:16.380 | Is it PyCharm, VS Code, Vim, or Emacs?

01:45:21.340 | - Historically, I actually started out with using Vim,

01:45:26.340 | but when it was still called VI.

01:45:28.540 | For a very long time, I think from the early 80s to,

01:45:34.900 | I'd say two years ago, I was Emacs user.

01:45:41.820 | - Nice.

01:45:42.860 | - Between, I'd say 2013 and 2018,

01:45:48.740 | I dabbled with PyCharm,

01:45:51.620 | mostly because it had a couple of features.

01:45:56.660 | I mean, PyCharm is like driving an 18-wheeler truck,

01:46:01.660 | whereas Emacs is more like driving

01:46:08.420 | your comfortable Toyota car

01:46:11.540 | that you've had for 100,000 miles

01:46:15.540 | and you know what every little rattle of the car means.

01:46:19.300 | I was very comfortable in Emacs,

01:46:21.820 | but there were certain things it couldn't do.

01:46:23.540 | It wasn't very good at that sort of,

01:46:26.380 | at least the way I had configured it.

01:46:29.140 | I didn't have very good tooling in Emacs

01:46:33.300 | for finding a definition of a function.

01:46:35.500 | - Got it.

01:46:37.540 | - When I was at Dropbox exploring

01:46:40.500 | a five million line Python code base,

01:46:44.460 | just grabbing all that code for where is there a class,

01:46:49.460 | foobar, well, it turns out that if you grab

01:46:52.260 | all five million lines of code,

01:46:54.460 | there are many classes with the same name.

01:46:56.700 | And so PyCharm sort of, once you fired it up

01:47:02.260 | and once it's indexed, your repository was very helpful.

01:47:07.260 | But as soon as I had to edit code,

01:47:10.540 | I would jump back to Emacs and do all my editing there

01:47:14.020 | because I could type much faster and switch between files

01:47:18.180 | when I knew which file I wanted much quicker.

01:47:21.700 | And I never really got used

01:47:23.100 | to the whole PyCharm user interface.

01:47:26.900 | - Yeah, I feel torn in that same kind of way

01:47:29.140 | 'cause I've used PyCharm off and on

01:47:31.540 | exactly in that same way.

01:47:33.540 | And I feel like I'm just being an old grumpy man

01:47:37.700 | for not learning how to quickly switch between files

01:47:40.340 | and all that kind of stuff.

01:47:41.180 | I feel like that has to do with shortcuts,

01:47:42.580 | that has to do with, I mean, you just have to get accustomed

01:47:45.500 | just like with touch typing.

01:47:46.700 | - Yeah, you have to just want to learn that.

01:47:49.100 | I mean, if you don't need it much.

01:47:51.140 | - You don't need touch typing either.

01:47:53.340 | You can type with two fingers just fine in the short term,

01:47:56.180 | but in the long term, your life will become better

01:47:59.780 | psychologically and productivity wise

01:48:01.900 | if you learn how to type with 10 fingers.

01:48:03.980 | - If you do a lot of keyboard input.

01:48:06.820 | - But for everyone, emails and stuff, right?

01:48:09.260 | Like you look at the next 20, 30 years of your life,

01:48:13.380 | you have to anticipate where technology is going.

01:48:15.820 | Do you want to invest in handwriting notes?

01:48:19.900 | Probably not.

01:48:20.860 | More and more people are doing typing

01:48:22.860 | versus handwriting notes.

01:48:24.700 | So you can anticipate that.

01:48:26.460 | So there's no reason to actually practice handwriting.

01:48:28.700 | There's more reason to practice typing.

01:48:30.660 | You can actually estimate, back to the spreadsheet,

01:48:33.980 | the number of paragraphs, sentences, or words you write

01:48:39.100 | for the rest of your life.

01:48:40.460 | (laughing)

01:48:42.580 | You can probably--

01:48:43.420 | - You go again with the spreadsheet of my life, huh?

01:48:45.820 | - Yes.

01:48:47.140 | All of that is not actual, like converted to a spreadsheet,

01:48:50.140 | but it's a gut feeling.

01:48:51.700 | Like I have the same kind of gut feeling about books.

01:48:54.580 | I've almost exclusively switched to Kindle now,

01:48:57.020 | so ebook readers, even though I still love

01:49:01.380 | and probably always will the smell,

01:49:03.380 | the feel of a physical book.

01:49:05.820 | And the reason I switched to Kindle is like,

01:49:09.700 | all right, well, this is really paving,

01:49:12.180 | the future is going to be digital

01:49:15.180 | in terms of consuming books and content of that nature.

01:49:18.820 | So you should let your brain get accustomed

01:49:21.980 | to that experience.

01:49:23.700 | In that same way, it feels like PyCharm or VS Code.

01:49:27.180 | I think PyCharm is the most sort of sophisticated,

01:49:31.540 | featureful Python ID.

01:49:35.060 | It feels like I should probably at some point very soon,

01:49:38.940 | switch entire, like I'm not allowed to use anything else

01:49:42.140 | for Python than this ID or VS Code.

01:49:45.220 | It doesn't matter, but walk away from Emacs

01:49:47.300 | for this particular application.

01:49:49.140 | So I think I'm limiting myself in the same way

01:49:51.980 | that using two fingers for typing is limiting myself.

01:49:54.660 | This is a therapy session.

01:49:57.100 | This is not, I'm not even asking a question.

01:49:58.420 | (laughing)

01:50:00.100 | But I'm sure a lot of people are thinking this way, right?

01:50:00.940 | - I'm not gonna stop you.

01:50:04.740 | - I think that sort of everybody has to decide

01:50:07.940 | for themselves which one they want to invest more time in.

01:50:12.420 | I actually ended up giving VS Code a very tentative try

01:50:18.660 | when I started out at Microsoft and really liking it.

01:50:23.580 | And it sort of, it took me a while

01:50:27.820 | before I realized why that was.

01:50:30.900 | But, and I think that actually the founders

01:50:33.820 | of VS Code may not necessarily agree with me on this.

01:50:38.540 | But to me, VS Code is in a sense,

01:50:42.420 | the spiritual successor of Emacs.

01:50:45.620 | Because as you probably know, as an old Emacs hack,

01:50:51.700 | the key part of Emacs is that it's mostly written in Lisp.

01:50:57.140 | And that sort of new features of Emacs

01:51:02.780 | usually update all the Lisp packages

01:51:06.380 | and add new Lisp packages.

01:51:08.700 | And oh yeah, there's also some very obscure thing

01:51:13.140 | improved in the part that's not in Lisp.

01:51:16.500 | But that's usually not why you would upgrade

01:51:19.220 | to a new version of Emacs.

01:51:21.260 | There's a core implementation that sort of can read a file

01:51:26.260 | and it can put bits on the screen

01:51:29.740 | and it can sort of manage memory and buffers.

01:51:33.700 | And then what makes it an editor full of features

01:51:37.740 | is all the Lisp packages.

01:51:39.780 | And of course the design of how the Lisp packages

01:51:42.860 | interact with each other and with that sort of

01:51:46.420 | that base layer of the core immutable engine.

01:51:51.420 | But almost everything in that core engine in Emacs case

01:51:55.300 | can still be overridden or replaced.

01:51:59.340 | And so VS Code has a similar architecture

01:52:04.340 | where there is like a base engine

01:52:10.660 | that you have no control over.

01:52:14.220 | I mean, it's open source, but nobody except the people

01:52:18.460 | who work on that part changes it much.

01:52:22.820 | And it has a sort of a package manager

01:52:28.220 | and a whole series of interfaces for packages

01:52:32.300 | and an additional series of conventions

01:52:35.660 | for how packages should interact with the lower layers

01:52:38.700 | and with each other.

01:52:40.620 | And powerful primitive operations

01:52:43.260 | that let you move the cursor around

01:52:47.460 | or select pieces of text or delete pieces of text

01:52:51.580 | or interact with the keyboard and the mouse

01:52:54.860 | and whatever peripherals you have.

01:52:57.940 | And so the sort of the extreme extensibility

01:53:02.940 | and the package ecosystem that you see in VS Code

01:53:08.220 | is a mirror of very similar architectural features in Emacs.

01:53:13.220 | - Well, I'll have to give it a serious try

01:53:16.580 | 'cause as far as sort of the hype and the excitement

01:53:20.620 | in the general programming community,

01:53:22.340 | VS Code seems to dominate.

01:53:24.340 | The interesting thing about PyCharm and what is it,

01:53:29.260 | PHP Storm, which are these JetBrains specific IDs

01:53:33.980 | that are designed for one programming language.

01:53:36.340 | It's interesting to, when an ID is specialized, right?

01:53:41.060 | - They're usually actually just specializations of IntelliJ

01:53:45.980 | because underneath it's all the same editing engine

01:53:50.460 | with different veneer on top.

01:53:55.460 | Where in VS Code, many things you do

01:54:00.060 | require loading third-party extensions.

01:54:05.780 | In PyCharm, it is possible to have third-party extensions

01:54:10.780 | but it is a struggle to create one.

01:54:14.540 | - Yes, and it's not part of the culture,

01:54:16.340 | all that kind of stuff.

01:54:17.180 | - Yeah, I remember that it might've been five years ago

01:54:21.580 | or so we were trying to get some better MyPy integration

01:54:26.180 | into PyCharm 'cause MyPy is sort of Python tooling

01:54:30.260 | and PyCharm had its own type checking heuristic thing

01:54:35.260 | that we wanted to replace with something based on MyPy

01:54:42.300 | because that was what we were using in the company.

01:54:44.860 | And for the guy who was writing that PyCharm extension,

01:54:49.860 | it was really a struggle to sort of find documentation

01:54:55.420 | and get the development workflow going

01:54:59.860 | and debug his code and all that.

01:55:02.460 | So that was not a pleasant experience.

01:55:06.060 | - Let me talk to you about parallelism.

01:55:08.740 | In your post titled "Reasoning about AsyncIO Semaphore,"

01:55:13.460 | you talk about a fast food restaurant in Silicon Valley

01:55:16.180 | that has only one table.

01:55:17.140 | Is this a real thing?

01:55:18.140 | I just wanted to ask you about that.

01:55:20.020 | Is that just like a metaphor you're using

01:55:21.860 | or is that an actual restaurant in Silicon Valley?

01:55:25.380 | - It was a metaphor, of course.

01:55:27.540 | - I can imagine such a restaurant.

01:55:29.380 | So for people who don't then read the thing, you should.

01:55:33.620 | But it was a idea of a restaurant where there's only

01:55:38.140 | one table and you show up one at a time

01:55:41.540 | and you're prepared.

01:55:43.220 | And I actually looked it up and there is restaurants

01:55:45.260 | like this throughout the world.

01:55:47.380 | And it just seems like a fascinating idea.

01:55:50.500 | You stand in line, you show up, there's one table.

01:55:53.660 | They ask you all kinds of questions,

01:55:56.540 | they cook just for you.

01:55:58.020 | That's fascinating.

01:55:58.860 | - It sounds like you'd find places like that in Tokyo.

01:56:02.380 | It sounds like a very Japanese thing.

01:56:04.380 | Or in the Bay Area, there are popular places

01:56:06.540 | that probably more or less work like that.

01:56:08.540 | I've never eaten at such a place.

01:56:10.460 | - The fascinating thing is you propose it's a fast food.

01:56:12.580 | This is all for a burger.

01:56:14.340 | - It was one of my rare sort of more literary

01:56:19.220 | or poetic moments where I thought I'll just open

01:56:23.380 | with a crazy example to catch your attention.

01:56:26.980 | And the rest is very dry stuff about locks and semaphores

01:56:31.500 | and how a semaphore is a generalization of a lock.

01:56:35.060 | - Well, it was very poetic and well delivered.

01:56:36.980 | And it actually made me wonder if it's real or not

01:56:39.420 | because you don't make that explicit.

01:56:41.300 | And it feels like it could be true.

01:56:43.660 | And in fact, I wouldn't be surprised if somebody

01:56:45.460 | like listens to this and knows exactly a restaurant

01:56:48.260 | like this in Silicon Valley.

01:56:49.780 | Anyway, can we step back and can you just talk

01:56:52.860 | about parallelism, concurrency, threading, asynchronous,

01:56:57.660 | all of these different terms?

01:56:59.500 | What is it, sort of a high philosophical level?

01:57:02.300 | The fisherman is back in the boat.

01:57:04.720 | - Well, the idea is if the fisherman has two fishing rods,

01:57:10.780 | since fishing is mostly a matter of waiting

01:57:13.340 | for a fish to nibble, well, it depends

01:57:15.820 | on how you do it actually.

01:57:16.940 | But if you're doing the style of fishing

01:57:20.400 | where you sort of, you throw it out

01:57:22.380 | and then you let it sit for a while

01:57:25.380 | until maybe you see a nibble,

01:57:27.300 | one fisherman can easily run two or three

01:57:30.300 | or four fishing rods.

01:57:32.900 | And so as long as you can afford the equipment,

01:57:35.700 | you can catch four times as many fish

01:57:37.700 | by a small investment in four fishing rods.

01:57:41.700 | And so since your time, you sort of say

01:57:45.020 | you have all Saturday to go fishing,

01:57:47.540 | if you can catch four times as much fish,

01:57:50.540 | you have a much higher productivity.

01:57:52.820 | - And that's actually, I think, how deep sea fishing is done.

01:57:55.340 | You could just have a rod and you put in a hole

01:57:57.260 | so you can have many rods.

01:57:58.660 | What, is there an interesting difference

01:58:01.860 | between parallelism and concurrency and asynchronous?

01:58:06.540 | Is there one subset of the other to you?

01:58:09.380 | Like, how do you think about these terms?

01:58:10.820 | - In the computer world, there is a big difference.

01:58:14.860 | When people are talking about parallelism,

01:58:18.780 | like a parallel computer,

01:58:20.580 | that's usually really several complete CPUs

01:58:26.780 | that are sort of tied together

01:58:29.620 | and share something like memory or an IO bus.

01:58:35.620 | Concurrency can be a much more abstract concept

01:58:40.620 | where you have the illusion

01:58:45.100 | that things happen simultaneously,

01:58:48.620 | but what the computer actually does

01:58:50.660 | is it spends a little time running this program for a while,

01:58:55.660 | and then it spends some time running that program

01:58:58.060 | for a while, and then spending some time

01:58:59.860 | for the third program for a while.

01:59:02.540 | - So parallelism is the reality,

01:59:05.620 | and concurrency is part reality, part illusion.

01:59:08.380 | - Yeah, parallelism typically implies

01:59:11.820 | that there is multiple copies of the hardware.

01:59:15.700 | - You write that implementing synchronization primitives

01:59:18.660 | is hard in that blog post,

01:59:20.700 | and you talk about locks and semaphores.

01:59:23.580 | Why is it hard to implement synchronization primitives?

01:59:27.020 | - Because at the conscious level,

01:59:29.980 | our brains are not trained to sort of keep track

01:59:34.980 | of multiple things at the same time.

01:59:39.380 | Like, obviously you can walk and chew gum at the same time,

01:59:43.460 | because they're both activities

01:59:45.980 | that require only a little bit of your conscious activity,

01:59:50.980 | but try balancing your checkbook

01:59:53.700 | and watching a murder mystery on TV.

01:59:57.900 | You'll mix up the digits,

01:59:59.540 | or you'll miss an essential clue in the TV show.

02:00:03.700 | - So why does it matter that the programmer,

02:00:05.420 | the human, is bad?

02:00:08.540 | - Because the programmer is,

02:00:10.660 | at least with the current state of the art,

02:00:12.700 | is responsible for writing the code correctly,

02:00:17.500 | and it's hard enough to keep track of a recipe

02:00:22.500 | that you just execute one step at a time.

02:00:27.660 | Chop the carrots, then peel the potatoes, mix the icing.

02:00:32.660 | You need your whole brain,

02:00:35.660 | when you're reading a piece of code,

02:00:38.980 | what is going on?

02:00:40.780 | Okay, we're loading the number of mermaids in variable A,

02:00:45.780 | and the number of mermen in variable B,

02:00:49.180 | and now we take the average or whatever.

02:00:52.040 | - I like how we're just jumping

02:00:54.980 | from metaphor to metaphor, I like it.

02:00:57.020 | - You have to keep in your head,

02:00:58.660 | what is in A, what is in B, what is in C?

02:01:02.380 | Hopefully you have better names.

02:01:04.180 | And that is challenging enough.

02:01:08.060 | If you have two different pieces of code

02:01:12.540 | that are sort of being executed simultaneously,

02:01:17.060 | whether it's using the parallel or the concurrent approach,

02:01:23.700 | if like A is the number of fishermen,

02:01:28.140 | and B is the number of programmers,

02:01:30.860 | but in another part of the code,

02:01:32.740 | A is the number of mermaids,

02:01:34.260 | and B is the number of mermen,

02:01:36.820 | and somehow that's the same variable.

02:01:40.100 | If you do it sequentially,

02:01:41.540 | if first you do your mermaid merpeople computation,

02:01:45.060 | and then you do your people in the boat computation,

02:01:48.380 | it doesn't matter that the variables are called A and B,

02:01:51.620 | and that is literally the same variable,

02:01:53.540 | because you're done with one use of that variable.

02:01:56.980 | But when you mix them together,

02:01:59.460 | suddenly the number of merpeople

02:02:02.540 | replaces the number of fishermen,

02:02:04.340 | and your computation goes dramatically wrong.

02:02:08.100 | - And there's all kinds of ordering of operations

02:02:11.940 | that could result in the assignment of those variables,

02:02:14.380 | and so you have to anticipate all possible orderings.

02:02:17.060 | - And you think you're smart,

02:02:18.580 | and you'll put a lock around it.

02:02:21.100 | And in practice, in terms of bugs per line,

02:02:25.260 | per 1,000 lines of code,

02:02:27.940 | this is an area where everything is worse.

02:02:30.860 | - So a lock is a mechanism by which you forbid

02:02:35.860 | only one chef can access the oven at a time.

02:02:40.020 | - Something like that.

02:02:42.420 | - And then semaphores allow you to do what, multiple ovens?

02:02:46.420 | - That's not a bad idea,

02:02:47.740 | because if you're sort of,

02:02:49.900 | if you're baking cakes,

02:02:52.220 | and you have multiple people all baking cakes,

02:02:55.020 | but there's only one oven,

02:02:56.940 | then maybe you can tell that the oven is in use,

02:02:59.500 | but maybe it's preheating.

02:03:01.940 | And so maybe you make a sign that says, "Oven in use,"

02:03:06.060 | and you flip the sign over,

02:03:08.780 | and it says, "Oven is free

02:03:10.180 | "when you're done baking your cake."

02:03:12.020 | That's a lock, that's sort of,

02:03:15.180 | and what do you do when you have two ovens,

02:03:18.940 | or maybe you have 10 ovens?

02:03:21.060 | You can put a separate sign on each oven,

02:03:23.860 | or maybe you can, sort of,

02:03:25.380 | someone who comes in wants to see at a glance,

02:03:29.060 | and maybe there's an electronic sign that says,

02:03:32.420 | "There are still five ovens available."

02:03:34.700 | Or maybe there are already three people waiting for an oven,

02:03:40.860 | so you can, if you see an oven that's not in use,

02:03:45.780 | it's already reserved for someone else

02:03:47.580 | who got in line first.

02:03:49.380 | And that's sort of what the restaurant metaphor

02:03:51.900 | was trying to explain.

02:03:53.700 | - Yeah, and so you're now tasked,

02:03:55.980 | you're sitting as a designer of Python

02:03:59.140 | with a team of brilliant core developers,

02:04:01.220 | and you have to try to figure out

02:04:03.020 | to what degree can any of these ideas be integrated and not.

02:04:06.820 | So maybe this is a good time to ask,

02:04:08.740 | what is AsyncIO,

02:04:11.220 | and how has it evolved since Python 3.4?

02:04:15.780 | - Wow, yeah, so we had this really old library

02:04:19.820 | for doing things concurrently,

02:04:24.540 | especially things that had to do with IO,

02:04:27.700 | and networking IO was especially sort of a popular topic.

02:04:32.700 | And in the Python standard library,

02:04:38.860 | we had a brief period where there was lots of development,

02:04:45.100 | and I think it was late '90s, maybe early 2000s,

02:04:49.300 | and like two little modules were added

02:04:53.260 | that were the state of the art of doing asynchronous IO,

02:04:56.820 | or sort of non-blocking IO,

02:04:59.180 | which means that you can keep

02:05:00.820 | multiple network connections open

02:05:02.980 | and sort of service them all in parallel

02:05:05.860 | like a typical web server does.

02:05:08.020 | - So IO is input and output,

02:05:09.740 | so you're writing either to the network,

02:05:11.820 | or reading from the network connection,

02:05:13.820 | or reading and writing to a hard drive, to storage.

02:05:16.660 | - Also possible.

02:05:17.700 | - And you can do the ideas you could do to multiple

02:05:21.940 | while also doing computation.

02:05:24.940 | So running some code that does some fancy stuff.

02:05:28.100 | - Yeah, like when you're writing a web server,

02:05:30.820 | when a request comes in,

02:05:32.740 | a user sort of needs to see a particular web page,

02:05:37.100 | you have to find that page maybe in the database

02:05:40.580 | and format it properly and send it back to the client,

02:05:43.700 | and there's a lot of waiting,

02:05:46.540 | waiting for the database, waiting for the network,

02:05:48.860 | and so you can handle hundreds or thousands

02:05:51.500 | or millions of requests concurrently on one machine.

02:05:55.700 | Anyway, ways of doing that in Python were kind of stagnated,

02:06:00.460 | and I forget, it might've been around 2012, 2014,

02:06:05.460 | when someone for the umpteenth time actually said,

02:06:12.620 | these async chat and async core modules

02:06:16.660 | that you have in a standard library

02:06:18.500 | are not quite enough to solve my particular problem,

02:06:22.780 | can we add one tiny little feature?

02:06:25.660 | And everybody said, no, that stuff is not,

02:06:29.380 | you're not supposed to use that stuff,

02:06:31.100 | write your own using a third-party library,

02:06:34.140 | and then everybody started the debate

02:06:35.940 | about what the right third-party library was.

02:06:39.020 | And somehow I felt that there was actually a cue for,

02:06:44.020 | well, maybe we need a better state-of-the-art module

02:06:50.620 | in the standard library for multiplexing input/output

02:06:55.460 | from different sources.

02:06:57.540 | You could say that it spiraled out of control a little bit,

02:07:00.540 | it was, at the time, it was the largest

02:07:03.380 | Python enhancement proposal that was ever proposed.

02:07:07.060 | - And you were deeply involved with that.

02:07:09.060 | - At the time, I was very much involved with that,

02:07:11.740 | I was like the lead architect.

02:07:13.700 | I ended up talking to people

02:07:18.780 | who had already developed serious third-party libraries

02:07:23.020 | that did similar things,

02:07:24.500 | and sort of taking ideas from them,

02:07:26.820 | and getting their feedback on my design,

02:07:30.860 | and eventually we put it in the standard library,

02:07:34.220 | and after a few years, I got distracted,

02:07:36.740 | I think the big thing that distracted me

02:07:38.860 | was actually type annotations.

02:07:40.820 | But other people kept it alive and kicking,

02:07:45.300 | and it's been quite successful, actually,

02:07:47.940 | in the world of Python web clients.

02:07:51.300 | - So initially, what are some of the design challenges there

02:07:54.740 | in that debate for the PEP,

02:07:56.540 | and what are some things that got rejected,

02:07:58.460 | what are some things that got accepted to stand out to you?

02:08:01.900 | - There are a couple of different ways

02:08:03.700 | you can handle parallel IO,

02:08:06.980 | and this happens sort of at an architectural level

02:08:09.940 | in operating systems as well,

02:08:11.820 | like Windows prefers to do it one way,

02:08:14.380 | and Unix prefers to do it the other way.

02:08:17.500 | You sort of, you have an object

02:08:20.260 | that represents a network endpoint,

02:08:23.180 | say a connection with a web browser that's your client,

02:08:26.660 | and say you're waiting for an incoming request.

02:08:32.020 | Two fundamental approaches are,

02:08:34.340 | okay, I'm waiting for an incoming request,

02:08:38.500 | I'm doing something else, come wake me up,

02:08:41.260 | or of course, sort of come tell me

02:08:42.780 | when something interesting happened,

02:08:45.100 | like a packet came in on that network connection.

02:08:48.340 | And the other paradigm is,

02:08:52.340 | we're on a team of a whole bunch of people

02:08:56.460 | with maybe a little mind,

02:08:58.660 | and we can only manage one web connection at a time,

02:09:02.020 | so I'm just sitting,

02:09:05.380 | looking at this web connection,

02:09:09.940 | and I'm just blocked until something comes in,

02:09:13.740 | and then I'm already waiting for it.

02:09:17.340 | I get the data, I process the data,

02:09:21.860 | and then I go back to the top and say,

02:09:23.620 | no, sort of, I'm waiting for the next packet.

02:09:27.460 | Those are about the two paradigms.

02:09:29.180 | One is a paradigm where there is sort of notionally

02:09:34.140 | a thread of control,

02:09:35.420 | whether it's an actual operating system thread

02:09:37.900 | or more an abstraction in async IO, we call them tasks.

02:09:41.340 | But a task in async IO or a thread in other contexts

02:09:46.940 | is devoted to one thing,

02:09:49.500 | and it has logic for all the stages,

02:09:52.580 | like when it's a web request,

02:09:54.620 | like first wait for the first line of the web request,

02:09:58.900 | parse it, because then you know if it's a get or a post

02:10:02.380 | or a put or whatever, or an error.

02:10:05.700 | Then wait until you have a bunch of lines

02:10:09.300 | until there's a blank line,

02:10:10.860 | then parse that as headers,

02:10:12.420 | and then interpret that,

02:10:14.540 | and then wait for the rest of the data to come in

02:10:18.380 | if there is any more that you expect,

02:10:21.220 | that sort of standard web stuff.

02:10:24.420 | And the other thing is,

02:10:26.260 | and there's always endless debate

02:10:27.980 | about which approach is more efficient

02:10:30.100 | and which approach is more error prone,

02:10:33.140 | where I just have a whole bunch of stacks in front of me,

02:10:36.980 | and whenever a packet comes in,

02:10:41.140 | I sort of look at the number of the packet,

02:10:43.660 | that there's some number on the packet,

02:10:45.540 | and I say, "Oh, that packet goes on this pile,"

02:10:49.100 | and then I can do a little bit,

02:10:51.260 | and then sort of that pile provides my context,

02:10:54.260 | and as soon as I'm done with the processing,

02:10:57.820 | I sort of, I can forget everything about what's going on,

02:11:02.060 | because the next packet will come in

02:11:03.780 | from some random other client,

02:11:05.660 | and it's that pile or it's this pile.

02:11:07.740 | And every time a pile is maybe empty or full

02:11:11.580 | or whatever the criteria is,

02:11:13.500 | I can toss it away or use it for a new space.

02:11:16.580 | But several traditional third-party libraries

02:11:21.060 | for asynchronous I/O processing in Python

02:11:24.180 | chose the model of a callback,

02:11:27.140 | and that's the idea where you have a bunch

02:11:29.500 | of different stacks of paper in front of you,

02:11:32.500 | and every time someone gives you a piece,

02:11:34.820 | gives you a new sheet,

02:11:36.220 | you decide which stack it belongs to.

02:11:38.660 | And that leads to a certain style of spaghetti code

02:11:44.580 | that I find sort of aesthetically not pleasing,

02:11:50.500 | and I was sort of never very successful,

02:11:54.260 | and I had heard many stories about people

02:11:56.460 | who were also sort of complaining

02:12:00.140 | about that style of coding.

02:12:01.840 | It was very prevalent in JavaScript at the time at least,

02:12:06.340 | because it was like how the JavaScript event loop

02:12:10.260 | basically works.

02:12:11.860 | And so I thought, well, the task-based model

02:12:15.620 | where each task has a bunch of logic,

02:12:19.420 | we had mechanisms in the Python language

02:12:22.020 | that we could easily reuse for that.

02:12:25.940 | And I thought, I want to build a whole library

02:12:29.180 | for asynchronous networking I/O,

02:12:31.500 | and all the other things

02:12:33.940 | that may need to be done asynchronously,

02:12:36.840 | based on that paradigm.

02:12:39.980 | And so I just chose a paradigm

02:12:41.900 | and tried to see how far I could get with that.

02:12:45.820 | And it turns out that it's a pretty good paradigm.

02:12:48.980 | - So people enjoy that kind of paradigm programming

02:12:51.460 | for asynchronous I/O, relative to callbacks.

02:12:55.760 | Okay, beautiful.

02:12:58.620 | So how does that all interplay with the infamous GIL,

02:13:03.620 | the Global Interpreter Lock?

02:13:05.600 | Maybe can you say what the GIL is,

02:13:08.620 | and how does it dance beautifully with asyncio?

02:13:12.060 | - The Global Interpreter Lock solves the problem

02:13:17.380 | that Python originally was not written

02:13:19.740 | with either asynchronous or parallelism in mind at all.

02:13:24.740 | There was no concurrency in the language.

02:13:26.700 | There was no parallelism.

02:13:27.840 | There were no threads.

02:13:30.320 | Only a small number of years

02:13:32.660 | into Python's initial development,

02:13:35.980 | all the new cool operating systems

02:13:38.660 | like SunOS and Silicon Graphics,

02:13:42.300 | Irix, and then eventually POSIX and Windows,

02:13:46.660 | all came with threading libraries

02:13:49.820 | that lets you do multiple things in parallel.

02:13:53.300 | And there is a certain sort of principle,

02:13:57.260 | which is the operating system handles the threads for you.

02:14:01.720 | And the program can pretend that there are as many CPUs

02:14:07.620 | as there are threads in the program.

02:14:13.220 | And those CPUs work completely independently.

02:14:17.260 | And if you don't have enough CPUs,

02:14:20.380 | the operating system sort of simulates those extra CPUs.

02:14:25.140 | On the other hand, if you have enough CPUs,

02:14:28.300 | you can get a lot of work done

02:14:31.540 | by deploying those multiple CPUs.

02:14:34.420 | But Python wasn't written to do that.

02:14:40.860 | And so as libraries for multithreading were added to C,

02:14:45.860 | but every operating system vendor

02:14:52.220 | was adding their own version of that.

02:14:55.380 | We thought, and maybe we were wrong,

02:14:58.220 | but at the time we thought,

02:14:59.580 | "Well, we quickly want to be able

02:15:01.300 | "to support these multiple threads."

02:15:04.300 | Because they seemed at the time in the early '90s,

02:15:07.420 | when they were new, at least to me,

02:15:09.980 | they seemed a cool, interesting programming paradigm.

02:15:13.820 | And one of the things that Python,

02:15:16.020 | at least at the time, felt was nice about the language

02:15:20.020 | was that we could give a safe version

02:15:23.900 | of all kinds of cool new operating system toys

02:15:28.820 | to the Python programmer.

02:15:30.220 | Like I remember one or two years before threading,

02:15:36.140 | I had spent some time adding networking sockets to Python.

02:15:41.140 | And they were very literal translation

02:15:45.580 | of the networking sockets

02:15:46.940 | that were in the BSD operating system, so Unix BSD.

02:15:50.480 | But the nice thing was if you were using sockets from Python,

02:15:55.460 | then all the things you can do wrong with sockets in C

02:15:59.100 | would automatically give you a clear error message

02:16:01.740 | instead of just ending up

02:16:03.220 | with a malfunctioning hanging program.

02:16:06.300 | And so we thought,

02:16:07.140 | "Well, we'll do the same thing with threading."

02:16:10.100 | But we didn't really want to rewrite the interpreter

02:16:15.100 | to be thread safe,

02:16:17.220 | because that would be a very complex refactoring

02:16:22.220 | of all the interpreter code and all the runtime code,

02:16:27.500 | because all the objects were written with the assumption

02:16:30.280 | that there's only one thread.

02:16:32.300 | And so we said, "Okay, well, we'll take our losses.

02:16:35.940 | We'll provide something that looks like threads.

02:16:39.860 | And as long as you only have a single CPU on your computer,"

02:16:43.260 | which most computers at the time did,

02:16:45.500 | "it feels just like threads."

02:16:48.540 | Because the whole idea of multiple threads in the OS

02:16:53.540 | was that even if your computer only had one CPU,

02:16:57.420 | you could still fire up as many threads as you wanted.

02:17:01.020 | Well, within reason, maybe 10 or 12, not 5,000.

02:17:04.480 | And so we thought we had conquered

02:17:09.220 | the abstraction of threads pretty well,

02:17:14.360 | because multi-core CPUs were not

02:17:18.820 | in most Python programmers' hands anyway.

02:17:22.860 | And then, of course, a couple of more iterations

02:17:26.500 | of Moore's law, and computers getting faster.

02:17:29.600 | And at some point, the chip designers decided

02:17:34.600 | that they couldn't make the CPUs faster,

02:17:38.300 | but they could still make them smaller.

02:17:40.060 | And so they could put multiple CPUs on one chip.

02:17:43.740 | And suddenly there was all this pressure

02:17:46.520 | about do things in parallel.

02:17:49.520 | And that's where the solution we had in Python didn't work.

02:17:55.340 | And that's sort of the moment that the GIL became infamous.

02:18:00.340 | Because the GIL was the solution we used

02:18:04.320 | to sort of take this single interpreter

02:18:09.020 | and share it between all the different operating system

02:18:12.440 | threads that you could create.

02:18:14.920 | And so as long as the hardware physically only had one CPU,

02:18:19.640 | that was all fine.

02:18:21.600 | And then as hardware vendors were suddenly telling us all,

02:18:25.680 | "Oh, you gotta parallelize.

02:18:27.480 | "Everything's gotta be parallelized."

02:18:30.120 | People started saying, "Oh, but we can use multiple threads

02:18:34.640 | "in Python."

02:18:35.680 | And then they discovered, "Oh, but actually all threads

02:18:39.220 | "run on a single core."

02:18:41.600 | - Yeah.

02:18:42.640 | I mean, is there a way, is there ideas in the future

02:18:46.000 | to remove the global interpreter log GIL?

02:18:49.560 | Like maybe multiple sub-interpreters,

02:18:52.240 | some tricky interpreters on top of interpreters

02:18:56.840 | kind of thing?

02:18:57.680 | - Yeah, there are a couple of possible futures there.

02:19:02.520 | The most likely future is that we'll get multiple

02:19:07.280 | sub-interpreters, which each run a completely

02:19:11.640 | independent Python program.

02:19:14.040 | - Nice.

02:19:15.220 | - But there's still some benefit of sort of faster

02:19:20.220 | communication between those programs.

02:19:25.060 | - But it's also managing for you this running

02:19:28.220 | a multiple Python programs.

02:19:30.580 | - Yeah.

02:19:31.420 | - So it's hidden from you, right?

02:19:33.540 | - It's hidden from you, but you have to spend more time

02:19:36.980 | communicating between those programs.

02:19:39.180 | Because the sort of, the attractive thing about the

02:19:43.960 | multi-threaded model is that the threads can share objects.

02:19:48.860 | At the same time, that's also the downfall of the

02:19:51.620 | multi-threaded programming model.

02:19:53.900 | Because when you do share objects, you weren't,

02:19:58.260 | and you didn't necessarily intend to share them,

02:20:01.460 | or there were aspects of those objects that were not

02:20:06.460 | reusable, you get all kinds of concurrency bugs.

02:20:11.420 | And so the reason I wrote that little blog post

02:20:15.880 | about semaphores was that concurrency bugs are just harder.

02:20:20.360 | It would be nice if Python had no global interpreter lock,

02:20:26.280 | and it had the so-called free threading,

02:20:28.540 | but it would also cause a lot more software bugs.

02:20:34.380 | The interesting thing is that there is still a possible

02:20:39.080 | future where we are actually going to, or where we could

02:20:43.220 | experiment at least with that, because there is a guy

02:20:48.220 | working for Facebook who has developed a fork of CPython

02:20:54.240 | that he called the no-gill interpreter, where he removed

02:21:00.220 | the gill and made a whole bunch of optimizations

02:21:05.000 | so that the single-threaded case doesn't run

02:21:08.300 | too much slower, and multi-threaded case will actually

02:21:13.300 | use all the cores that you have.

02:21:15.740 | And so that would be an interesting possibility

02:21:22.680 | if we would be willing as Python core developers

02:21:27.780 | to actually maintain that code indefinitely.

02:21:34.920 | And if we're willing to put up with the additional

02:21:38.540 | complexity of the interpreter and the additional

02:21:42.300 | sort of overhead for the single-threaded case.

02:21:45.120 | And I'm personally not convinced that

02:21:49.080 | there are enough people needing the speed of multiple

02:21:56.720 | threads with their Python programs that it's worth

02:22:03.640 | to sort of take that performance hit and that complexity hit.

02:22:08.640 | And I feel that the gill actually is a pretty nice

02:22:13.720 | Goldilocks point between no threads and all threads

02:22:18.720 | all the time, but not everybody agrees on that.

02:22:21.800 | So that is definitely a possible future.

02:22:24.760 | The sub-interpreters look like a fairly safe bet for 3.12.

02:22:29.080 | So say a year from now.

02:22:32.120 | - A year, so the goal is to do a new version

02:22:34.360 | every year for Python.

02:22:36.880 | Let me ask you perhaps a fun question,

02:22:39.080 | but there's a philosophy to it too.

02:22:42.400 | Will there ever be a Python 4.0?

02:22:45.640 | Now, before you say it's currently a joke and probably not,

02:22:50.040 | we're gonna go to 3.99 or 3.999,

02:22:55.040 | can you imagine possible features that Python 4.0

02:23:02.360 | might have that would necessitate the creation

02:23:07.200 | of the new 4.0 given the amount of pain and joy,

02:23:12.200 | suffering and triumph that was involved in the move

02:23:16.640 | between version two and version three?

02:23:19.320 | - Yeah, well, we, as a community

02:23:26.200 | and as a core development team,

02:23:28.480 | we have a large amount of painful memories

02:23:33.480 | about the Python 3 transition,

02:23:36.720 | which is one reason that sort of everybody is happy

02:23:43.280 | that we've decided there's not going to be a 4.0

02:23:47.160 | at least not anytime soon.

02:23:50.080 | And if there is going to be one,

02:23:52.800 | we'll sort of plan the transition very differently.

02:23:56.800 | Because clearly we underestimated the pain

02:24:00.080 | that transition caused for our users in the Python 3 case.

02:24:05.080 | And had we known we could have sort of designed

02:24:10.720 | Python 3 somewhat differently without making it any worse,

02:24:16.760 | we just thought that we had a good plan,

02:24:20.240 | but we underestimated what sort of the users

02:24:25.600 | were capable of when it comes to that kind of transition.

02:24:28.840 | - By the way, I think we talked way before,

02:24:32.080 | like a year and a half before the Python 2 officially--

02:24:37.080 | - End of life? - End of life.

02:24:39.480 | - Oh, yeah.

02:24:41.040 | - What was that?

02:24:41.880 | What was your memory of the end of life?

02:24:43.720 | Did you shed a tear on January 1st, 2020?

02:24:47.040 | Was there, were you standing alone?

02:24:49.360 | - Everyone on the core team had basically moved

02:24:51.480 | on years before.

02:24:54.000 | - Yeah.

02:24:54.840 | - It was purely, it was a little symbolic moment

02:24:58.040 | to signal to the remaining users

02:25:03.520 | that there was no longer going to be any new releases

02:25:08.520 | or support for Python 2.7.

02:25:12.880 | - Did you shed a single tear

02:25:15.920 | while looking out over the horizon?

02:25:18.040 | (laughing)

02:25:19.000 | - I'm not a very poetic person

02:25:21.440 | and I don't shed tears like that, but no.

02:25:25.000 | (laughing)

02:25:26.720 | - Yeah, we actually had planned a party,

02:25:29.680 | but the party was planned for the Python,

02:25:32.920 | the US Python conference that year,

02:25:35.040 | which would never happened, of course,

02:25:36.720 | because of the pandemic.

02:25:38.040 | - Oh, was it like in March or something?

02:25:39.720 | - Yeah, the conference was going to be,

02:25:42.720 | I think, late April that year.

02:25:44.880 | - Oh.

02:25:45.960 | - So that was a very difficult decision to cancel it,

02:25:49.000 | but they did.

02:25:51.680 | So anyway, if we're going to have a Python 4,

02:25:54.600 | we're going to have to have both a different reason

02:25:57.920 | for having that and a different process

02:26:02.480 | for managing the transition.

02:26:04.040 | - Can you imagine a possible process that,

02:26:07.400 | so I think you're implying that if there is a 4.0,

02:26:11.280 | in some ways it would break back compatibility?

02:26:14.920 | - Well, so here is a concrete thought I've had,

02:26:20.640 | and I'm not unique, but not everyone agrees with this,

02:26:23.920 | so this is definitely a personal opinion.

02:26:26.360 | If we were to try something like that Nogill Python,

02:26:32.400 | my expectation is that it would feel just different enough,

02:26:40.600 | at least for the part of the Python ecosystem

02:26:49.480 | that is heavily based on C extensions,

02:26:54.480 | and that is like the entire machine learning,

02:26:58.040 | data science, scientific Python world

02:27:02.080 | is all based on C extensions for Python.

02:27:06.400 | And so those people would likely feel the pain the most

02:27:14.760 | because even if we don't change anything

02:27:19.760 | about the syntax of the language

02:27:21.720 | and the semantics of the language

02:27:23.320 | when you're writing Python code,

02:27:25.720 | we could even say, suppose that after Python say 3.19,

02:27:30.120 | instead of 3.20, we'll have 4.0.

02:27:33.720 | Suppose that's the time when we flip the switch to 4.0,

02:27:38.560 | we'll not have a GIL.

02:27:41.040 | Imagine it was like that.

02:27:43.520 | So I would probably say that particular year,

02:27:48.520 | the release that we name 4.0 will be syntactically,

02:27:54.840 | it will not have any new syntactical features,

02:27:58.800 | no new modules in the standard library,

02:28:01.360 | no new built-in functions.

02:28:03.360 | Everything will be at the Python level

02:28:07.480 | will be purely compatible with Python 3.19.

02:28:12.520 | However, extension modules will have to make a change.

02:28:17.520 | They will have to be recompiled.

02:28:21.720 | They will not have the same binary interface.

02:28:26.720 | The semantics and APIs for some things

02:28:34.240 | that are frequently accessed by C extensions

02:28:38.520 | will be different.

02:28:40.200 | And so for a pure Python user, 4.0 would be a breeze,

02:28:45.200 | except that there are very few pure Python users left

02:28:50.000 | because everybody who is using Python

02:28:52.560 | for something significant is using third-party extensions.

02:28:56.880 | There are like, I don't know,

02:28:58.480 | several hundreds of thousands of third-party extensions

02:29:01.800 | on the PyPI service.

02:29:06.240 | And I'm not saying they're all good,

02:29:08.520 | but there is a large list of extensions

02:29:12.000 | that would have to do work.

02:29:13.360 | And some of those extensions

02:29:14.920 | are currently already low on maintainers

02:29:19.920 | and they're struggling to keep afloat.

02:29:23.680 | - So there you can give a huge heads up to them

02:29:26.520 | if you go to 4.0 to really keep developing it.

02:29:30.400 | - Yeah, we'd probably have to do something like

02:29:32.800 | several years before, who knows, maybe five years earlier,

02:29:38.280 | like 3.15, we would have to say,

02:29:42.000 | and I'm just making the specific numbers up,

02:29:45.320 | but at some point we'd have to say

02:29:47.440 | the Nogail Python could be an option.

02:29:52.920 | It might be a compile time option.

02:29:55.240 | If you want to use Nogail Python,

02:30:00.760 | you have to recompile Python from source for your platform

02:30:04.240 | using your tool set.

02:30:06.240 | All you have to do is change one configuration variable

02:30:09.760 | and then you just run make or configure and make

02:30:13.440 | and it will build it for you.

02:30:15.560 | But now you also have to use

02:30:18.560 | the Nogail compatible versions

02:30:21.400 | of all extension modules you want to use.

02:30:24.720 | And so as long as many extension modules

02:30:27.720 | don't have fully functional sort of variants

02:30:32.720 | that work in the Nogail world,

02:30:35.720 | that's not a very practical thing for Python users,

02:30:39.560 | but it would allow extension developers

02:30:43.880 | to test the waters,

02:30:46.080 | see what they need to syntactically

02:30:49.200 | to be able to compile at all.

02:30:51.000 | Maybe they're using functions

02:30:54.400 | that are defined by the Python 3 runtime

02:30:56.960 | that won't be in the Python 4 runtime.

02:30:59.040 | Those functions will not work.

02:31:00.520 | They'll have to find an alternative,

02:31:04.400 | but they can experiment with that

02:31:06.240 | and sort of write test applications.

02:31:09.120 | And that would be a way to transition.

02:31:11.120 | And that could be a series of releases

02:31:15.960 | where the Python 4 is more and more imminent.

02:31:20.760 | We have supported more and more third-party

02:31:24.600 | extension modules to have solid support

02:31:28.280 | that works for Nogail Python for that new API.

02:31:33.880 | And then sort of Python 4.0 is like the official moment

02:31:38.880 | that the mayor comes out and cuts the ribbon.

02:31:43.200 | And now Python, now the sort of Nogail mode

02:31:47.640 | is the default and maybe the only mode there is.

02:31:50.480 | - The internet wants to know from Reddit.

02:31:54.040 | It's a small and fun question.

02:31:58.840 | There's many fun questions,

02:31:59.920 | but out of the PyPI packages,

02:32:04.360 | PyPI packages, do you have ones you like?

02:32:09.360 | In your opinion, are there must-have PyPI libraries

02:32:12.760 | or ones you use all the time constantly?

02:32:15.240 | - Oh my, I should really have a standard answer

02:32:19.800 | for that question, but like a positive standard answer.

02:32:23.600 | But my current standard answer is that

02:32:25.840 | I'm not a big user of third-party packages.

02:32:30.080 | When I write Python code, I'm usually developing

02:32:33.880 | some tooling around building Python itself.

02:32:37.080 | And the last thing we want is dependencies

02:32:41.960 | on third-party packages.

02:32:43.440 | So I tend to just use the standard library and--

02:32:46.920 | - That's where your focus is, that's where your mind is.

02:32:50.280 | But do you keep an eye of what's out there

02:32:53.320 | to understand where the standard library

02:32:56.240 | could be moving, should be moving?

02:32:58.360 | It's a good kind of landscape of what's missing

02:33:01.520 | from the standard library.

02:33:02.560 | - Well, usually when something's missing

02:33:04.800 | from the standard library, nowadays,

02:33:07.120 | it is a relatively new idea

02:33:13.920 | and there is a third-party implementation

02:33:17.800 | or maybe possibly multiple third-party implementations,

02:33:22.040 | but they evolve at a much higher rate

02:33:25.760 | than they could when they're in the standard library.

02:33:28.040 | So it would be a big reduction in activity

02:33:33.040 | to incorporate things like that in the standard library.

02:33:38.200 | So I like that there is a lively package ecosystem

02:33:41.880 | and that sort of recent trends in the standard library

02:33:45.560 | are actually that we're doing the occasional

02:33:47.960 | spring cleaning where we're just,

02:33:49.880 | we're choosing some modules

02:33:56.840 | that have not had a lot of change in a long time

02:34:02.040 | and that maybe would be better off not existing

02:34:07.040 | at all at this point, because there might be

02:34:09.880 | a better third-party alternative anyway,

02:34:13.440 | and we're sort of slowly removing those.

02:34:17.240 | Often those are things that I sort of,

02:34:20.360 | I spiked somewhere in 1992 or 1993.

02:34:24.080 | If you look through the commit history, it's very sad.

02:34:29.240 | All cosmetic changes, like changes in the indentation style

02:34:34.760 | or the name of this other standard library module

02:34:38.160 | got changed or nothing of any substance.

02:34:42.800 | The API is identical to what it was 20 years ago.

02:34:47.320 | - So speaking of packages, they have a lot of impact

02:34:52.320 | on a lot of people's lives.

02:34:54.000 | Does it make sense to you why Python has become

02:34:57.440 | the primary, the dominant language

02:34:59.240 | for the machine learning community?

02:35:00.920 | So packages like PyTorch, TensorFlow, Scikit-learn,

02:35:05.160 | and even like the lower level stuff like NumPy, SciPy,

02:35:08.160 | Pandas, Matplotlib with visualization.

02:35:11.080 | Can you like, does it make sense to you why it,

02:35:15.440 | permeated the entire data science,

02:35:18.800 | machine learning, AI community?

02:35:21.080 | - Well, part of it is an effect that's as simple

02:35:25.320 | as we're all driving on the right side of the road, right?

02:35:29.320 | It's compatibility.

02:35:33.360 | - Yeah.

02:35:34.200 | - It's, and part of it is not quite as fundamental

02:35:42.880 | as driving on the right side of the road,

02:35:44.640 | which you have to do for safety reasons.

02:35:46.960 | I mean, you have to agree on something.

02:35:48.920 | They could have picked JavaScript or Perl.

02:35:52.400 | There was a time in the early 2000s

02:35:54.680 | that it really looked like Perl was going to dominate

02:35:58.280 | like biosciences, because DNA search was all based

02:36:02.600 | on regular expressions and Perl has the fastest

02:36:05.120 | and most comprehensive regular expression engine, still does.

02:36:08.600 | - I spent quite a long time with Perl.

02:36:11.760 | That was another letting go.

02:36:14.040 | Letting go of this kind of data processing system.

02:36:19.040 | - The reasons why Python became the lingua franca

02:36:24.520 | of scientific code and machine learning in particular

02:36:29.520 | and data science, it really had a lot to do

02:36:36.840 | with anything was better than C or C++.

02:36:42.400 | Recently, a guy who worked

02:36:44.560 | at Lawrence Livermore National Laboratories

02:36:47.520 | in the sort of computing division wrote me his memoirs

02:36:52.520 | and he had his own view of how he helped something

02:37:00.160 | he called computational steering into existence.

02:37:04.880 | And this was the idea that you take libraries

02:37:10.040 | that in his days were written in Fortran

02:37:12.960 | that solved universal mathematical problems.

02:37:17.280 | And those libraries still work,

02:37:20.920 | but the scientists that use the libraries,

02:37:25.640 | use them to solve continuously different

02:37:30.440 | specific applications and answer different questions.

02:37:34.760 | And so those poor scientists were required

02:37:39.960 | to use say Fortran because Fortran was the language

02:37:44.440 | that the library was written in.

02:37:47.120 | And then the scientists would have to write an application

02:37:51.360 | that sort of uses the library to solve a particular equation

02:37:55.880 | or set off of answer a set of questions.

02:37:59.560 | And the same for C++ because there's interoperability.

02:38:06.080 | So the dusty decks are written either in C++ or Fortran.

02:38:10.720 | And so Paul Dubois was one of the people who,

02:38:16.800 | I think in the mid '90s saw that you needed

02:38:21.800 | a higher level language for the scientists

02:38:26.240 | to sort of tie together the fundamental

02:38:31.400 | mathematical algorithms of linear algebra and other stuff.

02:38:36.000 | And so gradually some libraries started appearing

02:38:42.160 | that did very fundamental stuff

02:38:46.240 | with arrays of numbers in Python.

02:38:49.280 | I mean, when I first created Python,

02:38:52.080 | I was not expecting it to be used

02:38:54.400 | for arrays of numbers much.

02:38:55.840 | I thought that was like an outdated data type

02:38:59.280 | and everything was like objects and strings

02:39:02.800 | and like Python was good and fast at string manipulation

02:39:06.680 | and objects obviously, but arrays of numbers

02:39:09.920 | were not very efficient and the multidimensional arrays

02:39:13.080 | didn't even exist in the language at all.

02:39:15.320 | But there were people who realized

02:39:19.880 | that Python had extensibility that was flexible enough

02:39:25.720 | that they could write third party packages

02:39:30.600 | that did support large arrays of numbers

02:39:33.280 | and operations on them very efficiently.

02:39:35.800 | And somehow they got a foothold

02:39:39.640 | through sort of different parts of the scientific community.

02:39:44.640 | I remembered that the Hubble Space Telescope people

02:39:47.920 | in Baltimore were somehow big Python fans in the late '90s.

02:39:52.800 | And at various points, small improvements were made

02:39:57.800 | and more people got in touch with using Python

02:40:02.680 | to derive these libraries of interesting algorithms.

02:40:07.680 | And like once you have a bunch of scientists

02:40:12.840 | who are working on similar problems,

02:40:14.880 | say they're all working on stuff that comes in

02:40:18.920 | from the Hubble Space Telescope,

02:40:20.440 | but they're looking at different things.

02:40:21.840 | Some are looking at stars in this galaxy,

02:40:24.760 | other are looking at galaxies.

02:40:26.800 | The math is completely different,

02:40:28.240 | but the underlying libraries are still the same.

02:40:33.240 | And so they exchange code.

02:40:36.840 | They say, well, I wrote this Python program

02:40:39.480 | or I wrote a Python library to solve this class of problems.

02:40:43.960 | And the other guys either say, oh, I can use that library too

02:40:48.440 | or if you make a few changes, I can use that library too.

02:40:52.600 | Why start from scratch in Perl or JavaScript

02:40:57.480 | where there's not that infrastructure

02:40:59.400 | for arrays of numbers yet, whereas in Python you have it.

02:41:04.720 | And so more and more scientists at different places

02:41:07.800 | doing different work discovered Python

02:41:12.800 | and then people who had an idea

02:41:16.360 | for an important new fundamental library decided,

02:41:20.040 | oh, Python is actually already known to our users.

02:41:25.040 | So let's use Python as the user interface.

02:41:28.880 | I think that's how Tensor, I imagine at least

02:41:31.200 | that's how TensorFlow ended up with Python

02:41:33.360 | as the user interface.

02:41:35.520 | - Right, but with TensorFlow,

02:41:37.920 | there's a deeper history of what the community,

02:41:42.840 | so it's not just like what packages it needs.

02:41:45.120 | It's like what the community leans on

02:41:47.360 | for a programming language 'cause TensorFlow

02:41:50.360 | had a prior library that was internal to Google

02:41:55.160 | but there was also competing machine learning frameworks

02:41:58.960 | like Theano, Caffe, they were in Python.

02:42:02.760 | There was some Scala, some other languages

02:42:06.780 | but Python was really dominating it.

02:42:08.820 | And it's interesting because there's other languages

02:42:13.320 | from the engineering space like MATLAB

02:42:16.560 | that a lot of people used but different design choices

02:42:20.920 | by the company, by the core developers

02:42:23.960 | led to it not spreading.

02:42:26.000 | And one of the choices of MATLAB by MathWorks

02:42:29.920 | is to not make it open source, right?

02:42:31.820 | Or not having people pay.

02:42:34.600 | - It was a very expensive product

02:42:36.640 | and so universities especially disliked it

02:42:40.600 | because it was a price per seat.

02:42:42.880 | I remember hearing.

02:42:45.120 | - Yeah, but I think that's not why it failed

02:42:48.880 | or it failed to spread.

02:42:50.820 | I think the universities didn't like it

02:42:52.920 | but they would still pay for it.

02:42:55.120 | The thing is it didn't feed into that GitHub

02:42:58.320 | open source packages culture.

02:43:02.720 | So like, and that's somehow a precondition

02:43:05.080 | for viral spreading, the hacker culture,

02:43:09.140 | like the tinkerer culture.

02:43:11.800 | With Python it feels like you can build a package

02:43:13.920 | from scratch or solve a particular problem

02:43:16.120 | and get excited about sharing that package with others.

02:43:19.040 | And that creates an excitement about a language.

02:43:22.300 | - I tend to like Python's approach to open source

02:43:25.160 | in particular because it's sort of,

02:43:27.120 | it's almost egalitarian.

02:43:30.040 | There's little hierarchy.

02:43:34.920 | There's obviously some because like you all need to decide

02:43:38.960 | whether you drive on the left or the right side

02:43:40.780 | of the road sometimes.

02:43:42.680 | But there is a lot of access for people with little power.

02:43:47.340 | You don't have to work for a big tech company

02:43:50.000 | to make a difference in the Python world.

02:43:52.500 | We have affordable events that really care about community

02:43:59.200 | and support people and sort of the community

02:44:03.360 | is like a big deal at our conferences

02:44:08.600 | and in the PSF.

02:44:11.100 | When the PSF funds events,

02:44:12.940 | it's always about growing the community.

02:44:17.900 | The PSF funds very little development.

02:44:21.580 | They do some, but most of the money that the PSF forks out

02:44:28.780 | is to community fostering things.

02:44:35.600 | - So speaking of egalitarian,

02:44:37.560 | last time we talked four years ago,

02:44:39.940 | it was just after you stepped down from your role

02:44:43.280 | as the benevolent dictator for life, BDFL.

02:44:47.500 | Looking back, what are your insights and lessons

02:44:51.040 | you learned from that experience

02:44:52.440 | about Python developer community, about human nature,

02:44:56.320 | about human civilization, life itself?

02:45:00.840 | - Oh my.

02:45:04.080 | I probably held onto the position too long.

02:45:07.320 | I remember being just extremely stressed for a long time

02:45:13.800 | and it wasn't very clear to me what was leading,

02:45:21.040 | what was causing the stress.

02:45:23.780 | And looking back,

02:45:30.920 | I should have sort of relinquished my central role

02:45:35.920 | as BDFL sooner.

02:45:39.080 | - What were the pros and cons of the BDFL role?

02:45:42.880 | Like what were the, you not relinquishing it,

02:45:45.320 | what are the benefits of that for the community?

02:45:48.440 | And what are the drawbacks?

02:45:50.560 | - Well, the benefits for the community would be things like

02:45:58.920 | clarity of vision and sort of a clear direction

02:46:03.920 | because I had certain ideas in mind when I created Python.

02:46:10.880 | And while I sort of let myself be influenced

02:46:16.440 | by many other ideas as Python evolved

02:46:19.980 | and became more successful and more complex and more used,

02:46:27.760 | I also stuck to certain principles.

02:46:30.480 | And it's still hard to say

02:46:31.840 | what are Python's core principles.

02:46:34.140 | But the fact that I was playing that role

02:46:40.000 | and sort of always very active,

02:46:43.900 | grew the community in a certain way.

02:46:47.780 | It modeled to the community how to think about

02:46:52.000 | how to solve a certain problem.

02:46:55.280 | - Well, that was a source of stress,

02:46:57.720 | but it was also beneficial to the community.

02:46:58.560 | - It was a source of stress for me personally,

02:47:01.120 | but it was beneficial for the community

02:47:03.520 | because people sort of over time

02:47:08.520 | had learned how I was thinking and could predict

02:47:13.080 | but how I would decide about a particular issue

02:47:18.520 | and not always perfectly, of course.

02:47:20.360 | But there wasn't a lot of jerking around

02:47:24.800 | like this year, we're all,

02:47:26.560 | but this year the Democrats are in power

02:47:29.360 | and we're doing these kinds of things.

02:47:31.160 | And now the Republicans are in power

02:47:33.280 | and they roll all that back and do those kinds of things.

02:47:36.640 | There is a clear, fairly straight path ahead.

02:47:41.200 | And so fortunately the successor structure

02:47:45.080 | with the steering council has sort of found a similar way

02:47:50.680 | of leading the community in a fairly steady direction

02:47:55.680 | without stagnating.

02:47:58.480 | And for me personally, it's more fun

02:48:00.640 | because there are things I can just ignore.

02:48:03.640 | Yeah, oh yeah, there's a bug in multi-processing.

02:48:07.800 | Let someone else decide whether that's important

02:48:10.080 | to solve or not.

02:48:11.040 | I'll stick to typing in the async IO

02:48:16.120 | and the faster interpreter.

02:48:18.640 | - Yeah, it allows you to focus a little bit more.

02:48:20.800 | - Yeah.

02:48:21.640 | - What are interesting differences in culture

02:48:25.120 | if you can comment on between Google, Dropbox

02:48:27.480 | and Microsoft from a Python programming perspective,

02:48:30.600 | all places you've been to, the positive.

02:48:32.920 | Is there a difference or is it just about people

02:48:37.520 | and there's great people everywhere

02:48:40.160 | or is there culture differences?

02:48:41.760 | - So Dropbox is much smaller than the other two

02:48:46.360 | in your list.

02:48:47.720 | - Yeah.

02:48:48.560 | - So that is a big difference.

02:48:52.600 | - The set of products they provide is narrower

02:48:55.400 | so they're more focused.

02:48:57.000 | Smaller code base.

02:48:57.840 | - Yeah, and Dropbox sort of,

02:49:00.480 | at least during the time I was there,

02:49:03.720 | had the tendency of sort of making a big plan,

02:49:08.720 | putting the whole company behind that plan for a year

02:49:12.400 | and then evaluate and then suddenly find that

02:49:17.680 | everything was wrong about the plan

02:49:19.960 | and then they had to do something completely different.

02:49:22.800 | So there was like the annual engineering reorg

02:49:28.480 | was sort of an unpleasant tradition at Dropbox

02:49:31.800 | because like, oh, there's a new VP of engineering

02:49:34.520 | and so now all the directors are being reshuffled

02:49:37.280 | and this guy was in charge of infrastructure one year

02:49:42.280 | and the next year he was made in charge of,

02:49:45.880 | I don't know, product development.

02:49:48.480 | It's fascinating 'cause like,

02:49:49.600 | you don't think about these companies internally

02:49:52.240 | but Dropbox to me from the very beginning

02:49:55.160 | was one of my favorite services.

02:49:57.440 | There's certain like programs and online services

02:49:59.840 | that make me happy, make me more efficient

02:50:03.440 | and all that kind of stuff

02:50:04.360 | but one of the powers of those kinds of services,

02:50:07.440 | they disappear.

02:50:08.680 | You're not supposed to think about how it all works

02:50:10.940 | but it's incredible to me

02:50:12.000 | that you can sync stuff effortlessly

02:50:15.760 | across so many machines so quickly

02:50:19.120 | and like don't have to worry about conflicts.

02:50:21.720 | They take care of the,

02:50:23.440 | you know, as a person that comes from a version

02:50:25.880 | of repositories and all that kind of stuff

02:50:27.560 | or merge is super difficult

02:50:30.120 | and just keeping different versions of different files

02:50:33.280 | is very tricky.

02:50:34.120 | The fact that they could take care of that is just,

02:50:35.880 | I don't know.

02:50:36.920 | The engineering behind the scenes must be super difficult

02:50:40.440 | both on the compute infrastructure and the software.

02:50:43.520 | - A lot of internal sort of hand-wringing

02:50:47.080 | about things like that

02:50:49.100 | but the product itself always worked very smoothly.

02:50:53.440 | - Yeah.

02:50:54.560 | Well, there's probably a lot of lessons to that.

02:50:56.720 | You can have a lot of turmoil inside

02:50:58.720 | on the engineering side

02:50:59.920 | but if the product is good, the product is good

02:51:03.040 | and maybe don't mess with that either.

02:51:05.600 | You know, when it's good,

02:51:06.840 | it's like with Google, focus on the search and the ads.

02:51:10.760 | Right?

02:51:12.800 | - The money will come.

02:51:13.840 | - Yeah.

02:51:14.680 | And to make sure that's done extremely well

02:51:16.520 | and don't forget what you do extremely well

02:51:19.640 | in what ways do you provide value and happiness

02:51:23.120 | to the world?

02:51:23.960 | Make sure you do that well.

02:51:25.700 | Is there something else to say about Google and Microsoft?

02:51:29.680 | Microsoft has had a very fascinating shift recently

02:51:33.320 | with a new CEO,

02:51:35.400 | with, you know, recent CEO,

02:51:37.760 | with purchasing GitHub,

02:51:39.940 | embracing open source culture,

02:51:41.760 | embracing the developer culture.

02:51:43.320 | It's pretty interesting to see.

02:51:44.560 | - That's like why I joined Microsoft.

02:51:47.600 | I mean, after retiring and thinking

02:51:50.400 | that I would stay retired for the rest of my life,

02:51:53.780 | which of course was a ridiculous thought,

02:51:56.360 | but I was done working for a bit

02:51:59.840 | and then the pandemic made me realize

02:52:01.880 | that work can also provide a source of fulfillment,

02:52:06.380 | keep you out of trouble.

02:52:11.280 | Microsoft is a very interesting company

02:52:14.120 | because it has this incredible,

02:52:17.400 | very long and varied history

02:52:21.160 | and this amazing catalog of products

02:52:25.200 | that many of which also date way back.

02:52:30.000 | I mean, I've been talking to a bunch of Excel people lately

02:52:35.000 | and Excel is like 35 years old

02:52:39.960 | and they can still read spreadsheets

02:52:42.280 | that they might find on an old floppy drive.

02:52:46.500 | - Yeah.

02:52:49.000 | Yeah, there's, man,

02:52:49.840 | there've been so many incredible tools through the years.

02:52:53.360 | Excel, one of the great shames of my life

02:52:57.640 | is that I've never learned how to use Excel well.

02:53:02.080 | I mean, it just always felt like so many features are there.

02:53:05.640 | It's similar with ideas like PyCharm.

02:53:08.680 | It feels like I converge quickly

02:53:11.200 | to the dumbest way to use a thing to get the job done

02:53:14.120 | when clearly there's so much more power at your fingertips.

02:53:17.760 | - Yeah.

02:53:18.600 | - But I do think there's probably expert users of Excel.

02:53:22.280 | - Oh, Excel is a cash cow actually.

02:53:26.040 | - Oh, it actually brings in money.

02:53:27.160 | Oh, that's interesting. - Oh, yeah.

02:53:28.760 | A lot of the engineering,

02:53:31.000 | sort of if you look deep inside Excel,

02:53:33.640 | there's some very good engineering,

02:53:36.480 | very, very impressive stuff.

02:53:39.720 | - Okay, now I need to definitely learn Excel a little better.

02:53:42.960 | I had issues because I'm a keyboard person,

02:53:45.080 | so I had issues coming up with shortcuts.

02:53:47.760 | I mean, Microsoft sometimes, it's changed over the years,

02:53:51.420 | but sometimes they kind of want to make things easier

02:53:54.000 | for you on the surface,

02:53:55.760 | and therefore make it harder for like people

02:54:00.600 | that like to have shortcuts and all that kind of stuff

02:54:03.320 | to optimize their workflow.

02:54:05.120 | Now, Excel's probably, people are probably yelling at me.

02:54:07.760 | It's like, no, Excel probably has a lot of ways

02:54:10.000 | to optimize workflow.

02:54:11.080 | - In fact, I keep discovering

02:54:13.240 | that there are many features in Excel

02:54:15.360 | that only exists at keyboard shortcuts.

02:54:18.340 | - Yeah, that's the sense I have.

02:54:21.040 | And now, like I'm embarrassed that it's just--

02:54:23.680 | - You just have to know what they are.

02:54:25.280 | - Yeah.

02:54:26.120 | - That's like, there's no logic or reason

02:54:30.480 | to the assignment of the keyboard shortcuts

02:54:32.680 | because they go back even longer than 35 years.

02:54:37.160 | - Can you maybe comment about Satya Nadella

02:54:39.880 | and how hard it is for a CEO to sort of pivot a company

02:54:43.480 | towards open source, towards developer culture?

02:54:45.640 | Is there something you could see about like,

02:54:48.200 | what's the role of leadership in such a pivot

02:54:52.280 | and definition of a new vision?

02:54:54.120 | - I've never met him, but I hear he's just a really sharp,

02:54:59.720 | he's just a really sharp thinker,

02:55:03.800 | but he also has an incredible business sense.

02:55:09.000 | He took the organization that had very solid pieces,

02:55:12.800 | but that was also struggling

02:55:17.120 | with all sorts of shameful things,

02:55:20.520 | especially the Steve Ballmer time.

02:55:23.640 | I imagine in part through his personal charm and thinking,

02:55:27.480 | and of course the great trust

02:55:29.960 | that the rest of the leadership has in him,

02:55:32.920 | he managed to really turn the company around

02:55:36.160 | and sort of change it from openly hostile to open source

02:55:41.160 | to actively embracing open source.

02:55:47.040 | And that doesn't mean that suddenly Excel

02:55:49.280 | is going to go open source,

02:55:51.440 | but that means that there's room for a product like VS Code,

02:55:54.640 | which is open source.

02:55:56.840 | - Yeah, that's fascinating.

02:55:57.880 | It gives me faith that large companies

02:56:01.320 | with good leadership can grow, can expand,

02:56:04.480 | can change and pivot and so on, develop,

02:56:07.720 | 'cause it gets harder and harder as the company gets large.

02:56:10.720 | You wrote a blog post in response to a person

02:56:13.960 | looking for advice about whether with a CS degree

02:56:16.640 | to choose a nine to five job or to become an entrepreneur.

02:56:21.520 | It's an interesting question.

02:56:23.000 | If you just think from first principles right now,

02:56:26.200 | somebody has took a few years in programming,

02:56:29.120 | has loved software engineering,

02:56:31.040 | in some sense creating Python is an entrepreneurial endeavor.

02:56:35.520 | That's a choice that a lot of people

02:56:39.000 | that are good programmers have to make.

02:56:40.640 | Do I work for a big company or do I create something new?

02:56:45.480 | - Or you can work for a big company

02:56:50.160 | and create something new there.

02:56:52.200 | - Oh, inside the-

02:56:54.520 | - Yeah, I mean, big companies have individuals

02:56:58.600 | who create new stuff that eventually grows big all the time.

02:57:03.600 | - And if you're the person that creates a new thing

02:57:06.120 | and grows big, you'll have a chance

02:57:08.360 | to move up quickly in the company to run that thing.

02:57:11.280 | - If that's your aspiration, what can also happen

02:57:15.720 | is that someone is a brilliant engineer

02:57:19.320 | and sort of builds a great first version of a product

02:57:25.320 | and has no aspirations to then become a manager

02:57:30.320 | and grow the team from five people to 20 people

02:57:33.880 | to 100 people to 1000 people

02:57:36.120 | and be in charge of hiring and meetings.

02:57:40.400 | And they move on to inventing another crazy thing

02:57:45.200 | inside the same company,

02:57:46.800 | or sometimes they found a startup

02:57:51.200 | or they move to a different great large or small company.

02:57:55.920 | There's all sorts of models.

02:57:58.560 | And sometimes people sort of do have this whole trajectory

02:58:03.400 | from engineer buckling down, writing code,

02:58:07.520 | not nine to five, but more like noon till midnight,

02:58:13.360 | seven days a week, and coming up with a product

02:58:18.400 | and sort of staying in charge.

02:58:22.840 | I mean, if you take Drew Houston, Dropbox's founder,

02:58:27.240 | he is still the CEO.

02:58:28.800 | And at least when I was there,

02:58:32.000 | he had not checked out or anything.

02:58:33.920 | He was a good CEO, but he had started out

02:58:39.280 | as the technical inventor or co-inventor.

02:58:43.120 | And so he was someone who, I don't know,

02:58:47.200 | if he always aspired that, I think when he was 16,

02:58:50.760 | he already started a company.

02:58:52.240 | So maybe he did, but he sort of,

02:58:54.960 | it turned out that he did have the personal

02:59:00.600 | sort of skillset needed to grow and stay on top.

02:59:05.600 | And other people sort of are brilliant engineers

02:59:10.640 | and horrible at management.

02:59:12.200 | I count myself at least in the second category.

02:59:16.160 | - So your first love and still your love

02:59:19.400 | is to be the quote unquote individual contributor.

02:59:22.480 | So the programmer.

02:59:23.400 | - Yep.

02:59:24.240 | - Do you have advice for a programming beginner

02:59:29.640 | on how to learn Python the right way?

02:59:32.520 | - Find something you actually want to do with it.

02:59:41.880 | If you say, I want to learn skill X,

02:59:46.600 | that's not enough motivation.

02:59:48.920 | You need to pick something,

02:59:50.720 | and it can be a crazy problem you want to solve.

02:59:55.400 | It can be completely unrealistic.

02:59:57.800 | But something that challenges you

03:00:03.680 | into actually learning coding in some language.

03:00:11.200 | - And there's so many projects out there

03:00:12.760 | you can look for, like that doesn't have to be

03:00:14.560 | some big ambitious thing.

03:00:15.760 | It could be writing a small bot.

03:00:18.520 | If you're into social media,

03:00:19.640 | you can write a Reddit bot or a Twitter bot

03:00:21.960 | or some aspect of automating something

03:00:26.800 | that you do every single day,

03:00:28.640 | processing files, all that kind of stuff.

03:00:30.560 | - Nowadays, you can take machine learning components

03:00:34.200 | and sort of plug those things together.

03:00:38.880 | So you can do cool stuff with them.

03:00:40.200 | - So that's actually a really good example.

03:00:41.920 | So if you're interested in machine learning,

03:00:43.480 | the state of machine learning is such

03:00:45.680 | that a tutorial that takes an hour

03:00:49.400 | can get you to start using pre-trained models

03:00:53.080 | to do something super cool.

03:00:54.680 | And that's a good way to learn Python

03:00:56.200 | 'cause you learn just enough to run this model,

03:00:58.600 | and that's a sneaky way to get in there

03:01:01.600 | to figure out how to import stuff,

03:01:04.200 | how to write basic I/O, how to run functions.

03:01:09.800 | I'm not sure if it's the best way

03:01:11.280 | to learn the basics in Python,

03:01:13.280 | but it could be nice to just fall in love first

03:01:15.920 | and then figure out the basics, right?

03:01:17.600 | - Yeah, you can't expect to learn Python

03:01:21.200 | from a one-hour video.

03:01:23.120 | Kind of blanking out on the name of someone

03:01:26.440 | who wrote a very funny blog post

03:01:31.440 | where he said, "I see all these ads for things

03:01:35.320 | like learn Python in 10 days or so."

03:01:40.320 | And he said, "The goal should be learn Python in 10 years."

03:01:45.240 | - That's hilarious, but I completely disagree with that.

03:01:49.160 | I think the criticism behind that is that

03:01:51.480 | the places just like the blog post from earlier,

03:01:55.560 | the places that tell you learn Python

03:01:57.280 | in five minutes or 10 minutes,

03:01:58.880 | they're actually usually really bad tutorials.

03:02:01.040 | So the thing is, I do believe that you can learn a thing

03:02:05.360 | in an hour to get some interesting, quick, it hooks you.

03:02:11.680 | But it just takes a tremendous amount of skill

03:02:14.560 | to be that kind of educator.

03:02:16.080 | Richard Feynman was able to condense a lot of ideas

03:02:18.880 | in physics in a very short amount of time,

03:02:21.120 | but that takes a deep, deep understanding.

03:02:23.200 | And so yes, of course, the actual,

03:02:25.440 | I think the 10 years is about the experience,

03:02:29.800 | the pain along the way, and there's something fundamental.

03:02:31.920 | - Well, you have to practice.

03:02:33.200 | You can memorize the syntax, but, well, I couldn't,

03:02:37.840 | but maybe someone else can,

03:02:39.820 | but that doesn't make you a coder.

03:02:42.240 | - Yeah, actually, coding has changed in fascinating ways

03:02:46.520 | 'cause so much of coding is copying, pasting

03:02:49.600 | from Stack Overflow and then adjusting,

03:02:52.240 | which is another way of coding.

03:02:53.560 | And I don't wanna talk down to that kind of style of coding

03:02:56.360 | because it's kind of nicely efficient.

03:02:58.680 | But you know where that is going?

03:03:00.520 | - Code generation?

03:03:03.360 | - No, seriously, GitHub Copilot.

03:03:05.280 | - Yeah, Copilot.

03:03:06.200 | - I use it every day and it-

03:03:08.440 | - Really?

03:03:09.280 | - Yeah, it writes a lot of code for me.

03:03:11.880 | And usually it's slightly wrong,

03:03:13.680 | but it still saves me typing

03:03:16.120 | because all I have to do is change one word

03:03:20.000 | in a line of text that otherwise it generated perfectly.

03:03:23.760 | And how many times are you looking for,

03:03:27.920 | oh, what was I doing this morning?

03:03:29.800 | I was looking for an begin marker

03:03:31.480 | and I was looking for an end marker.

03:03:34.120 | And so begin is blah, blah, blah, search for begin.

03:03:39.120 | This is the begin token.

03:03:43.240 | And then the next line I type E

03:03:46.520 | and it completes the whole line with end instead of begin.

03:03:51.200 | That's a very simple example.

03:03:52.720 | Sometimes it sort of, if I name my function right,

03:03:56.480 | it writes a five or 10 line function.

03:03:58.680 | - And you know Python enough

03:04:03.560 | to very quickly then detect the issues.

03:04:06.440 | It becomes a really good dance partner then.

03:04:09.080 | - It doesn't save me a lot of thinking,

03:04:11.080 | but since I'm a poor typist,

03:04:13.080 | I'm very much appreciative of all the typing it does for me.

03:04:18.080 | Much better actually than the previous generation

03:04:23.360 | of suggestions that are also still built in VS Code

03:04:26.760 | where when you hit like a dot,

03:04:29.960 | it tries to guess what the type is of the variable

03:04:34.640 | to the left of the dot.

03:04:35.760 | And then it gives you a list,

03:04:37.320 | a pop down menu of what the attributes of that object are.

03:04:42.120 | But Copilot is much, much smoother than that.

03:04:44.680 | - Well, it's fascinating to hear

03:04:46.360 | that you use GitHub Copilot.

03:04:49.320 | Do you think, do you worry about the future of that?

03:04:52.720 | Did the automatic code generation,

03:04:56.400 | the increasing amount of that kind of capability,

03:04:59.880 | are programmers jobs threatened

03:05:03.600 | or is there still a significant role for humans?

03:05:05.440 | - Are programmers jobs threatened

03:05:07.320 | by the existence of Stack Overflow?

03:05:09.480 | I don't think so.

03:05:12.280 | It helps you take care of the boring stuff

03:05:14.640 | and you shouldn't try to use it to do something

03:05:18.920 | that you have no way of understanding what you're doing yet.

03:05:23.320 | A tool like that is always best

03:05:26.600 | when the question you're asking is,

03:05:29.640 | please remind me of how I do this,

03:05:32.920 | which I could do, I could look up how to do it,

03:05:37.920 | but right now I've forgotten

03:05:40.840 | whether the method is called foo or bar

03:05:44.160 | or what the shape of the API is.

03:05:47.320 | Does it use a builder object or a constructor

03:05:50.920 | or a factory or something else?

03:05:55.760 | And what are the parameters?

03:05:57.800 | It serves that role.

03:05:59.280 | It's like a great assistant,

03:06:02.040 | but the creative work of sort of deciding

03:06:05.040 | what you want the code to do is totally yours.

03:06:09.200 | - What do you think is the future of Python

03:06:12.440 | in the next 10, 20, 50 years, 100 years?

03:06:15.080 | You look forward.

03:06:16.400 | You ever imagine a future of human civilization

03:06:20.840 | or living inside the metaverse on Mars,

03:06:25.560 | humanoid robots everywhere?

03:06:27.160 | What part does Python play in that?

03:06:29.440 | - It'll eventually become sort of a legacy language

03:06:35.000 | that plays an important role,

03:06:36.880 | but that most people have never heard of

03:06:39.280 | and don't need to know about,

03:06:41.840 | just like all kinds of basic structures in biology,

03:06:46.840 | like mitochondria.

03:06:51.120 | - So it permeates all of life, all of digital life,

03:06:55.440 | but people just build on top of it

03:06:57.680 | and they only know the stuff that's on top of it.

03:06:59.680 | - Yeah.

03:07:01.280 | - Because you build layers of abstractions.

03:07:03.160 | I mean, most programmers nowadays

03:07:06.800 | rarely need to do binary arithmetic, right?

03:07:11.280 | - Yeah.

03:07:12.120 | - Yeah, or even think about it or even learn about it,

03:07:16.800 | or they can go quite far without knowing.

03:07:19.760 | - I started building little digital circuits

03:07:24.120 | out of NAND gates that I built myself

03:07:26.920 | with transistors and resistors.

03:07:29.240 | So I'd sort of, I feel very blessed that

03:07:33.480 | with that start when I was a teenager,

03:07:38.240 | I learned some of the basic, at least concepts

03:07:43.000 | that go into building a computer.

03:07:46.400 | And I sort of, every part,

03:07:49.200 | I have some understanding what it's for

03:07:54.560 | and why it's there and how it works.

03:07:57.440 | And I can forget about all that most of the time,

03:08:00.080 | but I sort of, I enjoy knowing, oh, if you go deeper,

03:08:05.080 | at some point you get to NAND gates

03:08:08.520 | and half adders and shift registers.

03:08:11.760 | And when it comes to the point of how do you actually

03:08:16.080 | make a chip out of silicon, I have no idea.

03:08:18.840 | That's just magic to me.

03:08:20.280 | - But you enjoy knowing that you can walk a while

03:08:23.640 | towards the lower and lower layers, but you don't need to.

03:08:27.520 | It's nice.

03:08:28.360 | - The other day as a sort of a mental exercise,

03:08:32.360 | I was trying to figure out if I could build

03:08:35.640 | flip-flop circuits out of relays.

03:08:42.640 | I was just sort of trying to remember,

03:08:47.240 | oh, how does a relay work?

03:08:49.280 | Yeah, there's like this electromagnetic force

03:08:52.880 | that pulls a switch open or shut.

03:08:55.120 | And you can have like, it can open one switch

03:09:00.160 | and shut another, and you can have multiple contacts

03:09:05.160 | that go at once.

03:09:07.200 | And how many relays do I really need to sort of represent

03:09:10.960 | one bit of information?

03:09:12.560 | Can the relay just feed on itself?

03:09:14.560 | And it was, I don't think I got to the final solution,

03:09:18.480 | but it was fun that I could still do a little bit

03:09:23.400 | of problem solving and thinking at that level.

03:09:26.960 | - And it's cool how we build on top of each other.

03:09:29.520 | So there's people that are just,

03:09:31.520 | you stood on the shoulders of giants

03:09:33.520 | and there's others who'll stand on your shoulders,

03:09:35.400 | and it's a giant, beautiful hierarchy.

03:09:38.360 | - Yeah, I feel I sort of covered this middle layer

03:09:41.920 | of the technology stack where it sort of peters out

03:09:45.920 | below the level of NAND gates.

03:09:50.920 | And at the top, I sort of, I lose track

03:09:54.680 | when it gets to machine learning.

03:09:56.920 | - And then eventually the machine learning

03:09:58.640 | will build higher and higher layers

03:10:00.800 | that will help us understand the lowest layer

03:10:03.120 | of the physics, and thereby the universe figures out

03:10:07.080 | how it itself works.

03:10:10.120 | - Maybe, maybe not.

03:10:12.560 | Yeah, I did, I mean, it's possible.

03:10:14.920 | I mean, if you think of human consciousness,

03:10:17.840 | if that's even the right concept,

03:10:20.100 | it's interesting that sort of we have this

03:10:25.920 | super parallel brain that does all these

03:10:29.400 | incredible parallel operations like image recognition.

03:10:33.480 | I recognize your face.

03:10:35.840 | Does huge amount of processing that goes on in parallel.

03:10:40.120 | There's lots of nerves between my eyes and my brain,

03:10:43.560 | and the brain does a whole bunch of stuff all at once,

03:10:46.360 | because it's actually really slow circuits,

03:10:48.960 | but there are many of them that all work together.

03:10:51.460 | On the other hand, when I'm speaking,

03:10:54.880 | everything is completely sequential.

03:10:57.220 | I have to sort of string words together one at a time,

03:11:03.360 | and when I'm thinking about stuff,

03:11:07.040 | when I'm understanding the world,

03:11:09.100 | I'm also thinking of everything like one step at a time.

03:11:13.880 | And so we've sort of, we've got all this incredible

03:11:19.880 | parallel circuitry in our brains,

03:11:23.540 | and eventually we use that to simulate

03:11:26.680 | a single threaded, much, much higher level interpreter.

03:11:31.680 | - That's exactly, I mean, that's the illusion of it.

03:11:36.760 | That's the illusion of it for us,

03:11:39.280 | that it's a single sequential set of thoughts,

03:11:42.560 | and all of that came from a single cell

03:11:45.260 | through the process of embryogenesis,

03:11:47.320 | so DNA is the code.

03:11:49.540 | DNA holds the entirety of the code.

03:11:53.240 | The information and how to use that information

03:11:56.520 | to build up an organism, the entire like,

03:11:58.620 | the arms, the legs. - How is it built?

03:12:01.720 | - Yeah, the brain.

03:12:03.040 | And so you don't buy a computer, you buy like a--

03:12:06.920 | - You buy a seed, a diagram.

03:12:10.080 | - And then you plant the computer,

03:12:11.960 | and it builds itself in almost the same way,

03:12:15.560 | and then does the computation,

03:12:17.560 | and then eventually dies.

03:12:21.560 | It gets stale, but gives birth to young computers

03:12:25.720 | more and more, and gives them lessons,

03:12:27.500 | but they figure stuff out on their own,

03:12:29.400 | and over time it goes on that way.

03:12:32.560 | And those computers, when they go to college,

03:12:35.000 | try to figure out how to program,

03:12:36.840 | and they built their own little computers.

03:12:38.960 | They're increasingly more intelligent,

03:12:40.680 | increasingly higher and higher levels of abstractions.

03:12:44.040 | - Isn't it interesting that you sort of,

03:12:46.700 | you see the same thing appearing at different levels, though,

03:12:51.360 | because you have like,

03:12:53.000 | cells that create new cells,

03:12:58.220 | and eventually that builds a whole organism,

03:13:02.400 | but then the animal, or the plant, or the human,

03:13:06.280 | has its own mechanism of replication,

03:13:11.040 | that is sort of connected in a very complicated way

03:13:16.040 | to the mechanism of replication of the cells.

03:13:19.520 | And then if you look inside the cell,

03:13:22.600 | if you see how DNA and proteins are connected,

03:13:26.640 | then there is yet another completely different mechanism

03:13:29.860 | whereby proteins are mass-produced

03:13:33.520 | using enzymes and a little bit of code from DNA.

03:13:39.640 | And of course, viruses break into it at that level.

03:13:44.160 | - And while the mechanisms might be different,

03:13:46.940 | it seems like the nature of the mechanism is the same,

03:13:51.640 | and it carries across natural languages,

03:13:55.000 | and programming languages, humans,

03:13:59.440 | maybe even human civilizations,

03:14:01.340 | or intelligent civilizations,

03:14:03.560 | and then all the way down to the single-cell organisms.

03:14:07.720 | - It is fascinating to see what abstraction levels

03:14:12.160 | are built on top of individual humans,

03:14:15.840 | and how you have whole societies

03:14:18.600 | that sort of have a similar self-preservation,

03:14:24.500 | I don't know what it is, instinct, nature, abstraction,

03:14:30.100 | as the individuals have and the cells have.

03:14:33.400 | - And they self-replicate and breed in different ways.

03:14:37.120 | It's hard for us humans to introspect it,

03:14:38.880 | 'cause we were very focused

03:14:40.360 | on our particular layer of abstraction.

03:14:42.920 | But from an alien perspective,

03:14:45.040 | looking on Earth, they'll probably see

03:14:48.480 | the higher-level organism of human civilization

03:14:51.720 | as part of this bigger organism of life on Earth itself.

03:14:55.480 | In fact, that could be an organism just alone,

03:14:57.700 | just life, life, life on Earth.

03:15:01.300 | This has been a wild,

03:15:03.960 | both philosophical and technical conversation.

03:15:06.200 | Guido, you're an amazing human being.

03:15:08.480 | You were gracious enough to talk to me

03:15:11.200 | when I was first doing this podcast,

03:15:13.240 | and one of the earliest first people I've talked to,

03:15:16.880 | somebody I admired for a long time.

03:15:18.520 | It's just a huge honor that you did it at that time,

03:15:20.680 | and you do it again.

03:15:21.600 | You're awesome.

03:15:22.440 | - Thank you, Lex.

03:15:23.960 | - Thanks for listening to this conversation

03:15:25.720 | with Guido Van Rossum.

03:15:27.440 | To support this podcast,

03:15:28.600 | please check out our sponsors in the description.

03:15:31.240 | And now, let me leave you with some words from Oscar Wilde.

03:15:34.760 | Experience is the name that everyone gives

03:15:37.400 | to their mistakes.

03:15:39.080 | Thank you for listening, and hope to see you next time.

03:15:41.920 | (upbeat music)

03:15:44.500 | (upbeat music)

03:15:47.080 | [BLANK_AUDIO]

Guido van Rossum: Python and the Future of Programming | Lex Fridman Podcast #341

Chapters