Jim Keller: Most People Don't Think Simple Enough | AI Podcast Clips
00:00:00.000 | - So what have you learned about the human abstractions
00:00:05.000 | from individual functional human units
00:00:09.000 | to the broader organization?
00:00:12.020 | What does it take to create something special?
00:00:15.160 | - Well, most people don't think simple enough.
00:00:18.860 | All right, so do you know the difference
00:00:21.760 | between a recipe and the understanding?
00:00:24.240 | There's probably a philosophical description of this.
00:00:29.280 | So imagine you're gonna make a loaf of bread.
00:00:31.600 | The recipe says, get some flour, add some water,
00:00:34.160 | add some yeast, mix it up, let it rise, put it in a pan,
00:00:37.820 | put it in the oven.
00:00:39.520 | It's a recipe.
00:00:40.360 | Understanding bread, you can understand biology,
00:00:44.840 | supply chains, grain grinders, yeast, physics,
00:00:49.840 | thermodynamics, there's so many levels
00:00:55.700 | of understanding there.
00:00:57.360 | And then when people build and design things,
00:01:00.360 | they frequently are executing some stack of recipes.
00:01:03.800 | And the problem with that is the recipes
00:01:07.040 | all have limited scope.
00:01:09.040 | Like if you have a really good recipe book for making bread,
00:01:11.600 | it won't tell you anything about how to make an omelet.
00:01:14.960 | But if you have a deep understanding of cooking,
00:01:17.480 | then bread, omelets, sandwich,
00:01:23.200 | there's a different way of viewing everything.
00:01:27.800 | And most people, when you get to be an expert at something,
00:01:32.340 | you're hoping to achieve deeper understanding,
00:01:36.520 | not just a large set of recipes to go execute.
00:01:40.040 | And it's interesting to walk groups of people
00:01:42.920 | because executing recipes is unbelievably efficient
00:01:46.720 | if it's what you want to do.
00:01:49.360 | If it's not what you wanna do, you're really stuck.
00:01:53.660 | And that difference is crucial.
00:01:56.720 | And everybody has a balance of,
00:01:59.320 | let's say, deeper understanding of recipes.
00:02:01.080 | And some people are really good at recognizing
00:02:03.880 | when the problem is to understand something deeply.
00:02:06.560 | Does that make sense?
00:02:09.180 | - It totally makes sense.
00:02:10.680 | Does every stage of development,
00:02:12.920 | deep understanding on the team needed?
00:02:15.680 | - Well, this goes back to the art versus science question.
00:02:18.760 | - Sure.
00:02:19.600 | - If you constantly unpack everything
00:02:21.360 | for deeper understanding, you never get anything done.
00:02:24.360 | And if you don't unpack understanding when you need to,
00:02:27.040 | you'll do the wrong thing.
00:02:28.600 | And then at every juncture,
00:02:31.400 | like human beings are these really weird things
00:02:33.360 | because everything you tell them
00:02:35.360 | has a million possible outputs.
00:02:37.240 | And then they all interact in a hilarious way.
00:02:41.240 | - Yeah, it's very nice.
00:02:42.080 | - And then having some intuition
00:02:43.280 | about what do you tell them, what do you do,
00:02:45.080 | when do you intervene, when do you not, it's complicated.
00:02:48.840 | - Right, so--
00:02:50.280 | - It's essentially computationally unsolvable.
00:02:53.320 | - Yeah, it's an intractable problem, sure.
00:02:55.480 | Humans are a mess.
00:02:58.120 | But with deep understanding,
00:03:01.920 | do you mean also sort of fundamental questions
00:03:04.680 | of things like what is a computer?
00:03:09.680 | Or why, like the why questions,
00:03:15.120 | why are we even building this?
00:03:17.600 | Like of purpose?
00:03:18.880 | Or do you mean more like going towards
00:03:22.320 | the fundamental limits of physics,
00:03:24.400 | sort of really getting into the core of the science?
00:03:27.400 | - Well, in terms of building a computer,
00:03:29.640 | think a little simpler.
00:03:31.480 | So common practice is you build a computer,
00:03:34.760 | and then when somebody says I wanna make it 10% faster,
00:03:37.880 | you'll go in and say, all right,
00:03:39.360 | I need to make this buffer bigger,
00:03:40.960 | and maybe I'll add an add unit.
00:03:43.120 | Or I have this thing that's three instructions wide,
00:03:45.480 | I'm gonna make it four instructions wide.
00:03:47.760 | And what you see is each piece
00:03:50.600 | gets incrementally more complicated.
00:03:52.840 | And then at some point you hit this limit,
00:03:57.240 | like adding another feature or buffer
00:03:59.200 | doesn't seem to make it any faster.
00:04:01.320 | And then people say,
00:04:02.160 | well, that's because it's a fundamental limit.
00:04:05.520 | And then somebody else will look at it and say,
00:04:07.080 | well, actually the way you divided the problem up
00:04:09.560 | and the way that different features are interacting
00:04:12.120 | is limiting you, and it has to be rethought, rewritten.
00:04:15.160 | So then you refactor it and rewrite it,
00:04:18.280 | and what people commonly find
00:04:20.200 | is the rewrite is not only faster, but half as complicated.
00:04:23.720 | - From scratch?
00:04:24.560 | - Yes.
00:04:25.400 | - So how often in your career,
00:04:27.520 | but just have you seen as needed,
00:04:29.880 | maybe more generally, to just throw the whole thing out?
00:04:34.400 | - This is where I'm on one end of it,
00:04:37.160 | every three to five years.
00:04:39.240 | - Which end are you on?
00:04:40.480 | - Rewrite more often.
00:04:42.880 | - Rewrite, and three to five years is?
00:04:45.360 | - If you wanna really make a lot of progress
00:04:47.120 | on computer architecture,
00:04:48.240 | every five years you should do one from scratch.
00:04:50.640 | - So where does the x86-64 standard come in?
00:04:57.040 | How often do you?
00:04:58.920 | - I wrote the, I was the co-author of that spec in '98.
00:05:02.480 | That's 20 years ago.
00:05:04.000 | - Yeah, so that's still around.
00:05:06.000 | - The instruction set itself
00:05:07.520 | has been extended quite a few times.
00:05:09.320 | - Yes.
00:05:10.160 | - And instruction sets are less interesting
00:05:12.640 | than the implementation underneath.
00:05:14.900 | There's been, on x86 architecture,
00:05:17.680 | Intel's designed a few,
00:05:18.800 | Ames designed a few, very different architectures.
00:05:22.640 | And I don't wanna go into too much of the detail
00:05:26.640 | about how often, but there's a tendency
00:05:30.740 | to rewrite it every 10 years,
00:05:32.680 | and it really should be every five.
00:05:34.380 | - So you're saying you're an outlier in that sense,
00:05:37.680 | in the-- - Rewrite more often.
00:05:39.080 | - Rewrite more often.
00:05:40.200 | - Well, and here's the problem.
00:05:41.040 | - Isn't that scary?
00:05:42.240 | - Yeah, of course.
00:05:43.800 | Well, scary to who?
00:05:45.320 | - To everybody involved,
00:05:46.900 | because like you said,
00:05:48.320 | repeating the recipe is efficient.
00:05:50.760 | Companies wanna make money,
00:05:54.280 | no, individual engineers wanna succeed,
00:05:56.460 | so you wanna incrementally improve,
00:05:59.120 | increase the buffer from three to four.
00:06:01.400 | - Well, this is where you get
00:06:02.840 | into diminishing return curves.
00:06:05.560 | I think Steve Jobs said this, right?
00:06:07.040 | So every, you have a project,
00:06:09.100 | and you start here, and it goes up,
00:06:10.720 | and you have diminishing return.
00:06:12.480 | And to get to the next level,
00:06:13.720 | you have to do a new one,
00:06:14.900 | and the initial starting point
00:06:16.880 | will be lower than the old optimization point,
00:06:20.260 | but it'll get higher.
00:06:21.980 | So now you have two kinds of fear,
00:06:23.680 | short-term disaster and long-term disaster.
00:06:27.680 | - And you're-- - So grown-ups, right?
00:06:31.240 | Like, people with a quarter-by-quarter business objective
00:06:35.400 | are terrified about changing everything,
00:06:38.000 | and people who are trying to run a business
00:06:41.200 | or build a computer for a long-term objective
00:06:44.120 | know that the short-term limitations
00:06:46.700 | block them from the long-term success.
00:06:49.500 | So if you look at leaders of companies
00:06:52.880 | that had really good long-term success,
00:06:55.320 | every time they saw that they had to redo something,
00:06:57.760 | they did.
00:06:59.120 | - And so somebody has to speak up?
00:07:01.200 | - Or you do multiple projects in parallel.
00:07:03.200 | Like, you optimize the old one
00:07:04.640 | while you build a new one,
00:07:06.840 | but the marketing guys, they're always like,
00:07:08.600 | promise me that the new computer
00:07:10.080 | is faster on every single thing.
00:07:12.840 | And the computer architect says,
00:07:14.040 | well, the new computer will be faster on the average.
00:07:16.880 | But there's a distribution of results and performance,
00:07:19.600 | and you'll have some outliers that are slower.
00:07:22.040 | And that's very hard 'cause they have one customer
00:07:23.880 | who cares about that one.
00:07:25.120 | (upbeat music)
00:07:27.700 | (upbeat music)
00:07:30.280 | (upbeat music)
00:07:32.860 | (upbeat music)
00:07:35.440 | (upbeat music)
00:07:38.020 | (upbeat music)
00:07:40.600 | [BLANK_AUDIO]