Luís and João Batalha: Fermat's Library and the Art of Studying Papers

00:00:00.000 | The following is a conversation with Louise and João Botala,

00:00:03.920 | brothers and co-founders of Fermat's Library, which is an incredible platform for annotating papers.

00:00:10.400 | As they write on the Fermat's Library website, "Just as Pierre de Fermat scribbled his famous

00:00:16.880 | last theorem in the margins, professional scientists, academics, and citizen scientists

00:00:21.920 | can annotate equations, figures, ideas, and write in the margins."

00:00:26.400 | Fermat's Library is also a really good Twitter account to follow. I highly recommend it.

00:00:32.000 | They post little visual factoids and explorations that reveal the beauty of mathematics. I love it.

00:00:39.840 | Quick mention of our sponsors. Skiff, SimpliSafe, Indeed, NetSuite, and 4Sigmatic.

00:00:47.760 | Check them out in the description to support this podcast.

00:00:50.880 | As a side note, let me say a few words about the dissemination of scientific ideas.

00:00:56.000 | I believe that all scientific articles should be freely accessible to the public. They currently

00:01:02.320 | are not. In one analysis I saw, more than 70% of published research articles are behind a paywall.

00:01:08.480 | In case you don't know, the funders of the research, whether that's government or industry,

00:01:14.640 | aren't the ones putting up the paywall. The journals are the ones putting up the paywall,

00:01:20.000 | while using unpaid labor from researchers for the peer review process.

00:01:25.520 | Where is all that money from the paywall going? In this digital age, the costs here should be

00:01:31.520 | minimal. This cost can easily be covered through donation, advertisement, or public funding of

00:01:37.520 | science. The benefit versus the cost of all papers being free to read is obvious, and the fact that

00:01:43.120 | they're not free goes against everything science should stand for, which is the free dissemination

00:01:48.640 | of ideas that educate and inspire. Science cannot be a gated institution. The more people can freely

00:01:56.560 | learn and collaborate on ideas, the more problems we can solve in the world together, and the faster

00:02:02.160 | we can drive old ideas out and bring new, better ideas in. Science is beautiful and powerful,

00:02:10.080 | and its dissemination in this digital age should be free. This is the Lex Friedman Podcast,

00:02:17.200 | and here's my conversation with Luis and Joao Batalla. Luis, you suggested an interesting idea.

00:02:24.960 | Imagine if most papers had a backstory section, the same way that they have an abstract. So,

00:02:32.960 | knowing more about how the authors ended up working on a paper can be extremely insightful.

00:02:38.720 | And then you went on to give a backstory for the Feynman QED paper. This is all in a tweet,

00:02:43.760 | by the way. We're doing tweet analysis today. How much of the human backstory do you think is

00:02:48.320 | important in understanding the idea itself that's presented in the paper or in general?

00:02:53.680 | - I think this gives way more context to the work of scientists. I think a lot of people have this

00:03:00.800 | almost kind of romantic misconception that the way a lot of scientists work is almost as the sum of

00:03:08.320 | eureka moments where all of a sudden they sit down and start writing two papers in a row,

00:03:13.200 | and the papers are usually isolated. And when you actually look at it, the papers are chapters of a

00:03:19.440 | way more complex story. And the Feynman QED paper is a good example. So, Feynman was actually going

00:03:26.800 | through a pretty dark phase before writing that paper. He lost enthusiasm with physics and doing

00:03:33.840 | physics problems. And there was one time when he was in the cafeteria of Cornell, and he saw a guy

00:03:39.600 | that was throwing plates in the air. And he noticed that when the plate was in the air, there

00:03:44.720 | were two movements there. The plate was wobbling, but he also noticed that the Cornell symbol was

00:03:50.480 | rotating. And he was able to figure out the equations of motions of those plates. And that

00:03:58.240 | led him to kind of think a little bit about electron orbits in relativity, which led to

00:04:08.080 | about quantum electrodynamics. So, that kind of reignited his interest in physics,

00:04:13.840 | and ended up publishing the paper that led to his Nobel Prize, basically. And I think

00:04:20.560 | there are a lot of really interesting backstories about papers that readers never get to know.

00:04:26.160 | For instance, we did a couple of months ago an AMA around a paper, a pretty famous paper,

00:04:34.640 | the Gans paper with Ian Goodfellow. And so, we did an AMA where everyone could ask questions

00:04:40.160 | about the paper, and Ian was responding to those questions. He was also telling the story of how

00:04:46.240 | he got the idea for that paper in a bar. So, that was also an interesting backstory.

00:04:51.760 | I also read a book by Cedric Villani. Cedric Villani is this mathematician, a Fields Medalist.

00:05:00.080 | And in his book, he tries to explain how he got from a PhD student to the Fields Medal,

00:05:06.480 | and he tries to be as descriptive as possible, every single step, how he got to the Fields Medal.

00:05:11.840 | And it's interesting also to see just the amount of random interactions and discussions with other

00:05:17.200 | researchers, sometimes over coffee, and how it led to fundamental breakthroughs in some of his

00:05:23.040 | most important papers. So, I think it's super interesting to have that context of the backstory.

00:05:27.280 | - Well, the Ian Goodfellow story is kind of interesting, and perhaps that's true for Feynman

00:05:31.680 | as well. I don't know if it's romanticizing the thing, but it seems like just a few little

00:05:37.200 | insights and a little bit of work does most of the leap required. Do you have a sense that for a lot

00:05:43.440 | of the stuff you've looked at, just looking back through history, it wasn't necessarily the grind

00:05:50.880 | of like Andrew Wiles or the Fermat's Last Theorem, for example. It was more like a brilliant

00:05:57.120 | moment of insight. In fact, Ian Goodfellow has a kind of sadness to him almost, in that at that

00:06:03.760 | time in machine learning, like at that time, especially for GANs, you could code something

00:06:11.680 | up really quickly on a single machine and almost do the invention, go from idea to experimental

00:06:18.960 | validation in like a single night, a single person could do it. And now there's kind of a sadness

00:06:24.000 | that a lot of the breakthroughs you might have in machine learning kind of require large scale

00:06:28.320 | experiments. So it was almost like the early days. So I wonder how many low hanging fruit there are

00:06:37.120 | in science and mathematics and even engineering where it's like, you could do that little

00:06:43.200 | experiment quickly. Like you have an insight in a bar. Why is it always a bar? But you have an

00:06:47.760 | insight at a bar and then just implement and the world changes. It's a good point. I think it also

00:06:55.040 | depends a lot on the maturity of the field. When you look at a field like mathematics,

00:07:00.400 | like it's a pretty mature field. A field like machine learning, it's growing pretty fast.

00:07:06.400 | And it's actually pretty interesting. I looked up like the number of new papers

00:07:14.640 | on archive with the keyword machine learning and like 50% of those papers have been published

00:07:19.920 | in the last 12 months. So you can see just the sense-

00:07:23.440 | Five zero?

00:07:24.160 | Five zero, 50%. So you can see the magnitude of growth in that field. And so I think like

00:07:31.440 | as fields mature, like those types of moments, I think naturally are less frequent. It's just

00:07:39.040 | a consequence of that. The other point that is interesting about the backstory is that it can

00:07:44.160 | really make it more memorable in a way. And by making it more memorable, it kind of sediments

00:07:49.760 | the knowledge more in your mind. I remember also reading the sort of the backstory to

00:07:54.960 | Dijkstra's shortest path algorithm, right? Where he came up with it essentially while he was

00:08:02.320 | sitting down at a coffee shop in Amsterdam. And he came up with that algorithm over 20 minutes.

00:08:09.680 | And one interesting aspect is that he didn't have any pen or paper at the time. And so he had to do

00:08:13.920 | it all in his mind. And so there's only so much complexity that he can handle if you're just

00:08:18.480 | thinking about it in your mind. And that like when you think about the simplicity of Dijkstra's

00:08:24.000 | shortest path finding algorithm, it's knowing that backstory helps sediment that algorithm

00:08:30.480 | in your mind so that you don't forget about it as easily.

00:08:33.280 | It might be from you that I saw a meme about Dijkstra.

00:08:37.520 | It's like he's trying to solve it and he comes up with some kind of random path. And then

00:08:43.760 | it's like my parents aren't home. And then he does, he figures out the algorithm for the shortest

00:08:49.280 | path. He's trying through words to convey memes, but it's hilarious. I don't know if it's in post

00:08:57.520 | that we construct stories that romanticize it. Apparently when Newton, there was no apple.

00:09:02.400 | Especially when you're working on problems that have a physical manifestation or a visual

00:09:08.160 | manifestation, it feels like the world could be an inspiration to you. So it doesn't have to be

00:09:16.160 | completely on paper. Like you could be sitting at a bar and all of a sudden see something in a pattern

00:09:23.120 | will spark another pattern and you can visualize it and rethink a problem in a particular way.

00:09:29.440 | Of course, you can also load the math that you have on paper and always carry that with you.

00:09:34.560 | So when you show up to the bar, some little inspiration could be the thing that changes it.

00:09:38.720 | Is there any other people almost on the human side, whether it's physics with Feynman,

00:09:44.640 | Dirac, Einstein, or computer science, Turing, anybody else? Any backstories that you remember

00:09:51.600 | that jump out? Because I'm also referring to not necessarily these stories where something

00:09:57.600 | magical happens, but these are personalities. They have big egos. Some of them are super friendly.

00:10:04.160 | Some of them are self-obsessed. Some of them have anger issues. Some of them, how do I describe

00:10:11.360 | Feynman? But he appears to have appreciation of the beautiful in all its forms. He has a wit

00:10:19.040 | and a cleverness and a humor about him. So does that come into play in terms of the construction

00:10:24.400 | of the science? - I think you brought up Newton. Newton is a good example also to think about his

00:10:30.000 | backstory because there's a certain backstory of Newton that people always talk about, but then

00:10:35.600 | there's a whole another aspect of him that is also a big part of the person that he was. But he was

00:10:41.920 | really into alchemy. And he spent a lot of time thinking about that and writing about it. And he

00:10:48.720 | took it very seriously. He was really into Bible interpretation and trying to predict things based

00:10:54.160 | on the Bible. And so there's also a whole backstory then. And of course, you need to look at it in the

00:10:59.040 | context and the time that when Newton lived, but it adds to his personality. And it's important to

00:11:07.280 | also understand those aspects that maybe people are not as proud to teach to little kids, but

00:11:14.880 | it's important. It was part of who he was and maybe without those, who knows what he would have

00:11:19.760 | done otherwise. - Well, the cool thing about alchemy, I don't know how it was viewed at the time,

00:11:27.440 | but it almost like to me symbolizes dreaming of the impossible. Like most of the breakthrough

00:11:34.400 | ideas kind of seem impossible until they're actually done. It's like achieving human flight.

00:11:39.040 | It's not completely obvious to me that alchemy is impossible or like putting myself in the mindset

00:11:45.120 | of the time. And perhaps even still, everything that, some of the most incredible breakthroughs

00:11:54.640 | would seem impossible. And I wonder the value of believing almost like focusing and dreaming

00:12:03.200 | of the impossible such that it is actually is possible in your mind and that in itself manifests,

00:12:09.200 | whether the accomplishing that goal or making progress in some unexpected direction. So alchemy

00:12:16.800 | almost symbolizes that for me. - I distinctly remember having the same thought of thinking,

00:12:21.920 | when I learned about atoms and that they have protons and electrons, I was like, okay, to make

00:12:26.240 | gold, you just take whatever has an atomic weight below it and then shove another proton in there

00:12:30.800 | and then you have a bunch of gold. So like, why don't people do that? It seemed like conceptually

00:12:35.920 | is like, this sounds feasible. You might be able to do it. - And you can actually, it's just very,

00:12:40.800 | very expensive. - Yeah, yeah, exactly. So in a sense, we do have alchemy and maybe even back

00:12:47.440 | then it wasn't as crazy that he was so into it, but people just don't like to talk about that as

00:12:52.800 | much. Yeah, but Newton in general was a very interesting fella. - Anybody else come to mind?

00:12:57.520 | In terms of people that inspire you, in terms of people that you just are happy that they have once

00:13:06.880 | or still exist on this earth. - I think, I mean, Freeman Dyson for me. Yeah, Freeman Dyson was,

00:13:15.360 | I've had a chance to actually exchange a couple of emails with him. He was probably one of the

00:13:19.680 | most humble scientists that I've ever met and that had a big impact on me. We were trying,

00:13:25.360 | we're actually trying to convince him to annotate a paper on Fermat's library and I sent him an email

00:13:33.200 | asking him if he could annotate a paper and his response was something like, I have very limited

00:13:40.080 | knowledge. I just know a couple of things about certain fields. I'm not sure if I'm qualified to

00:13:45.360 | do that. That was his first response. And this was someone that should have won a Nobel Prize

00:13:51.120 | and worked on a bunch of different fields, did some really, really great work. And then just

00:13:57.440 | the interactions that I had with him, every time I asked him a couple of questions about his papers

00:14:02.640 | and he always responded saying, I'm not here to answer your questions. I just want to open more

00:14:09.280 | questions. And so that had a big impact on me. It was like just an example of an extremely humble

00:14:17.200 | yet accomplished scientist. And Feynman was also a big, big inspiration in the sense that he was

00:14:25.680 | able to be, again, extremely talented and scientist, but at the same time, socially,

00:14:32.080 | he was able to, he was also really smart from a social perspective and he was able to interact

00:14:38.560 | with people. He was also a really good teacher and was also to, did awesome work in terms of

00:14:45.520 | explaining physics to the masses and motivating and getting people interested in physics.

00:14:52.400 | And that for me was also a big inspiration. - Yeah, I like the childlike curiosity of some

00:14:57.760 | of those folks, like you mentioned, Feynman. I have Daniel Kahneman, I got a chance to meet

00:15:01.920 | and interact with. Some of these truly special scientists, what makes them special is that even

00:15:08.320 | in older age, there's still that fire of childlike curiosity that burns. And some of that is like

00:15:18.000 | not taking yourself so seriously that you think you've figured it all out, but almost like thinking

00:15:22.880 | that you don't know much of it. And that's like step one in having a great conversation or

00:15:31.680 | collaboration or exploring a scientific question. It's cool how the very thing that probably earned

00:15:38.240 | people the Nobel prize or work that's seminal in some way is the very thing that still burns even

00:15:46.320 | after they've won the prize. It's cool to see. And they're rare humans, it seems. - And to that

00:15:53.120 | point, I remember like the last email that I sent to Freeman Dyson was like in his last birthday,

00:15:57.840 | he was really into number theory and primes. So what I did is I took like a photo of him,

00:16:04.160 | picture, and then I turned that into like a giant prime number. So I converted the picture into a

00:16:10.880 | bunch of one and eights, and then I moved some numbers around until it was a prime.

00:16:14.800 | And then I sent him that. - Oh, so the visual, like it still

00:16:19.440 | looked like the picture, it was made up of a prime. That's tricky to do. That's hard to do.

00:16:23.760 | - It looks harder than it actually is. So the way you do it is like you convert the darker regions

00:16:29.920 | into eights and the lighter regions in ones. And then there's- - And then you just keep flipping

00:16:35.200 | numbers until- - Yeah, but there's like some

00:16:38.080 | primality tests that are cheaper from a computational standpoint. But what it tells you is

00:16:44.000 | it excludes numbers that are not prime. Then you end up with a set of numbers that you don't know

00:16:48.480 | if they are prime or not. And then you run the full primality test on that. So you just have

00:16:53.200 | to keep iterating on that. And it's funny because when he got the picture, he was like, "How did

00:16:58.160 | you do that?" He was super curious too. And then we got into the details. And again, it was already

00:17:03.840 | 90, I think 92 or something. And that curiosity was still there. So you can really see that in

00:17:12.160 | some of these scientists. - So could we talk about Fermat's library?

00:17:16.000 | - Yeah, absolutely. - What is it? What's the main goal?

00:17:21.280 | What's the dream? - It is a platform for annotating papers

00:17:25.280 | in its essence. And so academic papers can be one of the densest forms of content out there and

00:17:32.000 | generally pretty hard to understand at times. And the idea is that you can make them more accessible

00:17:38.800 | and easier to understand by adding these rich annotations to the side. And so we can just

00:17:43.680 | imagine a PDF view on your browser, and then you have annotations on each side. And then when you

00:17:48.000 | click on them, a sidebar expands and then you have annotations that support LaTeX and Markdown.

00:17:54.160 | And so the idea is that you can say, explain a tougher part of a paper where there's a step

00:17:59.680 | that is not completely obvious, or you can add more context to it. And then over time,

00:18:05.520 | papers can become easier and easier to understand and can evolve in a way. But it really came from

00:18:12.640 | myself, Luis and two other friends. We've had this long running habit of kind of running a

00:18:19.600 | journal club amongst us. We come from different backgrounds. I studied CS, we studied physics. And

00:18:24.720 | so we'd read papers and present them to each other. And then we tried to bring some of that online.

00:18:32.320 | And that's when we decided to build Fermat's library. And then over time, it kind of

00:18:39.200 | grew into something with a broader goal. And really what we're trying to do is trying to help

00:18:49.360 | move science in the right direction. That's really the ultimate goal and where we want to take it now.

00:18:55.920 | So there's a lot to be said. So first of all, for people who haven't seen it,

00:18:59.600 | the interface is exceptionally well done. Execution is really important here.

00:19:05.600 | Absolutely.

00:19:06.560 | The other thing is just to mention for a large number of people, apparently, which is new to me,

00:19:12.960 | don't know what LaTeX is. So it's spelled like latex. So be careful googling it if you haven't

00:19:19.600 | before. It's a, sorry, I don't even know the correct terminology.

00:19:24.640 | Type setting language?

00:19:25.280 | It's a type setting language where you're basically writing a program that then generates

00:19:32.240 | something that looks from a typography perspective, beautiful.

00:19:36.880 | Absolutely.

00:19:37.840 | And so a lot of academics use it to write papers. I think there's like a bunch of communities that

00:19:44.640 | use it to write papers. I would say it's mathematics, physics, computer science.

00:19:49.200 | Yeah.

00:19:50.480 | That's, yeah. That's it.

00:19:51.840 | That's the main-

00:19:52.400 | Because I'm collaborating currently on a paper with two neuroscientists from Stanford.

00:19:57.360 | And they don't know what.

00:19:58.240 | So I'm using Microsoft Word and Mendeley and like all of those kinds of things.

00:20:08.560 | And I'm being very Zen like about the whole process, but it's fascinating.

00:20:13.200 | It's a little heartbreaking actually, because it actually, it's funny to say,

00:20:20.160 | but we'll talk about open science, actually the bigger mission behind it for Mars Libraries, like

00:20:25.120 | really opening up the world of science to everybody.

00:20:28.480 | Is these silly two facts of like one community uses LaTeX and another uses Word,

00:20:37.600 | is actually a barrier between them.

00:20:40.080 | It's like boring and practical in a sense, but this makes it very difficult to collaborate.

00:20:45.440 | Just on that, I think that if there are some people that should have received like a Nobel

00:20:50.720 | Prize, but will never get it. And I think one of those is like Donald Knuth because of tech

00:20:55.920 | and LaTeX then, because it had a huge impact in terms of like just making it easier for

00:21:02.560 | researchers to put their content out there, like making it uniform as much as possible.

00:21:07.680 | Oh, you mean like a Nobel Peace Prize?

00:21:10.000 | Maybe a Nobel Peace Prize.

00:21:11.600 | Maybe a Nobel Peace Prize. Yeah, I think so.

00:21:17.120 | I mean, he had a very young age, got the Turing Award for his work in algorithms and so on.

00:21:21.440 | So like an incredibly, I think it's in, it might be even the sixties, but I think it's the seventies.

00:21:27.760 | So when he was really young and then he went on to do like incredible work with his book and yeah,

00:21:34.480 | with tech that people don't know.

00:21:36.320 | And going back just on the reason why we ended up, because I think this is interesting.

00:21:41.360 | The reason why we ended up using the name Fermat's Library, this was because of Fermat's

00:21:46.560 | Last Theorem. And Fermat's Last Theorem is actually a funny story. So Pierre de Fermat,

00:21:51.440 | he was like a lawyer and he wrote like on a book that he had a solution to Fermat's Last Theorem,

00:21:59.120 | but that didn't fit the margin of that book.

00:22:03.520 | And so Fermat's Last Theorem basically states that there is no solution.

00:22:08.720 | If you have integers a, b and c, there's no solution to a to the power of n plus b to the

00:22:14.480 | power of n equals to c to the power of n if n is bigger than two. So there's no solutions.

00:22:21.920 | And he said that, and that problem remained open for almost 300 years, I believe. And a lot of the

00:22:30.560 | most famous mathematicians tried to tackle that problem. No one was able to figure that out until

00:22:35.680 | Andrew Wiles, I think was in the nineties, was able to publish the solution, which was, I believe,

00:22:41.920 | almost 300 pages long. And so it's kind of an anecdote that, you know, there's a lot of knowledge

00:22:49.200 | and insights that can be trapped in the margins, and there's a lot of potential energy that you

00:22:55.520 | can release if you actually spend some time trying to digest that. And that was the origin story for

00:23:03.920 | the name. - Yes, you can share the contents of the margins with the world.

00:23:08.000 | - Exactly. - That could inspire a solution

00:23:11.360 | or a communication that then leads to a solution. - And if you think about papers, like papers are,

00:23:16.640 | as João was saying, probably one of the densest pieces of text that any human can read. And you

00:23:22.080 | have these researchers, like some of the brightest minds in these fields, working on like new

00:23:27.680 | discoveries and publishing these work on journals that are imposing them restrictions in terms of

00:23:32.800 | the number of pages that they can have to explain a new scientific breakthrough. So at the end of

00:23:38.400 | the day, papers are not optimized for clarity and for a proper explanation of that content,

00:23:46.320 | because there are so many restrictions. So there's, as I mentioned, there's a lot of potential energy

00:23:51.200 | that can be freed if you actually try to digest a lot of the contents of papers.

00:23:57.120 | - Can you explain some of the other things? So margins, librarian, journal club?

00:24:02.240 | - So journal club is what a lot of people know us for, where we, every week, we release an

00:24:07.600 | annotated paper and in all sorts of different fields, but physics, CS, math. Margins is kind of

00:24:13.760 | the same software that we use to run the journal club and to host the annotations, but we've made

00:24:18.880 | that available for free to anybody that wants to use it. And so folks use it at universities and

00:24:25.760 | for running journal clubs. And so we've just made that freely available. And then librarian is a

00:24:33.440 | browser extension that we developed that is sort of an overlay on top of archive. So it's about

00:24:38.560 | bringing some of the same functionality around comments, plus adding some extra niceties to

00:24:46.480 | archive, like being able to very easily extract the references of a paper that you're looking at,

00:24:51.200 | or being able to extract the BibDeck in order to cite that paper yourself. So it's an overlay on

00:24:56.640 | top of archive. - The idea is that you can have that commenting interface without having to leave

00:25:01.280 | archive. - It's kind of incredible. I didn't know about it. And once I learned of it, it's like,

00:25:07.680 | holy shit, why isn't it more popular given how popular archive is? Like everybody should be

00:25:14.240 | using it. Archive sucks in terms of its interface. Or let me rephrase that, it's limited. Yeah.

00:25:20.560 | - Yeah. - In terms of its interface.

00:25:21.600 | - Archive is a pretty incredible project, right? And it is, in a way, it's, you know,

00:25:28.880 | the growth has been completely linear over time. If you look at like number of papers published on

00:25:33.760 | archive, like, you know, it's pretty much a straight line for the past 20 years. Especially

00:25:38.480 | if you're coming from a startup background and then you were trying to do archive,

00:25:43.840 | you'd probably try to like all sorts of growth acts and like try to then maybe like have paid

00:25:49.600 | features and things like that. And that would kind of maybe ruin it. And so there's a subtle

00:25:55.360 | balance there. And I don't know what aspects you can change about it.

00:25:59.280 | - Yeah. For some tools in science, it just takes time for them to grow. Archive is just turned 30,

00:26:05.920 | I believe. And for people that don't know, archive is this kind of online repository where people put

00:26:11.440 | preprints, which are versions of the papers before they actually make it to journals.

00:26:16.800 | - ARXIV. - Exactly.

00:26:20.320 | - For people who don't know. And it's actually a really vibrant place to publish your papers and

00:26:25.440 | in the aforementioned communities of mathematics, physics and computer science.

00:26:30.800 | - It started with mathematics and physics. And then over the last 30 years, it evolved. And now

00:26:36.400 | actually computer science now, it's a more popular category than physics and math on archive.

00:26:42.080 | - And there's also, which I don't know very much about like a biology, medical version of that.

00:26:47.520 | - BioRxiv. Yeah, BioRxiv. - More recent.

00:26:50.560 | - It's interesting because if you look at like these platforms for preprints,

00:26:56.320 | they actually play a super important role. Because if you look at a category like math,

00:27:02.800 | for some papers in math, it might take close to three years after you click upload paper on the

00:27:11.440 | journal website and the paper gets published on the website of the journal. So this is literally

00:27:16.720 | the longest upload period on the internet. And during those three years, that content is just

00:27:26.400 | locked. And so that's why it's so important for people to have websites like ARXIV so that you

00:27:32.560 | can share that before it goes to the journal with the rest of the world. That was actually on ARXIV

00:27:37.440 | that Perelman published the three papers that led to the proof of the Poincaré conjecture.

00:27:44.240 | And then you have other fields like machine learning, for instance, where the field is

00:27:49.440 | evolving at such a high rate that people don't even wait before the papers go to journals before

00:27:54.960 | they start working on top of those papers. So they publish them on ARXIV, then other people

00:27:59.200 | see them, they start working on that. And ARXIV did a really good job at like building that core

00:28:04.800 | platform to host papers. But I think there's a really, really big opportunity in building more

00:28:10.240 | features on top of that platform, apart from just hosting papers. So collaboration, annotations,

00:28:15.520 | like having other things apart from papers like code and other things. Because, for instance,

00:28:23.440 | in the field like machine learning, there's a really big... As I mentioned, people start working

00:28:28.000 | on top of preprints and they are assuming that that preprint is correct. But you really need a

00:28:34.960 | way, for instance, to maybe... It's not peer review, but distinguish what is good work from

00:28:41.280 | bad work on ARXIV. How do you do that? So like a commenting interface like Librarian, it's useful

00:28:46.960 | for that so that you can distinguish that in a field that is growing so fast as machine learning.

00:28:53.440 | And then you have platforms that focus, for instance, on just biology. BioARXIV is a good

00:28:59.040 | example. BioARXIV is also super interesting because there's actually an interesting experiment

00:29:07.520 | that was run in the '60s. So in the '60s, the NIH supported this experiment called the Information

00:29:16.400 | Exchange Group, which at the time was a way for researchers to share biology preprints via mail

00:29:24.320 | or using libraries. And that project in the 1960s got canceled six years after it started. And it

00:29:30.800 | was due to intense pressure from the journals to kill that project because they were fearing

00:29:36.080 | competition from the preprints for the journal industry. Creek was one of the famous scientists

00:29:46.640 | that opposed to the Information Exchange Group. And it's interesting because right now, if you

00:29:53.040 | analyze the number of biology papers that appear first as preprints, it's only 2% of the papers.

00:29:59.600 | And this was almost 50 years after that first experiment. So you can see that pressure from

00:30:06.400 | the journals to cancel that initial version of a preprint repo had a tremendous impact on the

00:30:14.160 | number of papers that are showing up in biology as preprints. So it delayed a lot that revolution.

00:30:21.680 | But now platforms like BioRxiv are doing that work. But there's still a lot of room for growth

00:30:27.040 | there. And I think it's super important because those are the papers that are open,

00:30:30.000 | that everyone can read. - Okay, so but if we just look at the entire process of science as a big

00:30:35.600 | system, can we just talk about how it can be revolutionized? So you have an idea, depending

00:30:43.280 | on the field, you wanna make that idea concrete, you wanna run a few experiments in computer science,

00:30:49.040 | there might be some code, there'd be a data set for some of the more sort of biology, psychology,

00:30:58.000 | you might be collecting the data set, that's called a study, right? So that's part of that,

00:31:06.720 | that's part of the methodology. And so you are putting all of that into a paper form.

00:31:12.640 | And then you have some results. And then you submit that to a place for review through the

00:31:20.400 | peer review process. And there's a process where, how would you summarize the peer review process,

00:31:25.680 | but it's really just like a handful of people look over your paper and comment and based on that,

00:31:31.040 | decide whether your paper is good or not. So there's a whole broken nature to it. At the same

00:31:37.840 | time, I love the peer review process when I buy stuff on Amazon, for like the commenting system,

00:31:46.800 | whatever that is. So, okay, so there's a bunch of possibilities for revolutions there. And then

00:31:51.440 | there's the other side, which is the collaborative aspect of the science, which is people annotating

00:31:56.720 | people commenting sort of the low effort collaboration, which is a comment. Sometimes

00:32:02.880 | as you've talked about, a comment can change everything, but, or a higher effort collaboration,

00:32:08.400 | like more like maybe annotations or even like contributing to the paper. You can think of like

00:32:14.000 | a collaborative updating of the paper over time. So there's all these possibilities for

00:32:21.920 | doing things better than they've been done. Can we talk about some ideas in this space,

00:32:29.200 | some ideas that you're working on, some ideas that you're not yet working on, but should be

00:32:34.400 | revolutionized? Because it does seem that archive and like open review, for example,

00:32:41.120 | are like the Craigslist of science. Like, yeah, okay. I'm very grateful that we have it, but it

00:32:49.840 | just feels like it's like 10 to 20 years. Like it doesn't feel like that's a feature. The simplicity

00:32:56.400 | of it is a feature. It feels like it's a bug. But then again, the pushback there is Wikipedia has

00:33:05.200 | the same kind of simplicity to it. And it seems to work exceptionally well in the crowdsourcing

00:33:13.040 | aspect of it. So, sorry, there's a bunch of stuff going on on the table. Let's just pick random

00:33:18.400 | things that we can talk about. - Wikipedia, for me, it's the cosmological constant of the internet.

00:33:24.640 | I think we are lucky to live in the parallel universe where Wikipedia exists. Because if

00:33:29.680 | someone had pitched me Wikipedia, like a publicly edited encyclopedia, like a couple of years ago,

00:33:36.320 | like it would be, I don't know how many people would have said that that would have survived.

00:33:41.280 | - I mean, it makes almost no sense. It's like having a Google doc that everybody on the internet

00:33:45.840 | can edit. And like, that will be like the most reliable source for knowledge.

00:33:50.240 | - I don't know how many, but hundreds of thousands of topics. - Yeah. It's insane.

00:33:56.880 | - It's insane. And then you have users, like there's a single user that edited one third

00:34:02.960 | of the articles on Wikipedia. So we have these really, really big power users. They are

00:34:07.520 | a substantial part of like what makes Wikipedia successful. And so, like, no one would have ever

00:34:16.640 | imagined that that could happen. And so that's one thing. I completely agree with what you just said.

00:34:23.680 | - Sorry to interrupt briefly. Maybe let's inject that into the discussion of everything else.

00:34:28.880 | I also believe, I've seen that with Stack Overflow, that one individual or a small collection

00:34:35.120 | of individuals contribute or revolutionize most of the community. Like if you create a really

00:34:42.000 | powerful system for archive or like open review and made it really easy and compelling and exciting

00:34:50.960 | for one person who is in like a 10X contributor to do their thing, that's going to change everything.

00:34:57.840 | It seems like that was the mechanism that changed everything for Wikipedia. And that's the mechanism

00:35:02.240 | that changed everything for Stack Overflow is gamifying or making it exciting or just making it

00:35:07.760 | fun or pleasant or fulfilling in some way for those people who are insane enough to like answer

00:35:15.280 | thousands of questions or write thousands of factoids and like research them and check them,

00:35:21.920 | all those kinds of things, or read thousands of papers. - Yeah. No, Stack Overflow is another

00:35:27.120 | great example of that. And it's just, and those are both two incredibly productive communities

00:35:34.080 | that generate a ton of value and capture almost none of it. And in a way it's almost like counter,

00:35:43.520 | it's very counterintuitive that people, that these communities would exist and thrive.

00:35:51.120 | And it's really hard to, there aren't that many communities like that.

00:35:57.840 | - So how do we do that for science? Do you have ideas there? Like what are the biggest

00:36:02.800 | problems that you see? Are you working on some of them? - Just on that, there are a couple of

00:36:06.960 | really interesting experiments that people are running. An example would be like the polymath

00:36:11.680 | projects. So this is a kind of a social experiment that was created by Tim Gowers,

00:36:17.600 | Fields medalist. And his idea was to try to prove that is it possible to do mathematics in a

00:36:24.560 | massively collaborative way on the internet? So he decided to pick a couple of problems and test

00:36:32.240 | that. And they found out that it actually, it is possible for specific types of problems,

00:36:38.080 | namely problems that you're able to break down in little pieces and go step by step. You might need,

00:36:44.320 | as with open source, you might need people that are just kind of reorganizing the house every

00:36:50.960 | once in a while. And then people throw a bunch of ideas and then you make some progress, then you

00:36:56.560 | reorganize, you reframe the problem and you go step by step. But they were actually able to prove

00:37:01.920 | that it is possible to collaborate online and do progress in terms of mathematics. And so I'm

00:37:11.040 | confident that there are other avenues that could be explored here. - Can we talk about peer review,

00:37:15.760 | for example? - Absolutely. I think like in terms of the peer review, I think it's important to look

00:37:22.400 | at the bigger picture here of like, of what the scientific publishing ecosystem looks like.

00:37:30.320 | Because for me, there are a lot of things that are wrong about that entire process. So if you

00:37:36.640 | look at, for instance, at what publishing means in like a traditional journal, you have journals that

00:37:43.520 | pay authors for their articles, and then they might pay like reviewers to review those articles.

00:37:52.400 | And finally, they pay people to, or distributors to distribute the content. In the scientific

00:37:59.680 | publishing world, you have scientists that are usually backed by government grants, they are

00:38:04.800 | giving away their work for free in the form of papers. And then you have other scientists that

00:38:10.160 | are reviewing their work. This process is known as the peer review process, again for free. And

00:38:16.640 | then finally, we have government backed universities and libraries that are buying back all that work

00:38:27.040 | so that other scientists can read. So this is, for me, it's bizarre. You have the government that is

00:38:32.240 | funding the research, is paying the salaries of the scientists, is paying the salaries of the

00:38:36.560 | reviewers, and it's buying back all that product of their work again. And I think the problem with

00:38:43.600 | this system, and it's why it's so difficult to break this suboptimal equilibrium, is because of

00:38:51.680 | the way academia works right now and the way you can progress in your academic life. And so,

00:38:58.720 | in a lot of fields, the competition in academia is really insane. So you have hundreds of PhD

00:39:06.320 | students that are trying to get to a professor position, and it's hyper competitive. And the

00:39:14.320 | only way for you to get there is if you publish papers, ideally in journals, with a high impact

00:39:23.360 | factor. In computer science, it's often conferences that are also very prestigious, or actually more

00:39:28.240 | prestigious than journals now. Okay, interesting. So that's the one discipline where, I mean,

00:39:33.120 | that has to do with the thing we've discussed in terms of how quickly the field turns around.

00:39:38.960 | But like, NeurIPS, CVPR, those conferences are more prestigious, or at the very least,

00:39:45.520 | as prestigious as the journals. But yeah, but doesn't matter. The process is what it is.

00:39:49.840 | So for people that don't know, the impact factor of a journal is basically the average number of

00:39:56.080 | citations that a paper would get if it gets published on that journal. But so, you can

00:40:02.400 | really think that the problem with the impact factor is that it's a way to turn papers into

00:40:10.480 | accounting units. And let me unpack this, because the impact factor is almost like a nobility title.

00:40:18.880 | So because papers are born with impact, even before anyone reads them. So the researchers,

00:40:24.560 | they don't have the incentive to care about if this paper is going to ever have a long-term impact

00:40:31.200 | on the world. What they care, their goal, their end goal is the paper to get published.

00:40:35.680 | So that they get that value up front. So for me, that is one of the problems of that. And that

00:40:42.480 | really creates a tyranny of metrics. Because at the end of the day, if you are a dean, what you

00:40:48.240 | want to hire is like people, researchers that publish papers on journals with high impact

00:40:53.600 | factors, because that will increase the ranking of your university and will allow you to charge

00:40:58.400 | more for tuition, so on and so forth. And that, especially when you are in super competitive areas,

00:41:06.720 | that people will try to gamify that system and misconduct starts showing up.

00:41:12.720 | There's a really interesting book on this topic called Gaming the Metrics. It's a book by a

00:41:19.920 | researcher called Mario Biagioli. It goes a lot into like how the impact factor and metrics

00:41:27.440 | affect science negatively. And it's interesting to think, especially in terms of citations,

00:41:32.400 | if you look at the early work of like looking at citations, there was a lot of work that was

00:41:37.280 | done by a guy called Eugene Garfield. And this guy, the early work in terms of citation, they

00:41:42.560 | wanted to use citations as from a descriptive point of view. So what they wanted to create was

00:41:49.600 | a map. And that map would create a visual representation of influence. So citations

00:41:56.080 | would be links between papers. And ideally what they would show, they would represent is that you

00:42:02.560 | read someone else's paper and it had an impact on your research. They weren't supposed to be counted.

00:42:07.920 | I think this inspired like Larry and Sergey's work, right? For Google.

00:42:12.720 | Exactly. I think they even mentioned that. But what happens is like, as you start counting citations,

00:42:17.680 | you create a market. And the same way, like, and this was the work of Eugene Garfield was a big

00:42:24.240 | inspiration for Larry and Sergey for the page rank algorithm that led to the creation of Google.

00:42:30.960 | And they even recognized that. And if you think about it, it's like the same way there's a gigantic

00:42:36.400 | market for search engine optimization, SEO, where people try to optimize the page rank and how a

00:42:45.360 | web page will rank on Google. The same will happen for papers. People will try to optimize

00:42:51.680 | the impact factors and the citations that they get. And that creates a really big problem.

00:42:57.440 | And it's super interesting to actually analyze them. If you look at the distribution of the

00:43:02.960 | impact factors of journals, you have like nature with nature, I believe it's like in the low 40s.

00:43:09.760 | And then you have, I believe science is high 30s. And then you have a really good set of good

00:43:16.480 | journals that will fall between 10 and 30. And then you have a gigantic tale of journals that

00:43:23.040 | have impact factor below two. And you can really see two economies here. You see the universities

00:43:31.520 | that are maybe less prestigious, less known, that where the faculty are pressured to just

00:43:37.920 | publish papers, regardless of the journal, what I want to do is increase the ranking of my

00:43:42.960 | university. And so they end up publishing as many papers as they can in journals with low impact

00:43:49.200 | factor. And unfortunately, this represents a lot of the global South. And then you have the luxury

00:43:57.520 | good economy. So for instance, and there are also problems here in the luxury good economy.

00:44:03.360 | So if you look at the journal like nature, so with impact factor of like in the low 40s,

00:44:09.600 | there's no way that you're going to be able to sustain that level of impact factor by just

00:44:15.440 | grabbing the attention of scientists. What I mean by that is like, for the journals,

00:44:22.080 | the articles that get published in nature, they need to be New York Times great. So they need to

00:44:29.280 | make it to the big media, they need to be captured by the big media. Because that's the only way for

00:44:35.120 | you to capture enough attention to sustain that level of citations. And that of course, creates

00:44:41.360 | problems because people then will try to, again, gamify the system and have like titles or abstracts

00:44:48.320 | or that are bigger, make claims that are bigger than what is actually can be sustained by the data

00:44:57.120 | or the content of the paper. And you will have clickbait titles or clickbait abstracts. And

00:45:02.720 | again, this is all a consequence of metrics and science of metrics. And this is a very dangerous

00:45:10.320 | cycle that I think it's very hard to break, but it's happening in academia in a lot of fields

00:45:16.240 | right now. - Is it fundamentally the existence of metrics or the metrics just need to be significantly

00:45:21.840 | improved? Because like I said, the metrics used for Amazon for purchasing, I don't know, computer

00:45:30.720 | parts is pretty damn good in terms of selecting which are the good ones, which are not. In that

00:45:35.920 | same way, if we had Amazon type of review system in the space of ideas, in the space of science,

00:45:44.000 | it feels like those metrics would be a little bit better. Sort of when it's significantly more open

00:45:52.800 | to the crowdsource nature of the internet, of the scientific internet, meaning as opposed to,

00:46:00.000 | like my biggest problem with peer review has always been that it's like five, six, seven people,

00:46:06.880 | usually even less. And it's often nobody's incentivized to do a good job in the whole

00:46:13.360 | process. Meaning it's anonymous in a way that doesn't incentivize, like doesn't gamify or

00:46:22.480 | incentivize great work. And also it doesn't necessarily have to be anonymous. Like there

00:46:30.080 | has to be, the entire system doesn't encourage actual sort of rigorous review. For example, like

00:46:39.680 | open review does kind of incentivize that kind of process of collaborative review,

00:46:47.280 | but it's also imperfect. It just feels like the thing that Amazon has, which is like thousands

00:46:52.960 | of people contributing their reviews to a product. It feels like that could be applied to science

00:47:00.640 | where the same kind of thing you're doing with Fermat's library, but doing at a scale that's much

00:47:08.000 | larger. It feels like that should be possible given the number of grad students, given the number of

00:47:16.240 | general public that's, like for example, I personally, as a person who got an education

00:47:22.560 | in mathematics and computer science, like I can be a quote unquote like reviewer on a lot bigger

00:47:32.480 | set of things than is my exact expertise. If I'm one of thousands of reviewers, if I'm the only

00:47:41.600 | reviewer, one of five, then I better be like an expert in the thing. But if I, and I've learned

00:47:47.680 | this with COVID, which is like, you can just use your basic skills as a data analyst as a,

00:47:54.320 | and to contribute to the review process on a particular little aspect of a paper and be able

00:47:59.200 | to comment, be able to sort of draw in some references that challenge the ideas presented

00:48:05.680 | or to enrich the ideas that are presented, or, you know, it just feels like crowdsourcing

00:48:12.240 | the review process would be able to allow you to have metrics in terms of how good a paper is

00:48:20.480 | that are much better representative of its actual impact in the world, of its actual value to the

00:48:26.080 | world, as opposed to some kind of arbitrary gamified version of its impact. - I agree with

00:48:35.440 | that. I think we, there's definitely the possibility, at least for more resilient,

00:48:39.520 | a more resilient system than what we have today. And it's, I think that's kind of what you're

00:48:43.280 | describing, Alex. And I mean, to an extent we kind of have like a little bit of a Heisenberg

00:48:50.160 | uncertainty principle. When you pick a metric, as soon as you do it, then maybe it works as a good

00:48:54.480 | heuristic for a short amount of time, but soon enough people would start gamifying and, but then

00:49:00.480 | you can definitely have metrics that are more resilient to gamification and they'll work as

00:49:05.200 | a better heuristic to try to push you in the best direction. - But I guess the underlying problem

00:49:12.000 | you're saying is there's a shortage of positions in academia. - That's a big problem for me. - Yeah.

00:49:18.080 | And that, and so they're going to be constantly gamifying the metrics. - It's a bit of a zero

00:49:22.800 | sum game. - It's a very competitive, it's a very competitive field. And that's what usually happens

00:49:28.080 | in very competitive fields. - Yeah. - But I think some of like the peer review problems, like scale

00:49:34.880 | helps, I think. And it's interesting to look at like what you're mentioning, breaking it down,

00:49:38.880 | maybe in like smaller parts and having more people jumping in. But this is definitely a problem.

00:49:47.040 | And the peer review problem, as I mentioned, is correlated with the problem of like academic

00:49:52.240 | career progression and it's all intertwined. And that's why I think it's so hard to break it.

00:49:57.920 | There are like a couple of really interesting things that are being done right now. There are a

00:50:02.800 | couple of, for instance, journals that are overlay journals on top of platforms like archive and

00:50:09.440 | bio-archive that want to remove like the more traditional journals from the equation.

00:50:14.480 | So essentially a journal is just a collection of links to papers. And what they are trying to do

00:50:21.360 | is like removing that middleman and trying to make the review process a little bit more transparent

00:50:28.480 | and not charging universities. Like there are a couple of more famous ones. There's one discreet

00:50:36.000 | analysis in mathematics. There's one called the Quantum Journal, which we're actually working with

00:50:41.280 | them. We have a partnership with them for the papers that get published in Quantum Journal.

00:50:45.840 | They also get the annotations on formats and they're doing pretty well. They've been able to

00:50:50.480 | grow substantially. The problem there is getting to critical mass. So it's again, convincing the

00:50:55.200 | researchers and especially the young researchers that need that impact factor, need those

00:51:01.120 | publications to have citations to not publish on the traditional journal and go on an open journal

00:51:07.520 | and publish their work there. I think there are a couple of really high profile scientists,

00:51:12.960 | people like Tim Gowers, that are trying to incentivize like famous scientists that already

00:51:18.240 | have tenure and that don't need that to publish that to increase the reputation of those journals

00:51:24.000 | so that other maybe younger scientists can start publishing on those as well. And so they can try

00:51:29.440 | to break that vicious cycle of the more traditional journals. - I mean, another possible way to break

00:51:36.720 | this cycle is to like raise public awareness and just by force, like ban paid journals. Like what

00:51:45.520 | exactly are they contributing to the world? Like basically making it illegal to forget the fact

00:51:54.560 | that it's mostly federally funded. So that's a super ugly picture too. But like, why should

00:52:00.800 | knowledge be so expensive? Like where everyone is working for the public good and then there's

00:52:08.880 | these gatekeepers that, you know, most people can't read most papers without having to pay money.

00:52:16.000 | And that doesn't make any sense. Like that should be illegal. - I mean, that's what you're saying

00:52:23.680 | is exactly right. I mean, for instance, right, I went to school here in the US, we studied in Europe

00:52:28.880 | and you would sit, like you'd ask me all the time to download papers and send it to him because he

00:52:33.600 | just couldn't get it and like papers that he needed for his research. And so- - But he's a student,

00:52:38.240 | like he's- - Yeah, he's a grad student. - He was a grad student, but that, you know, I'm even

00:52:42.560 | referring to just regular people. - Oh yeah, okay, that too. - And I think during 2020, because of

00:52:50.880 | COVID, a lot of journals put down the walls for certain kind of coronavirus related papers.

00:52:56.320 | But like that just gave me an indication that like, this should be done for everything. It's absurd.

00:53:04.080 | Like people should be outraged that there's these gates because, so the moment you dissolve the

00:53:11.280 | journals, then there'll be an opportunity for startups to build stuff on top of archive. It'd

00:53:18.720 | be an opportunity for like Fermat's library to step up, to scale up to something much even larger.

00:53:25.360 | I mean, that was the original dream of Google, which I've always admired, which is make the

00:53:31.040 | world's information accessible. Actually, it's interesting that Google hasn't, maybe you guys

00:53:36.000 | can correct me, but they have put together Google Scholar, which is incredible. And they've

00:53:41.920 | did the scanning of books, but they haven't really tried to make science accessible in the following

00:53:50.000 | way. Like besides doing Google Scholar, they haven't like delved into the papers, right?

00:53:57.200 | - Which is especially curious given what Luis was saying, right? That it's kind of in their

00:54:01.600 | genesis, there's this research that was very connected with how papers reference each other

00:54:06.960 | and like building a network out of that. - Interesting enough, like Google, I think

00:54:11.520 | there was not intended, Google Plus was like the Google social network that caught cancel,

00:54:16.800 | was used by a lot of researchers. - Yes, it was.

00:54:18.800 | - With what I think was just a side, kind of a side effect, but then a lot of people ended up

00:54:24.000 | migrating to Twitter, but it was not on purpose. But yeah, I agree with you, like they haven't

00:54:27.840 | gone past Google Scholar and I don't know why. - Well, that said, Google Scholar is incredible.

00:54:33.440 | For people who are not familiar, it's one of the best aggregation of all the scientific work that's

00:54:40.000 | out there and especially the network that connects all of them, what sites, what, and also trying to

00:54:45.680 | aggregate all of the versions of the papers that are available there and trying to merge them in a

00:54:50.480 | way that one particular work, even though it's available in a bunch of places, counts as,

00:54:55.200 | you know, like a central hub of what that work is across the multiple versions. But that almost

00:55:01.280 | seems like a fun pet project of a couple of engineers within Google, as opposed to a serious

00:55:08.000 | effort to make the world science accessible. - But going back to just the journals, when

00:55:13.920 | you're talking about that, Lex, I believe that in that front, I think we might be past the event

00:55:20.720 | horizon. So I think the business model for the journals doesn't make sense. They're a middle

00:55:27.840 | layer that is not adding a lot of value and you see a lot of motions, whereas like in Europe,

00:55:33.040 | a lot of the papers that are funded by the European Union, they will have to be

00:55:41.280 | open to the public. And I think there's a lot of... - Bill Gates too, like what the Gates Foundation

00:55:46.880 | funds, like the demand that it's accessible to everybody. - So I think it's a question of time

00:55:54.560 | before that wall kind of falls. And that is going to open a lot of possibilities. Because, you know,

00:56:00.880 | imagine if you had like the layer of, that gigantic layer of papers all available online,

00:56:08.160 | that unlocks a lot of potential as a platform for people to build things on top of that.

00:56:13.680 | - But I think to what you're saying, it is weird. Like you can literally go and listen to any song

00:56:20.400 | that was ever made on your phone, right? You open Spotify and you might not even pay for it. You

00:56:25.840 | might be on the free version and you can listen to any song that was ever made pretty much.

00:56:29.760 | But there's like, you don't have access to a huge percentage of academic papers, which is just like

00:56:38.240 | this fundamental knowledge that we're all funding, but you as an individual don't have access to it.

00:56:43.280 | And somehow, you know, like the problem for music got solved, but for papers, it's still like...

00:56:49.440 | - It's just not yet. It could be ad supported, all those kinds of things. And then hopefully

00:56:53.840 | that would change the way we do science. That's the most exciting thing for me,

00:56:57.600 | is especially once I started like making videos and this silly podcast thing, I started to realize

00:57:06.400 | that if you want to do science, one of the most effective ways is to do a couple,

00:57:12.160 | the paper with a set of YouTube videos, like explaining it.

00:57:17.040 | - Yeah. That also seems like there's a lot of room for disruption there. What is the paper 2.0

00:57:23.920 | going to look like? I think like LaTeX and the PDF seems like, if you look at the first paper

00:57:31.680 | that got published in Nature, and if you look at the paper that got published in Nature today,

00:57:36.320 | look at the two side by side, they are fundamentally the same. And even though like

00:57:41.280 | the paper that gets published today, you know, you get even code, like right now people put like

00:57:47.840 | code on a PDF. And there are so many things that are related to papers today. You have data,

00:57:56.240 | you have code, you might need videos to better explain the concepts. So I think for me, it's

00:58:03.840 | natural that there's going to be also an evolution there, that papers are not going to be just the

00:58:08.000 | static PDFs or LaTeX. There's going to be a next interface. - So in academia, a lot of things that

00:58:16.000 | you're judged by is often quantity, not quality. I wonder if there's a opportunity to have like,

00:58:23.360 | I tend to judge people by the best work they've ever done as opposed to...

00:58:28.960 | I wonder if there's a possibility for that to encourage sort of focusing on the quality

00:58:34.000 | and not necessarily in paper form, but maybe a subset of a paper, subset of idea,

00:58:39.200 | almost even a blog post or an experiment. Like why does it have to be published in a journal

00:58:44.000 | to be legitimate? - And it's interesting that you mentioned that. I also think like, yeah,

00:58:52.240 | why is that the only format? Why can't a blog post or... We were even experimenting with these

00:59:00.080 | a few months ago. Can you actually like publish something or like a new scientific breakthrough

00:59:08.160 | or something that you've discovered in the form of like a set of tweets,

00:59:13.280 | a Twitter thread? Why can't that be possible? And we were experimenting with that idea.

00:59:21.840 | We even, yeah, we ran a couple of, like some people submitted a couple of those,

00:59:27.440 | like I think the limit was three or four tweets. Maybe it's a new way to look at a proof or

00:59:32.880 | something, but I think it just serves to show that there should be other ways to publish like

00:59:38.160 | scientific discoveries that don't fit the paper format. - Well, but so even with the Twitter thread,

00:59:44.880 | it would be nice to have some mechanism of formalizing it and making it

00:59:50.400 | into an NFT. - Maybe. - Like a concrete thing that you can reference with a link that's unique.

00:59:58.960 | Because I mean, everything we've been saying, all of that, while being true, it's also true that

01:00:09.120 | the constraints and the formalism of a paper works well. It like forces you, constraints forces you

01:00:17.920 | to narrow down your thing and literally put it on paper, but you know. - I agree. - Make concrete.

01:00:27.200 | And that's why, I mean, it's not broken. It's just could be better. And that's the main idea.

01:00:34.160 | I think there's something about writing, whether it's a blog post or Twitter thread or a paper,

01:00:41.040 | that's really nice to concretize a particular little idea that can then be referenced by

01:00:49.520 | other ideas, then it can be built on top of with other ideas. So let me ask, you've read quite a

01:00:58.400 | few papers, you've annotated quite a few papers. Let's talk about the process itself. How do you

01:01:06.080 | advise people read papers? Or maybe you want to broaden it beyond just papers, but just read

01:01:12.640 | concrete pieces of information to understand the insights that lay within. - I would say for papers

01:01:19.200 | specifically, I would bring back kind of what Luis was talking about, is that it's important to keep

01:01:24.160 | in mind that papers are not optimized for ease of understanding. And so, right, there's all sorts of

01:01:31.920 | restrictions in size and format and language that they can use. And so it's important to keep that

01:01:38.640 | in mind. And so that if you're struggling to read a paper, that might not mean that the underlying

01:01:45.280 | material is actually that hard. And so that's definitely something that, especially for us,

01:01:51.760 | that we read papers and most of the times we'll read papers that are completely outside of our

01:01:57.760 | comfort zone, I guess. And so it'd be completely new areas to us. So I always try to keep that in

01:02:05.040 | mind. - So there's usually a certain kind of structure, like abstract introduction, there's

01:02:09.200 | methodology, depending on the community and so on. Is there something about the process of like

01:02:17.440 | how to read it, whether you want to skim it to try to find the parts that are easy to understand

01:02:22.080 | and not, reading it multiple times? Is there any kind of hacks that you can comment on?

01:02:28.720 | - I remember like Feynman had this kind of hack when he was reading papers where he would basically,

01:02:35.520 | I think I believe he would read the conclusion of the paper and we would try to just see if he would

01:02:45.360 | be able to figure out how to get to the conclusion in like a couple of minutes by himself.

01:02:51.040 | And he would read a lot of papers that way. And I think Fermi also did that almost. And Fermi was

01:02:58.640 | known for doing a lot of back of the envelope calculations. So he was a master at doing that.

01:03:03.520 | In terms of like, especially when reading a paper, I think a lot of times people might

01:03:10.640 | feel discouraged about the first time you read it, it's very hard to grasp or you don't understand a

01:03:18.480 | huge fraction of the paper. And I think it's having read a lot of papers in my life, I think

01:03:24.000 | I'm in peace with like the fact that you might spend hours where you're just reading a paper

01:03:29.360 | and jumping from paper to paper, reading citations. And like your level of understanding of sometimes

01:03:36.960 | of the paper is very close to 0%. And all of a sudden, everything kind of makes sense in your

01:03:44.400 | mind. And then you have this quantum jump where all of a sudden you understand the big picture

01:03:51.360 | of the paper. And this is an exercise that I have to do when reading papers, and especially like

01:03:58.240 | more complex papers, like, okay, you don't understand because you're just going through

01:04:01.840 | the process and just keep going. And it might feel super chaotic, especially if you're jumping

01:04:07.280 | from reference to reference. You might end up with like 20 tabs open and you're reading a ton of

01:04:12.800 | other papers, but it's just trusting that process, that at the end, like you'll find light. And I

01:04:19.200 | think for me, that's a good framework when reading a paper. It's hard because you might end up

01:04:25.840 | spending a lot of time and it looks like you're lost, but that's the process to actually understand

01:04:34.000 | what they're talking about in the paper. - Yeah, I think that process, I've found a lot of value

01:04:39.840 | in the process, especially for things outside my field, reading a lot of related work sections

01:04:46.560 | and kind of going down that path of getting a big context of the field, because what's,

01:04:52.560 | especially when they're well-written, there's opinions injected into the related work. Like

01:04:58.080 | what work is important, what is not. And if you read multiple related work sections that cite or

01:05:03.600 | don't cite each other, like the papers, you get a sense of where the field, where the tensions of

01:05:09.520 | the field are, where the field is striving. And that helps you put into context, like whether the

01:05:16.400 | work is radical, whether it's overselling itself, whether it's underselling itself, all those

01:05:22.480 | things. And added on top of that, I find that often the related work section is the most kind

01:05:31.520 | of accessible and readable part of a paper, because it's kind of, it's brief to the point,

01:05:38.240 | it's trying to like summarizing, it's almost like a Wikipedia style article. The introduction is

01:05:43.280 | supposed to be a compelling story or whatever, but it's often like overselling, there's like an

01:05:48.800 | agenda introduction. The related work usually has the least amount of agenda, except for the few

01:05:54.800 | like elements where you're trying to talk shit about previous work, where you're trying to sell

01:05:59.680 | that you're doing much better. But other than that, when you're just painting where the field

01:06:05.520 | came from or where the field stands, that's really valuable. And also, again, just to agree

01:06:11.120 | with Fiona in the conclusion, but it's like, I get a lot of value from the breadth first search,

01:06:16.640 | kind of read the conclusion, then read the related work, and then go through the references in the

01:06:23.040 | related work, read the conclusion, read the related work, and just go down the tree until

01:06:29.280 | you like hit dead ends or run out of coffee. And then through that process, you go back up the tree

01:06:35.280 | and now you can see the results in their proper context. Unless of course the paper is truly

01:06:40.960 | revolutionary, which even that process will help you understand that is in fact truly revolutionary.

01:06:47.920 | You've also, you talked about just following your Twitter thread in a depth first search.

01:06:55.920 | You talked about that you read the book on Grisha Perlman, Grigori Perlman, and then you had a

01:07:03.360 | really nice Twitter thread on it and you were taking notes throughout. So at a high level,

01:07:09.840 | is there suggestions you can give on how to take good notes? Whether it's, we're talking about

01:07:14.880 | annotations or just for yourself to try to put on paper ideas as you progress through the work in

01:07:22.000 | order to then like understand the work better? For me, I always try not to underestimate how

01:07:28.320 | much you can forget within six months after you've read something. I thought you were going to say

01:07:33.040 | five minutes, but yeah, six months is good. Yeah, or even shorter. And so that's something

01:07:38.400 | that I always try to keep in mind. And it's often, I mean, every once in a while I'll read back a

01:07:45.200 | paper that I annotated on Fermat and I'll read through my own annotations and I've completely

01:07:52.960 | forgotten what I had written. But it also, it's interesting because in a way, after you just

01:08:00.800 | understood something, you're kind of the best possible teacher that can teach your future self.

01:08:06.720 | After you've forgotten it, you're kind of your own best possible teacher at that moment.

01:08:14.640 | And so it can be great to try to capture that. It's brilliant. It just made me kind of

01:08:21.840 | realize it's really nice to put yourself in the position of teaching an older version of yourself

01:08:30.240 | that returns to this paper, almost like thinking it literally. That's under explored, but it's super

01:08:36.000 | powerful because you were the person that you can, if you look at the scale from one, not knowing

01:08:41.760 | anything about the topic, and 10, you are the one that progressed from one to 10 and you know which

01:08:46.720 | steps you struggled with. So you're really the best person to help yourself make that transition

01:08:52.240 | from one to 10. And a lot of the times, I really believe that the framework that we have to expose

01:09:01.600 | ourselves to be talking to us when we were an expert, when we were taking that class and we

01:09:06.880 | knew everything about quantum mechanics. And then six months later, you don't remember half of the

01:09:11.360 | things. How could we make it easier to have those conversations between you and your past expert

01:09:19.680 | self? I think there might be, it's an under explored idea. I think notes on paper are probably

01:09:27.120 | not the best way. I'm not sure if it's a combination of video, audio, where it's like you

01:09:32.960 | have a guided framework that you follow to extract information from yourself so that you can later

01:09:38.480 | kind of revisit to make it easier to remember. But that's, I think it's an interesting idea worth

01:09:47.040 | exploring that I haven't seen a lot of people kind of trying to distill that problem.

01:09:53.200 | Yeah, creating the kind of tools. I find if I record, it sounds weird, but I'll take notes. But

01:10:00.400 | if I record audio, like little clips of thoughts, like rants, that's really effective at capturing

01:10:10.160 | something that notes can't. Because when I replay them, for some reason, it loads my brain back

01:10:17.120 | into where I was when I was reading that in a way that notes don't. Like when I read notes,

01:10:23.520 | I'll often be like, "Whoa, what? What was I thinking there?" But when I listened to the audio,

01:10:30.640 | it brings you right back to that place. And maybe with video, with visual, that might be even more

01:10:36.160 | powerful. I think so. Yeah. And I think just the process of verbalizing it, that alone kind of

01:10:44.080 | makes you have to structure your thought and put it in a way that somebody else could come and

01:10:49.680 | understand it. And just the process of that is useful to organize your thoughts. And yeah,

01:10:57.120 | just that alone. Does the Fermat's Library Journal Club have a video component or no?

01:11:02.480 | Not natively. Sometimes we'll include videos, but it's always embedded.

01:11:07.840 | Do people build videos on top of it to explain the paper? Because you're doing all the hard work

01:11:12.720 | of understanding deeply the paper. We haven't seen that happening too much, but we were actually

01:11:20.880 | playing around with the idea of creating some sort of podcast version where we try to distill

01:11:26.160 | the paper on an audio format that not maybe you can have access to. Might be trickier, but there

01:11:32.720 | are definitely people that could be interested in the paper and that topic, but are not willing

01:11:36.640 | to read it. But they might listen to a 30-minute episode on that paper. You could reach more people

01:11:42.480 | and you might even bring the authors to the conversation, but it's tricky, especially for

01:11:47.200 | more technical papers. We've thought about doing that, but we haven't converged. I'm sure if you

01:11:54.080 | have any... Well, I'm going to take that as a small project to take one of the Fermat's, almost

01:12:00.560 | half advertisement and half as a challenge for myself, to take one of the annotated papers and

01:12:05.760 | use it as a basis for creating a quick video. I've seen, hopefully I'm saying the name correctly, but

01:12:14.720 | Machine Learning Street Talk. I think that's the name of the show that I recommend highly.

01:12:21.840 | That's the right name. But they do exactly that, which is multiple hour breakdown of a paper with

01:12:27.760 | video component. Sometimes with authors, people love it. It's very effective.

01:12:33.120 | There's also, I've seen, I haven't seen the entire, in its entirety, but I've seen the founder of

01:12:40.000 | comma.ai, George. I've seen him just taking a paper and then distilling the paper and coding it,

01:12:48.960 | coding it sometimes doing 10 hours. And he was able to get a lot of people interested in that

01:12:55.600 | and viewing him. - So I'm a huge fan of that.

01:12:58.480 | George is a personality. I think a lot of people listen to this podcast for the same reason.

01:13:05.520 | It's not necessarily the contents. They like to listen to a silly Russian who has a childlike brain

01:13:13.760 | and mumbles and all those like struggle with ideas. And George is a madman. People just enjoy,

01:13:20.240 | how is he going to struggle in implementing this particular paper? How is he going to

01:13:24.560 | struggle with this idea? It's fun to watch and that actually pulls you in. The personality

01:13:28.800 | is important there. - True. I agree with you,

01:13:32.160 | but there also, it's visible. There's an extraordinary ability that is there. He's

01:13:39.200 | talented and you need to have, there's a craft and this guy definitely has talent and he's doing

01:13:44.960 | something that is not easy. And I think that also draws the attention of people. And the other day,

01:13:50.640 | we were actually, we ran into this YouTube channel of this guy that was restoring art.

01:13:55.840 | And it was basically just a video of him. The production is really well done. And it's just

01:14:05.760 | him taking really old pieces of art and then paintings and then restoring them. But he's

01:14:12.480 | really good at that. And he describes that process and that draws the attention of people,

01:14:18.080 | regardless of your craft, be it like annotating a paper, restoring it.

01:14:22.960 | - Craftsmanship, excellence. Yeah. George is incredibly good at programming.

01:14:27.200 | You know, those competitive programmers, like Top Coater and all those kinds of stuff. He has

01:14:33.760 | the same kind of element where the brain just jumps around really quickly. And that's, yeah,

01:14:39.040 | just like with art restoration. - And it's also motivating.

01:14:41.280 | - Yeah, it's motivating. But you're right, in watching people who are good at what they do,

01:14:48.080 | it's motivating, even if the thing you're trying to do is not what they're doing.

01:14:51.600 | It's just like contagious when they're really good at it. And the same kind of analysis with

01:14:56.160 | the paper, I think, so not just like the final result, but the process of struggling with it.

01:15:02.400 | That's really interesting. - Yeah. I think Twitch proved that

01:15:05.840 | like, you know, that there's really a market for that, for watching people do things that they're

01:15:11.760 | really good at and you'll just watch it. You will enjoy that, that might even spike your interest

01:15:18.800 | in that specific topic. And yeah, people will enjoy watching sometimes hours on end of great

01:15:26.080 | craftsmen. - Do you mind if we talk about some of the

01:15:29.200 | papers, do any papers come to mind that have been annotated on Fermat's library?

01:15:33.680 | - The papers that we annotated can be about completely random topics, but that's part of

01:15:38.960 | what we enjoy as well. It forces you to explore these topics that otherwise maybe you'd never

01:15:43.360 | run into. And so the ones that come to mind are, to me, are fairly random, but one that I really

01:15:51.680 | enjoyed learning more about is a paper written by a mathematician, actually, Tom Apostol,

01:16:00.080 | and about a tunnel in a Greek island off the coast of Turkey. So it's very random.

01:16:10.240 | So this, okay, so what's interesting about this tunnel? So this tunnel was built in the 6th

01:16:18.000 | century BC and it was built in the island of Samos, which is, as I said, off the coast of Turkey.

01:16:27.360 | And they had the city on one side and the other mountain, and then they had a bunch of springs

01:16:34.160 | on the other side and they wanted to bring water into the city. Building an aqueduct would be

01:16:40.320 | pretty hard because of the way the mountain was shaped. And it would also, if they were under a

01:16:45.440 | siege, they could just easily destroy that aqueduct and then the water wouldn't have any

01:16:51.520 | water supply, the city wouldn't have any water supply. And so they decided to build a tunnel

01:16:56.480 | and they decided to try to do it quickly. And so they started digging from both ends at the same

01:17:07.280 | time through the mountain. And so when you start thinking about this, it's a fairly difficult

01:17:14.400 | problem. And this is like 6th century BC. So you had very limited access to the mathematical tools

01:17:22.640 | that you had at the time were very limited. And so what this paper is about is about the story

01:17:27.680 | of how they built it and about the fact that for about 2000 years, the accepted explanation of how

01:17:35.760 | they built it was actually wrong. And so this tunnel has been famous for a while. There are

01:17:40.240 | a number of historians that talked about it since ancient Egypt. And the method that they described

01:17:46.800 | for building it was just wrong. And so these researchers went there and were able to figure

01:17:57.040 | that out. And so basically, kind of the way that they thought they had built it was basically,

01:18:02.720 | if you can imagine looking at the mountain from the top and you have the mountain, then you have

01:18:07.920 | both entrances. And so what they thought, and this is what the ancient historians described,

01:18:14.640 | is that they effectively tried to draw a right angle triangle with the two entrances at each end

01:18:24.480 | of the hypotenuse. And the way they did is like they would go around the mountain and kind of

01:18:29.440 | walking in a grid fashion. And then you can figure out the two sides of the triangle. And then after

01:18:36.560 | you have that triangle, you can effectively draw two smaller triangles at each entrance that are

01:18:43.680 | proportional to that big triangle. And then you kind of have arrows pointing in each way.

01:18:49.360 | And then you know at least that you have a line going through the mountain that connects

01:18:56.400 | both entrances. The issue with that is like, once you go to this mountain and you start thinking of

01:19:03.680 | doing this, you realize that, especially given that the tools that they had at the time, that

01:19:08.480 | your error margin would be too small. You wouldn't be able to do it. Just the fact of trying to build

01:19:17.440 | this triangle in that fashion, the error would accumulate and you would end up missing. You'd

01:19:22.080 | start building these tunnels and they would miss each other. - So the task ultimately is to figure

01:19:26.000 | out like really perfectly, as close as possible, the direction you should be digging. First of all,

01:19:31.280 | that it's possible to have a straight line through and then what that direction would be.

01:19:37.040 | And then you are trying to infer that by constructing a right triangle. I'm not exactly

01:19:44.320 | sure about how to do that rigorously, like by tracing the mountain, by walking along the mountain.

01:19:50.560 | You said grids? - Yeah, you kind of walk as if you were in a grid and so you just walk in right

01:19:56.720 | angles. - But then you have to walk really precisely then. You have to use tools to measure

01:20:02.880 | this and the terrain is probably a mess. So this makes more sense in 2D and 3D gets even weirder.

01:20:09.760 | So, okay, gotcha. - But so this method was described by an ancient Egyptian historian,

01:20:16.640 | I think, Hero of Alexandria. And then for about 2000 years, that's how we thought that they'd

01:20:24.160 | built this tunnel. And then these researchers went there and found out that actually they must have

01:20:32.720 | to use other methods. And then in this paper, they describe these other methods. And of course,

01:20:39.680 | they can't know for sure, but they presented a bunch of plausible alternatives. The one that

01:20:46.000 | for me is the most plausible is that what they probably must have done is to use something that

01:20:51.120 | is similar to an iron sight on a rifle, the way you can line up your rifle with a target off in

01:20:59.280 | the distance by having an iron sight. And they must have done something similar to that effectively

01:21:09.280 | with three sticks. And that way they were able to line up sticks along the side of the mountain

01:21:17.600 | that were all on the same height. And so that then you could get to the other side,

01:21:22.400 | and then you could draw that line. So this for me is the most plausible way that they might have

01:21:29.200 | done that. But then they described this in detail and other possible approaches in this paper.

01:21:36.240 | - So this is a mathematician doing this? - Yeah, this is a mathematician that did this.

01:21:41.040 | - Which I suppose is the right mindset instead of skills required to solve an ancient problem.

01:21:47.520 | - Yeah. - Mathematicians and engineers,

01:21:51.040 | there are a lot of things. - Because they didn't have computers or

01:21:54.960 | drones or LIDAR back then or whatever technology you would use modern day for civil engineering.

01:21:59.680 | - Yeah. And another fascinating thing is that effectively after the downfall of the Roman

01:22:07.120 | civilization, people didn't build tunnels for about a thousand years. We go a thousand years

01:22:12.160 | without tunnels. And then only in late middle ages that we start doing them again.

01:22:17.600 | But here is a tunnel like sixth century BC, like incredibly limited mathematics. And they build it

01:22:23.920 | in this way. And it was a mystery for a long time exactly how they did it. And then these

01:22:30.880 | mathematicians went there and basically with no archeology kind of background, were able to figure

01:22:37.440 | it out. - How do annotations for this paper look like? What's a successful annotation for paper

01:22:43.920 | like this? - Yeah. So sometimes you're for this paper,

01:22:47.360 | sometimes adding some more context on a specific part. Like sometimes they mentioned, for instance,

01:22:56.880 | these instruments that were common in ancient Greece and ancient Rome for building things.

01:23:04.640 | And so in some of those annotations, I described these instruments in more detail and how they

01:23:10.800 | worked. 'Cause sometimes it can be hard to visualize these. Then this paper, I forget exactly

01:23:20.560 | when this was published. I believe maybe the seventies. But then there was further research

01:23:27.840 | into this tunnel and other interesting aspects about it. I add those to that paper as well.

01:23:34.080 | There's historical context that I also go into there. For instance, the fact that as I said,

01:23:40.640 | that effectively after the downfall of the Roman empire, no tunnels were built. Like this is

01:23:44.960 | something that I added to the paper as well. Yeah. So those are- - So when other people look at the

01:23:50.960 | paper, how did they usually consume the annotations? So it's like, is there a commenting feature?

01:23:58.000 | I mean, like this is a really enriching experience the way you read a paper.

01:24:02.240 | What aspects do people usually talk about that they value from this?

01:24:07.760 | Yeah. So anybody can just go on there and either add a new annotation or other comments to an

01:24:15.680 | existing annotation. And so you can start a kind of a thread within an existing annotation.

01:24:21.280 | And that's something that happens relative frequency. And then because I was the original

01:24:26.800 | author of the initial annotation, I get pinged. And so oftentimes I'll go back and add on to that

01:24:34.080 | thread. - How'd you pick the paper? I mean, first of all, this whole process is really exciting.

01:24:39.360 | I'm gonna, especially after this conversation, I'm gonna make sure I participate much more

01:24:44.320 | actively on papers that I know a lot about and on paper I know nothing about. I should- - You get

01:24:49.280 | to annotate the paper. - I would love to. I also, I mean, I realized that there's a,

01:24:56.880 | like, it's an opportunity for people like me to publicly annotate a paper. Like- - Or do an AMA

01:25:06.560 | around the paper. - Yeah, exactly. But yeah, but like be in the conversation about a paper. It's

01:25:13.760 | like a place to have a conversation about an idea. The other way to do it that's much more ad hoc is

01:25:19.680 | on Twitter, right? But this is more like formal and you could actually probably integrate the two.

01:25:24.880 | They have a conversation about the conversation. So the Twitter is the conversation about a

01:25:29.360 | conversation and the main conversation is in the space of annotations. - There's an interesting

01:25:34.400 | effect that we see sometimes with the annotations on our papers is that a lot of people, especially

01:25:39.280 | if the annotations are really well done, people sometimes are afraid of adding more annotations

01:25:46.960 | because they see that as a kind of a finished work. And so they don't want to pollute that.

01:25:51.680 | And especially if it's like a silly question. I don't think that's good. I think we should as

01:25:59.280 | much as possible try to lower the barrier for someone to jump in and ask questions. I think

01:26:04.720 | most of the times it adds value, but it's some feedback that we got from users and readers.

01:26:10.480 | I'm not exactly sure how to kind of fight that, but... - Well, I think if I serve as an inspiration

01:26:21.120 | in any way is by asking a lot of dumb questions and saying a bunch of dumb shit all the time.

01:26:27.840 | And hopefully that inspires the rest of the other folks to do the same, because that's the only way

01:26:33.600 | to knowledge, I think, is to be willing to ask the dumb questions. - And there are papers that are

01:26:38.720 | like... We have a lot of papers on Fermat's where it's just one page, or really short papers. And

01:26:44.800 | we have like the shortest paper ever published in a math journal, with just a couple of words.

01:26:50.720 | One of my favorite papers on the platform is actually a paper written by Enrico Fermi. And

01:26:57.600 | the title of the paper is, I think, is "My Observations at Trinity." So basically Fermi

01:27:02.640 | was part of the Manhattan Project. So he was in New Mexico when they exploded the first atomic bomb.

01:27:10.240 | And so he was a couple of miles away from the explosion, and he was probably one of the first

01:27:15.840 | persons to calculate the energy of the explosion. And so the way he did that was he took a piece of

01:27:22.560 | paper and he tore down a piece of paper in little pieces. And when the bomb exploded,

01:27:28.560 | the Trinity bomb was the name of the bomb, he waited for the blast to arrive at where he was.

01:27:33.840 | And then he threw those pieces of paper in the air, and he calculated the energy based on the

01:27:39.520 | displacement of the paper, the pieces of paper. And then he wrote a report, which was classified

01:27:45.040 | until a couple of years ago. One page report, calculating the energy of the explosion.

01:27:50.720 | - Oh, that's so badass. - And we actually went there and

01:27:53.760 | kind of unpacked and I think he just mentions basically the energy, and we actually went,

01:27:59.600 | and one of the annotations is like explaining how he did that.

01:28:02.080 | - I wonder how accurate he was. - It was maybe, I think, like 20 or 25% off.

01:28:08.240 | Then there was another person that actually calculated the energy based on images after

01:28:14.640 | the explosion and the rate at which the mushroom of the explosion expanded. And it's more accurate

01:28:22.240 | to calculate the energy based on that. And I think it was like 20, 20% off. But it's really

01:28:28.000 | interesting because Fermi was known for all these, being a master at these back of the envelope

01:28:33.280 | calculations, all those, like the Fermi problems are well known for that. And it's super interesting

01:28:39.600 | to see that just one page report, it was also actually classified. And it's interesting because

01:28:45.360 | a couple of months ago, when the Beirut explosion happened, there was a video circulating of these

01:28:52.000 | a bride that was doing a photo shoot when the explosion in Beirut happened. And so you can see

01:28:57.200 | a video of her with the wedding dress, and then the explosion happens and the blast arrives at

01:29:01.440 | where she was. She was a couple of miles away from the blast. And you can see like the displacement

01:29:07.840 | of the dress as well. And I actually looked, and that video went viral on Twitter. And I actually

01:29:12.320 | looked at that video and used the same techniques that Fermi used to calculate the energy of the

01:29:18.240 | explosion based on the displacement of the dress. And you could actually see where she was at the

01:29:24.240 | distance from the explosion, because there was a store behind her and you could look the name of

01:29:28.080 | the store. And so I calculated that. - The distance and then you can...

01:29:32.000 | - Based on the distance where she was from the explosion and also on the displacement of the

01:29:37.680 | dress, like because you can, when the blast happens, like you can see the dress going back

01:29:42.160 | and then going back to the original position. And like, by just looking at how much the dress moved,

01:29:49.200 | you can estimate the energy of the explosion. - I assume you published this.

01:29:53.760 | - On Twitter, it was just a Twitter thread. But actually like a lot of people share that and it

01:29:59.920 | was picked up by a couple of news outlets. - I was hoping it would be like a formal title

01:30:05.920 | and it would be an archive. - No, no, no, no.

01:30:07.760 | - It may be submitted. - Just the Twitter thread. But it was

01:30:10.560 | interesting because it was exactly the same method that Fermi used.

01:30:14.560 | - Is there something else that jumps to mind? Like, is there something, I know like in terms

01:30:20.480 | of papers, like I know the Bitcoin paper is super popular. Is there something interesting to be said

01:30:25.600 | about any of the white papers in the cryptocurrency space? - Yeah, the Bitcoin paper was the first paper

01:30:32.720 | that we put on for months. - Why that choice as the first paper?

01:30:38.400 | - This was a while ago and it was one of the papers that I read and then kind of explained it

01:30:46.000 | to Luiz and there were two other friends that do this journal club with us. And I did some research

01:30:52.400 | in cryptography as an undergrad. And so it was a topic that I was interested in. But even for me,

01:30:59.040 | that I had that background, but reading the Bitcoin paper, it took me a few reads to really

01:31:06.400 | kind of wrap my head around it. It uses very spartan, precise language in a way. It's like,

01:31:13.120 | you feel like you can't take any word out of it without something falling apart. And it's all

01:31:19.200 | there. I think it's a beautiful paper and it's very well-written, of course, but we wanted to

01:31:27.920 | try to make it accessible so that anybody that maybe is an undergrad in computer science could

01:31:32.960 | go on there and know that you have all the information in that page that you're going to

01:31:40.000 | need to understand the mechanics of Bitcoin. And so I explain the basic public key cryptography

01:31:48.720 | that you need to know in order to understand it. I explain, okay, what are the properties of a hash

01:31:53.760 | function and how they are useful in this context. Explain what a Merkle tree is. So a bunch of those

01:32:00.880 | basic concepts that maybe if you're reading it for the first time and you're an undergrad and

01:32:04.320 | you don't know those terms, you're going to be discouraged because maybe, okay, now I have to

01:32:08.720 | go and Google around until I understand these before I can make progress in the paper. And this

01:32:15.120 | way it's all there. So there's a magic to... Also to the fact that over time, more people went on

01:32:21.920 | there and added further annotations. So the idea that the paper gets easier and more accessible

01:32:28.480 | over time, but you're still looking at the original content the way the author intended it to be.

01:32:35.440 | But there's just more context and the toughest bits have more in-depth explanations.

01:32:41.920 | - Okay, I think there's just so many interesting papers there. I remember reading the paper that

01:32:50.400 | was written by Freeman Dyson on the first time that he explained, he came up with the concept

01:32:57.280 | of the Dyson sphere and he put that out. Again, it's a one-page paper. And what he explained was

01:33:04.560 | that eventually if a civilization develops and grows, there's going to be a point where when

01:33:12.480 | the resources on the planet are not enough for the energy requirements of that civilization.

01:33:18.560 | So if you want to go, the next step is you need to go to the next star and extract energy from

01:33:24.000 | that star. And the way to do it is you need to build some sort of cap around the star that

01:33:30.000 | extracts the energy. So he theorized this idea of the Dyson sphere and he went on to kind of analyze

01:33:37.520 | how he would build that, the stability of that sphere. Like if something happens, if there's

01:33:42.320 | like a small oscillation with that fear collapse into the star or no, what would happen? And even

01:33:48.640 | went on to kind of say that a good way for us to look for signs of intelligent life out there

01:33:55.600 | is to look for signals of these Dyson spheres. And because according to the law of second law

01:34:01.920 | of thermodynamics, like there's going to be a lot of infrared radiation that is going to be emitted

01:34:06.480 | as a consequence of extracting energy from the star. And we should be able to see those signals

01:34:12.480 | of like infrared if we look at the sky. But all these, like from the introduction of the concept,

01:34:18.000 | like how to build the Dyson sphere, the problems of like having a Dyson sphere, how to detect how

01:34:23.440 | that could be used as a signal for intelligence. - Wait, really? That's all in the paper?

01:34:26.880 | - Yeah, all in one, like one page paper. And it's like, for me, it's beautiful. It's like-

01:34:31.040 | - Where was this published? - I don't remember.

01:34:33.280 | - It's fascinating that papers like that could be, I mean, the guts it takes to put that all

01:34:40.320 | together in a paper form. You know, that kind of challenges our previous discussion of paper. I

01:34:46.320 | mean, papers can be beautiful. You can play with the format, right?

01:34:49.440 | - But there's a lot to unpack there. That's like the starting point, but it's beautiful that you're

01:34:56.720 | able to put that in one page and then people can build on top of that.

01:35:01.360 | - But the key ideas are there. - Yeah, exactly.

01:35:03.760 | - What about, have you looked at any of the big seminal papers throughout the history of science?

01:35:09.680 | Like you look at simple, like Einstein papers. Have any of those been annotated?

01:35:15.600 | - Yeah, yeah, no. We have some more seminal papers that people will have heard about. You know,

01:35:22.080 | we have the DNA double helix paper on there. We have the Higgs boson paper. Yeah, there's

01:35:32.560 | papers that we know that it's, they're not gonna be finding out about them because of us, but it's

01:35:39.920 | papers that we think should be more widely read and that folks would benefit from having some

01:35:46.000 | annotations there. And so we also have a number of those.

01:35:48.720 | - Yeah, a lot of like discovery papers for fundamental particles and all that. We have

01:35:54.240 | a lot of those on Fermat's library. I would like to, we haven't annotated that one, but I'd like

01:36:00.000 | to on the Riemann hypotheses. That's a really interesting paper as well. But we haven't

01:36:06.400 | annotated that one, but there's a lot of like more historical landmark papers on the platform.

01:36:12.400 | - Have you done Poincaré conjecture with Perlman?

01:36:16.960 | - That's too much.

01:36:17.760 | - Too much?

01:36:18.480 | - That's too much for me, but it's interesting that, you know, and going back to our discussion,

01:36:25.040 | like the Poincaré paper was like published on archive and it was not on a journal,

01:36:29.840 | like the three papers.

01:36:30.800 | - Yeah, what do you make of that? I mean, he's such a fascinating human being.

01:36:34.400 | - Exactly.

01:36:34.720 | - I mentioned to you offline that I'm going to Russia. He's somebody I'm really-

01:36:38.800 | - You should try to interview him.

01:36:40.080 | - Yeah, well, so I definitely will interview him. And I believe I will, I believe I can,

01:36:48.480 | I just don't know how to, I know where he lives. So here, okay. My hope is, my conjecture is that

01:36:57.520 | if I just show up to the house and look desperate enough, that, or threatening enough,

01:37:03.040 | or some combination of both, that like the only way to get rid of me is to just get the thing done.

01:37:08.400 | That's the hope.

01:37:08.960 | - It's actually interesting that you mentioned that because I, after I, so a couple of weeks

01:37:13.840 | ago I was searching for like stuff about Perlman online and ended up on this Twitter account of

01:37:20.240 | this guy that claims to be Perlman's assistant. And he's like, he has been posting a bunch of

01:37:26.800 | pictures like next to Perlman. You can see like Perlman in a library and he's like next to him,

01:37:31.600 | like taking a selfie or like Perlman walking on the street and like, maybe you could reach out to

01:37:36.960 | this assistant then I'll send you this Twitter account.

01:37:41.520 | - So maybe you're onto something.

01:37:43.360 | - No, but going back to like Perlman is super interesting because the fact that he published

01:37:48.160 | the proofs on archive is what was also like a way for him to, because he really didn't like

01:37:55.120 | the scientific publishing industry and the fact that you had to pay to get access to articles.

01:38:02.080 | And that was a form of like protest. And that's why he published those papers there. I mean,

01:38:07.760 | I think Perlman is just a fascinating like character. And for me, it's this kind of ideal of,

01:38:13.360 | platonic ideal of what a mathematician should be. It's someone that is, it's just cares about,

01:38:20.640 | deeply cares about mathematics. It cares about fair attribution of disregards money. And like

01:38:30.080 | the fact that he published like on archive is a good example of that.

01:38:33.840 | - What about the Fields Medal that he turned down the Fields Medal? What's, what's,

01:38:37.200 | what do you make of that?

01:38:39.360 | - Yeah, I mean, if you look at like the reasons why he rejected the Fields Medal. So after,

01:38:44.160 | so Perlman did a postdoc in the US and when he came back to Russia.

01:38:49.360 | - Do you know how good his English is?

01:38:51.520 | - I think it's fairly good.

01:38:53.120 | - I think it's pretty good.

01:38:53.760 | - I think it's really good.

01:38:54.800 | - Especially given lectures in American universities.

01:38:57.520 | - But I haven't been able to...

01:38:58.560 | - Listen to anything.

01:38:59.600 | - Well, certainly not listen, but I haven't been able to get anybody. Cause I know a lot of people

01:39:03.840 | have been to those lectures. I'm not able to get a sense of like, yeah, but how strong is the

01:39:09.920 | accent? What are we talking about here? Is this going to have to be in Russian? Is it going to

01:39:13.120 | have to be in English? It's fascinating. But he writes the papers in English.

01:39:16.080 | - True. But there's so many, like such a fascinating character. And there are a couple

01:39:21.920 | of examples like him, like at, I think 28 or 29, he proved like a really famous conjecture called

01:39:27.920 | the Salk Conjecture, I believe it was like in a very short four page proof of that. It was a

01:39:32.400 | really big breakthrough. Then he went to Princeton to give a lecture on that. And after the lecture,

01:39:38.240 | the chair of the math department at Princeton, a guy called Peter Sarnak, went up to Perlman,

01:39:44.640 | was trying to recruit him, trying to offer him a position at Princeton. And at some point he asked

01:39:51.520 | for Perlman's resume. And Perlman responded saying, just gave a lecture on like this

01:39:57.920 | really tough problem. Why do you need my resume? Like, I'm not going to send you, like I just

01:40:02.880 | proved my value. But going back to the Fields Medal, like when Perlman went to back to Russia,

01:40:11.760 | he arrived at a time where the salary of postdocs were so much off in regards to inflation that they

01:40:20.400 | were not making any money. Like people didn't even bother to pick up the checks at the end of the

01:40:25.920 | month because they were just like ridiculous. But thankfully he had some money that he had

01:40:30.560 | gained while he was doing his postdocs. So he just concentrated on like the Poincare conjecture

01:40:37.600 | problem, which he, when he took that, he took it after it was reframed by this mathematician

01:40:44.640 | called Richard Hamilton, which posed the problem in a way that it turned into this super like math

01:40:50.720 | Olympiad problem with like perfect boundaries, well-defined, and that was perfect for Perlman

01:40:56.160 | to attack. And so he spent like seven years working on that. And then in 2002, he started

01:41:01.520 | publishing those papers on archive. And people started jumping on that, reading those papers.

01:41:07.840 | And there was like a lot of excitement around that. A couple of years later, there were two

01:41:12.960 | researchers, I believe it was, they were from Harvard that took Perlman's work. They sanded

01:41:19.680 | some of the edges and they republished that saying that, you know, based on Perlman's work,

01:41:26.080 | they were able to figure out the Poincare conjecture. And then there was at the time at

01:41:32.560 | the international conference of mathematics in 2006, I believe that's when they were going to

01:41:40.080 | give out the Fields Medal. There was a lot of debate of like, oh, who's like, who should get

01:41:45.920 | the credit for solving this big problem. And for Perlman, it felt really sad that people were even

01:41:54.240 | considering that he was not the person that solved that. And the claims that those like researchers,

01:42:01.680 | when they published after Perlman, they were false claims that they were the ones, they just

01:42:05.440 | sanded a couple of edges, like Perlman did all the really hard work. And so just the fact that

01:42:12.080 | they doubted that Perlman had done that, like was enough for him to say, I'm not interested in this

01:42:17.680 | prize. And that was one of the reasons why he rejected the Fields Medal. Then he also rejected

01:42:22.960 | the Clay prize. So the Poincare conjecture was one of the millennium prizes. There was a million

01:42:28.800 | dollar prize associated with that problem. And that has to add to do with the fact that for them

01:42:34.080 | to attribute that prize, I think it had to be published on a journal. The proof. And again,

01:42:39.360 | Perlman's principles of like interfered here. And he also just didn't care about the money. He was

01:42:46.080 | like, Clay, I think was a businessman and he's like, doesn't have to do anything with mathematics.

01:42:51.760 | I don't care about these. Like, that's one of the reasons why he rejected it.

01:42:55.920 | - Yeah, it's hard to convert into words, but at MIT, I'm distinctly aware of the distinction

01:43:05.440 | between when I enter a room, there's a certain kind of music to the way people talk when we're

01:43:10.960 | talking about ideas versus what that music sounds like when we're talking, when it's like bickering

01:43:21.200 | in the space of like, whether it's politics or funding or egos, it's a different sound to it.

01:43:30.080 | And I'm distinctly aware of the two. And I kind of, sort of, to me personally, happiness was just

01:43:38.960 | like swimming around the one that like is the political stuff or the money stuff and all that,

01:43:46.080 | or egos. And I think that's probably what Perlman is as well. Like the moment he senses there's any,

01:43:53.440 | as with the Fields Medal, like the moment you start to have any kind of drama around credit

01:43:59.440 | assignment, all those kinds of things, it's almost not that it's important who gets the credit. It's

01:44:04.560 | like the drama in itself gets in the way of the exploration of the ideas or the fundamental thing

01:44:10.000 | that makes science so damn beautiful. - And you can really see that this is also

01:44:14.240 | a product of that Russian school of like doing science. And you can see that people were,

01:44:21.760 | during the Cold War, a lot of mathematicians, they were not making any money. They were doing math

01:44:27.760 | for the sake of math, like for the intellectual pleasure of like solving a difficult problem.

01:44:33.600 | And even if it was a flawed system and there were a lot of problems with that, they were able to

01:44:41.200 | actually achieve these. And there were a lot of, and Perlman for me is the perfect product of that.

01:44:46.560 | He just cared about like working on tough problems. He didn't care about anything else. It was just

01:44:51.920 | math, pure math. - Yeah, there's a, like for the broader

01:44:57.120 | audience, I think another example of that is like professional sports versus Olympics. Especially

01:45:03.680 | in Russia, I've seen that clear distinction where, because the state manages so much of the Olympic

01:45:11.840 | process in Russia, as people know, with the steroids, yes, yes, yes. But outside of steroids

01:45:17.280 | thing is like the athlete can focus on the pure artistry of the sport. Like not worry about the

01:45:27.440 | money, not just in the way they talk about it, the way they think about it, the way they define

01:45:32.400 | excellence versus like in the, perhaps a bit of a capitalist system in the United States with

01:45:39.040 | American football, with baseball, basketball, so much of the discussion is about money.

01:45:47.040 | Now, of course, at the end of the day, it's about excellence and artistry and all of that.

01:45:52.400 | But when the culture is so richly grounded in discussions of money and

01:46:00.240 | sort of this capitalistic like merch and businesses and all those kinds of things,

01:46:06.400 | it changes the nature of the activity. And it's in a way that's hard again to describe in words,

01:46:12.160 | but when it's purely about the activity itself, it's almost like you quiet down all the noise

01:46:20.400 | enough to hear the signal, enough to hear the beauty. Like whenever you're talking about the

01:46:25.200 | money, that's when the marketing people come and the business people, the non-creators come and

01:46:30.640 | they fill the room and they create drama and they know how to create the drama and the noise,

01:46:35.200 | as opposed to the people who are truly excellent at what they do, the person in their arena.

01:46:41.120 | Like when you remove all the money and you just let that thing shine, that's when true excellence

01:46:50.000 | can come out. And that was of the few things that worked with the communist system,

01:46:55.200 | the Soviet Union, to me at least, as somebody who loves sport and loves mathematics and

01:47:01.280 | science, that worked well, removing the money from the picture.

01:47:06.320 | Not that I'm saying poverty is good for science. There's some level in which not worrying about

01:47:16.880 | money is good for science. It's a weird, I'm not exactly sure what to make of that because capitalism

01:47:22.560 | works really damn well, but it's tricky how to find that balance. - One Fields Medalist that is

01:47:32.080 | interesting to look at, and I think you mentioned it earlier, but it's Cédric Villani, which is,

01:47:36.240 | might be the only Fields Medalist that is also a politician now. But so it's this brilliant French

01:47:44.400 | mathematician that won the Fields Medal. And after that, he decided that one of the ways that he

01:47:52.560 | could have the biggest leverage kind of in pushing science in the direction that he thinks

01:47:59.680 | science should go would be to try to go into politics. And so that's what he did. And he has

01:48:07.680 | ran, I'm not sure if he has won any election, but- - I think he's running for mayor of Paris.

01:48:13.440 | - Paris or something like that. But it's this brilliant mathematician that before winning the

01:48:18.880 | Fields Medal had only been just a brilliant mathematician. But after that, he decided to

01:48:24.720 | go into politics to try to have an impact and try to change some of the things that he would complain

01:48:30.320 | about before. So there's that component as well. - Yeah, and I've always thought mathematics and

01:48:37.840 | science should be like, like James Bond would, in my eyes, I think be sexier if he did math.

01:48:46.000 | Like we should, as a society, put excellence in mathematics at the same level as being able to

01:48:52.480 | kill a man with your bare hands. Like those are both useful features, like that's admirable. It's

01:48:57.600 | like, oh, like that makes you like, that makes the person interesting. Like being extremely well read

01:49:03.440 | about history or philosophy, being good in mathematics, being able to kill a man with

01:49:08.160 | bare hands. Those are all the same in my book. So I think all are useful for action stars.

01:49:13.120 | And I think the society will benefit for giving more value to that. Like one of the things that

01:49:18.640 | bothers me about American culture is the, I don't know the right words to use, but like the nerdiness

01:49:27.360 | associated with science. Like, I don't think nerd is a good word in American culture because

01:49:37.440 | it's seen as like weakness. There's like images that come with that. And it's fine, you could be

01:49:44.640 | all kinds of shapes and colors and personalities, but like, to me, having sophisticated knowledge

01:49:52.960 | in science, being good at math, doesn't mean you're weak. In fact, it could be the very opposite.

01:50:00.560 | And so it's an interesting thing because it was very much differently viewed in the Soviet Union.

01:50:07.600 | So I know for sure as an existence proof that it doesn't have to be that way. But it...

01:50:14.080 | - I also feel like we lack a lot of role models in terms, if you ask people to mention one

01:50:22.400 | mathematician that they know that is alive today, I think a lot of people would struggle

01:50:26.400 | to answer that question. - And I also think, I love Neil deGrasse Tyson.

01:50:32.960 | But there is, having more role models is good. Like different kinds of personalities. He has

01:50:43.520 | kind of fun and it's very, it's like Bill Nye, the science guy, I don't know if you guys know him.

01:50:49.680 | - Net spectrum. - That, yeah. But there's not, like

01:50:54.240 | Feynman is no longer there. Those kinds of personalities.

01:50:58.160 | - Carl Sagan, man. - Even Carl Sagan, yeah.

01:51:00.160 | Like a seriousness that's like not playful. - Not apologetical.

01:51:06.800 | - Yeah, exactly. Not apologetic about being knowledgeable. And in fact, the kind of energy

01:51:15.760 | where you feel self-conscious about not having thought about some of these questions.

01:51:23.280 | Just like when I see James Bond, I feel bad about that I don't, have never killed a man. Like I need

01:51:30.160 | to make sure I fix that. That's the way I feel. So that same way, I want to feel like that way

01:51:34.720 | with Carl Sagan talks, I feel like I need to have that same kind of seriousness about science. Like

01:51:40.000 | if I don't know something, I want to know it well. What about Terence Tao? He's kind of

01:51:45.680 | a superstar. What are your thoughts about him? - True. He's probably one of the most famous

01:51:50.160 | mathematicians alive today. And probably one of, I mean, regardless of like, is of course,

01:51:54.560 | he won a Fields Medal, is a really smart and talented mathematician. It's also like a big

01:52:04.000 | inspiration for us, at least for some of the work that we do with Fermat's library. So Terence Tao

01:52:12.240 | is known for having a big blog and he's pretty open about like his research. And he also,

01:52:18.960 | he tries to make his work as public as possible through his blog posts. In fact, there's a really

01:52:27.280 | interesting problem that got solved a couple of years ago. So Tao was working on a problem,

01:52:34.480 | on an Erdos problem actually. So Paul Erdos was this mathematician from Hungary, and he was known

01:52:42.000 | for like the Erdos, for a lot of things, but one of the things that he was also known was for the

01:52:47.280 | Erdos problem. So he was always like creating these problems and usually associating prizes

01:52:52.960 | with those problems. And a lot of those problems are still open. And there will be, some of them

01:52:58.000 | will be open for like maybe a couple of hundred years. And I think that's actually an interesting

01:53:02.800 | hack for him to collaborate with future mathematicians. His name will keep coming up

01:53:09.680 | for future generations. But so Tao was working on one of these problems called the Erdos discrepancy

01:53:15.440 | and he published a blog post about that problem and he reached like a dead end. And then all of

01:53:23.920 | a sudden there was this guy from Germany that wrote like a comment on his blog post saying,

01:53:29.680 | "Okay, so this problem is like a Sudoku like flavor and some of the machinery that we're using

01:53:36.240 | to solve Sudoku could be used here." And that was actually the key to solve the Erdos discrepancy

01:53:42.160 | problem. So there was a comment on his blog. And I think that for me is an example of like how to do,

01:53:47.680 | again, going back to collaborative science online and the power that it has. But Tao is also like

01:53:55.920 | pretty public about like some of the struggles of being a mathematician. And even he wrote about

01:54:04.400 | some of the unintended consequences of having extraordinary ability in a field. And he used

01:54:10.960 | himself as an example. When he was growing up, he was extremely talented in mathematics from a young

01:54:16.560 | age. Like Tao was a person, he won a medal in like one of the IMOs at the age I think was a gold

01:54:22.960 | medal at the age of 10 or something like that. And so he mentioned that when he was growing up,

01:54:28.240 | like, and especially in college, when he was in a class that he enjoyed, it just came very natural

01:54:34.960 | for him and he didn't have to work hard to just ace the class. And when he found that the class

01:54:40.160 | was boring, like it didn't work and he barely passed. I think in college, he almost failed two

01:54:47.600 | classes. And he was talking about that and how he brought those studying habits or like in existence

01:54:54.880 | of studying habits when he went to Princeton for his PhD. And in Princeton, when he started kind of

01:55:01.200 | delving into more complex problems in classes, he struggled a lot because he didn't have that,

01:55:08.320 | those habits, like he wasn't taking notes and he wasn't studying hard when he faced problems.

01:55:14.720 | And he almost failed out of his PhD, he almost failed his PhD exam. And it talks about like

01:55:22.960 | having this conversation with his advisor and the advisor pointing out like, you're not,

01:55:27.040 | this is not working, you might have to get out of the program. And like how that was a kind of a

01:55:32.640 | turning point for him. And like it was super important in his career. So I think Tao is also

01:55:39.440 | like this figure that apart from being just an exceptional mathematician, he's also pretty open

01:55:44.320 | about what it takes to be a mathematician and some of the struggles of these type of careers.

01:55:50.000 | And I think that's super important. - In many ways, he's a contributor to open science and open

01:55:55.920 | humanity. He's being an open human by communicating. Scott Aronson is another in computer

01:56:02.800 | science world who's a very different style, very different style, but there's something about a

01:56:07.680 | blog that is authentic and real and just gives us a window into the mind and soul of these brilliant

01:56:16.080 | folks. So it's definitely a gift. Let me ask you about Fermat's library on Twitter, which,

01:56:22.240 | I mean, I don't know how to describe it. People should definitely just follow Fermat's library

01:56:27.520 | on Twitter. I keep following and unfollowing Fermat's library because it's so, it gives,

01:56:36.960 | when I follow it leads me on down rabbit holes often that are very fruitful, but anyway, so

01:56:48.800 | the posts you do with the, on Twitter are just these beautiful, are things that reveal some

01:56:55.760 | beautiful aspect of mathematics. Is there something you could say about the approach there?

01:57:04.480 | And maybe broadly what you find beautiful about mathematics and then more specifically how

01:57:13.600 | you convert that into a rigorous process of revealing that in tweet form.

01:57:19.200 | - That's a good point. I think there's something about math that a lot of the mathematical content

01:57:24.560 | and papers are like little proofs, has in a way sort of an infinite half-life. What I mean by

01:57:33.760 | that is that if you look at like Euclid's elements, it's as valid today as it was when it was created

01:57:40.080 | like 2000 years ago. And that's not true for a lot of other scientific fields. And so in regards to

01:57:48.080 | Twitter, I think there's also a very, it's a very under-explored platform from a learning perspective.

01:57:56.400 | I think if you look at content on Twitter, it's very easy to consume. It's very easy to read.

01:58:03.920 | And especially when you're trying to explain something, we humans get a dopamine hit if we

01:58:12.080 | learn something new. And that's a very, very powerful feeling. And that's why people go to

01:58:18.800 | classes when you have a really good professor, it's looking for those dopamine hits. And

01:58:24.640 | that's something that we try to explore when we're producing content on Twitter. Imagine if we could,

01:58:32.560 | if you would, on a line to a restaurant, you could go to your phone to learn something new instead of

01:58:37.600 | going to a social network. And I think it's very hard to sometimes to

01:58:46.000 | kind of provide that feeling because you need to sometimes digest content and put it in a way that

01:58:54.560 | it fits 280 characters. And it requires a lot of sometimes time to do that, even though it's easy

01:59:02.400 | to consume, it's hard to make. But once you are able to provide that Eureka moment to people,

01:59:08.320 | that's very powerful to get that dopamine hit and you create this feedback cycle and people come

01:59:14.400 | back for more. And in Twitter, compared to an online course or a book, you have a 0% dropout.

01:59:20.960 | So people will read the content. So it's part of the creators, the person that is creating the

01:59:28.480 | content, if you're able to actually get that feedback cycle, it's super, super powerful.

01:59:33.200 | Yeah. But some of this stuff is like, how the heck do you find that? And I don't know why it's

01:59:39.200 | so appealing. This is from a, what is it? A couple of days ago. I'll just read out the number. 2,

01:59:49.200 | 3, 4, 5, 6, 7, 8, 9 is the largest prime number with consecutive increasing digits.

01:59:54.720 | I mean, that is so cool. That's like some weird like glimpse into some deep universal truth,

02:00:04.640 | even though it's just the number. I mean, that's like so arbitrary. Like why is it so pleasant

02:00:10.880 | that that's a thing? But it is in some way, it's almost like it is a little glimpse at some

02:00:16.240 | much bigger like. - And I think like, especially if we're

02:00:20.880 | talking about science, there's something unique about you go, and with a lot of the tweets,

02:00:26.000 | you go sometimes from a state of not knowing something to knowing something. And that is very

02:00:30.800 | particular to science, math, physics, and that again, is extra extremely addictive. And that's

02:00:36.800 | how I feel about that. And that's why I think people engage so much with our tweets and go

02:00:44.320 | into rabbit holes. And then they, you start with prime numbers and all of a sudden you are spending

02:00:49.440 | hours reading number theory things and you go into Wikipedia and you lose a lot of time there.

02:00:55.760 | - Well, the variety is really interesting too. There's human things, there's physics things,

02:01:01.840 | there's like numeric things, like I just mentioned, but there's also more rigorous mathematical

02:01:08.400 | things. There's stuff that's tied to the history of math and the proofs and there's visual,

02:01:13.760 | there's animations, there are looping animations that are incredible, that reveal something.

02:01:18.800 | There's Andrew Wiles on being smart. This is just me now, like ignoring you guys and just going

02:01:26.960 | through your Twitter. - No, yeah, we're a bit like

02:01:28.480 | math drug dealers. We're just trying to get you hooked. We're trying to give you that hit

02:01:32.960 | and trying to get you hooked. - Yes, some people are brighter than

02:01:36.400 | others, but I really believe that most people can really get to quite a good level of mathematics

02:01:42.080 | if they're prepared to deal with these psychological issues of how to handle the

02:01:46.000 | situation of being stuck. - Yeah.

02:01:47.760 | - Yeah, there's some truth to that. - That's truth. I feel that's like really,

02:01:51.600 | it's some truth in terms of research and also about startups. You're stuck a lot of the time

02:01:57.840 | before you get to a breakthrough and it's difficult to endure that process of like being stuck

02:02:03.440 | because you're not trained to be in that position. I feel, yeah, that's-

02:02:09.280 | - Yeah, most people are broken by the stuckness or like they're distracted. I've been very

02:02:16.960 | cognizant of the fact that more and more social media becomes a thing, like distractions become

02:02:25.520 | a thing that that moment of being stuck is your mind wants to go do stuff that's unrelated to

02:02:32.960 | being stuck and you should be stuck. I'm referring to small stucknesses, like you're like trying to

02:02:40.160 | design something and it's a dead end, basically little dead ends, dead ends of programming,

02:02:44.800 | dead ends in trying to think through something. And then your mind wants to like, this is the

02:02:51.680 | problem with this like work-life balance culture is like take a break, like as if taking a break

02:02:59.760 | will solve everything. Sometimes it solves quite a bit, but like sometimes you need to sit in a

02:03:03.920 | stuckness and suffer a little bit and then take a break, but you definitely need to be,

02:03:09.040 | and like most people quit from that psychological battle of being stuck.

02:03:14.640 | So success is people who persevere through that. - Yeah, and in the creative process,

02:03:22.960 | that's also true. I was, the other day I was, I think it was reading about, what is his name?

02:03:28.400 | Ed Sheeran, like the musician was talking a little bit about the creative process and he was using

02:03:33.360 | this analogy of a faucet, like where you, when you turn on a faucet, it's as like the dirty water

02:03:38.800 | coming out in the beginning and you just have to keep trusting that at some point your clean,

02:03:44.960 | clean, clear water will come out, but you have to endure that process. Like in the beginning,

02:03:49.040 | it's going to be dirty water and just embrace that. - Yeah, actually this, the entirety of my

02:03:56.880 | YouTube channel and this podcast have been following that philosophy of dirty water.

02:04:01.120 | Like I've been, you know, I do believe that, like you have to get all the crap out of your system

02:04:07.200 | first. And sometimes it's all, sometimes it's all crappy work. I tend to be very self-critical,

02:04:14.240 | but I do think that quantity leads to quality for some people. It does for my, the way my mind

02:04:19.760 | works is like, just keep putting stuff out there, keep creating and the quality will come as opposed

02:04:27.840 | to sitting there waiting, not doing anything until the thing seems perfect. Cause the perfect may

02:04:34.240 | never come. - But just, just on like on, on, on our Twitter profile, I really, and sometimes when

02:04:41.200 | you look on, on some of those tweets, they might seem like pretty kind of, you know, why is this

02:04:47.120 | interesting? It's like, so raw, like it's just a number, but I really believe that especially with

02:04:53.200 | math or physics, it is possible to get everyone to love math or physics, even if you think you hate

02:04:59.680 | it. It's, it's not a function of the student or the person that is on the other side. I think it's

02:05:04.320 | just purely a function of like how you explain a hidden beauty that they hadn't realized before.

02:05:10.400 | It's not easy, but I think it's like, a lot of the times it's on like on the creator's side

02:05:16.560 | to, to be able to like show that beauty to the other person. - I think some of that is native to,

02:05:22.720 | to humans. We just have that curiosity and you look at small toddlers and babies and like

02:05:28.160 | them trying to figure things out. And there's just something that is born with us that we, we,

02:05:32.640 | we want for that understanding. We want to figure out the world around us. And, and so,

02:05:36.880 | yeah, it shouldn't be like, whether or not people are going, are going to, to enjoy it. Like

02:05:43.920 | I, I, I also really believe that everybody has that capacity to fall in love with, with math and

02:05:49.440 | physics. - You mentioned startup. What do you think it takes to build a successful startup?

02:05:55.360 | - Yeah, that it's what, what Louise was saying that you need to, to be able to endure being stuck.

02:06:03.840 | And, and I think the best way to put it is that startups don't have a linear reward function,

02:06:12.160 | right? You oftentimes don't get rewarded for effort. And, and, and, and most of our lives,

02:06:18.000 | we go through these processes that do give you those small rewards for effort, right? In school,

02:06:25.360 | you study hard, generally you'll get a good grade and then you good, you get like good grades ever,

02:06:30.560 | or you get grades every semester. And so you're, you're slowly getting rewarded and pushed in the

02:06:36.160 | right direction for, for startups and startups are, are not the only thing that is like this,

02:06:41.600 | but for startups, it's, you know, you can put in a ton of effort into something that, and then get

02:06:47.280 | no reward for it, right? It's, it's like, like Sisyphus boulder, where you were pushing that

02:06:51.920 | boulder up the mountain and, and, and you get to the top and then it just rolls all the way back

02:06:57.760 | down. And, and so that's something that I think a lot of people are not equipped to deal with and

02:07:03.120 | can be incredibly demoralizing, especially if that happens more than, than a few times.

02:07:09.200 | And so, but I think it's absolutely essential to, to power through it because by the nature

02:07:16.160 | of startups, it's oftentimes, you know, you're dealing with, with, with non-obvious ideas and

02:07:22.240 | things that there might be contrarian. And so you're gonna, you're gonna run into, into that a

02:07:28.320 | lot. You're going to do things that are not going to work out and you need to be prepared to deal

02:07:33.120 | with that. But, but if we're not coming out of college, you're, you're just not equipped.

02:07:37.600 | I'm not sure if there's a way to train people to deal with those non-linear reward functions,

02:07:44.080 | but it's definitely, I think one of the most difficult things to, you know, about doing a

02:07:49.360 | startup. And also happens in research sometimes, you know, we're talking about the default state

02:07:53.760 | is being stuck. You just, you know, you don't like, you try things, you get zero results,

02:07:58.240 | you close doors, you constantly closing doors until you, you know, find something. And yeah,

02:08:04.640 | that is a big thing. - What about sort of this point when you're stuck, there's a kind of decision,

02:08:10.160 | whether you, if you have a vision to persist through, through with this direction that you've

02:08:15.760 | been going along or what a lot of startups do or businesses is pivot. How do you decide whether

02:08:22.720 | like to give up on a particular flavor of the way you've imagined the design and to like adjust it

02:08:31.520 | or completely like alter it? - I think that's a core question for startups that I've asked

02:08:39.200 | myself exactly. And like, I've never been able to come up with a great framework to make those

02:08:44.640 | decisions. I think that's really at the core of, yeah, out of a lot of the toughest questions that

02:08:52.320 | people that started a company have to deal with. - I think maybe the best framework that I

02:08:59.280 | was able to figure out is like when you run out of ideas, you just, you know, you're exploring

02:09:05.920 | something is not working, you try it in a different angle, you know, we try a different business model.

02:09:11.760 | When you run out of ideas, like you don't have any more cards, just switch. And yeah,

02:09:18.560 | it's not perfect because you also, it's, you have a lot of stories of startups was like,

02:09:23.520 | people kept pushing and then, you know, that paid off. And then you have philosophies,

02:09:30.240 | there's like fail fast and pivot fast. So it's, you know, it's hard to, you know,

02:09:36.080 | balance these two worlds and understand what is the best framework. - And I mean,

02:09:41.120 | if you look at Fermat's library, you're maybe you can correct me, but it feels like you're

02:09:46.800 | operating in a space where there's a lot of things that are broken or could be significantly improved.

02:09:54.960 | So it feels like there's a lot of possibilities for pivoting or like, how do you revolutionize

02:10:01.200 | science? How do you revolutionize the aggregation, the annotation, the commenting, the community

02:10:09.360 | around information of knowledge, structured knowledge? I mean, that's kind of what like

02:10:13.840 | Stack Overflow and Stack Exchange has struggled with to come up with a solution. And they've come

02:10:19.520 | up, I think, with an interesting set of solutions that are also, I think, flawed in some ways,

02:10:24.160 | but they're much, much better than the alternatives. But there's a lot of other

02:10:28.560 | possibilities. If we just look at papers, as we talked about, there's so many possible revolutions

02:10:33.120 | and there are a lot of money to be potentially made as revolutions, plus coupled with that,

02:10:38.480 | the benefit to humanity. And so like you're sitting there, like, I don't know how many

02:10:43.760 | people are legitimately from a business perspective, playing with these ideas. It feels like

02:10:48.880 | there's a lot of ideas here. - True, it varies.

02:10:51.360 | - Are you right now grinding in a particular direction? Like, is there a five-year vision

02:10:56.960 | that you're thinking in your mind? - For us, it's more like a 20-year vision

02:11:02.000 | in the sense that we've consciously tried to make the decision of... So we run Formats as

02:11:10.800 | it's a side project. And it's a separate in the sense like it's not what we're working on full

02:11:15.840 | time. But our thesis there is that we actually think that that's a good thing, at least for

02:11:25.280 | this stage of Formats library. And also because some of these projects, if you're coming from

02:11:32.960 | a startup framework, you probably try to fit every single idea into something that can change

02:11:41.920 | the world within three to five years. And there's just some problems that take longer than that.

02:11:47.280 | Right? And so, we were talking about archive and I'm very doubtful that you could grow

02:11:52.560 | like archive into what it is today, like within two or three years, no matter how much money you

02:11:57.840 | throw at it, there's just some things that can take longer. But you need to be able to power

02:12:02.560 | through the time that it takes. But if you look at it as, "Okay, this is a company, this is a

02:12:09.520 | startup, we have to grow fast, we have to raise money," then sometimes you might forego those

02:12:15.040 | ideas because of that, because they don't very well fit into the typical startup framework.

02:12:23.680 | And so, for us, Formats, it's something that we're okay with having it grow slowly and maybe taking

02:12:30.720 | many years. And that's why we think it's not a bad thing that it is a side project, because it

02:12:36.800 | makes it much more acceptable in a way to be able to be okay with that.

02:12:44.000 | - That said, I think what happens is if you keep pushing new little features,

02:12:49.040 | new little ideas, I'd feel like there's like certain ideas will just become viral.

02:12:53.760 | And then you just won't be able to help yourself, but it'll revolutionize things. It feels like

02:12:59.920 | there needs to be, not needs to be, but there's opportunity for viral ideas to change science.

02:13:08.080 | - Absolutely.

02:13:08.880 | - And maybe we don't know what those are yet. It might be a very small kind of thing.

02:13:13.280 | - Maybe you don't even know if, should this be a for-profit company doing this?

02:13:17.520 | - Right. It's the Wikipedia question.

02:13:19.040 | - Yeah. There are a lot of questions, like really fundamental questions about this space

02:13:24.960 | that we've talked about.

02:13:26.800 | - I mean, you take Wikipedia and you try to run it as a startup, and by now it would have a paywall,

02:13:31.120 | you'd be paying 9.99 a month to read more than 20 articles.

02:13:34.320 | - I mean, that's one view. The other, the ad-driven model. So, they rejected the ad-driven model.

02:13:42.640 | I don't know if we could, I mean, this is a difficult question. If Archive was supported

02:13:47.600 | by ads, I don't know if that's bad for Archive. If Fermat's Library was supported by ads,

02:13:54.160 | I don't know. It's not trivial to me. Unlike, I think a lot of people,

02:13:59.760 | I'm not against advertisements. I think ads, when done well, are really good. I think the problem

02:14:06.320 | with Facebook and all the social networks are the way, the lack of transparency around the way they

02:14:11.280 | use data and the lack of control the users have over their data. Not the fact that data is being

02:14:17.120 | collected and used to sell advertisements. It's the lack of transparency, lack of control. If you

02:14:22.880 | do a good job of that, I feel like it's a really nice way to make stuff free.

02:14:26.560 | - Yeah. It's like Stack Overflow, right? I mean, I think they've done a good job with that,

02:14:32.720 | even though, as we said, they're capturing very little of the value that they're putting out

02:14:37.120 | there, right? But it makes it a sustainable company and they're providing a lot of, it's a

02:14:44.160 | fantastic and very productive community. - Let me ask a ridiculous tangent of a

02:14:49.920 | question. Luis, you wrote a paper on Game of Thrones, Battle of Winterfell, just as a side

02:14:56.560 | little, I'm sorry, I noticed, I'm sure you've done a lot of ridiculous stuff like this. I just

02:15:00.800 | noticed that particular one. By ridiculous, I mean ridiculously awesome. Can you describe

02:15:06.560 | the approach in this work, which I believe is a legitimate publication?

02:15:10.240 | - So going back to the original, like when we were talking about the backstory of papers and

02:15:16.480 | the importance of that. So this is actually, when the last season of the show was airing,

02:15:22.880 | this was during a company lunch. In the last season, there's a really big battle against the

02:15:32.800 | forces of evil and the forces of good. And this is called the Battle of Winterfell. And in this

02:15:41.840 | battle, there are like these two armies, and there's a very particular thing that they have

02:15:46.400 | to take into account is that in the army of dead, like if someone dies in the army of the living,

02:15:52.320 | like that person is gonna be reborn as a soldier in the army of the dead. And so that was an

02:16:00.880 | important thing to take into account. - And the initial conditions, as you

02:16:04.000 | specify, it's about 100,000 on each side. - Exactly. So I was able to, like based on

02:16:08.800 | some images, like on previous episodes to figure out what was the size of the armies. And so what

02:16:13.040 | I want, what we wanted to, what we were theorizing was like, how many soldiers does like a soldier on

02:16:19.840 | the army of the living has to kill in order for them to be able to destroy the army of the dead

02:16:26.880 | without like losing, because every time one of the good soldiers dies, it's gonna turn into like

02:16:32.880 | the other side. And so we were theorizing that and I wrote a couple of differential equations,

02:16:39.760 | and I was able to figure out that based on the size of the armies, I think was the ratio had to

02:16:44.400 | be like 1.7. So it had to kill like 1.7 soldiers of like the army of the dead in order for them to

02:16:51.040 | win the battle. - Yeah, that's science. It's the most powerful. And this is also somehow a pitch

02:17:00.000 | for like a hiring pitch in a sense like this is the kind of important science you do at lunch.

02:17:07.360 | - Exactly. Well, it turned out to be, you know, as for people that have watched these shows,

02:17:12.640 | it's like they know that every time you try to predict something that is gonna happen,

02:17:16.640 | it's gonna, you're gonna fail miserably. And that's what happened. So it was not at all important

02:17:21.440 | for the show, but we ended up like putting that out and there was a lot of people that shared

02:17:26.960 | that. I think it was some like elements of the show, the cast of the show that actually retweeted

02:17:32.400 | that and shared that. So it was fun. - I would love if this kind of calculation

02:17:36.960 | happened like during the making of the show or, you know, I love it like in,

02:17:44.640 | for example, I now know Alex Garland, the director of Ex Machina, and I love it. And he doesn't seem

02:17:51.200 | to be some, not many people seem to do this, but I love it when directors and people who wrote the

02:17:58.560 | story really think through the technical details. Like whether it's knowing like how things, even

02:18:05.120 | if it's science fiction, if you were to try to do this, how would you do this? Like Stephen Wolfram

02:18:10.960 | and his son were collaborating with the movie Arrival in designing the alien language of how

02:18:18.320 | you communicate with the aliens. Like how would you really have a math-based language that could

02:18:25.680 | span the alien and being and the human being? So I love it when they have that kind of rigor.

02:18:33.280 | - The Martian was also big on that. Like the book and the movie was all about like,

02:18:36.960 | can we actually, is this plausible? Can this happen? It was all about that.

02:18:42.240 | - And that can really bring you in. Like sometimes the small details, I mean,

02:18:47.360 | the guy that wrote the Martian book is another book that is also filled with those like things

02:18:53.440 | that when you realize that, okay, these are grounded in science can just really bring you in.

02:18:58.960 | - Yeah. - Like he has a book about a colony on-

02:19:02.800 | - A colony on the moon. And he goes about like all the details that would be required about

02:19:07.600 | setting up a colony in the moon and like things that he wouldn't think about. Like the fact that

02:19:12.800 | they would, it's hard to bring like air to the moon so they wouldn't like, how do you make that

02:19:20.400 | breathable, that environment breathable? You need to bring oxygen, but like you probably wouldn't

02:19:26.480 | bring nitrogen. So what you do is like, instead of having an atmosphere that is a hundred percent

02:19:33.280 | oxygen, you like decrease the pressure so that you have the same ratio of oxygen on earth, but like

02:19:39.040 | lowering the pressure here. And so like things like water boils at the lower temperature. So

02:19:45.120 | people would have coffee and the coffee would be colder. Like there was a problem in this environment

02:19:50.880 | in the moon. So like, and these are like small things in the book, but I studied physics. So

02:19:56.880 | like when I read these, that throws me into like tangents and I start researching that. And it's

02:20:04.080 | like, I really like to read books and watch movies when they go to that level of detail about science.

02:20:11.840 | - Yeah. I think Interstellar was one where they also consulted heavily with a number of people.

02:20:15.840 | - Yeah, yeah, yeah. - I think even resulted in-

02:20:17.440 | - In a couple of papers. - A couple of papers about like the black

02:20:19.840 | hole visualizations and yeah. But there's even more examples of interesting science around like

02:20:27.840 | these fantasy. We were reading at some point, like these guys that were trying to figure out if

02:20:34.080 | the Tolkien's middle earth, if it was round, if it was like a sphere, if it was like a flat earth.

02:20:43.040 | - Based on the map. - Based on the map and some of the

02:20:46.160 | references in the books. And so- - Yeah, we actually, I think we tweeted

02:20:51.680 | about that. - Yeah, we did.

02:20:53.040 | - Based on the distance between the cities, you can actually prove that that could be like a map

02:20:58.480 | of a sphere or like a spheroid and you can actually calculate the radius of that planet.

02:21:03.760 | - That's fascinating. I mean, yeah, that's fascinating. But there's something about like

02:21:12.080 | calculating the number, like exactly the calculation you did for the battle of Winterfell

02:21:20.320 | is something fascinating about that because that's not like being, that's very mathematical versus

02:21:27.520 | grounded in physics. And that's really interesting. I mean, that's like injecting mathematics into

02:21:34.720 | fantasy. There's something magical about that. - I see what you're saying. And that for me,

02:21:40.320 | that's why I think it's also when you look at things like Fermat's Last Theorem, like problems

02:21:47.200 | that are very kind of self-contained and simple to state. I think like that's the same with that

02:21:52.640 | paper. It's very easy to understand the boundaries of the problem. And that for me, that's why math

02:22:01.360 | is so appealing. And those like problems are also so appealing to the general public. It's not that

02:22:06.400 | they look simple or that people think that they're easy to like solve, but I feel that a lot of the

02:22:12.400 | times they are almost intellectually democratic because everyone understands the starting point.

02:22:18.240 | You know, you look at Fermat's Last Theorem, everyone understands like, this is the universe

02:22:24.000 | of the problem. And the same, maybe with that paper, everyone understands, okay, these are

02:22:27.440 | the starting conditions. And yeah, the fact that it becomes intellectually democratic, and I think

02:22:34.640 | that's a huge motivation for people. And that's why so many people gravitate towards these like

02:22:40.160 | Riemann hypotheses or Fermat's Last Theorem or that simple paper, which is like just one page.

02:22:44.560 | It was very simple. - And I just talked to somebody, I don't know if you know who he is,

02:22:48.880 | Jocko Willink, who is this person who among many things loves military tactics. So he would

02:22:58.080 | probably either publish a follow-on paper, maybe you guys should collaborate, but he would see the

02:23:04.000 | fundamental, the basic assumptions that he started that paper with is flawed because, you know,

02:23:08.960 | there's like dragons too, right? There's like, you have to integrate tactics because it's not

02:23:15.280 | a homogeneous system. - I don't take into account the dragons and like- - And he would say tactics

02:23:20.880 | fundamentally change the dynamics of the system. And so like- - That's what happened. (laughs)

02:23:26.280 | - So yeah, so at least from a scientific perspective, he was right, but he never

02:23:31.840 | published, so there you go. Let me ask the most important question. You guys are from

02:23:36.400 | Portugal, both? - Yeah. - So who is the greatest soccer player, footballer of all time?

02:23:44.320 | - Yeah, I think we're a little bit biased on this topic, but I mean- - Maradona?

02:23:50.160 | - I have a huge, I have a tremendous respect for what- - Here we go.

02:23:56.880 | (laughs) - This is the political-

02:23:58.960 | - We can convince you. - I mean, I have tremendous respect for what

02:24:03.440 | Ronaldo has achieved in his career. And I think soccer is one of those sports where I think you

02:24:08.960 | can get to maybe be one of the best players in the world if you just have like natural talent.

02:24:15.520 | And even if you don't put a lot of hard work and discipline into soccer, you can be one of

02:24:20.880 | the best players in the world. And I think Ronaldo is kind of like, of course, he's naturally talented,

02:24:26.000 | but he also- - Cristiano Ronaldo,

02:24:27.200 | say the football from Portugal. - Exactly, from Portugal, and not the

02:24:32.320 | Brazilian in this case. And Ronaldo came from nothing. He's known from being probably one of

02:24:38.960 | the hardest working athletes in the game. And I see that sometimes a lot of these discussions about

02:24:44.320 | the best player, a lot of people tend to gravitate towards like, this person is naturally talented

02:24:51.760 | and the other person has to work hard. And so, as if it was bad, if he had to work hard to be good

02:24:59.360 | at something. And I think so many people fall into that trap. And the reason why so many people fall

02:25:07.520 | into that trap is because if you're saying that someone is good and achieved a lot of success by

02:25:14.480 | working hard, as opposed to achieving success because he has some sort of God-given natural

02:25:20.000 | talent that you can't explain why the person was born with that. What does it tell you about you?

02:25:25.040 | It tells you that maybe if you work hard on a lot of fields, you could accomplish a lot of great

02:25:31.280 | things. And I think that's hard to digest for a lot of people. - So in that way, Ronaldo is

02:25:36.160 | inspiring that- - I think so.

02:25:37.600 | - So you find hard work inspiring, but he's way too good looking. That's the-

02:25:41.760 | - Yeah, yeah. - I don't like him probably.

02:25:45.120 | - No, I like the part of the hard work and like of him being like one of the hardest working

02:25:49.760 | athletes in soccer. - So he is to you the greatest of all time?

02:25:53.920 | Is he up to, is he will be number, okay. - I agree.

02:25:57.760 | - Do you concur with this? - I disagree.

02:25:59.360 | - Well, I definitely disagree. I mean, I like him very much. He works hard. I admire,

02:26:04.880 | I admire, you know, what, like he's incredible goal scorer, right? But I,

02:26:17.440 | so first of all, Leo Messi, and there was some confusion because I've kept saying Maradona is

02:26:22.000 | my favorite player, but I think Leo has surpassed them. So it's Messi, then Maradona, then

02:26:30.800 | Pelé for me. But the reason is, there's certain aesthetic definitions of beauty that I admire,

02:26:40.400 | whether it came by hard work or through God given talent or through anything. It doesn't really

02:26:46.720 | amount to me. There's certain aesthetic, like genius when I see it to me, and especially it

02:26:53.840 | doesn't have to be consistent. It is in the case of Messi, in the case of Ronaldo, but just even

02:26:59.360 | moments of genius, which is where Maradona really shines. - Even if that doesn't translate into like

02:27:06.640 | results and goals being scored. - Right, right. And that's the challenge.

02:27:10.400 | I'm like, I did that because that's where people that tell me that Leo Messi's never,

02:27:17.520 | even on strong teams have led his, the national team, people aspire to the world cup, right?

02:27:24.080 | That's really important. And to me, no, it's the moment, like winning to me was never important.

02:27:29.680 | What's more important is the moments of genius. And, but you're talking to the human story and

02:27:40.000 | yeah, Cristiano Ronaldo definitely has a beautiful human story.

02:27:42.640 | - Yeah. And I think you can't, for me, it's hard to decouple those two. I don't just look at the

02:27:49.920 | list of achievements, but I like how he got there and how he keeps pushing the boundaries. It's like

02:27:54.960 | almost 40 and how that sets up an example. Like maybe 10 years ago, I wouldn't have ever imagined

02:28:00.960 | that like one of the top players in the world could be a top player at like 37 or.

02:28:05.280 | - But so, and there's an interesting, the human story is really important, but like,

02:28:09.520 | if you look at Ronaldo, he's like, he's somebody like kids could aspire to be. But at the same

02:28:16.240 | time, I also like Maradona who like is a tragic figure in many ways. It's like the, you know,

02:28:23.280 | the drugs, the temper, all of those things, that's beautiful too. Like I don't necessarily think to

02:28:29.920 | me, the flaws are beautiful too. And athletes, I don't think you need to be perfect from a

02:28:39.120 | personality perspective. Those flaws are also beautiful. So, but yeah, there is something about

02:28:44.720 | hard work and there's also something about the being an underdog and being able to carry a team.

02:28:52.640 | That's an argument from Maradona. I don't know if you can make that argument for Messi and Ronaldo

02:28:59.040 | either, 'cause they've all played on superstar teams for most of their lives. So I don't know

02:29:05.760 | how, you know, it's difficult to know how they would do when they had to work, like did what

02:29:15.200 | Maradona had to do to carry a team on his shoulders. And Pelé did as well, depending on the

02:29:22.000 | context. - Maybe you could argue that with the Portuguese national team, but we have a good team.

02:29:26.720 | Yeah, but maybe with what Maradona did with, you know, Naples and a couple other teams,

02:29:33.200 | it's incredible. - It speaks to the beauty of the game that, you know, we're talking about

02:29:37.040 | all these different players that have, or especially, you know, if you're comparing

02:29:41.040 | Messi and Ronaldo, they have such different, you know, styles of play and also even

02:29:45.280 | their bodies are so different. But these two very different players can be at the top of the game.

02:29:54.160 | And that's not, there are not a lot of other sports where you have that, you know, like you

02:30:00.000 | have kind of a mental image of a basketball player and like the top basketball players kind of fit

02:30:06.400 | that mental image and they look a certain way. But for soccer, there's, it's not so much like

02:30:15.120 | that. And I think that's beautiful, but that really adds something to the sport. - Well,

02:30:21.920 | do you play soccer yourself? Have you played that in your life? What do you find beautiful

02:30:26.160 | about the game? - Yeah, I mean, it's one of the, I'd say it's the biggest sport in Portugal. And

02:30:30.960 | so growing up, we played a lot. - Did you see the paper from DeepMind? I didn't look at it,

02:30:35.520 | where they're like doing some analysis on soccer strategy. - Yeah, interesting. - I saved that

02:30:42.160 | paper. I haven't read it yet. It's actually, when I was in college, I actually did some

02:30:49.680 | research on applying machine learning and statistics in sports. In our case, we're doing

02:30:57.440 | it for basketball. But what they're effectively trying to do was, have you ever watched Moneyball?

02:31:05.440 | So they're trying to do something similar, right? Taking that, in this case, basketball,

02:31:10.640 | taking a statistical approach to basketball. The interesting thing there is that baseball is much

02:31:17.840 | more about having these discrete events that happen kind of in similar conditions. And so it's easier

02:31:22.960 | to take a statistical approach to it. Whereas basketball, it's a much more dynamic game.

02:31:28.080 | It's harder to measure. It's hard to replicate these conditions. And so you have to think about

02:31:36.800 | it in a slightly different way. And so we were doing work on that and working like with the

02:31:41.600 | Celtics to analyze the data that they had. Like they had these cameras in the arena, they were

02:31:46.720 | tracking the players. And so they had a ton of data, but they didn't really know what to do with

02:31:51.120 | it. And so we were doing work on that. And soccer is maybe even a step further. It's a game where

02:31:58.960 | you don't have as many... In basketball, you have a lot of field goals. And so you can measure

02:32:03.360 | success. Soccer, it's more of a poisson process almost, where it's like you have a goal or two

02:32:11.120 | in a game. In terms of metrics, I wonder if there's a way... And I've actually have thought

02:32:15.120 | about this in the past, never coming up with any good solution. If there's a way to definitively

02:32:19.840 | say whether it's messy or not, they're the greatest of all time. Honestly, measure,

02:32:24.160 | like convert the game of soccer into metrics, like you said, baseball. But those moments of genius,

02:32:30.880 | if it's just about goals or passes that led to goals, that feels like it doesn't capture

02:32:38.800 | the genius of the play. You have more metrics, for instance, in chess, and you can try to

02:32:48.080 | understand how hard of a move that was. There's Bobby Fischer has this move that I think it's

02:32:56.080 | called the move of the century, where you have to go so deep into the tree to understand that

02:33:01.680 | that was the right move and you can quantify how hard it was. So it'd be interesting to try to

02:33:06.560 | think of those types of metrics, but say, yeah, for soccer. - Computer vision unlocks some of that

02:33:10.720 | for us. That's one possibility. - I have a cool idea, a computer vision product, Lex, that you

02:33:15.840 | could build for soccer. - Let's go. I'm taking notes. - If you could detect the ball and imagine

02:33:22.640 | that... It seems totally doable right now, but if you could detect when the ball enters one of the

02:33:29.120 | goals and just had a crowd cheering for you when you're playing soccer with your friends, every

02:33:35.680 | time you score a goal, or you had the Champions League song going on, and having that, you go play

02:33:42.080 | soccer with your friends, you just turn that on and there's a computer vision program analyzing

02:33:46.320 | the ball. - Detects the ball. - Detects the ball every time there's a goal. If you miss, there's

02:33:49.680 | the fans are reacting to that. - And then... - Should be pretty simple by now. I think there's

02:33:55.280 | an opportunity there. Just throwing that. - I'm going to go all out. By the way, I did...

02:33:59.920 | I've never released... I was thinking of just putting it on GitHub, but I did write exactly

02:34:04.640 | that, which is the trackers for the players, for the bodies of the player. This is the hard part,

02:34:10.240 | actually. The detection of player bodies and the ball is not hard. What's hard is very robust

02:34:19.040 | tracking through time of each of those. So I wrote a tracker that's pretty damn good. - Is that

02:34:26.560 | open source? You open source? - No, I've never released it. - Interesting. - Because I thought

02:34:31.920 | I need to... This is the perfection thing, because I knew it was going to be like...

02:34:37.520 | It's going to pull me in and it wasn't really that done. And so I've never actually been part

02:34:44.080 | of a GitHub project where it's like really active development. - Interesting. - And I didn't want

02:34:48.080 | to make it... I knew there's a non-zero probability that it will become my life for like a half a

02:34:52.720 | year. That's just how much I love soccer and all those kinds of things. And ultimately, it will be

02:34:58.800 | all for just the joy of analyzing the game, which I'm all for. - I remember you also, in one of the

02:35:05.760 | episodes, you mentioned that you did also a lot of eye tracking analysis on Joe Rogan. - That was

02:35:11.040 | the research side of my life. - Interesting. And you have that library, right? You kind of

02:35:15.280 | downloaded all the episodes. - Allegedly. And of course I didn't, if you're a lawyer and listening

02:35:20.720 | to this. - I was listening to the episode where you mentioned that and I was actually... There

02:35:25.200 | was something that I might ask you for access to that, to allegedly that library. But I was

02:35:32.080 | doing some, not regarding like eye tracking, but I was playing around with analyzing the distribution

02:35:38.880 | of silences on one of the Joe Rogan episodes. So I did that for the Elon conversation, where it's

02:35:46.560 | like, you just take all the silences, after Joe asked the question and Elon responded, and you

02:35:52.240 | plot that distribution and see how that looks like. - Yeah, I think there's a huge opportunity,

02:35:59.200 | especially with long form podcasts, to do that kind of analysis, bigger than Joe. - Exactly. It

02:36:05.600 | has to be a fairly unedited podcast so that you don't cut the silences. - So one of the benefits

02:36:10.400 | I have doing this podcast is, what we're recording today is there's individual audio that's being

02:36:17.840 | recorded. So I have the raw information, when it's published, it's all combined together,

02:36:22.960 | and individual video feeds. So even when you're listening, which I usually do, I only show one

02:36:28.000 | video stream, I can track your blinks and so on. But ultimately, the hope is you don't need that

02:36:37.520 | raw data, because if you don't need the raw data for whatever analysis you're doing, you can then

02:36:41.920 | do a huge number of podcasts. It's quickly growing now, the number, especially comedians,

02:36:47.760 | there's quite a few comedians with long form podcasts, and they have a lot of facial

02:36:54.400 | expressions, they have a lot of fun and all those kinds of things, and it's prone for analysis. -

02:37:00.000 | There's so many interesting things. That idea actually sparked because I was watching

02:37:04.800 | a Q&A by Steve Jobs, and I think it was at MIT. And then, people did a talk there, and then

02:37:13.520 | the Q&A started, and people started asking questions. I was working while listening to it,

02:37:18.400 | and someone asked a question, and he goes on a 20-second silence before answering the question.

02:37:23.760 | I had to check if the video hadn't paused or something. And I was thinking about if that

02:37:31.840 | is a feature of a person, how long on average you take to respond to a question, and if it's like-

02:37:37.680 | - Oh, that's fascinating. - Has to do with how thoughtful you are,

02:37:40.800 | and if that changes over time. - But it also could be, this is a really

02:37:44.720 | fascinating metric, 'cause it also could be, it's certainly a feature of a person, but it's also a

02:37:49.440 | function of the question. - True.

02:37:51.040 | - Like, if you normalize to the person, you can probably infer a bunch of stuff about the

02:37:55.680 | question. So, it's a nice flag, like it's a really strong signal, the length of that silence,

02:38:00.640 | relative to the usual silence they have. So, one, the silence is a measure of how thoughtful they

02:38:06.480 | are, and two, the particular silence is a measure how- - Thoughtful the question was.

02:38:11.040 | - Thoughtful the question was. It's really interesting. I mean, yeah.

02:38:13.600 | - Yeah. I just analyzed Elon's episode, but I think there's room for exploration there.

02:38:21.040 | - I feel like the average for comedians would be, I mean, the time would be so small,

02:38:25.840 | 'cause you're trained to, I would think you're reacting to hecklers, you're reacting to all

02:38:30.160 | sorts of things, you have to be so quick. - Maybe, maybe. Yeah. But some of the greatest

02:38:34.640 | comedians are very good at sitting in the silence. I mean, there's Louis CK, they play with that,

02:38:41.040 | 'cause you have a rhythm. Like Dave Chappelle, a comedian who did a Joe show recently,

02:38:49.760 | he has a, especially when he's just having a conversation, he does long pauses. It's kind

02:38:56.400 | of cool. It's one of the ways to have people hang in your word, is to play with the pauses.

02:39:04.960 | To play with the silences and the emphasis and mid-sentence. There's a bunch of different things

02:39:11.200 | that it'd be interesting to really, really analyze, but still soccer to me is, that one's

02:39:17.520 | fascinating. I just want a conclusive, definitive statement about, 'cause there's so many

02:39:23.520 | soccer highlights of both Messi and Ronaldo. I just feel like the raw data is there.

02:39:31.040 | - In jest, and decide. - Definitive statement.

02:39:35.440 | 'Cause you don't have that with Pelé and Mardona. - Yeah, true.

02:39:38.800 | - But here's a huge amount of high-def data, the annoying, the difficult thing,

02:39:43.840 | and this is really hard for tracking. And this is actually where I kind of gave up.

02:39:47.920 | I didn't really give much effort, but I gave up to the way that highlights or usually football

02:39:57.360 | match filmed is they switch the camera. So they'll do a different switch perspective. So you have to,

02:40:03.360 | it's a really interesting computer vision problem. When the perspective is switched,

02:40:07.280 | you still have a lot of overlap about the players, but the perspective is sufficiently different that

02:40:12.320 | you have to like recompute everything. So there's two ways to solve this. One is doing it the full

02:40:19.600 | way where you're constantly doing the slam problem. You're doing a 3D reconstruction the whole time,

02:40:24.480 | and projecting into that 3D world. But you could also, there could be some hacks.

02:40:28.960 | Then I wonder like some trick where you can hop, like when the perspective shifts,

02:40:35.360 | do a high probability tracking hops from one object to another. But I thought,

02:40:42.480 | especially in exciting moments when you're passing players, like you're doing a single ball dribble

02:40:51.840 | across players and you switch perspective, which is when they often do when you're making a run on

02:40:56.000 | goal. If you switch your perspective, it feels like that's going to be really tricky to get right

02:41:01.040 | automatically. But in that case, for instance, I feel like if somebody released that data set,

02:41:06.640 | where it's like, you just have all like this data set, a massive data set of all these games from

02:41:13.440 | say Ronaldo and Messi, like, and you just add that in like whatever, CSV format and some publicly

02:41:19.680 | available data set like that. I feel like people would just, there would be so many cool things

02:41:25.120 | that you could do with it. And you just set it free and then like the world would like do its

02:41:29.360 | thing. And then like interesting things would come out of it. - By the way, I have this data set. So

02:41:34.480 | the two things I've did of this scale is soccer. So it's body pose and ball tracking for soccer.

02:41:42.320 | And then it's the pupil tracking and blink tracking for, it was Joe Rogan and a few other

02:41:49.120 | podcasts that I did. So those are the two data sets I have. - Did you analyze any of your podcasts?

02:41:54.400 | - No, I think I really started doing this podcast after doing that work and it's difficult to,

02:42:04.080 | maybe I'd be afraid of what I find. I'm already annoyed with my own voice and video,

02:42:12.800 | like editing it. But perhaps that's the honest thing to do. 'Cause one useful thing about doing

02:42:18.880 | computer vision about myself is like, I know what I was thinking at the time. So you can start to

02:42:24.480 | connect the particular, the behavioral peculiarities of like the way you blink, the way

02:42:32.000 | you squint, the way you close your eyes, like talking about details. It's like, for example,

02:42:39.520 | I just closed my eyes. Is that a blink or no? Like figuring that out in terms of timing, in terms of

02:42:46.240 | the blink dynamics is tricky. It's very doable. I think there's universal laws about what is a blink

02:42:53.920 | and what is a closed eye and all those things, plus makeup and eyelashes. I actually have

02:42:59.920 | annoyingly long eyelashes. So I remember when I was doing a lot of this work, I would cut off my

02:43:05.200 | eyelashes, which when like, especially it was funny, like female colleagues were like, what the

02:43:11.040 | fuck are you doing? Like, no, keep the eyelashes. But it, cause it got in the way, made the computer

02:43:16.240 | vision a lot more difficult, but. - Super interesting topics.

02:43:20.240 | - Yeah. But speaking about the one, still on the topic of the data sets for sports, there's one,

02:43:26.400 | one paper and I actually edited it on format and it's, it was published in 90s, 90s, I believe,

02:43:35.440 | 90s or 80s, I forget. But the researcher was effectively looking at the hot end phenomena

02:43:43.920 | in basketball, right? So whether like the fact that you just made a field goal,

02:43:48.960 | if, you know, if on your next attempt, if you're more likely to make it or not.

02:43:54.400 | And it was super interesting cause I mean, he pulled like, I think a hundred undergrads and

02:44:01.760 | I think from Stanford and Cornell and asking people like, do you, do you think that's, that

02:44:06.320 | do you have a higher likelihood of making your free throw if you just, just made one? And I think

02:44:10.880 | it was like 68, 68% said yes, they believe that. And then he looked at the data and this was back

02:44:20.240 | in, as I said, like few decades ago. And so I think he had the data set of about, he looked at

02:44:25.600 | it specifically for free throws and he had a data set of about 5,000 free throws. And

02:44:31.200 | and effectively what he found was that specifically in the case of free throws,

02:44:37.280 | he didn't, for the aggregate data, he didn't find that he couldn't really spot that correlation,

02:44:44.800 | that hot end correlation. So if you made the first one, you weren't more likely to, to make

02:44:50.480 | the second one. What he did find was that they were just better at the second one,

02:44:54.400 | because you just got like maybe a tiny practice and you just attempted once and then, and then

02:44:59.920 | you're going to be better at the next one. And then I, I, then I went and there's a data set

02:45:04.320 | on Kaggle that has like 600,000 free throws. And I re-ran the, the same computations and,

02:45:11.440 | and, and confirmed, like, you can see a very clear pattern that they're just better

02:45:16.080 | at their second free throw. - That's interesting. Cause I think there's

02:45:20.080 | similar, that kind of analysis is so awesome. Cause I think with tennis, they have like,

02:45:25.200 | like a fault, like when you serve, they have analysis of like, are you most likely to miss

02:45:30.000 | the second serve if you missed the first, obviously. I think that's the case. So that

02:45:35.200 | integrates, that's so cool when psychology is converted into metrics in that way. And in sports,

02:45:41.440 | it's especially cool because it's such a constrained system that you can really study

02:45:46.640 | human psychology because it's repeated, it's constrained. So many things are controlled,

02:45:51.520 | which is something you rarely have in, in the wild psychological experiments. So it's cool.

02:45:57.200 | Plus everyone loves it. Like sports is really cool to analyze.

02:46:01.600 | - People actually care about the results. - Yeah. I still think, well, like I,

02:46:08.800 | I know we'll definitely publish this work on Messi versus Ronaldo and

02:46:12.560 | objective, fully objective. - I'd love to peer review.

02:46:19.200 | Yeah, this is very true. This is not past peer review. Let me ask sort of an advice question

02:46:26.560 | to the young folks. You've explored a lot of fascinating ideas in your life. You built a

02:46:32.880 | startup, worked on physics, worked in computer science. What advice would you give to young

02:46:38.640 | people today in high school, maybe early college about life, about career, about science and

02:46:46.000 | mathematics? - I remember like, I read, like, I remember

02:46:51.520 | reading that Poincaré was once asked by a French journal about his advice for young people and

02:47:00.320 | what was his teaching philosophy. And he said that like, one of the most important things that

02:47:05.200 | parents should teach their kids is how to be enthusiastic in regards to like the mysteries

02:47:11.360 | of the world. And that he said, like, striking that balance was actually one of the most important

02:47:16.480 | things between like in education, you know, you want to have your kids be enthusiastic about the

02:47:21.200 | mysteries of the world, but you also don't want to traumatize them. Like if you really force them

02:47:24.640 | into something. And I think like, especially if you're young, I think you should be curious.

02:47:32.640 | And I think you should explore that curiosity to the fullest, to the point where you even become

02:47:40.320 | almost as an expert on that topic. And you might start with something that it's small,

02:47:46.400 | like you might start with, you know, you're interested in numbers and how to factor numbers

02:47:50.720 | into primes. And then all of a sudden you go and you're like lost in number theory and you

02:47:56.160 | discover cryptography. And then all of a sudden you're buying Bitcoin. And I think you should do

02:48:01.520 | this. You should really try to fulfill this curiosity and you should live in a society that

02:48:06.800 | allows you to fulfill this curiosity, which is also important. And I think you should do this

02:48:12.000 | not to get to some sort of status or fame or money, but I think this is the way, this iterative

02:48:17.360 | process, I think this is the way to find happiness. And I think this is also allows you to find the

02:48:23.760 | meaning for your life. I think it's all about like being curious and being able to fulfill

02:48:29.440 | that curiosity and that path to fulfilling that, your curiosity. - Yeah, the star small,

02:48:36.560 | the fire build is kind of interesting way to think about it. - And you never know where you're

02:48:40.320 | going to end up. It's like, for us, Formaz is just a really good example. We started like by

02:48:46.800 | doing this as an internal like thing that we did in the company. And then we started putting out

02:48:52.080 | there and now a lot of people follow it and know about it. And so... - And you still don't know

02:48:56.960 | where Formaz library is going to end up actually. - True, exactly. So yeah, I think that would be my

02:49:03.440 | piece of advice with very limited experience, of course, but yeah, I agree. - I agree. I mean,

02:49:10.800 | is there something from particular, Joao, from the computer science versus physics perspective,

02:49:15.920 | do you regret not doing physics? Do you regret not doing computer science? Which one is the

02:49:22.320 | wiser, the better human being? This is Messi versus Ronaldo. Those are very, I don't know

02:49:29.920 | if you would agree, but they're kind of different disciplines. - True. - Yeah. - Very much so.

02:49:35.280 | I actually, I had that question in my mind. I took physics classes as an undergrad or like

02:49:45.280 | besides what I had to take. And it's definitely something that I considered at some point.

02:49:53.440 | And I do feel like later in life, that might be something that I'm not sure if regret is the right

02:50:04.720 | word, but it's kind of something that I can imagine in an alternative universe, what would

02:50:09.840 | have happened if I had gone into physics. I try to think that like, well, it depends on what your

02:50:19.280 | path ends up being, but that it's not super important, right? Like exactly what you decide to

02:50:25.360 | major on. I think Tim Urban, the blogger had a good visualization of this where it's like,

02:50:34.160 | he has a picture where you have all sorts of paths that you could pursue in your life. And then

02:50:40.480 | maybe you're in the middle of it. And so there's maybe some paths that are not accessible to you,

02:50:44.480 | but like the tree that is still in front of you, it gives you a lot of optionality. And so-

02:50:48.960 | - There's two lessons to learn from that. Like we have a huge number of options now,

02:50:53.440 | and probably you're just one to reflect, like to try to derive wisdom from the one little path

02:51:02.960 | you've taken so far may be flawed because there's all these other paths you could have taken.

02:51:07.040 | So it's like, so one, it's inspiring that you can take any path now. And two,

02:51:13.120 | it's like the path you've taken so far is just one of many possible ones. But it does seem that

02:51:18.880 | physics and computer science both open a lot of doors and a lot of different doors.

02:51:24.800 | - True. - It's very interesting.

02:51:26.480 | - It is. I feel like in this case, and especially in our case, because I could see the difference.

02:51:32.080 | I studied, I went to college in Europe and João went to college here in the US. So I could see

02:51:38.080 | the difference in like, the European system is more rigid in the sense that when you decide

02:51:43.760 | to study physics, you don't have a lot, especially in the early years, you don't have a lot of,

02:51:47.200 | you can't choose to take like a class from like computer science course or something like that.

02:51:52.880 | You don't have a lot of freedom to explore in that sense in university, as opposed to here

02:51:56.640 | in the US where you have more freedom. And I think that's important. I think that's what

02:52:02.880 | constitutes a good kind of educational system is one that gravitates towards the interests of a

02:52:09.120 | student as you progress. But I think in order for you to do that, you need to explore different

02:52:14.480 | areas. And I felt like if I had a chance to take say more computer science class when I was in

02:52:19.680 | college, I would have probably have taken those classes. But yeah, but I ended up like focusing

02:52:24.800 | maybe too much in physics. And I think here, at least my perception is that you can explore more

02:52:32.560 | fields. - But there is a kind of, it's funny, but physics can be difficult. So I don't see too many

02:52:39.440 | computer science people then exploring into physics. It's like the one, not the one, but

02:52:46.400 | one of the beneficial things of physics, it feels like it, what was it, Rutherford that said like,

02:52:54.160 | like basically that physics is the hard thing and everything is easy. So like there's a certain

02:53:00.640 | sense once you've figured out some basic like physics, that it's not that you need the tools

02:53:05.600 | of physics to understand the other disciplines. It's that you're empowered by having done difficult

02:53:11.280 | shit. I mean, the ultimate, I think is probably mathematics there. - Yeah, true. - So maybe just

02:53:17.600 | doing difficult things and proving to yourself that you can do difficult things, whatever those

02:53:22.080 | are. - That's net positive, I believe. - Net positive. - Yeah. And I think like, before I

02:53:27.120 | started a company, I worked in the financial sector for a bit. And I think having a physics

02:53:34.000 | background, I felt I was not afraid of learning finance things. And I think when you come from

02:53:40.000 | those backgrounds, you are generally not afraid of stepping into other fields and learning about

02:53:44.960 | those because I feel they've learned a lot of difficult things and that's an added benefit,

02:53:52.320 | I believe. - This was an incredible conversation, Luis, João. We started with, who do we start

02:54:00.480 | with? Feynman, ended up with Messi and Ronaldo. So this is like the perfect conversation. It's

02:54:05.360 | really an honor that you guys would waste all this time with me today. It was really fun. Thanks

02:54:09.600 | for talking. - Thank you so much for having us. Yeah, thank you so much. - Thanks for listening

02:54:13.840 | to this conversation with Luis and João Batala. And thank you to Skiff, SimplySafe, Indeed, NetSuite

02:54:20.880 | and Four Sigmatic. Check them out in the description to support this podcast. And now

02:54:26.560 | let me leave you with some words from Richard Feynman. "Nobody ever figures out what life is

02:54:31.280 | all about and it doesn't matter. Explore the world. Nearly everything is really interesting

02:54:37.600 | if you go into it deeply enough." Thank you for listening. I hope to see you next time.

02:54:43.280 | [BLANK_AUDIO]

Luís and João Batalha: Fermat's Library and the Art of Studying Papers | Lex Fridman Podcast #209

Chapters