fast.ai APL study session 6

00:00:00.000 | >> Hi, Taneesh, how are you?

00:00:12.440 | >> Hello.

00:00:13.440 | Oh, I'm behind, but, yeah, hello.

00:00:17.200 | >> You're a silhouette today, huh?

00:00:21.800 | >> Yeah.

00:00:22.800 | Hello.

00:00:23.800 | >> Hello, Taneesh, how are you?

00:00:25.880 | >> Good to see you, me, how are you?

00:00:29.480 | >> Good.

00:00:30.480 | Thank you.

00:00:31.480 | Where are you joining us from?

00:00:32.480 | >> I'm from Austin, Texas.

00:00:35.240 | >> Okay.

00:00:36.960 | >> So, our company, when we initially started doing image recognition based deep learning,

00:00:48.280 | we followed your course, and that was the starting point for us on our deep learning

00:00:53.280 | journey.

00:00:54.280 | >> Oh, excellent.

00:00:55.280 | >> Thanks for the course in Austria.

00:00:57.280 | >> I hope it's going okay.

00:01:00.280 | >> Yeah, it's going okay.

00:01:01.280 | I'll learn hardworking on deep learning.

00:01:04.920 | It's the go-to resource for new hires.

00:01:17.960 | >> Hello, hello.

00:01:27.520 | >> Hi, how are you?

00:01:31.880 | >> Oh, I can hear you.

00:01:33.280 | You can hear me.

00:01:34.280 | That's wonderful.

00:01:35.280 | Good, good.

00:01:36.280 | >> Sounds personalized.

00:01:37.280 | >> Yes, I thought I had some audio issues there, but it seems all is good.

00:01:45.400 | How are you doing, Jeremy?

00:01:46.400 | >> I'm okay.

00:01:47.400 | I'm making some good progress on getting the course ready.

00:01:52.240 | It's mainly been about getting the process working well.

00:01:59.400 | >> Wonderful.

00:02:00.400 | >> And I'm illustrating it with Dali two pictures.

00:02:05.920 | >> Wow.

00:02:06.920 | Wow.

00:02:07.920 | That's going to be awesome.

00:02:13.520 | >> Do you have access?

00:02:15.240 | >> I just got it.

00:02:16.240 | So, good timing.

00:02:17.240 | >> Sweet.

00:02:18.240 | >> I still haven't got an access.

00:02:19.680 | I think I signed up like a week later, so that might have been why.

00:02:22.800 | I'm still waiting.

00:02:27.840 | >> I can try and do a demo if you like.

00:02:31.880 | >> That would be awesome.

00:02:33.440 | This is so wonderful with abstract concepts.

00:02:36.560 | I mean, for illustrating articles, you know, I think this is going to have an amazing

00:02:46.240 | -- well, a group of people, a very wide group of people who might benefit from it.

00:02:56.400 | If I were a journalist and were working on an article, I mean, wow, this is so much better

00:03:02.240 | than going to stock photo or whatever it's called.

00:03:05.920 | >> I mean, they've got a non-commercial restriction at the moment, but presumably at some point

00:03:10.720 | they'll have some kind of paid service.

00:03:18.320 | And --

00:03:19.320 | >> Oh, look, Molly's here.

00:03:20.640 | First time I think Molly's joined us.

00:03:22.480 | Hi, Molly.

00:03:23.480 | Have you joined us before?

00:03:28.480 | Possibly not able to chat.

00:03:29.880 | That's all right.

00:03:30.880 | Hello, anyway.

00:03:31.880 | >> No, I haven't had a chance to join before.

00:03:35.040 | >> Oh, well, welcome.

00:03:36.640 | Have you watched any of the videos, done any APLing as yet?

00:03:41.920 | >> I have started the first video I just saw that the study group had started yesterday.

00:03:50.960 | So yeah.

00:03:51.960 | >> No worries.

00:03:52.960 | I'm going to be caught up in no time.

00:04:00.080 | This is what it looks like.

00:04:02.480 | So who wants me to -- yeah, who wants to suggest a prompt?

00:04:14.520 | You can always put it in the chat if you like or just tell me.

00:04:21.480 | What does the fox say?

00:04:22.480 | Wait, you're meant to be giving something to draw a picture of.

00:04:26.640 | Is that so you can draw a picture of?

00:04:29.440 | >> Maybe a student working on an assignment using array programming language.

00:04:36.040 | >> A student working on an assignment using an array programming language.

00:04:45.920 | Do you want like a photo realistic photo?

00:04:49.600 | Do you want a 3D render?

00:04:51.460 | Do you want a pencil drawing?

00:04:54.040 | What kind?

00:04:57.840 | >> Maybe something with colors.

00:05:01.920 | So pencil drawing, not so much, but -- >> You can do a color pencil drawing or oil

00:05:06.240 | pastels.

00:05:07.240 | >> Oh, yeah.

00:05:08.240 | Yeah.

00:05:09.240 | That sounds great.

00:05:10.240 | >> A color pencil drawing.

00:05:13.960 | I don't think Darlene is interested to know what a thing about -- oopsie dozy -- about

00:05:20.960 | array programming from what I've been able to see.

00:05:24.280 | >> Yeah, I don't imagine it should, but, you know, maybe it will surprise us in some way.

00:05:51.680 | >> Digital art for striking high quality images, oh, that's good to know.

00:05:57.600 | It's taking a bit longer than usual for some reason.

00:06:04.960 | >> I wonder if people are going to make movies with Gally and so on, like explain every scene

00:06:13.880 | to it, and it makes -- >> Something's happening.

00:06:17.040 | >> Okay, so this is a problem.

00:06:23.160 | I often find when you kind of put these extra details, it tries to write things about it,

00:06:28.080 | but it doesn't know how to write.

00:06:30.200 | So it doesn't really know what array programming is, so it's written some words that it thinks

00:06:35.040 | look a bit like that.

00:06:36.320 | >> Right.

00:06:37.320 | >> So if we did, like, a programming assignment -- >> Yeah, the first and sixth ones, they

00:06:46.800 | look somewhat accurate, I would say.

00:06:50.160 | >> Yeah, this person's coding by drawing on a screen.

00:06:54.560 | >> Do you have to pay for this, to get access to Gally?

00:06:58.960 | >> This is free.

00:07:01.080 | >> Anyone can just go in here, is it?

00:07:02.800 | >> You have to apply, and then you just wait for, well, months, I think it was.

00:07:09.480 | >> I may try.

00:07:16.360 | >> All right, well, while we're waiting --

00:07:21.080 | >> That's been happening.

00:07:43.120 | Not too much.

00:07:49.200 | >> I added a blog post on my favorite advent of code problem from last year, and --

00:07:56.640 | >> This one here?

00:07:59.200 | >> Yeah, that one was, I don't know, just really, really cool.

00:08:06.240 | It kind of worked for APL very, very well.

00:08:11.200 | >> Smoke basin.

00:08:26.960 | So this is a map representing the height of the ground at that grid point.

00:08:33.200 | This is the high bits, okay, that's cool.

00:08:38.880 | >> Part one, you have to find -- what do you do?

00:08:44.960 | You have to find the load points, I think.

00:08:48.400 | >> Yep, find the load points.

00:08:50.720 | Add one to the value of each load point, and add them up.

00:09:03.680 | Cool.

00:09:10.160 | >> That's a good one from Tanishk, much more creative, a professor who is a cat teaching

00:09:22.480 | deep learning in a classroom, photorealistic.

00:09:28.080 | >> Let's see how this went.

00:09:37.920 | Okay.

00:09:39.600 | >> It is a pencil drawing.

00:09:41.280 | >> Yeah.

00:09:42.280 | >> I wonder if it would do better if you told it was programming in hieroglyphics, but hieroglyphics

00:09:50.560 | are more than array programming.

00:09:53.280 | >> Interestingly, when I started watching something about APL on YouTube, it started

00:10:02.360 | recommending new videos on hieroglyphs and I guess Egypt and stuff, so it can draw the

00:10:12.280 | connection.

00:10:15.280 | All right, let's open up our one.

00:10:31.280 | I don't think there's anything to pull, but just in case -- there is, oh, because of the

00:10:49.680 | GitHub pages.

00:11:02.000 | I thought we could make a custom operator today.

00:11:27.360 | >> A cat is not a professor.

00:11:32.720 | >> A professor who is a cat, hmm.

00:11:35.520 | >> I guess at least it's got glasses on the top left.

00:11:40.240 | >> Yes, that could well, but clearly somebody else in the background is doing the teaching.

00:11:54.600 | I did an interview this morning with A16 sets, a newish online magazine kind of thing.

00:12:03.120 | We talked about this idea that it's kind of prompt engineering as a skill now.

00:12:09.000 | It's a skill I definitely don't have yet.

00:12:13.440 | >> I feel like it doesn't generalize from one model to another.

00:12:17.720 | >> Exactly.

00:12:25.280 | All right.

00:12:31.480 | Has anybody got a favorite, like, drawing program for Mac?

00:12:43.560 | Have you tried Procreate?

00:12:49.480 | >> Not something that fancy, like something just for doing, you know, these kind of things.

00:12:58.600 | And I also need to find out why my right mouse button doesn't work, because since it doesn't

00:13:07.480 | work, I'm not sure how to insert page.

00:13:14.360 | File, no, home, oh, yeah, a page.

00:13:29.040 | I'm guessing everybody is probably familiar with the idea of gradients, but it's fine

00:13:39.520 | if you're not, because I thought we could just briefly mention it.

00:13:44.240 | So if we've got something like a quadratic, then the gradient

00:14:05.560 | at some point is the slope at that point.

00:14:15.640 | So this would be the gradient at this point, is that slope.

00:14:21.760 | And you may or may not remember that the slope is equal to the rise over the run, which is

00:14:32.720 | the change in y over the change in x.

00:14:38.120 | So if this is x2 comma y2, and this is x1 comma y1, then the slope is y, is rise over

00:15:03.040 | the run, I said, so it's from around, is y2 minus y1 over x2 minus x1.

00:15:21.600 | OK, and so what you could do is you could pick some point a bit, you know, like, so this

00:15:28.840 | is the point we're going to pick, and then you could just add a little bit to x, like

00:15:34.720 | 0.01x, and this is the point here.

00:15:38.680 | So in this case, you could actually write y2 in a different way.

00:15:47.600 | So if you've got, so this is like, this is, this is some function of x, like x squared,

00:15:52.920 | this is x, right?

00:15:55.960 | So you could change y2 to instead be function of x plus a little bit, like, say, 0.01, and

00:16:15.520 | y1 would be function of, oops, would be the function of x at the starting point, and then

00:16:26.400 | you divide by the amount that you moved x by, which in this case would be 0.01.

00:16:30.360 | That would be another way of doing the change in y over change in x.

00:16:35.760 | Does that make sense so far?

00:16:40.720 | And so this is like an approximation of the derivative, it's the slope at a point.

00:16:46.800 | And I thought we could try and do this in APL.

00:16:55.600 | So if we picked a function, I'll show you something interesting.

00:17:00.160 | You can do a function a couple of ways.

00:17:01.880 | I think we've already learnt that you can do a function like this.

00:17:14.440 | If you want to do y squared, you can do omega squared, that's one way to do it.

00:17:34.600 | But you could also just do this, right?

00:17:45.680 | That just means power of, so you could do something like this, if that makes sense.

00:17:58.760 | So then actually what we might do first is a nice operator.

00:18:06.880 | So it just returns the power operator.

00:18:09.200 | Yeah, so this is saying that f is the power function.

00:18:15.240 | And that might not sound very interesting yet, but you could make it a bit more interesting

00:18:18.720 | by saying f is a combination of a function and an operator, and we could give that a

00:18:25.920 | more sensible name, for example.

00:18:34.280 | And that's exactly the same as doing, oops, it's exactly the same as doing that.

00:18:47.560 | So here's an example of something we could do.

00:18:49.200 | Let's learn a new operator, which is called bind, which is J.

00:19:12.480 | So we can get the help, oh, that didn't work.

00:19:25.000 | So we can get the help for it by typing help.

00:19:29.360 | Okay, so this thing, this symbol, you'll hear it a lot, it's called Jot.

00:19:37.720 | And it said, oh, this is finished.

00:19:41.440 | There we go.

00:19:42.440 | Professor, there was a cat, it's too hard, but a cat professor is fine.

00:19:48.000 | Not sure what happened with four, but clearly professors point at things, this cat professor's

00:19:55.400 | just faking it because there's a real, in fact, most of the cat professors, not most,

00:20:01.000 | some of them are faking it, but yeah, professor's point at whiteboards with chalk or sticks,

00:20:08.760 | apparently.

00:20:10.880 | It's not obvious, it's deep learning, but it certainly looks very messy.

00:20:14.560 | Yeah, it's not good.

00:20:17.200 | Well, yeah, that one has like some sort of network or something.

00:20:20.120 | It looks like, yeah, interesting.

00:20:26.600 | The first one is maybe looking at some activation function.

00:20:29.800 | Yeah.

00:20:30.800 | Could well be, okay.

00:20:36.640 | Now Jot is a dyadic operator.

00:20:47.080 | So that means that there won't be a monadic.

00:20:59.040 | And it can be a couple of things, it can be beside or bind depending on whether you pass

00:21:08.800 | it functions or arrays.

00:21:14.960 | Now, for Python programmers, let's call this Python equivalence partial.

00:21:39.200 | It's the same as using partial in Python.

00:21:53.200 | So that's a kind of a functional programming idea, and most of the functional programming

00:21:57.800 | ideas in the standard library are in the func tools module.

00:22:01.760 | That's where you can import partial from, and here's what partial does.

00:22:08.880 | We define something, and so let's say we could call this power, which does that, three squared.

00:22:28.160 | Okay, so what if we wanted to now define squared using power?

00:22:34.480 | We could say square equals partial l comma y equals two.

00:22:43.440 | And what that says is, so partial is a higher order function or an APL operator, something

00:22:50.200 | that returns a function, and the function it returns is something that's going to call

00:22:54.280 | this function passing in this parameter that's going to always set y in the function to two.

00:23:02.680 | So I could then say squared three.

00:23:05.800 | Does that make sense?

00:23:11.940 | So bind does the same thing.

00:23:18.040 | So we could say squared equals, and we can create a function.

00:23:31.560 | So we're doing a power of function, we're doing to the power of two.

00:23:40.680 | So this is a function, we don't say equals, obviously.

00:23:54.840 | And so you can see here, this is this idea that we don't need to use curly brackets and

00:24:03.080 | omega and stuff, we can just define a function directly.

00:24:08.520 | This is called point free programming in other languages, and it's basically where you write

00:24:17.220 | functions without specifically referring to the parameters they're going to take.

00:24:21.680 | And in this case, because this is an operator that therefore returns a function, there's

00:24:30.000 | no need for round brackets and whatnot.

00:24:34.320 | Does that make sense?

00:24:43.360 | An operator that returns a function.

00:24:45.780 | All operators return a function, that's the definition of an operator in APL.

00:24:49.360 | Do you remember last time we did operators, we learned about backslash and slash?

00:24:54.920 | Yes.

00:24:55.920 | And those are?

00:24:57.520 | So it's the B site or bind that's an operator, right?

00:25:00.920 | So it's return, it returns a function.

00:25:04.520 | Okay.

00:25:05.520 | So it takes the argument on the left and on the right.

00:25:10.920 | That's right.

00:25:11.920 | And returns a function.

00:25:12.920 | And returns a function.

00:25:13.920 | So this is the function to apply, and then this is the right hand side to bind.

00:25:21.880 | And it returns a function.

00:25:23.720 | And it's an operator, therefore it returns a function.

00:25:25.960 | Got it.

00:25:26.960 | And you can do it the other way as well, which is you could say a power of 2, and then you

00:25:37.880 | could just do the opposite, 2 to the power of, so this means 2 to the power of.

00:25:43.800 | Rather than to the power of 2, so 2 to the power of 3 is 8.

00:25:52.440 | So that's the equivalent of, this is the equivalent of partial where I bind the right hand side.

00:25:57.520 | This is the equivalent of partial where I bind the left hand side.

00:26:00.520 | Wow.

00:26:01.520 | That makes sense?

00:26:02.520 | It makes sense, it's just surprising how succinct it all is, and it's like Lego bricks.

00:26:10.840 | It is, yeah.

00:26:15.980 | So let's move some of this stuff into the right spot.

00:26:26.440 | Okay, so then, yeah, let's just go like this.

00:26:41.800 | So this one's called bind.

00:26:44.100 | And bind's a very common computer science term for this, partial function application

00:26:51.080 | or bind, in C++ it's called bind.

00:26:56.440 | Now the other thing we could do is we could do beside, and beside is what happens if you

00:27:06.200 | put, if you just pass two functions to it.

00:27:13.160 | So for example, we can create a function that first does reciprocal, remember monadic divide

00:27:25.200 | is reciprocal, and then it does e to the power of, remember star is e to the power of.

00:27:33.120 | So this, so if we go, for example, reciprocal of three, and then we do e to the power of

00:27:42.200 | that, it's that, and we should find f of three is the same thing.

00:27:52.280 | Does that make sense?

00:27:55.800 | So this is called function composition, it's at least as one form of it.

00:28:01.440 | So first it does this function, and then takes the result and passes to that function.

00:28:12.480 | You can also use the function that is returned diatically, like so, and that is going to

00:28:36.720 | be the same as, I think, oh it's not, is it, yeah, two, yeah it is, it's going to be two

00:28:51.280 | to the power of a third.

00:28:59.600 | So this will be the cube root of two, yeah, okay, so it first, so in the case if you do

00:29:05.520 | it diatically, then it first applies this to the right hand side, and then it applies this

00:29:11.080 | to the left hand side and the result of that.

00:29:17.320 | And they've got some really nice pictures of this somewhere, I wonder if they're here.

00:29:27.840 | Oh yeah, bind is also called carrying.

00:29:36.320 | So there's lots of ways of doing function composition, and so beside, as you can see,

00:29:40.520 | it takes y on the right, it passes it through g, which is this function, which was reciprocal,

00:29:47.880 | and then f, which was power of, gets the left hand side and the result of that, which interestingly

00:30:00.600 | enough is exactly the same thing as if you just put them next to each other, I think,

00:30:12.800 | oh no that's different, that's interesting, okay, I'll take that back, there's something

00:30:24.920 | in the, it's a, API wiki is good for this stuff, I suppose.

00:30:54.640 | Wait, it says it's the same, that's very confusing, oh it's without the parentheses, okay, so that

00:31:20.880 | does the same thing. So there's a few ways of doing the same thing here.

00:31:31.960 | So it's a bit confusing that, I mean it doesn't need to be, but it can be if you're not careful

00:31:37.200 | that this is a dyadic operator because it takes the left hand side and the right hand

00:31:42.200 | side, it returns a function, the function it returns can either be used monadically or diadically,

00:31:55.800 | that makes sense? And so the way it behaves is based on, if it's used monadically it's

00:32:08.040 | reasonably straightforward, we just first apply this and then we apply that. If it's

00:32:14.800 | diadically then it behaves like this. It makes sense, it's just interesting that

00:32:22.520 | it exists like one questions, you know, what it could be used for.

00:32:28.240 | So the purpose of it is to create, the way I think of it is to expand mathematics, right?

00:32:36.520 | So in mathematics we stick symbols next to each other and they have meanings, right?

00:32:45.400 | But the rules for what symbols you can stick next to each other and when vary a lot depending

00:32:51.480 | on the symbol and stuff like that, you know. So APL tries to just lay out the ground rules

00:32:59.080 | and says this is how you can do it and so let me show you my favourite so far example

00:33:09.040 | which is calculating the golden ratio. We're going to need some more operators for this.

00:33:26.960 | Okay, so we're going to use, the next operator we're going to learn is star diuresis, yeah,

00:33:43.160 | which is shift P.

00:34:03.720 | And this is also called power but it's actually the power operator rather than the power function

00:34:11.440 | which is confusing so you do actually have to be careful. The power function is just

00:34:23.120 | a star. Okay, it is a operator therefore it returns a function, it's dyadic therefore

00:34:39.560 | it needs a left-hand side and a right-hand side. So let's define, I don't know if any

00:34:47.000 | of you have done metamathematics but metamathematics is the philosophical foundations of mathematics

00:34:54.640 | and there's a small set of theorems that you can use which I think are called the piano

00:35:02.360 | theorem, piano axioms, which people used to think you could then derive all of math from

00:35:12.280 | although it turns out it's not necessarily true. And so basically the idea is you'd say

00:35:18.360 | okay we're going to create something called zero and the way to create something called

00:35:23.120 | equals and equals is defined by saying for every number x equals x and that if x equals

00:35:34.800 | y then y equals x and if x equals y and y equals z then x equals z, I mean these are

00:35:42.480 | all obviously true things which are kind of defining some basic theorems. And then it

00:35:49.040 | defines this function called capital S, we just basically say it exists and it returns

00:35:57.040 | another number and basically it defines it in such a way that it's it's the successor

00:36:03.360 | of a number so the successor of zero is one, the successor of one is two, the successor

00:36:07.280 | of two is three and so forth. And we actually can easily define S now because it's plus

00:36:19.280 | one, right? And so if we assume that zero exists and we assume the successor exists

00:36:28.240 | then we can create the number one and we can create the number two and we create the number

00:36:32.840 | three so forth. And so at this point we want to invent addition because in metamathematics

00:36:40.540 | you basically are not allowed to assume anything exists except what you put in your premises.

00:36:44.880 | So addition is what happens when we apply S a bunch of times. If we want to add three

00:36:51.160 | to zero we would write three S's followed by zero. Does that make sense? So the power

00:37:00.040 | function, the power operator simply says how many times to repeat a function. So I want

00:37:08.980 | to repeat the function S three times. So that's the same as writing S space S space S. Okay?

00:37:24.440 | So if we want to create a function called add then we basically are going to apply the S

00:37:37.800 | function alpha times to omega. So that's going to start with, so for example, two added to

00:37:55.320 | three, we'll start with three and then it will apply S twice. That would be the same

00:38:03.080 | as writing S space S space three. Does that make sense? So we just invented addition.

00:38:09.960 | Yes, absolutely it does. So we can do the same thing for modification. We can apply, multiply,

00:38:26.840 | we need to start at one and then we can multiply. So we're going to multiply by, I'm not sure

00:38:40.960 | if it matters which way around this goes, we're going to multiply omega, we're going

00:38:46.560 | to multiply by alpha omega times. Oops. Oh, I wrote volt, which obviously is going to break

00:38:58.600 | everything. I need to add. Okay. I'm adding, oh, I'm adding to zero. There we go. So I'm

00:39:11.280 | adding to zero. This is what I add each time. This is how many times I add. So we just invent

00:39:19.680 | a model play. So we can of course now invent to the power off and that would be where we'd

00:39:31.680 | start with one and we multiply. The number of times we multiply will be, yeah, the thing

00:39:45.400 | on the right. So two to the power of three. Okay. Yeah, it is quite mind bending to look

00:40:00.440 | at the syntax though, because you know, with the star thingy, the star diuresis, star diuresis,

00:40:08.880 | it look already quite interesting when it was, was there a monadic S function, you know,

00:40:18.640 | and now it can also, so yeah, this, this S thing you have in the cell 26, it's just,

00:40:27.320 | you know, on the right hand side of it and it does what it does, but then you can also

00:40:32.880 | use it with dyadic. Yeah. And it's, you know, another example of just composing this very

00:40:44.080 | small piece of functionality that, yeah. So it's applying its left upper end function

00:40:49.440 | cumulatively G times. And if it's a dyadic function, it's applied repeated. Oh no, that's

00:40:58.320 | not what we want. It's bound. So if there's a left argument, it's bound as the left argument.

00:41:05.180 | So we've basically seen this idea, right, of binding an argument. So that's basically

00:41:12.120 | what it's doing. It's saying that this is multiplied by alpha each time. Okay. Now where

00:41:25.240 | it gets really fun is that you don't have to put a, a number on the right. You can put

00:41:39.880 | a function on the right. And so this is going to come some way towards answering your question

00:41:46.960 | Radek about what's all this for. So we've now got a sequence of five glyphs in a row.

00:41:58.640 | Okay. So what does that mean? Well, this cliff here, we know it means take the reciprocal

00:42:12.520 | and then add, and as this is on the left, so it's going to be take the reciprocal and

00:42:16.360 | add one. So let's try it. Let's just grab that copy. We'll call that F. Whoops. And

00:42:34.880 | so if we go F of one and that's sorry, one F one, I should say that equals two, right?

00:42:46.640 | Because it takes the reciprocal of this and then it applies plus to the result and the

00:42:52.960 | thing on the left. So one plus one is two because that's because of this, this is what

00:43:02.520 | we get if we do a beside, we first apply reciprocal to the right and then we apply plus to the

00:43:10.960 | left and the result. So the reciprocal of one is one and then the left hand side is

00:43:20.640 | one and the result of reciprocal is one. And so one plus one is two. And we could take

00:43:26.520 | the result of that and pass it back into exactly the same function with the same left hand

00:43:31.400 | side. And we could do that again, take the result of that and pass it into the right

00:43:37.940 | hand side. And if we do this a bunch of times, we actually are doing something quite interesting,

00:43:49.640 | which is we're creating something called a continued fraction and the continued fraction

00:43:55.200 | that we're creating is this one. So we started with one plus one over one and then we made

00:44:10.320 | it one plus one over, then we made it one plus one over that and then we made it one

00:44:15.440 | plus one over that and then we made it one plus one over that. Now, if we keep doing

00:44:19.400 | that enough times, eventually we're going to get a number called phi. And phi is also

00:44:29.080 | known as the golden ratio. And the golden ratio by the golden ratio appears in nature

00:44:42.400 | and art basically everywhere. So basically, you know, it appears in nature when we look

00:45:00.440 | at kind of the proportions of things, it appears as the ratios in famous paintings, it appears

00:45:06.720 | on the snails shell, it's this number that appears everywhere. And why are we talking

00:45:17.120 | about it? Well, we can calculate it by typing this again and again and again. But that's

00:45:24.240 | going to get pretty boring. We could do this, right? So that's going to do one over, sorry,

00:45:41.800 | one plus one over one. And then it's going to do one over that. And that's going to do

00:45:45.960 | one over that. It's going to do one over that. It's going to do one over that. And I think

00:45:54.320 | we now know that we could replace that with just do f a bunch of times, I don't know,

00:46:05.880 | five times. So that's nice because now we can let go a bit further and get, that's actually

00:46:12.320 | a pretty good estimate of the golden ratio. There you go, yep, about one to 1.618. Does

00:46:24.120 | that make sense so far? Yes. All right. But how do we know how far to go? Well, basically,

00:46:33.240 | we want to keep on applying f until the next time we apply f, the result doesn't change

00:46:40.600 | to within floating point error. If you replace 15 with equals, then the power operator, if

00:46:54.260 | you put this on the right-hand side, repeats this function again and again and again. And

00:46:59.200 | each time it passes to this function, the previous result and the current result. And

00:47:04.880 | it stops if this function returns one. And in math, we call that the fixed point. The

00:47:13.200 | fixed point of a function is the point at which, or a sequence, I guess, is a point

00:47:17.520 | at which it stops changing. So there's exactly the same thing, but iterated exactly around

00:47:25.440 | a amount of times. I'm not sure how to change the precision that we print out things here.

00:47:29.800 | But if you printed this out in high precision and then passed it to itself again, it wouldn't

00:47:34.400 | change. And so if you replace f with its definition, which is this, then you get that. And so the

00:47:52.320 | answer to your question of what's all this for is so that we can write short, concise

00:47:59.920 | mathematical expressions for things like here's the fixed point of the continued fraction

00:48:04.360 | that calculates five. Is that kind of mind blowing? It is very much so. But it's amazing.

00:48:14.600 | It is amazing. And, you know, yeah, there's something delightful, I think, at least to

00:48:21.040 | me about, you know, like, yeah, realizing notation can take you this far. And that, like, you

00:48:33.200 | know, I would much rather write this than this, you know. And this doesn't even -- you

00:48:47.360 | can't even put this in a computer, because what the hell does dot, dot, dot mean? That

00:48:50.680 | means, like, oh, a human ought to be able to guess what goes here. So, yeah, I think

00:48:59.120 | it's beautiful. It's a beautiful notation. It's a powerful notation. And it lets us

00:49:02.640 | express complex things fairly simply, like, once we get the idea. And the nice thing is

00:49:09.560 | then, like, because we were able to express this quite simply, then we can use that as

00:49:18.240 | a thing that we then create another layer of abstraction on top of that. We use that

00:49:22.000 | as an input to something else. You know? So it's because that, you know, if we didn't

00:49:27.360 | have this kind of bind -- sorry, this subside composition idea, then this whole thing wouldn't

00:49:33.920 | really have worked, you know. We can use these ideas in Python as well. In Python, you can

00:49:45.520 | do function composition. And I think Vastcore might have something, if I'm going to correctly.

00:49:55.200 | I can't quite remember what's where. Yeah, compose. So, this is the same as the side

00:50:11.080 | in APL. So you can pass it a list of functions. So, for example, here's one function and here's

00:50:21.960 | another function and here we are composing the two together. It goes in the opposite

00:50:28.080 | order, I think. So it applies F1 first and then F2 correctly. Sorry, what were you saying?

00:50:41.440 | Oh, no, I was going to put you in the compose function. There's a bunch of things like this

00:50:51.760 | in Vastcore. So, for example, I really like partial, so I created a better version of

00:50:55.400 | partial which doesn't throw away documentation. This is basically the same as kind of broadcasting.

00:51:06.120 | There we are. I created my own bind class. Now, I created all these before I did much

00:51:15.680 | with APL because they used another functional programming languages. Possibly or maybe even

00:51:22.720 | probably APL did them first, I'm not sure. There's actually a whole academic world, or

00:51:42.320 | combinatorial logic, which is all about, eliminate the need for variables, kind of like point-free

00:51:53.600 | programming. And the ideas that are in APL, I wouldn't say they come from combinatorial

00:52:03.800 | logic because nobody knows if Ken Iverson had ever seen this stuff before. It's quite

00:52:08.120 | possible he reinvented all this from scratch. But it turns out they're all exactly the same.

00:52:15.000 | And something that Connor in the ArrayCast podcast mentioned, which has just arrived,

00:52:25.600 | is that there's a book of puzzles which actually cover lots of the combinatorial logic ideas.

00:52:39.720 | So I will let you know once I start going through what I think. So there's a lot of

00:52:51.920 | ideas of the onion we can peel off. And it turns out that we kind of find ourselves in

00:52:59.480 | all these different areas of math and logic and philosophy and whatever.

00:53:07.520 | Jeremy, if you check that book out, you have to let me know how it goes. I checked it out

00:53:13.440 | before maybe a couple of years ago. Steven Wolf from Mathematica was talking about combinators.

00:53:23.280 | Maybe it was only a year ago, and I happened to get interested and I checked that book

00:53:27.040 | out and I kind of worked through some, but I got lost pretty quickly. I'm sure that you

00:53:32.720 | will probably fare much better than me. I don't know about that. I'm not sure.

00:53:37.360 | See how it goes. Well, you know, maybe at some point we can start working through it

00:53:42.440 | together if people get interested. Yeah, it's interesting listening to an earlier Raycast

00:53:52.840 | episode where kind of talked about how he just started with APL and he went to some

00:53:58.200 | APL user group meeting or something and said to people, wait, trains? Are they combinators?

00:54:06.120 | And combinators are trains? And everybody at the APL user group meeting, he said, no,

00:54:11.080 | this is a new APL invention. It's not some other thing. Because nobody, you know, intellectual

00:54:19.240 | worlds don't mash very much. Nobody realized. Particularly the APL intellectual world. I've

00:54:26.680 | noticed it's pretty cloistered, to be honest. So anyway, yeah, they are the same thing.

00:54:39.720 | So I'm trying to remember why the hell we were doing all this. This is definitely our

00:54:48.840 | most intense study session, right? And please don't be worried if this feels a bit intimidating

00:55:02.360 | because it is. And that's fine. And we're meant to be learning new interesting parts

00:55:09.560 | of the world and when we find them, we shouldn't expect them to make sense straight away. So

00:55:19.160 | we were doing JWT. That's probably it. That's a good place to finish, isn't it? Yeah, that's

00:55:38.200 | fine. I don't think we need any of this. Oh, that's right. We're going to do custom operators.

00:55:46.600 | Okay. I think we've got some good background to tackle that. Oh, yeah. Okay. So we've got

00:55:55.000 | to here. I guess before we go, we should at least write down something so that we've made

00:56:02.760 | some start here. Oh, yeah. Okay. I remember now. I was going to do squared. That's right.

00:56:19.840 | So I wanted my function to be squared. And so to do that, I had to show you that this

00:56:25.160 | is min squared. Okay. So then to calculate the derivative of this function at some point,

00:56:36.680 | we're going to use this formula we wrote down. This one here. So it's going to be f of let's

00:56:51.280 | create some number to hold up 0.01. So the difference we're going to work on is going

00:56:56.000 | to be 0.01. So it's going to be the fact f of, and so let's calculate this at some point,

00:57:03.880 | say two. Now let's do three. Okay. So we're going to calculate it at X equals three. So

00:57:10.320 | we're going to go function of X plus D. Okay. Minus function of X. Okay. And then that's

00:57:23.240 | going to have to go in parentheses. Okay. And then we're going to divide that by 0.01.

00:57:31.760 | There are better ways to write this in APL, but I want to make it like somewhat familiar.

00:57:39.520 | Okay. And for those of you who remember calculus, the actual derivative of X squared is 2x. So

00:57:46.960 | the correct answer would have been six. So which is, you know, we're on the right track.

00:57:58.660 | If we set D to a smaller number, we would get a more precise answer. Yeah. So what we're

00:58:10.080 | going to try and head to is that we can actually create our own operator that will calculate

00:58:16.080 | the derivative of function at a point. And so maybe that's what we'll try to do next

00:58:20.140 | time. Any comments or questions before we go? That's really cool. Awesome. I'm glad to

00:58:34.040 | hear. Yeah, that's great again. Thank you. All right. All right. Bye all. Thank you for

00:58:40.320 | showing us this. Bye bye. Bye. Thank you.

00:58:43.920 | Thank you.

00:58:45.920 | [BLANK_AUDIO]

fast.ai APL study session 6

Chapters