fast.ai APL study session 5

00:00:00.000 | [BLANK_AUDIO]

00:00:10.000 | [BLANK_AUDIO]

00:00:20.000 | >> That's Rada.

00:00:33.760 | >> Good morning, Jeremy.

00:00:34.880 | [BLANK_AUDIO]

00:00:44.880 | >> Hello, everybody.

00:00:46.880 | [BLANK_AUDIO]

00:00:51.040 | >> I'm not hearing anybody, is anybody hearing me?

00:00:53.080 | >> Yes, we can hear you.

00:00:55.120 | >> Yeah, we can hear you.

00:00:56.040 | >> We can hear you.

00:00:57.200 | >> Cool.

00:00:57.700 | [BLANK_AUDIO]

00:01:03.800 | >> How are you doing?

00:01:05.800 | [BLANK_AUDIO]

00:01:07.800 | >> Doing good.

00:01:08.800 | It's a good day.

00:01:09.800 | [BLANK_AUDIO]

00:01:13.800 | >> I'm here.

00:01:15.800 | >> Hi, Marty, I think you're a new face, right?

00:01:17.800 | [BLANK_AUDIO]

00:01:18.800 | >> What's that?

00:01:19.800 | >> I think you're a new face, right?

00:01:21.800 | [BLANK_AUDIO]

00:01:22.800 | >> I am new, yes.

00:01:24.800 | I've been on actually all of them, but I just don't speak a whole lot.

00:01:29.800 | >> I see.

00:01:30.800 | Nice to see you.

00:01:31.800 | Thanks for joining.

00:01:32.800 | >> Thank you.

00:01:33.800 | [BLANK_AUDIO]

00:01:42.800 | >> I'm getting a bit bright.

00:01:43.800 | Can I close this?

00:01:44.800 | >> Mm-hm.

00:01:45.800 | [BLANK_AUDIO]

00:01:51.800 | >> Is it raining there, Jeremy?

00:01:53.800 | >> Nope, it's nice and sunny.

00:01:55.800 | >> No.

00:01:56.800 | >> It's normally pretty much always sunny here during winter.

00:02:02.800 | Nearly always.

00:02:04.800 | It's our best season.

00:02:06.800 | [BLANK_AUDIO]

00:02:20.800 | >> Getting my windows organized.

00:02:27.800 | [BLANK_AUDIO]

00:02:37.800 | >> Hi, Radek, how are you?

00:02:39.800 | >> Hey, I'm good.

00:02:41.800 | How about yourself?

00:02:43.800 | >> Pretty good.

00:02:45.800 | >> I read the blog post yesterday by Isaac,

00:02:51.800 | the one about the dot operator.

00:02:54.800 | Yeah.

00:02:55.800 | Wow, what a mind bender.

00:02:57.800 | >> [LAUGH]

00:02:58.800 | >> That was--

00:02:59.800 | >> Pretty cool, I agree.

00:03:01.800 | >> Very cool.

00:03:03.800 | >> I guess I should share that so people can see what you're talking about.

00:03:08.800 | >> I read it yesterday before going to sleep, and today I took another look.

00:03:14.800 | It will take me a few more attempts probably to fully understand what's going on there.

00:03:22.800 | I probably need to play with the dot operator because it's, yeah, it is so interesting.

00:03:29.800 | Like, how does it know?

00:03:36.800 | >> I didn't understand it until I finished writing the post myself, so.

00:03:41.800 | >> [LAUGH]

00:03:44.800 | >> How does it know to broadcast the--

00:03:49.800 | because that's what it essentially does.

00:03:52.800 | I guess it broadcasts the--

00:03:55.800 | here it's just a vector, right?

00:03:58.800 | The second-- I'm not sure what works to use because it's--

00:04:02.800 | I guess the first thing is a matrix, right?

00:04:09.800 | >> Maybe go up a level.

00:04:11.800 | Isaac, maybe you can give us a quick overview.

00:04:14.800 | Maybe share your screen or something and give us a quick picture of what you found

00:04:19.800 | and then see if we can answer Radek's question.

00:04:26.800 | Because I really like the inner product operator in APL because I think it's a great example

00:04:32.800 | of the ways in which APL has a tendency to generalize things beyond the original math notation

00:04:40.800 | to make them more expressive, in my opinion.

00:04:52.800 | >> And you're doing the course by Gilbert Strunk.

00:04:54.800 | That's quite an awesome choice.

00:04:59.800 | >> Yeah, I love that course.

00:05:03.800 | It's really good.

00:05:05.800 | >> You know, another course which is great is fast.ai's numerical linear algebra course.

00:05:12.800 | >> Oh, yeah.

00:05:13.800 | I was even-- I was even thinking like yesterday that it might be fun to do the course

00:05:20.800 | but do the exercises in APL.

00:05:22.800 | >> Oh, yeah.

00:05:23.800 | That would actually be very interesting.

00:05:24.800 | >> You know, for that course, yeah.

00:05:28.800 | >> So let's skip ahead a little bit but talk about really what the dot does.

00:05:39.800 | I think this was the clearest.

00:05:40.800 | So here we have this matrix times this matrix, 2, 5, 1, 3, 8 times 1, 2, 3, 4.

00:05:48.800 | And so we're going to multiply these together.

00:05:51.800 | And you can't use the dot by itself because it has to have functions on either side of them.

00:05:57.800 | But if you could use the dot by itself, this is what it would do.

00:06:04.800 | It would-- it gives you the structure of this problem so that it's going to kind of lay it out in this way.

00:06:15.800 | It's going to lay your problem out here.

00:06:17.800 | And you can see, you know, your 1, 2, 3, 4 and your 1, 2, 3, 4, it's all multiplying it.

00:06:23.800 | And it's going to take these-- I guess these are called double alpha and double omega.

00:06:27.800 | I guess I don't know.

00:06:29.800 | But it's going to take whatever is on the left and the right of the dot and apply those functions in this way.

00:06:35.800 | So I kind of think of the inner product operator is it just sets the framework for the problem and then drops,

00:06:42.800 | in this case, the plus and the times functions into these particular spots.

00:06:48.800 | So, for example, double omega will often be times and double alpha will often be plus.

00:06:55.800 | And if you do it in that way, then it's matrix multiplication.

00:07:00.800 | I could swear that yesterday it was a different example of a matrix and a vector.

00:07:06.800 | And now it's a matrix times a matrix.

00:07:09.800 | Oh, yeah.

00:07:10.800 | I got a question. So I added a section.

00:07:12.800 | Sorry about that. Yeah, I got a question. So I added a section.

00:07:15.800 | And it's the same. It's the same idea here.

00:07:18.800 | We do in this case, when we're doing a matrix, this matrix times the vector, we get the exact same structure of the problem.

00:07:29.800 | But they're all scalars because we only have one number multiplied by another.

00:07:35.800 | In this case, that's the double omega multiplication.

00:07:38.800 | It's 1 times 2 plus 2 times 5, 1 times 1 plus 2 times 3.

00:07:43.800 | And so we get we get it in that way.

00:07:46.800 | If we were to do the same thing, right, if one of these, the second one was a matrix, instead of, you know, our one to vector, we had a one, two, three, four matrix.

00:08:01.800 | Instead of this one, two being just numbers, these could be vectors, but we'd still do the same thing. This one could be a vector one, two, three, four, which would use whatever this double omega operator is.

00:08:16.800 | In this case, times and multiply it by this two. So this could be a scalar in the, in this example, in a matrix example, it's a vector.

00:08:28.800 | I haven't tried it, but I suspect kind of as you get into more and more dimensions, it could be, you know, higher dimensional array as well.

00:08:40.800 | Oh, that's that's that's actually, you know, I think what we should do maybe today is learn about operators, because that is not a function.

00:08:51.800 | It's a operator. And this is a PL terminology for what in some languages we would call higher order functions, their functions that take a function and return a function.

00:09:04.800 | So in Python, we don't give them a special name, you can have functions that take a function and return a function, although there is special syntax for doing it more conveniently and that syntax is called decorator syntax, which is where you can pop an at sign.

00:09:20.800 | Before the function before a decorator name and put that at the top of a function and that will cause a function to get passed in. In fact, should we look at that first maybe that might be interesting, and then talk about operators.

00:09:38.800 | Yeah, okay, let's try it because I know Zach Biola did a thing about decorators the other day, which I think people found interesting.

00:09:48.800 | So

00:09:58.800 | I guess I should do a get poll first.

00:10:05.800 | And while we're there, I guess we should also just talk about other things that have been happening on the forum.

00:10:16.800 | Actually, it's getting quite active, which is amazing.

00:10:22.800 | So, one particularly interesting thing, which was also Isaac is a way to create flashcards more conveniently.

00:10:35.800 | So if you look for Anki deck generator, you will find that.

00:10:42.800 | By the way, these, these ugly spaces are caused by this.

00:10:53.800 | You're not sharing your screen. I'm not sharing my screen of course I'm not sharing my screen I never remember to share my screen.

00:11:00.800 | There we go. Okay, let's try that again.

00:11:03.800 | All right, so this is the Anki deck generator thread from Isaac.

00:11:11.800 | And so I think now there's what three different decks you can import Isaac.

00:11:17.800 | Yes.

00:11:19.800 | So here's some of those examples.

00:11:23.800 | There will be more, I'm going to be after after each session, because I don't want to have to, you know, recreate the decks after each session I'm going to create a new, a new deck for that day.

00:11:39.800 | So, import that, because I'm sorry if you already know this, but if you import a deck with a name of a deck you already have.

00:11:50.800 | And then any

00:11:55.800 | cards you already have will not be duplicated.

00:12:00.800 | You can just keep exporting your updated deck and people can keep importing that, and it won't. And as long as you make sure you don't include scheduling information in the exported deck.

00:12:12.800 | Then it'll work fine.

00:12:15.800 | Oh, even better.

00:12:18.800 | I think it would be useful for, I didn't know that but do you think it would be useful for people who maybe aren't in the group and are either trying to catch up or going to watch this in the future to have kind of wanted each day.

00:12:29.800 | Not really because there'll be an order right.

00:12:34.800 | Oh, that's true. At, at most I'd say like add a tag to each card to say what session it's from.

00:12:42.800 | But otherwise people will just like, they'll be like, Oh, I haven't got to this bit yet. So, so stop. Yeah, so like, yeah, if you just go file export in Enki, and then you choose deck package, and you choose the package, remove scheduling information.

00:13:00.800 | And then you say export.

00:13:02.800 | This is what I do for the kids.

00:13:05.800 | And so that will export the deck, and then when they import it, it'll say these ones already existed and these ones are new and it'll bring in the new ones.

00:13:15.800 | Awesome.

00:13:19.800 | So I was just saying earlier that Mark, that's, that's such a cognitive dissonance for me to see Jeremy use a mark.

00:13:28.800 | Yeah, I, I've been using Windows largely since 1995.

00:13:37.800 | And I before that I used quite a mixture I use Windows 3.0 3.1 3.11 for work groups but I also used Mac from like 1990 or so.

00:13:55.800 | But I gotta say Windows 11. It's really getting a lot worse. It's the first time like I've really felt like it's getting worse so for the first time in my life I'm actually thinking of switching to Mac, and I actually ordered a MacBook Air M2 yesterday.

00:14:14.800 | Oh, yeah.

00:14:17.800 | So yeah, I did. I mean I do like being able to write on the screen of the surface pros and stuff.

00:14:24.800 | But what I do instead now is I've got a Wacom tablet, which is Bluetooth, and it's actually in some ways like it's not strictly worse than writing on a screen because I can just, you know, write with it in front of me, or else with a screen based approach I kind of have to move the whole thing.

00:14:46.800 | Yes. And how are you finding the UI of a market is it easy to get used to it.

00:14:54.800 | It's all right I don't, I don't love it.

00:14:58.800 | Like, it's definitely much worse than Windows used to be, but Windows is getting, you know, apparently everybody in the UX team in Windows or uses Macs so they seem to be trying to make it more like a Mac, but of course they don't.

00:15:15.800 | They don't really get it right so that's the worst of both worlds.

00:15:20.800 | So I would say yeah I kind of like it better than Windows 11. I've realized what I really like actually is Ubuntu's default desktop environment, because that's modeled much more closely on on Windows, you know, like,

00:15:41.800 | yeah Windows keyboard shortcuts just so ubiquitous, for example, which I really like, and, and they are on Ubuntu as well which I think users know nowadays.

00:15:55.800 | I kind of remember I just use whatever's there.

00:15:59.800 | There are things like for example to move a window in Mac to the other screen.

00:16:05.800 | You know, you have to either download an extension to use a keyboard for it.

00:16:11.800 | Or you know like it's all very much assumes mouse stuff, and specifically a lot of things really assume trackpads, you know, a lot of instructions you get will say what to do on a trackpad.

00:16:22.800 | That's okay. You know like I can like I can hit control left and right to jump to my different virtual desktops even if I can't move things to different virtual desktops to do it like if I want to move this to different virtual desktop I can press

00:16:34.800 | control up to get oh it's on the wrong screen.

00:16:38.800 | Honestly works I can press control up to bring up mission control and then I can like drag it to the other desktop, which is a bit clunky but yeah, sorry.

00:16:48.800 | How do you figure out how the UI works like when I, that's a great question I

00:16:56.800 | it's not easy right like I. So, part of me kind of thinking maybe I'm giving up on Windows was thinking okay well maybe I should act like this literally last weekend as if that were a cash probably learn to use the Ubuntu desktop environment, rather than just doing everything

00:17:10.800 | in the terminal.

00:17:12.800 | So partly it's knowing like what kind of features ought to exist and what they generally get called. So, you know, things like what Mac calls mission control is generally called virtual desktops, you know, so learning so learning how to use virtual

00:17:28.800 | desktops properly I just googled for Ubuntu virtual desktop, and then start, you know, keyboard shortcuts.

00:17:37.800 | And, I mean, they've got some pretty decent documentation, I found, so it tells you here.

00:17:49.800 | So, it means the windows key. So yeah windows page up and windows page down, which is slightly like not particularly well chosen because a lot of keyboards don't have a page up and page down keyboard on on laptops but generally on laptops if you press

00:18:04.800 | function up and down you get the same things this is basically windows function up and down.

00:18:10.800 | And then, you know, there's move away to a different workspace it's you know quite good and so if you just add shift to it, it moves the window.

00:18:19.800 | So yeah, it's kind of like knowing what the things are called and that they exist, then you can search for them.

00:18:25.800 | I haven't found a lot of good like here's how to productively work in a graphical windowing environment into 22 kind of material, that's, it probably must exist somewhere but I haven't found it.

00:18:43.800 | That's what defeated me when I got a marker myself I just couldn't figure out how to use it in a way that felt comfortable to me and I couldn't find the information in them.

00:18:54.800 | Yeah, easy to follow away. And I guess one observation that I have here is when you talk to somebody who's just starting to learn to program or starting to do machine learning they always ask that question, which is the one programming language I should learn.

00:19:12.800 | And, you know, which is the one library that I should learn and people spend two weeks, trying to answer this question.

00:19:22.800 | You know, in reality it is just start with something, anything I did something that is geared towards people starting out but you know if you do hang around that field, you will learn a lot of libraries, that's just the nature of it you might have one that you prefer.

00:19:39.800 | But, you know, you will try various things out and maybe it's the same thing with operating systems, you know, for part of my life I have been thinking, I have to, you know, just focus on this single operating system and making my home, and that's first

00:19:53.800 | into then it was Windows through no small influence of Jeremy, but maybe it is sort of, yeah, you know, if you are a computer user, and you will become a computer user for next X amount of years, maybe it makes sense to invest some time actually to,

00:20:11.800 | you know, try different things and it's also that if you probably attempt to use a mark, you might become a better Windows user or you want to desktop user, because of, yeah, yeah, absolutely.

00:20:29.800 | Yeah, I mean it's like using multiple programming languages I think it's a good idea.

00:20:35.800 | And just trying to be good at each of them. Yeah, I think that's fair.

00:20:40.800 | I see people putting stuff in the chat. All right. Well, I see even Marty I know you guys can both talk so you can actually say yes.

00:20:48.800 | Marty asked what Mac I got. I just got a well I've the one I'm on right now is a Mac mini and one.

00:20:55.800 | I just bought it.

00:20:57.800 | A couple of months ago for testing fast AI releases, honestly.

00:21:02.800 | But then once I plugged it in I figured I should try using it.

00:21:07.800 | And the laptop I got was just the MacBook Air into live 12 gig drive.

00:21:18.800 | Yes, right.

00:21:21.800 | I was just changing topics I put a little link to a handy vocabulary dialogue things that I've been using over the weekend that I found helpful as well.

00:21:33.800 | I will share this one.

00:21:47.800 | Yeah, so it kind of gives you your magnetic versus diabetic names, those links the document patients.

00:21:56.800 | It tells you kind of what they are too. So, I found it helpful.

00:22:03.800 | Great.

00:22:06.800 | Sweet.

00:22:08.800 | Some of these I don't recognize.

00:22:12.800 | And that's where the double alphas and double omegas. Yeah, so we'll talk about this in a moment.

00:22:19.800 | And then also share this one.

00:22:23.800 | Oh, so I also use Windows for a long time and I don't know I use Mac.

00:22:29.800 | I don't use most of the like gestures and the way that they expect you to do things I think I just keep like one window with lots of one virtual desktop with lots of windows.

00:22:42.800 | And then alter between them and use rectangle to like send windows to like the top left or the left or whatever. It's quite cool.

00:22:52.800 | Thanks.

00:22:56.800 | One which I didn't immediately find, but once I found it, it helped was

00:23:06.800 | Command tab or Windows tabs which is between applications, but to switch between windows within applications, it's command back to which is

00:23:18.800 | not something that necessarily find my mistake.

00:23:21.800 | That's quite helpful.

00:23:26.800 | I'm using a Microsoft keyboard so I still have a Windows key. I'll try to remember to call it command.

00:23:35.800 | Yeah, I always try and have my screens maximize so I can switch between windows and terrible at sticking to that rule. Yeah.

00:23:43.800 | Oh, and then other news.

00:23:47.800 | Oh, yeah, so there's some ugly blank lines we have in our

00:23:57.800 | page caused by

00:24:06.800 | this, which is actually added by the APL dialogue kernel, so I contacted them and told them that they shouldn't be adding

00:24:18.800 | forced styling into elements because they're impossible for us to make it look good. So, then I made it worse by accidentally putting line breaks in the middle of them too.

00:24:29.800 | Yeah, that's why they look even bigger. Yeah.

00:24:32.800 | So, yeah, they're going to try and fix it if they can't then we could use MP process credit processor to delete this but he said he's going to check it out next week. So, that was nice of him.

00:24:45.800 | And then the other weird thing that happened was our actual study group got on the front page of Hacker News. So that was weird.

00:24:56.800 | A lot of APL stuff seems to get on the front page of Hacker News for reasons I don't understand.

00:25:02.800 | So, is anybody here actually from the Hacker News thread.

00:25:12.800 | Not mainly people I recognize I think so.

00:25:16.800 | Yeah, a lot of fascinating stuff also often gets on Hacker News, which is good I guess.

00:25:23.800 | That's fine.

00:25:25.800 | It's terrible comments, but

00:25:29.800 | nice of people to know what we're doing.

00:25:35.800 | I thought this was an interesting question of like how the hell do you actually do multi line editing. So, as I kindly link to the section of the book, which is

00:25:45.800 | in the GUI, there's basically you can use this closed parenthesis ED to open up the editor.

00:25:55.800 | And then something I haven't tried apparently is you can use square bracket D input which I guess lets you put a multi line.

00:26:04.800 | Yeah, it does.

00:26:07.800 | I think so, if I, yeah, I don't know if that's all only thing what it does I haven't really looked at the documentation but yeah you can copy that directly in your dialogue raffle.

00:26:18.800 | Because it's weird that they say we define it in the notebook. But anyway, yeah. Okay.

00:26:23.800 | Put that directly in the raffle and it'll, it'll work.

00:26:27.800 | And then the other thing I just wanted to kind of show off about is

00:26:31.800 | the

00:26:34.800 | we now have hyperlinks in

00:26:39.800 | our APL notebook,

00:26:42.800 | which goes to the documentation.

00:26:46.800 | And we actually get those for free.

00:26:50.800 | Thanks to Isaac

00:26:53.800 | for putting this over to be an nb process repo, which is what nb devs, going to be called very soon.

00:27:00.800 | And then

00:27:04.800 | now who did it somebody then

00:27:10.800 | added the nb dev index to it.

00:27:14.800 | Where would I find that

00:27:18.800 | search should be from the last lesson. Okay.

00:27:24.800 | That's weird I thought I

00:27:29.800 | categories.

00:27:33.800 | Search.

00:27:36.800 | Yes, last lesson, quite right throughout a thank you.

00:27:41.800 | So nb process slash nb dev has uses this thing called nb dev dash index, which basically creates

00:27:51.800 | pi pi

00:27:56.800 | modules, such as

00:28:01.800 | nb dev numpy, for example, which if you pip install this basically you automatically it will automatically hyperlink

00:28:08.800 | every numpy keyword to its documentation.

00:28:13.800 | And so

00:28:18.800 | Johan kindly added an APL

00:28:23.800 | index,

00:28:28.800 | which he manually

00:28:32.800 | created semi manually just using Python requests.

00:28:37.800 | So yeah, so now thanks to the magic of

00:28:42.800 | nicely decoupled APIs, it automatically

00:28:47.800 | just worked. So we now have hyperlinked study notes, which I think is very fun.

00:28:58.800 | All right. Let's talk about operators.

00:29:04.800 | Yes, we were first going to talk about

00:29:08.800 | decorators.

00:29:13.800 | So let's create a function

00:29:20.800 | at one.

00:29:34.800 | Okay, and so we're going to pass it off something to add one to.

00:29:38.800 | Okay, so there's a function.

00:29:41.800 | Now, let's create another function

00:29:46.800 | which we're called log_args.

00:29:51.800 | And this is a function which is going to take a function

00:29:55.800 | and it's going to return another function. And the function

00:30:00.800 | it returns will be one that calls this function

00:30:05.800 | but prints the arguments it gets first.

00:30:09.800 | So to start with, let's do something really ridiculously easy, which is just to return the function that's passed.

00:30:15.800 | So we could create something called g

00:30:20.800 | which equals log_args of at one.

00:30:25.800 | And so log_args is going to be passed this function and let's return it.

00:30:30.800 | And so g will be exactly the same thing as at one.

00:30:35.800 | Makes sense so far? Okay.

00:30:40.800 | So we could define a function inside a function.

00:30:57.800 | Which is going to take some arguments and some keyword arguments.

00:31:06.800 | And it will simply pass them along to f.

00:31:16.800 | So now, when I call log_args, it's going to create a new function called _inner

00:31:22.800 | or actually called like, well, yeah, we'll call it _inner.

00:31:27.800 | And that function will return f, so it depends what we pass to it.

00:31:33.800 | So this is going to do exactly the same thing as before.

00:31:37.800 | Okay. But then what we could do is say print

00:31:46.800 | received_args_args_args.

00:31:58.800 | So now we could do things other than just running the function.

00:32:04.800 | I didn't actually say g. That was my mistake. Okay.

00:32:08.800 | And so you can see now it's done the same thing that the function did before,

00:32:12.800 | but it's doing something else as well. Does that make sense so far?

00:32:16.800 | So this is a function which takes a function and returns a function.

00:32:22.800 | And so people often seem to call these higher order functions,

00:32:26.800 | which I find confusing or not particularly helpful because they're just functions.

00:32:30.800 | But there you go. So there's a special way of doing this.

00:32:39.800 | So if I wanted to create a logged version of add one, I would have to do two steps.

00:32:43.800 | I'd have to first create add one and then say, you know,

00:32:47.800 | create this new version of it and then call that new version.

00:32:51.800 | So Python has a special syntax for doing these two steps.

00:32:57.800 | And that special syntax is to put at.

00:33:10.800 | This is a special syntax. And so this syntax,

00:33:13.800 | when you put at and a function before the definition of a function,

00:33:17.800 | is going to take the function that's which is defined here.

00:33:21.800 | It's going to pass it to this function.

00:33:24.800 | And it's going to replace this with whatever gets returned from it.

00:33:30.800 | So this is exactly the same as what I did before.

00:33:32.800 | So if I now go add one.

00:33:36.800 | Makes sense. So I can now add logging to any function I want to.

00:33:41.800 | That is my world changing mult2 function.

00:33:54.800 | Does that make sense?

00:34:00.800 | This is slightly by the by, but something that you can do is you don't have to use a function.

00:34:10.800 | It just has to be something that's callable.

00:34:12.800 | Something that's callable is something that you can put around brackets after.

00:34:17.800 | So let's say we wanted to change this to.

00:34:27.800 | With that we wanted to be able to change this.

00:34:31.800 | This year to some other prefix.

00:34:33.800 | So you can do that with a function, but you can actually just do a class.

00:34:37.800 | So we could create replaces for the class.

00:34:41.800 | Right.

00:34:43.800 | And then we could.

00:34:49.800 | So not many people seem to know this is possible.

00:34:55.800 | I put something in the, this is somewhat related, but I put something in the chat that I learned something about strings that allow you to actually shorten the string a little bit.

00:35:07.800 | Same thing.

00:35:09.800 | I found about today.

00:35:13.800 | This was only added in some what recent Python, but I don't remember which version it was.

00:35:20.800 | Maybe you can try and figure it out.

00:35:22.800 | 3.7 okay well 3.7 is is 3.6 is AOL. So that means you can safely use this now.

00:35:30.800 | I had not been using it because I knew that until at least until recently.

00:35:36.800 | There were in supported versions that didn't support this, but yeah okay that's great that now exists.

00:35:42.800 | Let's use it here.

00:35:52.800 | And then there's a question from a live thing.

00:36:00.800 | We do this and then I'll take a look. Thanks for letting me know. Okay, so we should be able to do this.

00:36:06.800 | Oh, that's right. I was halfway through fixing this.

00:36:26.800 | Okay, let's try that.

00:36:28.800 | Okay, yep, so that works.

00:36:32.800 | That does a question from somebody.

00:36:36.800 | So I can read it out. What does the conventional starting function name underscore signify. Oh yeah.

00:36:43.800 | When we start a function name with an underscore.

00:36:48.800 | It's generally means it's private. It generally means it's not something we expect people to be calling from from outside of this place.

00:36:58.800 | In nb dev, things that start with underscore by default not exported so that I don't appear in done to all starting and ending with two underscores is a special Python thing which means it's what it's a magic method it's a magic that will automatically be called

00:37:19.800 | at some time by Python. So, for example, done to it is automatically called when it's created.

00:37:29.800 | So if I was to turn this into a class.

00:37:43.800 | Yeah, it's. So we're going to say let's let's say we're going to call it the prefix, which we could default to received like so.

00:37:55.800 | And so then we could go self prefix equals prefix. Okay, so this is now a class which

00:38:08.800 | prefix it is. Okay. And then when you call it. So if you so.

00:38:19.800 | To explain if you have this a very interesting magic method called done call.

00:38:37.800 | Which does this prefix it basically makes your class callable. So if I create an instance of it.

00:38:53.800 | Then I can call it.

00:39:07.800 | They say I'm treating it as a function.

00:39:10.800 | And so when he treated class as a function, this is what it calls.

00:39:17.800 | So then we could use this instead of passing X, we're going to pass some function and.

00:39:39.800 | Keep all that stuff. Oops.

00:39:47.800 | And so now rather than putting received here.

00:39:52.800 | We would put self prefix.

00:39:58.800 | Okay, so when you call this now.

00:40:02.800 | As a function, you're going to pass it a function and it's going to return a function.

00:40:12.800 | Which means we would now write.

00:40:17.800 | For example.

00:40:22.800 | Okay, for example, like this.

00:40:27.800 | Okay, so. This is calling the function or the callable logs, which is a function.

00:40:35.800 | Sorry, which this is a class. So this instantiates the class passing in this prefix.

00:40:42.800 | And so.

00:40:45.800 | So this gets replaced with an object of this class. So if we have a look at that one.

00:40:55.800 | This.

00:40:59.800 | This class then gets called.

00:41:03.800 | Being passed the function at one.

00:41:06.800 | And so it then gets replaced with this. And that's why we end up with a.

00:41:12.800 | Logags dot under call dot underscore inner.

00:41:19.800 | Anyhow, so.

00:41:25.800 | So yeah, so higher order functions are functions that take functions and return functions, and Python has some tricky funds.

00:41:36.800 | Hi Jeremy. Yes. Okay. So I just wanted to say something for for people who are seeing this for the very first time.

00:41:44.800 | So I have a tip. The tip is to just use them anywhere. So, when you start thinking about these inner things, when you're first introduced to it.

00:41:56.800 | You would be confused. But if you just use it, you can use it on. There are a lot of days in flask framework. So in the Python classroom with a lot of decorators being used.

00:42:07.800 | And, and, and you can also see this in like Python as LRU cache LRU cache for caching older variables.

00:42:18.800 | So if you just start using it then you will not be confused very much.

00:42:23.800 | So, always whenever you see a decorator, don't start thinking what is going on inside just use it.

00:42:29.800 | So,

00:42:32.800 | here's the, here's the long handed way of doing this.

00:42:36.800 | Well I guess the long handed way actually would be to define it would be to call this something else like underscore at one.

00:42:42.800 | And then this could be called add one.

00:42:46.800 | And so here's a version which doesn't use decorators.

00:42:56.800 | But it behaves the same way.

00:43:02.800 | Oops, this should be underscore at one.

00:43:06.800 | There we go.

00:43:08.800 | Okay, so in APL,

00:43:14.800 | functions that take a function and return a function aren't called functions. They're called operators.

00:43:22.800 | And you will find therefore if we look up the language elements.

00:43:31.800 | I should have looked at dialogue language elements.

00:43:43.800 | They're separated out okay there's a table of functions.

00:43:47.800 | And that's a table of operators.

00:43:51.800 | So as you can see dot, which Isaac told us about means in a product.

00:44:01.800 | It's an operator because that returns a function which does matrix model play.

00:44:10.800 | So I think we should start with some simpler.

00:44:17.800 | Simple versions. But first of all,

00:44:26.800 | let's play this up a little bit.

00:44:33.800 | Okay, so let me cut this version.

00:44:49.800 | Okay, and then we've got this version.

00:45:00.800 | And I guess we should have the manual version of this one as well,

00:45:06.800 | which would be to say define underscore add one x and x plus one.

00:45:22.800 | So define underscore mult2.

00:45:30.800 | Turn X times two and then we'd say mult2 equals

00:45:37.800 | logags of underscore mult2. So here's the version without a decorator.

00:45:45.800 | And it does the same thing.

00:45:48.800 | So, well, let's look at APL and show you everybody's favorite operator,

00:45:58.800 | which is slash.

00:46:03.800 | And this one here appears to be in the wrong spot.

00:46:09.800 | So you had to hear.

00:46:13.800 | Guess we didn't know where to put it maybe.

00:46:29.800 | No, it's here.

00:46:32.800 | Let's move it there.

00:46:43.800 | Oh, I see why it's because we had strings.

00:46:48.800 | So maybe we should move strings before all that.

00:47:09.800 | Okay.

00:47:13.800 | Oh, no, that's got row it out. Man, this stuff's so complicated, isn't it?

00:47:23.800 | Fine.

00:47:32.800 | I'll leave it here now.

00:47:41.800 | Okay.

00:47:47.800 | So operators.

00:47:50.800 | We do have a few more Boolean things to go that we can always come back to them later.

00:48:03.800 | So operators.

00:48:22.800 | All right, so let's do slash.

00:48:27.800 | It's called, this is an operator.

00:48:34.800 | reduce called slash.

00:48:40.800 | We've got to have

00:48:46.800 | monadic slash called reduce.

00:48:53.800 | All right, so here is.

00:49:01.800 | Oh, and we need to turn on our APL thing.

00:49:08.800 | Okay.

00:49:10.800 | Okay, so let's say A is

00:49:15.800 | a five.

00:49:22.800 | I wonder if I can do this.

00:49:26.800 | Quad is

00:49:36.800 | an L. Oh, it's an L. Thanks.

00:49:42.800 | Okay, plus slash.

00:49:46.800 | All right, so

00:49:52.800 | here's A.

00:49:55.800 | And here's plus slash A. Now notice here that slash is not a function.

00:50:03.800 | It's an operator and therefore it has different rules.

00:50:07.800 | The first rule to know about operators is that they bind more tightly than functions.

00:50:15.800 | So normally if slash was behaving as a function here, then this would happen first.

00:50:23.800 | But because slash can behave as an operator and there's a function on its left, this happens first.

00:50:30.800 | So operators bind more tightly than functions. So so this is

00:50:36.800 | passed as plus slash first.

00:50:40.800 | Okay, so then what does plus slash do? Well, slash is an operator.

00:50:45.800 | And so therefore it returns plus slash returns a function.

00:50:50.800 | And what does that function do?

00:50:58.800 | It takes a function, so it's going to, here it is here, f slash, so it takes a function.

00:51:06.800 | The function has to be dyadic.

00:51:10.800 | And what it does is it applies the function between items

00:51:15.800 | of the vectors that it is passed. And so literally what this does is it inserts the symbol plus

00:51:23.800 | slash between 1 and 2 and between 2 and 3 and between 3 and 4 and between 4 and 5.

00:51:30.800 | So you get 1 plus 2 plus 3 plus 4 plus 5.

00:51:35.800 | So therefore plus slash means sum. And here is the sum of the numbers from 1 to 5.

00:51:41.800 | So this is the same as capital sigma in math.

00:51:49.800 | Now when you say it doesn't 1 plus 2 plus 3 plus 4 plus 5, but is in the order from right to left?

00:51:56.800 | It is. Or how does that work? It is. Yeah, so it it so it inserts it between its inserts in between these pairs.

00:52:03.800 | So it becomes 1 plus 2 plus 3 plus 4 plus 5. And then to evaluate it, it would do this first 4 plus 5 and then this 3 plus 9.

00:52:11.800 | This is 2 plus 12. So we could do the same thing with times.

00:52:21.800 | And this is now the same as capital pi in math. It's the product.

00:52:30.800 | So yeah, this is called a reduction in computer science. Reduction is something that basically reduces the number of dimensions.

00:52:42.800 | Not necessarily down to a scalar, but if you start with a vector or reduce it down to a scalar, as is happening here.

00:52:52.800 | And there's a very similar one which does exactly the same thing, but it shows you everything that it does along the way from left to right.

00:53:18.800 | So so now that I think about this backslash version, I wonder if maybe it doesn't work the way I just said it to Tanishk.

00:53:25.800 | Maybe it always maybe slashes doing it left to right as well.

00:53:29.800 | The left to right thing is about how it passes a statement like this or an expression.

00:53:38.800 | But I guess evaluating a reduction, maybe it is actually evaluating left to right.

00:53:43.800 | I mean, neither of these cases doesn't matter. Hey, let's test. Let's do divide.

00:53:48.800 | A bit of a weird one, but why not? Let's just make this 3.

00:54:05.800 | Oh, that's strange. Thank you.

00:54:12.800 | Let's see. One divided by two is a half and a half divided by three.

00:54:19.800 | Okay, no, it is doing it right to left. So two divided by three is two thirds and then the reciprocal of that is one.

00:54:25.800 | Okay, so it's definitely going right to left. So backslash must just reverse the order as well.

00:54:33.800 | Oh, wait, what was meant three divided by two and then divided by one to get one point five.

00:54:41.800 | It's it's doing two divided by three to get two over three and then one divided by that to get three over two.

00:54:50.800 | Because this is the same as one divided by two divided by three.

00:54:58.800 | Which is the same as one divided by two divided by three.

00:55:04.800 | Which is the same as one divided by two divided by three.

00:55:10.800 | Okay, so I guess backslash is not quite working in either of the ways I thought I think what it's doing is it's printing out.

00:55:25.800 | I need all these spaces.

00:55:28.800 | I just I think it's just printing out each of these.

00:55:35.800 | It's doing that.

00:55:40.800 | It's kind of like a cumulative function. It's kind of cumulative, but it's not right, because this last one actually is doing it right to left.

00:55:50.800 | It's not really cumulative at all. So I think it is doing every line. So, yes, every two numbers, and one more printed.

00:56:01.800 | Yeah, it's doing it's doing every line independently.

00:56:04.800 | So it's not necessarily going to be as efficient as I thought. So we could see that.

00:56:12.800 | Hopefully in the docs.

00:56:16.800 | Okay.

00:56:18.800 | This is called scan, which is a common computer science term for this idea.

00:56:30.800 | It's formed by success excessive reductions.

00:56:35.800 | It doesn't really explain exactly how it's done it better we can see from the divide example how it must be doing it.

00:56:46.800 | So, there's this reduce thing happening.

00:56:50.800 | But I also recall from functional programming, reduce with memo, is there such a thing.

00:56:59.800 | Is it reduce. So, no, like, when you start with an initial value.

00:57:06.800 | Oh, an initial value. So, there's not really any need for that here because you can always just insert the initial value.

00:57:14.800 | This is the first element. Right. Yeah.

00:57:17.800 | I think we haven't covered this yet but I think we might be able to use like comma.

00:57:28.800 | Yeah, so comma concatenates.

00:57:31.800 | So you could always just put like comma with your first value here.

00:57:37.800 | Thank you.

00:57:47.800 | All right.

00:58:01.800 | One way I've seen these, this reduce work as well as if you combine it with equals or eventually by the name or Max or whatever you can kind of look for either minimum or the maximum or or see if the entire race.

00:58:19.800 | Yes, absolutely. Not a unique value.

00:58:23.800 | So we actually haven't done it and Max yet but I think that's a great idea to put them before this.

00:58:31.800 | So, in and Max.

00:58:39.800 | And they're going to have monadic versions. They do actually I'm pretty sure.

00:58:52.800 | Okay, so,

00:58:56.800 | and everybody remember which way around they are but there's s and there's d. Okay, so s is this one.

00:59:03.800 | And it's called something style up style.

00:59:14.800 | Yep, up style.

00:59:16.800 | Okay, so up style.

00:59:22.800 | Okay, so the monadic version is slightly less relevant it's called ceiling.

00:59:36.800 | I'm assuming

00:59:50.800 | such as they're around that thing.

00:59:53.800 | Yep, just to round up thing. So it's normally called ceiling and floor in both programming languages, it's some finding the nearest integer that's greater than or equal to the number

01:00:10.800 | and d, I assume is called down style

01:00:29.800 | and therefore this one going to be called floor, I assume.

01:00:43.800 | And so there's a good you know kind of mnemonic for these right this is you can see this is going up to the ceiling this is going down to the floor.

01:00:51.800 | So you can see this one's being rounded down.

01:00:54.800 | This is the nearest integer. Okay, so apropos of

01:01:01.800 | Isaac's point the dyadic ones are interesting when we reduce them, because dyadic

01:01:09.800 | ones be called min and max, yep, maximum

01:01:24.800 | returns the greater than of two things.

01:01:29.800 | Now notice here that even though I don't have a space.

01:01:33.800 | This is the list of the array 32 maximum, the array 23. So it'll be element wise always element wise that's the maximum of three and two will be the first element and the maximum of two and three will be the second element.

01:01:49.800 | And then yeah where it gets pretty interesting is what if we create a

01:02:10.800 | random numbers yet something like that.

01:02:21.800 | Actually, let's just do a few of them or six to four, six, two, so that's going to pass it first of all is maximum of six and two, which is six, and then maximum of six and four.

01:02:39.800 | And maximum of six and four, which is six.

01:02:42.800 | And so where that gets interesting is we can then reduce slash.

01:02:50.800 | Like so reduce.

01:02:53.800 | slash I get the same thing. Does that make sense because that's literally going to insert the operator between each pair of elements.

01:03:12.800 | So that is the same as that.

01:03:17.800 | Does that make sense.

01:03:21.800 | So I think that's really quite great.

01:03:26.800 | The thing that I've done helpful with the upstill downstyle is you can do like four and then one of the styles and then an array, and it will kind of set your own custom minimal or maximum value, kind of a cut off on a right.

01:03:44.800 | Right, right, right. So we could do a style 345.

01:03:48.800 | So we could do operator right so we could do.

01:03:56.800 | Yeah, let's do that. Okay, so let's do negative four.

01:04:04.800 | Negative four, six, two. And so then we could do

01:04:10.800 | that.

01:04:12.800 | And that's going to do a value operator, because it's truncating negative zero is that what you meant Isaac.

01:04:17.800 | That's exactly what I meant. Excellent.

01:04:21.800 | One glyph activation.

01:04:23.800 | Yes, exactly. Just go really should be expressed there in your unit. There's actually a paper I found somewhere which was apl.

01:04:40.800 | I think they actually have a GitHub repository with their code as well.

01:04:46.800 | That would make all that sense to me, but hopefully one day I will. Yeah, yeah.

01:04:50.800 | This is all of the from scratch implementations of layers of a conf net, at least the types that were common in 2019.

01:05:02.800 | Just need backwards.

01:05:06.800 | I think I think is here.

01:05:12.800 | And so for convolutions, they actually had this version here although when I read a bit further.

01:05:20.800 | They said that there are.

01:05:23.800 | There's an easier way to do convolutions.

01:05:26.800 | CNN building blocks convolution.

01:05:32.800 | This is they said you can do this.

01:05:35.800 | Anyway, I guess we'll get there.

01:05:38.800 | I think it's in the chat if anybody wants to add it to our session notes, chat chat chat where's my chat button.

01:05:49.800 | Oh, I've got a pop down.

01:05:55.800 | Oh, and I could put the link to GitHub repository if you want to look at. Great. More of their code to be great.

01:06:03.800 | And so curian mentioned that the, that the little that that f string trick with the equals in it is actually only from Python 3.8.

01:06:18.800 | So, probably best not to use it just yet.

01:06:29.800 | It was.

01:06:48.800 | I see.

01:06:50.800 | CNN.

01:06:59.800 | There are networks.

01:07:02.800 | And this is the Rodrigo who I was chatting to about dialogue kernel.

01:07:09.800 | Very cool.

01:07:11.800 | Actually, I'm looking at that.

01:07:14.800 | And we realized that we covered the lamp.

01:07:17.800 | Yeah, I don't think so.

01:07:21.800 | Wouldn't be a bad thing to include quite early on because otherwise it's just going to be confusing isn't it.

01:07:30.800 | Actually, first of all, let me fix, just get this sorted out.

01:07:37.800 | So, in terms of getting the D and the S that the minimum the maximum the right way around,

01:08:06.800 | the fact that this is D for down style is actually quite helpful. That's a character that one.

01:08:19.800 | Wait, what happened to the

01:08:26.800 | value thing. Okay, we've got it in the wrong spot because this is not a slash. So I should put this up here.

01:08:35.800 | Okay, so what character what what keyboard is the comment lamp thing.

01:08:44.800 | Back to Connor.

01:09:05.800 | This is a comment.

01:09:20.800 | Comment.

01:09:27.800 | Great.

01:09:29.800 | All right.

01:09:31.800 | Seems like a good place to stop.

01:09:34.800 | So, we now know about operators and I guess maybe next time we can try making our own operators.

01:09:42.800 | We'll try doing a derivative operator,

01:09:46.800 | which means we're going to have to do alpha alpha and Omega Omega.

01:09:52.800 | Cool.

01:09:53.800 | Cool.

01:09:54.800 | All right. Thanks.

01:09:56.800 | See you next time.

fast.ai APL study session 5

Chapters