Okay, is that better? Can you see and hear me now? Great. Sorry about the delay. Okay. I just had to restart Chrome for some reason. All right. So there's a few possible directions we can go, but maybe we could just dive into transforms. Not because particularly it's code that many people need to understand, but I think it's kind of interesting code.

It's interesting Python code. So it's fun to see how the basics, the lowest level stuff is put together. And then we can kind of build back from there, look at data source and look at data blocks. Maybe first of all, though, we'll have a quick look at data blocks.

Because you can kind of see where we're heading with this, which we can simplify. I guess that makes sense. Okay. So quickly, where we're getting to is here are some examples. So here's an example of MNIST using the new data blocks API. And as you can see, it's not a fluent API anymore.

Just a moment. Sorry about that. Okay. So it's not a fluent API anymore. But now it's more of a, I don't know how you describe it. It's kind of similar to these other callbacks/subclass APIs like the data loader one. So you can now subclass from a data block and define the normal things that we're used to from version one, or you can construct it and pass in exactly the same things as functions.

So the data box API ends up looking kind of similar. But in some ways it's kind of easier to use because you can see easily what things you can pass to it. But then it ends up looking pretty similar. So there's MNIST, here's pets, as you can see, looks pretty familiar.

Here's planet, same kind of output. And so planet, there's a few ways to do it. One is, again, bypassing things in, actually, here's a super, super simple way to do it. Very nice and easy. Here's segmentation, sorry, here's Canva. Here is the buoy points one. And finally, here is Cocoa, bounding boxes.

So you can see we're going to end up with a nice simple data block API. But here's the coolest part. Here is the entire definition of the data block class. As you can see, it fits on less than one screen of code. And the reason for that is all that intermediate API stuff we built makes creating stuff like this super, super simple.

For questions about any of the data sets I just showed, have a look at the most recent courses, because they're all described there. And yes, David, this new API does present accidental wrong ordering, because there's no order to these things, they're just either subclasses, they're either subclasses or parameters.

I think I'm getting a cold, excuse me. So yeah, that's one of the nice things here. So that's the kind of payoff of all this. It's just so easy to create these kinds of things. So yeah, let's start pretty close to the bottom of the layers now. And let's look at transforms.

So before we do, I will answer this question from Gokor, which is my thought process when designing an API. I don't design really anything much at all. I just throw code out there, and I then refactor it again and again and again. So I try to make sure I've got pretty good tests.

And I keep refactoring it until it gets to the point where I feel like it's so easy that even I will understand it in six months' time. With the previous versions of fast.ai, I haven't had the time to fully do that. But with this version, I do. So as you can see, every function is super short and easy, which is really nice.

So yeah, lots and lots of refactoring. So transforms, the basic transform class is here, again, less than a line of code-- sorry, a screen of code. But the work's really happening in the meta class, because what happens is when we call done to call or decode, it's going to call end codes and decodes by this little intermediate thing, underscore call, which we haven't really looked at filters much yet.

But once we get to data source, you'll see this is how we do things on just the training set or just the validation set or both. That's what filters are. Basically, we check whether we're doing as item as true or false. If we're doing as item, then we just call the function, decodes or end codes.

If it's tuple behavior rather than item behavior, then we call it for each element of the list and turn it into a tuple. So that's all pretty simple, but the trick is what are end codes and decodes, because we know these are things which have a few cool things we can do.

We can-- let's see what we can do. We can pass methods in or we can subclass. Most interestingly, we can have a type annotation and we can have multiple end codes/decodes with each with different type annotation. So there's some other things we'll look at. So let's look at how we-- OK, so the first two are pretty straightforward.

Now in it, you can pass in an encoder and/or a decoder. And if you pass either of those in, then it will create the encoder and decoder-- encodes and decodes based on what you pass in. But as you can see, they are not just methods. There's something called type dispatch.

So that's what we're going to have to look at. If you don't pass in an enc or dec, which will be-- that means this will be false, then somehow encodes and decodes have already been created for us. So we have to figure out how that happens. OK, so let's take a look.

So answering questions, mixing, tabular and text, for example, yes, we certainly want to do that, but we haven't started on it. Not sure what you're asking exactly, AJ, about post-student projects at USF, but maybe ask on the forums. Do I have to subclass something? Simply means to type class something and then put something in brackets.

So for example, there is a subclass of transform. In other words, I am subclassing transform. OK, so if we look underneath here, we can see some of the kind of behaviors being used. So here's another interesting behavior, which is that we can use a class of transform as a decorator to add a encodes or decodes to an existing class.

So let's add that, decorator behavior. OK, yeah, Max, we're going to be talking about filters later when we look at data source, but basically just it's something that we're going to have zero generally as a training set, filter zero as a training set, filter one is generally the validation set, you can have more than just those two.

Transforms know basically which data set they're being applied to so they can have different behavior for training versus validation optionally, but we'll look at that later. OK, so here's an example of a class transform a, which is getting an encodes, which is just x plus one. So we can check that that works, we should be able to subclass that transform, so that checks that that works, and here's an empty transform, so it should do nothing at all.

So that checks that that works, but then later on, we'll get to the tests where, for example, we are making sure things only work on a particular subclass, in this case, tensor image. So here's the test, we're going to create one of these a objects, and we will call it, we should end up with a negative number, assuming that this was a something of type and tensor image, or else if we pass something which is not a type tensor image, in this case an int, nothing should happen at all.

Also, when we do this, we should not be changing the type, so we check that the type is still a tensor image. OK, so let's see how some of that works. So the trick is that all the stuff is happening in the meta class. What's a meta class? Redex given some good examples and code walkthroughs of some meta class stuff in the forums, so we check that out, but basically, when you go class something like that, that's creating a new class.

By the way, if you don't know, dot dot dot is basically the same as typing pass in Python. So that's created a new class, as you can see. And how does it do that? Well class foo, writing this, is basically syntax sugar for calling type and passing in these three things, a name, so foo, let's call this one foo2, some bases, so the base class is always implicitly, unless you write otherwise, object, so it's object.

And a dictionary of stuff to put in the class, which in this case is empty, and so let's call that foo2, you can see they're very similar looking things. So in other words, this is just syntax sugar for calling this thing called type. What is type? You can find out the type of something by using the single argument version of type, and the type of type is type, in other words, type is a class, and so this is a constructor, and it constructs something, so if I go type foo2, it constructs something of type type, and so there are all kinds of attributes that a type has.

So in particular, one of the important ones is I can put here a colon one. The most important is dundadict, and dundadict is basically a dictionary, it's a slightly special kind of dictionary, but it's a dictionary which contains a mapping from names to values, and in particular anything that I passed in here ends up in that dictionary.

There's another way to put things in the dictionary, which is to use this special syntax sugar that we get in Python, which is like this. And so now, if I say foo.dict, you can see it's got the same thing, a equals one. So when you type that into a class, it's actually just a shortcut for creating an A attribute in the dundadict attribute of a class.

Okay, a bunch of questions, what's for recommended systems, no plans yet. Why are we using meta class? Basically among the reason I started using meta classes in version 2 is because I wanted to change the things about how Python worked that I didn't really have the time to change version 1.

So things like this, all the stuff we're going to look at doing with transforms are impossible to do without meta classes. So yes, max type creates class objects and then class objects create instances, exactly right. So, it also means in Python, it's worth knowing how these syntax sugar things work.

When you go like this, if you go like that, for example, then that's the same as saying f equals and then passing in some function. And so if you look at the dict, you'll see you end up with f and then some function. So really, there's not that much - there's a small, concise, elegant set of foundations in Python and the kind of stuff that we type day to day is a bunch of little bits of syntax sugar for those foundations.

So if I go foo.a, that is also syntax sugar. And specifically, it's syntax sugar for dunder dict a. So this is all important to understand when we look at meta classes. And the reason why is because a meta class is something where we're going to replace type. We're going to say I want to create a class that does not use a type constructor but uses some other constructor.

And the way you do that in Python is you type meta class equals and then you type the name of the class you want to construct this class. You can create a class from scratch, but normally you wouldn't. It's easiest to subclass type. So here I'm going to inherit from type.

And if you remember, type takes three things, object or name, basis, dict. So dunder new always requires class first and then here it is, name, basis, dict. So if I wanted to create a super simple meta class, then I could just... So here's something which just returns super. So it's not going to change anything at all, but it will work, right?

Meta class equals m. There we go, I have a class. It has a dunder dict. So you can now start inserting things in here. And see how this printed as soon as I typed class t pass, right? It didn't print after I created an object of type t, it appeared as soon as I created the class.

So Python is going to call this code any time I try to actually create a class, not when I try to instantiate it. Yep, res is result. So in this case, we are placing three things, new, call and prepare. So there is a really cool piece of documentation called the Python data model.

And the Python data model describes how all this works. And not just all this, but everything, it describes how Python is, how everything happens. So there's a section called customizing class creation. Where you can see all of the stuff that happens, including three, three, three, one meta classes. By default, they're constructed using type.

So I could click on type. And we can see in three arguments, it returns a new type object. And then we can find out about type objects. And so forth. So meta classes, as it says, you say meta class equals blah. And the first thing that happens is it has to prepare the class namespace.

And the class namespace is the dunderdict, is the dunderdict object. So if we were to keep creating our underscore m, we could just return an empty dictionary. And as you can see, it all works. And I guess we should be able to put something in it, even. OK, there it is, right?

So you can see dunderdict is created by calling your dunderprepare. And this is actually a way you can insert something into every single class. Which has a meta class. So to train the different arguments to the meta class constructor, you would want to read 3.3.3 of the data model reference in the Python docs.

OK. So in our case, we've replaced prepare. So it's not returning a dictionary, but instead it's returning some special kind of dictionary called a tuffundict. So a tuffundict is a dictionary that overrides dunder set item. Actually I'm planning to change this. So I had things set up so you could use either underscore or encodes.

I'm actually going to get rid of the thing that lets you use underscore. So ignore that. So basically, if you're calling something that isn't encodes or decodes, then it's just the normal dictionary. As you can see, this inherits from dict. But if it is encodes or decodes, then what I do is I check whether or not I already have different encodes or decodes in my class.

And if I don't, then I create one using dicts set item. And what I set it to is I set it to a type dispatch object. So in other words, this tuffundict is something which behaves exactly like a normal dictionary and so it's going to be inside my dunder dict for anything that uses this meta class and it's not going to work differently at all except for two special things called encodes and decodes.

And in those cases, it's going to use something called type dispatch. So let's try that. So if I go plus a meta class equals to for meta, well, let's and then let's say def encodes self, turn X, so a dot encodes, oh, that's not a normal function, what type is it?

Oh, it's a type dispatch. And that's because of this. So do I create a meta class instead of inheriting from the class? They do different things. By inheriting from a class, you can't change the behavior of type creation. So we're trying to create something where anything that is inheriting from transform gets a different class behavior.

And so it's impossible to do the things we're describing by inheriting. There's no way, for example, normally to be able to say, I want to have two different encodes for two different types, for instance. That would normally be impossible. But thanks to meta classes and to done to prepare, we know that each time it sees this, it's going to call our replacement dicts under set item with a key of encodes and the value of the function, and we can do whatever we like.

And so what we're going to do is we're going to create an encodes, decodes type dispatch object if we don't already have one, and then we're going to add this function to that type dispatch object. So let's try that. If I add this twice with two different things, you can see now float has one thing and int, it's a different thing.

So let's then have a look at type dispatch. So yeah, this is like a huge rabbit hole. It's basically how do you make Python do whatever you wanted to do. Python is a dynamic language, and so they created this amazing data model to allow us to customize anything we like in Python, but it takes some time to get used to.

So it taught me a lot of reading and studying and looking at lots of different places to get the hang of all this, and so I don't expect to understand it all the first time around. I'm not sure what built-in functionality you're referring to, Max. I don't think Python has anything which does what I just described, which is why I'm doing something I don't think it does before.

Okay, so here is the type dispatch class, and basically it's going to be something which we know it's going to have to work a lot like a dictionary because we're going to be adding things to it, and what's going to happen--actually, I haven't quite shown you all this yet--what's going to happen is we're going to--let's see, we're going to grab encodes, we're going to call--oh, yes, so we're going to call the function, and when we call the function--well, Max, we're not just checking types, we're dispatching on types.

There's a big difference between checking types and dispatching on types, so we're actually building something where we're able to call different code depending on types, so yeah. So there isn't a way to do that in Python, so as we've discussed in previous walkthroughs, when you look at, for example, data augmentation, we're going to have class rotate, which when we first define it might be empty, and then later on when you say, "Oh, I've got something for a tensor image," then we'll say @rotate, and it'll say def encodes, x colon tensor image, and then degrees, and then there'll be some functionality for that, and then in some other place there's going to be x colon bounding box, and it's going to be different also whether we're encoding or decoding.

So it's a--it's quite different functionality. So yeah, so type dispatch. Dispatch is referring to how do you--how does a programming language decide what piece of code to run when you call something? So for example, there's all kinds of different ways of doing dispatch in Python. The main one that is used for methods, for example, is something called MRO, which is a method resolution order.

Yeah, so it's like--it's basically all the rules in a language about how do you decide which piece of code to call when you call some function, and different languages do it all kinds of different ways, and yeah, check out some of the earlier walkthroughs if you want more information about why we're doing this dispatch.

Yes, thank you, I mentioned that should have been transformed if we want that to work. So, what we want is we want something which works--looks like a function, so that means it has to have it done to call, but when you call it with some argument, we're not just going to call a function, but we're going to look at the type of that argument and we're going to try to find the appropriate function or method to call based on the type of that argument, and based on which methods have been created so far.

And so, what's going to happen is that inside our type dispatch object, there will be a dictionary called "funx" and that's going to contain a dictionary where the key is the type. So, for example, in this case, we're going to have keys "tensor image" and "bbox" and the value is the actual function to call.

So that's the key thing, is the "funx". So then, there was an "add" method and that is what FIFM dict calls. It adds this function, and the "add" method is going to find out the type annotation for the first parameter, "p1-anno" is parameter number one annotation, so it will grab the type.

If there is none, then it's assuming that it's object because that's the highest level of the type hierarchy, and it's going to pop that into our functions dictionary. So then, later on, when you call "dumda call", it's going to look up the type of the parameter that you're calling this function on, and it's going to look it up in this object.

If it doesn't find it, then it does nothing at all, so that's kind of the rule, right? If you only have like a "rotate" defined for "tensor image" and "bbox" and then you call it with "int", then nothing happens because that function isn't defined. If we did find it, then we just call it with that argument and anything else that you passed along.

And you can actually tag things to say, "I want you to turn it into a method", which is something we might talk about later if people are interested, but basically it's just going to create a normal function unless you ask it to be a method. So the key thing, then, is how does this line of code work?

How does it look up the type? So as you can see, it's calling "dunda get item", and basically what we're going to do is we're going to keep a cache, which is a dictionary mapping from types to functions. And we need a special cache dictionary because the way type dispatch works is it doesn't just look up, say, "tensor image" or "bbox", but it also looks really subclasses of those things.

So, for example, we could have also, as we discussed in an earlier walkthrough, "tensor". And in this case, then it's going to, if you pass a tensor image, it'll grab the most specific version it can, which is a tensor image version. So I think I just answered your question, which is the opposite of that.

If you call it on something which is a subclass of "tensor image", it will be invoked. And the reason why is because of how we create this cache. And so if we don't find it in the cache, then we're going to add it to the cache. And what we do is we create a list of all of the types that are registered, for which there is the appropriate subclass relationship.

And we, how do we do this? And notice that the functions have been ordered by a comparator which checks for subclass relationships. And so we grab the first one. And so the first one is the most specific type of the ones that it's a subclass of. This takes a little bit of time.

We don't want to have to do this every time we call this function. So once we find the right one, we pop it in the cache. So next time around, we can just grab it. Okay. So that is type dispatch. And so the key way to understand it really is to look at the tests, right?

So you can see here we've got things at lots of different levels of hierarchy. There's a parameter that's a collection, some kind of integer, tensor, mask or image, or some kind of number, right? So these are all functions. So we can create a types dispatch object with all of those different functions in.

And so we should try, if we look up int for instance, then we should get back this one because none of them were defined with int specifically, but it's going to match number and integral. Integral is more specific type than number. So that's why it matches that. String doesn't match any of them.

So it's a none. So here's the same kind of thing, but this time after creating the object, we'll actually start calling some functions and make sure that they do the right thing. Okay. So that is type dispatch. So yeah, to get the Python data model in your head requires kind of putting all these things together.

But the nice thing is once it is all in your head, you can put it together the way we have here. So we were able to replace the normal dictionary by replacing prepare with to firm dict. The firm dict is something which instead of a normal dictionary, when you set the item as encodes or decodes, it actually creates a type dispatch object and adds to it.

And that means that now when we inherit from transform and we create an encodes or decodes, um, attribute, uh, it will actually add it to the encodes or decodes type dispatch. And so when we call encodes or decodes, we get this behavior, which is to get a negative version of this because it's a tensor image and a non-negative version of this because it's not a tensor image.

Um, so, um, this specifically is single dispatch, not multiple dispatch. So, um, it only looks up. Let's find it. Um, when you add it, it only adds the, so P one ano is a function, any tiny little function here, which just grabs the, um, added the first parameter annotation.

So this will only work. Uh, so this will only do type dispatch based on the first, um, non-self parameter. So for me, that's a very good question. Why not throw an error? Um, basically because of what I was describing before, if you are defining say, uh, rotate data augmentation for various types, um, then generally speaking, if you don't have it defined for your type, then probably what we want to do is nothing at all.

Um, so the basic idea is that these kind of transforms are things you can opt into. Um, if you do want to create a transform, um, um, if you do want to create a transform which, uh, throws an error, uh, if you, um, if you call it with something that doesn't exist, you certainly can.

Let's do it. So here's a transform. And so if we say def encodes, and we'll say here, raise not implemented, and then we'll go def encodes self comma x colon int return x plus one. So let's go a equals a, and then we can go a one that returns two a high, and that's going to return.

Oh, is that not the right one? There we go. Not implemented error. So you can certainly, um, um, add that. And so the reason this works is because to remember that if you don't have an annotation, it's the same as saying object. It's kind of the way Python normally does things.

That's the way we do things too. And since that's the highest thing up in the inheritance, in the inheritance hierarchy, that's the one that it would end up calling if you don't provide some other behavior. Uh, so thank you for that. Excellent question. Um, okay, so that is the way that we handle dispatch.

So the next thing is what about the way we can treat a transform sub class as a decorator? So if you remember a decorator, um, uh, when you see something like this, um, we'll actually call this as a callable and pass this function to it. So that is basically identical to saying something like def underscore encodes equals that.

And then encodes equals, um, a parenthesis. And actually what I should say is no, I just say, yeah, a underscore encodes. It's basically the same as doing that. Um, and we should find that the same test then passes. Um, but we have to say a equals that. Um, let's see.

Ah, yes, it's slightly different because it thinks of it as not as a method. Okay. So it's not exactly the same. It's nearly the same, well because of Python, because of stuff we do. So we'll come back to that later. Um, okay. So the reason we can use this, um, as a decorator is because that means that our class, um, is going to have to support, um, the callable basically.

Um, now classes are normally callables because you can, um, instantiate a class, obviously. So if you go class B like this, then you go B equals B, then you're calling B as a callable or B's type, B's metatype as a callable. Uh, so that means if we go to our meta class, we can redefine under call.

And that means we can redefine what happens when we instantiate this class. Um, so if we instantiate this class or if we, if we done the call this class, um, and we pass in some argument, um, and if that argument is callable, um, and it's not decodes or encodes, then we are going to, um, add that function, uh, to our type dispatch.

Um, and so that's exactly what happens when we say at a def encodes, blah, blah, blah. It's just going to add that function to this class's type dispatch. So you can see here it calls dot add, which is the dot add that we had here in type dispatch. So this is the thing that basically registers another type.

Um, okay. So, uh, that's the hat. Uh, there's one last piece, which is done to new. So this is the thing that first gets called when you're creating a new type. Um, and one of the things I discovered, which is super annoying, is that if I create a new type, like so any title type, even without being a meta type, and I define a done to new, like so, so here's some class and then I define some sub class and I say in it, like so.

And then so I want to instantiate that and then I shift tab. Um, oh, that was not what I expected to happen. Oh, sorry. And plus B inherits from a shift tab. You can see the signature is star, star, star quags, or else I would have expected the signature to be a, as it would be if I removed my base class.

Um, so I found that super annoying because like you want to customize new all the time. Not all the time, but very frequently. Um, and pretty much most of the time when you customize it, you're going to use star, star, star quags because you don't want to define what base classes can do.

Um, but as soon as you do that, you, you kill the signature, like so. So, um, I don't know why it works that way and maybe I'm missing something, but what I did here was I replaced the signature for the class with a signature for the dunder init so that we get the right nice signature.

So if we try that, here we have a look at transform. You can see we get the correct signature as we would want. Um, because otherwise, uh, it's not just under new, it would also be done to call from the meta class, uh, would also replace the signature otherwise.

Um, so that's why that's there. Um, so believe it or not, that's actually all the pieces. Um, and so if you want to study this, like, so the first thing to point out is none of this is at all necessary to understand fast AI version two. It's no more important than understanding the meta object data model is to, um, use Python day to day.

It's an advanced technique which you can learn about if you're interested. And if you're interested in learning more about how Python works behind the scenes, so you can try doing stuff like this yourself if you want to create, um, change how Python works and fully use its dynamic features.

So if you want to fully understand what transform does, just check out all the tests. Um, we've tried to make it so that each test does one thing, you know, one shows one clear type of behavior. We try to add comments explaining what each behavior is that it's showing.

Um, so yeah, hopefully that all is useful for those of you that are interested. So then you can see tuple transform and item transform just force an item to be true or false and lots more tests of that behavior, as you can see. Um, all right. Well, I think that's enough for today because that was super dense.

Um, and, uh, yeah, if you start looking through this code and want to learn more about it, feel free to ask any questions you like. All right. Thanks, everybody. Bye.