back to index

fastai v2 walk-thru #4


Whisper Transcript | Transcript Only Page

00:00:00.000 | Hi, everybody.
00:00:10.880 | Can you see and hear okay?
00:00:19.800 | Great.
00:00:27.600 | And that response came back pretty quickly.
00:00:29.640 | So hopefully the latency is a bit lower.
00:00:37.920 | So Sam asked before we start to give you a reminder that here's conference in San Francisco.
00:00:44.320 | Today is the last day to get early bed tickets.
00:00:51.600 | So remember if you're interested, the code is fast AI to get an extra 20% off.
00:01:04.000 | Okay.
00:01:05.000 | I was thinking I would do something crazy today, which is to do some live coding, which
00:01:13.680 | is going to be terrifying because I'm not sure I've really done that before.
00:01:17.720 | Oh, hey there, Max, welcome.
00:01:23.440 | So I was going to try and do some live coding to try to explain what transform does and
00:01:34.480 | try and justify like why it does it and why it exists to kind of try to, yeah, basically
00:01:44.320 | see why we ended up where we ended up.
00:01:46.480 | I think we did something like 26 rewrites from scratch of the data transform functionality
00:01:59.120 | over a many month period, because it was just super hard to get everything really easy and
00:02:07.160 | clear and maintainable and stuff like that.
00:02:11.480 | So I'm really happy with where we got to and that might be helpful for you to see why we
00:02:15.160 | got here.
00:02:18.400 | So maybe let's start by writing a function.
00:02:26.440 | So our function is going to be called neg and it's going to return negative, which you
00:02:38.600 | might recognize actually is the same as the operator.neg function, but we're writing it
00:02:44.360 | ourselves.
00:02:46.400 | So we can say neg three, for instance.
00:02:53.960 | And that's how Python will do that.
00:02:58.400 | What if we did neg of some tuple, let's create a tuple, one kernel three, we get neg of some
00:03:08.920 | tuple.
00:03:10.920 | Oh, it doesn't work.
00:03:17.000 | That's actually kind of annoying because, I mean, it's the first thing to recognize is
00:03:23.280 | like this is this is a behavior.
00:03:27.840 | It's a behavior that can be chosen from a number of possible behaviors and this particular
00:03:32.200 | behavior is how Python behaves and it's totally fine.
00:03:37.320 | And as you know, if we had an array, for instance, we would get different behavior.
00:03:52.360 | So one of the very, very, very nice things, in fact, to me, the best part of Python is
00:03:58.680 | that Python lets us change how it behaves according to what we want for our domain.
00:04:07.880 | In the domain of data processing, we're particularly interested in tuples or tuple-like things
00:04:17.880 | because generally speaking, we're going to have, you know, an x and a y.
00:04:23.400 | So for example, we would have an image and a label such as a pet.
00:04:37.080 | Other possibilities would be a source image and a mask for segmentation.
00:04:46.600 | Another possibility would be an image tuple and a bool, which would be for Siamese, segmentation,
00:04:59.960 | Siamese, categorization, or it could be a tuple of continuous columns and categorical
00:05:15.400 | columns and a target column such as for tabular.
00:05:27.920 | So we would quite like neg to behave for our tuple in a way that's different to Python
00:05:36.600 | and perhaps works a little bit more like the way that it works for array.
00:05:45.360 | So what we could do is we could repeat our function and to add new behavior to a function
00:06:00.000 | or a class in Python, you do it by adding a decorator.
00:06:07.400 | So we created a decorator called transform, which will cause our function to work the
00:06:18.320 | way we want.
00:06:20.680 | So now here we have it, minus one, minus three.
00:06:25.360 | So let's say we grab an image.
00:06:31.440 | And so we've got a file name, FM, which is an image, so let's grab the image, which will
00:06:37.440 | be log image FM, there's our image, and so let's create a tensor of the image as well,
00:06:59.280 | which we could do like this.
00:07:07.920 | OK, maybe just look at a few rows of it.
00:07:14.720 | OK, so we could let's look at some rows more around the middle.
00:07:26.760 | OK, so we could, of course, do negative of that, no problem.
00:07:40.640 | Or if we had two of those in a tuple, we could create a tuple with a couple of images.
00:07:49.480 | And actually, this is not doing the negative, we're not getting a negative here.
00:07:57.520 | OK, interesting question, tag the image, row, color, date.
00:08:13.840 | Hmm, interesting, what am I doing wrong?
00:08:28.040 | Oh, yes, thanks, sir, I mean, the reason my negative is wrong is because I have an unsigned
00:08:39.560 | int, so, yes, thank you, everybody.
00:08:44.680 | So let's make this a dot float, there we go, OK, and that will work on tuples as well.
00:08:57.920 | There we go, OK.
00:09:05.280 | So that's useful, and that would be particularly useful if I had an image as an input and an
00:09:11.640 | image as an output, such as a mask as an output or super resolution or something like that.
00:09:23.200 | So this is all looking pretty hopeful.
00:09:26.180 | So the next thing, let's try and make a more interesting function.
00:09:29.520 | Let's create a function called normalize, and that's going to take an image and a mean
00:09:37.560 | and a standard deviation, and we're going to return x minus m over s.
00:09:49.560 | So I could go normalize, t image, comma, let's say the mean is somewhere around 127, and
00:10:04.400 | standard deviation, let's say it's somewhere around 150, and here, let's-- and again, actually,
00:10:21.200 | let's just grab our subset so it's a little bit easier to see what's going on.
00:10:25.960 | So we'll go the subset of the image is a few of those rows, OK, there we go.
00:10:43.320 | So we could check now t image dot mean, the sub dot mean, output t image dot mean, I should
00:11:01.440 | say, let's say, normed image across that, dot com, t, so, normed image dot mean, OK,
00:11:31.320 | so, it's the right idea, OK, so here's a function that would be actually pretty useful to have
00:11:42.000 | in a transformation pipeline, but I will note that we would generally want to run this particular
00:11:56.360 | transformation on the GPU, because even though that doesn't look like a lot of work, it actually
00:12:04.480 | turns out that normalizing takes an astonishingly large amount of time as part of something
00:12:10.920 | like an ImageNet pipeline, so we would generally want to run it on the GPU.
00:12:16.560 | So by the time something has gotten to the GPU, it won't just be an image, it's the thing
00:12:22.400 | that our data set returns will be an image and a label, so let's create a image tuple,
00:12:34.160 | will be image, comma, let's say it's a one, so this is the kind of thing that we're actually
00:12:42.120 | going to be working with, oh, we should use a test, we're actually going to be working
00:12:48.680 | with on the GPU, so if we want to normalize that, and again, let's grab a mean and a standard
00:13:08.880 | one, and again, we have a problem, so let's make it a transform, so that's something that
00:13:22.520 | will be applied over tuples, so you'll see that none of the stuff we already have is going
00:13:26.720 | to change at all, because, oh, I made a mistake, I did it wrong, oh, took two additional arguments,
00:13:40.440 | that four were given, ah, yes, that's true, so we do have to use n equals s equals, okay,
00:13:49.560 | so one changes that we have to use, um, named arguments, so now we can go m equals s equals,
00:14:01.960 | so there we go, so now it's working again, um, but, oh, that's no good, it's also transformed
00:14:10.920 | our label, which is definitely not what we want, so, um, we need some way to have it
00:14:21.200 | only apply to those things that we would want to normalize, um, we can't, so, as we saw,
00:14:31.840 | you can add a type notation to say what to apply it to, um, the problem is that generally
00:14:43.560 | by the time you are getting through the data loader, everything's a tensor, so how can
00:14:50.880 | we decide which bits of our tuple to apply this to, um, so the answer is that we need
00:14:58.120 | to, um, create things that inherit from a tensor, um, and so we have something called
00:15:06.860 | tensor base, um, which is just a base class for things that inherit the tensor, so we
00:15:12.080 | could go plus, um, my tensor image, which will inherit from tensor base, um, oops, and
00:15:25.920 | I don't have to add any functionality to it, um, what I could just do is now say p image
00:15:34.120 | equals my tensor image, p image, actually let's call this, uh, my tensor image, uh,
00:15:45.560 | and so this, because this inherits from tensor, it looks exactly like a tensor, as all the
00:15:53.240 | normal tensory things, um, but if you check its type, it's one of these, so the nice thing
00:16:03.360 | is that we can now start constraining our functions to only work on things we want,
00:16:11.280 | so I want this to only work on that type, um, and so this is the second thing that a
00:16:18.600 | transform does, is that it's going to make sure that a new e gets applied, um, to these
00:16:24.000 | parts of a tuple, if you are applying it to a tuple. So again, it's not going to change
00:16:28.000 | this behaviour, because it's not a tuple, um, but this one, we are applying it to a
00:16:34.880 | tuple, ah, perfect, okay, so now that's not being changed. So it's important to realise
00:16:41.280 | that this is like, um, it's, it's no, uh, more or less weird or more or less magical or anything
00:16:50.840 | than Python's normal function dispatch, different languages and different libraries have different
00:16:58.080 | function dispatch methods, um, Julia tends to do, uh, function dispatch nearly entirely
00:17:06.360 | using these kinds of, um, this kind of system that I'm describing here, um, um, Python by
00:17:16.320 | default has a different way of doing things, um, so we're just picking, we're just picking
00:17:22.560 | another way, uh, equally valid, uh, no more strange or no less strange. Um, so Max is
00:17:31.600 | asking does transform only apply to tensor? No, not at all. It's just a, it's just a,
00:17:36.920 | so let's go back and have a look. Remember that, um, uh, in our image tuple, the second
00:17:43.360 | part of the tuple is just a one. So when I remove that, you'll see it's being applied
00:17:50.000 | to that int as well. This is just a, this is, it's nothing PyTorch specific or anything
00:17:55.360 | about this. It's just another, um, way of doing, uh, function dispatch in Python. And
00:18:04.400 | so in this case, uh, we're constraining it to this particular type, but yeah, I mean,
00:18:09.160 | I could have instead said, um, class my int. And then let's create a, my int equals my
00:18:22.240 | int one. And then let's say this only applies to a my int. Um, and so now let's not use
00:18:34.600 | a one, but let's use my int. So you can see now the, uh, tensor, the image tensor hasn't
00:18:44.320 | changed, but the int has. So it's just a, yeah, it's just a different way of doing, um, function
00:18:54.480 | dispatch. Um, but it's a way that makes a lot more sense for data processing, um, than
00:19:04.040 | the default way that Python has. And so the nice thing is that like Python is specifically
00:19:09.320 | designed to be able to provide this kind of functionality. So every, all everything we've
00:19:15.560 | written as you'll see, uh, we've written that all, um, uh, okay. Got a few questions here.
00:19:27.680 | Uh, so let's answer these first. Um, you defined MTI, but when did that get passed to norm?
00:19:37.280 | Uh, it got passed to norm. Oh, it got passed to norm. Yeah. That I think. Yes. Okay. So
00:19:49.720 | let's go back and check this. So we're going to do my tensor image. So there it is with
00:19:59.040 | a by tensor image. If we made the tuple non my tensor image, just a regular tensor. Oh,
00:20:08.240 | it's still running. Why is that? Um, right. Oh, I actually changed the type of T image
00:20:22.320 | at some point. I didn't mean to do that. Let's run it again. Okay. Uh, oh, uh, because I'm
00:20:51.080 | actually changing the original way. Uh, that looks like we found a bug. Okay. Let's make
00:20:59.440 | that bug. Okay. So let's do it separately. Yes, I do need a job. Oh, I've got fun. Um,
00:21:22.600 | I shouldn't, but I have a bug. Okay. Let's do it again. There we go. Okay. So T image
00:21:32.440 | is now a tensor. And so, okay. So now when I run this on the tuple, which has a normal
00:21:40.920 | tensor, it doesn't run. Okay. And so then if I change it to the subclass, then it does
00:21:52.600 | run. Um, and oh, and we wanted to make this a bug. Uh, and that should be a tensor array.
00:22:14.400 | Okay. There we go. Um, so that one is just being applied to the image and not to the label.
00:22:29.060 | And so, uh, to answer Max's question, we absolutely could just write in here and put that back
00:22:39.360 | to T here. And now it is being applied. Uh, the reason I was showing you sub classing
00:22:51.400 | is because this is how we add a kind of semantic markers to existing types, um, so that we
00:22:59.000 | can get all the functionality of the existing types, but also say that they represent different
00:23:04.360 | kinds of information. Um, um, so for example, in fast AI vision two, there is a capital
00:23:13.360 | I int, um, which is basically an int that has a, uh, show method. Um, but it knows how
00:23:22.320 | to show itself. Uh, what kind of work do you put in after the pipe? Not sure what you mean,
00:23:30.360 | Caleb. We don't have a pipe. Uh, so feel free to re ask that question if I'm not understanding.
00:23:38.360 | Um, okay. So this is kind of on the right track. But one problem here, though, is that
00:23:47.040 | we don't want to have to pass the main and standard deviation in every time. So it would
00:23:52.160 | be better if we could, uh, grab the main and standard deviation and we could say mean on
00:24:01.440 | our standard deviation equals T image dot mean comma T image dot standard deviation.
00:24:10.400 | Oops. He calls. I'm not sure what I just did. Okay. So now we've got a name and standard
00:24:31.320 | deviation. So what we could do is we could create a function, which is a partial application
00:24:37.440 | of norm over that main and that standard deviation. Uh, so now we could go norm image equals f
00:24:49.920 | of that instance. Um, and then of course, we'll need to put this back to let's just make this
00:25:02.400 | tensor for now, say, there we go. Um, or we could do it over our tuple. Um, we don't need
00:25:17.440 | an F, but more and more. So, and this is kind of more how we would want something like normalization
00:25:23.560 | to work, right? Because, um, we generally want to normalize a bunch of things with the
00:25:30.840 | same main and standard deviation. Um, so this is more kind of the behavior we want, except
00:25:37.800 | there's a few other things we'd want in, um, normalization. Um, one is we generally want
00:25:44.880 | to be able to serialize it. We'd want to be able to save the transform that we're using
00:25:50.360 | to disk, including the main and standard deviation. Um, uh, the second thing we'd want to be able
00:25:57.080 | to do is we want to be able to undo the normalization so that when you get back, uh, an image or
00:26:03.500 | whatever from some model, we'd want to be able to display it, which means we'd have to
00:26:07.560 | be able to denormalize it. So that means that we're going to need proper state. So the easiest
00:26:14.080 | way to get proper state is to create a class. Um, Oh, before I do that, I'll just put something
00:26:22.080 | out. Um, for those of you that haven't spent much time with decorators, this thing where
00:26:26.680 | I went at transform and then I said def norm, um, that's exactly the same as doing this.
00:26:31.800 | I could have just said def say underscore norm equals this and then I could have said
00:26:37.560 | norm equals transform. Um, and that's identical as you can see, because exactly the same answers
00:26:49.120 | that's literally the definition of a decorator in Python is it literally takes this function
00:26:58.440 | and passes it to this function. That's what at does. So anytime you see at something,
00:27:06.960 | you know that, oh, thank you transform underscore norm. Um, you know that that's what it's doing.
00:27:16.640 | Thanks very much for picking up my silly mistakes. Um, okay. So yeah, so it's important to remember
00:27:25.800 | when you see a decorator, it's something you can always run that manually on functions
00:27:32.000 | and um, yeah, okay. Uh, so what I was saying is in order to have state in this, we should
00:27:40.520 | turn it into a class, let's call it capital N norm. And so the way transform works is
00:27:48.480 | you can also inherit from it. And so now if you inherit from it, you have to use special
00:27:56.680 | names and encodes is the name that transforms expect. Uh, and so normally you wouldn't then
00:28:05.520 | pass in those things, but instead good pass it here. And remember, uh, so we could just
00:28:19.600 | go self dot m comma self dot s equals m comma s, or we could use our little convenient thing
00:28:27.920 | we mentioned last time or doesn't really add as much, um, so small number, but just to
00:28:37.640 | show you the different approaches. So that one. Um, so now we need self dot m self dot
00:28:49.720 | s. Um, I still have my, uh, annotation here. So now we will go f equals capital N norm
00:29:07.160 | equals M s equals s. Um, Oh, and of course these states are still, and there we go. So
00:29:24.640 | Oh, and why isn't that working? So it worked on our, this one. That's not on our tuple.
00:29:44.560 | Um, so David asks, why would we inherit versus use decorator? Yeah. So basically, um, yeah,
00:30:02.160 | if you want some state, it's easier, easier to use a class. If you don't, it's probably
00:30:06.160 | easy to use a decorator. Um, that sounds about right to me. Um, okay. So let's try and figure
00:30:14.000 | out why this isn't working. Um, Oh, I think it's because we forgot to call the super pass.
00:30:26.360 | Perhaps. Yes, that's probably why. Thank you. Oh yeah. But we both put it at the same time.
00:30:37.560 | There we go. Okay. Um, so that's an interesting point. Um, it's, it's quite annoying to have
00:30:50.320 | to call supers in it. Um, we actually have something to make that easier, which is borrowed
00:30:58.760 | from the idea that, uh, Python standard libraries, data classes use, which is to have a special
00:31:06.200 | thing called post in it that runs after you're in it. Um, uh, okay, Juvie and I will come
00:31:16.360 | back to that question. It's a very good one. Um, um, so we actually have, yes, if we inherit
00:31:33.560 | from base object, we're going to get a meta class, which we'll learn more about later called
00:31:37.920 | pre post in it. Meth meta, which allows us to have a pre in it and a post in it. And
00:31:44.700 | they may already be using that. Let's have a look. Um, transforms. Transform. This isn't
00:32:07.340 | a trick from meta. Yeah. Okay. So we'll have to come back to this. This isn't a great opportunity
00:32:23.840 | to use this one. So must be impaired. Um, let's answer some questions first. Uh, so
00:32:34.080 | can you specify multiple types? Um, so as max said, it would be nice if we could use
00:32:42.920 | union. I can't remember if I actually currently have that working. The problem is that Python's
00:32:49.680 | typing system, um, isn't at all well designed for using at runtime. Um, and last time I
00:32:59.900 | checked, I don't think I had this working. Let's see. No, there's this really annoying
00:33:09.720 | check that they do. Um, so basically if you want this, um, then you have to go like this.
00:33:24.760 | Um, probably actually probably the easiest way would be to go underscore norm, get rid
00:33:41.780 | of that there. And then in codes here would be return self underscore norm X and we'll
00:33:52.220 | do the same here. I think that's what you have to do at the moment. So have it twice
00:33:57.380 | both calling the same thing with the two different types. Um, if anybody can figure out how to
00:34:03.260 | get union working, that would be great because this is slightly more clunky than it should
00:34:08.220 | be. Um, so if you need multiple different encodes for different types, you absolutely
00:34:25.420 | don't need to use, um, class. Um, and, um, so just to confirm, you know, again, you can
00:34:35.980 | do it with a class, uh, like so, um, so if that, uh, you can also do it like this. Um,
00:34:58.580 | well, actually, um, yeah, there's a few ways you can do it, but one way is you partially
00:35:10.380 | use it. You partially use a class, uh, you define it like this, um, but you just write
00:35:18.340 | pass here. And then what you do is you write in codes here and you actually use this as
00:35:27.620 | a decorator, um, like so, and then you can do this a bunch of times like so. Um, and
00:35:40.100 | this is another way to do it. So we now have multiple versions of encodes, um, uh, in separate
00:35:49.940 | places. So it's still defining a class once, but it's an empty class. Um, but then you
00:35:56.180 | can put the encodes in these different places. This is actually, we'll see more about using
00:35:59.700 | this later, but this is actually super important because, um, let's say the class was, well,
00:36:06.100 | actually norms, norms are great class. So let's say you've got a normalized class and there's
00:36:09.540 | somewhere where you, uh, defined it for your, you know, answer image us. Um, but then later
00:36:15.420 | on you create a new data type for audio, say, um, you might have an audio tensor and then
00:36:25.420 | that might have some different implementation where, I don't know, maybe you have to, uh,
00:36:32.940 | transpose something or you have to add a non somewhere or whatever. Um, so the nice thing
00:36:38.900 | is this separate implementation can mean a totally different file, totally different
00:36:43.220 | module, totally different project, and it will add the functionality to normalize audio
00:36:49.020 | tensors, um, to the library. Um, so this is one of the really, uh, helpful things about
00:36:56.580 | this, uh, about this style. And so you'll see, yeah, the data augmentation in particular,
00:37:01.980 | this is great. Anybody can add data augmentation of, um, their own types, uh, to the library
00:37:10.540 | with just one line of code. Well, two lines of code, one for the decorator. Um, oh, great
00:37:18.220 | question, Juvian. So yes, um, the, the way that, uh, encodes works or the way that in
00:37:25.140 | general, um, these, uh, this dispatch system works is, um, let's check it out. Um, let's
00:37:40.700 | put here, print a, print b, and this is a tensor and this is a tensor image. Um, so if
00:37:58.740 | we call it with MTI, it prints b. Why is it printing b? Because my tensor image is a more,
00:38:14.140 | um, is further down the inheritance hierarchy. So that's the more, uh, what you would call
00:38:20.740 | Juvian closest match, a more specific, uh, match. Um, so it uses the most, um, precise
00:38:27.900 | type that it can. Um, and this does also work for, um, uh, kind of generic types as well.
00:38:38.900 | So we could say type being dot, somewhere there's a class for this. Ah, what's it called?
00:38:52.580 | There you go. Just a little bit slowly. So Python standard library comes with various
00:39:01.500 | type hierarchy. So we could say numbers dot integral, for instance. Uh, and then we could
00:39:07.260 | say f3, as you can see, uh, where else if we had one that was on int. And again, it's
00:39:26.060 | going to call the most specialized version. Um, okay. So lots of good questions. All right.
00:39:46.060 | Let me check this out later. Thank you for the tip. Um, all right.
00:40:15.980 | Um, the next thing that we want to be able to do is we need to be able to go in the opposite
00:40:21.860 | direction. Um, so to go in the opposite direction, they create something called decodes. And so
00:40:36.220 | this would be X plus self dot S. Sorry, self times self dot S plus M. Um, and so let's
00:40:53.140 | check normalized image. And so to run decodes, you have to go F dot decode. And then we would
00:41:05.300 | say an image. And there it is. Um, and this works for tuples and everything exactly the
00:41:14.980 | same way. So if we go, uh, F of our tuple, and then we could say F dot decode F of our
00:41:27.500 | tuple. Yep. We've gone back to where we were for the whole couple, just to confirm that
00:41:34.420 | this did impact. Oh, that's not working. Oh, because this should be an MTI. This we're
00:41:41.060 | now saying this image type. There we go. And decode. There we go. Okay. So, um, so that
00:41:55.020 | is, um, some of the key pieces around how this dispatch works and why it works this
00:42:07.320 | way. Um, but of course, it's not enough to have just one transform. Um, in practice,
00:42:17.380 | we need to, um, not only normalize, but we need to be able to do other things as well.
00:42:24.900 | So for example, maybe we'll have something, uh, that flips. Um, and so generally things
00:42:33.620 | that flip, we don't need to unflip because data augmentation, we want to see it even,
00:42:38.100 | um, later on, we don't generally want to decode data augmentation. So we could probably just
00:42:43.620 | use a function for that. Um, so it's going to take one of our image tenses and it's gonna
00:42:56.980 | return porch.flip, uh, our X and we'll do it on the, uh, zero, the zeroth axis, uh, no,
00:43:13.720 | the fifth axis. Uh, if random random is greater than 0.5, otherwise we'll leave it alone. So
00:43:24.660 | that would be an example of some data augmentation we could do. Uh, so how would we combine those
00:43:30.400 | two things together? Um, so one simple approach would be, we could say, um, uh, trans, uh,
00:43:42.220 | transformed image, uh, equals, uh, in fact, let's do it for the whole couple. So transform
00:43:48.020 | tuple, all right, tuple is image tuple. So transform tuple equals, um, let's say first
00:43:57.300 | we normalize, which was f of our image and then we flip it. Um, actually that won't quite
00:44:13.900 | work because, um, um, because we can't apply it to couples, so we could tell it that this
00:44:24.940 | is a transform. And so now that works, which is good. Um, it's kind of annoying having
00:44:32.140 | to say everything's a transform and it's also kind of annoying because this kind of, um,
00:44:37.180 | uh, normalize and then flip and then do something else and then do something else. These kinds
00:44:42.740 | of pipelines are things we often want to define once and use multiple times. Um, one way could
00:44:48.360 | do that is we could say, uh, pipeline, uh, equals compose. Uh, so compose is just a standard,
00:44:57.500 | um, functional and mathematical thing that, um, runs a bunch of functions in order. So
00:45:03.940 | we could first of all run our normalization function and then our flip image function.
00:45:10.100 | So now we could just go pipe. Uh, so that, that works. Um, probably is that composed
00:45:17.940 | doesn't know anything about, um, transforms. So if we now wanted to go, um, to decode it,
00:45:33.180 | we can't. So we would have to go, um, you know, the image dot decode, uh, which actually isn't
00:45:41.260 | going to do anything because we don't have one, but we don't necessarily know which ones
00:45:44.620 | have one, which don't, you'd have to do something like this. Um, so it would be great if we could
00:45:50.680 | compose things in a way that know how to compose, um, decoding as well as encoding. Um, so to
00:45:57.660 | do that, you just change from compose to pipeline and it's going to do exactly the same thing.
00:46:05.660 | But now we can say pipe.decode and it's done. Um, so as you can see, this is kind of idea
00:46:17.440 | of like trying to get the pieces all to work together in this data transformation domain
00:46:26.180 | that we're in specifically kind of couple transformation domain that we're in. Um, because pipeline
00:46:32.480 | knows that we want to be dealing with transforms. Um, it knows that if it gets a normal image
00:46:36.940 | and not a transform, it should turn it into a transform. So let's remove the at transform
00:46:40.860 | decorator from here and run this again. And I thought that was going to work. Um, oh, so
00:47:01.480 | I thought I had made that the default. Sorry, I think I forgot to export something.
00:47:17.660 | Yes, that is correct. So let's do that. So let's do that. So let's do that. So let's do,
00:47:43.000 | okay. Let's get the right order. So pipeline. Okay. So let's do that. So let's do that. So,
00:48:12.080 | let's add a new class. Let's say class. Okay. So let's do that. So let's do that. So let's,
00:48:41.160 | let's do that. Um, okay. So we put an image tuple pipeline, which is going to go. So if
00:49:08.080 | I add a transform back on here, and I've managed to break everything. Let's go do that. That's
00:49:26.720 | a problem with live coding, I guess. Oh, and now it's working again. What did I do wrong?
00:49:42.000 | Maybe I just want to restart properly. I guess I did. Okay. All right. Sorry about that delay.
00:49:52.080 | So what I was saying is that a pipeline knows that the things that's being passed need to
00:50:04.360 | be transforms. If we pass it something that's not a transform, so I've made this not a transform
00:50:11.160 | now. Um, so the cool thing is that pipeline will turn it into a transform automatically.
00:50:23.080 | So yes, David from people in the San Francisco study group, they say one of the best things
00:50:29.640 | is finding out how many errors I make. Um, apparently that's extremely, um, encouraging
00:50:34.720 | that I make lots. Uh, and I want the default behavior of the transform to encode and decode.
00:50:41.120 | Uh, yeah. So the default behavior of encode and decode is to do nothing at all. Um, so
00:50:50.600 | the decoder want, the decoder doesn't know what the image prior to the flip is. The decoder
00:50:55.720 | is not storing any state about what was decoded. It's just a function. So the decoder simply
00:51:02.320 | just returns whatever it's passed. It doesn't do anything. Uh, we haven't got to data box
00:51:09.760 | yet. So we'll talk about data box when we get there. Um, okay. And Kevin, I find I have
00:51:22.280 | a lot of troubles with auto reload, so I don't tend to use it, but you can actually just
00:51:27.520 | tap zero twice on the keyboard and it'll restart the kernel, um, which I thought I did before,
00:51:33.720 | but I might've forgotten. Um, and the other thing I do, I find super handy is I go edit
00:51:38.200 | keyboard shortcuts and well, why don't I see any edit shortcuts? Edit keyboard shortcuts.
00:51:49.200 | Apparently you put it is currently broken. That's weird. Um, anyway, I add a keyboard
00:51:55.320 | shortcut for, um, um, running all the cells above the current cursor, which normally you
00:52:02.760 | can do with edit keyboard shortcuts. Um, so I can just go zero zero to restart and then
00:52:08.480 | I go a to run above and that gets me back to where I am. Um, okay. Um, so hopefully you
00:52:23.280 | can kind of see how, um, this approach allows us to, um, kind of use normal Python functions,
00:52:34.040 | but in a way that is kind of dispatched and composed in a way that makes more sense for
00:52:39.480 | data processing pipelines than the default approach that Python gives us. Um, but the
00:52:47.840 | nice thing is they're, um, they're not, they're not that different, you know? So like it's,
00:52:55.360 | it's, it's just kind of call, you know, the function just behaves like a normal function
00:52:59.440 | unless you add type annotations, uh, and so forth. Um, okay. So this is making good progress
00:53:10.320 | here. Um, so then, um, yeah, so we don't need to actually define my tensor image because,
00:53:26.200 | um, uh, we already have, um, a answer image class, which we can use. Um, but as you can
00:53:39.440 | see, it's actually defined exactly the same way. Um, so that's the one that we should
00:53:47.120 | be using, um, so that's, um, and I guess we might as well just say T image dot on and
00:54:06.200 | let's restart. And then we don't need my int because we've actually defined something called
00:54:12.880 | int as I mentioned in exactly the same way. Int again, just pass, but it's also got a
00:54:21.040 | show on it. Um, so that's now going to be a tensor image. There we go. Oh, um, okay.
00:54:50.840 | There we are. So then the last piece of this puzzle, um, for now is that we might want,
00:55:01.200 | let's make our load image thing into a transform as well. Um, so the fine, um, create image
00:55:11.720 | with some file name and we're going to, it's actually going to be like, uh, I guess we're
00:55:21.120 | actually going to create an image. And so we're going to return, um, answer array. Um,
00:55:32.000 | image, sorry, load image X, um, that we could do that. And so now we could use that here.
00:55:51.280 | And let's restart the image, the image done all above. Okay. So there you go. Now the point
00:56:12.320 | about this though, is that what we actually want risk to be is because you know, it's
00:56:17.840 | an image, not just any odd tensor. We want to avoid having kind of non semantically typed
00:56:23.080 | tensors wherever possible. Cause then we, or first AI doesn't know what to do with them,
00:56:28.600 | but this is currently just a tensor. So we can just say that should be a and so our image
00:56:42.560 | and it will cast it, uh, should cast it for us. Oh, and of course, unless it's in a pipeline,
00:56:50.120 | we need to tell it it's a transform for any magic to happen. Okay. Um, in practice, I basically
00:57:02.000 | pretty much never need to run transforms outside of a pipeline, um, except for testing. So,
00:57:09.200 | um, you'll see a lot of things that don't have decorators, um, because we basically always
00:57:14.080 | use them in the pipeline, which you can see. Um, for example, we just went pipeline and
00:57:23.680 | pass that in. That would work as well. Okay. Cause there's a pipeline with just one, uh,
00:57:30.880 | something being composed, um, or of course, alternatively, it could do this and that would
00:57:44.720 | work too. Um, but heck out of having return types is kind of nicer because it's just going
00:57:55.120 | to ensure that it's, it's cast properly for you and it won't bother casting it if it's
00:58:00.240 | already the right type. Um, just kind of nice. Um, okay. Random orientations we will get to
00:58:09.520 | later. Um, but if you're interested in looking ahead, you'll find that there is a, um, thing
00:58:18.160 | called efficient augment where you can find that stuff developed. Um, okay. So, um, you
00:58:34.960 | know, you don't need to learn anything about or understand anything about pipe dispatch
00:58:41.200 | or meta classes or any of that stuff to use this any more than to use, um, Python's default,
00:58:48.080 | uh, function dispatch. You don't have to learn anything about, you know, um, the, the workings
00:58:55.520 | of Python's type dispatch system. Um, you just have to know what it does. Um, but we're going
00:59:01.600 | to be spending time looking to see how it works because that's what code walkthrough
00:59:05.680 | is all about, but you certainly don't have to understand it to use any of this. Um, so
00:59:14.480 | the next thing to be aware of is that both pipelines and tenses, um, when you create
00:59:21.040 | them, you can pass an optional additional argument as item, uh, which is true or false.
00:59:30.720 | Um, as item equals false gives you the behavior that we've seen, um, which is that it, um,
00:59:38.640 | uh, well, if given a tuple or some kind of listy type of thing, it will operate on each
00:59:44.480 | element of the collection. Um, but if you say as item equals, uh, true, um, then it won't,
00:59:53.800 | it basically turns that behavior off. Um, so you can actually see that in the definition
00:59:59.600 | of transform, um, here it says if use as item, which returns soft as item, then just call
01:00:11.360 | the function. Uh, and then otherwise it will do the loop that in this case it's a couple
01:00:19.560 | comprehension. So, uh, you can easily turn off that behavior, um, by saying as item equals
01:00:30.360 | true either in the pipeline, uh, or in the transform. Um, the, um, there are predefined,
01:00:41.880 | uh, subclasses of transform as you can see called couple transform and item transform.
01:00:49.240 | And they actually set as item force equals false and true. And the force versions of
01:00:54.360 | these mean that even if later on you create a pipeline with as item equals something,
01:01:00.120 | um, it won't change those transforms. So if you create an couple transform or an item transform,
01:01:06.840 | then you can know that all the time, no matter what, it will have the behavior that you requested.
01:01:13.880 | So why does pipeline have this, uh, as item equals true false thing. So what that does
01:01:21.400 | is pipeline simply will go through. So you can see pipeline in the constructor called
01:01:28.120 | set as item that simply goes through each function in the pipeline and sets as item.
01:01:35.920 | Um, and the reason we do that is that, um, depending on kind of where you are in the
01:01:43.840 | data, the full data processing, processing pipeline, um, sometimes you want kind of tuple
01:01:49.600 | behavior and sometimes you want item behavior and specifically, um, up until you, um, uh,
01:02:01.160 | create your batch together, you kind of have, uh, two separate, uh, pipelines going on.
01:02:07.520 | So each, each pipeline is being separately applied to mind you from, uh, the pets tutorial.
01:02:18.600 | We have two totally separate pipelines. So since each of those is only being applied
01:02:25.760 | to, um, one item at a time, not to tuples, um, the, uh, uh, uh, the, the fast.ai pipeline
01:02:37.040 | will automatically set as item equals true for those because they don't have, uh, they
01:02:44.040 | haven't been combined into tuples yet. Um, and then later on in the process, it will
01:02:50.280 | set as item equals false, uh, basically after they've been pulled out of datasets. So by
01:02:56.120 | the time they get to the data loader, um, it'll use, uh, false. Um, and we'll see more
01:03:03.000 | of that later. Um, but that's a little, little thing to be aware of. So basically it just
01:03:10.040 | tries to do the right thing, um, for you. Um, but you know, you don't have to use any
01:03:17.080 | of that functionality, of course, if you don't want to. Um, okay. Okay. All right. Well,
01:03:39.840 | I think that's, um, enough for today. Give us a good introduction to pipelines and transforms.
01:03:48.100 | Um, and, uh, yeah, again, happy to take requests on the forum for what you'd like to look at
01:03:53.360 | next. Otherwise, um, yeah, maybe we'll start going deeper dive into how these things are
01:03:59.480 | actually put together. Um, we'll see where we go. Okay. Thanks everybody. See you tomorrow.
01:04:05.680 | [BLANK_AUDIO]