Live coding 9

00:00:00.000 | - Okay, we're recording.

00:00:01.280 | So I know there was some questions on the forum.

00:00:04.240 | Matt, I think it was you, right?

00:00:07.160 | A couple of questions.

00:00:08.360 | - Yep, that's correct.

00:00:12.160 | I can read them out for you if you like.

00:00:13.760 | - Oh yeah, or you can express them in your own terms,

00:00:17.240 | whatever, yeah, tell me.

00:00:19.280 | - So I guess there weren't so much,

00:00:23.400 | first of all, there weren't a couple of,

00:00:25.520 | there weren't questions.

00:00:26.400 | There were just sort of differences

00:00:28.840 | when working on paper space,

00:00:30.080 | you were working on your local GPU.

00:00:32.480 | And I found sim linking the Kaggle folder

00:00:38.680 | into storage with my API keys,

00:00:43.440 | something that I did to make it

00:00:45.360 | a little bit easier to restart.

00:00:47.680 | Just wanting to verify that

00:00:49.280 | that's sort of a good thing to do.

00:00:51.080 | And, but the one thing that I would be really interested

00:00:58.800 | and be really keen to know about

00:01:00.000 | is when I did the pip install of Tim,

00:01:04.720 | it was fine, it was installed,

00:01:09.280 | but I had to restart the kernel.

00:01:11.120 | And I'm wondering,

00:01:13.160 | that might be a bit of a pain going forward.

00:01:15.800 | I'd prefer to have it persistently there, ready to go.

00:01:20.800 | But it's not a Conda package.

00:01:23.440 | So I wasn't sure how to...

00:01:26.280 | - Yeah, so it's actually pip packages are the only ones

00:01:29.560 | we've actually got the persistence working for.

00:01:31.800 | So let's do that one first.

00:01:33.440 | So the key thing when you install Tim,

00:01:37.200 | let's see, do I already have it installed?

00:01:39.240 | Import Tim.

00:01:42.520 | Okay, great, I don't, so let's do it.

00:01:46.360 | So that the key thing to remember

00:01:51.840 | is when you install Tim is to do it with dash dash user.

00:01:55.920 | Now, in order to make that easier,

00:02:00.920 | I think what I would be inclined to do

00:02:03.320 | would be to edit our slash storage slash dot bash dot local

00:02:08.320 | and add to it alias.

00:02:13.920 | Let's do PI for pip install equals pip install.

00:02:25.240 | Let's do minus U for upgrade.

00:02:28.320 | That should work even if it's not installed already.

00:02:32.120 | Minus minus user.

00:02:33.680 | Okay, now, if I...

00:02:37.280 | So I could close and reopen my terminal

00:02:40.840 | or I could just type source and then the name of the script,

00:02:44.040 | which of course in this case is exclamation mark dollar.

00:02:46.840 | Oopsie dozy, exclamation mark VIM will rerun this.

00:02:51.240 | This whole thing needs to be in quotes.

00:02:55.120 | Because it's a single thing, it's my alias.

00:02:58.440 | Okay, I'll pair it twice.

00:03:01.280 | Okay, so now I can just type PI.

00:03:05.000 | And by the way, if you wanna know what something is,

00:03:11.440 | if you type which PI, it won't tell you anything useful

00:03:16.840 | because it's an alias, not a binary.

00:03:20.480 | But if you type type PI,

00:03:22.600 | it will tell you exactly what it is in this case.

00:03:24.720 | Oh, it's something that's an alias.

00:03:26.920 | So I can type PI, Tim.

00:03:29.560 | And the key thing about minus minus user

00:03:34.720 | is that's gonna put it in my dash dash local directory.

00:03:38.320 | Sorry, my dot local directory.

00:03:40.160 | So there it is, Tim.

00:03:46.520 | So then all you need to make sure is that

00:03:53.560 | your local directory is simlinked

00:03:57.560 | to dot slash storage config local.

00:04:02.120 | Now that's interesting.

00:04:03.360 | Our, this is here is telling us we've got a broken simlink.

00:04:07.760 | So that's what that means.

00:04:09.040 | Yeah, dot config is simlinked to slash storage.

00:04:13.120 | But there is no dot get config there.

00:04:22.160 | So I might've maybe forgot to move that or something.

00:04:27.160 | So that's okay.

00:04:32.800 | Next time we try to commit,

00:04:35.800 | it'll tell us and we'll know to fix that then.

00:04:38.440 | To create a file that's empty, you just use touch.

00:04:46.000 | So I'm just gonna go ahead and create an empty file.

00:04:49.040 | So at least it exists and then things

00:04:52.160 | won't get horribly confused.

00:04:53.880 | Did I not touch it correctly?

00:05:02.400 | Slash storage slash dot get config.

00:05:07.400 | Oh, there's a slash at the end.

00:05:11.320 | Oh, that's why.

00:05:15.040 | That's why it's confused.

00:05:16.280 | So that would be a directory,

00:05:18.760 | this is not a directory.

00:05:20.040 | So my guess is that there's a bug in our pre-run script

00:05:28.080 | for dot get config.

00:05:29.640 | Yes, I've got a slash at the end.

00:05:33.080 | So that's why that didn't work.

00:05:35.360 | So if I

00:05:48.000 | source that.

00:05:48.960 | Now it's happy.

00:05:54.920 | Great.

00:05:59.240 | So now, yeah, so now since it's been installed

00:06:04.120 | into something that's similar to back to slash storage,

00:06:06.040 | Tim will be available.

00:06:08.160 | And if I run ipython, we can confirm it did install.

00:06:12.320 | That should be all good.

00:06:13.160 | Does that answer that part of the question, Matt?

00:06:15.480 | - Yes, thank you.

00:06:16.640 | - That was.

00:06:17.800 | So then the second one, yeah, it was not a question,

00:06:20.280 | but a comment, which was about Kaggle.

00:06:23.200 | So yeah, when I get back to using Kaggle on this machine,

00:06:27.240 | we will do that for sure,

00:06:29.160 | which will probably be next time.

00:06:31.600 | And you also had a question about jumping around to,

00:06:36.600 | you know, the end of a string, for example,

00:06:44.440 | which, let's grab Bast.ai's repo, for example.

00:06:49.440 | Let's grab Bast.ai's repo, for example.

00:06:55.440 | Bast.ai's repo, for example.

00:06:57.840 | (upbeat music)

00:07:00.420 | (upbeat music)

00:07:03.000 | (upbeat music)

00:07:05.580 | (upbeat music)

00:07:08.160 | Oh, and you also had a question

00:07:34.500 | about loading and serving bottles, right?

00:08:03.660 | So, I mean, one thing obviously

00:08:05.260 | is it'd be nice to have a tags file.

00:08:07.160 | At some point, we could even talk about

00:08:10.900 | how to set up VIM to automatically create that

00:08:13.100 | for us from time to time.

00:08:15.740 | But let's have a look at, I don't know, layers, for example.

00:08:21.320 | So a few things to mention.

00:08:29.780 | The first is something which sounds very obscure,

00:08:33.300 | but actually isn't, is F in VIM.

00:08:38.300 | F in VIM is like slash.

00:08:41.300 | Now slash searches, so we've seen it before.

00:08:43.820 | Slash init, we'll search for the next thing called init.

00:08:47.540 | Okay.

00:08:48.820 | Oh, maybe something we haven't discussed

00:08:50.300 | is to go back to where we were,

00:08:52.340 | regardless of whether it was a tag or a search or anything,

00:08:55.860 | it's Control + O.

00:08:56.960 | And right next to Control + O is the letter I,

00:09:00.720 | which goes forward again.

00:09:03.640 | Okay, so Control + O and Control + I go kind of,

00:09:06.440 | it's like pressing the back button

00:09:07.520 | and the forward button on your browser.

00:09:09.880 | There's something a lot like slash,

00:09:11.740 | but it just finds a single letter, which is F.

00:09:14.200 | If I type F, it's gonna,

00:09:15.800 | and it's under the search on the current line,

00:09:17.300 | it will search on this line for the next thing I type.

00:09:19.720 | So if I type F double quote,

00:09:22.520 | actually maybe more interesting would be F full stop.

00:09:26.040 | So if I type F full stop,

00:09:27.320 | it's gonna jump to the full stop, F dot.

00:09:30.080 | So you see it jumps to the full stop, right?

00:09:32.520 | And so your question was,

00:09:33.560 | well, what about jumping to the end of a string?

00:09:35.660 | Now, in this case,

00:09:36.500 | the end of a string is the last character of the line.

00:09:39.900 | So there's a better answer,

00:09:41.020 | which is to start inserting at the end of the line,

00:09:43.560 | it's Shift + A.

00:09:44.520 | Just one moment.

00:09:47.440 | (mouse clicking)

00:09:50.200 | (mouse clicking)

00:10:19.000 | My daughter's got kicked off her Zoom call,

00:10:21.280 | always technical problems.

00:10:23.160 | Okay, so I can undo that.

00:10:26.020 | Control + O to go back to where I was.

00:10:28.920 | But yeah, so let's say there was some stuff at the end,

00:10:34.960 | hash some comment, right?

00:10:38.240 | And we wanted to, yeah,

00:10:40.400 | we wanted to go to the next double quote.

00:10:42.980 | I can just type F double quote, and it takes me there.

00:10:48.000 | And then Shift + F does the opposite, it searches backwards.

00:10:50.840 | And the reason it's interesting mainly

00:10:54.000 | is that that's a motion,

00:10:55.120 | and therefore I can combine things with it.

00:10:56.840 | So for example, if I wanted to delete everything up

00:11:00.400 | to the next quote, I can press D, F double quote, right?

00:11:04.720 | And then I could press slash double quote

00:11:06.880 | to search the next one and press dot,

00:11:09.240 | and it'll do the same thing again, right?

00:11:12.080 | Or maybe delete everything up to the next comment

00:11:15.800 | would be D, F hash.

00:11:18.200 | So yeah, those are a couple of useful things.

00:11:24.400 | Another really useful one is percent.

00:11:28.640 | Percent jumps between the start and the end

00:11:32.600 | of a set of paired parentheses or braces or brackets.

00:11:37.600 | So if I press percent here,

00:11:40.640 | it goes to the start of end of the next parentheses

00:11:43.760 | and then press it again,

00:11:44.680 | and you can see it jumps between the two, right?

00:11:47.520 | And so if I do it from here,

00:11:49.600 | you can see it jumps to the end of this one, right?

00:11:54.600 | If I do it at the very end, it'll jump to this one.

00:11:57.200 | So if I want to delete from here

00:12:00.560 | to the end of the parenthesized parenthetical expression,

00:12:05.280 | like let's say to delete this bit,

00:12:07.440 | I could press D, F, sorry, D, F percent.

00:12:14.640 | Sorry, not D, F percent, just D percent.

00:12:16.600 | There you go, D percent, you see?

00:12:19.440 | Although there's actually something even better for that,

00:12:23.440 | which is I, and I refers to an area,

00:12:28.440 | the whole area that is surrounded

00:12:34.920 | by some kind of parentheses.

00:12:36.640 | So even when I'm in the middle of these parentheses,

00:12:40.080 | the enclosing parentheses would go from here

00:12:44.400 | to here, and so I stands for inside.

00:12:46.920 | So if I want to delete everything inside those parentheses,

00:12:49.080 | I can type DI, open round, do that again,

00:12:53.920 | DI, open round parentheses, and it deletes the contents,

00:12:58.040 | which is really nice.

00:13:00.560 | So let's say I wanted to replace all my parameters

00:13:03.520 | with something else, like A comma B,

00:13:06.960 | then I would use C for change inside parentheses.

00:13:11.600 | So type my change, like A comma B, right?

00:13:15.040 | And then I can come down here and type dot,

00:13:18.240 | and it'll do the same thing.

00:13:21.000 | So yeah, it's like medial work.

00:13:25.640 | You can kind of really crush with these tricks.

00:13:29.520 | - Great, it's fantastic.

00:13:35.240 | - Yeah, it's cool.

00:13:36.080 | There's a lot of them,

00:13:36.920 | and you don't have to know them all, you know?

00:13:38.480 | It's like you can learn one thing each day or something,

00:13:41.040 | but yeah, I'm not using any plugins or anything, you know?

00:13:45.480 | Okay, so we're gonna save a model in a moment.

00:13:56.560 | Any other questions or comments

00:13:57.840 | before I go back to our notebook?

00:13:59.880 | - I want to make one comment about the Tim installation.

00:14:04.880 | I don't know if maybe you discussed this yesterday

00:14:07.000 | 'cause I came a little late,

00:14:08.280 | but with the Tim installation,

00:14:11.560 | sometimes it might be better to install from master

00:14:14.280 | because there are some changes that Ross has made

00:14:17.480 | that you might not receive from-

00:14:18.760 | - Yeah, we did mention that yesterday.

00:14:21.800 | Actually, I think the conclusion we came to

00:14:24.960 | was to install the latest pre-release

00:14:27.720 | because that's like something that's more stable

00:14:33.200 | than installing from master,

00:14:35.120 | but, you know, better than his,

00:14:40.120 | sometimes like here he went six months without updating.

00:14:46.400 | So yeah, I agree.

00:14:47.280 | In fact, so let's do that.

00:14:49.360 | So this is 0.6.2 dev.

00:14:52.520 | So I think we decided that we'd go,

00:14:54.880 | let's use our new PI thing.

00:14:56.440 | Tim is greater than or equal to 0.6.2 dev.

00:15:03.560 | 0.2 dev.

00:15:04.400 | Great, yeah, thanks for the reminder, Tanisha.

00:15:11.400 | All right, great.

00:15:18.360 | It's kind of this thing in Python modules

00:15:27.320 | and quite a lot of other things.

00:15:28.360 | If there's like an extra .dev at the end,

00:15:31.560 | that means it's a pre-release basically.

00:15:34.040 | And so PIP has this convention that if you say,

00:15:39.040 | I want to install something that is at least as recent

00:15:44.200 | as 0.6.2 dev, then that's a way of signaling to PIP

00:15:48.400 | that you're happy to include pre-release options.

00:15:52.340 | - Is there any reason that when you do the installation

00:15:57.240 | of a theme and then you try to use the learner,

00:16:02.240 | it says that theme doesn't exist

00:16:07.040 | when you try to load the model.

00:16:08.800 | - Right, that's because you have to restart the kernel

00:16:12.040 | after installing it.

00:16:13.360 | And so now that it's installed in .local

00:16:18.040 | every time I start a machine,

00:16:20.080 | it's going to be there anyway.

00:16:21.120 | So you wouldn't have to worry about that again.

00:16:23.120 | Okay, so this was our notebook from yesterday.

00:16:28.120 | And I wanted to try and improve the model.

00:16:49.680 | And one of the reasons I wanted to try

00:16:52.720 | to improve the model is because

00:16:55.660 | our result was worse than the top 50%.

00:17:11.120 | There you go, top 56%.

00:17:12.600 | I didn't know that was a tip, that's handy.

00:17:14.800 | And so I want to aim to at least,

00:17:22.880 | be as good as this helpful fast AI out of the box person.

00:17:27.880 | So they got .97385, how far off are we?

00:17:33.880 | You know, which is quite a bit better than ours, right?

00:17:40.260 | - That was me, that was my number.

00:17:42.480 | - Fantastic, I like it.

00:17:43.840 | It's a good notebook.

00:17:44.680 | So we're going to try to beat you.

00:17:45.760 | I hope you don't mind, but then you'll know how to beat us

00:17:48.160 | because well, at least you know how to match us.

00:17:50.920 | - My only.

00:17:51.760 | - So yeah, I saw that what you did here

00:17:58.800 | was you trained for longer, which makes sense.

00:18:03.140 | And you also used some data augmentation, which makes sense.

00:18:08.400 | So let's talk about this.

00:18:11.980 | So if we're going to train for,

00:18:14.980 | so what's your name, Gerardo?

00:18:19.120 | Is it Gerardo or Gerardo?

00:18:22.600 | - Either way, that's fine.

00:18:25.680 | - Which is right, I want to be accurate.

00:18:28.560 | - Well, my name is Gerardo in Spanish.

00:18:30.960 | - I see, so both of them, no worries.

00:18:32.960 | Thank you, Gerardo.

00:18:34.880 | Okay, so if we're going to train as long as Gerardo did,

00:18:38.360 | then, you know, if you train more than about five epochs,

00:18:42.780 | you're in danger of overfitting.

00:18:45.360 | And certainly 10, I feel like you're in significant danger

00:18:48.480 | of overfitting because your model's going to have seen

00:18:51.160 | every image, you know, 10 times.

00:18:53.800 | So in order to avoid overfitting

00:18:56.240 | to the specific images it's seeing,

00:18:58.280 | we should make it so that it sees

00:18:59.880 | a slightly different image each time.

00:19:02.200 | And this is discussed in the book in some detail.

00:19:07.200 | But basically, if you pass in batch transforms,

00:19:17.160 | these are things that are going to be applied

00:19:18.660 | to each mini batch, so to each bunch of,

00:19:21.560 | however many, 32 or 64 or whatever images.

00:19:24.460 | And this is basically a bunch of functions

00:19:27.520 | that are going to be applied.

00:19:29.440 | So what does this function do?

00:19:31.320 | Org transform, so this is transforms for data augmentation.

00:19:35.040 | So we know that the best way to find out

00:19:37.340 | what something's going to do is to check its help.

00:19:39.840 | So let's start there.

00:19:43.600 | Not help, doc.

00:19:46.500 | Okay, so it's going to do things like flip our images,

00:19:51.500 | rotate them, zoom them, change their brightness,

00:19:58.820 | their warp, let's see, show in docs.

00:20:02.880 | Okay, and here's some examples of a very cute puppy

00:20:09.860 | that Sylvia found, I think Sylvia found it.

00:20:12.100 | So this is all the same puppy, it's all the same picture.

00:20:16.020 | And as you can see, each time the model sees it,

00:20:18.660 | it sees a somewhat skewed or rotated or brightened

00:20:22.540 | or darkened or whatever version of that picture.

00:20:25.580 | And so this is called data augmentation.

00:20:27.780 | So let's try then running that.

00:20:43.940 | And so all transforms actually returns a list, right?

00:20:48.660 | It returns a list of transformations.

00:20:51.820 | So here's the flip transformation

00:20:54.540 | with a probability of 0.5 at all flip.

00:20:56.940 | It's got a brightness transformation

00:21:01.100 | with a probability of one,

00:21:02.940 | it will change the lighting by up to 0.2.

00:21:05.500 | And then a random resized crop

00:21:09.420 | is perhaps the most interesting one,

00:21:11.660 | which is it will zoom in

00:21:13.780 | such that it has at least 75% of the height width.

00:21:18.500 | And it will basically pick a smaller zoomed in section

00:21:23.500 | randomly chosen each time.

00:21:25.900 | So what we can do is when we say show batch,

00:21:29.380 | if you say unique equals true,

00:21:31.020 | it'll show the same picture each time.

00:21:34.540 | And so here you can see four versions of the same picture.

00:21:36.780 | And you can see sometimes it's flipped,

00:21:38.580 | sometimes it's moved a little bit up and down,

00:21:40.980 | sometimes it's a little bit darker or less dark,

00:21:43.660 | and it's also a little bit rotated.

00:21:45.460 | So that's what data augmentation is.

00:21:49.340 | And that really helps us

00:21:50.300 | if we wanna train a few more epochs.

00:21:52.260 | Then the second thing I figured we should do is,

00:21:57.140 | ResNet's actually great,

00:22:01.340 | but there are things which are greater.

00:22:04.100 | And as we talked about,

00:22:05.380 | Tim has a bunch of them

00:22:06.500 | and in particular comes next pretty good.

00:22:09.660 | And the other thing we could do

00:22:12.660 | is think about learning rates.

00:22:15.540 | The default learning rate used by Fast AI is one where,

00:22:20.020 | I would say I picked it on the conservative side,

00:22:23.180 | which means it's a little bit lower than you probably need

00:22:27.140 | because I wanted things to always be able to train.

00:22:29.780 | But there's actually a downside to using,

00:22:34.780 | a couple of downsides to using a lower learning rate

00:22:36.740 | than you need.

00:22:37.560 | One is that given fixed resources, fixed amount of time,

00:22:41.040 | you're gonna have less epochs,

00:22:45.980 | not less epochs, sorry,

00:22:46.820 | less distance that the weights can move.

00:22:50.420 | The second is it turns out a high learning rate

00:22:54.100 | helps the optimizer to explore the space of options

00:22:59.700 | by jumping further to see if there's better places to go.

00:23:05.420 | So the learning rate finder is suggesting things

00:23:10.420 | around about 0.002, which is indeed the default.

00:23:15.260 | But you can see that all the way up to like 10

00:23:19.020 | to the negative two,

00:23:19.860 | it still looks like a pretty nice slope.

00:23:23.420 | And the other thing to remember is,

00:23:25.100 | as we saw after answering Nick's question yesterday,

00:23:28.280 | we're using one cycle training schedule,

00:23:31.500 | which means we're gradually increasing the learning rate.

00:23:33.980 | And my claim was that by doing that,

00:23:36.340 | we can reach higher learning rates.

00:23:38.260 | So I would also say that even these recommendations

00:23:41.620 | are gonna be a bit in the conservative side.

00:23:43.880 | So what I did just before I started this call

00:23:46.100 | was I tried training at a learning rate of 0.01,

00:23:51.000 | which is

00:23:51.840 | five times higher than the default.

00:23:58.380 | And so that's up here.

00:24:00.440 | And I did find actually that that did give us

00:24:02.860 | a quite a better result with a 2% error.

00:24:07.860 | So let's see.

00:24:09.540 | I mean, obviously we've got different training sets,

00:24:11.340 | but this is hopeful, right?

00:24:14.060 | That we're gonna get a better result than our target.

00:24:18.600 | It's nice to have a target to aim for.

00:24:20.500 | Okay, so that was the next thing.

00:24:24.700 | So then since this took six minutes to train,

00:24:28.940 | it's probably a good idea to save it.

00:24:32.400 | So there's a couple of different things we can save with.

00:24:35.360 | One is .save and the other is .export.

00:24:39.040 | So learner.export saves the contents,

00:24:53.200 | that's just not very well written.

00:24:56.100 | So the learner, self means that this learner,

00:25:01.380 | and it saves it to self.path/fname.

00:25:04.820 | So learner.path/fname using pickle.

00:25:07.860 | So basically what that means is if you call this

00:25:16.620 | learner.export, it's gonna save it into learner.path.

00:25:21.620 | So let's find out learner.path is what train images.

00:25:27.120 | And so this is actually whatever we passed in here.

00:25:31.620 | So if we were to save things somewhere else,

00:25:39.740 | we've got a couple of options.

00:25:40.920 | One is to change learner.path

00:25:42.500 | by setting it equal to some other path.

00:25:45.060 | Or we can just use an absolute path.

00:25:48.860 | So an absolute path is something that starts with slash.

00:25:52.100 | And so if I were to save it somewhere in storage,

00:25:55.140 | for example, then I can type slash storage slash whatever.

00:26:00.140 | Or maybe I wanna put it in slash notebooks somewhere.

00:26:05.400 | So these are some ways you can change

00:26:11.100 | where it's gonna save.

00:26:22.820 | I might even just put it into the current directory.

00:26:25.940 | I think that seems fine to me.

00:26:27.540 | Well, actually, where are we current directory?

00:26:34.000 | Yeah, put it in get paddy.

00:26:35.700 | That sounds fine.

00:26:37.020 | Or maybe to be a bit more sure

00:26:40.740 | just in case the directory ever changes.

00:26:43.020 | Be more specific.

00:26:48.100 | So then the other option is learn.save.

00:26:53.100 | So learn.save doesn't save the whole learner.

00:26:59.260 | It just saves the model and the optimizer state.

00:27:04.020 | The difference is that remember a learner

00:27:06.460 | doesn't just contain the model,

00:27:08.700 | but it also contains,

00:27:10.800 | it also contains information about the data looters

00:27:17.580 | and specifically what transformations are applied.

00:27:21.180 | So I don't really often, if ever use.save.

00:27:27.460 | The only reason I would use.save

00:27:31.940 | is like if I was writing something to like,

00:27:34.660 | and we've already have stuff this in fast.ai.

00:27:36.980 | Well, let's give an example.

00:27:37.820 | In fast.ai, we have something that's a callback

00:27:40.540 | that can save the model at the end of each e-park

00:27:43.800 | or each time it gets a better result

00:27:45.940 | than its previous best or whatever.

00:27:47.780 | In those cases, we might use.save

00:27:50.540 | 'cause then you recreate a learner

00:27:53.040 | and you can .load into the learner.

00:27:56.640 | But yeah, for exporting something,

00:27:57.920 | I wanna be able to just load that exact thing

00:28:00.720 | with all the same details next time, .export's the way to go.

00:28:04.740 | So I'm gonna call .export.

00:28:06.540 | I'm gonna use, it's a conv next, it's small,

00:28:09.380 | and I did 12 epochs.

00:28:11.360 | Oh, and this needs to be an actual path.

00:28:16.360 | Normally we actually try to make these things

00:28:22.800 | do that for you, but this is less friendly than I would like.

00:28:25.440 | Sorry about that.

00:28:26.320 | There we go.

00:28:28.980 | Okay, so we should now be able to see it.

00:28:32.760 | There it is.

00:28:35.840 | Okay, and it looks like we need to give it a .pick or,

00:28:38.640 | or whatever.

00:28:39.480 | Okay.

00:28:40.320 | By default, it'll, with org transforms,

00:28:47.760 | which uses random resource crop,

00:28:49.120 | it'll randomly pick a subset of the crop of the image

00:28:53.720 | of this, up to this, of this size or bigger.

00:28:56.060 | And the validation set, it will pick out the center,

00:29:01.960 | it'll, you know, as all the width it can

00:29:05.120 | or all the height it can without changing the aspect ratio.

00:29:08.720 | If you say squish instead,

00:29:12.200 | it will grab the whole thing and change the aspect ratio

00:29:16.960 | to squish it into a square.

00:29:18.600 | Matt, you don't have to raise your hand.

00:29:19.680 | Just talk to me, mate.

00:29:20.680 | What's up?

00:29:21.520 | - Can you hear me?

00:29:25.440 | - I can't hear you.

00:29:26.440 | Does that mean you can't hear me?

00:29:28.440 | - I can hear, I can hear you,

00:29:30.680 | but I don't think you can hear you.

00:29:33.320 | You can't hear anybody.

00:29:34.580 | - So you do need to raise your hand.

00:29:36.960 | - I can't hear you, but you guys can hear me.

00:29:41.960 | Okay.

00:29:44.240 | - Yes.

00:29:45.720 | - Yes, we can hear you, Jeremy.

00:29:47.620 | - We can hear you.

00:29:48.620 | - I see why.

00:29:51.480 | Okay, say something.

00:29:59.020 | - Can you hear me now?

00:30:01.280 | - Yeah, yeah, I can.

00:30:03.040 | All right, ah, okay.

00:30:05.660 | All right, did you guys, were you guys saying anything

00:30:10.660 | I was meant to be hearing?

00:30:14.500 | Did I miss anything?

00:30:15.500 | - Yeah, why didn't you choose 12 epochs?

00:30:19.540 | - Oh, no particular reason.

00:30:22.660 | I just saw that this one was using 14,

00:30:27.180 | and I thought, oh, aim for something around there,

00:30:30.340 | but maybe just do a little bit less.

00:30:32.300 | I guess I often do around 12ish epochs.

00:30:35.260 | Like, it seems to, like, for fine-tuning things,

00:30:40.260 | which are somewhat similar to the original,

00:30:43.700 | it often seems to get pretty close to, you know,

00:30:49.700 | getting all the information it can,

00:30:53.020 | just as a rule of thumb.

00:30:54.220 | And that's a reasonable amount of time, too, I'd say.

00:31:02.980 | - My assumptions were that the number 460

00:31:06.740 | is because of the size of the images were 460.

00:31:11.740 | And then another assumption was 224,

00:31:14.980 | because when you show the team with the different,

00:31:19.980 | they come next, and the image size was 224,

00:31:23.780 | that's the reason that I selected that.

00:31:25.640 | Is that, okay, is that a correct assumption?

00:31:27.620 | - Well, they were 640 by 480, so actually,

00:31:30.780 | so we do this, look it up in the book,

00:31:33.940 | it's under the section called pre-sizing,

00:31:35.940 | and I think this is around what we always pre-size to.

00:31:39.660 | So actually, maybe 480 would have been better,

00:31:42.600 | because then it wouldn't have had to change

00:31:44.220 | one of the dimensions, 'cause there was 640 by 480.

00:31:46.780 | And then your size, you picked, I actually changed it.

00:31:50.840 | So Gerardo picked 230, but actually,

00:31:54.720 | most of these models that are trained on ImageNet

00:31:57.740 | are generally trained on 224.

00:32:00.700 | So I wanted them to be the same size

00:32:02.380 | as what they trained on.

00:32:04.260 | So that's why I picked 224.

00:32:05.860 | Yeah, so then Squish I've talked about.

00:32:09.540 | Oh, and then the other thing is the model I picked

00:32:14.740 | is one with a suffix in 22K.

00:32:17.620 | IN here refers to ImageNet,

00:32:21.180 | and the 22K refers to the version of ImageNet

00:32:24.400 | with 22,000 categories,

00:32:26.340 | as opposed to the version that's normally used,

00:32:28.900 | which only has 1,000 categories.

00:32:30.540 | So this is a ConvNEXT, which is small,

00:32:33.940 | but it's trained on ImageNet with a 22,000 category version.

00:32:38.580 | The 22,000 category version,

00:32:40.120 | it just has a lot more images

00:32:41.340 | covering a lot more different things.

00:32:43.540 | So there's a much higher chance

00:32:44.700 | that it's gonna have seen something like

00:32:46.660 | rice paddy illness than the one with 1,000 images,

00:32:51.780 | and it's just seen a lot more different pics, you know.

00:32:58.100 | So yeah, I would recommend always using

00:33:00.500 | the IN22K pre-trained models.

00:33:03.780 | So those are, I think, the key differences

00:33:09.460 | at the training stage.

00:33:11.100 | - Yeah, I think when you had put the export

00:33:18.280 | and then the error came, that's when it cut off.

00:33:20.540 | So I don't think you explained what you did to,

00:33:23.100 | we didn't catch the part where you explained the fix.

00:33:26.780 | - The fix.

00:33:28.700 | - Well, 'cause the export had an error, right?

00:33:31.940 | And then I guess you've now added the path.

00:33:34.460 | - I don't think it had an error, but I just,

00:33:37.240 | oh, I see, yes, yes, yes.

00:33:38.800 | Okay, yeah, the export had an error

00:33:40.120 | because this was a string and it actually is via path,

00:33:43.540 | which I'd say is an oversight on my part.

00:33:46.020 | I try to make it so that everything

00:33:47.420 | can accept a path or a string.

00:33:49.260 | So I would consider that a bug that ought to be fixed.

00:33:52.260 | So hopefully by the time people watch this video,

00:33:55.000 | that might've been fixed.

00:33:55.960 | But yes, at the moment, I had to change this to a path.

00:33:59.820 | Thank you.

00:34:00.660 | All right, so there's a few things we could do here, right?

00:34:08.400 | But one key issue is that the,

00:34:13.340 | is that particularly if you don't have method equals squish,

00:34:18.180 | when we do validation,

00:34:19.260 | it's only selecting the center of the image

00:34:22.260 | and that's a problem, right?

00:34:25.100 | We would like it to see all the image.

00:34:27.920 | And then another thing is that we've been training it

00:34:31.060 | with various different augmentations,

00:34:33.620 | but the validation set is any of those augmentations.

00:34:37.920 | So there's a trick you can use,

00:34:41.780 | which you should particularly use if you don't use squish

00:34:44.860 | and it's effectively cropping into the center,

00:34:47.620 | which is something called test time augmentation.

00:34:50.380 | And in test time augmentation,

00:34:52.980 | we basically get

00:34:56.400 | multiple versions of each image.

00:35:03.500 | We actually by default get

00:35:04.660 | four different randomly augmented versions of each image.

00:35:07.580 | And plus the unaugmented version,

00:35:12.960 | we get the prediction on every one

00:35:14.700 | and then we take their average.

00:35:16.960 | And that's called test time augmentation.

00:35:19.560 | And as I said, it's gonna work particularly well

00:35:21.580 | without the squish,

00:35:23.060 | but it ought to work well even with the squish.

00:35:25.260 | So to get those predictions,

00:35:28.060 | let's first of all make sure we can replicate

00:35:32.360 | this error rate manually, right?

00:35:35.180 | So if we go

00:35:37.220 | probabilities, targets equals learn.get preds

00:35:51.420 | and we pass in the validation set.

00:35:54.400 | And then we should find that if we ask now

00:36:01.040 | for the error rate, shift tab,

00:36:04.200 | so the inputs of the probabilities

00:36:08.220 | and the targets of the targets, there we go.

00:36:11.420 | Okay, so that's our 2.02% error rate.

00:36:14.440 | So we've replicated that.

00:36:16.100 | Okay, so now we've got that 2.02.

00:36:20.580 | I would then try out TTA.

00:36:24.240 | And of course, before we use a new function,

00:36:27.980 | we would always read its documentation.

00:36:30.320 | Here we are, .TTA.

00:36:36.240 | So return the predictions on some dataset

00:36:38.540 | or some data loader.

00:36:39.900 | We get the predictions n times by default four

00:36:44.280 | using the training set transformations.

00:36:49.580 | Great, oh, and instead of getting

00:36:51.300 | the average of predictions,

00:36:52.580 | we could also get the max of predictions.

00:36:54.580 | Cool, and the other thing

00:36:58.940 | which I definitely encourage you to do

00:37:01.380 | is it's always good to look at the source code

00:37:06.140 | 'cause my claim is that fast AI functions

00:37:09.420 | are generally not very big.

00:37:10.820 | And like also quite a bit of it's stuff

00:37:13.980 | you can kind of skip over, right?

00:37:15.300 | This kind of like, oh, what if it's none?

00:37:17.340 | What if it's none?

00:37:18.180 | Like this is just setting defaults.

00:37:19.540 | You can kind of skip it.

00:37:21.100 | Try finally is you can skip

00:37:22.780 | 'cause it's just error handling.

00:37:24.620 | With, you can pretty much split progress bars.

00:37:28.460 | You can pretty much skip.

00:37:29.940 | So the actual work starts happening here.

00:37:36.260 | We're gonna call self.get preds passes in a data loader

00:37:40.660 | and then it catenates that all together.

00:37:43.800 | And then it takes either the maximum or the main

00:37:47.020 | depending on whether you asked for the max or not.

00:37:50.100 | And it also grabs it for the validation set data loader.

00:37:55.060 | Yeah, so you kind of get the idea.

00:38:07.060 | So let's run it.

00:38:11.460 | See if we can beat 2.02%.

00:38:14.420 | So you can see here it's running at four times

00:38:19.420 | for each of the four augmented versions.

00:38:25.020 | And then it will run at one time

00:38:29.500 | for the non-augmented version.

00:38:31.020 | Okay, and it beat it, but just by a little bit.

00:38:37.100 | And then, you know, another thing is, well,

00:38:39.500 | what if we did the non-maximum?

00:38:44.500 | Use max equals false.

00:38:52.020 | Use max equals true.

00:38:55.620 | Use the maximum instead of the average.

00:38:58.180 | Yeah, I kind of wish I didn't have the squish in now,

00:39:00.100 | but I don't want you guys to have to wait 10 minutes

00:39:02.060 | for it to retrain 'cause then it's much more clearly

00:39:04.580 | see the benefit of using TTA.

00:39:06.380 | That's interesting, that one's worse.

00:39:11.380 | So I generally find that when not using squish,

00:39:24.900 | that using TTA and use max equals true is best.

00:39:30.060 | Okay, so now we've done all that.

00:39:34.340 | We can try and submit this one to Kaggle.

00:39:38.260 | So we can just repeat basically what we had yesterday,

00:39:44.060 | but instead of get preds,

00:39:45.780 | we'll do TTA.

00:39:50.860 | Now there's no width decoded, I don't think, for TTA.

00:40:00.340 | So we're gonna have to do a bit of extra work here.

00:40:03.260 | So this is gonna give us the probabilities and the targets.

00:40:07.780 | And so the probabilities, each row is gonna contain

00:40:26.820 | a probability for each element of the vocab.

00:40:30.900 | (laughs)

00:40:32.980 | So we can take a look.

00:40:47.100 | And so it's a,

00:40:50.500 | so for each of the 3,469 things in the test set,

00:40:56.740 | there are 10 probabilities,

00:40:58.620 | which presumably means the length of the vocab is 10,

00:41:01.460 | which it is.

00:41:07.260 | Great, so to find, so what we wanna do is find out,

00:41:14.340 | well, what's it actually predicting?

00:41:15.740 | And the thing it's predicting

00:41:16.660 | is whatever thing has the highest probability.

00:41:19.620 | So we're gonna go through each row

00:41:21.340 | and find the index of the thing

00:41:22.740 | with the highest probability.

00:41:24.340 | So in math and PyTorch, NumPy, that's called argmax.

00:41:28.940 | So argmax is the index of the thing with the highest value.

00:41:32.740 | So robs.orgmax.

00:41:35.740 | And so what do we want to take the maximum over,

00:41:45.460 | which dimension?

00:41:46.500 | So we wanna do it over rows,

00:41:49.340 | which I think we say dimension equals one.

00:41:53.100 | (silence)

00:41:55.260 | There we go.

00:42:03.420 | So that's the correct shape.

00:42:04.820 | So now we should be able to do

00:42:07.740 | the same thing we did yesterday,

00:42:09.220 | which is to convert that into a series.

00:42:11.860 | And now we should be able to run

00:42:21.860 | this mapping.

00:42:22.700 | Now, I realized actually this thing we did yesterday

00:42:25.500 | where we went K colon V for K comma V in enumerate,

00:42:28.980 | is actually a really long way

00:42:32.420 | of just saying create a dictionary

00:42:34.900 | from those tuples.

00:42:43.700 | So when you create a dictionary,

00:42:47.740 | you can do it like this.

00:42:48.860 | (silence)

00:42:51.020 | Right?

00:42:55.380 | Or you could do this.

00:42:59.900 | Here's a tuple of tuples.

00:43:13.420 | (silence)

00:43:15.580 | Okay, sorry.

00:43:20.100 | Here's a tuple of tuples.

00:43:21.740 | And ideally, what we'd like is to call dict

00:43:29.860 | and pass in each pair of these as an argument to it.

00:43:34.860 | And so Python actually has syntax to do exactly that

00:43:39.740 | for any function, not just dict,

00:43:41.660 | which is the function star star.

00:43:43.980 | And star star means take a mapping

00:43:48.500 | and pass it in as pairs.

00:43:52.220 | So that's what this does, right?

00:43:58.860 | That's gotta be a mapping which enumerate already is.

00:44:05.580 | So that's what star star,

00:44:11.020 | let's just pop this here.

00:44:12.660 | Why is this not working?

00:44:15.660 | I expect this to work.

00:44:17.140 | How annoying.

00:44:20.060 | Well, so much for that discussion.

00:44:26.220 | Annoying.

00:44:38.540 | All right, I'm gonna have to try to think of a better way

00:44:40.740 | to make this work, 'cause so far,

00:44:43.220 | similar problem to what we had yesterday.

00:44:47.540 | - I think you don't need the star star in that case.

00:44:56.460 | - Wow, that's nice, isn't it?

00:45:00.540 | Even better.

00:45:01.380 | Thanks for the trick.

00:45:04.140 | Okay, I didn't quite get to show you

00:45:06.460 | about how call star star is, but nevermind.

00:45:08.980 | Okay, so what I'm gonna do is I'm gonna make a copy

00:45:17.540 | of the last time we did a head of the submission.

00:45:20.500 | And one reason I'd like to do that for my new submission

00:45:23.940 | is to confirm that our new one looks somewhat similar.

00:45:26.940 | So previously we went hispanormal, downy, blast, blast.

00:45:30.060 | Now we go hispanormal, blast, blast, blast.

00:45:32.500 | And so this makes me feel comfortable that, okay,

00:45:34.820 | we haven't totally broken things.

00:45:36.740 | It's still giving largely the same results as before

00:45:40.700 | with a few changes.

00:45:41.820 | And so that's just something I like to do, okay.

00:45:45.140 | And then another thing I like to do is kind of keep track

00:45:51.540 | of stuff I've done before.

00:45:53.620 | I try not to delete things I've used before,

00:45:55.660 | and just pop it into a different notebook or a comment.

00:45:57.700 | So down here, I'm just gonna have non-TTA version,

00:46:01.520 | just in case I want that again later.

00:46:03.420 | All right, so we should be able to submit that now.

00:46:08.420 | Okay, so I use Control + R

00:46:26.500 | and then to start typing competitions.

00:46:30.140 | Okay, so this is now a Squish,

00:46:35.140 | conv next small, 12 Epoch, fine-tune, TTA.

00:46:41.720 | Fine-tune, TTA.

00:46:46.720 | (sniffs)

00:46:48.800 | Oh, what on earth did it do to my window?

00:47:13.760 | How do I get it back?

00:47:17.940 | Oh, it, oh, I see, I've got to, how does it happen?

00:47:29.080 | I've got two desktops going, I didn't notice that.

00:47:32.820 | All right, let's go and check out Kaggle.

00:47:36.780 | (sniffs)

00:47:38.860 | My submissions.

00:47:47.660 | Oh, look at that.

00:47:52.260 | Harada's still beating us, I think,

00:47:53.940 | but at least we've beaten our previous one.

00:47:56.140 | (laughs)

00:47:58.500 | - That's amazing.

00:47:59.980 | - That's great.

00:48:00.820 | Jump to our leaderboard position,

00:48:02.900 | we're gonna have a good battle on.

00:48:04.980 | (sniffs)

00:48:07.060 | 34.

00:48:09.740 | - No, I think you beat me up.

00:48:14.620 | - Wait, I thought yours was better than that.

00:48:18.740 | - I think I'm a little bit lower.

00:48:20.480 | - Code.

00:48:24.740 | Oh, you were 97385.

00:48:28.380 | Okay, 979, oh, okay.

00:48:35.260 | That's not bad, right?

00:48:36.100 | Actually, 34th out of, I mean, it's just a fun competition.

00:48:40.500 | Nobody's really trying too hard, but still,

00:48:43.580 | it's nice to feel like you're in the mix.

00:48:47.720 | How far are we?

00:48:48.580 | Okay, so this person's still way ahead, right?

00:48:53.580 | They've got an error of 1.3%

00:48:57.300 | and we've got an error of 2.1%.

00:49:00.140 | You know, something else that would be fun

00:49:05.140 | would be, you know, like you can kind of

00:49:12.380 | super easily create an ensemble.

00:49:15.780 | So maybe I'll show you how I would go

00:49:17.100 | about creating an ensemble.

00:49:18.600 | To create an ensemble, I would be inclined to maybe,

00:49:24.620 | we could create an ensemble with an unsquished version,

00:49:29.620 | for instance.

00:49:37.600 | So what I would do is I'd kind of like copy all the stuff

00:49:40.180 | that we used to get our predictions, right?

00:49:45.180 | And then I would kind of paste them down here,

00:49:50.380 | go through and remove the stuff that isn't quite needed.

00:49:54.940 | Like so.

00:49:57.460 | This one's gonna be no squish.

00:50:09.900 | And we'll do max.

00:50:17.460 | Is max equals true.

00:50:21.100 | And so then to merge cells, it's shift M, M for merge.

00:50:28.100 | And don't need the error rate anymore.

00:50:37.980 | And so this is gonna be a second set

00:50:47.380 | of probabilities and a second set of targets.

00:50:50.080 | Yeah, so we could just run that

00:50:54.420 | and take the average of these two models.

00:50:59.100 | I'll remove squish here.

00:51:00.680 | Okay, so that might be our third model.

00:51:05.660 | And then another model I would be inclined to try

00:51:08.240 | is one that doesn't use square.

00:51:13.300 | So we've got 640 by 480 images, right?

00:51:18.300 | So the aspect ratio is four to three.

00:51:23.900 | So I would be inclined to say, take that

00:51:31.220 | and multiply that by the smaller side we want.

00:51:35.300 | Okay, that gives us 298.66.

00:51:40.060 | Nice to find something that works a bit more evenly,

00:51:45.060 | wouldn't it?

00:51:48.140 | What if we did it the other way around?

00:51:51.820 | So we could create 168 by 224 images, for instance,

00:52:00.300 | or 168 by 224 images, for instance.

00:52:06.500 | 336 maybe, 336 by 252 images.

00:52:11.500 | Yeah, let's do that.

00:52:17.660 | So let's change their size.

00:52:21.860 | And I never quite remember which way round it is,

00:52:33.700 | but that's okay, we'll check it.

00:52:35.380 | So 336 by 252 images.

00:52:40.380 | And so the reason I'm doing rectangular,

00:52:47.700 | sorry, rectangular images is that,

00:52:50.100 | yeah, all of our input images are the same aspect ratio.

00:52:53.340 | So there's no particular reason to make them square.

00:52:56.500 | When some of your images are wider than tall

00:52:59.660 | and some are taller than wide,

00:53:01.600 | then it makes perfect sense to use square

00:53:05.940 | as the thing that everything gets changed to.

00:53:09.900 | But when everything's wider than they are tall,

00:53:12.500 | especially when they're all the same aspect ratio,

00:53:14.740 | it makes more sense to keep them

00:53:16.740 | at that same aspect ratio.

00:53:18.220 | And another thing I guess we should consider doing

00:53:22.160 | for 640 by 380 is to,

00:53:26.240 | you can change their resolution more gracefully

00:53:31.540 | without weird interpolating fuzziness

00:53:33.900 | by doing it by a factor of two.

00:53:38.340 | So we could do 320 instead of 640 and by 240.

00:53:43.340 | So that would be another one I'd be inclined to try.

00:53:48.300 | Yeah, in fact, let's just do that.

00:53:50.040 | Let's make that the aspect ratio.

00:53:51.740 | There we go.

00:53:59.860 | And so obviously we should check it

00:54:02.360 | and we know how to check it, which is to go show batch.

00:54:06.680 | Okay, so you can see I've got it the wrong way around.

00:54:13.620 | There we go, that's better.

00:54:23.180 | Cool.

00:54:24.000 | Oops.

00:54:41.060 | And like given that we're gonna have such nice clear images,

00:54:44.700 | I would probably do the affine transforms

00:54:48.140 | or the ones where we're zooming and rotating and stuff.

00:54:52.540 | So to say don't do those so often,

00:54:54.900 | we can change the probability of affine transforms

00:54:57.500 | from 0.75 to 0.5.

00:54:59.340 | Probability of affine transforms to 0.5.

00:55:04.780 | So in theory, I feel like this one feels the most correct

00:55:11.820 | given that the data that we have is a fixed input size

00:55:17.940 | of that type.

00:55:20.100 | So I would be inclined to,

00:55:23.620 | well, we'll take a look afterwards, but.

00:55:29.260 | Oh, what did I just do?

00:55:42.300 | Copy.

00:55:49.520 | We'll save a different set.

00:55:54.520 | And so we can easily then check the accuracy

00:55:58.800 | of each of them.

00:56:00.000 | And this one's gonna be rectangular, rectangular.

00:56:04.120 | There we go.

00:56:07.680 | Now that we're saving a few,

00:56:14.040 | I guess I'm a little worried that paper space

00:56:18.640 | might disappear.

00:56:22.520 | And so I'm actually inclined to save these

00:56:25.200 | into my notebooks directory just to be a bit paranoid.

00:56:29.480 | Copy, paste.

00:56:41.160 | And so let's move,

00:56:46.880 | let's move this one into slash notebooks.

00:56:50.520 | Oh, that's right, I'm not using paper space,

00:56:54.320 | so I don't have to.

00:56:55.800 | I forgot, nevermind.

00:56:57.840 | I'm on my own machine.

00:57:00.480 | I like the fact that I've got paper space so well set up

00:57:03.880 | now that I don't even remember I'm using paper space.

00:57:07.740 | Okay.

00:57:09.560 | Great.

00:57:14.960 | I think that's that.

00:57:16.080 | All right, I'm gonna not have you guys watch that run

00:57:18.160 | for 20 minutes, so I'm gonna go.

00:57:20.280 | Any questions or comments before we wrap up?

00:57:24.840 | - So I see that you're like focusing a lot

00:57:31.040 | on like the data transformations and augmentations.

00:57:34.360 | When would you focus on that versus, you know,

00:57:37.360 | playing around with different models

00:57:38.640 | and things like that instead?

00:57:40.080 | - Given that this is a image classification task

00:57:45.080 | for natural length, for natural photos,

00:57:54.760 | it will almost certainly have exactly the same

00:57:59.040 | characteristics as ImageNet in terms of accuracy,

00:58:04.040 | or at least any fine-tuning on ImageNet.

00:58:08.360 | So I'm working on the assumption,

00:58:12.840 | which we can test later,

00:58:14.520 | but I'm pretty sure it's gonna be true,

00:58:15.660 | that the things that are in that notebook

00:58:19.080 | showing which TIM models are better than others

00:58:22.800 | will apply to this dataset.

00:58:25.040 | So I would, once everything else is working really well,

00:58:30.040 | you know, I would then try it on a couple of models

00:58:35.040 | or at least run it on a bigger one,

00:58:36.760 | like base or large or whatever I can get away with.

00:58:39.280 | If it was like a segmentation problem

00:58:44.960 | or an object detection problem

00:58:46.560 | or a medical imaging dataset,

00:58:48.520 | which has the kind of pictures that aren't in ImageNet,

00:58:50.640 | you know, for all of these things,

00:58:51.800 | I would try more different architectures.

00:58:56.480 | But then for those cases,

00:58:58.000 | I would, let's say it was a segmentation problem,

00:59:01.560 | which is about recognizing what each pixel is,

00:59:04.000 | it always is a pixel of.

00:59:07.160 | Even there, I would not try to replicate

00:59:10.880 | the research of others.

00:59:11.920 | Instead, I would go and look at something like

00:59:14.120 | paperswithcode.com to find out

00:59:16.520 | which techniques have the best results on segmentation.

00:59:21.080 | And better still, I would go and find

00:59:22.680 | two or three previous Kaggle competitions

00:59:25.040 | that have a similar problem type

00:59:27.080 | and see who won and see what they did.

00:59:30.000 | Now, when you look at who won,

00:59:32.680 | they always say, oh, we made an ensemble, which is fine.

00:59:36.800 | But the important thing isn't that they did an ensemble,

00:59:38.960 | it'll be, they'll always say pretty much,

00:59:40.920 | the best model in our ensemble was X.

00:59:43.520 | And so I would just use X.

00:59:45.040 | And I would use this kind of like smallest version of X

00:59:47.400 | I can get away with.

00:59:48.560 | And yeah, generally fiddling with architectures

00:59:54.840 | tends not to be very useful nowadays

00:59:57.200 | for any kind of problem that like

01:00:00.360 | people have fairly regularly studied,

01:00:03.080 | which almost any computer vision problem is of that type.

01:00:06.640 | I guess the only interesting question for this one would be

01:00:11.720 | there is something saying what kind of rice

01:00:13.600 | is in this patty, which is like a category,

01:00:17.760 | but I'm fairly sure that using that information

01:00:22.720 | is not gonna be helpful in this case,

01:00:24.720 | because the model can perfectly well see

01:00:26.240 | what kind of rice it is.

01:00:27.400 | So I had very much doubt we have to tell it

01:00:30.000 | 'cause it's got pictures.

01:00:31.240 | - Jeremy, it's gonna take me a while

01:00:35.840 | to work our way through all of the videos.

01:00:38.320 | - Yeah.

01:00:39.160 | - Are they gonna be actually available?

01:00:41.880 | - Yes.

01:00:42.840 | - Cool, thank you.

01:00:44.400 | - And don't feel like you can only join

01:00:49.400 | if you've watched all the previous videos

01:00:51.440 | and don't feel like you can only ask a question

01:00:53.720 | if you've watched all the previous videos,

01:00:55.360 | like it's totally fine to ask a question about a video

01:00:59.640 | we did a week ago or about something

01:01:03.360 | that we just covered yesterday or whatever.

01:01:05.280 | If the answer to your question is,

01:01:07.000 | oh, we covered this in this video, here's where you go,

01:01:09.720 | I will tell you that, and that's totally fine.

01:01:11.560 | But, and if it's like, okay, you said this thing

01:01:15.080 | in this other video, but I don't get it, say it again,

01:01:17.480 | that's totally fine too.

01:01:18.840 | Like we're moving at quite a fast pace

01:01:23.140 | because people can go back and rewatch the videos

01:01:25.860 | and because people can come back later

01:01:27.600 | and ask questions about things that aren't clear.

01:01:30.220 | So yeah, it definitely does rely on people turning up

01:01:34.160 | and saying, I'm not clear on this or whatever.

01:01:38.280 | - Yeah, well, I sort of started from ground zero

01:01:40.880 | in this whole environment,

01:01:42.320 | but it is starting to make sense now.

01:01:45.400 | I'm starting to feel a little bit more comfy with it.

01:01:48.200 | - Nice, well done.

01:01:49.040 | - And I just wanna take the time to work through,

01:01:52.360 | my way through and absorb what you've been talking about.

01:01:56.240 | - Well, also Daniel, I will say like,

01:01:59.000 | there's a couple more lesson lessons to come.

01:02:02.760 | Like what is it next week or the week after?

01:02:04.840 | I suspect during those two weeks,

01:02:06.080 | I'll probably stop the walkthroughs.

01:02:07.920 | So there'll be a couple of weeks there to catch up.

01:02:11.200 | But yeah, like feel free to join in any time

01:02:16.080 | or not join in any time and ask questions about any video

01:02:19.720 | or even about things that's not covered in a video,

01:02:21.960 | but you feel like would be something useful to know

01:02:25.720 | in order to understand.

01:02:26.600 | - Okay, I'm really looking forward

01:02:28.800 | to the tabular data actually.

01:02:30.760 | - Oh, cool.

01:02:31.680 | - Yeah, okay, thank you.

01:02:34.040 | - Thanks all, see you next time.

01:02:35.920 | - Bye.

01:02:40.160 | - Take care.

01:02:42.160 | [BLANK_AUDIO]

Live coding 9

Chapters