Okay, so let's, yeah, let's look at how we can automate a process like the one that we packed together yesterday. And actually what we might do is have a look at something that Raddick put on the forums because I kind of want to have all the stuff we want to include there and then we can automate the whole thing.

Yeah, here's Raddick's thing. Okay, so Raddick, do you want to tell us about what this thing is? What your project, your goal, or what is, yeah, what's this forum post about? This is another of Raddick's crazy inventions, but essentially this is how I learned, right? So if I, this is the first AI way of learning.

I think of this as the first AI way of learning. So for those who haven't read it yet, can you just take us through what it is? Oh, yeah, absolutely. So yesterday in the works through, we covered some material and I wanted to find a fun way to practice it.

So I went to Kaggle, I looked for competitions and there was a competition that seemed like participating in it could allow me to practice what we learned about the work through. So I created this as a resource for others. You know, I've been creating such things for a couple of years now and maybe it requires also a little bit of understanding of Kaggle, which somebody who has not played around with Kaggle yet, they might not have it.

So I thought, Hey, let me put this together, share it with others. So this is a getting started part with a community competition. It features images of plants. And you're supposed to detect one of nine or one of 10 classes of plant diseases that they can be affected with.

I haven't looked too much at the data. Essentially, I relied on first AI's functionality, where if I present the image data loaders, and then by that token, if I present the learner with data that seems to be appropriately formatted, then the learner will do the rest. So I tried to bridge through this as quickly as I could just to put it together and to practice on my own as well while doing this.

And hopefully this can be useful to others. Great. Shall we try going it through together? There's somebody's got a fast AI one here already. Sounds fun. Let's try it. So that's great. And I like the fact that you're looking to run it on paper space rather than on Kaggle.

I think that's good practice. So let's do the same thing that Radek did. So on Kaggle, you can see all the competitions that are running, active competitions. So there are various different types, right? There's kind of the normal ones that have money involved. And as well as having money involved, they also have ranking points.

So that gives you the opportunity to try to become a master or grandmaster or whatever by getting ranking points. Then there are some which are just for kudos. So there's no money involved and no ranking points. And you can kind of use the little buttons at the top to find getting started competitions.

So those are just knowledge. Some have prizes. I wonder what prizes they have. A TPU star. 20 extra hours of TPU time for four weeks. There you go. Kind of sounds like a drug. Like you use TPUs to win and then you need more to keep going. I love it.

And then there's also playground competitions which they kind of repeat each month. All right. So looks like Radek's picked out a kudos competition. So we're not going to get any money or ranking points. We're just doing it for the enjoyment and the learning. So we have to click, before you can download the data, you have to click join competition.

And this is a really common mistake people make is they try to download the data without doing that. You'll get an error. So as it says you can download the data by running this command here. Now that's not going to be installed yet. So we need to install it.

So we can install it with pip install minus minus user etc. I feel like given how much we're typing pip install minus minus user whatever it might be worth creating an alias for. This is taking a long time. >> Do you like paper space so far generally? >> Very very much.

Yeah. It's the first platform I found which I feel like I can use this you know. >> Nice. I have to give another try then. >> Not opening my cheaper 11. Yeah, make sure you do all the previous walkthroughs because it really does take you through like how to take advantage of this and it's yeah, otherwise it's not particularly exciting.

All right. I'm having some trouble with Jupyter Labs. So I'm just going to go to and use their rather unappealing GUI. Hopefully that'll work. No, even that's not working. >> Do they have the ability to SSH into these machines? Do you know? >> They do not. But because you've got Jupyter Lab installed, you get a full terminal.

So it doesn't really make any difference. And you can also connect VS Code to them. Anyway, for some reason, this is the first time this has happened. It's not liking me. >> I can't open a Jupyter Lab notebook either. It's timing out. I'll see if I can fire it up in VS Code then.

It's unusually sluggish today. >> This is my experience whenever I've tried it before. Maybe it's just I'm here, so the curse came with me. >> It is to happen to me all the time. And they seem to have really improved. I'm going to try a paid one. >> I've never had one before.

>> I'll try a green dot. It's promising. >> I've got two tabs. The green dot is the free one. >> I can't connect through VS Code either. >> Not wanting to start. >> It should be a red dot. It's not working. >> No worries. Change of plans. We'll do things locally.

We will yell at paper space. What the hell just happened? Let me just switch to my other user. Let's see if Kaggle's installed. There's no Kaggle. Here I am with no Kaggle. I'll start by creating a Tmux session. I like to be able to run a few things at the same time.

I would run pip install minus user Kaggle. There we go. Now, Kaggle is not just a Python library, but it also has a command line tool. And because I did minus minus user, it installed the command line tool into my home directory, into the dot local folder. And binaries, things you can execute, are generally put in bin.

But dot local slash bin in my home directory isn't in my path, and therefore I can't type Kaggle. As we know, to fix that, if you're on paper space, you would modify slash storage slash bash dot local, or here in my local machine, I would just modify my home directory.bashrc.

>> On paper space, if you do this, you can modify and bash dot local. It will not run before Jupyter notebook runs. >> Correct. Which is fine. >> Okay. >> Yes. Because Jupyter notebook doesn't need access to this, unless you want to do exclamation mark Kaggle. If you want to put exclamation mark Kaggle in Jupyter, you would need to put it in tree dash run dot sh.

Is that your point there? >> It is, yes. >> Cool. Cool. Great. Yeah. So, something dot bash dot local only will execute in a new terminal. Sorry. Go on. >> So, I suppose, like, in some operating systems, I think the local bin directory is on the path by default.

So, maybe, do you think that it's just whatever reason? >> I've never seen that. But it's possible. There's a lot of distributions around. >> I could be confusing with, like, user local bin or whatever. There's some of them on Mac. >> Yeah, slash user slash local slash bin is not in your home directory, and that is always part of your path.

But this is something in your home directory. >> Oh, yeah. Okay. >> All right. Okay. So, by default, Ubuntu has a bunch of stuff in your bash RC, by the way. So, I'm just going to go to the bottom. So, to go to the bottom in Vim, it's shift G to go to the bottom.

And then O to open up a new line underneath this one. So, insert beneath. And so, we will export path equals Tilda slash dot local slash bin. And then colon and then everything that's already in your path. So, that prepends it to our path. So, I could close and reopen my bash RC, or I can re-execute it.

And any exported variables I want to go into my shell. So, to execute stuff and put variables into your current shell, you can type source. So, source dot bash RC is going to save me having to close and reopen my terminal. And dot bash RC is the last thing on the last line.

So, I can just do that. And so now we can Kaggle. Okay. So, the next thing we need is somewhere to authenticate. And Kaggle uses something called Kaggle dot JSON to do that. So, if I go to Kaggle, you can grab it. There we are. You can grab it by clicking create new API token.

And what that will do is it will download a file called Kaggle dot JSON to your computer. And so, once it's downloaded, depending on where you are on Mac, it might be in your Twitter slash downloads directory. And on Windows, it will be in slash mount slash C is your Windows C drive.

And it will be in your user's user name downloads directory. So, they say it needs to be in a directory called dot Kaggle. So, I'll go make dot Kaggle. I think it's probably just created that for us when we tried to run it. That's good. So, now I can copy it.

And in this case, I think what I'm going to do is just copy it from my other account. Copy dot Kaggle slash there's my JSON. And I'll copy it into my by the way, so I want to get the JPH 00's home directory. So, tilde JPH 00 refers to the home directory belonging to JPH 00.

Tilde on its own means the current user's own directory. So, I'm going to copy it over to dot Kaggle. There we go. And change its ownership so it's owned by JPH 00. So, you won't have to do this because you'll be downloading it and copying it from downloads. I'm just doing this because I'm copying it from a different user.

All right. So, yep, that now belongs to JPH 00. So, now I should be able to go back into that username and type Kaggle. Okay, great. So, I've got Kaggle installed. And we'll do a check from time to time to see whether anything's working. Not really. Okay. So, the Kaggle competition said we can download it with this command.

So, I'll copy that. And let's create a directory for the competition. Patty. And run that command. Nice. Gigabyte of data. All right. Did anybody have any questions or anything about this as we wait for that to download? So, Jeremy, we can use Mamba install here since you're doing it on local, right?

It's just demonstrating paper space. If it's on, yes, it is on kind of forge. So, yeah, Mamba install Kaggle should be fine. Although, you know, to be honest, like, for simple pure Python stuff like this, I often just use pip anyway. Because things like this, like, pretty much most tools are Python libraries.

Like, pip is the main thing people are kind of targeting. So, you can be sure that that's going to be the most recent version. Unless the documentation explicitly says, like, we provide kind of packages as well, there's often a good chance that the kind of packages will be behind.

So, if I was going to do a member install, I would be inclined to, like, double check that this is actually the most recent version. But, yeah, as you see, I just use pip anyway, I suspect, for something like this. >> This used to be the case that you -- I remember something about, like, cookies and there was a browser extension and maybe you had your own tool for this.

Or am I just hallucinating? Did it used to be this way? From, like, an older, faster? Okay. >> Okay. So, there are always things so we can unzip it. Okay. So, I hate it when that happens. Because it actually takes ages. So, minus Q for quietly unzip it. Right.

Okay. So, that's going to give us our data. I guess one thing is for getting our Kaggle.json onto paper space, the easiest way is to click the file upload button in JupyterLab. So, there's just a little upward pointing arrow button. If you click that, it'll upload it. And then, yes, copy it to tilde slash dot Kaggle.

And it does have to have the correct permissions. Which is hopefully you might be able to recognize this. So, that's 4 plus 2 is 6. And then 0 0. So, chmod 600 on that file will give you the correct permissions. Okay. So, now, the only problem is that this is my desktop, which does not have a GPU.

So, that was actually a stupid place to put this. So, I've got to copy this to my GPU server. So, to copy files from one Linux or Mac thing to another, a very easy way to do it is SCP, secure copy, and type the name of the file. And then type where you want to send it to.

Oh, except I don't have that set up here. All right. So, I'm just going to go back to my normal user. So, you know, copy tilde jph00, get patty disease classification. I'm going to copy that here. So, you can use SCP to copy a file to another machine. And off that goes.

So, how does it know what local colon is? So, there's a very underutilized handy file called .ssh/config where you can type things like host local. And when I SSH to that, it will actually SSH to this host name. And it will actually use this user name. And it will set up, we haven't talked about SSH forwarding, but if you know about that, it will set up SSH forwarding.

So, this is just a little trick for people who do use SSH, that using the SSH config file is great. And it's not just for SSH, it's also for anything that uses SSH, including SCP. SCP is a secure copy over SSH. All right. So, now that's done, I can log in to that machine.

And now we're on a GPU machine. So, to check your GPUs, you can type nvidia-smi. And so, this has got three GPUs. And I can move that file, I just copy it into here, into here. So, should we use SCP or Rsync? Oh, that's fine. Yeah. I use SCP just because I don't have to type any flags to it.

Strictly speaking, SCP is kind of considered deprecated nowadays, but it actually works fine. Unzip that. Cool. Okay. Making good progress. Let's see what we've got. Okay. So, there's a sample submission.csv, there's a train.csv, train images, test images. So, ls train images, if this has got like 10,000 things in it, that's going to be annoying.

So, if you pipe to head, so remember this vertical bar is called pipe, means take the input of this, output of this program and pass it in as the input to this program. And this program shows you the first 10 lines of its input. Okay. So, actually, it turns out that's got folders for each category.

So, I don't really need to pipe it to head. Okay. And so then we could do the same thing with one of these, bacterial leaf blight and pipe that to head. There we go. So, now we might want to know like how many of those are there. So, instead of piping to head, we can pipe it to word count, which is wc.

But despite the name, it doesn't only count words. If you pass in L for line, it'll do a line count. So, that's how many bacterial leaf blight images there are. So, it's really useful to play around with these things you can pipe into. So, head, wc, another useful one is tail, which is the last 10 lines.

And then one we've seen before is grep. So, not particularly useful, but show me all the ones with the number 33 in it. Okay. And you can use head and tail also on files. So, head is very useful for csv files. If you're in your Jupyter notebook and it's streaming at you, then it cannot read a csv file.

It cannot parse a csv file. You can just jump into console or even from Jupyter notebook, just do head. Yeah. Well, let's try it, right? Because so, I think we know that if you type cat and a file name, it will send it to the output, which by default prints it to the screen.

So, we could pipe that to head, right? Now, real Unix gurus will say, well, that was silly because actually if you look at the band page for head, if you pass it a file name, it does the same thing. But to me, I prefer to learn a small smaller number of composable things.

So, piping stuff to head is not a bad idea. And we could even, and, you know, another nice thing about cat is I can pipe it into grep and search for everything with, I don't know, how many of these ADT45s are there? That's grep for ADT45 and then pipe that into word count but count lines.

Yeah. So, you can quickly get some information at the console, which, yeah, I think can be quite useful. All right. So, next thing to do, I reckon, is to fire up a Jupyter. So, let's see the get Jupyter notebook. Excuse me, Jeremy, if you were interested in my paper space, Jupyter instances started up now.

Hooray. So, I don't know if yours would have to. Look at that. Fantastic. All right. So, it's probably worth just quickly going through the exact same process one more time, I guess, isn't it? So, we open up the terminal. Pip install Kaggle minus minus user. Ah, that's interesting. So, this is because I installed stuff to that Conda directory the other day.

And so, if I go which pip, it's actually finding that one. And I don't want there to be a pip there. So, we'll remove it. Conda, oh, in my home directory. Okay, let's try that again. Do I have to reopen this terminal? How confused is it? Which pip? There we go.

Okay, now it's happy. Control R, install. To find the last thing I typed, saying install. Okay, we've got the path issue again. So, vim slash, I think I prefer radix approach. I'm putting it in pre-run, so that way we have the ability to use this if we wish in Jupyter.

Not echo. So, export path equals dot local n and then the current path. One of the confusing things I find about pass, and it got me a couple of times, if you are then export something, a variable name, you need to have the equality sign straight after the variable name.

Oh, yeah, no space. It won't work. And, you know, it's just one of these little quirks where things are different. Yeah, you know, Bash is a very old program, and it has these weird old quirks about white space sensitivity. So, that's a really important point to mention. Thank you.

And I'll run it here as well, rather than restarting. And so, now Kaggle should exist. It does. It runs. That's good. All right. And so, let's copy this into my downloads directory. Or else, I guess what I could do... Yeah, let's just do that. Copy tilde slash dot Kaggle Kaggle slash mount slash the user's J downloads.

And so, we should be able to now upload it from my downloads directory. There it is. Okay. And so, it's created a dot Kaggle directory for us. Wait, oh, sorry, this is my wrong. Sorry, let's do that again. CD tilde slash dot Kaggle. Yeah, it's created a Kaggle directory for us.

And so, we should be able to move the thing that we just uploaded to slash notebooks into here. And the permissions will be wrong. So, we can fix them. Okay. And so, let's see if it works here as well. It does. And PaperSpaces network is faster than my connection in Australia, not surprisingly.

Although, you know, mine wasn't bad, actually. Okay, so... Oh, that was a dumb place to put it, obviously. I don't want to put it in paddy disease classification. You know, we're only going to use this for this notebook, I guess. So, maybe move that to slash notebooks. And so, let's create a paddy folder.

Pop it in there. And unzip it. So, that means... Okay, that's interesting. There's no unzip, but we know how to deal with that. Why isn't Control-R working for me? Oh, because Control-R does a refresh. Oh, that's annoying, isn't it? So, how do we search our history in these terminals?

Oh, well. That's fine. I will just type it in manually and we will figure out how to make Control-R working at some other point. So, micro member minus C condor forge minus prefix tilde slash condor install. Probably need the install first. Yeah, a lot of the keyboard shortcuts don't work in the browser-based terminal, which is actually pretty annoying.

They work a bit better on Mac than on Windows, because on Windows, the Control key is both used for the Linux terminal commands, and it's also used for the normal browser commands. Whereas on Mac, they use command for the browser commands, and so the Control key doesn't get overwritten.

So, this would probably be a better experience on Mac, actually, than Windows. Okay, so we're going to install Unzip. And hopefully, by the time people watch this video, if it's like July or later, things like Mamba and Unzip will already be installed. Okay, let's check. Okay, we have an Unzip.

That's good. Okay, so that is on its way. So, that's going to use up a gigabyte of space in my persistent storage, which you might not want to do that, right? And if you don't want to do that, then instead, you should unzip it into your home directory. If you unzip it into your home directory, it won't be there if you close it down and reopen it, right?

So, you might want to create a little script for yourself that does the Kaggle download and the Unzip on your notebook, and then you can run that each time you start it up. So, these are the issues. I mean, look, having said that, the average cost on paper space for storage, I believe, is $0.29 per gigabyte per month.

So, your convenience of putting it in storage is probably worth $0.29 for the one month. You're probably going to want it there. So, maybe that's just a better plan. I do know, though, that the... Well, maybe this is a problem, actually, because I do know the paper space /notebooks and /storage are very, very, very, very slow.

And we can actually see that when we're unzipping this. So, maybe this is a bad idea. Maybe we shouldn't put data, at least when there's lots of fails. Because this is painful. I'm going to cancel it and see how far it got. TU minus SH train images. 426. And how about test images?

Wouldn't you know it? It was nearly finished. But, yeah, I think this is actually slower. So, I'm going to remove that. And I have a strong feeling if we move it back to our home directory, it's going to be faster. I sure hope so. And the reason I care is not so much for the unzipping speed, but when it comes to training a model, we don't want it to be taking ages to open up each of those files.

You see, even RM minus RF takes a long time. So, while that's running, let's move patty/zipfile, pop it into our home directory. There we go. And then cd to our home directory. Okay, that's now finished. So, in terms of the steps we're going to do, it would be first we would make a directory for it.

We would then do the Kaggle download. Actually, which we can just copy easily enough from Kaggle. And then we would unzip. Let's see how long it takes. So, the time Unix command runs whatever command you put after it and tells you how long it took. Did I not move it there?

Oof.dot/patty. Oh, I didn't move it there. Yeah. Time, unzip, quietly, patty. So, yeah. So, I think what I would do, now I think about it, is I would have a patty directory in my notebooks. I wouldn't store anything big here. I just have my notebooks here. And I would put a script here called get data, say.

And it will just have each of the steps I need. So, the steps would be cd to my home directory, make the patty folder, cd to the patty folder, do the wget, or not wget, Kaggle competitions download, I should say, unzip it, patty disease. Yeah. And I think that's it, right?

So, we can make that executable with chmodu+x to add the executable permission to it. And so, yeah. So, then all I have to do is run that thing each time I start up PaperSpace. And, yeah, it's only going to take eight seconds to unzip, and it took about five seconds to download.

So, actually, that's not going to be really any trouble at all, is it? Cool. And that's, you know, /notebooks, remember, is persistent on this machine. So, that's all good. So, now we can create a notebook for it. And so, my first step is always just to import computer vision functionality in general.

Which is the same thing we used yesterday. And now you know exactly what that does. And then my second step is to look at the data. So, it's easiest to look at the data if we set a path to it. So, it's going to be in our home directory.

And it's going to be called patty.plash. Well, that's okay. It's just /patty, right? It can go past that home. Wow. I didn't know that. That's quite neat. Yeah, it is quite neat. So, that's that. Okay. So, we can path.ls tells me what's in there. And if you remember my trick from yesterday, I also like to set that to be the path.basepath just so that my LSs look a bit easier to read.

There we go. So, at this point, we could create a data frame by reading in the CSV of path/train.csv. Okay. So, we've got 10,000 rows. Each one is a JPEG. Each one's got a label. And so, let's take a look at one of the images, shall we? Oh, yeah, PIO image.

Path/train/ actually, you know, let's make life a little bit easier for ourselves by creating a train path. Because, you know, it's just so good to be lazy. /100330.jpg. Oh, no, because then they're inside the label directory. Yes. So, what we actually probably should have done would be to say turn path.ls patty_train, is that not right?

train_images. And that's another good reason to put it in a variable. So, you have to change it in one place. And so, there we have that. And so, let's create, I don't know, let's call it the bacterial_leaf_light_path=train_path/bacterial_leaf_light. So, now we should be able to go BLB and look at that image.

Oh, there we go. So, we have an image. Yay. All right. So, might be nice to, like, find out a bit about this. Maybe look at the size. So, it's a 480 by 640 image. Great. You know, another way we can take a look at an image, you might remember from yesterday, you can go files=get_image_files and pass in a path.

And this will be recursive. So, I can do this. As you can see. So, this has got the 10,000. Okay. And that number there matches that number there. So, that's a good sign. And so, another way to do that would have been to go image=pil.create_files 0. Okay. And we could even take a look at a few, right?

So, if we wanted to check that the image size seems reasonably consistent, we could go o.size for o in, well, actually, pil image dot create o dot size for o in files 10, for example. So, you know, this is not particularly rigorous, but it looks like they're generally 480 by 640 files.

They're all the same size, which is handy. That's interesting. And they're probably bigger than we normally need. You know, we normally use images that are about 224 or so. Having said that, I don't know if, like, presumably this is some disease thing, paddy disease competition. So, it's rice. Classify the images according to their disease.

So, I can't even tell that this thing has a disease. So, I don't know how big it needs to be to see the disease. So, it is possible it'll turn out that we actually need full-sized images. So, like, I would start by using smaller images and kind of see how we go.

Anyway, 640 by 480 is not giant. So, we should be fine. The CSV file has got one extra bit of information, which is the variety. Radek, did you find out what this variety thing is about? From the doubt, I didn't even know that that CSV file existed. But it's fun because we can build a multimodal model from data.

I see. It's the type of rice as opposed to the type of disease. Yeah. So, maybe the different diseases might look different depending on what type of rice it's on. My guess is that we wouldn't need to use that information because given how many images there are, I would guess that it's going to do a perfectly good job of recognizing the varieties by itself without us telling it.

Unless there's a whole lot of different types of varieties, which we can check easily enough by checking the data frame, grabbing the variety, and doing a .value counts. And we can see how many there are of each. Okay. So, look, I mean, there's a couple of tiny varieties, but on the whole, most of it is ADT 45 and quite a bit of Kanaka-Ponni.

It does seem like a bit of a rice session today, doesn't it? Lots of rice going on. Yeah. So, I think it's very unlikely that this variety field is going to help because there's so many examples of the main one anyway that it's going to be able to recognize it.

I mean, at some point we can try it, but I would be making that a pretty low priority for this competition. And so, yeah, given we're doing a practice walk through, I'd be inclined to fire up Fastbook and the intro and see if we can just basically do the same thing that we did last time.

So I'm going to merge these back together again. We've already got those, too. We've got those. Well, there's not much there, is there? Oh, I'm in APL mode. I wonder why things aren't working. I don't know how that happened. I haven't used APL today. Copy, paste. Okay. So this is how we did cats.

So we needed a labeling function. Now, in our case, the labels are very easy. Each image is inside the directory, which is its label. So the parent folder name is the label. And so we already have a function to label from folders. So we can actually just do image data loaders from folders, because that's all we need.

So we're still going to need the path. Train and valid actually have different names. So let's fill all of those in. So we're going to have path train equals train underscore, what was it? Images? Yep. And test images. Train underscore images. Valid percent. So that's fine. We'll do that the same as last time.

Okay. It's expecting to have train and valid subfolders. Oh, all valid percent. So hopefully that'll work. Let's try it. And we'll use the same resizes last time. Okay. All right. Oh, no, that did. Well, did that work? No, it didn't work because we've got, that's interesting. Test images. So my guess is it's got confused by the fact.

Yes. Okay. So possibly what we should instead do is use train path here. And use valid percent instead. I wonder if that'll fix that problem. There we go. Let's fix that problem. Okay. Great. So we should then be able to create a learner. And learn dot fine-tune. Let's just do one epoch to start with.

There it goes. So it can be useful to kind of make sure it's being reasonably productive as it's training. And we can do that with nvidia smi. nvidia smi minus help. Oh, so much help. So there's, let's take a look here. We've only got one GPU, so that's fine.

Loop query. Okay. We're not modifying anything. daemon. I think that's the one we want. nvidia smi. daemon. Okay. That's just finished. So while it was running, so this is something people often say to use watch nvidia smi to like have it refresh. But actually I don't think most people know that there's a daemon subcommand where you can use that just as you can see it shows you every second how it's going.

And it's showing me the most important thing is this column SM. SM stands for symmetric multiprocessor. That's kind of what they call it instead of a CPU for their GPUs. And it's showing me that it's being used 70 to 90 percent kind of effectively if you like. And that's a good sign.

That's fine. So if this was like under 50, then that would be a problem. But it looks like it's using my GPU reasonably effectively. Yeah, and it's got the error rate down to 13 percent. So we are successfully training a model. So that sounds good. So, Jeremy, just a quick question.

When you're saying that like if it's under 50 percent, then that can be a problem. Is that because you've oversized the GPU like when you selected it or like just just want to clarify what you know about that? What that would mean. Yeah, thanks. It's a good question. Just rename this.

It would probably mean that we're not able to read and process the images fast enough. And so in particular, my guess is that if they're in slash storage or slash notebooks, you would see the SM percent be really low because I think it would be taking a really long time to open each image because it's coming from a network storage.

And so generally, yeah, a low SM means that your IO, your input output, your reading or processing time is too high. And so the ways to fix that would be a few. One would be to move the images onto the local machine so they're not on a network drive.

A second would be to resize the images ahead of time to make them a more reasonable size. And a third would be to decrease the amount of kind of augmentation that you're doing. Or another would be to pick a different instance type with more CPUs. So those are basically the things.

All right. Okay. Just to end the system, my command also has a lot of useful information like your CUDA version and stuff like that. So you know, it's also useful command even without demo to know that it exists. Yep. A lot of details here. So if you're looking for the IDX of your GPU, it might be GPUs.

And some of the variables here are a little bit more descriptive. So it might be easier to get started with that command or to at least use it every now and then. And if you'd like to have this one running in a loop, which is what I generally do, just do nvdr-smi-i.

Yeah. Yeah. I mean, I agree this is useful, but I would suggest in a loop to use the daemon because there's only two columns you care about. And this one does not show you SM, right? So if you want to actually see it's being utilized, you need to use daemon.

And you can also see the percentage memory utilization. So just look at these two columns. The other ones you can actually ignore. Yeah. Okay. I think that's a pretty good place to stop. I'm glad you put us onto this competition Radek. It looks fun. And I feel like we've got a reasonable start.

So yeah, maybe next time we can try doing a submission. And we could also try creating a Kaggle notebook for other people to see. How does that sound? Sounds excellent. One thing I also like about this is that we're coming up across problems as we go and jumping through those hoops.

And these are the beginner sorts of roadblocks that we'll have to face, I guess. Exactly. And if you guys, you know, repeat these steps or do it on another dataset or whatever and hit some roadblocks, then it's really helpful. If you solve them, you know, come back tomorrow and tell us what happened and how you solved it.

And if you didn't, come back tomorrow and tell us to fix it for you. I think they're both useful things to do. So things like Radek's example of like doing a bash environment variable and having a space next to the equal sign, you know, that kind of stuff. I forget even to mention it, but really useful information.

You know, this competition is nice because it's relatively small, like 10,000 images, and it's aligned with what you're doing in the course. But if you'd like to try something out on a competition that is not active right now, you can still do this. Rascaggle allows you to do this late submission thing.

And this opens up many competitions to play around with. The current competitions that are, how do they call it, ranked competitions, so they award you points and there are prizes, they are not on images. So we explore something on your own to try the methods on another competition on an image that might be something quite useful.

So to find those, you need to scroll to the bottom and click explore all competitions. And yeah, this will let you see closed competitions as well. And you can even see, I guess, here you go, you can find out which were the ones with the most popular of all time.

That can be interesting. Crypto forecasting. Well, of course it would be. That's a bit sad, but there you go. That's interesting. This patent phrase one is super popular. That's good to see. Instant gratification. All right. Thanks all. See you next time. Bye.