Great. All right, so let's at least get introduction. So let's say hello. Hello. I guess probably people know me. I'm Jeremy, co-founder of Fast AI, and we're going to be doing a careful thorough and correct walkthrough of fast AI lessons and more generally how to data science, I guess.

That's the plan. And we will see who ends up being regular guests with us, but at least we have have we have at least two regular guests, hopefully being Radac and Nick. So Radac and Nick, do you want to say hello and introduce yourselves? Sure. Nick, want to go first?

Sure. I can go first, Radac, no worries. Hi, everyone. I'm Nick. I'm friends with Jeremy. I've been involved in machine learning and deep learning over the last, geez, longer than I care to think about, actually. But it's great to be here. And yeah, I'd love to help you out however I can.

And I'm looking forward to getting into it. So nice to meet you all. And so, Nick, your background is mainly trading, I guess. Yeah. Yeah, mainly quant finance. And it'd be fair to say like you're extremely numerate, extremely good coder, but you haven't had as much experience with deep learning using Python.

Like you seem to mainly, from what I've seen, you've used Wolfram quite a lot. Yeah, exactly. Technical background. Yeah, so I suppose I'm a one-trick pony in one sense in machine learning and deep learning in that it's usually been pretty heavily tabular data and financial data and some other bits and pieces along the way.

Yeah, I've always coded in R and less so in Python. It's a transition for me, so I'm learning stuff along the way here too, and especially with coming to these frameworks like fast.ai, like I'm making my way around them just like the rest of you are, but it's fantastic to be here and I'm glad to be on the journey with all of you.

Awesome. Hello, everyone. My name is Radek and super happy to be here. So, I guess I was trying to learn machine learning for quite a while. And then I did a fast.ai course, which was four years ago, and it's changed everything for me and I have been doing fast.ai courses ever since.

Hoping to contribute in whatever way I can. And just because the home of global.ai is clearly in Queensland, Radek and Nick and I all happen to be in Queensland, within about an hour's drive of each other in fact. Absolutely, absolutely, and you know, AI can take you to interesting places, because if it weren't for AI, I wouldn't be able to be here.

That's quite fun. You normally don't expect that to happen when you take an online course, but hey, fast.ai is the Nick in my mind. So, I thought we should start at step 0.1, which is making sure that we've got Python working. And not just working but working well. So, these sessions are not going to be about showing options for things, but showing you the one true way, by which I mean of course the way I do it.

Having said that, I do actually try a lot of things. I am totally addicted to trying out software libraries and tools and configurations or whatever and I've been doing that for 25 years so I have actually more than 25 years. So yeah, you know, I find a lot of people on the forums.

Watch a fast AI course and then go and have this conversation about like how to set something up. Nobody actually seems to just set it up the way I said just do it like this and when I say like no actually do it what I said. And then people are like, oh wow that doesn't actually work.

So, before you get into complicated things you read about in the blog somewhere. Just do this, and you're going to see me do it from scratch. So you'll see it really does work. So one of the things we're going to be doing a lot is using a terminal, which a terminal looks like this, which is not the most exciting looking thing.

You know, I started out, back in, you know, 30 years ago not using a terminal much at all. I was all kind of Excel, VB, whatever. And those are all very useful tools, but the terminal is great for data scientists because, rather than having to have a like a long list of click here, click next, minimize that, click there, drag here.

You can put it in a little script and you just say run the script, you know, you can do things very quickly you can automate things it's like it's also like the difference between me speaking to you with with English language versus me holding up pictures and kind of diagrams or hand signals, you know, language is a really powerful way to communicate with the human and also with a machine.

So there's a reason we're going to be using the terminal. A lot of the things we do, there will be gooey ways of doing it as well, but because this is the opinionated do it my way version. That's what we're going to be spending a lot of time on.

And I want to like assume nothing, except that you like know how to use a computer and you know the basics of Python. That's hard for me because I've been using terminals and Python stuff for a long time. So I'm highly reliant on people asking questions. And I would certainly appreciate it if people ask questions here in zoom directly on audio, if possible.

Otherwise, on YouTube live, don't use the zoom chat because it's a little bit harder for me to like zip in and out of. So yeah, before I get started anything particularly Nick and Radek that you were hoping to get out of these sessions or to share in these sessions I know Radek you were interested in learning a bit more about paper space, I believe.

Yes, yes, that's true. I'm wondering where it fits in my stock as a data scientist. If it has a use. So I'm thinking about having something that I could access in my browser without thinking about starting an instance somewhere and essentially just add a click. This could be helpful, not even for doing the course but just something, something useful to have so definitely looking forward to learning about how we can use paper space.

And of course Radek you're an AI expert in Nvidia so it's not like you have a shortage of GPUs you can access but they're not necessarily set up in as convenient a way as you described for just jumping into a browser and starting work right. That is true and I'm thinking, so I want to use tools in a way that speeds me up, you know, I want to create an environment where I feel comfortable, where I remove the tools from the equation and I can focus on doing the work.

That not only includes work for, you know, like professional work but a big part of being a data scientist is continuing to grow your skills. So if I could be working on something and very often in data science, you know, you still have to wait for something to start or for some calculation to compete.

And in those units of time where you know it's two minutes, three minutes, 10 minutes, I would jump into onto Reddit or you know would essentially waste my time and to my attention. If I would have a window in my browser a tab I could jump and do something useful something exciting like for example, submitting to a Kaggle competition or trying out a new library.

I think that would be a really, really nice outcome. So, yeah, I'm curious if that's, that's something that paper space fast iteration speeds important for sure. By the way, I can hear you okay but you're a little quiet and I don't know if you've got a headset you can use your microphone you can move or volume control you can change or anything.

How about you, Nick. Yeah, I would reflect what Reddick said, straight off the bat there that you want to just have as little friction as possible in your in your tool chain. And I think one of the great things about about doing these sessions is for people who, you know, it can be intimidating I think once you really start to get down into the tool chain when you go from perhaps having tried, you know, your first notebooks on Kaggle or you've been on CoLab or something else, you know, it can be intimidating to actually start getting down in the weeds of your system and getting to really know it and and some things can be so confusing it's like I should be using Tonda should be using member show.

What's the right Python environment all these things. And, and it can be intimidating to find that out. So, like, not always great right. Exactly. And almost everyone is opinionated about how they want to use their tool chain for their thing but I think what I hope that everyone gets out of this is a way of is that there's a canonical fast AI way of doing things and and to be able to have a tool chain as as friction free as possible so that you can get productive with whatever task in your general compute environment.

I think that's fantastic. And yeah, look forward to it. Anybody else listening or on YouTube live want to say what you're hoping to get out of this. Hello, everyone. My name is Ashley. Hi, Ashley. I'm a search on my video. Can you guys see me now. Sure can. Where are you joining from actually.

I'm in Brisbane. Yes. You're working in Brisbane or student. Yes, I'm working as a graduate data scientist at Rio Tinto. And it's so happened that today I don't have any meetings so it's like, this is good. Yeah, and I can totally reflect on what Nick and Radik said about being as about having as little friction as possible because I'm like since I'm a graduate data scientist, I'm just starting my journey as a data scientist so trying to become a data scientist is what how I describe myself.

And most of the times the problem that arises are just conflicts and libraries, and I want to do data science but I still end up doing that lots of that engineering stuff and realize which is good to realize that you're learning you're like learning what a data engineering role or that sort of thing what they do and help us in getting our environment set up, which is good to learn that they do a lot of stuff and I have huge respect for the data engineers at least at Rio now, but also learning that it's a good thing to, you know, what my interest like it was just data science and I would like to do that so that's something I'm looking forward to as well.

Yeah, I mean there's a lot of stuff I think we need to really dig into over the coming weeks you know and it needs to include stuff like you know SQL and, you know, working with CSV files and all this stuff and it's good that all of you have talked about kind of decreasing the friction of data science because it's not just about doing it more quickly but if you have a lower friction, then it's less cognitive overhead for your brain and thinking about the process which means there's more room for your brain to think about solving the problem.

Yeah, absolutely. Okay. Anybody else. Jeremy on. I'm just really enjoying the narrative. And also I'm hoping you'll be able to tune into the environment, but by the end of it so. Where are you joining from. A bit of a trial with paper space but I finally got that going after about what's this five weeks.

And so anyway that set me back a bit but. So I'm really grateful that you're running these these additional issues. That's great. Could you quickly introduce yourself, who are you and where are you joining from me. I'm a QUT in Brisbane, and I'm a late entry PhD student spec coming to the end of that fact I saw you at the SDM.

When you presented there. And it looks like your name is Daniel according to playing in the decomposition of heterogeneous data into a margin of sections and then using that for modeling of simulation. Oh, interesting. Okay. Great. Well, let's, let's, let's make a start then so I'm. Yeah, so please just dive in anytime.

I do recommend if you're planning to join, you know, on a regular basis, the best thing to do is to get a headset with a boom, like I have here and like I see Nick has there and then you don't even have to worry about muting yourself, especially if it's a noise headphone because you can then just chat anytime.

Yeah, and as I said just don't don't feel like you have to wait for the right moment just jump in and treat this like a conversation. Especially if I kind of like do something that you're like, what did Jeremy just do, or I talk about something as if everybody knows about it and you're like, I don't know what that is like, please ask.

So let's start. So step one to working with the terminal is is running a terminal on your computer. So, I guess, you know, people here hopefully either have the Mac or Windows or Linux, if you have a Chromebook things are a bit difficult. And so I would suggest trying not to use a Chromebook.

Right. If you're on a Mac to start your terminal just if you Google for Mac OS terminal Apple will tell you what to do. You can use command space type terminal you can click launchpad type terminal or in the finder. You'll see lots of conversations online about all the different terminals you can download and use as you'll see it really doesn't matter.

Okay, so just use the one you've got. If you're on Linux, you probably already know how to use the terminal. If you're on Windows. Windows now has a terminal as well. So if you just go to Microsoft Store and type terminal, it'll download that for free. Okay, so what's a terminal terminal is strictly speaking, you know, it's a program looking like this that knows how to display a console window for a shell window.

Now the thing inside it with a flashing cursor and these words and stuff is not strictly part of the terminal, right? That's something called a shell. So this thing called the Windows terminal can start a number of different shells. It can start a shell called PowerShell and so forth.

So there's two different things. There's a thing called the shell. There's a thing called the terminal. We often use them a little bit interchangeably and that's fine. But the first time you start the Windows terminal, it's actually going to start something called PowerShell, which looks like this. So if you're running Windows, the next thing you need to do is type WSL install, which stands for Windows Subsystem for Linux.

And nowadays in Windows, it's a single line. You just type WSL dash dash install and it says do that in administrator PowerShell. So if you type start PowerShell run as administrator, right? And then you can copy and paste that into the window and then reboot when it's done. And then you'll have actually an entire copy of a Linux distribution called Ubuntu running inside Windows.

And that's what I recommend for Windows users. OK, so once you've done that, you should now find that you can go click here and go Ubuntu rather than PowerShell. And one of the things I like to do is to get all of my tools working so that they do what I want kind of automatically each time.

So I don't use PowerShell very much. I mainly use Ubuntu as I expect you will be, especially for this course. So if you go into settings, you can choose the default profile as being Ubuntu. And then next time you open it, it will do Ubuntu by default. OK, so first time you run it, it'll ask you to choose a username and a password.

That's because this is a whole separate operating system. So I've actually set up a new user. Totally empty. So I've got an empty. I'm going to start with a totally empty kind of virtual machine here. OK. And then, you know, the other thing I think is a good idea is to kind of give your terminal a full screen.

So on Windows, for example, if you hit Windows tab, you have the ability to kind of move things around. And if you hit alt enter, you'll get a full screen. So now I've got lots of room for working with my terminal. So I'm not sure for whatever terminal you use on Linux or Mac, what the equivalent keyboard shortcuts are, but Google for keyboard shortcut Mac terminal full screen or something and virtual.

These different these different like separate kind of virtual windows are called virtual desktops. Very good idea to learn to use virtual desktops. OK. So this is here we are in a terminal and we're going to we're going to set up Python. Now, the first thing to do in your terminal is to type Python and this is bad.

I typed Python and something happened. I didn't want something to happen because we actually want to install our own version of Python. And so if you type something happens, that means you've already got a version and we definitely don't want to use it. And the reason for that is that the version that you're going to see by default is going to be what's called the system Python.

It's the Python that your computer uses for like running stuff. We want to put in a separate version of Python. So when I type a command like Python, what it's actually doing is it's finding this somewhere on your hard drive and running that program and you actually want to know which one is it running.

And so you can type which followed by the thing that you want to run Python and it's going to tell you this is running from something called slash user slash bin slash Python. Now, on the other hand, the user I created, which is called JPH 00, if I want to find out where I am right now, I can type print working directory.

So there's this idea of like directories, which hopefully you're kind of somewhat familiar with from using like, you know, the finder or explorer or whatever. So on your computer, you know, there's some directory that you're in in finder or explorer and that you're seeing the contents of it and you can kind of move up and down and move around and there's kind of a concept of a current directory you're in.

So in the terminal, it's exactly the same. There's a concept of what directory I'm in. And if I type PWD it tells you what directory you're in. So currently I'm in a directory called slash home slash JPH 00. So on all of the operating systems we're talking about today, I believe there's the same idea.

Actually, I don't know if this is true on Mac. There's an idea of a home. Radec or, Nick, are you guys Mac users? If you go into terminal and type PWD, do you end up in some kind of home directory? Yeah, you do by default in Mac. Yeah. Okay.

And that'll be like slash home slash Nick or whatever. Correct. Cool. Yeah, home slash user system username. Great. So all the same. Terrific. So generally speaking, we want to install everything into our home directory. We don't want to be using any versions of Python or anything that are in anywhere else such as this system directory.

So let's go ahead and install our own version of Python we can use. And there is one and only one correct answer at the moment, in my opinion, to what should I install, which is to install some kind of anaconda distribution. So Python is an open source program. Lots of different people package it up for your computer.

Which, which Python you install is going to make a huge difference to how easy everything is. Yeah, may ask a question, please. Yeah, are you using Windows 11 or Windows 10. I'm using Windows 11. I think they're almost exactly the same. I actually don't like it quite as much as Windows 10, to be honest, we're about to force everybody to upgrade anyways.

I just bit the bullet. So tell me again what's the difference with the WSL install, I guess, on the 10 you have to specify WSL two. I would suggest a good reason for installing. Yeah, I do think that Windows 11 that the WSL story is a little easier. So that's probably one reason to upgrade.

Also, if you do have a Nvidia GPU in your laptop like I do, it's much easier to use it in WSL. If you use Windows 11. Okay, so the particular distribution I strongly recommend using is called mini forge. Sorry, not actually we're not gonna use mini forge, we're gonna use mammer forge, mammer forge, I always get this confused, mammer forge.

So let's go ahead and install it. So, if we scroll down, we'll find installers for mini forge, keep scrolling down, mini forge pipo 3 is clicked on here, mammer forge. Okay. Oops, so I'm going to try to remember to put stuff in here as we go. mammer forge installer.

Save. All right. So, let's do everything through the terminal. I think it makes life easier. So not pipo 3 just mammer forge plain mammer forge. So we'll find. So I know I'm on Windows but I'm using WSL so that means I'm effectively on Linux, and I've got most computers AMD 64 unless you're on a Mac.

So right click on this and I'm going to choose copy address. Okay, let's go back to our terminal. So in our terminal, you know, just like in find or explore it's good to keep things nicely organized so let's create a directory to put things we download into. So to create a directory, we type mukter make a directory, and then we'll call it downloads.

So if I now list the directory that shows me or LS shows me all of the things that are in this directory. And so I've now got something called downloads. So to make that the current directory I change directory CD into it. Now, typing downloads takes far too long.

So instead I'll just type the first couple of letters and then hit the tab key on my keyboard, and it fills it out for me. So the tab key is very helpful key to use in in your shell in your terminal. So we need somewhere in the terminal of downloading a file, and the main one I recommend using is called w get.

So W get gets a URL and make sure you know how to paste things into your terminal. So I'm trying to remember I think there's some things you can figure around with. Or maybe it's just a default. So in, I think maybe it's a default in in WSL in Windows terminal you hit Ctrl shift V to paste.

So there's the thing that I just copied URL. So if I hit enter now. It's going to go ahead and download that for me. There we go. Okay. Now, with data science, you spend a lot of time waiting for things. And so don't spend time waiting for things like downloads do something else at the same time.

So you'll see here there's a little plus sign that lets me create another window, or I can use control shift one now I strongly recommend using keyboard shortcuts, wherever possible. So if I hit control shift one. I've now got a second terminal running and I can control tab between them.

And so now we can keep doing things where that's happening. It's going to switch to my fresh new user. Okay, so, so while that's happening, let's talk about if you've tried to install Python before and you've got a million different versions of things that can get really confusing. I strongly recommend trying to clear everything away and starting from scratch.

So the first thing is if you're in your home directory. So if you see this little tilde here that means I'm in my home directory. So although my working directory is slash home slash JPH whoa on Linux and on Mac most shells, the shortcut for your home directory is just to call it to order.

So if I CD into downloads change directory into downloads. It says you are now in tilde slash downloads says a shorthand way of saying slash homes as JPH 00 slash downloads. Okay, if you've got any other anaconda many condo many forge, whatever here right now. Delete it. And so let's say you've got a mini condo, for example.

So to delete a directory and all of its contents. Well, first of all, you could just move it out of the way. If you're worried you might lose something important. So to move something you type MV for move, move any condor to call it like many conduct old say and that now is going to get out of the way it's not going to interfere with anything so either remove or move out of the way any existing installations.

The other thing you want to make sure is you don't have any. We don't have voice. Sorry. We had interruption, I think. Oh, you lost me for a bit. Can you hear me now. Yes, it's working now. Okay, how long did you lose me for. Five seconds. Okay, so as well as making sure that you don't have any other copies of anaconda or many conduct or many forge lying around.

You also want to make sure you don't have any copies of Jupiter installed or anything like that installed in your system Python. So if you type I Python, you want nothing to happen that this is what you want. Or if you type Jupiter. You want nothing to happen. Okay, if something is happening.

Right, then try typing pip uninstall. For example, I Python and Jupiter. Like try to uninstall all the copies of this stuff. You don't want them in your system Python it's going to make things very very confusing. If you can't uninstall things no matter how hard you try, then find out where that is.

So for example type which Jupiter, right, and if you've got it installed it'll tell you since slash user slash bin slash Jupiter or whatever, and delete it. So to delete things to delete a directory and all of its contents. This is obviously very dangerous to be careful, you don't delete something you don't want to RM for remove minus R means recursively so all folders and files underneath it.

F for force so do this without asking many kind of old for example. And you could do something similar if you had a slash user slash bin slash Jupiter for example you could type RM slash user slash bin slash Jupiter, and that would get rid of it. So you wanted to be at a point where, yeah, you don't have I Python, you don't have Jupiter, and you've downloaded.

You don't have any any kind of directories in your home directory, and you've downloaded member forage. Alright, any questions or comments. So far. If you're kind of trying to follow along and something weird happens feel free to let us know you're welcome to share a screen, you know, debugging will help you know me.

Yeah, you can see me hopefully you can. I'm Rob I'm down in Hobart. Hi Robin Hobart. I love it. It's a great place. It's just got above 11 degrees outside. I'm going to play a royal tennis if you have been. Oh yeah, yeah I know the tennis club. Yeah, beautiful.

Anyway, I work for the bureau meteorology. There's a bit of background but the other guy that did the clouds detective. Yeah, that's me. Yeah, yeah. Yeah, that was fun. I was a quick question. Would you mind talking a little bit about Conda versus mamba. Yeah, we're going to do that now so absolutely.

I'm happy sorry please anytime jump in. Alright, so let's just get this installed first of all so we've got okay so we've learned that LS lists the directory that we're in right now, and LS minus L gives us like alpha long form gives you more information about each thing.

And so long form. It tells me how big it is now that's quite hard to read so another really helpful these are called flags really helpful flag is minus L minus h h is for human readable 89 big. And when you pass multiple flags to a program like this you don't actually have to put them each with a dash you can just put them next to each other.

So minus l h is the same as minus l minus h. So this is list long form human readable so we just downloaded an 89 megabyte file. And this is a dot sh file a dot sh file is a script. It's something that it's basically a shell script it's a list of shell commands like LS or CD or which that are listed in order one after another in a text file.

So you can look at a text file by typing less. And then the name of the file to look at now I'm going to type capital M a I'm not going to type all this out because I'm lazy I just hit tab. And then I'll press enter. And so you can see this is what a script looks like anything starting with a hash sign is a comment so it doesn't do anything.

And then everything else is literally the shells is going to go through each of these lines one at a time and run them in your terminal. And so this is the thing that installs Bamber Forge. So to run a script there's a few ways to do it. The best way to do it is to run it.

Well there's a few ways to do it. I think the way they suggest in the docs is that you type bash space and then there's such explain what's happening here. I mentioned that inside the terminal the thing that's like printing out this information and listening to my commands and doing stuff like LS is not the terminal itself, but something called a shell.

And a shell is just a program that somebody wrote to let me type commands and get answers and stuff like that. The shell that you bun to users by default is called bash and the shell that Linux most Linux is used by default is also called bash the shell that Mac uses by default is called zsh.

They're so similar. My suggestion is just don't worry about it. Like you'll find basically almost everything. I think everything we see today if you use a Mac will work just as well with bash. Max also come with bash. So it's like a pretty safe thing to run scripts with.

So if you type bash space and then that it says please run this shell script using this shell. So it's going to run this as a bash script. So if I press enter. OK so this is now running this program called Man of Man before Linux dot sh. So please review the license agreement.

Of course. Do I accept the license terms. I've read them very carefully. OK. And it's going to get installed for you. So I'll just press enter and you'll see here it's going to install it into my home directory to something called member Forge. Away it goes. OK. So while that's happening let's describe what's going on here.

So this is installing not just Python but a whole bunch of libraries. These are all the libraries that it's installing. OK so it's finished. That was quick. Do you want to this to initialize it. So the answer is yes we do. And what that means is each time in the future that we open our terminal.

It's going to automatically set up our Python environment for us. So type yes. There we go. So as it says for changes to take effect close and reopen your shell. So let's go ahead and do that. Jeremy I have a quick question. Yeah. I'm using a Mac and I went through lesson three and I think you had covered something about installing some of the software of including and then getting to the notebook.

I'm a bit confused. So you been to doesn't apply to Mac does it are everything described so far will work perfectly on that. Yes. But can should I be using the terminal directly. And I do see that. Yes I suggest doing everything exactly as I'm showing it. Although the Mac version of the install let's see OS X the Mac installer might be not a shell script.

No it is a shell script. Yeah. So it looks like it should work. Yeah. I think I can confirm it is. Great. Great. Yeah. So that should all work fine. If it doesn't let us know. But yeah sure. All the steps should be identical. Great. Thanks. All right. Okay so I just closed and reopened my shell.

So one way to do that is just to close your terminal and open the terminal again and my terminal now looks slightly differently. It looks good. It's got the word base before it. And this is the way we can tell that we are. We have our Python environment running we're inside it, if you like.

So we're all set up to use Python. So if I now type which Python it doesn't say slash user slash bin slash Python anymore. It actually shows me I'm in my home directory. There's a man before its directory and there's a bin slash Python there. This is what we want.

Okay. You know you've set this up correctly. If you type which Python and you get back an answer that's inside your home directory. There's a number of reasons for this but really a key one as I say is that the main alternative is that you'll be using your system Python that's the Python that your computer uses for like running stuff in your operating system and that's causes all kinds of confusion with between what your computer is trying to do and what you're trying to do.

It's also super easy when you're using the one in your home directory because if anything ever goes wrong. Like just RM minus RF member forage and that deletes everything and this run it again and you're up and running and you want to get to a point where you can do that.

Incredibly easily. So let me now describe. Okay, so what I'm now going to do is I'm going to I'm going to show you I'm going to start from scratch. Right. So currently I got that all set up there's my man before directory created so I'm going to go RM minus RF member forage.

There we go. I'm going to go to downloads and I'm going to even remove the installer we downloaded. Okay. And I got to show you how we can do all the same steps, but using a script. Oh, and also let's delete the thing that it that it added. Remember, I said, Do you want to automatically initialize member forage when you start your terminal.

Well, what it actually did was it edited a secret hidden file. Normally LS doesn't show you secret hidden files but if you add the minus a for all it does. And so you can see now there's a whole bunch of files that we didn't see before anything starting with a dot is by default hidden in the shell.

And this one here called bash RC, which if you're using a Mac will be called dot zsh RC is a script where all the things in that are run automatically. Every time you run your terminal. So if we edit that script. You can see at the end of it, they've the installer is set up stuff to initialize condo.

So I'm just going to delete all that so that we're really back to scratch. Okay, so if I now close and reopen my shell. You can see it no longer says base right so we've totally started again. So I don't like to do anything. I'm going to do more than a couple of times.

I don't like to do manually. So I'd like to automate this whole thing. So, which I have. There's a repo called fast setup. Which Radek knows well because he's helped set up some of this, which contains all the things I use to like very rapidly set up everything on a on a new computer or a new server or whatever.

And there's a thing here called setup Conda. And I just wanted to ask, can you hear me a little bit better now, I switched a different microphone. Oh, so much wonderful, wonderful. So, now that you can hear me if I could maybe make one tiny comment, please, please. So, with the comments like we've seen LS dash minus a or there was another one and the dash minus RF.

The flags, they often have the same unique for different programs. They have mv minus R that's recursive. I don't think you actually need that for me, but, for example, with the LS, the minus h flag, you will see the cross a bunch of other comments that the minus h will be human readable.

And it might sound a little bit overwhelming to see these flags that maybe don't seem to make that much meaning. But since you will be using them across many commands and then there's also a trick to very easy to list the flags for command. You know, it actually gets quite straightforward.

Yeah, that's a great point. Many programs have a special flag minus minus help, which will list all the flags, for example. They can be a bit overwhelming because there's a lot of flags but that doesn't mean you have to know them all. I mentioned something else which is like, you know, we're covering a lot of stuff pretty quickly and so people who haven't used the terminal before, there's going to be a lot of new information here.

Obviously, it's a good idea to watch the video afterwards, but something that would I think help a lot of people in the community would be if there are bits that you do read it once again afterwards because you want to learn that bit is if you write it down.

Like just write down what, you know, this is what Jeremy said and here's a link to the thing he was talking about, chuck it into a blog post that would like help a lot of people, because, you know, watching a video is takes more time than grabbing the best of bits out of a out of a blog post so that would be something if you're interested in both giving back to the community and documenting it for yourself.

That would be amazing. And I guess something we should probably talk about in more detail at some point is how to write a blog post. Sharing things like the blog post or tweets or on the forums. It has a lot of nice benefits. It's not only that you're helping others, but you're also remembering the stuff better or understanding the things better.

And I think that's something in words, you know, I often find myself writing something out and I think I understand it, but it's only at the point of putting it in words that I identify gaps in my understanding. It's definitely great for remembering things. The fact that I still have a hard time believing that it is a real thing, but it is.

You will write something and you will share it and then somebody else will read it and they will say, hmm, that's useful. This person maybe knows about what they're talking about and suddenly people will reach out to you asking you to collaborate or maybe even, you know, you might need to professional opportunities.

So it's something that is very far away from what we normally imagine that is possible, but apparently it is a very real thing. Well, it happened to you for sure. And lots of people in our community. When I look at the most, the biggest contributors to our community. All of them started out without having a job in AI and pretty much all of them have jobs or research positions in AI now.

So probably not a coincidence. Now this thing we're looking at right now is a Git repo in GitHub. We'll talk more about Git a lot, but for now it's enough to know that GitHub is a place that has lots and lots of coding libraries and coding tools that you can download, edit, play with, explore, and so forth.

So this is a very, very, very useful tool to get extremely familiar with. In this case, we want just one file from it, which is the file called setup condor.sh, which is a list of steps necessary to install member forage on your computer. As you can see, it's just 28 lines.

It's very short. So how do you download a single file from GitHub? The way, in my opinion, is just to click on the file, click on raw, because you see at the moment it's got all this stuff around it. We just want the file itself. Click on raw. And now there's just the script, right?

And now if I click on the URL and copy it, I can now go back here, and who remembers what we type? wget is the way we download something. So if I type wget setup condor, then that's now created a file called setup condor on my computer. And if I want to look at it, remember we type less.

And there we can see there's the file. So we want to run this script. And you can see a bash script, a shell is a little mini programming language. So it's got a case statement. So the case, I mean, we can look at what it's going to do, right?

It's going to check the operating system type. If it's Darwin, that's a Mac. It's going to check what kind of CPU you have, and it will download the appropriate member forage for your environment. And then it's going to check what shell you're using, and then it's going to set up bash or zsh_fish is appropriate.

So echo means printout, sort of printout downloading installer, it'll download it, cowl is just like wget, and it'll run that bash thing we just saw, and it'll initialize it. There's a handy flag for the member forage installer -b, which is something that causes it to not ask you to check the license and stuff like that, it all happens automatically.

So if I now do exactly what I did before to run my script, I could type bash setup condor.sh, that would work. Another thing we can do is the very first file here, hash exclamation mark, blah, blah, blah, blah. This tells our computer how to run this script that says to run it with bash.

So we actually don't need to say bash, then that we can just run it directly. So we could try doing that, setup bash condor.sh, with a dot slash, this means in the current directory. And that doesn't work. And the reason that doesn't work is because by default things you download from the internet, you can't just run.

Your computer assumes that, you know, things are dangerous unless you specify otherwise. So we need to tell it that this is something where I want to give it permission to execute this as a script. So if I type ls -la, actually just ls -a would be fine. No, ls -l.

Oh, here's a trick. I want to repeat the last command, but I want to change the a to an l. So just press up arrow to get back the last command. Backspace, make that an l. So this thing over here we skipped over before, this is what's called the permissions.

This tells you what you're allowed to do with a file. An r says you're allowed to read it. A w says you're allowed to write to it. And this dash here, if that was an x, it means you're allowed to execute it. So to change the permissions of a file.

So we want to say that the user is allowed to execute it. We use a program called chmod to type chmod and you say, OK, who do you want to change the permissions for? And I say this user, that's a w. What permissions do you want to change? I want to add the executable permission.

So you say plus x. And then you type the name of the program that you want to change the permissions for. So press tab. And so now if I go ls -l, I'll press up arrow twice and then hit enter. There's that x we wanted. OK, so now I should be able to rerun dot slash set up Conda.

So press up a few times to find that there it is and hit enter. And so that's all going to go ahead and download and set that up for us. OK, so while that's happening, does anybody that will take a minute or two, does anybody have any questions or comments up to this point?

Hi Jeremy, I had a doubt which version of Python are we using right now? And in case of some other kind of experimentations, if we want to switch Python versions, do we have any recommendations on that? So this is just going to use whatever the default is. And generally speaking, the default will be the most recent release.

So we can check which version it's going to be by looking at MambaForge and say which version. The latest installers have Python 3.9. So I just use whatever is like the default because that's going to be most likely what most other people are using. There is actually a Python 3.10 now, but the fact that they're making the default 3.9 suggests that folks are thinking that's probably the most reliable thing to use.

So that's what I would also recommend using. That's a nice thing about MambaForge and I call that it allows you to create environments very easy. So if you just want to experiment with a different Python version or with some different sets of packages, then it's just one command in your terminal that you have to execute and you're transported to a new environment, which is very helpful.

Yeah, although I would recommend not using different versions of Python if you can avoid it. Sometimes you need to because you need to check compatibility like a lot of workplaces use really old versions. There's actually if you look up Python EOL for end of life, there's a list of like when each version of Python became end of life, which means you shouldn't be using it anymore.

So 3.6 is now officially ended security support. So if your workplace is using 3.6 or older, you should be strongly encouraging them to upgrade because it's insecure. And so currently 3.7 to 3.10 are all supported versions of Python. I do tend to avoid using the very most recent version for probably a year after it's released because I find it takes a while for everybody to get compatible.

So 3.10 in particular introduced some nasty incompatibilities, which I try to make sure fast AI works with them with all the versions, as long as PyTorch does, but yeah, I'd say the penultimate versions are good one to choose. And that's the default. Okay, so look at that we just ran a single command and it downloaded and installed.

Okay, I have run into an error, saying, may I share my screen if that's all right? Yes, of course. Absolutely. Do I need to do anything? Yeah, you might need to give me permission to share. So I'll stop sharing. And then. Okay, got it. Thank you. Okay, so I just have to stop.

No worries. Great. This is great. Our first debugging session. Here we go. Yeah. So this is the error, I think. Expected environment not found at prefix aborting. Interesting. Okay, so, did you do the manual version first and then now you're doing the script version? That's right. But I removed the through RM and RF.

Oh, I see the problem. Do you see how you've got base in brackets before your... Yes. Okay, that means you're, you're inside Conda right now. So you've removed it but then you didn't close and reopen your shell. Oh, I see. So let's start again so close your, close your shell.

And then reopen it. Okay now so the next thing you'll see here is it's complaining. No search file or directory. Oh, that's something else. Never mind about that. Okay, great. So, oh, now, yeah, good. So, that's something that actually just did is she typed CD with nothing after it, and that put her in her home directory.

So you've been to on wsl is a bit weird. Most things put you there by default this one doesn't. Okay, great. So that should run the setup quantify now. So I would do an IRA minus IRF member forge at this point. And then, yeah, and then dot slash set up hundred.sh and then, yeah, come back.

That's it. Good. Nice. All right, then come back to us if that causes will do. All right. Thank you so much. My pleasure. And the Python versions we discussed before. It is very rare, but it happens that you might encounter some code on GitHub, you will want to use your project, and it will use, it will require some Python version, you know, that's much earlier from what you're using.

And in the environment that will be using Jupyter notebook, there is actually a way to run a specific version of Python for a for a given notebook so okay should we add that to our list of things to talk about them that sounds useful. I put that here things to cover.

Sure, running. I mean, it's, it's useful in the sense that you don't have to switch environments while you're working on something and usually that would be part of your workflow that you want to do something to your data. Right. Thanks for that. Great. Okay, so I have to close and read my terminal.

Done. And so again we've got that base there. Okay. And we'll also double check with which Python. Okay. So, with Python installed. We can run Python scripts, which isn't that useful of itself unless I mean because we want to be able to like, you know, have some environment we can we can work in, particularly in particular we want Jupyter.

So we can install. We can install stuff into our Python environment, but actually this Python environment it's more than just Python we can actually install just any program we like, in fact, into into here so basically what Mamba Forge and Conda and so forth do is they basically create a whole kind of directory of programs and libraries for you in your home directory so it's not just for setting up Python, but for installing literally any program you want.

So you'll find often when you install something you don't just end up with a Python library but there's actually programs that you can run like Jupyter, for example. Jeremy just very quickly just as a quick interruption. I think maybe some people might find it helpful even even I find it helpful still try and understand what I guess what it disambiguates Mamba from Conda from using pip, why would you choose one over the other.

So now, exactly. Okay, cool. Perfect. So, so how do you install software into your Python environment. Well, let's start just by talking about how do you install Python libraries into your Conda environment. And Nick just mentioned the three options you have which is Conda, pip, and Mamba you'll come across all of these three things.

Let's just focus first of all on Conda and Mamba. So, Conda and Mamba are literally two ways of doing the same thing. And they are fully compatible with each other. Mamba is a faster version of Conda. So why would you use Conda. Basically, you wouldn't. But Mamba is very new.

So the vast majority of documentation you'll find on the internet users Conda. Anytime you see something that says Conda blah, you can type Mamba blah, and generally speaking, that's going to happen much faster, at least if the blah is install. So, let's look at an example. If I want to, so a really helpful thing for playing around with Python at the command line is I Python that's interactive Python, which we'll play with now I don't have a Python doesn't exist.

Right. Now when I try to run it, it says, Oh, did you mean some of these other things. This is, this is Ubuntu talking to me. I don't want. Make sure you never install any Python stuff through through brew on Mac or apt on Ubuntu or whatever. That's just for the system Python only use Conda Mamba or PIP.

The vast majority of the time you're going to be using Mamba. So if I type Mamba. If I just type Mamba, then it tells me okay well what do you want to do. In fact, it even says Conda is a tool for blah blah blah so you can see Mamba and Conda really get mixed up right.

These are things you can do most of the time after typing Mamba you're going to be typing install. So, Mamba install, what do you want to install, I Python. And so what that's going to do is go away and check its index to see if anybody has uploaded something called I Python.

And if they have it's then going to see what are all of the things that I Python depends on. And so it's now going to check with me. It's going to say okay for in order to use I Python, you got to install all these other things as well.

Is that okay. Let's say yeah, that's absolutely fine. So that goes ahead and downloads those things. And it's awesome. And now I can type by Python. And there we go. So this is a Python REPL, a REPL is a read eval print loop, I can type things. And it will tell me the answers.

Okay. So this is like a good way for quickly experimenting with stuff in Python on the command line. I don't use it very much because I mainly use Jupyter. But, you know, sometimes you're really quick and dirty stuff can be helpful. If you've read Wes McKinney's fantastic pandas and so forth book, Python for Data Analysis, he does most things in I Python.

To close I Python, you close it in the same way as the fastest way also to close a bash or zsh shell, which is to hold down Ctrl and press D. And that exits. Okay, so Ctrl D is a nice fast way of closing probably most programs at your terminal.

Okay, so that's how you install stuff. So let's install PyTorch, shall we? So PyTorch install. PyTorch, probably get started I guess. There we are, start locally. All right, get the latest version. It's choosing Windows, but I'm actually not on Windows, like I'm not WSL is not considered Windows, it's considered Linux.

So, Linux, Conda, Python. And then there's various platforms you can choose. So if you've got an NVIDIA GPU, which you probably don't, you would choose CUDA. And if you don't, you would choose CPU. I actually do have an NVIDIA GPU on my laptop, but just to keep things simple.

I'm going to do this and you can see by default it says Conda install. So remember, it's going to be faster if I use Mamba install. So I'm going to copy everything except the Conda copy. And then over here I'll type Mamba and then Ctrl+Shift+V to paste and enter.

There we go. Now, you'll see here it's going to install. It's going to install PyTorch, Torch Vision, which is the computer vision bit of PyTorch, Torch Audio, the audio bit of PyTorch, just for the CPU. And this minus C is important. This is called a channel, a PyTorch channel.

When people upload stuff that you can install from Mamba, it can kind of be installed to the main default repository that people grab stuff from, which in the case of the Mamba Forge package we're using is called Conda Forge, or they can upload it to their own channel. So PyTorch and FastAI and NVIDIA, we all upload things to our own channel.

So to get stuff from the PyTorch channel, you say minus C PyTorch. That says, please search in PyTorch's channel for this software. OK, so it's now downloading all the packages that we asked for and also decompressing them. And so this is basically a list of all of the stuff that it figured out was needed to install PyTorch.

Jeremy, quick question. Let's say that I'm like I just come along and I type in Mamba install PyTorch or something similar, or even if I'm reading through a blog post or a tutorial and I accidentally type in pip install xyz. Have I created a cardinal scene for which there is no return?

You mean if you've pip install something rather than Mamba installing the thing? Correct, yeah. Yeah. So if you pip install PyTorch, like that is an option here, pip. The thing is, though, that they've got some extra things here to make sure that you get the version you want. Right.

Mambas, so for one thing, if you've got a graphics card, an NVIDIA graphics card, pip installing won't work unless you do a lot of futzing around with installing NVIDIA's CUDA software development kit, which you don't want to do that. It's annoying, right? If you use Condor or Mamba to install it, it installs those libraries automatically for you.

Even if you're using Windows, your NVIDIA GPU will still work. So, yeah, in general, installing with Condor or Mamba is going to more reliably give you the kind of correct setup of everything without you futzing around. Because Condor and Mamba are able to install what's called binary dependencies, so it installs all the libraries you need.

If you've accidentally pip installed something, you can just type pip uninstall, the same thing. Problem is, though, that pip's going to have installed also a bunch of dependencies. Again, they came from pip rather than Mamba, and again, they might not be the right versions that are optimized for your machine.

So, yeah, you know, I would be inclined to delete my Mamba Forge directory and start again, particularly if I did that for PyTorch, because it's just really important to have a good PyTorch installation. So that would be certainly my recommendation. I feel a bit the same way when we do Jupyter.

If you accidentally pip install Jupyter, you know, I'd just be inclined to start again. But if you always type Mamba or Condor instead of pip, just make that your habit, then you won't get out of that situation. That's helpful. Yeah, I mean. Can you hear me? Yes. So, for certain things I believe was Tim, there is no way to install it with Condor.

I didn't see it. There is, we will get to that, don't worry. But for now, my channel for fastchan will let you do that. Can I just type in or just make a comment? With pip, now, if I understand correctly, I think like if you just do pip, when you like set up a con environment, that pip is still going to be, I think, for like the system, right?

No, no, no, it's not. It'll be fine. In fact, let's check, shall we? Alright, so which pip? No, we're all good. So it should be a problem. No, but like if you were to create a new environment, right, then, because right now you're from the base, right? So if you create your own environment.

Well, we haven't got good environments yet, so I wouldn't worry too much. No worries. I mean, did you have a question or comment? It was the same as already discussed. Okay, so now that we've like installed this stuff, how do we know if it's working? So if I hit up arrow, I can find my ipython, and I could just try importing it.

It is, right? So I can now type torch. and if I hit tab in ipython, it actually gives me a nice list of things. So for example, torch.tensor. And you can see it's going to give me the arguments. There we go. So that's working. So remember, control D. So yeah, another thing we probably want to install is Jupyter.

So there's a couple of different versions of Jupyter. There's Jupyter lab, there's Jupyter notebook. So we'll probably use lab. Because we're using memberforge, this condorforge channel, we're going to be using automatically, right? So we don't need this -c condorforge. So we can just type member, install Jupyter lab. Okay, so that's got a lot of stuff.

It's a big piece of software. But that's okay. Doesn't take too long. There we go. It's finished. Okay, so now we're ready to create our first notebook. So it's nice to keep things organized. So let's create a directory for our notebooks. So I'll make a directory called nbs and change directory to nbs.

So that's empty. Okay. And so now we can type Jupyter lab. And there it is. Okay, so it got a little bit confused here in my version because WSL tried to open my browser. But my browser is a Windows application, not a Ubuntu application. So it tried to open it up and said there was an error.

Now it's actually fine. I can still use this. So you can see here, if I click, if these are hyperlinks, I can control click on it. And it opens it up. There it is. Right. If I don't want it to complain like that, you can say don't open a browser by typing no dash browser.

And that's it. Okay. And then click. Control click actually. Yes. Alan. Hello Alan. Hi everyone. I'm from Brisbane as well. I'm currently getting over COVID. How are you feeling? Apparently I'm loud out of isolation today. I don't feel that great yet. Sorry mate. Much better though. So that's fine.

You know, the most important thing to know about for COVID is to take it easy on exercise until you're fully recovered because it's actually makes long COVID higher probability if you exercise when you've still got any symptoms. So take it easy. That's good. Well, I'm going to go for a walk later, just to the shops.

A nice one. Just I was going to say with that JupyterLab and then no browser, this is one of the ones where I discovered the aliases were really my friend. I was about to show you that. Yes. Yeah, Alan. And why I got it. Yeah, so yeah, exactly. So I have.

Let's see, what do I got here? I actually have even an alias for alias. So for things that are kind of long and annoying to type, you can create an alias. So there's a few things we could do to not have to type JupyterLab no browser. One would be to hit up arrow.

Now, one nice thing about up arrow is if I close my shell and go back into it, up arrow still works. And by the way, none of this stuff is magic, right? If you if you're wondering, why does that still work? It's because bash creates a file called dot bash history, which is actually really useful to know about because everything you've ever done is listed in that file.

So here's all the things we do. This is another good reason to use the terminal is you can always go back and repeat things. This is actually how I create scripts. I do everything interactively. And then I copy and paste stuff from my bash history into a text file that creates a script.

So if I hit up arrow, I can get my JupyterLab on a browser. A very, very, very helpful thing that works in Bash and ZSH is the control R keyboard shortcut. If I type control R, it says reverse search. And then I type JU for Jupyter. That will show me the last thing I type with those letters in.

OK, and if that's not the right one, just hit control R again and it'll keep searching. So that's very useful. Jamie, is there any way to clean this is sort of clear. This is I assume that after a while it becomes such a long file. It's very difficult to make a script out of it.

It's just it's just text. I think it automatically like cleans itself. But yeah, just leave it. You can always delete it if you want to or you can even edit it with an editor. But it's kind of nice to have all your history there, in my opinion. No, another thing you can do.

Sorry. I think by default it's source 1009 or something like that, but you can set it to something more. Yeah, that's right. I think there's some environment variables. And then one other one, which is kind of interesting is exclamation mark followed by some letters runs the last thing that starts with those letters exclamation mark JU will run Jupyter lab no browser.

Something that you'll see quite often is exclamation mark exclamation mark and that means just rerun the last command. As you see. And you can put that anywhere, by the way. So for example, if I type echo dollar dollar that filtered out as echo Jupyter lab no browser. So there's a little trick for you as well.

OK, so Alan mentioned aliases. I could type alias Jupyter lab JL equals. And then I could put. This is going to work. Let's try it. Yep. Okay, so alias JL equals echo Jupyter lab. I don't want the echo there. Never mind. So let's just delete it. Oh, okay. When I am editing this line, there's a few shortcut keys to know about to go to the end of a line type control e.

To go to the start of a line type control a. And then to go forward and backward by a word at a time, press alt left and right, which radical Nick is that option left and right on a Mac. To move by a word. I had it for a month, but couldn't get used to it.

Nick, are you on a Mac? I'm using one as we speak right now, because it's better for running zoom on that. Yeah, you can do things like that. Although I'm used to all the Emacs key bindings, but that's a rabbit hole. I won't subject people to it. So now I've got this alias.

I can just type JL. And it does the same thing. Now, the problem is that's not going to appear automatically next time I run my terminal. But remember, I told you about that special hidden thing called dot bash I see that automatically runs. So if we put it in there.

Then it will automatically run. So if I close my terminal and reopen it. There's JL. Beautiful. Now you'll notice that when I edit files in the terminal, I tend to use a program called VIM, which as Nick well knows is obviously much better than the competition called Emacs. But you don't have to use either of these.

You can use whatever you like. And in fact, on a Mac, if you type open that open dot, it'll open the finder at this directory or on Windows type explorer dot xe dot. Right. Oh, why doesn't that working? That's weird. Okay, that's never happened before. All right. So, amazing.

Must be some recent change to WSL, which I will have to look into find out why explorer dot x is not working. I think you made a typo in the spelling explorer. What did I do wrong? I think you typed it. Okay, you added an extra period. The period at the end is intentional.

That's to say open it in this directory. And I tried to have complete and complete didn't work. So, is it because they've changed it to it to work on my end. No, it's not works like your user, maybe you need to go to other user. Yeah, I tried it and another user.

I think. Nevermind. Come back to that another day. I wonder if like. Yeah, what was he changed something. It seems to work for me. Yeah, I don't know if there's never not worked before. Okay, we'll figure this out and we'll cover it in the next session. Okay. Yes, this is amazing.

And one of the things I like particularly about it. WSL like was the expert that x dot trick I think that's super useful and super amazing. But another wonderful thing that WSL allows you to do is to run the Linux programs in Linux. That's true. For example, I for example, use password manager, I can't imagine anything without it.

And it's a pass. That's the US, I can just started in WSL, and it pops up a window in my windows. So I don't have to go there in WSL is you can just access through WSL dot localhost. So if I change this to JPH 00 I can do it this way.

And so here's my bash RC. And then I can edit it with whatever I want to. Visual Studio Code, for example. Alright, so okay. So let's take a quick look at Jupiter. CD and these. When I run Jupiter. My other window. jail. Okay, so I'll control click here. When I run Jupiter by default, it's going to, it's going to kind of be inside the nbs directory.

So when it says here. There's nothing in here, like it kind of says slash it doesn't really mean slash it means wherever I wherever you launched me from. So this is the contents of the nbs directory. So I can now create a new notebook. Like so. And run a cell and save.

Okay, by default, it's called untitled by pi and be so if I now. So that's now running here right so I can't like do stuff here because it's busy running Jupiter so I'll open up another tab. And I can LS. And so on Windows terminal is just control tab to switch between tabs and you can see there it is untitled by pi and be right and we can even take a look at it.

It's just a normal text file. Right. Okay, so we have Jupiter working, and we can import torch. Please object Jupiter and I pi widgets. Okay, that's interesting. Torch. Tensor. Cool. Okay. So, I'm guessing we need to install IPI widgets so we're getting pretty good at this. Let's try it.

Mamba install IPI widgets. Oh, this is needs to be reopened. Okay. I'm surprised that's not installed by default, but there you go. So close Jupiter. Three open it. Control click. Already had it open. And let's see if it now works okay. You didn't mean to say new. There we go.

And let's see if it now works. Yep, it's not complaining anymore. Okay. You can see Jupiter lab you can kind of do like normal gooey kind of stuff so if I want to delete this I could either go into my terminal and click and type RM untitled one, or I could right click on this and choose to leave.

This feels like a pretty good place to leave things does anybody have any questions or comments before we wrap up. the session, Jeremy, that was really helpful in setting up a very smooth as well.