Live coding 2

00:00:00.000 | turns out to be complicated so i think i've broken the back of it i think i've basically figured it out

00:00:06.320 | does anybody have any um questions or uh

00:00:14.000 | areas of interest or anything following our last chat

00:00:26.400 | hi yeah i am interested to know more about fast jam generally because i just google and found it

00:00:34.800 | very beautifully we will get to that that might be a little while though hi jeremy i'm i'm joe

00:00:42.480 | where are you i'm enjoying this from oh just about seven minutes away from where you're at

00:00:49.440 | oh you're in the red cliff peninsula somewhere no i'm sorry i thought you were in brisbane

00:00:56.320 | i'm in red cliff not too it's about seven minutes away so it's still um question i have um like

00:01:05.520 | really impressed with the blogging sort of set up out of jupiter and just i'm curious what your

00:01:10.800 | workflow is and how you are able to um do an experiment and be able to vlog about it very

00:01:17.920 | easily because i find i've got to move it from this get out to another sort of thing and there

00:01:24.240 | may be things in that space and yeah definitely i think yeah i think we've already let me just

00:01:31.840 | double check our list of things to cover yeah how to blog is actually number one of our list

00:01:36.960 | of things to cover in our walkthroughs topic and i don't think fast jam's there yet so let me add

00:01:42.320 | that fast chain there we go um

00:01:50.240 | all good questions um i'll uh introduce myself uh i'm matt from britain as well um i uh how do you

00:02:05.760 | i would like to um i think um we started looking at setting up jupiter lab last week um from

00:02:18.000 | remember correctly and uh so i think we'd installed jupiter lab um pytorch so i even managed to get

00:02:28.080 | that working successfully on my let that all work okay yeah it was it works apparently looks like

00:02:35.440 | it's working um so and i managed to um install um fast ai as far as i know um so i guess

00:02:47.440 | being confident that i can work on that to my machine would be yeah that's actually what we're

00:02:55.360 | going to do today i think we're going to focus on uh yep get our own kind of start doing some work

00:03:02.000 | on our own machine yep exactly um okay well let's get underway with that um so

00:03:16.160 | just to explain something oh um just to explain something you might have seen last time

00:03:34.880 | i i have two different users in my wsl installation i don't normally but it's just like

00:03:44.880 | i just wanted to mention that in a in in a linux you know distribution you can add multiple users

00:03:52.880 | and and switch to them um and uh so uh the way you switch between users is by typing uh pseudo

00:04:01.440 | minus u for username and then the username you want then minus i to open up an interactive

00:04:07.200 | terminal as that user so that's why you see me doing this sometimes so when i do that you can

00:04:12.800 | see it's like said oh welcome jph zero zero and kind of starts from scratch and you can now see

00:04:18.240 | before the @ sign it tells me that the only reason i'm doing that is so that i've got like an empty

00:04:23.120 | user that doesn't have anything installed or anything set up so i don't accidentally assume

00:04:28.880 | something that and um fail to create something um generally speaking there's you won't normally

00:04:38.080 | have a reason to create multiple users on your account unless you want to kind of set up something

00:04:43.760 | like this it's totally separate um so yeah i thought we could try to like create you know a notebook and

00:04:55.280 | python script and put them in a git repository and you know just kind of go through that

00:05:05.120 | workflow basically um so i normally create a directory to put my my git repository so what's

00:05:18.000 | a git repository a git repository is a folder containing files and potentially subfolders

00:05:27.360 | which um uh basically uh to simplify slightly you can easily store somewhere else um and the most

00:05:37.680 | commonplace people store it uh is github.com how do they provide free storage of git repositories

00:05:44.960 | um and most interestingly git keeps a kind of a copy of every version of your files

00:05:54.000 | and you can switch back to any previous version at any time so if you ever mess something up and

00:05:59.840 | save it and then realize later oh i messed it up you can you can go back and get an old version

00:06:05.120 | so maybe before we create our own github repo we could look at a github repo so we could look at

00:06:11.520 | the fastai one so github as i say is just one particular company that stores git repositories

00:06:18.800 | for you um you don't have to use them there's also one called git lab um but um i don't know

00:06:27.760 | most people use github so i think it's easier to go go you know be where everybody else is

00:06:32.800 | and so this is an example of repository um so or the name of the repository is fastai

00:06:40.320 | and then that belongs to some particular person or organization in this case it belongs to an

00:06:46.640 | organization the organization is called fastai um so if we get rid of the first of those the last

00:06:52.560 | of those fastai's this is the organization fastai and you can see the 119 repositories that are in

00:06:58.720 | that organization so if i click on one say fast release see how the url's changed this is this is

00:07:05.680 | now the fastai's organization's repository called fast release so this pair of things with a slash

00:07:12.800 | is basically a unique identifier in git lab for a repository because it has the username or

00:07:19.120 | organization and the repository so fastai is an organization my github username is jph00 and then

00:07:28.640 | that also has repositories and so i can click on one like tiny pets and you can see this is now the

00:07:37.760 | tiny pets repository in jph00 um okay so let's go back to the fastai repository

00:07:44.880 | so a repository as i mentioned contains folders and files and then folders can contain folders

00:07:54.320 | and files they can contain basically any kind of folder or file any kind of file you like

00:08:01.120 | they ask that your files that each individual file is smaller than 50 megabytes they don't

00:08:08.800 | particularly enforce that though um so that's what we see here there's a special file which

00:08:18.000 | is a file called readme or readme.md and that file is special because in this display it's

00:08:27.440 | automatically shown in this repository website so a kind of minimal useful repository

00:08:35.440 | is something that just contains a readme.md file and in fact as soon as you have that you have

00:08:40.160 | something that you can a website you can show people um so maybe we should start by doing that

00:08:50.560 | so if you go to your own github page github.com slash your username

00:08:59.200 | you can click repositories and so you might not have any repositories yet which is fine

00:09:09.680 | and here you could click new repository so why don't you go ahead and do that click new repository

00:09:17.440 | um and so you need to give it a name um walk through three

00:09:30.160 | if you want a description not a bad idea to remind yourself of what things are

00:09:37.360 | sample repo for walk through three of fast.ai course so you can make it private so nobody else

00:09:45.840 | can see it or public um something you'll hear a lot from fast.ai folks the community alumni myself

00:09:53.600 | is that the more you can work in public the better the more you can share your your portfolio the

00:10:00.320 | better um something which a lot of folks aren't used to and can feel uncomfortable but like here's

00:10:07.360 | you know here's a place you could start trying that is to make this public if your default is

00:10:11.280 | to always make things private maybe try not doing that this time um okay so we can yeah

00:10:18.400 | walk through second please walk through two sorry the second walk through the second walk here

00:10:25.280 | sorry yeah thank you i don't know why i imagined that there was another walk through

00:10:30.960 | maybe i dreamt it okay i guess it's because it's the weekend it's been a long time

00:10:36.960 | so we can say add a readme file now there are some things that you don't want to put in git

00:10:42.880 | kind of like temporary files and stuff like that and there's a special file called .gitignore which

00:10:49.120 | lists all the kinds of files you don't want to store um so if you click on here and start typing

00:10:55.520 | python this will kind of create a list automatically of the kinds of like pythons intermediate files

00:11:01.680 | and stuff um so again it's a good idea to use a license that allows other people to use this if

00:11:10.240 | you don't then other people can't use your code uh i always use apache license too and the reason

00:11:16.000 | for that is that it's like gives users you know really as much flexibility as possible and how

00:11:21.520 | they use the code but it also has a clause that minimizes the chance that i'm going to get sued

00:11:27.200 | for patent violations or that people would use this to kind of enforce patents in inappropriate

00:11:33.920 | ways um so wait can i ask a question of course um sorry my name is mark i'm from toronto i'm

00:11:41.280 | representing north america here tonight i think great or this morning so late night for you

00:11:46.080 | well it's only nine it's okay uh you're at the of course your west coast i see yeah

00:11:51.840 | that you're at east but yeah um the uh sorry can you just go a little bit more into the

00:11:58.880 | git ignore and i i didn't quite follow what that yeah let's create the repository and take a look

00:12:04.720 | at it um okay so i'm embarrassed i've actually been to toronto and i've completely butchering my

00:12:11.280 | geography is this the way you want questions asked or do you want hands put up or something

00:12:15.520 | i like yeah i like i like you know as informal a conversation as possible so yeah i'm not even sure

00:12:23.200 | i would see if hands go up so here's here's the repo we just created and if i click on git ignore

00:12:29.520 | it will show me the contents of that text file and so here is the text file called git ignore

00:12:37.040 | with line numbers on it um you can see it without anything else by clicking raw

00:12:44.240 | so that's this is just what the text file contains um and you can see for example um in linux a .so

00:12:53.360 | file is a compiled c library which is not something you would normally want to save into a repo so star

00:13:00.160 | .so star means anything .so so cx c library files would not be stored python has a cache of things

00:13:09.520 | that it's pre-compiled which it puts in a dunder pycache directory so this says don't put that in

00:13:15.520 | in into your git repo so basically when you when we add stuff to git repos shortly um it will by

00:13:22.880 | default not include anything that matches these these patterns does that answer your question mark

00:13:30.160 | it does in in the sort of literal sense i guess okay tell me more about what your non-literal

00:13:38.480 | sense is well so so i understand that it's um i understand that these are the files that will

00:13:44.240 | be excluded but i guess other than just always choosing python for git ignore i don't know how

00:13:50.080 | to decide which files to include in which files to oh yeah okay well we'll we'll see that as we go

00:13:55.520 | um what one one example that will come up if it's not already in here is um jupiter stores

00:14:04.080 | backups uh the most recent backup uh save copy uh in a backup directory called ipy checkpoints

00:14:10.400 | you would gen you know and when you go to git add you might say oh it's adding this directory called

00:14:16.320 | ipy checkpoints i don't want that and they'll be like oh okay i'll put it in my git ignore

00:14:20.160 | so it's generally like if you'll see things that you're like oh i don't

00:14:23.280 | need that to be backed up or like oh that's the thing that contains my password i definitely

00:14:28.960 | don't want that in my public git repo so those are the two main reasons inside the stuff that

00:14:33.280 | doesn't need to be there or stuff that you actively don't want to be there okay so but but choosing the

00:14:38.960 | python for git ignore is a generally a good first this is a good start so this is just kind of

00:14:43.280 | pre-populated it with a bunch of lines that we'll we'll add to later as we find stuff that we don't

00:14:48.480 | want to include okay so you can see it's created a readme.md file and if i click that you can see

00:14:56.080 | it's here's its contents and that's the same as the contents down here and so we could like create

00:15:03.680 | you know a little website now which is basically having websites better you know i guess it's a

00:15:08.480 | website by editing readme.md so if you click that little pencil

00:15:13.280 | we can now edit so try editing your readme.md and this is um this is markdown which we use in

00:15:23.520 | jupiter we use in readme files it's kind of used pretty much everywhere um and so it's well worth

00:15:30.080 | getting used to um so at this stage the markdown file you've got one line that starts with a hash

00:15:36.320 | that means this is a level one heading um a level two heading would have two hashes

00:15:42.320 | introduction okay and then you just put things um on separate lines of text so this is a sample repo

00:15:55.600 | we'll walk through of the fast today i course and then if you want a new paragraph just put an empty

00:16:00.960 | line between them this is another paragraph and then you know you can have lists by putting hyphens

00:16:10.240 | okay and so forth

00:16:19.840 | and maybe another header here okay so if i save that

00:16:25.680 | it doesn't say save it says commit um so this is um a concept in git is that you don't just save

00:16:36.480 | things but you create um you know a version it's called a commit um which is basically all the

00:16:43.200 | changes since last time you saved so commit means save but it means save with version control and

00:16:51.680 | so when you save um you need to have some description of what the change was that you made

00:16:58.560 | and optionally you can add more details about it so by default it's made a description for me

00:17:06.240 | so i can commit my changes and so now if we go to our repo you'll see that we've

00:17:13.440 | got more in our readme um you'll also see that this repository now has two commits so if i

00:17:24.000 | want to go back and see what it used to look like then i can see here's the initial commit

00:17:30.080 | and here's my update readme if i click on this commit code over here

00:17:34.880 | it shows me the difference so things with a minus and a red are things i deleted

00:17:42.880 | and things and plus and green are things i added and this is called a diff a diff for a difference

00:17:49.440 | and so very often people will you know say that they want to have a look at a diff

00:17:55.360 | to see like if you've made some changes to their code to see what changes did you make

00:17:59.920 | otherwise it's very difficult to know what somebody did so this commit gets automatically

00:18:04.320 | gets a little unique name as you see we can see the diff um or you can even click on this little

00:18:12.400 | button here browse repository at this point in the history so you can go back and pretend

00:18:17.120 | that life was before a minute ago and see exactly what this looked like back then

00:18:23.200 | so this is now looking at the state of this repo before i made the change

00:18:27.600 | so get get is pretty handy and github is a very useful place to store stuff in git

00:18:36.880 | so generally speaking we don't normally edit things directly with the github editor because

00:18:44.800 | it's pretty basic i mean i do sometimes you know if i if i just want to make a quick change to a

00:18:49.920 | readme i will sometimes click that pencil button but more often i'll do it on my computer

00:18:55.600 | so to put it on my computer

00:18:57.440 | we need to clone the repository so cloning it means making a

00:19:04.240 | kind of a linked copy on our computer so you see this code button here

00:19:07.920 | we can clone it we're going to clone it using something called ssh which is something we'll

00:19:16.720 | use a lot and ssh is a way of logging into remote computers and also automatically doing stuff with

00:19:23.440 | remote computers and copying stuff from remote computers so here is a a url it's not to a website

00:19:30.560 | but to a git repository using ssh so i can click here to click copy

00:19:38.560 | right so i've copied that so how do i clone it well the first thing i do is like say let's

00:19:46.320 | get things organized so i like to have a directory for all my cloned git repos so i'll go make to git

00:19:52.560 | cd git and so if i go print working directory you can see i'm in my home

00:19:57.120 | and remember that tilde is a shortcut for your home

00:20:00.720 | so i can now type so git is a program on your computer and so you can type git

00:20:08.240 | clone and then the url so i've just pasted it

00:20:20.560 | okay so there's a lot of security stuff built into ssh and one thing is to make sure that you

00:20:25.600 | don't accidentally connect to places you didn't mean to you know i don't use that much of the

00:20:32.240 | security stuff because i'm not too worried you know for most things i do so in this case it's

00:20:36.880 | just saying you sure you want to connect to this new place and i'll just say yes and it's going

00:20:42.960 | to save that okay so permission denied all right so i've asked to clone a repository using ssh

00:20:53.600 | but i got permission denied and that's because to use ssh you have to use an ssh key

00:21:02.240 | so the first thing i mentioned is if you want to clone somebody else's repository

00:21:07.360 | you can use http instead of ssh so i can click here http copy and then i could clone that

00:21:16.720 | and that works fine that's a perfectly reasonable way to work with other people's code

00:21:24.240 | so if i go cd walkthrough 2 here it is and i can edit the readme

00:21:30.800 | and then i could go down and i could add something else

00:21:35.600 | and

00:21:37.600 | the problem is i can't save that back again without logging into github so if i try to

00:21:47.680 | commit that i'll explain committing from the command line in a moment but i just want to show you that

00:22:02.080 | oh okay so the first next thing to know is if you want to save stuff back to

00:22:08.480 | back to github it needs to know your name and email address so it tells you some commands you

00:22:17.920 | can use so one approach would be just to paste them in right and obviously that's not the right

00:22:24.320 | email and username but things like this in the terminal they basically always create

00:22:31.440 | hidden files in your home directory so if i just type cd it takes me to my home directory

00:22:37.440 | and you can actually see here there's now a file called .git config and so i could edit that

00:22:45.840 | and here we go so info-rap-fast.ai say jeremy howard

00:22:54.880 | all right now i'll show you a little trick which most people don't know i want to go back to the

00:23:05.200 | directory i was in before i could type cd git slash walkthrough 2 or i could just type cd space hyphen

00:23:12.160 | and space cd hyphen means change directory to your most recently used directory so that's very handy

00:23:18.720 | one issue with that is if you've like cd three or four times there's no kind of history there

00:23:27.120 | so what i would do if i want to come back here later and i'm going all over the place

00:23:32.000 | is instead of typing cd you can go push d which is the same as cd but it remembers where you were

00:23:37.200 | so if i go push d and change to my home directory i'm now in my home directory i can cd to downloads

00:23:43.600 | i can cd to nbs how do i get back to where i were where i was before well i pushed

00:23:51.920 | from git walkthrough 2 so i can pop if i type pop d

00:23:56.400 | there we go i'm back where i got words originally so there's a couple of good tricks for

00:24:05.760 | zipping around in your directory structure it's a bit like pressing the back button i guess

00:24:10.960 | almost on on a browser um anyhow if i now try to save my change and the way you do that we'll

00:24:19.200 | talk about this more in a moment um is you push um you can see here it's asking for a username

00:24:24.880 | and so you obviously can't save things to other people's repositories if you don't have their

00:24:31.520 | credentials um it's also annoying to type in your username and password all the time

00:24:37.280 | so i never use almost never use HTTP get clone for my own repositories instead i use ssh

00:24:47.200 | so let's delete that directory and get clone with ssh so click ssh and copy

00:24:56.160 | and let's take a look again at this error we got okay so why do we not have permission that's

00:25:04.720 | because it doesn't know who we are and it doesn't know that we have permission to copy and change

00:25:10.080 | this repository um ssh is really nifty it doesn't use passwords instead we create a secret key on

00:25:19.680 | our computer for the private key and the second thing called the public key which we can give to

00:25:26.320 | other people and then anybody who has our public key will be able to accept uh we will be able to

00:25:32.560 | log into basically but they can't log into us that's why it's cool it's not like a password

00:25:37.280 | it's a one-way thing so we can get into github into our repos but nobody on github could like

00:25:47.760 | log in through our computer or anything so we need to create um it's called a key pair

00:25:53.760 | a public key and a private key so to do that you type ssh oops dash key gen ssh keygen

00:26:04.720 | so that's generate ssh keys and then just hit enter hit enter again hit enter again

00:26:14.000 | all right so that's created keys in my home directory

00:26:20.240 | n.ssh this one here's the private key and this one here's the public key so we need to tell

00:26:29.760 | github about our public key so that we can log in there so to display a file in the terminal

00:26:42.640 | without like uh basically show it all at once you can just type cat so if i go cat in my home

00:26:50.320 | directory dot sh dot pub there it is now there's no problem with me showing this on a live stream

00:26:57.840 | and it being on a video this is not in any way private or secret because this is just something

00:27:02.800 | that lets me if it's placed on another computer it lets me log into another computer

00:27:08.320 | in fact it's even stored publicly like anyone can access this public key because it's on github

00:27:20.160 | yeah so let's put it we're gonna put it on github so let's do that yeah so let's go back to github

00:27:28.400 | click on my face and um probably going to be in settings somewhere

00:27:37.360 | all right uh here we are ssh and gpg keys there we go um

00:27:53.440 | do we not add them here here we go new ssh key title i'll just call this the walkthrough key

00:28:00.240 | and paste there we go add maybe add the username i used

00:28:15.600 | all right so that is now in my account so i should now be able to rerun that git clone command so

00:28:34.960 | remember if you press ctrl r you can type a few letters from a previous command so i'll start

00:28:40.480 | typing clone and there it is and so if i hit enter now it's working so i don't have to

00:28:48.400 | worry about ever typing in a github password so here we are great um

00:29:03.760 | let's make sure i've closed my other jupyter session

00:29:10.000 | jeremy i noticed when i did that when i did the ssh key gen yeah i already already had one okay

00:29:24.160 | no worries so if you've already got one just use that one yep just use that one there should

00:29:28.400 | be a id r r i d rsa or id dsa or something like that that's your private key and the one that ends

00:29:34.000 | in pub is the one you'll copy over yep and so you don't need like how did you just how did you just

00:29:42.560 | split your terminal like that uh we will learn it's something called tmarks t mux but we'll

00:29:47.920 | certainly be covering that one of the best things ever um all right jimmy one thing that i just

00:29:57.040 | thought might be interesting for people or just something that can catch people out is making sure

00:30:01.600 | that your identity that you set up in your terminal as you showed before and that um the email that

00:30:09.360 | you use in github said that that they're consistent sometimes if you are using a different email or

00:30:14.960 | especially if you it's probably not a case here but if you have multiple github accounts you can

00:30:19.520 | end up um you know having one identity making commits to a repository that you didn't intend

00:30:24.480 | to under that identity and back and forth so just um just be careful when you first get started that

00:30:29.840 | you're using a consistent email identifier like you know it's not necessarily always important for

00:30:36.000 | your public key but certainly for your identity on github and for where you're going to be making

00:30:40.800 | commits and things like that um it's often helpful when you get started to make sure they're

00:30:44.640 | consistent yeah i mean i don't honestly ever think about that i kind of just chuck in any

00:30:50.480 | old email address it doesn't like it's it comes up sometimes but not too often um

00:30:57.440 | because it's kind of mainly informational i think this information so that other people can

00:31:02.960 | i find i've found it's more down the line if especially if you have a separate github account

00:31:09.280 | for personal and a separate one for work then um if you if you have different identities or

00:31:14.080 | different emails being so said with it then when you make a commit to a work repo you've accidentally

00:31:18.400 | committed it from your personal one which you didn't want to reveal your identity on or whatever

00:31:21.680 | so yep okay and somebody else have a question or comment just in the same context like you said

00:31:30.480 | the public key even if somebody else knows that they cannot commit to your git is that because

00:31:37.200 | the private key is unique only on your computer like just a high level correct the private key

00:31:42.080 | is the secret code so if you wanted to be able to um you know log in from another computer you

00:31:48.560 | would you would copy the private key over there and then that that computer can log in yep so the

00:31:53.360 | private key is the thing that says i i can prove i'm germany howard and the public key is a thing

00:31:59.920 | that says anyone who could prove that germany howard can log in here thank you no worries

00:32:08.160 | this is something that this this way of authentication is used across many different contexts

00:32:14.880 | so github uses it but for instance today i was installing gibuntu server on my local machine

00:32:23.040 | and i put points the installation to take my public key from github just to make things easier

00:32:33.440 | it grabbed the public key and it set it up so that i can easily connect to my computer without

00:32:39.600 | any additional setup steps so there is absolutely no danger given the current state of mathematics

00:32:46.880 | you know to making the information public and yeah without that you know going to the

00:32:56.560 | HTTPS and password authentication that makes it for such a cumbersome github workflow that

00:33:03.920 | this is so so much better yeah yeah ssh is used everywhere so for example i've got a GPU

00:33:09.120 | server sitting next to me here which um and then i've set up a um somebody at my door

00:33:20.560 | and a tick

00:33:31.040 | our doorbell keeps switching itself to a christmas chimes for some reason

00:33:49.200 | yeah so i've actually created a profile here for logging into my box which uh

00:33:53.840 | as you can see it just calls ssh so and then that's uh attached to

00:34:00.640 | control shift three so anytime if i want to log into my GPU box i hit control shift three and here

00:34:06.960 | i am i'm now you know typing in just as if i was at my own computer but i'm actually now typing into

00:34:12.640 | my GPU server um so ssh is a good thing to learn about properly um okay so we have

00:34:23.600 | Jimmy uh with regards to creating repos and committing and i mean what we are practicing

00:34:36.960 | right now i guess i'm in the course itself you show how we can deploy an application and sometimes you

00:34:42.960 | need to do the same thing having git push to those applications yeah and just wanted to see is it

00:34:48.880 | possible to have both github and also pushing a repo communicate with two different destination

00:34:57.920 | sources if it makes sense yeah yeah we can do that and we'll be saying that once we start looking at

00:35:03.360 | hugging face spaces yep yep you can so um you can have as many repos as you like and each repo kind

00:35:11.520 | of has this kind of default destination it's connected to which will normally be github but

00:35:16.800 | you can even connect one to multiple destinations um choose which one you're pulling and pushing

00:35:23.840 | from and to yeah can i ask one more with regards to the token password that you mentioned that for

00:35:32.400 | for that ignore file just wanted to see if that is important only for the public reports or even

00:35:38.080 | a private report on github you shouldn't share token passwords so yeah i would tend not to i

00:35:44.560 | mean it depends how important it is you know um i mean at least the people on great hub can see it

00:35:51.200 | um so you know and yeah i would i would try to avoid putting private stuff onto public websites

00:35:59.520 | in in general um particularly things like that where it's like at some point you might decide

00:36:06.080 | oh i want to make this repo public and then you forget that somewhere there was a at some point

00:36:10.800 | there was a secret file in there and you know one thing to be very aware of is even if i um in fact

00:36:18.000 | let me show you um let's create a super secret file and my secret is i don't like bananas i

00:36:28.720 | don't want anybody to know this right but let's say you know it was a currently uh um

00:36:35.360 | uh private repo pretend and so if we put that into github because it's fine it's all private

00:36:51.120 | and then i push that over to github.com and then later on i say oh okay i want to

00:36:59.440 | actually make this a public repo so i need to delete this from from from github so i'll get rm

00:37:05.360 | that file remove secret

00:37:17.360 | and like okay now i make my repo public everything's fine nobody has to know that

00:37:21.760 | i don't like bananas and then i can check on my repo

00:37:25.200 | and okay my super secret file's not there but remember commits let's go back to the world as

00:37:37.920 | it was oh it's here in my history right when you remove things from github it removes it from the

00:37:44.640 | current commit but it's not changing history so people could still find out my secret

00:37:50.160 | Jeremy i just um i just noticed there as well that was a good example when you looked at those

00:37:59.600 | commits of the um the identity issue we were just talking about before because on that commit screen

00:38:04.560 | you had some commits that were verified because you will have had an ssh key that was tied to your

00:38:10.000 | github email you see you've got those first two commits there are verified and then the latest two

00:38:14.400 | aren't that's because you just created a new key so that's the the key you won't have had the same

00:38:19.040 | email id as your first one so that's that's an example where especially like if you are trying

00:38:23.280 | to make verified commits if that's important to some project you're involved in um yeah that's

00:38:27.920 | where that'll check you out so i just saw that i mentioned that never been of interest in my

00:38:31.280 | projects i guess i never thought about it but yeah so this one was actually yeah because you know

00:38:38.960 | anybody could create a dot get config file that says i'm minus 12 volts and push and it'll say

00:38:45.360 | this is a commit from minus 12 volts um it won't say it's verified um yeah yeah um okay so um

00:38:56.800 | let's put a a notebook in here

00:39:03.440 | so um i can run jupyter lab and i think we created an alias last time jl

00:39:10.720 | and i pointed out that at this point it's it's you know this terminal i can't use it anymore

00:39:19.120 | because it's busy running jupyter now i could run another terminal session in windows terminal

00:39:31.200 | but i actually never do um i actually uh always use um instead something called tmux

00:39:38.160 | uh tmux is something which um actually runs inside your terminal so regardless of what

00:39:44.400 | operating system you're on and what your term preferred terminal is you'll always be able to

00:39:48.560 | use tmux and let me show it to you if you type tmux um well the first thing i'd mention is if you

00:39:58.320 | type tmux um it might say command not installed um and if it does say that then in linux you can

00:40:08.000 | type sudo and sudo um if you don't say what user to run it is it says run it as an admin it's called

00:40:15.920 | root right so so to install software you have to be root say sudo apt install means install linux

00:40:23.520 | software or ubuntu software tmux and so that's how you would install tmux if you got that error

00:40:30.080 | and so to run stuff as root you have to put in your password and so in my case i already have tmux

00:40:35.760 | okay um but yeah so that's that's what you would do on um mac you would normally use brew

00:40:42.400 | which unfortunately does not come with mac so if you as it says it's the missing package manager

00:40:53.280 | so if you just copy and paste this into your terminal in mac you'll then be able to should

00:40:59.360 | be able to do brew install tmux if you double check that it's available yep it is

00:41:04.000 | so brew and apt uh equivalence for mac and linux and i remember when i say linux i'm

00:41:12.720 | including windows as linux because it's it is linux so once it's installed if you type tmux

00:41:19.440 | you'll get a new screen that looks exactly like your old screen but now it's got a green bar on

00:41:23.680 | the bottom and this is what tmux looks like and it behaves the same way as usual um

00:41:31.600 | you know one thing is if i kind of like go off the end of the screen it's nice i can

00:41:43.040 | use my scroll um if you've got this mouse set up um but you know basically just looks like a normal

00:41:50.880 | terminal where things get interesting is that i can add additional windows um and so in tmux

00:41:57.600 | everything um almost everything you do tmux starts by pressing the uh key the keyboard shortcut

00:42:06.800 | ctrl b ctrl b is the tmux shortcut and so let's say i cd into get and i run jupyter lab i always

00:42:15.360 | like to run it from the place where i've kind of got my notebooks and repositories and i'm like

00:42:22.240 | oh okay what do i do now well i'll create another github window uh sorry another um tmux pane i

00:42:29.360 | should call them tmux pane um and i can create another uh either split them horizontally or

00:42:33.760 | vertically so if you hold hit hold hold down ctrl press b nothing happens and then press percent

00:42:39.680 | so ctrl b percent and you can see what that's done is it's created a second window over here

00:42:45.360 | as opposed to ctrl b double inverted commas

00:42:49.600 | which creates another window down here a pane and you can close them in the usual way and remember

00:42:57.200 | the usual way is ctrl d and each time you close them they just disappear so i pretty much always

00:43:07.120 | do that um now then how do you move around between these different panes

00:43:18.880 | depending on how your terminal is configured you might be able to click with your mouse

00:43:23.440 | i never do um i just press ctrl b and press arrow keys ctrl b right ctrl b up see how

00:43:30.400 | my cursor is moving here i am in the bottom right ctrl b left now i'm in the bottom left

00:43:36.160 | ctrl b left up now i'm in the top

00:43:40.800 | so that's how you can move around between the panes

00:43:47.680 | and then another thing to know is um this window looks very small now this pane looks very small

00:43:55.360 | i'd like more room please to get more room you zoom with z so ctrl b z so if i press ctrl b z

00:44:02.240 | that makes that pane take up the whole screen and once i'm done you know and i don't want the whole

00:44:09.440 | the thing to be maximized anymore i just press ctrl b z again and now it goes back

00:44:15.360 | okay um yeah Jeremy what how did you get the four by four right so um we got again so it's

00:44:37.680 | let me go back to where it was so um i'm going to first of all split vertically so that'll be

00:44:42.000 | ctrl b double quotes sorry ctrl b percent and then i'm going to split the one on the right

00:44:49.040 | up and down so ctrl b double quotes and then i'll press ctrl b left to go to the left hand tab

00:44:57.760 | the pane i should say and then i'll go ctrl b double quote again

00:45:01.520 | and there you've got it and you kind of like there are things you can do to make that automatic

00:45:09.040 | but after a while you get so used to it that like i show you when i do it i just go ctrl b percent

00:45:13.920 | ctrl b double quote ctrl b left ctrl b double quote and i'm done you know it only takes three

00:45:19.200 | seconds um and then the other nice thing about tmux is it sits there running in the background

00:45:24.880 | for as long as you like so i can actually detach from this tmux session as long as i don't turn on

00:45:30.080 | off my computer by pressing ctrl b d for detach and then i'll close down my whole terminal

00:45:38.080 | everything's gone okay don't have a terminal running anymore um oh that's going to be annoying

00:45:43.600 | to have to set everything up again get to the right directory rerun jupyter notebook blah blah blah

00:45:48.480 | but actually i don't because i detached from tmux so no all i have to do is attach to read tmux

00:45:56.720 | again um i do have to be the right person so let me go back to my extra account to attach to tmux

00:46:05.760 | again you type tmux a a for attach and everything's back so that is one of the very handy things about

00:46:17.920 | tmux and so if you yeah if you've got some long running job or something it's totally fine you can

00:46:24.560 | detach if you need to and come back to it later obviously if you reboot your computer it won't

00:46:30.560 | work otherwise it should be fine i think they're the main yeah yeah just to confirm if you close

00:46:38.640 | any of these windows that ends the process that's killed the process no i just closed the window it

00:46:43.040 | didn't close the process but i detached first so ctrl b d to detach and i close the part that it's

00:46:48.080 | all gone it's closed sure but i mean that at the beginning when you split into the four and then

00:46:53.520 | you close the tree and then went back to the original so for example the one with the jupyter

00:46:58.160 | if you close if you close a pane yeah so

00:47:01.600 | yeah so like the the pane here i can't close it because it's running a program

00:47:16.400 | so the only way to close that would be to actually cancel the program

00:47:20.560 | this pane's not running anything so i can close it so i can just hit ctrl d

00:47:26.560 | and so that just closes that session then if i do ctrl b double quote it just creates a new session

00:47:32.240 | a new a new interactive login if you like so each of these windows are totally separate to each

00:47:40.480 | other so if i overhear cd and to walk through two you can see none of the other ones you know these

00:47:46.320 | are all like separate copies of bash running this one over this copy over here is actually running

00:47:52.720 | jupyter all right there'll be one more doubt uh like can we assess it into a different session

00:48:02.640 | from uh team at little yeah so um absolutely so i you know i could SSH into my machine

00:48:09.840 | into my GPU machine from here um oh i need to be not jph 00 so let me just do that as somebody else

00:48:19.120 | yeah so i could uh ssh into another machine no worries you can even run tmux inside tmux if you

00:48:30.400 | want to gets a bit crazy um yeah they're all totally separate um okay so let's so i got jupyter

00:48:47.920 | running so i can control click on this url to open it up

00:48:52.400 | ah that's interesting so that didn't work because you see how it's wrapped to the next line

00:49:04.000 | so i didn't get the fa 91 it's kind of not considered part of it um that's why this

00:49:09.760 | didn't log in correctly so what i could do is to um zoom in with control bz and then click on it

00:49:19.760 | there we go um the other thing you can do is to set a password which isn't a bad idea um but for

00:49:29.680 | now we can just use this it's because it's using this so unique token that it auto creates um okay

00:49:36.800 | so because i launched um jupyter lab from inside the git directory that's why i'm here inside the

00:49:44.080 | git directory so here's walkthrough too so i could now create a notebook um turn that to a markdown

00:49:53.520 | cell so um i strongly recommend learning keyboard shortcuts um which

00:50:02.560 | what's the easiest way to see that in the lab i'm much more familiar with classic notebook

00:50:11.280 | so here's launch classic notebook which is what i normally use in classic notebook

00:50:19.600 | you can hit h to bring up the keyboard shortcuts

00:50:24.160 | let's see how to do it in jupyter lab jupyter lab keyboard shortcuts

00:50:39.840 | advanced settings editor in the settings menu doesn't sound very friendly settings

00:51:00.160 | advanced settings okay that's control comma keyboard keyboard keyboard there we go all right

00:51:10.160 | um

00:51:12.160 | great so anyway the keyboard shortcut to turn a um change a cell to markdown is just to hit m

00:51:25.360 | and so this is now markdown as you see um or to switch it back to code you press y

00:51:34.480 | and that's now code another useful quick keyboard shortcut is you can just press one two or three to

00:51:40.720 | create a first level second level or third level header or change one's one to a header as you can

00:51:47.040 | see um so um walk through sample notebook here is how i calculate

00:52:05.120 | one plus one put things in back text to say this is to start it as code

00:52:14.640 | here is how i calculate

00:52:26.800 | so if you hit shift tab after typing a function it'll tell you

00:52:42.640 | the parameters and so forth of a function a equals array 1.0 comma pi over two whatever

00:53:02.160 | sign a okay um so by default things get called untitled dot i pi nb which is not a great name

00:53:12.480 | so you can rename it to sample whatever now if i close that um it's actually running

00:53:27.920 | so that that that python session is still in memory most of the time that doesn't matter

00:53:34.000 | unless it's something where you're like training a model on a gpu in which case it continues to

00:53:38.720 | use your gpu memory so a couple of things you can do one is rather than closing from the x you can

00:53:46.160 | click close and shut down notebook which is ctrl shift q or alternatively you can click the x here

00:54:00.800 | all right um since we've got a proper terminal um on our computer we don't really need to use

00:54:11.280 | it much but if i you know and we will use it later you can create a terminal inside your browser

00:54:16.000 | which is identical to the terminals we've seen before and just like before ctrl b will close it

00:54:22.720 | all right so we've now got um

00:54:28.960 | another file called sample.i pi nb so we would like to put that into our github repo

00:54:38.960 | for other people to to share and just so i put a backup and so it's version control and so forth

00:54:45.200 | um so everything you want to do in git you first type git

00:54:48.480 | so what you'll often do at this point is type git status which tells you what's in git and

00:54:56.880 | what's not in git and so it'll give you a list of untrapped files so these are things you haven't

00:55:00.560 | yet got in git so to put it into git you have to commit it um so if you type um so if you type git

00:55:10.720 | commit um that'll commit anything that you've added to git so first of all you have to say okay what

00:55:17.520 | do i want in my next commit i want sample.i pi nb so git add sample.i pi nb i'll zoom in

00:55:23.600 | okay so now this is a it's not untrapped it's now a change to be committed is that we've added a file

00:55:37.200 | so now we can commit what we've added by saying git commit

00:55:40.160 | and then it'll say okay tell me your commit message and so by default it's opening up an

00:55:49.520 | editor called nano and we can change this but nano is probably like the easiest editor to

00:55:55.680 | get started with so it's not a bad place to start um so let's put in our commit message

00:56:02.640 | for people to see what have you done we added sample notebook and you can see down at the bottom

00:56:11.200 | that we can accept by pressing control x and it'll say do you want to save and i'll press y for yes

00:56:17.440 | and by default just leave the file name as it is so hit enter okay so that has now added the

00:56:28.800 | file to my commit so if i now say git status it's now in a third place it's gone from untrapped

00:56:37.120 | to do be committed to be committed and it is now committed and that means it is now actually

00:56:42.320 | version controlled on your computer so git actually version controls things on your computer

00:56:48.080 | even without using github um in fact originally when git was created by liners 12 volts there was

00:56:55.440 | no such thing as github and people kind of sent changes to each other directly rather than going

00:57:01.200 | through a server but in this case we are connected to a server and it said oh your your branch so

00:57:07.360 | your copy of this repo is ahead of the copy the place that you got it from by one commit

00:57:14.160 | so use push to to send your commits back to the server so git push

00:57:21.200 | there we go um so now if we do git status again there we go so that's the whole cycle

00:57:35.360 | and so if we now go back to github

00:57:40.960 | there's our notebook

00:57:46.560 | and github does have a basic notebook viewer it's not amazing but it does the job

00:57:57.920 | most of the time and so there's our notebook and so here's like a really minimal way of like it's

00:58:04.400 | not a blog or anything but you know to just like quickly share things with people this is the

00:58:08.560 | easiest way to do it you know and you can just say here's here's a repo and you know you can create

00:58:14.320 | as many repos as you like so like don't feel like it's in some way inappropriate to create

00:58:20.720 | a repo for one or two notebooks you want to share with somebody it's totally fine you know

00:58:28.000 | i generally have kind of a repo i put somewhat temporary things into but you know often when

00:58:37.280 | we're kind of sharing something with somebody else for example the bug we found we want to show how to

00:58:43.600 | recreate the bug we're just like create a repo just to send somebody an example of a bug for

00:58:49.680 | instance so yeah so repositories you can create as many as you like um

00:58:55.280 | a really good way to use repositories is let's go to the fastbook repository

00:59:04.240 | is it would be nice to have your own copy of the book

00:59:10.640 | because you would like to run cells edit things stuff like that so

00:59:16.800 | if you clone this right you won't be able to do an ssh clone at all because my because your

00:59:24.720 | public key is not in my account you can http clone it but you won't be able to save changes back to

00:59:32.320 | github so ideally you'd like your own separate copy of this and so you can create your own separate

00:59:38.080 | copy of this by clicking fork and so fork's going to create your own copy so you just say create fork

00:59:46.000 | there's a fork's a copy and you can experiment with changes without affecting the original

00:59:51.520 | and here you go so it looks exactly the same right but now i can

00:59:59.520 | the ssh copy this and then cd so cd dot dot means go to the parent directory which in this case is get

01:00:09.760 | get clone paste

01:00:13.840 | um actually yeah no that's fine yeah so we'll go get clone there we go

01:00:29.200 | and so now it downloads the the whole book and so now if we in fact let's do it let's open up

01:00:36.560 | a notebook so let's go back to jupiter lab

01:00:40.400 | fastbook

01:00:49.040 | feminist basics actually you know what would be good would be let's go into the clean version

01:00:59.440 | because this is really the one you want to be experimenting with

01:01:10.800 | and so i can now start running cells oh can't run fastai fair enough so um at this point we need to

01:01:29.840 | install fastai so we can see here it says condor install -c fastchan fastai

01:01:41.520 | so i'm going to copy that except for the condor bit because i want to use mamba

01:01:47.600 | so -c remember says to bring it from some other channel so this is not coming from the main

01:01:59.440 | channel but from a channel called fastchan so fastchan is a channel that fastai provides

01:02:04.480 | where we try to put in you know a lot of the kind of pieces of software that

01:02:10.160 | data scientists are likely to want to use

01:02:13.600 | okay so we can now say yep that all looks fine

01:02:22.400 | so fastai relies on the kind of the whole you know kind of ecosystem of python scientific

01:02:48.800 | libraries and since this is a brand new user you know things like matplotlib that's used for

01:02:53.440 | plotting for example and scikit-learn that's used for classic machine learning then none of those

01:02:58.080 | were there so because it relies on all these it it grabs them all downloads them and installs them

01:03:06.640 | can i ask a question while it's downloading please um is your i notice you're installing it in the

01:03:14.480 | base directory is that usually how you do it that's how i always do it yes yeah you don't

01:03:19.840 | create separate virtual environments never never like okay a lot of people do but i strongly dislike

01:03:26.240 | them particularly for beginners or unless you've got some very special reason i always put stuff

01:03:32.720 | in the base okay and the other thing i noticed was that the notebooks all use pip yes they do

01:03:42.000 | but it won't do anything for you because it says here see if there's a this is a bash thing it says

01:03:49.200 | see if there's a directory called slash content which there isn't this exists for something called

01:03:54.160 | colab which is a free jupiter server environment which doesn't generally have up-to-date things

01:04:00.560 | installed in fact i think they still have fastai version one so this cell will on colab install

01:04:07.920 | everything because colab uses pip basically um but yeah won't do anything on our computer

01:04:16.160 | um oh and we also need fastbook

01:04:21.600 | now fastbook

01:04:25.200 | i'm not even sure i'm not sure if there is a

01:04:32.160 | conda package at the moment for fastbook so i will use well let's check mamba install minus c fast

01:04:39.440 | chan fastbook i have a feeling that won't work

01:04:46.160 | oh apparently it does work there you go

01:04:59.760 | oh that was easy

01:05:05.440 | and actually fastbook includes fast fastai so we could have skipped the whole fastai one

01:05:12.960 | fastbook is just a it's basically just a list of dependencies of like all the different things we

01:05:23.120 | use in the book um it doesn't really have much code at all of its own it's just a kind of quick

01:05:30.240 | and easy way to grab all the stuff that you'll need for the book all right so now we should be

01:05:34.880 | able to run this okay um that is actually i think a little out of date i think we could use any

01:05:47.120 | sentence piece nowadays and i think we might have that in fast gen 2 sentence piece

01:05:57.680 | that's something we use in the nlp chapter

01:06:03.280 | yep looks good

01:06:13.200 | wow that's a slow download that's beating up at last

01:06:16.960 | let's try again okay so now we've got a copy of everything we need so this is like a unusually

01:06:27.360 | you know big uh kind of set of dependencies because it's a big book that teaches lots and

01:06:34.400 | lots of things but the nice thing is once this works um you know you'll generally find

01:06:39.120 | everything you need is gonna is gonna work um okay so at this point you know i've started

01:06:44.720 | doing things you know maybe i'll make some notes to myself um to install um we have to go remember

01:06:53.200 | install basically this is all we'd need would be mamper install minus c fastchan fastbook

01:07:02.320 | sentence piece that would actually be all you would need to do

01:07:08.400 | um so if i save this and so now you know we've created a note to ourselves in our own copy of

01:07:14.320 | the book so we'll close that and if we now cd to fastbook git status you can see it tells us we've

01:07:25.600 | modified that file so we would like to save that back to our copy um typing git add and then git

01:07:36.640 | minus m and then changing things in the editor is a bit slow so a shortcut is if you type git commit

01:07:44.800 | minus am that minus a means add everything that's not committed

01:07:48.720 | and this this m here says i'm going to put a message right here on the command line

01:07:53.680 | it's got to be in quotes um change m nist uh maybe add install notes

01:08:06.000 | so at that point if i type git status you can see now it's gone ahead and

01:08:10.800 | added it and committed it and it now says all right we're ready to push so if i take git push

01:08:14.720 | that saved that change so that change has not been saved to fastai's copy of the book

01:08:25.760 | of course but it has been saved to the jph fork so at this point my fork is one commit ahead

01:08:36.800 | of fastai right so that is i've made changes that are not in fastai's copy

01:08:41.440 | and so i could see what commits there are and that and here's all the commits that are in the

01:08:50.560 | one i forked and here's my addition and here's the change which says that i added this one cell

01:08:58.880 | okay i think that's probably about enough

01:09:12.640 | for one day does anyone have any questions or comments about that

01:09:20.480 | um the only one i found was that sometimes if you sort of we started off creating a repo

01:09:31.440 | within github sometimes i've done the reverse where i've created the repo on my computer

01:09:38.880 | and then had a few issues trying to get it into github and yeah yeah okay well let's add that

01:09:46.640 | promise to things to do in a future session i find it easier to do it in github so i actually

01:09:54.640 | always try to do it that way um like literally like because because i find it so easy like i

01:10:01.200 | always have to look it up to remind myself how to do it otherwise um so often i'll like i'll create

01:10:06.240 | it in github pull it and then copy over things that are on my computer you know like that's the

01:10:11.600 | really lazy way to do it um but let's do how to create a git repo locally all right let me pop that

01:10:21.280 | all right thanks gang um so we're doing tuesday wednesday thursday friday this week

01:10:34.560 | so i hope to see you then, thanks for joining us for Toronto, bye, thanks bye

Live coding 2

Chapters