- Yeah. - Okay, hi everybody. Welcome to this tutorial for NBDev. And I am joined by Silva. Say hello, Silva. - Hi. - Who you guys will know from Fast AI and Alexis. - Hello. - Who you won't know necessarily since he's known as a swift guy, but he's gonna be here to help with clarifications and questions and so forth along the way.

You might recognize him actually briefly mentioned in part two of the most recent course during the swift bit. So we're gonna start, if you go to NBDev.fast.ai and on the left click on NBDev tutorial, we're just gonna step through it step by step. So there won't really be much information here that isn't already available in that tutorial.

So if you're more of a reading person than a video person, feel free to do the reading version. So I won't go into too much detail about what NBDev is. There is as linked here, a post with a lot of information about why it exists and what it's for.

But in brief, it's a system for exploratory programming, which is kind of a way of writing software, which involves or is kind of cognizant of the fact that I think for most of us, 90% of the time that we're programming, we're figuring out how to do things rather than just typing things in already knowing how to do things.

So there's a lot of exploration and experimentation. And so we wanna support that and even make their process of developing it something that other people can see and learn from themselves. So the platform that is, this is all built on is as the name suggests, Jupyter Notebooks. So this is all gonna assume that you're at least somewhat familiar with Jupyter Notebooks already.

Jupyter Notebooks are a very nice environment for exploring code and data. The only other thing I'll say is, Silva and I both feel like we're significantly as in like three to three times more productive using this notebook based way of programming than what we're used to being kind of traditional IDEs and editors and TDD and such like.

But I will say it does require a somewhat different mindset, somewhat different way of doing things kind of involves really embracing the experimental side of things rather than trying to do everything totally top down in the way that we often learn it at school. All right, so anyway, let's get started.

And so step one, so this thing we're looking at here, nbdev tutorial, not surprisingly comes from a notebook. So actually, if you click on GitHub over here, you can see nbdev's repository and in nbs is the notebooks for nbdev. And here is tutorial. So everything in nbdev itself comes from notebooks.

So if you ever wanna know how something is written, an easy way would be just to actually have a look at those various notebooks. So for example, if we look in within Jupyter, here is the nbdev tutorial we're looking at right now. And you'll see that some things are added automatically for us, like a table of contents, we didn't have to write it ourselves.

So step one is to create a repo. So I'm gonna try and show the easy low fiction way to do things once you learn more about how nbdev works, you'll find that there are different choices you could make always along the way. I'm gonna assume we're using GitHub 'cause that's the lowest fiction way.

You can certainly use other repositories as well. So step one is to click on this link here, which takes advantage of a handy thing in GitHub called repository templates. And it will just make a copy of a kind of bare bones skeleton repository. So let's call this hello nbdev.

- And just to add one position since we had a lot of people asking the link only works if you are logged in in GitHub. - Thank you, correct, correct. A tutorial example for nbdev. Okay, let's make that a public repository. Okay, here it is. And I realized I didn't open that in a new window.

So let's go back here. Okay, so that is going ahead and creating a new repo. Here it is. So here's our nbdev repo that just got created. Not much to see at this stage. So step two, while we're there, is we're gonna set up our documentation site. The documentation for nbdev uses something called Jekyll, which is a simple thing that builds your site to allow it to be hosted on any kind of static host.

GitHub Pages supports Jekyll. So you can host it there for free. You don't have to use that. You can use pretty much any host you can think of. But GitHub Pages is gonna be the easiest way. So let's go ahead and use that. So we have to enable it.

So to enable GitHub Pages, we click Settings. And we scroll down to GitHub Pages and we click on Docs folder. And if we now scroll down again, you'll see your site is now ready to be published at this address. So I'm gonna go ahead and copy that address. And then as it says here, copy that URL, go back to your main repo page, click Edit next to the description and place the URL there.

So we'll go back to here, click Edit, paste it here. Paste it here, save. Okay, so we've done the first bit. Step two, edit settings.py. Sorry, that should be settings.ini. We will fix that. In fact, let's go ahead and fix it now so that by the time you watch this, it will be fixed.

Edit settings.ini, there we go. Okay, edit settings.ini. So this is basically the file that everything in your project will be, all the configuration will be created from that one place. That makes life really easy 'cause you don't have to edit anything else for your documentation, for your packaging, et cetera.

So we're gonna edit all the commented out bits. So let's go ahead and find settings.ini and edit it. And all these sections here which are commented as it says they're required. So let's uncomment them. And we'll call this, hello, nvdev, user jphoo. User jphoo, description, a tutorial project for nvdev.

Keywords, fast.ai, nvdev, tutorials. So these are just things that are gonna end up in the GitHub metadata and maybe in your PIP installers metadata. - So that's one question I'd sort of ask. What does all this give you? When I first interacted with it, I realized that there was this config file but I didn't realize how many things it drove.

So it drives automatic document generation and then also other downstream configurations that let you easily share your library onto PyPy. Are those the main two sort of fallouts from it or is there anything else? - Also the actual Python project that is created from your notebooks. Yeah, those are basically the artifacts we end up creating an installer, a Python project, and documentation.

- Okay, so those will all be things that's been in the nuisance to figure out individually but they'll come out-- - Well, and also when they're in different files, it's kind of like probably six or seven different files that end up grabbing stuff for this, from the make file through to the navigation bar or through to the Jekyll config file, the setup.py file.

So yeah, that's make life significantly easier. It also means we can create tools that know to read and write stuff in this one place. So it makes it easier to kind of modify the tooling. Other things you could put here, this is where the version number is. As you'll see, that's gonna be managed for us automatically.

If you set your minimum Python here, then that'll change how your setup.py pip installation is done. This is also for your setup.py. The sidebar is the thing over here in your documentation, which is automatically created for you. If you wanna create a multi-layered sidebar, you can make it custom.

You can choose a license. PyPy expects you to say one of these seven statuses. You can change where things live. This stuff down here, these %s things is because we're using a Python standard library module called config parser to read this. And this is how Python standard library config parser uses kind of variables.

So in this case, user will be replaced with the value of user automatically by Python when it's read. So that is that file. So let's go ahead and commit that. Okay. So, oh, and we did skip a step which is to clone the repo. So let's go ahead and clone our repo as well.

Okay, so JPHO/LONBDev. LONBDev. Cool, all right. Okay, so the next thing is Git and Jupyter Notebook don't always play nice together. A Jupyter Notebook, if you've never looked inside one, looks like this. As you can see, it's basically, it's, well, it is JSON. And when you move cells around or edit cells, or even run a cell to create outputs, it can create a whole lot of a big def basically, which can cause conflicts really easily.

So one easy way to avoid those conflicts in the first place is to not save stuff to GitHub like cell metadata or notebook metadata. And so NBDev comes with things that will clean out that stuff for you. And so there's a whole bunch of command line tools in NBDev.

If you type NBDev and hit tab, you'll see them all. And one of them is called install Git Hocks. So if we run that, then that will install things into Git, which will now automatically clean up that stuff, which could cause conflicts. If you do get a conflict anyway later on, if you're collaborating with somebody else, you can run NBDev fix merge with a file name, and it will try to merge them with an understanding of how Jupyter Notebooks work.

And if it can't, it'll create a new readable notebook where the different cells are, the conflict cells are actually represented directly in the notebook. Okay, so we're now ready to edit our notebook. So you can run Jupyter Notebook if it's not already running. Personally, I tend to just have Jupyter Notebook always running in my notebooks directory.

And so I can jump to any repo. So in this case, I can go to hello NBDev. And you'll see there's something here called 00 core. So core, you can call it anything you like. The core bit is gonna be what the HTML page will be called. The number is totally optional, but so that and I have found it pretty handy to have incrementing numbers as you add more notebooks, 'cause it kind of gives the reader a sense of it all.

How was this built? What order should I read to understand if I wanna understand all the details of how something is put together? - Are any special Jupyter extensions required to be installed on the Jupyter server that's used for notebooks? - No, nothing's required. - Okay. - Okay, so when we open it up, as it says, you see something like this.

So let's look at what this all means. You'll see there's some comments which are special. So the first special comment is default export comment. And this is the name of the Python module that is going to be created. So in our case, our Python module is gonna be called Hello NBDev.

So this will create a sub module called Hello NBDev.core. So that's what the special comment means. So when we run the various NBDev commands, they look for various special comments, and this is an important one. This is another special comment, and it means when you create the documentation, don't include this in it.

Otherwise, pretty much everything in the notebook is included in the documentation, unless you specifically say hide. - Just one thought since we're here for the default export name. Pretty sure that if you want all the links to work properly in the documentation, the default export name should be kind of the same as the notebook.

So if you want to export in core, your notebook should be named something underscore core, like 00 underscore core. And if it's like core.utils, it should be named 001.core.utils, and so on and so forth. - Yeah, well, I've never tried not doing it that way. - So that's probably gonna make a lot of-- - Miniver, but some users have.

Yeah, some users haven't reported that the links didn't work, so. - Fair enough, yes. Okay, so, all right, so that's default X. The next one to know about is hash export. So we're gonna add a function to this notebook. So let's just go ahead and copy and paste this one.

And it's just a perfectly normal Python function with one exception, which is it says hash export at the top. So hash export means please include this cell in the exported, in this case, hello nbdev.core package. So now that we've done that, oh, and the other thing it does is it's gonna include, include it in the documentation.

So if you have a look at the tutorial, for instance, you can see that I've got this function here. And you can see in the tutorial, the function has automatically created this documentation with the docstring, a link to the source, and parameters and so forth. So that's what export does, it does those two things, puts it in the Python module, and if it's a function or a class or a variable assignment, it'll put it in the documentation.

Sorry, if it's a function or a class, it'll put it in the documentation in that special form. Okay, so the whole point of one of the key points of this is to experiment and to show people that experimentation. So let's go ahead and actually run that. And so, and we can add some more information.

There's just in markdown or whatever. Okay, hello, so far. - Hi. - And yeah, as it mentions here, you can, and it's a good idea to put in plots and images and stuff like that, so here's an example of SVG. And what we do a lot, and we find it works pretty well, is to also include tests.

So, all right, so we've already got an error in our test, that's good, oh, and that's why, because it's printing rather than returning. So you can see what happens when you get a test failure, it just appears here, and that will also appear later on when you see the test runner, it can also appear in the terminal or continuous integration, which we'll also learn about.

And this will by default appear in the documentation. So I probably don't really want this import line to appear, so let's move that into the hidden cell. But it's not a bad idea to actually include at least a few basic tests in the documentation, 'cause it can show people exactly what behavior they should expect.

So that's a bit of a, I don't know, taste issue as to how many tests you hide and how many you show. So far, and I tend to err on the side of showing rather than hiding. All right, so now we can create our Python module. So nbdev build-lib, we'll build it.

And you can run most nbdev commands from anywhere inside your repo, you don't have to be at the root of the repo. And as you can see, it's converted the two Python files we have. And so we now have a hello nbdev folder, and it contains these files, and there's the one we just created.

And as you see, it's got a little bit more than what we asked for. In particular, it's defined under all. It's considered a best practice in Python to just export using under all the things that you actually want to export. And so it actually gets a little bit easier with nbdev to follow that best practice.

- So just to spell it out here, the package or the project becomes a Python package, and then every notebook becomes a Python module. Is that correct? - Exactly, and you can create sub-modules as well, just by putting, instead of export this one, instead of default exp-core, it'd be default exp-blah.

It would be default exp-blah.blah to create a sub-module. - An export means that it goes into the module and the docs. Hide means it goes into neither, and if there's no annotation, then that means it goes into the docs, but not the module. - Exactly. - You also have exports with an S for export and show, and this will go into the module, and the source code will sustain the documentation if you want the source code to be in your documentation website.

- Yeah, which is actually what I probably should have done back here for the tutorial. I wrote the source code in the markdown and also put it here so that you would see the source code as well, but I probably should have just used exports, would be another way to kind of do the same thing.

But I don't use exports very often. Normally you don't really want to show the source code of a function in the documentation, but sometimes it happens. - Is there any support for the, how is, how is non-markdown cell content treated at all? Or maybe it's not. But does it make it into the docs successfully?

- Non-markdown meaning, I mean, there's only really two types who would use markdown or code. Code appears directly, outputs appear, and markdown appears as HTML. - I guess I meant like if you had markdown that was effectively including an image of some kind. - Yep, all that kind of markdown will work fine.

Yep, yep, so images and all that stuff is all supported, plots, et cetera. Okay, so now we can create or edit index notebook and that's gonna be what your documentation homepage and readme is automatically generated from. So here's index. And so the first thing to do is to note that it's importing your lib.core.

So we should import, change that to hello nvdev.core and let's see if that cell runs. It did, so we have successfully imported something. You'll see that in both here and here, there's a kind of slightly special markdown which is an h1 followed by a quote. This is used to create the HTML documentation title and initial heading in your documentation.

So you should keep it in this format. So welcome to, hello. Wow, nvdev, what's up? - Just to learn nvdev but you can put world if you want. - Fair enough, not the world, just to nvdev. An amazing tutorial. Okay, we hope you enjoy this project. Okay, so the main thing to do here is to make sure that it works.

So say hello, Alexis. Great, let's see this at work. Okay, so that's done. So now we can create our documentation. So we now go nvdev build ducks. And so you'll now see there is in ducks a index.html and a core.html. All right, so let's try to commit this. So we do git status.

We'll see there's a whole bunch of extra stuff here. So let's add it. And so you can see the things that it's modified for us if you want to dig into the details, things like the sidebar, the navigation bar, the config for YAML. One thing you might want to do actually is the init.py that's created for you is by default empty.

So you might, for example, want to go, oh, from.core import star. So if you did that, then that would allow us to modify our index example to just import directly from hello nvdev. Let's test that still works. It does, cool. So let's commit that. And it commit, push. There we go.

Okay, so GitHub is gonna go ahead and process that commit. And so if we go back to the main GitHub page, we'll see we now have more than one commit. So if I click on here, you can see that initially there's an X. And that X, if I click on it, tells me it failed to build my project.

And so specifically, you can see that it... Let's see. It failed to find the description flag in my settings.any. So when you first clone the repo by definition, it's not in a state that's ready. So this is kind of the way that nvdev with the help of GitHub actions automatically checks that all of your bits and pieces work, as you can see.

So it's gonna make sure it can install it, that it can read the notebooks, that they don't have that extra metadata, that the library matches the notebooks, and that the tests all run. So that generally takes a couple of minutes. Okay, so this one's now finished, and we still have an X.

So let's go and find out what went wrong. Now a test didn't run. Ah, so in this case, I tried to use tests from FastCore, and they failed. So let's go ahead and fix that. So to fix that, the best way to do it is to tell nvdev, or which it really talks to Python's setup tools, that we need FastCore installed.

So to do that, as you would hopefully expect, it's in settings.any, and you'll see here, there's a section called requirements, and we can just add space separated any requirements we have. In this case, FastCore. - And that's the PyPy package name? - That is correct, yep. And if you're a more advanced user, and you want to kind of debug things, or customize things more carefully, you'll see that the setup.py is, it's only 46 lines of code, right?

We try to make everything super simple. And you can see it's just like, as I said, it's just using config parser. And so if you want to see how those requirements are being read, you can see it's just grabbing the requirements thing, and splitting it, and then setting that to install requires.

So it's a pretty thin wrapper over, for example, setup tools. Okay, so, add FastCore depth, get push. Okay. And let's go check our documentation, because it did say that the page built correctly. So we should now be able to click on the web link we created earlier. And there it is, there's our documentation.

And you'll see this module name here has automatically been grabbed from over here. So while we're here, let's go ahead and fix that. Hello, nbdev, I guess it should be this really. And you can see the title's been set over here. And here is the automatically generated documentation, the docstrings being put here, and you can see the inputs and outputs, markdown all in one place.

Great. Oh, and now we have a tick. Okay, so that means that it successfully completed all the steps. So that's kind of like enough to get started, the basics are now all in place. So from here, we can start adding some more advanced functionality. So for instance, class. Okay, so here's a class.

And so when you have a class, it appears in the documentation like this. So it's got the docstring for the class, link to the source for the class, and the parameters for the constructor. The methods are not documented automatically. So to add method documentation, you have to add a call to the special showdoc, which is part of nbdev.showdoc.

And so when we do that, as you can see, it makes it appear here. And so it might be nice to include a docstring. You don't have to, but now it'll appear here. And in the documentation, the showdoc input would appear, just this markdown output. So this kind of makes it nice and easy to, intersperse markdown comments and images and plots and whatever in amongst your documentation of your class and add headings and stuff like that.

And I should mention, it's a really good idea to add headings because it's gonna help your table of contents. It makes it easier to get around. So I might say, for example, functions here. And then classes will automatically add their own heading. And so this method will already sit underneath the Hallowsayer headings.

We don't have to do that. Yeah, and let's add examples. Always a good idea. Or you could add tests. So we have a bug, which is now that should return. Great, and we could have used a test of course. All right. So here's something neat, which is, see here, it says say hello to using say hello.

And that say hello, if I click on it, jumps to the documentation to say hello. Even although I didn't create the link. So nbdev will look for anything in backticks, whether it be in markdown or docstrings or whatever. And it will attempt to see if it can resolve it at the Python symbol.

And if it can, it will create a hyperlink through the definition of that symbol. So you can see not only do we have it in the docstring here, but you can also see here that I've got hyperlinks in markdown. Again, they were just created using backticks. So that's something pretty handy.

- And does it look through all things in backticks or only cells that are exported? 'Cause I noticed that right now the class definition is not exported yet. - Yes, it'll only create links to things that it can actually resolve. So if something's not exported, then it won't be able to resolve it.

So yeah, let's fix that. Thank you. Okay, this is not an nbdev thing, but just a useful tip. These two lines will set up what's called auto reload, which is, so for example, in index, where we're using this module, if we have these two lines as the first two lines, then every time we update hello world nb.core, we don't have to manually restart this kernel.

That code will automatically be available inside this notebook. So that's a useful thing to add. The other useful thing to add is to put these two lines at the very end of your exported notebook along with a hash hide. And this is exactly the same as running nbdev buildlib.

So if I run this cell, and if we now have a look at our module, you'll see that class has been added. Okay, so that's that. So now that's all done. Let's make sure we haven't broken anything by running all of our tests. And tests can be in any notebook at all.

So to run all of our tests in all notebooks, we can just run nbdev test nbs. And that'll actually run the test in all notebooks in parallel. So even when you've got a pretty big repo, like we do for fast.aiv2, it just takes a few seconds. There we go.

And if there were any errors, it would pop them up and show you the stack trace of where the test failed. - One thing to add about the tests documentation and building the library is that if you're working on a notebook and it's not finished yet, you can put an underscore at the beginning of its name and it won't be considered by all of those commands.

- Let's try that. So, new Python three, one over zero, rename underscore, not finished. No problem. Okay. All right. Sometimes your documentation will fail to build. GitHub pages will give you a red cross next to the bit where it's trying to build the pages. But it tends not to give you any information about why.

So to find out why, it's a good idea to set up Jekyll locally, if there's a problem. Also, it's sometimes nice just to look at your docs locally rather than pushing and seeing how they look only after GitHub builds them. So you need to install Jekyll to do that.

If you're on Ubuntu, the easiest way is the official on Jekyll installation. Run these exact lines of code and that will install Jekyll. And then after you've done that, the next step is to CD into the docs directory and type bundle install. Okay. Then CD back to your repo root and type make docs serve.

Okay. Okay, that. So that's listening on port 4000. I'm actually not running this locally, it's remote. And I'm doing SSH forwarding. So I need to do something slightly different, but there we go. So here you can see it is running locally. Okay. Cool. When you're done, control C will kill it.

Okay, prerequisites we've already done. The next thing to mention is console scripts. So maybe you want to create a console script. There's actually a very nice system in setup tools for creating console scripts that will actually work on Windows or Mac or Linux automatically and the trick. And that's a surface through again, settings.any.

And the trick is to create a line like this console scripts equals. And so one easy way to see how this works is to look at nbdev. And you'll see there's a 06 CLI here. And we use our fast script library for creating these. But you can just have a normal, if name equals main or whatever.

In fact, you don't actually need that for console scripts. You can just create a function that looks at sys.argv. And so once you've got a function, all you need to do is to add to your settings.any console scripts equals. This is going to be the name of the script that you want.

This is the name of your library. This is the name of your module where you defined it. And this is the name of the function. - So this is showing us the configuration which actually creates the nbdev build live with console script itself. - Exactly, exactly. Now, just putting it in the settings.any doesn't do anything.

You have to install it to make it work. And so that you can test it, the easiest way is to type pip install minus e dot from the root of your repo. And that does something called an editable install. And that will install the console scripts in such a way and the module in such a way that it's actually linking back to the live code.

So you can change things without having to reinstall. And so that's a nice, easy way to test your module in other projects and to test your console scripts without having to reinstall it each time. - Sorry, I know you said this, but just looking at that configuration, why does nbdev build live appear twice before the second equal sign and after?

- This is the name of the console script that you're creating. - Okay. - This is the name of the function that is being called. - Okay. - So they could be different. - Okay, so nbdev.cli is naming a module. And then nbdev build lib is the function within that module.

- Yeah, in fact, maybe rather than looking at the notebook, let's look at the generated module. So if you look at nbdev, cli, here is nbdev build lib. So it is a function inside nbdev.cli. - Okay, so you could use this, if those two names weren't identical, to define a script that had a slightly different name from the function that you use.

- Yep, and for more information about this, click on this little cross-platform console scripts link in the tutorial where it'll explain, 'cause it's all just passing it through to setup tools, which is the standard library thing in Python. So you can see all the details there. - Thanks for the question.

All right, so since this is such a great module, and probably everybody on the internet will want to use it, we should make it easier for everybody to use by uploading it to PyPy, so that people can then go pip install hello nbdev. So, first of all, click where it says PyPy here, if you haven't ever done this before.

And then click on register, and go ahead and create an account. And then once you have your account, create this file called .pypyrc in your home directory, containing those three lines. I've already done that. So since I've done that, I can go ahead and say make release. And what that will do is it will bump the version, 'cause you can't have two releases with the same version number, and then upload it.

And so if you now look at cat settings.any, you'll see it's now 002, and you'll see that it has uploaded it to PyPy. And so now, we can go and have a look. There we go. So now you can pip install hello nbdev. And so you can see everything's gonna happen for you.

Your index has automatically been put into here. Your homepage has automatically been set to the GitHub repo, which is not working. Oh, 'cause I named it wrongly. Let's fix that. Vim settings.any. Yes, so the GitHub repo actually set a different name to the library name. That's gonna confuse everybody, but we can fix it in one place.

All right, so let's go ahead and commit that. Fix gh link. And let's do another release. And so this will bump the version to 003. And we should now be able to... There we go. Here's 003. Cool. All right, so. - So just, sorry, this may be the basic question, but make release, does that just call the same scripts that you were showing individually before, like nbdev?

- Okay, exactly. So here's the makefile. So the makefile to create a distribution is python-setup.py. This is just the standard setup tools. And release will bump clean, bump calls nbdev-bump-version. Plain calls... Plain calls rm-rf-dist. And then twine-upload is a thing. And you'll need to have installed twine. So pip install twine if you don't already have it.

So yeah, so you can see, for example, when you go docs-serve, it calls nbdev-build-docs and then calls the standard Jekyll thing. So yeah, the makefile is teeny tiny. And you don't have to use the makefile at all. If you're not into makefiles, you can just run any of those things manually.

So Alexis, you asked earlier about whether any extensions are required. And the answer was no. But there are two extensions that we do recommend, not just for nbdev, but for anything where you're just trying to create and use well-structured notebooks. The first is collapsible headings. And collapsible headings is the thing that is allowing me, for example, if you look at the tutorial, these little arrows here, I can collapse them.

I never use the arrows. I always hit right and left. So right goes to the end of a section. Left goes to the start. Left again to close it up. Left again to go to the start of the super section. Left again to close that up. So that's collapsible headings, which is super handy.

And the other one that's super handy is TOC2. Table of contents. And you can configure that in various different ways, but the main thing I find useful is that I get this nice navigate menu to jump straight to anything, which gives me the same functionality as I have in the documentation that gives me these table of contents.

Which is probably gonna be easier to see if I look at the tutorial. Oops, that should be last AI. Yeah, nbdev, great. So the contents is automatically created. So I don't need it in Jekyll. Okay. So yeah, finally, I don't know, it's often just helpful to see how things have already been written in another context.

And so going and having a look at the nbdev notebooks, all that functionality you've seen is just in one, two, three, four, five, six, seven, eight notebooks. And like, for example, have a look at our export, right? And it kind of gives you a sense of how we go about programming.

You can see like it starts with like, what's a notebook? A notebook is JSON. So here's something like starts right at step one. So like, if you're trying to understand all the detail of how nbdev works, then start here, because you can see the very first bit, which is where we actually read the JSON of a notebook.

And then you can see it's created a dict and here's the keys of the dict. And here's an example of the metadata and so forth. So kind of, we kind of try to write, this is how we explore, right? Is when we start, when we started this, we didn't know much about what a notebook is in terms of how it was formatted.

So as we started building the functions to read it and explore it, that became part of the documentation and part of the process. And so then you can also see that the checks that we did along the way as we tried to build something to test whether a cell has some particular or expression and so forth.

So yeah, and then as you go along, you might see, you know, other little things which we haven't described like default class level. And then you can go and search the documentation to find out what they are. And wow, that was good timing. That was exactly an hour. Any things you guys wanted to mention?

Anything we missed? Any questions? - I had one question. I know that you work in a notebook and then using these compile tools essentially, you can generate a Python module. It's like what you would have if you'd worked with a traditional text editor or an ID. But people already have tool choices that they're comfortable with.

Let's say you do that and then you want to go in and make a change directly to the Python code that's been generated. How do you get the change back into the notebook? Is that always a manual process? - Not at all. So that's a really good point. And we encourage people where possible where appropriate to edit things in the notebook.

And the reason for that is that in the notebook, you can see that context. There's a pros about why is it done this way or what else did we try? You can see the tests right next to it. You can explore to make sure the change you made really is working the way you thought it was.

But sometimes there's extremely good reasons to modify the actual Python files. And so in particular, when you're kind of starting to find your way around something, it can be really helpful to use kind of the tags features, for example. So if I'm reading through this and I'm like, oh, what's say hello do?

And so I'd like in VMI hit control right square bracket to jump straight to the definition of say hello. And obviously, particularly across files, this is really useful. And then maybe you just want to explore by like adding the print statement somewhere, or maybe you find like a typo in a doc string.

So there's no particular point to go all the way back to the notebook. So let's say we do that. Anybody at all? All right, so I've only changed the doc string. So there's no particular point to go back to the notebook, but I would like to now have that be modified in the notebook as well.

So to do that, I just type nb dev update lib instead of build lib. And so that will modify the notebook to match the changed module. So if we now go back to oocor, there is the change. Now, this is not for doing like major refactorings and stuff. It's kind of, you can only do small changes.

It's relying on these special cell comments to kind of know where cells are and to be able to find the right place to change. But for little changes like that, it'll work fine. Yes, the main thing is, as long as you don't want to add a new cell, it's gonna work.

But that's the only limitation. - Yep. Anything else? - And to complete what you just said, if at any point you want to see the diff between the notebooks and the library, there is a command for that, which is nb dev diff or something, we've stopped complete. I named them, but I forgot all the names.

So let's say we want to go back to what we had before. nb dev diff nb's. - Diff nb's, yeah. - And so you can see this is the difference between the notebook and the module. And this is run automatically when you push. And so if there's a difference, it'll tell you so.

And so one obvious way to fix that is just to run nb dev build lib, for instance. And then there won't be a difference anymore. Well, that's an impressive. - Down to the kernel there. - What just happened? That's terrifying. Did my whole machine just blow up when we were chatting here or something?

That was weird. I mean, get cwd, no such file or directory. Wow, oh, I see what happened. I said nb dev build lib when I was inside the lib. So the directory I was in had just got replaced by a new directory and didn't exist anymore. I don't quite know what that's got to do with the MKL, but it's all good.

Right. Okay, anything else? - I think I just want to pull things together to make sure I understood one thing. It sounds like there's a set of conventions that you need to follow for this to work, but they're pretty straightforward. Basic one is that the project becomes a package, the notebooks become modules.

It's advisable, maybe necessary, I'm not sure about that, that the notebooks be named with this number prefixing convention. - I'm not sure the number prefixing is necessary. The fact that they match the name with the modules and documentations is necessary. The numbers is just a recommendation, I think. - A lot of the names that need to be aligned could be, are specified in the settings on any file.

And then there's also a special markdown cell which needs to be set up in a certain way that has an h1 header, a quote, and then possibly text afterwards. - You could, if you don't have that, it'll just mean that you'll have some default or you won't have a proper title and stuff.

- All right, so that just affects how the docs look. That's not something where it's parsed. - The default exp is necessary. Now that I think about it, I don't know why it's necessary because we should be able to just grab it from the name. Maybe by the time you're watching this, that will be fixed.

- It's just because sometimes you have notebooks that are not exported at all. So you need a way to-- - Yeah, that makes sense. - Oh yeah, and then there's a convention around underscore notebooks not being exported. And in terms of what someone would need to use this, you need to have stuff like Python assault, obviously, but you need Jekyll installed to do the document generation locally.

This thing called Twine, it's also part of the-- - You can store Twine in order to push to PayPal. Other than those, everything is automatically installed when you pip install mbdev. - Okay. - Including Jupyter, which, but if you've installed from Anaconda, that'll be installed for you anyway. - And in terms of the key mechanics of interaction, there's a set of build tools for building the load, for building the docs.

There's a make file for like bumping versions and uploading docs. And then within the notebooks themselves, there's hide, which means don't put it in the module or the docs. There's export, which means put it in the module and the docs. There's nothing at all, which means only put it in the docs.

And then there's export s or exports, which means put the code as well as the-- - Yeah, the export code. - As well as the export in the docs in the module. - Right, exactly. - Just wanna make sure I had the essentials right. - No, that's very helpful, thank you.

Okay. Thanks, Kang. See you next time we do a thrilling tutorial. - Awesome. - All right. - See you. - Bye.