NVIDIA's NEW AI Workbench for AI Engineers

00:00:00.000 | Today, we are going to be talking about NVIDIA's new AI Workbench.

00:00:04.940 | AI Workbench is a software toolkit that is designed to help AI engineers and data scientists

00:00:11.720 | build in GPU-enabled environments.

00:00:15.660 | What AI Workbench does well, in my opinion, is extract away a lot of the kind of more

00:00:23.700 | fiddly parts of data science and local AI engineering and allow us to just start building

00:00:31.440 | something that can easily be reproduced by whoever we are developing with, if anyone,

00:00:37.840 | and also allow us to connect to more powerful remote GPU instances and very easily just

00:00:46.200 | switch context between our local machine and one of those remote instances.

00:00:51.000 | These are the things I think AI Workbench really seems to excel in from what I have

00:00:56.880 | seen of it so far.

00:00:58.760 | So in this video, we're going to introduce AI Workbench.

00:01:02.600 | I'll show you how to get set up with it, what features it comes with, where we should use

00:01:09.480 | it, and we'll walk through a little demo project that is actually from NVIDIA showing you how

00:01:15.080 | to run GPU-accelerated data processing.

00:01:18.960 | Now installation with AI Workbench is pretty straightforward, but it does require a few

00:01:25.180 | things outside of the core AI Workbench installation.

00:01:30.080 | A lot of things are handled by AI Workbench, but not all.

00:01:33.320 | So let's just go through that process quickly.

00:01:36.840 | So the things that are handled by AI Workbench for us, specifically for Windows, is of course

00:01:42.800 | AI Workbench itself, but also Windows Subsystem Linux 2, which is handled by Workbench.

00:01:49.580 | Now we do need to install separately Docker Desktop.

00:01:55.840 | If you are using Docker with Workbench, you can also use Podman.

00:01:59.920 | So in that case, you install Podman, but we do need Docker Desktop.

00:02:03.940 | If you're on Ubuntu, then you would just be using Docker, of course.

00:02:08.180 | And then the other thing we do need to install, if on Windows, is the GPU drivers.

00:02:13.720 | If you're on Ubuntu, you don't need to do this.

00:02:16.580 | So to get started, we will head on over to NVIDIA's AI Workbench page, and we can click

00:02:21.060 | this download button.

00:02:23.240 | During installation, we're going to go through a few steps.

00:02:27.020 | Just follow those.

00:02:28.280 | Use the recommended settings unless you have some reason not to.

00:02:31.640 | And then they're also going to ask you whether you want to use Podman or Docker.

00:02:35.600 | I chose Docker, most people use Docker.

00:02:38.720 | Of course, if you're using Podman, just go and use Podman.

00:02:42.600 | It doesn't make a difference, really, when you're actually using it.

00:02:46.640 | Basically within AI Workbench, every project that you build, it will be built within a

00:02:52.000 | container instance.

00:02:53.760 | So it's just whether that instance will be within Podman or Docker.

00:02:59.160 | Now once we've finished that installation and we've installed Windows Subsystem Linux,

00:03:03.960 | all that is left is to install the GPU drivers.

00:03:07.280 | Now, when running AI and ML processes with GPU acceleration, our code and GPU do not

00:03:15.360 | interact directly.

00:03:16.360 | Instead, they interact through several layers.

00:03:19.760 | So we go from, start with our code, like Python or C, Rust, whatever you're using.

00:03:26.600 | That's going to interact with CUDA, which is like a software layer, it's like a programming

00:03:32.720 | language from NVIDIA.

00:03:34.080 | That will then translate your code into instructions for your GPU drivers, which then run on your

00:03:40.600 | GPU.

00:03:41.600 | Now, we don't need to worry about CUDA, actually AI Workbench abstracts all that away for us,

00:03:47.040 | which is nice, but we do need to install the GPU drivers.

00:03:50.960 | Now the recommended way of doing this will depend on what GPU you have.

00:03:56.500 | If you're on a GeForce GPU, you should use a GeForce Experience.

00:04:01.120 | If you're on RTX, you should use RTX Experience.

00:04:05.200 | Now I'm on RTX, I'm using this Dell Mobile Precision Workstation on Windows.

00:04:13.140 | This comes with a NVIDIA RTX 5000 Arda GPU.

00:04:19.840 | So these vary a little in probably the GPUs like most of us are more familiar with, which

00:04:25.160 | is like 3090s, 4090s, those sort of GPUs, which are more consumer focused.

00:04:31.560 | These are more focused on professional workloads, which is pretty cool.

00:04:36.800 | And especially that it manages to fit into this laptop, which is, I'll be honest, it's

00:04:43.520 | a fairly beefy laptop.

00:04:45.840 | It's pretty heavy, but it's still a laptop, like I can carry it around with me wherever

00:04:49.280 | I go.

00:04:50.280 | And that's just insanely useful to be able to run my AI tasks locally on a CUDA enabled

00:04:58.240 | GPU.

00:04:59.240 | I mean, that is just incredibly, incredibly helpful.

00:05:04.600 | Now it comes with 16 gigabytes of GPU memory, which is nice.

00:05:08.220 | You're not going to be running like a huge, you know, 70 billion parameter LLM with that,

00:05:13.800 | but you can run small LLMs.

00:05:16.760 | And really the way that I would view this is more of like a local prototyping instance.

00:05:22.420 | If you want to do anything heavier, if you need to do anything heavier before shifting

00:05:26.280 | off using AI Workbench, which you can do, to a remote like A100 instance or something

00:05:34.280 | a bit bigger and more powerful.

00:05:36.240 | So back to our installation, I'm going to go ahead and install RTX Experience and that

00:05:42.920 | will handle the GPU drivers installation for me.

00:05:46.040 | Okay.

00:05:47.040 | So the first screen that we're going to see when we have gone through our installation

00:05:51.640 | is our locations.

00:05:54.720 | Now what we have here is our local machine.

00:05:56.800 | Our local machine is what's our local Workbench location.

00:06:01.640 | Ideally, we want to run on a machine that has an NVIDIA CUDA enabled GPU installed.

00:06:08.160 | But also thanks to Workbench, we don't actually need that because we can set up this remote

00:06:13.600 | location and just use that whenever we need a GPU.

00:06:17.560 | Now most AI tasks do require a ton of compute.

00:06:22.980 | More compute than most of us would have access to locally.

00:06:26.600 | And these scenarios are primary use case for what Workbench does.

00:06:32.060 | Workbench allows us to switch back and forth between a local dev instance and now remote

00:06:38.080 | GPU powered instance.

00:06:41.280 | Now if you do want a little more info on how to set it up, we do have a guide.

00:06:47.120 | So I will make sure that is linked in the comments and description of this video.

00:06:51.800 | And I will cover that in a future video as well.

00:06:55.080 | So let's go through to our local instance.

00:06:58.000 | And the first thing we're going to see is this.

00:07:00.000 | So we can either start a new project or clone a project.

00:07:02.880 | Let's just have a look at starting a new project, see what it gives us when we click through

00:07:07.080 | to this.

00:07:08.080 | Okay, so I'm just going to put some random stuff in here.

00:07:11.760 | It doesn't matter because I'm not actually going to create a new project.

00:07:14.640 | I just want to show you how it works when you create a new project.

00:07:19.580 | Basically I want to show you the template containers that they give you.

00:07:23.640 | Okay, so we click through and we have these.

00:07:26.600 | So these are just a few containers that we have.

00:07:29.700 | So we have like a basic Python container here.

00:07:33.360 | This doesn't include CUDA, for example, it's very stripped down.

00:07:36.840 | It does include JupyterLab.

00:07:38.400 | We have Python CUDA 11.7, CUDA 12, CUDA 12.2.

00:07:43.000 | And here we have PyTorch and this also includes CUDA 12.2.

00:07:47.220 | So these are like templates essentially that we can use to begin building from.

00:07:54.200 | It's like a foundation that we begin building our app from or whatever it is we're building.

00:08:00.680 | Now I'm not going to do that.

00:08:01.680 | I'm just going to go to clone a project so I can sort of get started and show you how

00:08:06.800 | everything works a little quicker.

00:08:11.000 | So we do need a URL for this.

00:08:14.360 | So to get that, I'm going to go over to GitHub and I'm going to go to the NVIDIA organization

00:08:19.960 | homepage and to find the examples that they have, the Workbench examples, you can just

00:08:26.300 | type in Workbench and they will pop up.

00:08:29.680 | Okay, so you see that we have Workbench example.

00:08:32.560 | All of them are called Workbench examples, so you could write this if you want to filter

00:08:36.200 | it down more.

00:08:37.200 | So you have all of these.

00:08:38.200 | I'm going to go over to the Rapids CUDF.

00:08:43.580 | So this is like CUDA accelerated data frames, like Pandas data frames, but faster with the

00:08:50.280 | CUDA.

00:08:51.280 | So I'm going to go over here.

00:08:52.280 | I'm going to copy the URL and I'm going to enter it in here.

00:08:56.440 | It will automatically create like a default path to save everything into for me.

00:09:03.120 | I'm going to use that.

00:09:05.760 | Okay, and pretty quickly I have the project built locally.

00:09:11.320 | So you can go and click build if you need to.

00:09:14.800 | That will build the project for you, basically set everything up ready for you to start interacting

00:09:20.400 | with it.

00:09:21.400 | But before we start interacting with it, I just want to show you a little bit around

00:09:24.800 | the project page that we find ourselves within right now.

00:09:28.600 | So this is where we can view and manage our project files, managing the environment, and

00:09:35.280 | of course start running JupyterLab or VS Code or whatever other apps we have installed

00:09:41.880 | from.

00:09:42.880 | All of this is set up to run within Docker because I set Docker as my preferred container

00:09:49.540 | instance.

00:09:50.960 | This would also, it would do the same for Podman as well.

00:09:54.120 | If we come down to here and click on build ready, I just want to show you how this works

00:10:01.040 | with our Docker container.

00:10:04.480 | So I'm just going to go up and I'm going to find the name of our Docker container, which

00:10:08.040 | is this here.

00:10:09.760 | So it's Rapids AI notebooks.

00:10:12.960 | Now if I go over to Docker desktop, which we installed earlier, and I'm just going to

00:10:18.080 | paste in Rapids AI notebooks and you'll see this pop up.

00:10:21.880 | This is our base image that we're using in our project and you can actually click on

00:10:26.120 | view on hub here and it will open it in your browser in Docker hub.

00:10:29.960 | And yeah, you can just see some information about the container here.

00:10:35.080 | If you come down here, we can see that we have all these, the Rapids libraries, Rapids

00:10:41.120 | is like a set of software libraries that NVIDIA have been developing.

00:10:46.960 | And if you come to here, we see that there are two types of these Rapids images, right?

00:10:52.240 | So one actually first to begin, they are based on this NVIDIA CUDA image.

00:10:59.600 | Then on top of that, we have this Rapids AI base, which just contains Rapids environment

00:11:04.440 | ready for use.

00:11:05.440 | And then the one that we're using is actually this one.

00:11:09.120 | So we have Rapids AI notebooks.

00:11:11.600 | This extends the base image by adding a Jupyter lab server, some example notebooks and other

00:11:16.800 | dependencies to it.

00:11:18.580 | And that's why when you come over here, we can actually just run Jupyter lab because

00:11:22.600 | that has actually, Jupyter lab has been installed via our Docker image.

00:11:27.800 | Okay, that's great.

00:11:29.280 | Now let's come over here.

00:11:30.280 | We can actually see some like a small description of what we were seeing before with our Docker

00:11:37.640 | image.

00:11:38.640 | So it has CUDA 12, it's Python 3.10.

00:11:42.720 | It contains Ubuntu 22.04, and it's like a high level view of what our container is.

00:11:51.400 | And if we scroll down a little bit, we can see we have these packages.

00:11:54.360 | Now it actually stays here.

00:11:56.560 | The package managers we are supporting here is apt, conda, and pip.

00:12:00.760 | And if we scroll through these, these are the packages that we, or package managers

00:12:04.760 | that we see, conda, apt, and pip.

00:12:07.280 | And we can also add more if we'd like here.

00:12:09.800 | So let's say I want to install PyTorch.

00:12:13.080 | I'm not sure what the current version of PyTorch is, but let's just say 1.2.0.

00:12:17.800 | And yeah, we can, I mean, PyTorch, we would go with pip or conda for that.

00:12:21.440 | And we can add that to our container like so.

00:12:26.120 | I don't need it.

00:12:27.120 | So I'm not going to add it.

00:12:28.800 | These do get stored either within the container, or we have our requirements.txt here for pip

00:12:36.080 | and apt.txt for the apt packages there.

00:12:39.640 | Let's scroll down a little bit more, and we have our variables and secrets.

00:12:42.560 | Now variables are, well, they're environment variables.

00:12:46.120 | These, whatever we add into here will actually be added into this variables.env file here,

00:12:53.120 | which is tracked by Git.

00:12:55.560 | So that means whenever you, you know, whatever you push, if you do push this to GitHub or

00:13:00.280 | GitLab or wherever else, any of the variables that you put into here will be included, which

00:13:07.040 | you need to be careful with, of course, you don't want to be putting, you know, secrets

00:13:10.680 | in there.

00:13:11.680 | Obviously you put them in secrets.

00:13:13.080 | It just means that whoever is cloning this, your colleagues, friends, you know, random

00:13:18.380 | people on the internet, they will be able to use the same environment variables.

00:13:23.280 | Then the other thing is secrets.

00:13:24.800 | Secrets are not tracked by Git.

00:13:26.440 | So they're not going to end up wherever you push this container.

00:13:30.040 | These are stored within your local workbench software.

00:13:34.720 | Okay.

00:13:35.720 | A few other things quickly to go through.

00:13:39.120 | So we have our mounts.

00:13:40.720 | That is what's a typical Docker container mount.

00:13:43.280 | So basically it's a place on your actual PC where data and models, whatever, can be stored

00:13:49.400 | so that when you shut down your project and container, that information doesn't get lost.

00:13:56.520 | You have applications.

00:13:59.000 | So we can also add VS code to this, or you can include your own custom apps.

00:14:05.160 | I'm just going to use JupyterLab for now, and I'm actually going to turn it on because

00:14:08.440 | we're going to use that very soon.

00:14:11.000 | Then we can come down to hardware.

00:14:12.720 | Now hardware, if you have multiple GPUs, you can allow the use of multiple GPUs here.

00:14:18.360 | Okay.

00:14:19.360 | So, I mean, that's basically everything I want to go through there.

00:14:23.240 | So I'm going to go over to my web browser, JupyterLab has just started up for me.

00:14:28.680 | So I'm going to go and open that.

00:14:30.520 | All right.

00:14:31.520 | Cool.

00:14:32.520 | So we're in JupyterLab.

00:14:33.520 | If it didn't start up automatically, you can open it by going to localhost 10,000 or just

00:14:39.080 | clicking the open JupyterLab button within Workbench.

00:14:44.980 | Now let's have a look at what we have running here.

00:14:49.500 | So NVIDIA SMI, you can see that we have a CUDA installed here, and we also have our

00:14:56.320 | GPU is being recognized.

00:14:58.800 | So you just want to confirm that you do have a CUDA version here and that you do have a

00:15:03.400 | GPU recognized here as well.

00:15:05.840 | And in the future, when you are running a process and you want to check how much memory

00:15:10.880 | your GPU is using, you can just run NVIDIA SMI again to check what we have in there.

00:15:21.520 | Cool.

00:15:23.800 | So let's see how the CUDF thing works.

00:15:28.080 | So I'm going to go into the CUDF panels demo, and we'll just go through a little bit of

00:15:33.720 | this.

00:15:34.720 | So we have NVIDIA SMI, that's what I just showed you, just so we can see again that

00:15:39.520 | things are as they should be.

00:15:42.180 | And we do want to confirm that CUDF is installed.

00:15:44.760 | So we just want to import CUDF, and this should run without any problems.

00:15:50.000 | Yeah.

00:15:51.000 | Perfect.

00:15:52.000 | So that looks good.

00:15:54.240 | What we need to do now is just download some data to sort of play around with.

00:15:59.040 | So we're going to be using this as NYC open data portal.

00:16:03.420 | It's this parking violations issued in 2022.

00:16:07.840 | Fascinating data set, but I'm not ready.

00:16:11.440 | I'm not too fussed about the content of this.

00:16:13.440 | It's more a case of just seeing what we get when we compare, you know, normal pandas without

00:16:19.160 | GPU acceleration with CUDF, which does have GPU acceleration.

00:16:25.520 | And the really nice thing about this is that we literally don't change any of our code

00:16:31.380 | to run this.

00:16:32.380 | There's like one line where we tell pandas to use the CUDF backend, and that's literally

00:16:38.400 | it.

00:16:39.400 | And it speeds things up quite a lot.

00:16:40.960 | So we'll see in a moment.

00:16:43.480 | So we'll just let that install.

00:16:44.960 | I'll skip ahead.

00:16:46.760 | Okay, cool.

00:16:48.140 | So we have that data set downloaded, and now we can go import pandas.

00:16:53.940 | This is not using GPU acceleration yet.

00:16:56.080 | I just want to run this first.

00:16:58.840 | So in this example, notebooks from NVIDIA, they go through like some example code, and

00:17:04.800 | they're just showing you what is happening.

00:17:07.240 | That's fine.

00:17:08.240 | And I appreciate it.

00:17:09.240 | But I want to just jump straight ahead to the bit where we're timing everything and

00:17:12.280 | just seeing what we get there.

00:17:14.720 | So we'll go through this.

00:17:16.640 | So right now, we're just loading the parquet file.

00:17:20.360 | And then we're displaying some stuff here.

00:17:22.440 | So we're doing like a group by, we're looking at the head, we're sorting the index, resetting

00:17:25.920 | the index.

00:17:26.920 | Okay, so the time for that total CPU time was 11.1 seconds.

00:17:32.040 | Let's do some more stuff here, run those, okay, 1.57.

00:17:38.960 | And this one is still running is 3.89.

00:17:41.560 | Okay, so it's not slow.

00:17:44.720 | But it does take a little bit of time.

00:17:46.920 | Now let's try with the GPU accelerated pandas.

00:17:51.040 | So this line here is just going to restart our kernel, because we, or we need to restart

00:17:56.640 | it, we want to load cdf pandas.

00:18:00.160 | And then so this here, we're loading cdf pandas, that is basically going to replace the back

00:18:05.880 | end with the cdf, basically GPU accelerated pandas, when we then import pandas.

00:18:12.840 | So I should actually, I think in the last one, where we time this, I don't think we

00:18:18.960 | imported pandas.

00:18:20.160 | No, we didn't.

00:18:21.160 | So let me just move this out of this cell, create a new cell here, I'm going to import

00:18:27.120 | pandas, then we're going to rerun what we saw before.

00:18:30.840 | So last time, it took like 11 something seconds, I thought, okay, now it took less than half

00:18:36.720 | a second.

00:18:37.720 | Right?

00:18:38.720 | So that's, that's pretty, it's really not too bad.

00:18:42.000 | Yep.

00:18:43.000 | So before it took 11.1 seconds.

00:18:45.560 | Now it takes less than half a second.

00:18:47.520 | So that's a pretty impressive speed up.

00:18:50.560 | Let's try the next one.

00:18:53.560 | 204 milliseconds.

00:18:57.400 | The last time we ran this without GPU, it was just over one and a half seconds.

00:19:04.460 | So pretty big speed up again.

00:19:06.920 | Now let's run this final one.

00:19:09.800 | So this is taking 0.7 seconds.

00:19:13.040 | And last time we ran, it was almost four seconds.

00:19:17.960 | So pretty big speed up.

00:19:20.460 | And we didn't really do all that much, to be honest.

00:19:24.640 | And it's worth noting that this is, you know, we're using a relatively small data set here.

00:19:30.160 | If you consider this with larger data sets, then you're going to see much bigger differences.

00:19:36.040 | You're going to, you're basically going to save a lot more time, which is quite nice.

00:19:39.680 | So if you do have, you know, that CUDA enabled GPU lying around, or, you know, you're willing

00:19:44.720 | to go and set up a remote instance, this is, it's worth it depending on what you're doing.

00:19:51.920 | So back to our project page, I'm going to go ahead and close this.

00:19:57.440 | So typically what we'd be doing if we're working on a project here, we're going to go to commit.

00:20:02.280 | You're going to commit everything you're doing.

00:20:05.680 | I am not going to do that because I'm not, I'm not actually working on this project,

00:20:09.760 | but I'm going to come over to here.

00:20:11.900 | I'm going to shut down JupyterLab.

00:20:15.400 | And then I would be able to just, you know, go ahead and close this.

00:20:18.780 | I am going to go ahead and actually delete the project because I'm not, you know, I'm

00:20:22.520 | not using it again.

00:20:24.000 | So you know, once you are done, you can delete that, save some space on your computer.

00:20:28.160 | And yeah, I'm with that, I'm done.

00:20:30.280 | Now that is it for this introduction to AI Workbench.

00:20:34.360 | As you can see, it's, it's very much like a, it's like a managed solution that for me

00:20:40.280 | feels best suited for data scientists and sort of AI engineers that are less familiar

00:20:47.920 | with things like CUDA and Docker and containerization and just want to, you know, get started working

00:20:54.840 | on a project and, you know, pretty easily switch across to like remote GPU instances

00:21:01.960 | without needing to worry about setting all that up because it can be kind of annoying,

00:21:06.160 | especially if it's the sort of thing that is, you know, relatively new to you as well.

00:21:10.080 | So I think this is actually, it's a pretty good solution for a lot of people.

00:21:14.920 | I think maybe if you're a, you know, you're a developer and you're working with a development

00:21:19.160 | team on like an AI project, it probably wouldn't be for you unless you're doing some like quick

00:21:25.280 | prototyping but within the space of more sort of data science and in some cases AI engineering,

00:21:33.080 | I think it's probably pretty useful.

00:21:34.720 | Okay, so that is it for this video on AI Workbench.

00:21:39.520 | I hope all this has been useful and interesting.

00:21:41.880 | So thank you very much for watching and I will see you again in the next one, bye.

00:21:48.360 | [Music]

00:21:58.360 | [Music]

NVIDIA's NEW AI Workbench for AI Engineers

Chapters