pythonrunscript - Answer.AI dev chat #3
00:00:00.000 | Hey, everyone. This is Alexis from Answer.ai. This video is one of our dev chats. These
00:00:07.120 | are internal/external chats that we do about anything that we're working on and we want
00:00:11.160 | to share with other people. This is about a tool I made called Python Run Script. It
00:00:17.080 | lets you define single file Python scripts that declare their dependencies internally
00:00:22.640 | in comments. So, you can just run them without having to manually set up or manage a Conda
00:00:28.420 | environment or a virtual environment or a VM environment or whatever. I've wanted to
00:00:34.000 | put this little comment right now in front of the video because some of the things in
00:00:39.100 | the video are a bit out of date. So, one thing is it's called Python Run Script, not Python
00:00:43.600 | Script or whatever I refer to it as in the video. Another thing is it's done. It's in
00:00:47.600 | a GitHub repo. You can go check it out there, grab the binary, or you can install it by
00:00:53.360 | just pip installing Python Run Script. And some of the syntax that we talked over ended
00:00:58.680 | up a little different. It uses markdown fences instead of yaml fences to let you declare
00:01:04.420 | your dependencies. You can use requirements.txt or environment.yaml or conda install specs
00:01:09.520 | and put them in there. And also it has a dry run mode or verbose mode and the other amenities
00:01:15.320 | you'd expect from a command line tool. It works on Linux and Mac OS. In addition to
00:01:21.020 | talking about this tool, we also talk more generally about scripting languages and what
00:01:26.760 | makes them good or bad. So, I hope you find this interesting, and thanks.
00:01:32.560 | >> Okay. Now it's working. >> There we go. Now we're recording. Hello,
00:01:40.000 | gentlemen. Okay. So, I thought I would talk about something that I've been hacking on
00:01:47.120 | the last couple days. >> Who are you?
00:01:51.440 | >> Oh, right. Sorry. I'm Alexis. >> Hi, Alexis.
00:01:55.240 | >> Hello, everybody. I'm at Answer AI. This is one of our Dev Chats, which may or may
00:02:02.920 | not be published. So, might as well introduce ourselves. Jeremy, who are you?
00:02:10.880 | >> They know me because I've done two of them already.
00:02:12.640 | >> Okay. Well, Jono, what about you? >> Yep. I'm also at Answer AI. I think maybe
00:02:17.800 | in some past videos on the channel. If not, also R&D, same job title as everyone. So,
00:02:23.240 | that should be enough for now. >> All right. So, I just wanted to talk to
00:02:28.160 | you guys about something that I started on Friday afternoon when the brainstorm hit me.
00:02:35.160 | And now it's mostly working, although it's not prettied up yet. But I'm actually quite
00:02:38.880 | happy with it. So, I wanted to get feedback from you both on, I don't know, ways to improve
00:02:44.840 | it, if it would be useful to you, or just if it's interesting.
00:02:48.520 | >> So, what was the brainstorm? What was the problem your brain was trying to solve?
00:02:52.960 | >> So, I have a very opinionated brain. And the problem I was trying to solve, which I've
00:02:59.640 | tried to solve before, is to try to find an adequate scripting language. Also, just an
00:03:08.200 | easy way to get started on development. Now, I started writing a mini rant about the whole,
00:03:14.640 | you know, what it means for a scripting language to be adequate, which I haven't published
00:03:21.320 | yet. But I'm aware that the moment you say something like best solution for scripting,
00:03:26.040 | you're going to get a bunch of responses. Like, hey, you should just use Bash because
00:03:30.020 | Bash is everywhere. Or someone will say, just use Python because, you know, Bash is terrible
00:03:34.800 | and you have to use Python anyway because it's the best thing for the data science stack.
00:03:38.760 | That's a better language than Bash. All those things are true.
00:03:41.800 | >> I'm in the just use Perl category for what it's worth.
00:03:45.400 | >> You're right. Or just use Perl. >> I think I should probably learn Ruby.
00:03:49.560 | >> I think Ruby came and went. Well, I'm sure that's not true. Don't send me an email.
00:03:54.800 | >> Careful! You don't know who's going to be watching this.
00:03:57.960 | >> It's great for web things. And I have published a Ruby gem, so I can't call it a Ruby hater.
00:04:03.340 | Or if you keep going down this line, you'll get reactions like, what's wrong with you?
00:04:08.340 | You're crazy. You're overthinking it. All things are relative. There is no best language.
00:04:13.020 | Just be practical. Turn off your brain. That's not -- I understand that objection. I just
00:04:19.060 | totally reject it. It's not the way I operate. So here are the criteria that I started putting
00:04:26.060 | together in my mind for what I think would be a kind of adequate, you know, scripting
00:04:29.780 | language or a property you really want to have in any language you want. First of all,
00:04:35.300 | a script means a single file. There's a lot of tasks where you want to be able to write
00:04:39.820 | a single file program quickly in order to get things done. This is what we like about
00:04:47.520 | Bash. Also, an adequate scripting language should be one where it's good for the sort
00:04:58.500 | of things you do with scripting. So what do we mean when we talk about scripting? Often
00:05:01.860 | we mean file manipulation, some text manipulation, and some process manipulation, like launch
00:05:07.900 | this process, launch that process, move these files around. This is what Bash is really
00:05:12.380 | good for. >> I don't know if you mentioned it, but a
00:05:14.900 | bit of network stuff, probably like grabbing something --
00:05:17.620 | >> Perling stuff or pulling stuff down and then doing things with it. So that's sort
00:05:22.380 | of the domain of scripting and where some languages, you know, might be really good
00:05:27.940 | in a general way, like I'm fond of Swift, they often fall down on scripting because
00:05:32.740 | some of their APIs are very verbose for these routine operations.
00:05:36.140 | >> And ideally, deployment would be attach it to an email and send it to somebody.
00:05:41.820 | >> Yes, yes. So this is another part of an adequate scripting language. I can even -- why
00:05:46.860 | not? I can share the sort of draft that I started writing on this, which you can see
00:05:53.860 | I'm now sort of speaking from vaguely when I talk about what we mean by an adequate scripting
00:05:58.860 | language. So easily deployable is one of these things. So I've had the experience a number
00:06:04.500 | of times where I'm roaming around on the Internet looking for a piece of software I might use
00:06:09.420 | for something that would be useful, and then I find some GitHub repo, and it looks beautiful,
00:06:13.740 | just does it. It's great. And then it's like, just, you know, launch your NPM and do your
00:06:18.380 | node and do your package.json. It's like, that's not my religion. I don't know that
00:06:22.340 | stack very well. I don't want to learn it very well. I've gotten other things --
00:06:26.460 | >> Just to mention, your window might be quite small or something, because it looks pretty
00:06:30.380 | pixelated over here. No, it's like your font size is fine. But, yeah, I think your window
00:06:37.620 | itself maybe was quite -- >> Is that better?
00:06:42.020 | >> Small, because it expands it to fill our window.
00:06:44.620 | >> Okay. It's probably not the right geometry. >> It's probably like a large window with
00:06:49.020 | normal sized font. That way we'll get, you know, high resolution.
00:06:53.780 | >> That will be legible. >> Cool.
00:06:55.580 | >> How's that? >> Good.
00:06:58.100 | >> Good? Okay. So easily deployable. This is another big one. And ideally for this easily
00:07:05.180 | deployable property, you want to have one file, and then you give another person the
00:07:09.260 | one file, and then they can just run the damn thing. You don't need to give them an instruction
00:07:13.700 | manual on how to understand Python virtual environments, or exactly what a Node project
00:07:18.660 | is, or why you should be using RBMs instead of RubyMs, instead of gem explode or whatever.
00:07:27.300 | And obvious things, too. You want it to be cross-platform. You don't want to have to
00:07:36.980 | learn the subtleties of exactly how this command is different on Linux versus different on
00:07:42.100 | Mac, before you can hand it around to your friends and family. And you want it to be
00:07:47.140 | not a crappy language. So decent syntax is part of that.
00:07:50.940 | >> And I will say Perl is definitely ticking all the boxes so far.
00:07:55.260 | >> Oh, really? Okay. Well, maybe I should --
00:07:57.580 | >> Yep. Perl is on everybody's computer already. Single file, yes. Easily deployable, yes.
00:08:06.260 | And the syntax is, like, just basically the same as Bash. If you want to run a process,
00:08:16.140 | you stick it in backticks. The result of that process ends up in the variable that you passed
00:08:20.840 | it to. It has, you know, regular expression, language support directly, a library.
00:08:31.060 | >> That's interesting. >> Yeah.
00:08:32.060 | >> So maybe Perl already -- >> And it's got a great standard library,
00:08:34.260 | so, yeah, your scripts will be cross-platform. And I would add another one, which is, like,
00:08:41.480 | you don't have to learn any language. Maybe that would be another thing to list here.
00:08:45.300 | And since no one knows Perl anymore, it would fail on that.
00:08:49.100 | >> Yeah. No, that's also a reasonable one. Although perhaps it's a temperamental weakness
00:08:56.140 | of mine that I wouldn't think of adding that. So another one I would say is that you want
00:09:02.660 | to have a good prompt, a good interactive prompt. And that's one that we -- that a lot
00:09:07.980 | of nice languages can sometimes underperform on. And it's also one of the strengths of
00:09:12.060 | Bash. Like, the shell is already your interactive prompt, so much so that you don't --
00:09:15.220 | >> Yes. It is. Sorry. >> All right. Is audio good again?
00:09:24.940 | >> Yep. >> So I'll just briefly say, like, why some
00:09:28.340 | obvious solutions don't work. >> Oh, we've lost you again. It's like -- I
00:09:35.300 | think there's, like, a wire that's loose or something. Because, like, you -- suddenly
00:09:40.060 | we just don't hear you at all. >> All right. Let me -- I'm on the wireless,
00:09:45.980 | so that seems unlikely. >> Yeah. I was just wondering if, like, the
00:09:48.300 | boom mic has got a loose wire in it or something. I don't know. It's like it's just --
00:09:52.340 | >> How about -- >> -- talking, everything's fine, and then
00:09:55.060 | suddenly it's, like, gone. Most of the time, it's fine. We can tell you what's up.
00:09:59.740 | >> I'll show a bunch of stuff in the background, and maybe that'll help. All right. Here I
00:10:07.620 | am. So just to kind of recap why some things don't -- don't solve this problem as well
00:10:14.660 | as we wished it would. And did I just close Emacs when I was having my frenzy of --
00:10:19.980 | >> And stop sharing your screen. >> -- stop sharing your screen, so --
00:10:23.060 | >> Okay. All right. I'll bring it up again later. But just to say, like, why some things
00:10:28.900 | don't meet this as well as they might. And the last thing is external dependencies. So
00:10:34.780 | it's all very well and good if you have this nice single file script. But if having a single
00:10:38.620 | file script means you're confined to using the built-in standard library, well, you've
00:10:44.300 | kind of already lost now. Because now you've exploded the other properties. Like, let's
00:10:49.220 | say I want to do something in Python, for instance, fairly modest. Like, I want to handle
00:10:53.220 | YAML. As far as I know -- I could be out of date with this -- YAML isn't in the standard
00:10:58.540 | library. So now I'm talking about either building my own YAML parser or emitter and putting
00:11:05.820 | it in my file. Now I've lost convenience. Or I'm using an external library. And now
00:11:10.460 | I need to explain to someone else how to make sure they can install an external library.
00:11:16.500 | So that's no good.
00:11:21.060 | So why some languages don't meet these properties right away. The lack of easy integration of
00:11:30.100 | the external library, that one you lose right away for a lot of them. So that's a shame.
00:11:38.020 | Languages that you might think would be -- even if it's not a very good language, it's sort
00:11:41.380 | of everywhere. Bash. You don't have the cross-platformness, really. Because the Bash standard library is
00:11:46.700 | actually all the external commands that are on your Unix system.
00:11:49.860 | Exactly.
00:11:50.860 | And those annoyingly are different in subtle ways from Mac OS to Linux. If you're writing
00:11:55.740 | a non-trivial script, you start becoming a specialist in the fact that the date command
00:12:00.260 | is going to be a little bit different. And the grep command is going to take different
00:12:04.340 | arguments. It's just nonsense. Python and Node have this problem that the package management
00:12:12.660 | system is a thing where you kind of need to create a whole directory around your script.
00:12:18.420 | So you've already lost the nice single file property. And you need whoever you're giving
00:12:23.080 | it to to know how to manage local environments. Or to be such a guru that they have one vast
00:12:29.180 | omni environment that they maintain in a state of cutting-edge excellence and consistency
00:12:34.620 | and self-compatibility, which not everyone knows how to do.
00:12:39.700 | Some languages that almost do this really well, I'd say, are Go and Rust. Why? Because
00:12:46.860 | they actually can produce a single static-linked executable. So whatever complexity you went
00:12:52.780 | through when you were writing the thing, you can't often just give someone a damn artifact
00:12:56.660 | and they can run it. They don't need to get a PhD in your build system in order to use
00:12:59.980 | the thing, which is a beautiful, beautiful property. That's why you have all these great
00:13:04.260 | command line tools now that come out of the Go and Rust community. Because they make something
00:13:09.900 | and then everyone can use it. And we don't need to care how it was made. Whereas that
00:13:13.580 | can't be said for a lot of other tools, unfortunately.
00:13:18.660 | So that's the kind of way I think about the problem. And then I'll show you solutions
00:13:23.980 | that I've run into. Two or three that struck me as interesting. And then my version of
00:13:28.180 | it in Python. And you guys can let me know maybe how it could be improved.
00:13:33.940 | So the original idea-- let me share my screen. All right. Can you see the browser? So--
00:13:52.160 | Yes.
00:13:53.160 | OK. So I was originally inspired by this library from years ago by a brilliant guy named Max
00:14:01.500 | Howell, who's responsible for the homebrew package management system, among other things.
00:14:08.460 | And he did a lot of work in the Swift language. And he came up with this thing called Swift
00:14:11.580 | Shell. And the way it works is--
00:14:13.900 | Oh, I remember that.
00:14:16.020 | Yeah. It's a brilliantly tasteful little piece of engineering. Swift is nice. You can write
00:14:22.540 | a single file script in Swift. But then Swift is crappy because you have a bunch of dependencies.
00:14:28.240 | You need to have a package file. You need to have a directory. You need to name them.
00:14:31.380 | It's a whole separate operation.
00:14:35.020 | But with his thing, instead of having your shebang line in your normal Swift script be
00:14:40.660 | this, you just change it to Swift sh. And then you write Swift as usual, except you
00:14:47.260 | add special comments that come after the normal Swift import statements. So this comment says
00:14:53.260 | that the Promise Git library can actually be got from the GitHub account associated
00:14:58.660 | with Max Howell. And this script needs a version 6.5 or greater. And you can get more involved
00:15:06.040 | with it if you want. You can have fairly elaborate statements of version dependencies.
00:15:12.780 | So in this way, you can write a single file script that actually can have half a dozen
00:15:17.480 | external dependencies with pinned versions if you want. And when you run the script,
00:15:22.360 | it will just work. It'll do all the compiling somewhere else you don't see it. And you just
00:15:26.280 | have the magic of, I give someone a single file. And as long as they have this Swift
00:15:30.240 | shell malarkey installed, then they can just run that file. And the first time they run
00:15:35.160 | it, it takes a while because it's fetching all this stuff and it's compiling. But then
00:15:38.320 | the second time, it just runs because you have an actually efficient compiled executable.
00:15:44.060 | So that's quite nice. I quite like that.
00:15:48.020 | Then the Rust folks never miss a beat these days. There's a similar thing in Rust called
00:15:54.820 | Rust Script. And it's exactly the same idea. You need to install Rust Script. That's actually
00:16:00.420 | not so great to install. You've got to go and install it with an installer. It's not
00:16:05.020 | just a binary you put in a magic place. But then once you do, it's the same gimmick or
00:16:13.260 | the same act of genius. You just replace your shebang with Rust Script. And then you have
00:16:18.460 | this comment block that comes right after Rust Script that states your Rust dependencies.
00:16:22.700 | OK, I need time version 0.1.2.5. And then once you've done that, you've magically created
00:16:29.080 | a single file using Rust that can exploit the whole Rust dependency ecosystem and is
00:16:34.100 | dead easy to run. You just run it and it goes. That's great for development if you just are
00:16:38.760 | getting something started with something small. And it's great for distribution. You just
00:16:43.060 | give someone the file and they don't need to learn about Rust. They just do it.
00:16:46.460 | I wrote a C# version of this about 20 years ago, which was great. That's in my insurance
00:16:54.020 | pricing company. And we just give it to clients and they would use it to write insurance pricing
00:17:00.660 | scripts. I think there might be a somewhat more official way to do it in C# now.
00:17:07.500 | One would think there might be. So basically, I thought this should exist in Python because
00:17:15.220 | I don't know if everyone here agrees, but I'd say Python has-- let me change the screen
00:17:24.460 | I'm sharing. How do I do that? Where's my stop sharing button? OK. Because in my opinion,
00:17:36.120 | one of the-- and OK, we're back in Emacs. And let me make the screen bigger. So it's
00:17:44.660 | a reasonable size again, Jeremy, in terms of legibility? Yeah. Because the Python packaging
00:17:54.620 | and distribution story, in my opinion, is terrible. Maybe it's great if you really know
00:17:59.460 | your way around it. But it takes a while to realize all the different things that are
00:18:03.720 | out there and which ones are redundant and which ones are official and which ones are
00:18:07.860 | good and which ones are bad. And every time I step away from the language for years and
00:18:13.260 | come back, I feel like I need to recheck to see if things are-- what the situation is.
00:18:18.300 | And my understanding of the current situation is that there's two things that work and they
00:18:23.300 | are all you need. If you don't need anything complicated, if you don't need anything that
00:18:28.940 | has complicated native dependencies, then you can do everything with Venv. And you probably
00:18:35.340 | should because Venv and Pip come built in. And that will get you everywhere you need
00:18:41.820 | to go in terms of creating a contained environment and installing everything in a contained environment
00:18:46.700 | and working from it. And that's not enough. Because if you are doing interesting machine
00:18:53.940 | learning stuff, for example, or things that have parts of the data science stack, the
00:18:58.340 | fact is not all of that can be packaged with Venv. And people who think so usually think
00:19:03.340 | so because they just haven't had to use those dependencies. So they haven't run into those
00:19:06.620 | cases yet. But when you hit those cases, then the answer is Conda. You don't need to like
00:19:12.140 | it, but that seems to be the truth. So my current--
00:19:15.340 | It's not bad. It's not bad advice. For what it's worth, I would say, actually, you should
00:19:22.740 | use Miniconda regardless. On Windows, if you run Python on a new Windows box, it's a wrapper
00:19:37.820 | which basically points you to something that lets you install it. Like it's not already
00:19:41.220 | there at all. On Mac, if you run Python, you're running the system Python, which Apple strongly
00:19:47.380 | asks everybody not to use for anything other than letting them run system stuff. And on
00:19:53.100 | Linux, it's pretty likely you've got a really old Python, you know, if you're using Ubuntu.
00:20:01.860 | So I would tell people just use Miniconda. That way, you've got your own Python, and
00:20:07.780 | you can still use Venv if you want to. But, you know, now it's just like one solution
00:20:13.300 | for everybody. So just use Miniconda and just use environments.
00:20:17.780 | I've only ever used Miniconda. I haven't used the whole sausage before. I think your knowledge
00:20:25.100 | of this is much better than mine. So everyone, you should listen to Jeremy's advice. But
00:20:30.820 | if you have an allergy to Conda, and you know you don't need the data science stack, then
00:20:35.620 | it's my understanding that you can get by with just pip and VM.
00:20:38.380 | You probably can get by. The other thing I'd say is if you think you have an allergy to
00:20:42.940 | Conda, you probably have an allergy to Anaconda as it existed two or more years ago, at which
00:20:53.460 | point it was slow and awful. And it's actually now fast and nice. So your allergy may have
00:21:03.260 | recovered.
00:21:04.260 | Yeah. And also, if I understand right, Miniconda now uses Mamba as its dependency resolver.
00:21:09.780 | So it's a lot faster than it used to be.
00:21:11.260 | Kind of. It uses Mamba's algorithm. But you don't ever have to think or know about that.
00:21:17.020 | So it's actually, yeah, it's using LitMamba behind the scenes, which uses a proper fast
00:21:22.260 | C++ optimized solver. So it is fast.
00:21:25.340 | OK. Yeah. So maybe some people haven't touched it for a while, have a memory of it being
00:21:29.020 | much slower than it actually is these days.
00:21:31.060 | Yeah. It used to be very, very annoying. But nowadays, it just works. And it's quick.
00:21:36.220 | So speaking of just works, why don't I show you what I have sort of working right now?
00:21:39.420 | And I can get your opinion on what would be a maybe tasteful way to improve it. So based
00:21:45.740 | on my picture of what one actually needs these days, being sometimes Conda and sometimes
00:21:51.700 | just PIP and VM, I ultimately-- let's see what the latest version of this thing is.
00:21:58.460 | Just one other thing to mention, if you do use Conda, you can still use VM. So if you
00:22:08.020 | want to-- if for some reason, Conda environments are not for you, that's totally fine. You
00:22:15.140 | can just use VM in all situations except the one that Alexis mentioned, which is you want
00:22:21.660 | more complicated binary things to install, like a compiler framework or compiler toolkit
00:22:32.220 | or CUDA or whatever.
00:22:37.220 | Yeah. So it's funny you mention that, because I've tried to cover both cases in the way
00:22:45.060 | this thing works now based on my understanding of what's sometimes necessary and what's always
00:22:50.820 | necessary. So the point you're making, Jeremy, is that if you create a Conda environment,
00:22:58.460 | once you're within the Conda environment, then you can then run Conda install, and you'll
00:23:01.820 | be installing packages that are Conda packages managed through Conda's distribution repos.
00:23:07.300 | But once you're in the Conda environment, you can also just run PIP install and install--
00:23:11.860 | And you don't even need to be in a Conda environment. Once you've installed Conda, you're always
00:23:17.180 | in an environment called the base environment, so you can actually never think about it.
00:23:21.580 | You can forget about the fact you're using Conda or Miniconda entirely. You can use nothing
00:23:27.940 | but PIP. You can use nothing but VNV. And your life will be just fine. You'll just be
00:23:33.180 | using a recent Python version that's installed into your home directory and therefore is
00:23:39.820 | up to date and doesn't upset your Mac and doesn't require sudo and so forth.
00:23:45.420 | You should definitely be using one of those. You don't want to use the system Python, because
00:23:48.460 | it's actually there for the system to use. It's not there with the intention that you
00:23:51.700 | would build on top of it and install new dependencies into it. Because at any given point, Apple
00:23:57.820 | might decide to upgrade it, change things around, and you don't want to be destabilizing
00:24:02.540 | your system.
00:24:03.540 | All right. So here's how this will work. I've got a couple test scripts here that I was
00:24:10.940 | using to test it out. So I'll show some of the cases. So very simple example would be
00:24:20.100 | one where you have a script and it has no dependencies at all. Here's a very boring
00:24:32.420 | script. And here you can see the shebang is something I'm now calling VM script. But I'll
00:24:40.620 | change that. I can change that now to --
00:24:42.020 | >> I'm just going to tell people what /usr/bin/env is, because a lot of people are confused about
00:24:48.860 | that. So you don't have to use that. It's just a thing which, I don't know, it's kind
00:24:55.140 | of a best -- I guess, you know, it's trying to find the correct place to find that from.
00:25:01.100 | You could avoid that and just directly write the full path of your combo script, too, if
00:25:08.460 | you wish.
00:25:09.460 | >> Yeah. All right. So the script I wrote, the latest version of it, is now -- this is
00:25:17.740 | the part that needs cleaning up. This one right here. And what it will do is when it
00:25:29.980 | reads a script, it will parse a script looking for a dependency block. A dependency block
00:25:43.980 | is a formatted set of comments which may specify conda commands to run in a new conda environment.
00:25:59.700 | And may specify the exact text of a requirements.txt file which will be pip installed, either in
00:26:18.900 | the conda environment or else in a VM. So, for example, I've got my test scripts here.
00:26:31.660 | I feel like I've let everyone into my house before I cleaned up. It's very embarrassing.
00:26:39.940 | If you look at this script, for instance, this is the dependency block. The conda part
00:26:48.700 | of the blocks always starts with a header which is conda commands.source. Because if
00:26:53.940 | you were actually in a conda environment, you would source those commands in order to
00:26:58.700 | do your installations. And then the things that you might only want to pip install are
00:27:04.980 | in here, introduced with a header, requirements.txt. And this text here in the lines under requirements.txt
00:27:12.140 | will literally be the text of a requirements.txt file that's used. And then Python 3 m pip install
00:27:20.820 | -r requirements.txt will be run to install that. So, with this script, for instance,
00:27:27.540 | if we run it, and let me put this in the - put my thing in the path here. All right. So,
00:27:57.360 | the one I was looking at was test script conda requirements. Make sure my latest version
00:28:02.580 | is in the path. Yes, it is. Move this chunk out of the way. So, I'm invoking it now by
00:28:13.780 | calling the script. But because it's in the shebang, you wouldn't need to do that in order
00:28:17.800 | to run it. >> Let's do it the cool way, then.
00:28:22.100 | >> All right. Let's do it the cool way. So, let me make sure I've updated this to be - so,
00:28:32.520 | just to recap what Jeremy was saying, people haven't used this before. This line in the
00:28:37.060 | beginning is what tells the operating system what interpreter to use to run the file. So,
00:28:43.680 | this file's been marked executable. Let me make sure I have it here. Conda and requirements.
00:28:50.780 | Yeah, there's the X there for the executable bit. So, if I just run the script, conva script
00:28:57.100 | 2, sorry, I run test script, conda and requirements, then the operating system will read this line.
00:29:08.780 | It'll say, okay, I got to go find this thing in the environment. Because I put it in the
00:29:13.580 | path, it will be in the environment. It will then run my conva script 2 and pass in this
00:29:20.580 | file test script conda requirements as the first argument to run it. And when I do that,
00:29:26.620 | I've got it doing - well, there it ran very quickly, because I'd already cached the building
00:29:32.980 | that it did. But I have it doing a lot of debug logging where it says - it parsed it,
00:29:38.660 | it found that the dependency block implies the script will need conda. It then went into
00:29:43.620 | a cache directory that's kept out of the way where it found the conda environment defined
00:29:48.460 | by that directory. >> Okay. So, even though that's called VN skipped cache, I shouldn't
00:29:52.300 | be confused. That's actually - >> Yeah, no, that's just the naming that I need to update
00:29:56.700 | on where to put it. But let's get - we can go look at those cache things, erase them,
00:30:02.740 | and then see it do the thing of figuring out what to do the first time. That's much noisier.
00:30:08.220 | So, I'm calling it VN script cache, but I'll erase the cached environments now. And those
00:30:23.620 | caches - those alphanumeric blobs were MD5 hashes that were based on the dependency block.
00:30:31.060 | >> So, there's no fanciness, like I know some systems will - oh, you know, if you only changed
00:30:39.540 | one of the pip requirements, there's no reason to, like, create a whole new conda environment
00:30:43.460 | and download PyTorch again and all of that. It'll just update the one package. >> Well,
00:30:47.420 | that won't happen anyway, because conda's very smart. So, first of all, conda stores
00:30:53.500 | all downloaded packages in a packages directory. So, it won't have to re-download anything.
00:31:01.300 | And the packages directory is stored, I think, in the base environments location. So, it
00:31:07.660 | shouldn't be re-downloading any of those things. And then, secondly, although it looks like
00:31:16.300 | it's using up a lot of space to have a whole second copy of Python and stuff, it's actually
00:31:21.380 | using hard links. If you've got that installed elsewhere in another environment or in your
00:31:26.660 | base environment. So, it's pretty smart. >> Cool. >> Yeah. So, this is - so, now it just
00:31:34.900 | actually ran and did the thing. And I haven't silenced the output by capturing standard
00:31:38.980 | out or dumping it. So, that's why it's so noisy. But as you can see here, it ran through
00:31:45.900 | and installed the bits that were needed for conda, created a conda environment associated
00:31:51.940 | with this directory only, did the conda installs, and then it also did the pip install into
00:31:59.660 | that. And because conda also lets you specify the version of Python, this effectively ends
00:32:09.420 | up being a way that you can, in your Python script, specify and ensure that it's run by
00:32:13.300 | a specific version of Python, rather than have that be information that you communicate
00:32:17.980 | out of channel and that someone else needs to keep track of. If you want to go see what
00:32:22.260 | it's done in that developer directory, you can see there, it's now created a new entry
00:32:30.060 | in that cache. And at the moment, I just write literally down here the conda commands that
00:32:36.380 | were detected. Oh, interesting. I also wrote the requirements thing in there. But this
00:32:41.140 | is just put here for information. It's not actually run. And then here's the stuff that
00:32:47.380 | conda actually put in place. >> I can see it's got a little error there.
00:32:48.900 | I can see in line 137, underneath, it's complaining about requirements.txt command.found and --prefix
00:32:55.500 | command.found on line 139. >> Oh, right. Good point. So, my parser is
00:33:00.340 | picking up the requirements.txt incorrectly. So, I should fix that.
00:33:06.740 | >> That's okay. We get the idea. >> Yeah.
00:33:08.620 | >> It'll be fixed by the time somebody else watches this video.
00:33:13.260 | >> Yes. And if we weren't using conda, if we only had -- let's look at another example.
00:33:18.980 | Well, if you have a script with no dependencies, it's just going to run like a normal Python
00:33:27.100 | script. Let me see if I can find that. Variant requirements. Yeah. So, this is had test script
00:33:44.020 | no deps. Yeah. Chmod. >> I haven't seen chmod plus before. If you
00:33:57.140 | don't say -- >> Plus x.
00:34:00.140 | >> U plus x, does it default to u and x? Or it defaults to ur, x, w?
00:34:06.300 | >> You got me. That was a typo. I don't know what it does.
00:34:08.660 | >> I'm surprised it didn't spit out an error. It must have done something. Maybe it added
00:34:13.740 | nothing at all to no groups at all or users. >> Yeah. Or here's an example where it only
00:34:19.260 | has a requirements.txt file. So, let's see that one go.
00:34:32.060 | >> Core kits use fast progress rather than TQDM, by the way. That's what I hear.
00:34:37.020 | >> So, here it's detected that -- you can see from the logging, which is noisy right
00:34:41.820 | now. It detects that the dependency block implies script will only need VM and pip.
00:34:47.740 | So, you don't need to have conda on your system to use it. That's the other reason I wanted
00:34:50.660 | to make it so it didn't require conda. The whole point of this is to create a thing where
00:34:55.100 | people need to install the minimum amount in order to be able to use scripts. So, it
00:35:01.300 | needs to create the VM and then it does that. And then within it, it installs this library.
00:35:07.060 | And if we go look at the cache directory, which, again, needs renaming, we'll see there's
00:35:11.060 | a new entry. There. This one, hopefully, just has the requirements.txt file properly formatted
00:35:18.060 | in the VM inside there. So, that's the gist of it. It's a little noisy right now. It doesn't
00:35:25.820 | do all the error checking, right? >> I like it. And, like, with -- people don't
00:35:32.660 | quite realize on the whole, I think, that conda has, like, every program you could imagine,
00:35:38.420 | you know. >> Yeah. It's not only Python.
00:35:42.060 | >> GraphViz, Node, whatever. So, if you wanted to have a script that creates database diagrams,
00:35:53.820 | you know, and send it to somebody using the thing I showed yesterday, you could have this
00:35:58.420 | and it would have a, you know, conda install. >> Yes.
00:36:01.740 | >> GraphViz there. >> It's very general. And there's a couple
00:36:05.780 | -- well, like, a little bit of context on this. Like, part of the reason I'm so militant,
00:36:11.540 | I guess, about, like, easy deployment in single files is that I've -- in the past operating
00:36:17.220 | environments where not everyone around me was a software engineer, at my -- where I
00:36:20.980 | was CTOing at my startup, we'd have a factory, like, a literal physical factory that was,
00:36:25.700 | like, right there as part of the office. And there would be technicians working with manufacturing
00:36:30.500 | equipment. And we created software just for them to use.
00:36:32.380 | >> That's, like, what you guys were making. >> Oh, yeah. We were making -- right. Making
00:36:36.620 | custom eyeglasses. And so, we had -- we used machine learning to scan the face.
00:36:42.620 | >> Like, per person customized, right? >> Per person. So, we scan your face with the
00:36:47.540 | front-facing iPhone camera, eventually the depth sensor, do inference to work out the
00:36:52.900 | 3D shape of the face. And those measurements of the shape of the face would get fed into
00:36:57.340 | computer-controlled milling machines, CNC machines, to make the glasses that exactly
00:37:02.420 | fit your face. And so, as part of the business operations, it wasn't just people doing ML,
00:37:06.920 | people doing iOS development. There were people there operating CNC machines and making glasses.
00:37:12.620 | And they would need to -- we needed to have a workflow so that they would get data that
00:37:17.180 | came out of this -- came out of our database properly and was fed then correctly into the
00:37:21.340 | CNC machines. You've got to do order tracking, customer tracking, taking payments, like,
00:37:26.420 | all that stuff. So, you need a lot of bits and pieces of custom software that everyone
00:37:30.740 | can use. The customer service reps, the factory technicians, the software engineers, the marketing
00:37:36.220 | professionals. Like, all these people are super capable, but they're not all people
00:37:40.660 | who want to know what the hell it even means to talk about, like, why are you bringing
00:37:44.260 | a snake onto my computer, you know? Python this. It's ridiculous. Oh, what? I need to
00:37:49.220 | install Swift first? What's Swift? Is that a bird? Like, you know, it's no good. Like,
00:37:53.220 | you just want to be able to give them a goddamn file and then have them run it. And any time
00:37:58.580 | you need to do anything else, like, oh, just, you know, update your node, like, you failed.
00:38:02.540 | Like, the whole stack has failed. If you're requiring that, you've already lost. So, before
00:38:08.900 | I, you know, want to make anything, I want to think about, is it going to be a thing
00:38:12.540 | that someone else can use easily? Not, like, that they can use if they care about it. And
00:38:19.740 | I couldn't do that with Python. And I could sort of barely do that with Swift, using the
00:38:22.820 | Swift shell, you know, getup. But what I like about this is, you know, when I was talking
00:38:29.500 | to you about this earlier, John, you were saying, well, this is good, but can you just
00:38:32.380 | make it a Python module? Like, that might be good for Python users, but then it's chicken
00:38:37.220 | and egg, because now they need to know how to install a Python module, right? Or they've
00:38:40.180 | got to pip install the thing that saves them from doing pip installs. Right now, my goal
00:38:45.180 | is for this thing just to be a single file that anyone can put in their path, if you
00:38:49.020 | could explain to them where a path is, and then if they do that, and then you can also
00:38:53.300 | explain the shebang. So, we're already deeper in than we want.
00:38:55.740 | >> So, Alexis, I have, I have things that might be helpful to you.
00:38:59.940 | >> Yes, yes. I'm interested, I have a bunch of questions, but I'd be interested in feedback,
00:39:03.140 | thoughts. >> I have a, I have a installer for Miniconda
00:39:10.780 | that you can just run, and it works, it'll install, and without any interactive anything,
00:39:17.060 | it'll Miniconda on Windows, Mac, and Linux. And I believe somewhere I also have that sitting
00:39:25.900 | on a, like, a URL, so you can just curl blah, pipe, bash, and you don't even have to download
00:39:35.740 | it. So, yeah, if you want to be like, okay, to run this script, you'll need Miniconda.
00:39:42.540 | If you're not sure if you've got that, or you know that you don't, copy and paste this
00:39:45.900 | one line, and you're done. >> Yeah, that's a good, that'd be very interesting,
00:39:53.500 | because I think I want to, I haven't, obviously, it's a bit of a mess right now, because I
00:39:57.140 | just got it working, but my goal is to make it extremely easy to adopt, so that, like,
00:40:05.300 | data scientists and people who are smart and have a need to, like, maybe do a little bit
00:40:10.020 | of scripting, can use, really, all of Python, without needing to learn the package.
00:40:16.540 | >> Exactly. Also, remember that you can pip install from a git repo directly. Things don't
00:40:24.780 | have to be on PyPy. So, two things. A, make sure your regular expression handles plus,
00:40:32.380 | because that's what you, you'll be git plus, blah, blah, blah. The second would be, you
00:40:37.580 | might want to, like, mention it, if you've got some kind of tutorial, or readme, or something,
00:40:42.140 | is, like, here's the absolute easiest way to create a package for somebody else to use.
00:40:48.860 | And nowadays, with pyproject.toml, that's, yeah, pretty easy. So, if you've got a repo
00:40:55.140 | with a pyproject.toml in it, you've now got something that people can pip install, which
00:40:59.340 | means you've now got something that people, so, for people who are slightly more advanced,
00:41:04.620 | they can create things to enhance other people's Python scripts.
00:41:11.100 | >> So this was the part where I was interested in getting feedback from both of you, because
00:41:16.460 | I feel like there's a lot of, there's some design choices to be made here that are shallow
00:41:21.420 | in engineering terms, but very consequential in user experience terms. And those are the
00:41:28.220 | design choices around what format to require/accept in this comment block.
00:41:34.780 | >> Yeah, I mean, don't, that's too complicated. Like, requirements.txt is fine. Pyproject.toml
00:41:41.220 | would be if you, you know, wanted to create a package for somebody else to install.
00:41:46.140 | >> Yeah, so I was experimenting with this, but then I rejected it, because when I started
00:41:50.300 | looking at what constitutes the simplest, the kind of minimum parsable, pyproject.toml,
00:41:58.180 | it was already longer than it needed to be. >> Yeah, yeah, and it's not, like, it doesn't
00:42:02.500 | buy you anything. >> Exactly. Yeah.
00:42:04.780 | >> The other thing I'd say is, for your examples, like, I don't know, I discourage people from
00:42:15.460 | putting specific version numbers in, in their requirements.txt. I'd say this for here as
00:42:23.460 | well, you know, I'd probably just say greater than or equal to four, less than five, or
00:42:31.420 | anything. >> Yeah, no, I think that's true.
00:42:34.220 | >> There's a tendency, I mean, thusly, this comes very much from the, from the NPM world
00:42:38.660 | where, where everybody has these lock files and stuff. So people in the Python world tend
00:42:44.700 | to use this equals equals thing, but I think it's not, I mean, it doesn't matter so much
00:42:50.540 | what you're doing, because everything's going to be in a separate env anyway.
00:42:53.340 | >> Yeah. >> But I kind of discourage people from thinking
00:42:56.780 | this is how we should package software, that it depends on the specific sub sub version
00:43:01.740 | of a package, you know. >> Well, why? Why do you say that? Let me,
00:43:07.140 | let me disagree a little bit, or at least check out where you're coming from.
00:43:08.980 | >> Yeah, no, it's fine. Like I say, in your particular case, it actually doesn't matter
00:43:11.860 | too much, because you're going to have, well, the only reason why is that if I've got a
00:43:17.820 | different pack, a different script that uses 4.66.5, it's going to have to download a separate
00:43:23.420 | one and store the separate one in my packages, and I can't hard link anymore. And so all
00:43:27.340 | those things I'm saying about how efficient and fast and low disk usage this will be,
00:43:31.700 | would be thrown away, thrown out. >> Well, they'd be compromised somewhat, because
00:43:34.580 | now we'd have two versions. >> Yeah.
00:43:36.460 | >> On the system, yeah. >> And you're not winning anything. And it's
00:43:39.580 | kind of, for people who are, you know, more beginner-y, just learning-y types, it's kind
00:43:46.220 | of saying to them, like, oh, this is how you're meant to package software. It has, you know,
00:43:49.860 | you're meant to assume that it only works in the specific sub sub version. That's a
00:43:54.860 | minor issue, but. >> I think it depends on the sophistication
00:43:58.500 | of the consumer. So if I, like, think back to my, like, prototypical use scenario, if
00:44:03.140 | I want to hand someone a script that they can run, even if they have less, you know,
00:44:08.380 | technical expertise around Python, I really want to do everything I can to make sure that
00:44:12.180 | it will run, like, no matter where it lands. Like, if it's raining, if it's snowing, if
00:44:18.700 | they're running Python 3.10, if they're running Python 3.7.
00:44:21.180 | >> No, no, absolutely. I agree. >> I want it just to work. So locking that
00:44:24.100 | down, because if I make it open-ended, then I face the risk that at some point there will
00:44:28.220 | be a future release. >> Well, that's what I'm saying. You don't
00:44:30.580 | make it open-ended. You make it slightly more open-ended, right? So for mine, I generally
00:44:34.780 | write greater than or equal to x, where x is the minimum version I know works, and then
00:44:39.940 | less than, and then I write the next major version number.
00:44:43.100 | >> Okay. >> Or, you know, you can make it slightly
00:44:47.180 | less annoying. Like, you could just write 4.66.star, you know, or 4.66, I think, might
00:44:53.100 | work. >> So pin major and minor version. I can see
00:44:57.220 | that. >> Anyway, it's a minor issue. I also noticed
00:45:00.500 | in your actual script, I think your regular expressions could be less strict. So I would
00:45:09.540 | use hash backslash s star in the conda header and pip header, rather than hash space. I
00:45:19.500 | would also put a backslash s star after the colon.
00:45:22.420 | >> Sorry, which line are you talking about? >> So, yeah, those two lines, 23 and 24. After
00:45:26.940 | the hash, I would change the space to backslash s star.
00:45:32.180 | >> Oh, so it could be an arbitrary number of whitespace tokens?
00:45:37.140 | >> Yeah, and I would also put arbitrary amount of whitespace before the new line.
00:45:41.620 | >> Yeah, I know, that's all. >> And rather than using a new line, I would
00:45:47.620 | put an R before that string, so that you then don't need new line. Instead, I'd say dollar,
00:45:53.460 | and then, well, it depends how you're doing this exactly, but I don't know. That backslash
00:45:58.060 | s, in some versions of Python, that might now complain that you haven't, this is not
00:46:06.100 | an R string. >> These aren't R strings.
00:46:10.120 | >> That should be backslash s star, not backslash s plus. You don't want to require spaces before
00:46:16.220 | your new line. >> I do, though, because I want everything
00:46:17.780 | to be hash and then a space, at least one space at the beginning.
00:46:20.980 | >> Yeah, but not at the end. >> That's true, yeah, yeah, yeah, okay, yeah,
00:46:25.500 | you're right. >> And then pipheader probably wants a plus
00:46:30.780 | as well. >> Yeah, what I'm curious about is why I picked
00:46:36.660 | up the pip block at all when, earlier, so there's probably a deeper problem here I haven't
00:46:43.500 | sorted out. >> People don't have to watch this, watch this
00:46:46.740 | debug together, but, yeah. It's more of a general comment. It's in general for these
00:46:51.540 | kinds of things. I try to make my matching things reasonably flexible.
00:46:59.820 | >> So this goes to the other thing I was wondering about. So I think it's almost certainly the
00:47:04.820 | right decision to use a syntax where, first of all, instead of putting things off of the
00:47:13.060 | comments line here, off of, you know, next to the thing, actually just have a block at
00:47:17.100 | the top. Because that means that the requirement, yeah, because that means in particular.
00:47:21.300 | >> But I would put them in, this is quite a common pattern nowadays, is to put things
00:47:27.180 | in a YAML fenced block, so that would be, except commented, so that would be a hash
00:47:34.180 | space dash dash dash at the top and a hash space dash dash dash at the bottom. And that
00:47:39.580 | way it's like very clear exactly where your block starts and stops. And if there's anything
00:47:44.820 | in there which you don't recognize in the block, you can tell people, you know, rather
00:47:48.540 | than it just mysteriously not running that thing. It's a YAML metadata block. It's just
00:47:58.540 | dash dash dash followed by some lines. And at the end it's another dash dash dash.
00:48:05.540 | >> But I'm going to have to put hashes around it.
00:48:11.620 | >> You're going to hash before it, yeah. And with, you know, as many spaces as you like.
00:48:15.860 | >> So the thing that was -- I want to follow up on that because I'm not 100% sure I understood.
00:48:23.180 | But the one particular thing that I really wanted to get both of your opinion on is what's
00:48:27.820 | the right format here? Because --
00:48:29.900 | >> Yeah, that's the next thing I want to mention. So I think you should have two separate sections.
00:48:36.460 | I think you should have a conda requirements. And conda requirements would all get passed
00:48:43.860 | to a single conda install minus YQ. And then the entire lot of requirements space delimited
00:48:49.660 | would all get done at once. Because most people don't need conda commands. And that's not
00:48:56.740 | really a conda command source. It's just a, you know, commands source. You know, it's
00:49:03.060 | like just commands to run.
00:49:04.540 | >> So I started looking at conda requirements. But as near as I could tell, there wasn't
00:49:07.700 | a well-defined format that was specified by conda that had all the properties --
00:49:17.340 | >> There is. It's the thing you've got there. It's the thing you've got there, which is
00:49:19.660 | the thing that appears after the minus minus quiet. So that's what we use in nbdev as well.
00:49:26.700 | I would restrict exactly like your requirements.txt, but I would have, you know, conda requirements.
00:49:32.740 | >> Right. So the reason I thought that might not work -- and you can correct me on this,
00:49:37.460 | because you know a lot more than I do -- is that if you happen to be installing something
00:49:42.540 | where you wanted to rely on different channels for different packages, then the binding through
00:49:51.700 | which this channel applies for these packages and that channel applies for those packages
00:49:55.900 | --
00:49:56.900 | >> There's a colon for that.
00:49:57.900 | >> Oh, okay. So there's a way to --
00:50:01.420 | >> And, you know, I would be inclined to have like in the, like, conda requirements, you
00:50:08.220 | know, space channels colon. And that way you've got like -- you know, because it's very rare
00:50:15.180 | that you need to like almost -- I don't think I've ever needed to specify different channels
00:50:18.700 | for different things. I just have channels for everything. So I would have, like, hash
00:50:24.940 | conda requirements space conda-forge space fast-chan colon, you know, and then I'd have
00:50:30.900 | just my requirements one per line. And if you did need a specific channel for a specific
00:50:38.580 | thing, you can just use -- like, and if all you're doing is just concatenating with a
00:50:42.340 | space, then it'll work, because you can, you know, a user can put channel colon.
00:50:45.980 | >> Okay. I think I need to see that syntax to understand it. You say it's using nbdev
00:50:50.940 | for the -- for the conda installation.
00:50:54.300 | >> There's no syntax. What you've got, Python equals 3.10. That's the syntax. NumPy, that's
00:50:58.700 | the syntax.
00:50:59.700 | >> But you could say --
00:51:00.700 | >> Go with a colon.
00:51:01.700 | >> Conda-forge colon.
00:51:02.700 | >> Yeah, that's what I say. You'll never -- like, I've never used it in my life. So I just like
00:51:08.020 | -- A, you don't have to worry about it. B, you don't have to do anything to support it.
00:51:12.620 | Because it'll just happen. If you just pass it along to conda install, it'll just work.
00:51:22.540 | So I guess I'm thinking about setups like this. So --
00:51:27.020 | >> So you just dump it. It's fine. You just dump them all, like, in one command. It'll
00:51:32.060 | work fine.
00:51:34.180 | >> So there's not a complicated binding that needs to be maintained here between cuda is
00:51:39.380 | enabled by this channel, this stuff is enabled by the PyTorch channel, the NVIDIA channel.
00:51:43.620 | >> No, no.
00:51:44.620 | >> These things are coming out of the default. And if I don't preserve that binding property,
00:51:48.500 | then the installation won't work, if I do it on one line?
00:51:51.060 | >> I mean, it would -- it's unusual for that to be the case. And if it is, then people
00:51:58.260 | can put it in the commands one, you know? But I would -- I suggest that for the normal,
00:52:04.820 | easy stuff that's, like, 99.999% of the time, just --
00:52:09.300 | >> Okay. Yeah. Yeah. Because I was trying to find an easy format to use that was also
00:52:17.260 | well-specified and natural. And I wasn't sure what there was. It sounds like what you're
00:52:20.780 | saying works. What I liked about this was -- what I didn't like about this was it wasn't
00:52:25.100 | a well-specified format. It was just the commands people would issue. But then what I sort of
00:52:28.700 | liked about it is, it was just the commands people would issue. So if you can imagine
00:52:31.820 | there's somebody who doesn't know their way around, but has made these commands before,
00:52:36.220 | it's not too frightening to see, oh, okay, these commands will be run.
00:52:39.500 | >> No, exactly. I just support both. But for the one you've got here, there's no reason
00:52:43.980 | it couldn't say conda, colon, Python equals 3.10, numpy.
00:52:51.660 | >> Yeah. So basically something like -- this is what you're suggesting?
00:53:05.420 | >> Yeah. I mean, I wouldn't put the .txt there, because it's kind of weird, but, yeah.
00:53:09.980 | >> Okay. And then the -- well, I'll look up what you mean by the YAML fencing. You mean?
00:53:16.780 | >> So I would -- yeah, but I would have run the whole thing, not just for the conda bit.
00:53:20.780 | Three dashes, not four. No, not there. So I would go right to the very -- to line two.
00:53:27.580 | I would insert a new line in line two. And I would put hash dash dash dash there. And
00:53:36.220 | then I would go to line 10 and put a new line after that and put another hash dash.
00:53:39.820 | >> Oh, I see. To fence the whole thing, not the separate parts.
00:53:42.700 | >> Yep. And then you just grab it. And then if somebody's accidentally typed the wrong
00:53:47.020 | thing somewhere, you can say, like, I didn't understand that line, you know?
00:53:50.540 | >> Yeah. >> Or if you don't find
00:53:53.660 | any requirements of any kind, you can say, like, oh, you've got a fence block, but there's
00:53:57.260 | nothing I recognize at all, or -- et cetera. >> Yeah. Yeah, that might be good. I'm just
00:54:04.620 | trying to think about how to make it easy for someone who's never -- doesn't really
00:54:07.500 | know what a shebang is, doesn't quite know what conda is, but they kind of know, you
00:54:11.340 | know, I found a thing and now I'm trying to modify it to get a new result out of it, which
00:54:14.380 | is a likely situation for -- >> Yeah, exactly.
00:54:17.420 | >> -- certain classes. >> I think this is fair enough,
00:54:19.180 | what you have to edit. >> But I'm very excited by it. I know
00:54:23.820 | it's a tiny little thing. It's less than, you know, less than 300 lines of code or whatever.
00:54:27.020 | But I think it will enable it to be easier to experiment with stuff. Just create a single
00:54:31.820 | file, run it, get on with your day. >> I don't remember the last time I
00:54:37.500 | wrote something with over 300 lines of code in it, so I wouldn't measure things based
00:54:41.420 | on how many lines of code it has. >> Yeah, I mean, that's a figure of
00:54:44.700 | merit. I'm just saying it's not a -- yeah, it's not a big thing. It's just a little thing.
00:54:48.860 | Oh, the other question is where to dump all the garbage.
00:54:51.340 | >> Yeah, XTG is the right thing to do, exactly. >> If it exists, but if it doesn't, then,
00:54:57.340 | I don't know, is it going to .cache on Linux and library developer on Mac OS?
00:55:01.420 | >> Yeah, I'm trying to remember. I think FastCore, let me check.
00:55:15.260 | >> I just copied this behavior from SwiftShell because Max usually makes good choices.
00:55:23.740 | >> Yeah, so there's a FastCore.xtg library, which you could borrow. Yeah, and it attempts to
00:55:34.380 | have something sensible for everything, although it could probably be improved because
00:55:40.220 | it -- yeah, it's -- I mean, I don't know if it's wrong, but it's, you know, if you don't have
00:55:49.500 | something, like if you don't have XTG cache home, it uses path.home/.cache, for example.
00:55:56.780 | >> What file is that, Defender? >> FastCore/xtg.py.
00:56:05.180 | >>
00:56:15.460 | Okay, well, definitely something -- >> So if you feel like there are things,
00:56:28.780 | better places to put things on Mac, I would rather a PR to this than --
00:56:33.340 | >> Yeah. >> -- something else. That way,
00:56:36.140 | everybody can benefit. >>
00:56:38.100 | Yeah, I'll give it a think. Apple often has pretty good docs if you look for them about where things
00:56:44.540 | are supposed to go. >> Yeah.
00:56:45.660 | >> How fully -- >> Yeah, I think I really only
00:56:48.860 | looked up the Linux or POSIX or something docs. I didn't really think about Mac specific.
00:56:55.740 | Nice, Alexis. Thank you. >> Yeah, my pleasure.
00:57:03.500 | It's nice to get excited about a little thing, because, you know, it'll be done quickly because
00:57:07.100 | it's little, and then, you know, you can actually then go on and use it and get on something else.
00:57:10.300 | >> Yeah. Maybe this is why I only ever do little things. But the nice thing is, like, if you kind
00:57:17.660 | of organize your little things in a nice way, eventually you discover that they all created
00:57:21.100 | a big thing by mistake. >> Yeah. Cool.
00:57:26.060 | >> Nice. All right. We done? We good? >> I think we're good.