back to index[Workshop] AI Pipelines and Agents in Pure TypeScript with Mastra.ai — Nick Nisi, Zack Proser

00:00:18.960 |
This is AI Pipelines and Agents in Pure TypeScript 00:00:32.020 |
but the patterns that we're going to use here, 00:00:36.060 |
of using TypeScript to build a production agent, 00:00:40.020 |
and that you can use to deploy to production with AI applications. 00:00:44.760 |
So the rough format that we're going to follow here 00:00:46.680 |
is we're going to do a 20-minute intros and concept just 00:00:52.620 |
and then the majority of the time this workshop 00:00:55.580 |
is going to be spent coding together collaboratively. 00:00:59.040 |
And then we'll have about 10 minutes wrap up in QA 00:01:01.540 |
and a little fun demo for you at the end to get some swag. 00:01:11.540 |
And just so you can get kind of a head start on it, 00:01:14.460 |
if you go to workshop AI agents with Mastra, the Slack channel, 00:01:18.840 |
in the AI engineer Slack, all the materials are in the pinned link there. 00:01:22.680 |
And we can have discussions there as well continuing. 00:01:28.240 |
We build developer tools and make enterprise features happy. 00:01:31.140 |
No, enterprise developers happy with easy-to-use tools like SSO and SAML, 00:01:35.740 |
directory sync, audit logs, fine-grained auth. 00:01:38.240 |
And we're really getting into AI and securing your AI. 00:01:41.240 |
Things like securing your MCP, attaching identity to those agents so that they can act on your behalf and know who you are. 00:01:47.140 |
Yep. Another good way to think about WorkOS is if you, like, let's say Nick and I built the ultimate app this weekend, 00:01:52.780 |
we're ready to go sell it, and then the first time we demo it to somebody, they say, 00:01:55.940 |
"Can you give us a single sign-on for our 2,000 users over here in a different path over here?" 00:02:01.120 |
So then we would go and buy WorkOS or use WorkOS, drop the JavaScript in, 00:02:04.960 |
and we would have those features ready to sell. 00:02:11.800 |
I'm also a host on the dysfunctional.fm podcast, formerly JS Party. 00:02:24.940 |
I'll be at SquiggleConf MCing that, which is a DevTools-focused conference in Boston later this year. 00:02:29.980 |
I'm also a TypeScript enthusiast, as you'll see. 00:02:32.820 |
I'm a Vim user, by the way, and I'm Nick Nisi everywhere. 00:02:40.380 |
I've mostly been an application and infrastructure developer my career, 00:02:44.080 |
but for the last two years, I've really been focused on Gen AI, MLOps, and then retrieval augmented generation pipelines, 00:02:50.340 |
So, yeah, really excited to be here building with you today. 00:02:58.340 |
I'll just embarrass two folks quickly and just point out that we have some core Mastra team members here, 00:03:03.340 |
so this is very exciting, so you'll not only be able to get some assistance from us hopefully, 00:03:06.600 |
but also from the Mastra core team for any questions that go over our heads. 00:03:10.600 |
Mastra is a framework, and so it's a framework for building production AI applications in an agentic manner. 00:03:16.940 |
And these are the components that we're all going to learn and get hands-on with today in this workshop 00:03:20.600 |
and understand very well by the time we leave here. 00:03:22.600 |
So we're going to build workflows, which think of them as composable and typed pipelines, 00:03:28.860 |
Tools that you can optionally use and give to agents. 00:03:31.860 |
Agents are like the interfaces that we're chatting with. 00:03:34.400 |
They're the ideal interface for humans to work with. 00:03:36.860 |
And then the Mastra framework does some incredibly cool things out of the box with batteries included that we'll see. 00:03:41.860 |
So not just a playground that makes it super easy to debug everything, 00:03:46.120 |
but built-in persistence, memory, observability, and even evaluation so that you can determine over time 00:03:52.120 |
how healthy your pipeline is, which is critical for successfully deploying an AI application in production. 00:04:01.120 |
We don't have time in two hours to go over everything, but there's a lot of really cool tools in there. 00:04:05.120 |
So we're going to be focusing on some core concepts for the example app that we're building, 00:04:09.500 |
and we'll kind of understand some of those building blocks as we get going. 00:04:16.160 |
What are workflows? Workflows are composable pipelines. 00:04:19.060 |
So the way that I think about this is I map it in my head mentally to if a coworker says, 00:04:22.960 |
I just keep on having to go scrape the site and download people that look interesting and look up their info 00:04:27.760 |
and then take an image and then write an email. 00:04:29.860 |
You know, that sounds to me like a workflow, right? 00:04:35.600 |
And as we can begin to see with this example code on the screen, 00:04:38.140 |
you can even have control over transforms that happen in the middle of your pipeline. 00:04:42.200 |
So the idea is you're chaining multiple steps together to get some work done. 00:04:48.300 |
And then very importantly, because as we know, LLMs are incredibly powerful, but they're non deterministic. 00:04:55.680 |
How do we coerce their output into something that we can actually build against dependably? 00:04:59.880 |
That's where we're going to validate inputs with Zod. 00:05:02.140 |
But we're also going to learn how Zod can even coerce at runtime and then handle errors gracefully. 00:05:08.540 |
But really, the key idea here is that workflows are a discrete set of steps to get a task done. 00:05:19.200 |
It reminds me of like RxJS, like very similar. 00:05:21.940 |
You're starting with one thing and then doing something and then mapping to something else and then doing something else and running through all of that to give a little bit of determinism to the call of all of these tools as they're being called by your agent or some other tool. 00:05:34.980 |
And speaking of tools, tools are just functions that agents can call. 00:05:40.340 |
They can give you things like file system access, API calls, database queries, custom business logic. 00:05:51.340 |
It gives the LLM the ability to act on your behalf or to read in data that it doesn't just inherently know from its latent space so that it can do more things and be more powerful for you. 00:06:06.560 |
I'll just say that, you know, we think of like AI systems getting better the more data and context they have access to. 00:06:11.900 |
And one of the things that MCP is really excellent at doing is giving, granting context to Claude to OpenAI that it normally wouldn't have and that's not in its training data. 00:06:21.180 |
And one note for today's workshop, we did build everything using tools, but then we switched as we brought in an agent to call and we had the agent call a workflow rather than the tools independently. 00:06:34.400 |
There's something powerful that you should look into on your own and figure out your use case for them. 00:06:38.180 |
But we didn't really use them in the final version of this workshop. 00:06:45.540 |
So we've all probably seen and used the chatbot type demos, but, you know, the way that we're thinking of agents these days is really that they're kind of an ideal human interface. 00:06:53.980 |
So if I set up a workflow that's a deterministic set of steps to get some discrete task done for my colleague, let's say that my colleague is in a hurry or they're busy or they're doing this while they're on the phone or they're not technical, then a chatbot can be an ideal interface for that human colleague. 00:07:08.840 |
Right. Because they can describe their problem or their issue, what they need with natural language, and then the system can be smart enough to actually do what they want. 00:07:18.340 |
This is a very core kind of pattern that we're seeing in a lot of AI applications. 00:07:23.620 |
And so we're actually going to build this and understand this here. 00:07:25.680 |
So after we create our workflow, we'll create an agent and we'll grant that agent the ability to call the workflow. 00:07:31.600 |
Yeah. And agents was something that when I first heard of them, like, like in a lot of these things, like the naming adds more confusion than is really necessary. 00:07:40.300 |
And we spent like an hour on the phone call and I kept being like, but what is an agent? 00:07:43.820 |
What how is that anything beyond just a simple prompt? 00:07:46.240 |
And it's really like the marriage of that prompt plus the workflows or the tools that it can use to do things in a specialized way. 00:07:52.660 |
Yeah. The other way I think of it is like if, you know, in the past, maybe five, ten years ago, you might have written deterministic, more deterministic software with agents. 00:07:59.620 |
You're expecting that the system overall can kind of interpolate or can understand, introspect, what's the state and then make a decision in like a graph sense of we need to actually go here. 00:08:10.560 |
So that's the other piece that you'll hear when you're discussing agentic systems. 00:08:14.740 |
Cool. Another thing, MCP or model context protocol, I told you I'd show you how to spell it. 00:08:21.500 |
This is a way for you to it's an open standard for a way to bring like universal plugins or these tools to AI agents 00:08:30.180 |
This was really developed by Anthropic and you can use it easily in Cloud Code, among other places like Cursor, Windsurf, I think. 00:08:39.180 |
And even OpenAI is is adopting these tools as well. 00:08:42.380 |
So it's really a nice, easy to use API that these tools know how to use and call. 00:08:47.500 |
And to also make it concrete, like you say you work on the DX team at WorkWest, you have constantly crushing a ton of bugs and all various SDKs and you use Cloud Desktop, right? 00:08:56.240 |
And so like what changed the day that you enabled the GitHub MCP server for Cloud Desktop? 00:09:00.780 |
Yeah. The GitHub MCP server was the one that really like made me understand what MCP is doing because I was like, why do I want to do this? 00:09:07.120 |
How is this different than just like having access to GitHub and, you know, calling all of that? 00:09:11.680 |
But it's really an easy way for me to have a conversation with Cloud and for it to bring in the context that it needs based on the tool, the issue that I'm working on. 00:09:20.740 |
For example, if you have an issue on a GitHub repo, I can just say, hey, can you check out that issue and tell me, you know, here's my example app. 00:09:27.940 |
Tell me how I might reproduce this and it can like go figure that out, pull in the comments, understand what's going on, link out to other repos or other examples from there. 00:09:37.500 |
And then look at my example repo and tell me how to to make that change. 00:09:42.640 |
Speaking of MCPs, we built a fun little demo called mcp.shop. 00:09:51.120 |
You can go there now and you can use MCP to order a shirt. 00:09:54.800 |
And it's the only way that you can order this shirt. 00:09:57.280 |
And for the low, low price of zero dollars, you can hit an MCP shirt. 00:10:03.280 |
So it's just a fun demo to to try out MCP, play with it. 00:10:07.240 |
It has instructions for adding it into Cloud or into any of the MCP servers. 00:10:18.460 |
So MCP is, you know, one of the protocols that's really hot right now in the space and a lot of folks are taking notice. 00:10:24.300 |
We recently had an MCP night and it was at the Exploratorium. 00:10:27.640 |
We blew out like the capacity of the event and we had people like wind up around the block, which we didn't expect. 00:10:33.560 |
We're almost certainly going to do something like this again soon. 00:10:36.920 |
So be sure to follow us if you want to hear about it. 00:10:39.160 |
And yeah, we'd love to see you there in the future. 00:10:45.640 |
What we're going to actually build today, though, to to get into it is the AI meme generator. 00:10:49.960 |
So we're going to if you imagine the input to this is going to be you venting any random frustration, 00:10:56.000 |
workplace frustration, frustration with your colleagues or boss into the pipeline and the workflow will process your frustration. 00:11:02.880 |
It's going to find the best base meme that already exists that would be ideal or might be humorous. 00:11:08.320 |
It's going to think about how to actually edit and create new captions and it's going to publish a new meme at a stable URL that's available for you. 00:11:16.800 |
And again, I just mentioned like this is a toy. 00:11:20.400 |
But the patterns that you see here and the master framework are definitely, you know, AI production ready. 00:11:25.280 |
We'll use open AI just for the understanding and image flip is the only other API involved here, which you can get free username and keys for that. 00:11:33.680 |
And there's information in the workshop MD file for that. 00:11:36.720 |
And it's a really fun demo just to kind of showcase some of the tooling that you can do with master and how easy it is to really get all of this going, but also not just to get your tools going, but to test them very easily along the way. 00:11:49.600 |
So we can start with just like a single step and a workflow and test that without an agent. 00:11:54.240 |
And then we can test the agent independent too. 00:11:56.560 |
And we can do all of this and not have to have like a full picture to understand how it's going to work, but kind of iterate along the way. 00:12:03.680 |
And the tooling that master provides is just super nice. 00:12:07.680 |
We're also huge fans of TypeScript, which makes it awesome as well. 00:12:13.360 |
Okay, this is the way that we've structured this workshop. 00:12:18.080 |
And so, you know, we weren't really sure the level that everyone's coming in at. 00:12:21.120 |
And some folks are experts and some folks are just getting started. 00:12:23.360 |
So it's self-paced learning and you can go at the speed that you want to go. 00:12:27.120 |
If you want to go and fly all the way through it, you're free to do that. 00:12:29.760 |
Workshop MD in that repo is the entirety of this course. 00:12:34.320 |
And so you can kind of follow along that way. 00:12:36.480 |
And then we also created Git branch checkpoints for you. 00:12:39.520 |
So in case, for example, you're to fall behind and you get stuck on something and then we get to phase two, you can also skip to phase two and stay, stay with 00:12:46.560 |
everybody just by, you know, checking out the right branch. 00:12:48.720 |
If you get stuck at any point, we're here to help. 00:12:54.400 |
Yeah, if you have questions that other folks likely have or might benefit from in the future, 00:12:59.040 |
please be sure to come up to a mic and we'll make sure that your question is heard. 00:13:02.480 |
And then, you know, the other way we're thinking about this workshop is, again, anyone can read online. 00:13:07.760 |
We can all talk online anytime, but we're actually here in person today. 00:13:10.400 |
So, you know, please shout out questions and raise your hand and ask us things you want to know. 00:13:15.520 |
We feel that understanding is more important than completing stuff here and it's OK to explore. 00:13:27.600 |
So it's it's fun, like snapshot in time to play with this stuff. 00:13:32.080 |
And if you have, like I said, you don't have to finish it. 00:13:34.880 |
We do have the whole workshop.md that you can follow step by step all the way through to the end. 00:13:40.240 |
And then those those branches to to get us there. 00:13:43.440 |
And if you just look at the main branch, that's the completed version. 00:13:50.720 |
Like I said, everything is in the Workshop AI Agents with Mastra channel on Slack. 00:13:59.280 |
There's also the get clone command right there to grab the repo. 00:14:08.080 |
That has everything kind of as a baseline to start from. 00:14:11.440 |
And then you can npm install from there and get going with the workshop workshop.md. 00:14:16.320 |
So here, I think we can stop and you can get started. 00:14:21.840 |
But if there are any questions, please come to the mics and we'd be happy to continue the chat. 00:14:30.400 |
So the before you showed the example of an agent and workflows, that's kind of different 00:14:35.840 |
from what I'm used to seeing in these like agent SDKs where they usually have tools. 00:14:40.320 |
So is the workflow kind of like a tool that I can use or? 00:14:43.520 |
I mean, I feel like correct me if I'm wrong here. 00:14:45.840 |
I feel to some degree that like the create tool and create step APIs are somewhat similar. 00:14:49.440 |
And in my mind, at least right now, it's sort of like an organizational preference. 00:14:52.960 |
Like if I have a simple use case and there might be three different tools and I want the 00:14:57.440 |
agent to decide the right time to call those tools, I might create three separate mostly 00:15:02.400 |
essentially functions, name them tools, grant them to the agent, and then start chatting with 00:15:06.720 |
the agent and ask for things and let the LLM determine at runtime which which one's to call. 00:15:10.240 |
The difference is with the workflow is I'm saying, I always want you to proceed in exactly this 00:15:15.440 |
sequence with these steps and these transforms. 00:15:17.920 |
And then you can also in master at least you can optionally grant an agent the ability 00:15:23.280 |
So, and then also in my conversation recently with Sam, that's kind of like, you know, 00:15:27.200 |
these are, these are various tools that you can build up and kind of assemble into the exact 00:15:36.240 |
Yeah, you can grant because it's basically like you, you're passing like this giant, 00:15:39.760 |
JSON object to the agent saying you can call these tools and you can call these workflows. 00:15:44.400 |
And then the tricky part is like if you give something 100 tools and then ask an LLM like 00:15:49.600 |
you've got 100 tools, let's like go to Disneyland. 00:15:51.840 |
I don't know exactly what we're going to get out of that. 00:15:54.240 |
But that's where like the determinism of a pipeline can be of a workflow can be beneficial. 00:16:00.560 |
Anybody want to raise any, there's no shame here. 00:16:26.400 |
I will also bring up the master docs because these are, oops, these are wonderful. 00:16:35.600 |
But also as Zach and I were working, we discovered something that was actually really awesome. 00:16:44.080 |
Once I installed this, things like my tooling got a lot smarter and knew how to work with Mastra. 00:16:53.840 |
So if you get stuck or if not, and you want to see MCP in action, I would recommend installing 00:17:02.160 |
I installed it into cloud code and into cloud desktop and played with it in both. 00:17:20.480 |
So if you go back to the repo, look at the GitHub repo, like in the browser. 00:17:34.640 |
So go to github.com, work OS, and then the master agent. 00:17:41.600 |
You have to change that from a colon to a forward slash thing. 00:17:44.080 |
So like grab this whole, grab everything after the ampersand. 00:17:48.480 |
And then go up here, and so make, but you got to change the URL. 00:18:16.640 |
It's searched for that URL instead of going directly. 00:18:19.600 |
So make that, if you can, and then change this whole into a forward slash and then get rid of that final git. 00:19:32.000 |
I'll post the HTTPS clone command in the Slack as well. 00:19:41.440 |
In case anyone's having trouble with SSH clone, 00:19:47.920 |
All right, we're going to do about five more minutes set up. 00:20:14.560 |
Feel free to raise your hand if you have an issue. 00:20:44.240 |
You still need that like JSON object, I think. 00:20:46.640 |
I mean, that's what I use for my fancy clients. 00:24:19.560 |
I'm going to do about two and a half minutes until we move on. 00:24:37.560 |
Put it on a flash drive and then we'll pass it around. 00:24:55.560 |
The end goal of this first step is really just seeing the playground. 00:25:05.560 |
And we'll kind of show that here in a minute. 00:25:07.560 |
there's not going to be any tools or agents or anything yet and we'll build those out. 00:25:12.560 |
But it's just seeing kind of what Mastra gives you right out of the box so that you have everything you need to get going fast. 00:25:22.560 |
Why Mastra compared to all the other frameworks? 00:25:25.560 |
The first couple of things I heard were pure type script, right? 00:25:26.560 |
And I'm already kind of bought into that with... 00:25:27.560 |
I like compilers finding my errors before I ship them. 00:25:28.560 |
Then there's the Vercel AI SDK that I think that the team wisely chose to build upon. 00:25:34.560 |
It's kind of mind bending to kind of wrap my head around. 00:25:35.560 |
But the idea that I can multiplex between any LLM provider, I mean, when GPT 4.0 came out, 00:25:41.560 |
I shipped a two character change in a PR and my entire pipeline got way smarter. 00:25:46.560 |
And that was the day where I was like, "This is good. 00:25:50.560 |
So the first couple of things I heard were pure type script. 00:25:53.560 |
And I'm already kind of bought into that with... 00:25:54.560 |
I like compilers finding my errors before I ship them. 00:25:56.560 |
Then there's the Vercel AI SDK that I think that the team wisely chose to build upon. 00:26:00.560 |
It's kind of mind bending to kind of wrap my head around. 00:26:01.560 |
And that was the day where I was like, "This SDK." 00:26:04.560 |
Then I found out that the master came out and it was like Vercel AI SDK and pure type script. 00:26:11.560 |
And I can tell you that there's a ton of stuff that they put in there, like including evals 00:26:17.560 |
So that's one of OpenAI's own recommendations about the right way to deploy agents at enterprise 00:26:22.560 |
and that scale is to start with a evaluated pipeline. 00:26:27.560 |
And I can tell you from working in like vector database companies too, that that's... 00:26:31.560 |
People that are actually in prod are using evaluations to determine the health of their pipeline 00:26:35.560 |
and the health of each individual generation. 00:26:39.560 |
So it's like import and then like also evaluate my pipeline. 00:26:43.560 |
And then finally, so it's like because of that, then it's on the Vercel AI SDK. 00:26:47.560 |
Then being able to deploy it quickly and then have a good experience that way too. 00:26:51.560 |
And honestly, what we'll see just in the local playground was pretty great. 00:26:58.560 |
Like every, everything has issues, but that was the experience I've had with it so far. 00:27:02.560 |
And real quick, I think that for me, like TypeScript moving fast with that. 00:27:08.560 |
I still don't have to figure out what PIP is. 00:27:14.560 |
But also like we had, how many of you have had these like come to AI moments within your 00:27:20.560 |
Like a big, Hey, we're going to have everybody, we're going to work hard with AI. 00:27:23.560 |
We're all doing AI now, whether you like it or not. 00:27:25.560 |
That Shopify post came out and then like, we had it. 00:27:30.560 |
Like everybody that I've talked to, like, Oh, we're going to have this meeting where we're 00:27:34.560 |
We're going to see how way I can, you know, solve every problem in every domain. 00:27:38.560 |
And, um, after we had that, we did like an open workshop thing. 00:27:43.560 |
And we had, uh, like immediately we showed Mastra and then we had like 10 people just 00:27:48.560 |
like start going off with 10 individual demos. 00:27:51.560 |
And cause they successfully built the exact workflow that they were looking to build. 00:27:55.560 |
And so then for me, the reason I'm the most excited about that piece of it is that I think 00:28:01.560 |
I think all of us are rapidly experimenting with this stuff. 00:28:04.560 |
And you know, do I want to live in a world where Nick's writing bespoke LLM scripts while 00:28:07.560 |
I'm sleeping and I wake up and I'm like, what is this spaghetti mess? 00:28:09.560 |
And he has to do the same with my, you know, crap or are we all using that? 00:28:14.560 |
Or are we all using like, you know, an actual framework that we can agree on certain primitives. 00:28:20.560 |
There's, there's even forethought I can see, um, in the way it's been organized so that once 00:28:24.560 |
you've defined an agent, a different project can reach in and call that agent via code or API. 00:28:29.560 |
So, um, all those reasons, I'm kind of excited to use it as sort of a unifying internal tools 00:28:39.560 |
This might be a little premature, but, um, step one, number five, where you, so you create 00:28:43.560 |
a workflow, you pass a discrete array of steps, um, which of which there's only one, which makes 00:28:50.560 |
And I would think you would just, I guess I'm asking the design decision behind then needing 00:28:55.560 |
to just also explicitly do test workflow dot then, and then invoke the step. 00:29:05.560 |
The ergonomics around, are you asking about the ergonomics around the commit method itself? 00:29:11.560 |
What's the, what's the, gentlemen, what's the decision behind that? 00:29:16.560 |
Um, I've looked at it recently and I actually checked that API today because I was curious 00:29:22.560 |
I actually argued with Nick because he correctly said, no, commit finalizes your pipeline, returns 00:29:31.560 |
What's the, what's the final idea behind commit? 00:29:32.560 |
Uh, that, and then, I mean, so, for sort of the local dot is kind of like a . 00:29:46.560 |
It's the final step that's like actually creates the whole object and adds the steps and returns 00:29:53.560 |
But why, sorry, I don't mean to berate this, but like why the dot then and then the name 00:29:57.560 |
Um, is that, is that step not already in the workflow steps already there? 00:30:01.560 |
Um, so you should be able to, um, so there's sort of two ways of defining steps. 00:30:08.560 |
Can you, I'm sorry, would you mind coming to the mic? 00:30:14.560 |
Um, uh, so there's, uh, there's two ways of, uh, coming to, of defining, um, steps. 00:30:20.560 |
You can either give it a step array, um, which as you might expect is kind of, um, the 00:30:27.560 |
most, a lot of times we just have these linear pipelines of steps. 00:30:30.560 |
And so you just give it a step array and it will execute things in a linear pipeline. 00:30:34.560 |
Um, or if you want to add sort of like branching and conditionals and a lot of that kind of 00:30:40.560 |
Um, we have this like fluent syntax where you can do dot then, and then you can do dot 00:30:44.560 |
if dot branch, dot do while and like loop and, and do conditionals and, and all the fun, 00:30:50.560 |
complex stuff, you know, but a lot of times you're just, you're running through the pipeline. 00:30:54.560 |
You also just do test workflow document and that it would run. 00:31:01.560 |
For what it's worth, I have also recently had the experience where I had built two different 00:31:05.560 |
Gen AI apps, like experimentally internally and realized that I had implemented a pipeline 00:31:10.560 |
And that was another reason why I was like, okay, I'm excited now about a single framework 00:31:18.560 |
We've mostly got packages installed or anybody getting into a hundred. 00:31:58.560 |
It's not, uh, you'll be able to catch up quickly or you can skip to the next step, which I'll 00:32:06.560 |
So I'm going to just, um, switch over to steps slash step, uh, one, which is the end of step 00:32:33.560 |
I didn't know how to spell switch apparently. 00:32:37.560 |
So once you have it all installed, then, uh, you really should just have kind of a master 00:32:44.560 |
It also has memory, um, set up for you automatically. 00:32:47.560 |
Uh, but this just creates the master instance. 00:32:50.560 |
Um, currently we have nothing enabled, no agents, no workflows, uh, no tools, and that's 00:32:58.560 |
And when we, oops, when we run NPM dev, uh, we will get this URL localhost 4,011. 00:33:06.560 |
And if we go there, this, nope, this is the, the end of step one, which is maybe, there we 00:33:15.560 |
Uh, you look here, you have agents, networks, tools, MCP servers, workflows, and then this runtime 00:33:24.560 |
Um, and that's currently by design because we've done nothing. 00:33:28.560 |
We've just gotten the project set up, but now we have all of these tools to help us with building 00:33:33.560 |
And so, um, once you get this set up, uh, you should be able to switch over to step one, 00:33:41.560 |
I think it might actually be the same because it's just NPM installing. 00:33:44.560 |
Um, and then you should be able to continue on in the, uh, the steps. 00:33:51.560 |
Uh, this is, I'll just say that this is one of my favorite things though about master, 00:33:59.560 |
And if you look in the console, you'll see that there is also a couple other things spun 00:34:04.560 |
There's also a, uh, swagger spec that you get automatically. 00:34:07.560 |
And so like the amount of thought that's been put into this, and then you could see how 00:34:10.560 |
once everyone's comfortable with it, you could, you know, very rapidly deploy applications in a 00:34:14.560 |
standardized way that other tools can consume, uh, reliably. 00:34:17.560 |
So that's, um, another exciting feature of it. 00:34:20.560 |
And again, this is like without us having built anything yet. 00:34:23.560 |
So the goals that we're going to do in step one now, uh, is just create our first workflow. 00:34:33.560 |
Uh, but it's going to be one workflow with a single step and it's not going to do anything 00:34:40.560 |
Uh, and then we'll build upon that in the next step to add additional steps. 00:34:46.560 |
Um, and then we will, we'll, uh, be able to put that together into a full workflow in 00:34:52.560 |
But we will more importantly, we'll be able to see it in the master playground and start 00:35:00.560 |
As always, any questions, uh, feel free to shout them out. 00:35:22.560 |
Um, and so, um, yeah, if you have questions, we'd love to, to chat or just ideas of this stuff. 00:35:36.560 |
Cause I think there's a lot of really cool powers, uh, that you can do with this. 00:35:43.560 |
Uh, I'm just kind of curious since you're obviously using Vim and kind of deep in the hole. 00:35:49.560 |
Have you played around with any of like the AI integrations into Neo Vim? 00:35:52.560 |
Kind of like how, what's your current workflow? 00:36:03.560 |
I haven't used cursor yet at all because it's not Vim. 00:36:15.560 |
I think that in Vim, I haven't played with that one yet, but it's on my list to play with. 00:36:18.560 |
Uh, the one that I have played with is Avante, uh, which I think I have still set up. 00:36:25.560 |
So it gives you kind of that like chat like interface here. 00:36:28.560 |
So I can just start having, uh, a chat with this code. 00:36:34.560 |
There's also, was it Neo AI use that for Neo Vim? 00:36:37.560 |
Even a year and a half ago, you could do that before, before most of the tools allowed you to chat in IDE. 00:36:44.560 |
That's always gonna be the benefit of open source is it lands first, right? 00:36:47.560 |
Um, but I honestly, like I didn't like the ergonomics of playing with this and like switching buffers and things like that. 00:36:53.560 |
And I honestly felt that, uh, for me, uh, a better workflow was just not using this at all. 00:37:02.560 |
So, um, you know, I will have like Vim open and then I'll just open a split. 00:37:06.560 |
And Claude, Claude code, um, gives me that kind of the same thing. 00:37:13.560 |
So I can work in Vim as I normally would without like weird buffer things. 00:37:16.560 |
And then it's just off doing things and I can, uh, you know, whoops, use, um, use T mux to like full screen Vim when I need it. 00:37:27.560 |
And then when I want it, I just bring it back in and go. 00:38:18.560 |
I would also be interested in hearing how all of you are using these AI tools. 00:38:22.560 |
Just for, like, not just what you're building, but how you're interacting with them day to day. 00:38:28.560 |
And I feel like everybody's, like, in their own bubble doing their own thing. 00:38:31.560 |
And then we, like, we have this opportunity to come together and learn, like, amazing new tips and tricks. 00:38:43.560 |
I don't know that I'm all for everybody getting on the same, you know, plus. 00:38:51.560 |
I think it should depend on, on, you know, product or, you know, service or whatever. 00:38:56.560 |
But I'm, I'm afeard of, of, you know, somebody coming down from on high with, like, this is the golden framework. 00:39:13.560 |
Like to mix and match and let everyone kind of choose their own lane. 00:39:24.560 |
To summarize for the video, mixing and matching tools is good. 00:39:27.560 |
And I think that that's really what is, like, powering this right now, especially among, like, us hackers, right? 00:39:33.560 |
It's a whole new world of figuring out what to do, how to do it. 00:39:40.560 |
I don't want to shame any specific editor, but. 00:39:59.560 |
There's all sorts of experimentation, all sorts of fun. 00:40:02.560 |
Also, I do worry that it's like, you know, the first hit is free and then we're all like, 00:40:13.560 |
Do you have a framework specific, like, set of rules, like, that would integrate? 00:40:18.560 |
Not that everyone has to use cursor, but just like, all of a sudden, like, at my company, 00:40:24.560 |
we've started to essentially say, hey, here's the top 10 tools that we use. 00:40:28.560 |
And here's cursor rules to do all of those the same kind of way. 00:40:31.560 |
And it's not so much trying to, like, box people in, but it's more trying to say, give 00:40:35.560 |
them, like, in Mario Kart, it's like, you get the three booster packs. 00:40:39.560 |
It's like, how can we give people booster packs from the beginning? 00:40:43.560 |
We were talking about this earlier, actually, with Nick Taylor. 00:40:46.560 |
we were talking about, like, cursor rules growing and evolving kind of like tests. 00:40:50.560 |
And then it's like this thing that evolves with your code base to prevent you from doing bad things. 00:40:56.560 |
I don't know what that means, though, because I've never used cursor. 00:41:00.560 |
It's not exactly the question you asked, but I will just say that that's also, like, when you run, you know, NPX install 00:41:06.560 |
mastra, it asks you, are you going to use cursor, are you going to use windsurf? 00:41:09.560 |
Because if so, I know I can inject the MCP server for our docs. 00:41:12.560 |
And then the experience that I had after accepting that, I saw, it's like, oh, it enabled that. 00:41:20.560 |
And then that session where I coded, there were zero hallucinations. 00:41:25.560 |
And so that means that every time I asked for a feature, it was one-shotted correctly. 00:41:31.560 |
Furthermore, because it was in TypeScript, I could quickly lint it and compile it and know it was good. 00:41:35.560 |
And so that's almost like, we were almost starting to, Nick and I were starting to suspect that MCP might be more powerful. 00:41:41.560 |
Because the other problem is the cursor rules are specific to cursor, right? 00:41:44.560 |
So what if, like, half your team's on windsurf, half's on cursor. 00:41:47.560 |
You know, we've got tons of licenses for cursor internally. 00:41:49.560 |
So it's kind of like, but it's not prescriptive. 00:41:51.560 |
I think the MCP docs server so far is more powerful of a pattern because it is, it's almost like the, the guarantee that the end user dev is going to get your latest docs is not going to get a hallucination. 00:42:04.560 |
And I really wonder if that's like, like, there's so much excitement around that. 00:42:08.560 |
We saw that at MCP night a couple of weeks ago, like there's so much excitement around that. 00:42:11.560 |
And I'm just starting to wonder, is it really just because it's like a standard emerging? 00:42:16.560 |
Um, and that's the reason because it's, it's very simple. 00:42:19.560 |
Uh, but it's, it's enabling all of these tools across, you know, it's not just an anthropic thing anymore. 00:42:30.560 |
I'm not sure if this is a better question for y'all or for the master team, but I saw that master has experimental support for ingest, which is a durable execution backend, but that it's labeled as experimental. 00:42:38.560 |
And I was curious, like how experimental that is because I'm really excited about durable execution for agents. 00:42:45.560 |
You know, if you've read the 12 factor agents manifesto, it's really good. 00:42:51.560 |
Uh, it's on GitHub, uh, Dex from human layer. 00:42:53.560 |
Um, originally did a lot of the work on that. 00:42:55.560 |
Um, but yeah, I just was curious about jamming on that. 00:43:00.560 |
Unfortunately, we have the folks in the room for that. 00:43:06.560 |
Um, uh, so the, we shipped in, uh, sort of a, what we call workflows V next about three, 00:43:16.560 |
Now, um, the idea being that basically it allows you to have the same syntax, but switch out the, 00:43:25.560 |
So the first, um, the, the first, uh, integration that we shipped, we, we sort of initially had, 00:43:31.560 |
we initially built our workflow engine using X state as a backend, because it kind of makes 00:43:37.560 |
Um, and then we kind of, we, we need to make a couple of syntax changes. 00:43:40.560 |
And then we also kind of realized that, uh, people started asking us, Hey, like, I love 00:43:44.560 |
the syntax, but I'd love to have this work with, and I love the visualization in the dev server, 00:43:49.560 |
but I would love to have this work with ingest or temporal, um, or Cloudflare work workflows 00:43:56.560 |
Um, and so we're like, yeah, that's a really good idea. 00:44:01.560 |
And then now we're just starting to, you know, do the list of integration. 00:44:04.560 |
And so injustice first, um, it's, you should use it. 00:44:08.560 |
Like it's experimental cause we shipped it like a week ago, but I would be surprised if 00:44:19.560 |
Um, just curious what, how does the team think about, uh, deployments going forward? 00:44:24.560 |
Like are you targeting specific platforms first? 00:44:27.560 |
Like, are you thinking about trying to go everywhere? 00:44:31.560 |
Uh, so by default, um, by, by default like master will bundle into a Hono server. 00:44:36.560 |
Put the Hono server in a Docker container and put it on EC2 or digital ocean or wherever you 00:44:42.560 |
We also have adapters for Cloudflare, Netlify, Vercel, we also have our own monstrad cloud. 00:44:51.720 |
So those are kind of the two, Hono is not a, you know, if you're only using Hono, great, 00:45:00.320 |
but also you can put it in Express or whatever, you can put it in the backend of a Next.js 00:45:09.320 |
operator app, you know, in general, I think like, you can't really, I mean, you can call 00:45:16.460 |
it open source and it cannot work everywhere, but like these days you should actually make 00:47:17.720 |
I was wondering, is there anything inbuilt into the framework that ensures that the output 00:47:35.220 |
of each step matches the schema kind of by default? 00:47:39.080 |
Because what if like you specific, like, you know, let's say in the prompt, you kind of 00:47:43.320 |
forgot what your schema was and you like, how does it reconcile that as well? 00:47:49.220 |
So I think the answer to that is essentially the Zod schema, right? 00:47:53.080 |
And the way that I look at that is like, LLMs are non-deterministic. 00:47:59.940 |
But then it's at the point where they're so useful. 00:48:01.940 |
We're trying to get specific JSON out of them while coding too, right? 00:48:05.080 |
And the Zod package and like validation library is now ubiquitous in AI engineering, at least 00:48:16.680 |
My understanding is that the open AI team couldn't get to a certain level of accuracy without adding 00:48:21.180 |
on top of training the Zod runtime validation, which can also do coercion into the right types. 00:48:28.180 |
So that's kind of doing the heavy lifting there. 00:48:31.000 |
I don't think there's anything framework specific that's doing structure generation, but correct 00:48:34.560 |
me if I'm wrong, but you can define like the transform steps between the workflow. 00:48:40.020 |
There's actually, we actually just did a blog post on this, but it's funny that the models 00:48:45.720 |
don't always actually respect the schemas that you give them, especially there's some, you 00:48:50.680 |
know, yellow, okay, Z dot string, like they'll respect that. 00:48:53.420 |
But if you know Z dot string dot, you know, length, you know, four or whatever, like they're 00:48:57.580 |
not necessarily going to respect that like level of, of nuance. 00:49:00.800 |
So we actually added, we added sort of four, four, this is especially challenging. 00:49:08.580 |
This is mostly worked, works reasonably well structure generation is more challenging with 00:49:13.340 |
tool calling for reasons that are not quite clear to us, but we're clear when we ran 16, 00:49:20.000 |
you know, with the 30 schemas across 16 models, so we added like a tool compatibility structured 00:49:26.980 |
layer that reduced, that like reduced the error rate a lot. 00:49:31.200 |
So in addition, on top of the side, like there's also master side, like optimizations 00:49:38.980 |
The, the longer version of this is in a blog post we published over the last week. 00:49:42.200 |
If you're, if you want to dive into the details. 00:49:44.200 |
And I don't know if it's related to this, but one thing that we were running into constantly 00:49:48.420 |
while we were developing this was we were like, as you'll see, we were accidentally sending, 00:49:53.420 |
sending like a base 64 encoded image, which was huge and failing and it would retry a lot. 00:49:59.420 |
So I assume that there's some retry logic built into it too. 00:50:02.420 |
If like it wasn't getting back what was expected from the schema. 00:50:06.640 |
Um, is it built into the, the schema validation piece? 00:50:09.860 |
I mean, I would assume probably what was happening was that the agents has like a max steps. 00:50:15.860 |
It was running until in a loop until hit max steps in it, but it, I guess what I'm asking 00:50:21.200 |
is if you, if you said, I want a string that's length four and I gave back a length five, would 00:50:25.640 |
it automatic would master automatically retry? 00:50:28.080 |
There are cases in which it would, it would retry. 00:50:30.680 |
I don't know if that particular case is one of them. 00:50:34.740 |
Does that, yeah, that does answer my question and to kind of, to add to that with, if you 00:50:39.460 |
were to specify a certain things into the schema that aren't inherently like supported, for 00:50:44.280 |
example, the, the length would that, um, would that be kind of like, would that error just 00:50:56.780 |
I would expect it's odd to coerce it actually at runtime, um, but I'm not sure. 00:51:02.040 |
Usually the model will fail silently and just give you a output that doesn't conform to the, 00:51:09.760 |
um, to the, um, to the, uh, to the schema and we kind of made a decision. 00:51:19.600 |
You, maybe you should decide if you want to error. 00:51:24.020 |
Uh, so just talking about deployment again, I, I noticed the server was the point is there's 00:51:42.020 |
Um, there's also a AWS Lambda, um, the player that the community had. 00:51:47.100 |
If you, if you Google, you'll probably find that you have issue. 00:51:51.100 |
And how does the, the serverless deploy work? 00:51:53.100 |
Does each step kind of map onto its own Lambda function or how's that? 00:51:58.100 |
I mean, we just sort of pack it, bundle it in a, in a, in a, in the, a platform specific way 00:52:05.940 |
So like Cloudflare has like a 300 meg, um, uh, bundle size limit and so we need to do some specific 00:52:13.600 |
I guess I'm just being in terms of the runtime model that each step kind of maps logically 00:52:19.140 |
to a function or are you putting multiple steps inside a single function? 00:52:23.680 |
I did not work on that part of that, so I would have scored. 00:52:26.960 |
I'm assuming they're all bundled into one, but yeah, I mean, I think they're all bundled 00:52:36.060 |
That would just be weird for long running stuff. 00:52:41.060 |
We, we all, we bundle it all into one, I'm pretty sure. 00:53:06.060 |
So we can kind of show that I'll go to, um, step two. 00:53:15.060 |
Um, and so now like looking at the code, uh, we've got, if I update this, we've now got our 00:53:28.060 |
Um, and then we've got a workflows directory with the test workflow and the schema for it. 00:53:34.060 |
And it's just going to have you, um, basically extract from the user input, uh, the type of 00:53:42.060 |
And the really cool thing now is like without building anything else, uh, we can go to our 00:53:47.060 |
master playground, refresh it and go to workflows. 00:53:51.060 |
And then we've got our test workflow right there and you can see the graph here. 00:53:58.060 |
Isn't very impressive with just one step in the workflow, but it's there. 00:54:03.060 |
And we can see that it would accept a raw user input about work frustration. 00:54:12.060 |
Please state the company you work for and your boss's name. 00:54:29.060 |
So we can see just by calling that it turned green in the, in the graph and you can see that 00:54:35.060 |
we click on it and we can see that that's the input that we gave. 00:54:38.060 |
It ran the step and it gave us back this output, this, uh, Jason object. 00:54:44.060 |
Uh, and we can see with the text conference wifi sucks. 00:54:50.060 |
Uh, and the analyzed frustration is technology. 00:54:53.060 |
Or so we're frustrated about technology in this room. 00:55:09.060 |
Uh, in the next step, um, we're going to create our frustration schema with Zod, uh, build our 00:55:20.060 |
In this one, we're going to, uh, create all of the workflow steps. 00:55:24.060 |
Uh, we're going to chain them together with data mapping. 00:55:27.060 |
Uh, and then we're going to successfully generate a meme. 00:55:31.060 |
Uh, and we'll understand the composition of that workflow as we go. 00:55:34.060 |
Uh, it still won't be like the perfect end result, but it will give us the full workflow where we 00:55:39.060 |
can like kind of plug it in, in that, uh, workflows tool and see it go through all of the steps 00:55:45.060 |
from our user input to an actual meme that's, uh, uploaded to image flip. 00:55:55.060 |
So you can skip to step two, uh, or continue on in your branch and, uh, continue from there. 00:59:54.020 |
Is this something, an object that I would then pass to another person? 00:59:59.220 |
And it's just right now, we're just in the early stages of showing exactly that and that 01:00:02.720 |
you could have structured this object however you want. 01:00:05.640 |
But in our case, but in our case, because of the pipeline that we're building, this is the 01:00:10.060 |
And so we formed it this way and then we're going to pass it further into the pipeline to 01:00:27.160 |
If I want markdown, if I want JSON, if I want whatever. 01:00:32.460 |
And that's part of what Zot will help you do too. 01:00:34.620 |
Because you can define like, maybe you have some giant object and this may fields and this 01:00:38.660 |
is a string and this is an array of strings and all that. 01:00:40.720 |
But it's really for you to compose together the exact structure that you need. 01:00:44.280 |
You would have to know what the next stage is. 01:00:52.340 |
A forethought or the third time you've written that app. 01:00:55.380 |
You've had enough, you've had two shitty first drafts and you're ready for the third. 01:01:01.340 |
But yeah, sometimes I'll plan it out and be like, what's the logical breakdown for these 01:01:06.620 |
The main point of this one is to just determine like the level of frustration and what you're 01:01:11.520 |
frustrated about so that it can be fed in in the next step to image flips API to get back 01:01:18.460 |
an image template that, or a mean template that would accurately fit that. 01:01:24.700 |
And then we kind of shoehorn it in from there. 01:01:26.760 |
But again, the like, I guess the mental mapping that we're all trying to do here too. 01:01:31.320 |
It's like to think about how do we take this home, then like turn it into something that 01:01:35.500 |
actually solves a problem or automate something at work. 01:01:37.440 |
So we get promoted before the machines come for us. 01:01:50.660 |
And just so we're level setting, this is like the memes that you've seen. 01:01:54.760 |
Image flip is a, they have an API for not only finding great memes, but also just having blank 01:02:01.000 |
So they, they can give us back information about this, the rock meme, the rock in a taxi meme 01:02:09.860 |
And then you can post to their API with whatever you want it to say in each of those. 01:02:14.440 |
And it'll tell, tell you like, Oh, I have three boxes. 01:02:18.620 |
And that's what we're eventually going to be doing in one of these steps. 01:02:21.320 |
But that's, that's a perfect real world example. 01:02:23.320 |
It's like knowing those box slots is critical to the rest of the pipeline. 01:02:26.360 |
If any other step in the pipeline didn't know the box slots correctly, right? 01:02:30.340 |
So you can define that in your schema and make sure that it's there and like that's part 01:03:05.780 |
putting this workflow together too, is probably the biggest of the steps. 01:03:12.420 |
Once we have this, we'll, we'll create an agent in the next one. 01:03:14.880 |
And that's pretty straightforward, as you'll see. 01:03:18.540 |
And so there's, there's time built in also to, to play with this experiment, tweak things 01:03:23.000 |
like the, like the prompt temperature, temperature, things like that. 01:03:29.220 |
That's the cool thing is like, once we get the agent and start chatting with it, you know, 01:03:33.120 |
like there's just a, iterating locally is so much faster, right? 01:03:36.760 |
And that's really what the playground's about. 01:03:38.760 |
So you can even tweak the prompt on a per request basis. 01:03:50.540 |
And, and, and, and, and, and, and, and, and, and. 01:07:30.300 |
Does it sound like the credentials are being... 01:07:40.300 |
Yeah, so in general, if you get an error at the generate meme step and you don't get 01:07:44.100 |
a clear error message and you have not created an image flip username and password, that's 01:07:50.340 |
And then if you sign in with Google, which is the same thing I did, that's fine. 01:07:57.100 |
And then you just have to go into image flip settings after and set a password because 01:08:02.340 |
you're OAuth-ing in so you didn't get to set a password, so the complexity is in the UI of image flip. 01:08:13.100 |
So in our testing, there's a public key that floats around. 01:08:21.860 |
So there is an anonymous key that's public, like git guardian will flag. 01:08:24.860 |
Others consider it a secret, but apparently it's not, and people are using it. 01:08:28.860 |
And the reason that we finally asked people to make their username and password is that it 01:08:35.620 |
And so you're less likely to run into this issue in the workshop, at least. 01:08:45.100 |
Actually, now I'm getting just like 20, but I don't know if it's just like you cannot run 01:09:03.100 |
You have no idea the struggles of trying to do a workshop with, when API keys are involved. 01:09:21.000 |
We originally had a step that it would not, it would take an inspiration from an image flip. 01:09:31.480 |
But in order to do that, you would have to go in and verify your OpenAI account, which means 01:09:37.480 |
like you would have to scan your passport to be able to do it. 01:09:40.480 |
We thought that might be a little much to ask in this workshop. 01:09:41.480 |
We thought that might be a little much to ask in this workshop. 01:09:51.480 |
Can you guys say more about like how you're, I guess, like interacting with these workflows 01:10:18.480 |
You said something about Vercel, like are you deploying them and then contact them through 01:10:22.480 |
like an API call or like, are they actually embedded into your application? 01:10:26.480 |
What I'm primarily doing right now is you still using the Verceli ISDK. 01:10:30.480 |
So everything I've got, like at least at work that's deployed already, that's mostly Verceli 01:10:35.480 |
And then we're building a ton of like internal tooling with master that started like two, three 01:10:40.480 |
And most of our stuff goes to Vercel just because we already have that and it's pretty fast. 01:10:45.480 |
And then most of the things I've built so far are web applications that might have a pipeline. 01:10:50.480 |
So there's a UI and there's still a user kind of requesting generations and asking for things. 01:10:55.480 |
What I think would be super cool and powerful though is like the workflows expose like an API 01:11:00.480 |
endpoint to just start them and hit them from anything. 01:11:03.480 |
And so I think that's also like a pretty powerful pattern. 01:11:05.480 |
I've not really used it really fully right now in Prod yet, but that's something I would 01:11:12.480 |
Workflows that need to run like every night, you know, maybe have a cron that fires that or 01:11:22.480 |
We've got about five minutes and then we're going to move to the next step. 01:11:36.480 |
So if anyone's stuck on anything or has questions, don't be shy. 01:11:52.480 |
Speaker 2: Does anybody have the memes generating right now through the workflow? 01:12:10.480 |
One, two, three, four, five, six, seven, eight, nine, 10, one, 12. 01:12:28.480 |
When you're dealing with memes like this, it really brings out the non-determinism of this. 01:12:34.480 |
If it generates offensive ones, we tried to curb that. 01:12:43.480 |
Speaker 2: If people who didn't want to raise their hands, I'll just go around and give 01:12:50.480 |
Speaker 2: If you want a free signed, are we talking autographs here, Sam? 01:12:59.480 |
Speaker 2: Autographed first edition from the CEO of Master Yourself. 01:13:02.480 |
If you got a meme working and you want, or you just want a book, raise your hand. 01:13:35.940 |
yeah i'm building agents and typescript got it 01:14:02.480 |
yeah i got some pretty racy ones by saying why did they put a systems engineer on an open floor plan 01:14:10.100 |
with the sales folks that are screaming all day that's a fun prompt 01:14:49.800 |
and there's something called daggs and airflow which are for whatever reason culturally they've caught on 01:14:55.840 |
um as you guys have designed workflows have you thought of daggs are they similar dissimilar 01:15:00.800 |
are you talking about directed acyclic graphs yeah control what does the control flow look like for 01:15:06.800 |
workflows so far stuff that i i've seen you know supported by the semantics of like a master pipeline 01:15:12.700 |
honestly um to be perfectly frank the place that my mind goes with that is just graph rag 01:15:17.360 |
like a graph plus retrieval augmented generation so like some vector database plus a graph 01:15:23.860 |
um to also answer like structured questions about entities in addition to like semantic search 01:15:29.180 |
capabilities um that's honestly what i've built mostly or i've seen people kind of like uh like 01:15:35.880 |
positing is going to be a next gen architecture but so far what i've needed has just been 01:15:41.100 |
pretty simple pipelines basically run these steps branch over here if this happens 01:15:45.280 |
um you know i see the dag is like that's like terraform in my mind from from devops like 01:15:52.560 |
land of like it must proceed this way and the entire thing fails if if like node 3 uh is unhealthy 01:15:59.200 |
kind of yeah that's a great question we were actually nick asked if we could trigger a specific 01:16:16.460 |
step i'm not sure but the other thing i'll say is that in those cases that's the other reason i love 01:16:21.340 |
the pipelines and also even just the concept of steps and agents because i can do a specific model 01:16:25.780 |
for each one so like extracting frustrations is like that's pretty simple and should be fast and 01:16:30.260 |
cheap so it's like do 4.0 or even something simpler or maybe even get away with like a 7 billion 01:16:35.060 |
parameter model that's deployed on like cloudflare workers right and then which it's like essentially 01:16:39.680 |
free um that's like and that question by the way this is like the uh like how do you right size ai 01:16:45.820 |
application workloads like that is top of mind for everybody now everyone's asking that questions 01:16:49.900 |
like how do i get the infra to be what's correct for the workload i have i think everyone's trying to 01:16:55.500 |
figure that out and another thing like you can explore different ways of doing this uh when we 01:17:00.100 |
first started with this we had all of these steps as tools and then kind of let the ai decide when it 01:17:05.380 |
should call them um so that could be another approach depending on if it can intelligently skip steps 01:17:10.600 |
um it might be able to do that um there was a question is this is in our chat right 01:17:16.780 |
is it okay so there's a question is is there a way to implement or configure caching with master i think 01:17:21.900 |
the answer is yes yeah for sure yeah this so i did i did i did the uh image 01:17:31.780 |
the same here show me the your dot n file you would 01:17:38.960 |
okay yeah right that looks great to me assuming those are your actual credits yeah and the other 01:18:00.340 |
other thing is did you uh did you restart in family i i saved it and then i 01:18:08.780 |
oh so restarted now because i bet you did you update them while that was wrong so if you 01:18:22.600 |
restart it it should um pick up the new dot n and let's see if it's the same or not 01:18:27.160 |
uh yeah i thought it was because it's two words instead of one and i still have a 01:18:35.600 |
is there an extra space anywhere and because i had that issue even after i had the correct values like there has to be 01:18:53.980 |
oh it'll take a few weeks probably oh is it not going through 01:19:02.420 |
oh it'll take a few weeks probably oh is it not going through did it try and execute the tool 01:19:19.860 |
uh what uh different key what um i don't think uh clandry is right no no no no it's 01:19:26.580 |
this is it should it should be relatively fast 01:19:29.980 |
second time trying it's just waiting for approval 01:19:49.680 |
uh so if you go if you have it installed which i do 01:19:58.820 |
set up this is claude by the way uh hey bro i here i can get a shirt 01:20:15.500 |
ah i can't provide physical shirts or merchandise that's interesting 01:20:44.960 |
hey bro here i can get a sweet mcp shirt now that we're maybe connected 01:21:01.440 |
you can definitely get an mcp shirt there's one available right now 01:21:13.400 |
i don't know that it says context is everything but it might 01:21:33.840 |
cool so it just needs for me my company name my mailing address and my t-shirt size so i'll say 01:21:42.800 |
we'll say excel um one infinite loop and company name uh work os 01:21:57.360 |
oh it needs my city state and zip code i don't i don't know what that is for apple 01:22:10.560 |
oh and look at that it's going to buy my shirt 01:22:26.660 |
we will we will actually send you a shirt so be sure to yeah be sure to submit it while you're here 01:22:33.660 |
you can also ask things like um can you tell me about my order 01:22:39.040 |
uh it's deployed on is this you put it on cloudflare yeah yeah i have a version deployed on cloudflare 01:22:51.960 |
um this one i think is it might be the original mcp shop yeah probably in versell i think it's yeah 01:22:57.240 |
it's deployed to versell um for this specific instance of it i i have a talk on thursday where i'm 01:23:02.960 |
also i have the same thing deployed to cloudflare so um we'll go into the code and i can send out 01:23:08.860 |
the repo for that but um yes that one just saves it saves the info to a kv store uh and then that's 01:23:16.500 |
that's what's used to order the shirts uh you can deploy mcp servers to cloudflare now using work os 01:23:21.700 |
as the auth layer yeah so if you check that we have a blog post and a couple of tutorials 01:23:26.060 |
yeah yeah that's the thing it's using work os to do that off you didn't see it because i'm already 01:23:30.380 |
often but when i hit connect it opened up and it was do completing that offload to get me a token 01:23:36.280 |
so that i could do that here and you can see that in here it's got like my order information 01:23:43.020 |
um i don't know do you know my name and email which is totally not something i've said yet 01:23:52.980 |
but but also to tie it again uh back together right because again like toy it's fun got a t-shirt 01:24:00.340 |
but imagine this as an e-commerce workflow right this is what people are building like this is what 01:24:04.760 |
everyone's trying to figure out how to do correctly what i actually want to do is just go i'm in a 01:24:08.340 |
hurry i want to get on the page and say it needs to be a product exactly like this does it support 01:24:11.980 |
that okay fine go not 40 less than that right that's what everyone wants to do um and so that's 01:24:17.220 |
what this interface is yeah pretty soon you'll be social engineering your way through llns to get a 01:24:21.860 |
free shirt when it's not free yep i can't wait for the future 01:24:26.660 |
uh cool yeah let's let's uh look at the workflow so i switched over to the step three branch which is 01:24:36.340 |
the end of step two uh where you should be at and you've got a new meme generation workflow uh and it's 01:24:43.140 |
got you know a bunch of schemas set up for the input and output schemas uh and all of the steps set up to 01:24:51.060 |
run each of these and you know that's the code for it let's go look at the the actual get out of here 01:24:57.940 |
um the tools in master so i'll refresh this and go back to work no workflows there we go now we have 01:25:07.940 |
our meme generation workflow and it's got four steps and so this is a little bit better a little 01:25:12.740 |
little more detailed in what what it's doing um it's got not including these mappings it's got our 01:25:18.580 |
extract functions find base meme generate captions generate meme oh come on uh and then a final mapping 01:25:27.300 |
um and so i don't know someone give me a work frustration 01:25:37.300 |
when no one in the workshop is someone was too busy preparing a workshop to review my pr 01:25:48.100 |
nick nick no never and won't review my pr that's a made up one um so yeah it can run you can see it 01:26:00.580 |
running um and then we can go we see all of the steps are now green and so we can see the input and 01:26:08.340 |
debug the input and output on each of the steps so there's the input that we got and it gave out a 01:26:26.180 |
but we can have it find the base meme and then generate the captions for it 01:26:31.700 |
find out what that is so when the meeting could have been an email but everyone is just smiling like 01:26:38.100 |
it's fine that's relatable um but then down in the generate meme one that's going to receive the 01:26:44.980 |
captions it's going to receive apparently a meme called disaster girl and 01:26:50.340 |
did i mention that these are non-deterministic and i don't know what it's going to generate 01:27:00.740 |
ah the meeting could have been an email but everyone is just smiling like it's fine 01:27:05.140 |
this is fine yeah um yeah so this is fun right we can see the workflows the inputs and outputs of it and 01:27:13.620 |
we can debug those steps along the way we can see exactly what is what we're starting with and how it 01:27:18.500 |
progresses through each of the steps uh to give us a full workflow which is awesome um we haven't even had to 01:27:26.180 |
do a deployment yet and we can just iterate locally change the prompts change the schemas until you get 01:27:30.820 |
it exactly the way you want and then you haven't wasted time going back and forth you've like 01:27:34.420 |
iterated rapidly yeah and so really easy debugging that's why we love this tool like it's it's just 01:27:41.300 |
really fun and easy to do so in the final step um the the final main step we have uh we're going to 01:27:49.460 |
like take that workflow now and call it with an agent so this is a pretty straightforward step we're 01:27:53.460 |
just going to create an agent give it a prompt to work with and give it the the knowledge of the this 01:27:58.340 |
workflow that it can run and then it'll run it for us so instead of having to use that interface we can 01:28:04.900 |
use a chat interface uh that we're more familiar with and go from there and so and again to tie it back 01:28:13.460 |
to actually taking this home and doing something useful with it like we've gotten to the point where we've 01:28:17.700 |
described the workflow of steps that completes a discrete task and maybe that's something that 01:28:22.340 |
our co-worker needs and then again maybe they are technical maybe they're not maybe they're just in 01:28:26.740 |
a hurry so by building them an agent and saying use this workflow and you have the ability to call this 01:28:31.620 |
workflow that becomes an ideal interface to give to our colleagues to start using this and then something 01:28:37.460 |
that they used to have to do piecemeally they're just now firing off a chat request and it's done for 01:28:41.300 |
them automatically yeah so with this you'll learn what agents are how you give them instructions 01:28:48.420 |
the memory systems for them the workflow integration through master and then how you can test them in the 01:28:53.940 |
and then how you can test them in the playground 01:29:16.340 |
wow these are awesome so these are new community memes pouring in in real time 01:29:20.420 |
we finally catch up on slack messages but there's already 20 more waiting nice 01:29:25.780 |
yeah that's a great idea yeah there's some great um feedback on the framework guys in the in the slack 01:29:43.220 |
channel there's like some ux requests as well you see 01:30:11.460 |
you can also play with the uh prompt uh the prompts a little bit to tweak it about what it's 01:30:18.260 |
how it's like analyzing the frustration the level of frustration the type of meme 01:30:23.700 |
it's really fun to get a vibe for it yeah indeed 01:30:36.420 |
all right all right is everyone more or less have uh what they need is anyone still stuck on anything 01:31:28.820 |
uh just a i guess a comment from us and i guess from the team as well like honestly like thanks 01:31:53.780 |
everybody for coming and building these we didn't even know this workshop was happening until last week 01:31:58.500 |
it was but um i mean previously like so like most of us at monster were previously building 01:32:06.420 |
gatsby like i was the co-founder and so we've been doing open source javascript stuff for a while but 01:32:12.580 |
it's like it's like pretty humbling when you show up for a conference and actually someone else is giving 01:32:17.780 |
a workshop about how to use the thing that you built 01:32:21.620 |
um and i think i speak for all of us so like thanks everybody 01:32:30.740 |
and on the same side we didn't know that you were going to be here until i think like yesterday 01:32:35.060 |
but we got very excited because it's like hey you got a question about commit let's go 01:32:39.300 |
i'm just personally curious how many folks have or like already have agents deployed at work 01:32:52.020 |
a couple how many folks are trying to get them in into prod right now 01:32:58.340 |
i'd be curious to know what you're using them for yeah i'd also be curious if anyone wants to share 01:33:02.100 |
either in the slack or just call it out or come up to mike like what are the top use cases that you're 01:33:07.380 |
thinking of for agents or struggling with i'd be curious to know too it's such a fun time because 01:33:12.980 |
there's like you have to get your head around what it's doing and then get your head around what it 01:33:17.620 |
can do and it just it starts to feel like the possibilities are endless and just super fun yep 01:33:24.900 |
that's awesome yeah that's awesome so like you've got um so you want to be able to chat with your 01:33:54.660 |
infrastructure and like ask questions like either an incident like show up to the incident and even 01:33:58.340 |
already have like initial guesses maybe yeah that's awesome cool any other use cases i think that 01:34:07.460 |
we're finding um we're mostly able to just knock out a ton of like low-hanging fruit that folks are doing 01:34:12.580 |
manually right now that's you know especially with tools like this like you you know create a simple 01:34:17.220 |
agent or even just a pipeline with a few steps and a couple api calls and that can like free up hours 01:34:22.420 |
of somebody's week i think that's the part that i'm most excited about right now yeah it's like all 01:34:27.700 |
the little paper cuts you can just so quickly iterate on them and get things going and not just with these 01:34:32.660 |
tools but just like vibe coding in general like you can you know be like i'm going to spend an hour just 01:34:37.940 |
seeing where i'm just seeing where i can get to and yeah it's amazing incredible how quickly you can get there 01:34:42.580 |
always review the code though please what's that always review the code please yeah please review the code 01:34:50.180 |
use tools like git guardian before you commit yeah 01:34:56.980 |
yeah we had our our ai moment a couple weeks ago at our last onsite and 01:35:05.620 |
we had like a couple hours just to spend doing something with ai and there was really no prescription 01:35:11.780 |
to it it was just explore and play and if you can solve a problem great if not you learned something 01:35:17.700 |
along the way and uh the thing that i worked on was like me as the maintainer of several sdks across 01:35:24.260 |
several languages uh a pain point i saw was like people adding in you know very simple prs to 01:35:30.580 |
one repo that's like the node repo the node sdk and then doing the same thing it's like adding a new 01:35:36.100 |
field to the node sdk and then adding the same field to the ruby go php kotlin like all of the sdks 01:35:43.060 |
i was like i wonder if we could do something to alleviate that and make it easier and after two 01:35:48.340 |
hours of pure vibe coding with clod i had a github action that when you label the first pr that you 01:35:55.220 |
do you just add a label to it it will go and generate the prs for every other repo and you know 01:36:00.180 |
every other automatic pr every other sdk yeah that's awesome were you using like a software 01:36:05.780 |
engineering like dagger or something to essentially make the changes making it oh that's a great 01:36:12.340 |
question making the changes is one thing and yeah that's something that yeah i think dagger would make 01:36:15.940 |
a lot of sense but what were you doing with that i wasn't i wasn't doing any of that um i vibe coded it 01:36:21.140 |
and i actually don't know what the code does and he had that i didn't review it a 4 300 line uh github 01:36:27.860 |
action that's working great now how much more about that but to be clear we didn't ship it uh it's 01:36:35.140 |
it's an experiment maybe maybe we will for sure um but the week after that i think um clod had come 01:36:42.260 |
out with like b the ability to run clod code as a uh like via an api so that you could like spin it up 01:36:49.620 |
run it give it the instructions on what to do and then let it figure out how to do it because like 01:36:53.780 |
most of the code that it generated was just handling how it was going to do the diffs and for several 01:36:58.820 |
hours of that it was just like it made the changes i wanted and then deleted the rest of the files 01:37:03.140 |
and that wasn't great um we're also for what's worth finding uh incredible utility with tools like v0 dev so if 01:37:18.020 |
you're already in the versell ecosystem or have used next js there's now they're they were originally 01:37:22.180 |
conceived of as like rapid prototyping tools that may be for designers or maybe for developers but like an 01:37:27.380 |
example is that somebody on our marketing and sales team is trying to figure out who would be interesting 01:37:31.460 |
engineers are already doing work with mcp to contact about conferences like this and their workflow is 01:37:37.380 |
completely manual and they're you know talking to me about it on on a call and four minutes after they 01:37:42.260 |
started telling me about it i had it working like in v0 dev and hand it to them and had a completely 01:37:47.700 |
separate you know tool that's available on a url that they can just paste in whatever they need 01:37:53.620 |
essentially and get like the analysis done um and so that was very eye-opening for me because it's in 01:37:59.300 |
some cases now it's faster to create the tool that your colleague needs than it is to continue talking 01:38:05.060 |
about the tool that your colleague needs and even ticketing it or saying next tuesday we should try 01:38:11.060 |
and like sometimes you can literally sit down in v0 dev with like a paragraph very descript prompt and get 01:38:29.380 |
no not in my experience not yet yeah totally i think that's a fascinating question yeah the question is 01:38:40.180 |
like when um folks in other roles that are not traditionally technical see how quickly you can 01:38:44.580 |
scaffold the stuff are they all clamoring for the keyboard that's not been my experience yet there 01:38:49.220 |
i think that's also like part of what the trepidation and anxiety is around is like well now everyone can 01:38:53.700 |
do it but then i also comfort myself and say that i define a pleasurable evening as like sitting down 01:38:58.740 |
for 12 hours on one of these machines and going back and forth to try and get it to go the way i want 01:39:04.820 |
and that's not most people's description of a fun time so like even when the tools are better 01:39:09.460 |
i i still think you kind of have to find this enjoyable you know what i mean i don't i don't 01:39:13.780 |
i i i think it might change to some degree like when like most generation becomes one shot able 01:39:18.660 |
and you say i want this app and it's of like this complexity and most people can get something 01:39:23.700 |
working immediately if that's correct i think that'll take a lot of low-hanging fruit out but 01:39:28.100 |
but they would do what they would do what lawyer like legal questions and data and just these fees and then they would 01:39:44.420 |
really uh almost like a blind amount start telling plot codes go to xyz and tell me what the results 01:39:59.780 |
yeah i think it's super fascinating and anecdotally um it's mostly been lawyers when i've been at meetups 01:40:11.860 |
sort of come up and asked the most like salient questions about gen ai and they'll say like i'm 01:40:15.220 |
not a coder but like this we're trying to do also semantic search and all like the information 01:40:19.140 |
compression that's available with a lot of the gen ai tools like i think uniquely speaks to their use 01:40:23.780 |
case too there's this gigantic corpus of case law and you need to know exactly this right now right 01:40:29.940 |
yeah it's but it is um it's an intense time i think for that reason yeah culturally right 01:41:32.820 |
in this uh session or in general sorry right now uh no yeah 01:41:40.100 |
but uh i have a i have a yeah yeah for the most part i have a tutorial for you that does that if you 01:41:45.940 |
want it thanks thanks for coming appreciate it yeah 01:41:49.220 |
all right so we can show the agent piece of it now um so i switched over to the step four branch 01:41:58.260 |
which is the end of step three uh where we should have the created uh agent now that can run 01:42:03.620 |
and actually have a conversation with the tooling which makes it really a lot easier to work with but 01:42:10.900 |
we still have the power of being able to use the workflow tool to see what's going on and 01:42:15.540 |
debug the steps along the way but now if we go to the agents tab in mastro we've got a meme generator 01:42:23.140 |
and this gives us a very sim a very familiar look and feel gives us a chat window but it gives us more 01:42:29.780 |
than that in the uh the sidebar here we've got an overview we can see the instructions 01:42:35.860 |
that are given to the agent so it's a helpful ui agent or ai agent that's going to help us do that 01:42:42.500 |
it's got some critical steps in there it's got the workflow that it should follow 01:42:46.020 |
and telling it what it has access to we also can see the model settings and we can adjust this as we go 01:42:54.660 |
so we can add more or less temperature uh top p we can change all of these on the fly and test it 01:43:00.660 |
to really hone in on like what works best for whatever we're trying to build 01:43:04.660 |
so i'll ask one more time work frustration he's got the gnarliest work for us who's the most frustrated 01:43:10.980 |
at work right now raise your hand and again you must state your company and your immediate superior's name 01:43:19.300 |
i always get asked to go to large meetings for four hours and then i don't speak at all 01:43:44.020 |
this bothers you too i see okay yes meeting fatigue um so now we can just ask the chat 01:43:50.980 |
and see the output and it's much cleaner it'll actually give us the image in line too 01:43:55.860 |
which makes it really nice and easy to work with that pointless meeting i was dragged into 01:44:03.780 |
but if you scroll up a little bit too the other thing uh to see here is like so the user asked a 01:44:09.940 |
question and you saw that the workflow got run um but now there's also additionally like the agent is 01:44:15.140 |
still chatting and it's in a familiar way that you already if it are familiar with if you use gpt 01:44:19.780 |
but then again just a reminder that this is kind of the ideal interface for folks that might be your 01:44:25.460 |
clients your customers your colleagues yeah and we can see like in line what happened you know the tool 01:44:32.660 |
arguments that were passed in the total result we can see what tools were called uh and how in this 01:44:37.140 |
case it was the workflow and here's all of the information and then the end result that we can 01:44:41.380 |
actually play with yeah so this could be you know chatbot that someone is requesting a report from 01:44:48.900 |
an analysis a sql query a new code a review of code review of a design like a new image 01:44:55.700 |
you can kind of put whatever you want behind the same interface 01:45:42.500 |
uh there's there's no hr for this conference i was told that in advance so uh kind of to wrap 01:45:52.740 |
that up uh what we built was a multi-step workflow that we could give to a conversational agent who 01:45:59.220 |
knows how to execute it knows what to pass into it and what it's going to get back and then it can 01:46:03.780 |
do without what it will it gave us you know human feedback uh it wasn't just spitting out jason in the 01:46:10.100 |
end it was giving us an actual response and then showing us the image inline and giving us a clickable 01:46:15.620 |
link so that we could go share that image to whoever we wanted to um and we were able to create some 01:46:21.620 |
awesome memes i highly recommend you deploy this and use it because it's cathartic to really vent to it 01:46:29.140 |
and if you're deploying it your data's safe probably i don't know 01:46:32.820 |
so some key takeaways uh from this uh we learned about the workflows agents we are uh type safety um 01:46:43.780 |
catching errors in that uh and the structured generation to ensure a reliable output uh and some 01:46:51.060 |
of the benefits of mastro yep um also just say that we didn't show this here but one of the examples 01:46:55.540 |
that's on master site you can go check out is like a 37 line of code uh pipeline that does retrieval 01:47:00.420 |
augmented generation and if you've worked if you've built those pipelines before like two years ago you 01:47:05.380 |
know you had to glue together at least three or four libraries it was a super crappy experience 01:47:09.140 |
so it's just really nice to have a single place you can go and define that pipeline at a high level 01:47:15.780 |
so yeah excited to experiment more with that as well there's a ton here that you can do that we did not 01:47:22.100 |
even scratch the surface of or point to yet yeah for sure coming up with uh what to do in two hours it's 01:47:27.540 |
uh not a lot of time and there's a lot a lot more that we could dig into and we highly encourage you to 01:47:32.660 |
so this was meant as a a primer uh you know have like an example yeah yeah uh now experiment with it 01:47:39.300 |
add things add tools um deploy it with mcp like figure out your co-workers yeah yeah ask them to 01:47:45.140 |
use it see what happens so yeah explore the tool creation uh add a vector database implement rag 01:47:52.580 |
uh all of those and just a final reminder mcp.shop um go order a t-shirt check it out uh it's really cool 01:48:02.900 |
so thank you with that um we've got 15 minutes left we wanted to save that for questions any 01:48:08.980 |
additional follow-up any just general casual conversation about these tools uh the fascinating 01:48:15.300 |
world of ai anything vote on memes that regenerated and see who is the ultimate winner in the slack channel 01:48:21.060 |
so i'm giving a talk on agents and workflows tomorrow and i really liked your agents and 01:48:28.260 |
workflow side can you bring them up again what i didn't hear what you said yeah i didn't hear 01:48:33.300 |
it sorry oh sorry i'm giving a talk on agents and workflows tomorrow agents and workflow sides 01:48:38.420 |
were really good can you bring them up again sure i can share them too they're open source because 01:48:52.420 |
there's a couple tools that nick nick likes uh yeah this was sly dot sly dev or sly dot dev 01:48:57.380 |
yeah it was something that we switched to relatively 20 minutes not that long ago 20 minutes before this 01:49:20.420 |
uh do make sure that you if you want a shirt buy a shirt because we will actually send you a shirt 01:49:26.660 |
we might not send it this week but uh we're tracking all the orders and we're going to honor all of them 01:49:31.940 |
so it's a limited edition mcp dot shop shirt that will never be seen again 01:49:37.460 |
yeah if you want an official master book um autographed copies are thirty dollars right and otherwise 01:49:53.700 |
cool um yeah yeah question where's the memory and execution step the memory and execution step 01:50:05.220 |
um yeah so you you can configure you know we configured sqlite for this demo and that's what's 01:50:10.500 |
powering like the ability as you create additional chats you'll see the previous chats on the left rail 01:50:14.820 |
but there's a ton of different storage drivers like there's the postgres and everything else right 01:50:19.860 |
so you can basically choose what you need to and then once you configure the memory you pass it into 01:50:25.700 |
either the workflow or agent um and then that's by by way of doing that you're telling it can use it 01:50:31.060 |
essentially is that your question the demo one doesn't have memory the demo one does have memories 01:50:36.020 |
using sqlite and it's solid uh it says check the execution step where do i find that uh check execution 01:50:50.580 |
great talks oh thank you thank you thanks so much for coming i appreciate it and uh yeah we'll be 01:50:55.220 |
around uh all week uh at the conference so reach out uh we'd love to chat more 01:51:00.660 |
talking about just uh like that graph this is amazing thank you