My name is Bianc. I'm the CTO and co-founder of a company called Sourcegraph. We build developer tools and today I want to share with you some of the observations and insights that we've had on the sort of like emerging skill set of how to wield coding agents. That sound good to everyone?

All right, cool. Okay, so let's check in on the agent discourse. I don't know if you all saw this, but a couple days ago there were some spicy tweets about the efficacy of AI coding agents or inefficacy, depending on your perspective. So Jonathan Blow, who is a really talented developer, he basically single-handedly coded up the indie game Braid, if you're familiar with that.

So he's kind of like god tier status in terms of coding ability. Retweeted Alex Albert, who is also someone I respect and admire a lot, works at Anthropic, basically claiming that all the hype around coding agents and code AI in general was just hype, right? There's no substance there.

And then there were some responses, and there was kind of a spectrum of responses too. We had some other big names in the developer world, like Jessie Frazell. She was one of the early contributors, maintainers of Docker. She's also really legit. She said basically something to the effect of like, I think you're right, but you're in like the top 0.01% of programmers, Jonathan, for the rest of us down here in this room that aren't on Mount Olympus.

It actually helps a lot, but not super helpful if you're really, really good. But then we also had folks like Eric S. Raymond, who is like one of the fathers of open source, who had a very spicy reply. He's basically like, look, I consider myself to be pretty decent at programming, and these things help a lot.

And then the kind of my favorite one of this was actually the Top of Hacker News post that was written by Thomas Tacek, who is a Thomas Tacek, who is a really legit security engineer. Some of you may have seen this trending. reply is basically like, look, I consider myself to be pretty decent at programming, and these things help a lot.

And then the kind of my favorite one of this was actually the Top of Hacker News post that was written by Thomas Tacek, who is a really legit security engineer. Some of you may have seen this trending. He was basically taking the opposite view of like, you know, there's some really smart people there who are very AI skeptical, but they're nuts.

Like, these things are really useful. So I'm guessing if you're at this conference, you probably lean toward coding agents are substantively useful, and there's something there. I don't know. Just a guess. But I think even within this room, there's probably a spectrum of best practices and opinions about like where agents are good.

You know, whether they're restricted to like small edits or like front end applications or weekend via coding, whether they actually work on your production code base. And I think this is just indicative of the dynamic technical landscape that we're in right now. And a couple months back, I read this blog post from this guy, Jeff Huntley.

So Jeff was a senior engineer at Canva at the time. And his role at Canva is really interesting. He basically went around interviewing all the developers inside of Canva using AI tools like cursor and other things and seeing how they're using it. And he basically came to the conclusion that like most people were holding it wrong, which is really interesting.

And he came up with the blog post about like all the different anti-patterns that he was seeing. But my summation of that blog post is like the number one mistake that people are using with coding agents right now is they're trying to use coding agents the same way they're using AI coding tools six months ago.

And therefore, they're probably wrong, which is kind of crazy because normally, if you're, you know, using a tool, the best practices don't change in six months. Typically, the things that you learn that are good will still be like present and, you know, topical and relevant six months down the line.

But I think we're in a really interesting moment in time right now. And, you know, why the sudden change? I think it's because of this step function transition that we've experienced in model capabilities in the past six months. So, you know, we've, we've all been around since the dawn of generative AI, the ancient year of 2022, right?

November 2022 was when ChatGPT launched, right? And every year now, you know, this is now the year three, you know, three after ChatGPT, right? We're now living in the AI future. But I think there's already been kind of like three distinct waves or eras largely driven by the evolution of frontier model capabilities.

And the model capabilities really dictate the ideal architecture that, that becomes dominant at the application layer. So in the GPT-3 era, all the models were text completion models, which meant all the applications that people were building were these like copilots or autocomplete tools. So the dominant UX paradigm was like, you type some stuff, it types some stuff, you type some more, and that's how you would interact.

And then ChatGPT came along with GPT-3.5, which was InstructTuned to interact like a chat bot. And suddenly people realized like, oh, it's not just completing the next thing I'm talking about, I can actually ask it questions like I can a human now. And then some other people came along, we were part of this crowd, we realized like, hey, you know what's even better than just like asking questions?

You can actually copy paste stuff into the chat and say like, here's some code for my code base, use that as an example, and pattern match against that. And that helps it generate, you know, a little bit better code or less fake code or less hallucinated code than it did before.

And that basically meant that everyone at the application layer was building a rag bot in 2023. So like a chat bot plus a rag retrieval engine. But now, I think we've entered a new era. And I'm not sure if everyone realizes it, or maybe this is, I don't know, like who agrees with this statement?

Like who thinks it's a real paradigm shift? Okay, and then who here is like, ah, that's a bunch of bullshit. Anyone? Feel free to, I like, okay, okay, so maybe I'm, maybe I could just skip this slide. So we're now living in the era of agents, and the new model capabilities really dictate a new application architecture.

And so one of the things that we asked ourselves at Sourcegraph is, you know, a lot of the existing tools in the market that were designed for the era of GPT-4 and CLAWD-3. So a lot of the application stuff, features, and UX and UI was really built around the capabilities, or in some cases, the limitations of the chat-based LLMs.

And so if we were going to design a coding agent from the ground up to unleash the capabilities of tool using LLMs, agentic LLMs, what would that look like? Okay, so here are my spicy takes. These are controversial design decisions that I think are better to make in the age of agents.

And many of these go against the best practices that kind of emerged in the chatbot era. Okay, so number one is the agent should just make edits to your files. It shouldn't ask you at every turn, like, hey, you know, I want to make this change. Should I apply it?

If it's asking you and it's wrong, it's already done the wrong thing and it's wasted your time. Humans need to get more out of the inner loop and more kind of like on top of the loop, like still steering it and guiding it, but less, you know, micromanaging and managing every change.

Second thing is, do we still need a thick client to manipulate the LLMs? Like, do we still need a forked VS code? That's like the salty way of saying this, right? The VS code fork became the culmination of the AI coding application, I think, for the chatbot era. But there's this question of like, you know, if the contract of an agent is you ask it to do stuff and then it does stuff, do you really still need all that UI built around like context management and applying the proposed change in the code base?

Or can you just ask it to do stuff and expect it to do the right thing? Third, I think we're going to move beyond the choose your own model phase. So I think in the chatbot era, it was very easy to swap models in and out. And you'd like, oh, you know, a new model came along, let me swap it out and see how well it attends to the context that my retrieval engine fetches.

In the agentic world, there's a much deeper coupling because the LLM that you're using essentially becomes the brains of these agentic chains. And so it's much harder to rip and replace. And I think a lot of people in this room who have tried mixing and matching, you know, different models in the context of agents have found that, you know, swapping out a different model and expecting similar results is very different.

A lot of the, like, a lot of the LLMs out there aren't even good at the basics of tool use yet. So it's very difficult to just replace the brains. Four is I think we're going to move past the era of fixed pricing. Agents eat up a lot of tokens.

And so they look expensive relative to chatbots. But the comparison that more and more people are making is how much human time is it saving? So they're still cheap relative to human time saved. And the fixed pricing model actually introduces a perverse incentive now where it's like selling gym memberships, right?

Like if I sold you a membership to my chatbot and you're now paying me, you know, 20 bucks a month, my incentive now is to push the inference cost as low as possible. And the easiest way to do that is to use dumber models. But dumber models just waste more of your time.

Sorry, this is a long list. Hopefully it's not too tedious. But I think these are important points. The second-to-last point I'll make is I think the Unix philosophy is going to be more powerful here than vertical integration. So in developer tools, the ability to use simple tools in ways that compose well with other interesting tools is really powerful.

And so I think, especially with agents where there's less of a need to create like a lot of UI around it, you're going to start to see more command-driven tools, command line tools, and things like that. And then last but not least is, you know, we had an existing RAG chat coding assistant.

Maybe some of you have used it. It was called Kodi. It still exists. We're still supporting it. It's still in heavy use across, you know, many Fortune 500 companies. But we decided to build a new application from the ground up for the agentic world because we didn't want to be constrained by all the assumptions and constraints that we built into the application layer for the previous generation of LLMs.

And one analogy I like to draw here is, you know, what the early days of the internet, right? Like in the early days of the internet, the way people, you know, jumped into the the web was using an interface on the left. This was before like most people knew what the internet was about, what it was capable of, and that was the right interface for the first generation of the internet because like what can you do with the internet?

Well, like there's a bunch of different things. You can look at like trending celebrities. You can, you know, buy automobiles. You can look at movie reviews, all these things you might not have thought of. And so it's useful to have in front of you. But at some point it gets a little tedious like clicking through all the different hyperlinks and navigating your way through.

And then the the real power of the web was sort of unleashed by just like the one simple text box where you just like type what you're looking for and you get to it. And I think, you know, with with agentic UIs, that's what we should be striving for both in developer tools and in a lot of different application paradigms.

Okay, so what does that look like in practice? So when we went to design this thing, our coding agent is called AMP. And AMP has two clients and this is what they look like. So both are like very, very bare bones. A lot of people, you know, look at this and like, what is this?

It's just a text box. What can I do with it? And that was by design, you know, that for all the reasons I just mentioned. One client is just a simple VS code extension that allows us to take advantage of some nice things that you get in VS code, like being able to view diffs.

That's really important in the agent decoding world. I often joke that like that's now I use that view more than the editor view now. And the second was a CLI. So just stripping things down to bare bones. It has access to all the same tools as the VS code extension does, but it's just something that you can invoke in your command line.

You can also script it, compose it with other tools. Okay. So what does this actually look like in practice? I want to do something a little bit risky here, which is in the past I've done a lot of like, you know, hey, here's me building a simple app, like those sorts of demos.

But I actually want to show off like where we think this is most useful, which is like, hey, I'm working on an application that has real users. Let me actually make a contribution to that code base, given all, with all the like existing constraints. And so I actually want to, I'm just going to code a little bit.

Well, I don't even know how far we're going to get. But this is, this is AMP. This is VS code running AMP in the sidebar, and it's open to the AMP code base. And what I want to do is implement like a simple change to this application. So the change that I'm going to make is AMP has a server component.

And the server exists as a way to provide the LLM inference point. It also provides like team functionality. We have a way to share like what different teams are doing, what different users are doing with AI. So you can kind of learn from other users. There's leaderboard. It's fun.

But there's also these things called connectors, which allow AMP to talk to external services. So our issue tracker is linear. And so I've integrated linear into AMP here, but I'm kind of annoyed because it's using this generic like network icon. And I would really like to customize this icon such that when you plug in the linear MCP endpoint, it uses a more appropriate icon like a checkbox or something issue-y.

So I've already filed this as a linear issue. And I'm just going to ask, can you find the linear issue about customizing the linear connector icon? Then implement it. So what this will do is it has access to a set of tools. I can go over here to the tool panel and see what tools it has access to.

Some are local, some are built in. It's got the standard tools like read and edit file or run bash command. You can also plug in things like Playwright and Postgres via MCP. And then linear is also plugged in through this. So we're basically talking to the linear API through the MCP server.

And what this will do is it will use the linear issues API. And it will search at issues. It found 50 issues. And the one that I was referring to is at the top here. So add a special icon for the linear connector. And now it's going to go and implement the thing for me.

And one thing to note here is it's just making these tool calls on its own. I'm not asking it to use specific tools. We've also tried to make the information that you see minimal. So you don't need to see all the API tool calls that it's making underneath the hood or crowd out the transcript with a bunch of things.

Most of the time, we just want to keep it simple. Because the contract we want to provide to users is like, the feedback loops here are more robust. And you don't have to micromanage this as much. Another thing I want to point out here is the search tool that this is using is actually a sub-agent.

So it's actually spinning off a sub-agent loop that uses a form of agentic search. It has access to a bunch of different search tools. Keyword search, regular graph, looking at file names. If you want to inspect what it's doing, you can click the expand thing and see what different path it's taking, what files it's reading, what things it uncovered.

But again, by default, we think this is an implementation detail. And hopefully, it should just surface the right thing. So it's working. It's gathering context. Another thing I want to call out in this interface is, as we've gotten more feedback, we've kind of designed this thing to be more multi-threaded.

So there's a quick keyboard shortcut that allows you to quickly tab through the different threads that you're running. And it's a common paradigm in our user community to be running more than one of these things at a time. And it takes a little bit to get used to the context switching.

Developers hate context switching. We like to be in flow, in focus. Typically, what we see here is the secondary thread will either be something that's a lot shallower, so that you can quickly page back to the main thread. Or what I like to do is, while the agent is working, I actually like to understand the code at a deeper level myself, so I can better understand what it's going to do.

So I could ask something like, "Can you show me how connectors and connections work in AMP?" I can ask it to draw a picture of that. So we'll kick that thread off in parallel. We'll check back in on what this guy is doing. So it's found -- it's read a bunch of files.

It's read some front-end files. Our front-end is written in Svelte. And as you can see, it's being fairly thoughtful about reading the appropriate files before it actually goes and does the work. And we find that this is really important to make the feedback cycles more robust. Otherwise, the anti-pattern is you just get into the weeds of steering it manually.

It's also got this to-do list thing at the bottom that helps it structure and plan out the longer-term tasks, so that it doesn't go immediately dive into the code. There's a classic mistake that human developers make too, where you dive into the code too early, and then you get lost in the weeds, and then it takes a while to dig yourself out.

Okay, so it's making some changes. One other thing that I like to point out here is -- you know, I mentioned that I use the diff view in VS Code now probably more than the editor view. VS Code actually has a really nice diff view. I have a hot keyed, so I can open it up quickly.

And most of my time in VS Code now is spent just reviewing the changes it makes. And I actually like this a lot better than GitHub PRs or Git diff on the command line. Just because it's in the editor, you can see the whole file, and jump to definition even works.

So, yeah. We'll just wait a little bit for it to do its thing. I actually think it's probably made -- looks like it's getting there. It's probably just running, like, tests. Let's see if we go back here, if it's updated the icon at all. Okay, so it hasn't gotten there yet, but I think it's on the right track.

The question was, does it write its own tests? Yes, it typically writes its own tests. And if it doesn't, you can prompt it to do so. So, it's doing a lot of things. It's reading a lot of files. It's making these edits incrementally and then checking the diagnostics. And then now let's see if it works.

Okay, cool. So, you see here the icon has been updated. And this is without me really steering it in any fashion. Notice here on this page that this icon didn't update, though. And so this is actually not surprising to me because this change -- as many changes in production code bases are -- often more nuanced than it seems at the surface.

So, in this case, the reason it's not getting it here is because this is the admin page. And the piece of data we need to know -- we need to read in order to tell that this is a linear MCP rather than a generic MCP is actually part of the config.

We have to look at the endpoint of the MCP URL. In order to do that, you have to read the config. But the config might also contain secrets. It doesn't contain secrets in this case, but might contain secrets in other cases. So, we actually prohibit those secrets from being sent to non-admin pages.

So, it's not surprising to me that the first pass, it didn't get that right. But let's see if it can get -- I'll just nudge it a little a bit. So, like, I noticed that the icon changed on admin connections, but not on settings. Can you investigate why? And in the interest of time, we'll check back on this later.

How about that? We'll let it run and we'll see if it can find its way to the right solution there. Is it okay if I go a little bit over since we started a little bit? Okay, cool. Is it okay with you all if I go a little bit over?

Okay. Are you still having fun? Okay, cool. So, that was like a brief demo of just like the interaction patterns and the UI. We try to keep it really minimal. We've released this to like a small group so far. The sign-up is now publicly open. It's been open for about two weeks, but we haven't done a lot of like marketing around it.

And that's kind of been intentional because we're really trying to design this for where we think the puck is going. And so, we've done a lot to curate this community of people who are trying to experiment with LLMs and figure out like how the interaction paradigms are going to change over the next six to 12 months.

And so, our user community is really people who are like spending nights and weekends a lot of time with this thing to see what they can get it to do. And so, actually, one of the most insightful things and actually the main topic of this talk is lessons that we've learned from just like looking at what our power users are doing and seeing what interesting behavior patterns they're kind of like implementing.

And so, like the average spend for agents is growing. It's a lot more than the average spend was for chatbots or autocomplete. But one other interesting thing that we've noticed among the user base is that there's a huge variance in terms of how much people use this think. To the point where like there's like an upper echelon of users that are spending like thousands of dollars per month just in inference costs.

And at first, we're like, this has got to be abuse, right? Like someone out there is poked, found some way to exploit the inference endpoint, is using it to power some Chinese AI girlfriend or whatever. But actually, no, when we spoke to the people using it, we actually found that they were doing real things.

And we're like, that's interesting. What the hell are you doing? And from those insights and the conversations, we basically have encapsulated a series of best practices or emergent power user patterns for how the very dominant users, the most active users are using this thing. And this has informed our product design process as well.

So one of the the first changes that we made was we noticed that a lot of the power users were very writing very long prompts. It was not like the simple kind of like Google style, like three keywords and just like read my mind and expect something good to happen.

They actually wanted to write a lot of detail because they realized that LLMs are actually quite programmable. If you give them a lot of context, they will follow those instructions and get a little bit further than if you just give them like a one line sentence. And so we made the default behavior of the enter key in the AMP input just new line.

So you have to hit command enter to submit. And this throws a lot of the new users off because they're like, wait a minute, why isn't it just enter? Like, you know, if I'm in like cursor or whatever, it's just enter. That's easy. That's intuitive. But actually, what we want to push users to do is to write those longer prompts because that actually yields better results.

And I think that's one of the things that prevents people who are still in the kind of like chat LLM mode from unlocking some of the, you know, cool stuff that agents can do. Another thing that people do very intentionally is direct the agent to look at relevant context and feedback mechanisms.

So, you know, context was very important in the chat bot era. It's still important in the agentic era. Now, agents do have a good amount of like built-in knowledge for how to use tools to acquire context. Like you saw that before when it was using the search tool to find different things.

And it was executing the test and linters to see if the code was valid. But there's still some cases, especially in production code bases, where it's like, oh, we do things in a very specific way that are kind of like out of distribution. And so like some like less agentically inclined users at that point will just give up.

They're like, ah, you know, agents aren't capable of working with like backend code yet. But what we've noticed is the power user like, actually, let me try to just tell it how to run, you know, the build in this particular sub directory, run the tests. And that helps it complete the feedback loop so that it can get the validation to get further.

Feedback loops are going to be a big theme of this talk. So another like dominant paradigm here is constructing these like frontend feedback loops. So like a really common formula is you have the playwright mcp server and then there's a thing called storybook, which is basically a way to encapsulate or componentize a lot of your fun components.

It makes it very easy to test individual components without loading your entire app. And you know, you probably should have been doing this anyways as a human developer because you get a fast feedback loop. You make a change, see it reflected instantly, you get the auto reload and then go back to your editor.

But with agents, you kind of notice it more because you're no longer like in the weeds doing the thing. You're like, oh, you're almost like the developer experience engineer for your agent. It's like, how can I make it loop back faster? And so what the agent will do is like, you know, make the code change, use playwright to open up the page in the browser, snapshot it, and then loop back on itself.

And it does that via storybook because it's much faster than reloading the entire app. You put playwright as a tool for you? Yes. So, it's one of the default recommended tools. So, it's right here. And actually, it looks like that run completed. I wonder if -- looks like it did approximately the right thing.

Sorry, just to jump out of the slides of her a little bit. So, now you can see like the icon is customized on the settings page, not just the admin page. And if you look at how it did that, I think it did the right thing. So, if you look at the diff, it actually looked at the surrounding code and it was like, oh, there is an existing mechanism for plumbing non-secret parts of the config through to the UI.

Let me use that as a reference point. And it actually plumbed exactly that field through to the front end. So, now if I add additional fields to the MCP config that do contain secrets, this is whitelisted. So, it will still only send the endpoint URL over to the client, basically what it needs to make that icon customization.

So, yeah. I know it's not a super impressively visual change, but a lot of such changes in messy production code bases are like that, and it's cool to see the agent be able to tease out that nuance. Okay. I know we're a little bit over time. Can people mind if I keep going?

Okay, cool. There's some additional tips and tricks. Most of this talk is just sharing what we've learned from our power users. So, another thing that we've noticed is this prevailing narrative that agents are going to make programmers lazy. It's going to make it so we don't really understand what's going on in the code, so we're going to ship more slop.

But we've actually found the inverse happen with the power users. They're actually using agents to better understand the code. And so, this is a really good onboarding tool. We just hired this guy, Tyler Bruno. He's a very precocious young developer. He's actually still in college, but he's working full-time in addition to taking classes.

So, really bright, but also a bit green. He's been using AMP to just quickly ramp up on how the different pieces connect together. He can draw diagrams and point you to specific pieces of the code. It's really good at accelerating that. And then a corollary to this is we all do a form of onboarding to new code whenever we do a code review.

By definition, code review is new code. And oftentimes, it's new code that contains blogs or is hard to understand or is a bit of a slog. And so, rather than just ignore the code that the AI generates and just commit it blindly, we find that our user base is actually using this tool to do more thorough code reviews.

So, I've adopted this practice myself where if I have to review a very large diff, the first thing I do is ask the agent to consume the diff and generate a high-level summary so I can have a high-level awareness. And then I ask it, "Hey, if you were a smart senior dev, what's the entry point into this PR?" Because often half the battle is just finding the right entry point.

And psychologically, I often put off code reviews because I'm like, "Oh, it's going to be a pain, and it's going to take forever just to figure out where I should start reviewing it. So, I'll just do it tomorrow." But this thing, just like it helps lower that activation energy and make code views more thorough and actually, dare I say, like a little bit fun and enjoyable now.

Sub-agents are also a thing. So, we implemented the search tool as a sub-agent in the very beginning, but we're seeing more and more use cases emerge for sub-agents. And the general best practice with sub-agents is that they often are useful for longer, more complex tasks because the sub-agent allows you to essentially preserve the context window.

So, like, the quality of the LLM will degrade over time. You know, Sonnet 4 has a context window of 200K, but we see degradation typically around like 120 or 130K. And by the time you hit 170 tokens, you start to see more kind of like off-the-rails and crazy behavior.

But sub-agents allow you to encapsulate the context used up by a specific sub-task, like implementing a small feature, such that it doesn't pollute the main agent. Okay. So, that was a quick tour of a lot of best practices. Just to recap, like the anti-practices, the common anti-patterns are just like micromanaging the agent, like using it like you would a chatbot, where you have to kind of like steer it at every interaction or review every edit it's making.

Another common anti-pattern is just like under-prompting, so not giving it enough detail. Like LLMs, their knowledge comes from two places. It either comes from their training data or from the context that you give it. And so, you know, it's fine if you do a five-word prompt if you're coding up like a 3D Flappy Bird game from scratch, because that's well represented in the training set.

They're really good at that. They're trained to do that. But if you're trying to make a subtle nuance change to your large existing code base, you should be giving it all the details that you would give a colleague on the team to point them in the right direction. And then last but not least, like agents are not a vehicle to like TLDR the code.

If anything, they're the opposite. You should be using them to do much more thorough code reviews more quickly. The human is still, you're ultimately responsible for the code that you ship, and you shouldn't view this as a human replacement. It's really a tool that you can wield to make yourself 10, 100x more effective.

Last tidbit. So, one of the things that we've noticed among the very, very, very top 1% of the 1% is this inclination to run multiple of these things in parallel. Jeff Huntley, who wrote that blog post that I showed earlier, he started putting out these Twitter streams. They're about four hours long each.

And it's basically just, he's working on a compiler on the side. And what he does is he constructs prompts for three or four different agents to work on different parts of the compiler. And he's gotten to the point where he's prompting it such that he feels confident enough in the feedback loops where he just hits enter, lets him run, and then he goes to sleep.

And then this thing just runs on Twitter for a while. And I think he's doing this to spread the word. It's like, hey, you can use this for serious engineering. Compilers are not some vibe-coded weekend project. They're real tech. They're difficult to build. And it is possible to use agents for code like this.

But it has to be a very intentional skill that you practice. And so I think it's cool. I think there's a lot of people thinking in terms of agent fleets and where the world is going. But I do think that the way that we'll actually get there is by building these composable building blocks that allow people like Jeff to go and combine them and come up with interesting UIs.

I think this is just running in like Tmux or some window manager. Okay, so the takeaways I just want to leave you with is, one, contrary to what some might say, and there's a lot of smart senior developers out there who think AI is overhyped, and maybe parts of it are.

But I think coding agents are very real. And it is, I think, a high ceiling skill. It's like, I think we will probably invest in learning how to use these things in the same way that we invest in learning how best to use our editor or our programming language of choice.

And I think the only way you can learn this stuff is by doing it and then sharing it out with others. And one of the reasons we built the thread sharing mechanism in AMP is to help encourage knowledge dissemination so that like if you discover an interesting way of using it, you can share that out with your team.

But yeah, that's it. If you want to kind of like see a recap of the best practices in this talk, we've actually put out like an AMP owner's manual that guides new users how to best use it. I'll also be around afterwards. We have a booth in the main expo hall.

I'm supposed to say, too, if you stop by the booth, we'll give you like $10 in free credits. So if anything you saw here was of interest of you and you want to try this out, stop by and say hi. I noticed you still type, can you, and then you correct your typos, which I guess you said you shouldn't do.

Yeah, it's part habit and it's part paranoia that in like a live demo setting, there will be some typo token that will trigger off the rails behavior. But it like, I think that was more of a concern that I learned in like 2023 when it actually mattered. Because like these days, LMs are more and more like typo robust, I would say.

Yeah.