Building Reactive AI Apps: Matt Welsh

00:00:00.000 | .

00:00:14.880 | How's everybody doing this morning?

00:00:16.960 | All right?

00:00:17.900 | Yeah?

00:00:18.420 | I can't see anybody past the first two rows,

00:00:20.500 | so I assume you're all there.

00:00:22.080 | Now, I know what you're all thinking.

00:00:23.920 | I look way better in person

00:00:25.860 | than that AI-generated headshot, right?

00:00:28.340 | Well, we're going to work on that.

00:00:30.000 | We're going to, you know, the AI models are not perfect yet,

00:00:32.580 | but we'll get there.

00:00:33.740 | I need the clicker.

00:00:34.620 | Did that-- oh, it's up here.

00:00:36.040 | Awesome.

00:00:36.540 | All right.

00:00:37.480 | Well, today I'm super excited to talk to you

00:00:39.720 | about what we've been working on at Fixie for a while,

00:00:42.680 | which is an open source framework for building

00:00:45.140 | what we call reactive AI applications called AI.jsx.

00:00:51.680 | So let's get into it.

00:00:53.180 | First of all, what's the problem we're trying to solve?

00:00:54.860 | Well, I think most of us here know this already,

00:00:57.880 | but building and deploying high-quality LLM apps

00:01:02.260 | is still super hard.

00:01:03.960 | It's a lot of pieces you've got to worry about.

00:01:05.600 | You've got your vector databases, your context window limits,

00:01:09.460 | your rag stack, your tool sets, all that stuff.

00:01:12.820 | Now, most of us in this room enjoy solving those problems.

00:01:16.460 | That's why we're here.

00:01:18.000 | But we think most developers probably would rather not

00:01:20.920 | have to solve these problems themselves.

00:01:23.380 | So we want to do this for them.

00:01:26.720 | So at Fixie, we are aiming to solve this problem

00:01:29.960 | by inventing the future of AI application development.

00:01:36.380 | Today, I'm going to be announcing something

00:01:38.800 | that I think everyone here will agree is the most revolutionary

00:01:43.420 | technology in the AI development landscape.

00:01:47.400 | It is going to change the way we all think

00:01:49.400 | about building AI-powered applications.

00:01:52.520 | It's going to blow you all away.

00:01:54.960 | And so, ladies and gentlemen, may I introduce to you

00:01:59.140 | the future of AI application development.

00:02:03.520 | Imagine the 2001 theme playing.

00:02:05.580 | Dun, dun, dun, dun, dun, dun.

00:02:08.540 | Angle brackets.

00:02:10.180 | This is the future.

00:02:12.180 | Yes.

00:02:13.180 | There was supposed to be sound with that.

00:02:16.700 | That's all right.

00:02:17.540 | OK.

00:02:19.240 | It was better with the sound.

00:02:20.440 | All right.

00:02:20.940 | So this is AI JSX, and I'm going to talk about this a bit.

00:02:25.160 | Think about this as elegant LLM development

00:02:28.100 | for a more civilized age in TypeScript.

00:02:32.600 | OK.

00:02:33.100 | So before I get into what AI JSX is,

00:02:35.260 | you might be wondering why TypeScript, right?

00:02:38.280 | Well, we believe that there are many front end and full stack

00:02:42.160 | devs that are just-- they don't have good tools

00:02:44.500 | for building AI-powered applications today.

00:02:46.800 | There's some stuff, right?

00:02:49.540 | But we also think that the AI devs, the front end devs,

00:02:53.520 | are the ones building the AI experiences of the future.

00:02:57.540 | But today they're kind of like that bear staring in the window,

00:03:00.040 | right?

00:03:00.540 | They're saying, hey, we want a piece of the action.

00:03:02.540 | We want to get involved in AI development.

00:03:04.700 | But it's all the Python devs that are in the back end

00:03:07.700 | that are having all the fun here, right?

00:03:09.200 | So why should the Python devs have all the fun?

00:03:11.860 | There's also a lot more JavaScript developers in the world

00:03:14.320 | than there are Python developers.

00:03:16.300 | So we think that helping this community that hasn't been well

00:03:19.200 | served so far makes a lot of sense.

00:03:23.200 | So without AI JSX, this is what an AI engineer looks like today.

00:03:28.200 | I think I met that guy out in the hall yesterday.

00:03:31.860 | But with AI JSX, we can all be Hacker Man or Hacker Woman.

00:03:35.700 | Yes.

00:03:36.200 | All right.

00:03:38.700 | So get ready.

00:03:39.700 | I'm going to rock you with this whole presentation on this.

00:03:42.200 | OK.

00:03:42.700 | So let's just show a hello world of AI JSX.

00:03:46.200 | It's a very simple example.

00:03:47.820 | What we're doing here is defining a complete application

00:03:52.460 | that uses a chat completion component.

00:03:54.980 | That's the foundation of anything that might call into an LLM.

00:03:59.700 | And we're providing it a user message prompt in the form

00:04:03.400 | of write a Shakespearean sonnet about large language models.

00:04:08.680 | You might get a result something like this.

00:04:10.860 | It's actually pretty good.

00:04:13.420 | That's a hello world though.

00:04:14.440 | That's very simple.

00:04:16.060 | But let's talk a little bit about what AI JSX is.

00:04:19.360 | Think about it like React, but for building LLM apps.

00:04:23.900 | My colleague at Fixie pointed out to me

00:04:26.600 | that we actually have a page on our documentation site

00:04:30.200 | that says that AI JSX is not React.

00:04:33.360 | But clearly that's wrong because here I am telling you it is.

00:04:35.660 | Anyway, so AI JSX, it's built in TypeScript,

00:04:40.200 | which means you get all the safety and performance of JavaScript

00:04:43.780 | with the exciting adventure of fucking around with your dev tooling.

00:04:49.040 | Any model, any provider, we can support both Anthropic and open AI.

00:04:56.700 | Full React integration.

00:04:58.620 | This is great for building full stack applications

00:05:01.180 | where you just want to drop your AI powered stuff into your React app.

00:05:05.700 | AI JSX supports RAG out of the box,

00:05:09.280 | uses multiple vector DBs behind the scenes.

00:05:12.280 | You can plug in your own as well.

00:05:14.540 | You can use it to invoke custom tools and APIs,

00:05:18.320 | so you can use AI JSX in situations where you want to invoke an external service or an API.

00:05:24.280 | One of the cool features of AI JSX is the ability to have the AI generate UI components for you.

00:05:32.240 | Because the AI JSX program is operating on effectively the DOM, if you will, as React components,

00:05:40.860 | you can use it to generate UI components.

00:05:42.860 | And, of course, it's fully programmable and extensible, so you can basically build whatever you want.

00:05:48.440 | I'm going to give you a whirlwind tour of all this, show you what's possible.

00:05:53.700 | Basic idea is you build components, just like you do in React.

00:05:58.440 | Here, I'm defining a component called makeSetting.

00:06:01.020 | I might define a setting for a story that we might want to write.

00:06:04.320 | And this component takes in its children elements as a parameter.

00:06:10.580 | And we basically say write a one-paragraph description of this setting

00:06:15.900 | and put the children components right there.

00:06:18.660 | Those children components can be anything.

00:06:20.060 | They can be a string or they can be the result of a different tree of JSX nodes

00:06:26.480 | that have been rendered and placed in line in that prompt.

00:06:31.600 | So to call it, all I need to do is say, take the makeSetting component, instantiate it,

00:06:36.680 | and give it the prompt that I want.

00:06:40.180 | Okay, it's pretty cool, very easy to use.

00:06:43.320 | But you might be saying to yourself, come on, this is basically writing Python code

00:06:48.860 | with different syntax, right?

00:06:50.740 | This is, the angle brackets are a little bit overplayed here.

00:06:54.300 | But it's not just about syntax, right?

00:06:58.700 | JSX defines an entire tree of nodes that are rendered as a stream asynchronously and in parallel.

00:07:07.020 | So instead of rendering to the DOM like React does, we're rendering effectively to the LLM, if you will.

00:07:16.620 | So this allows us to do extremely powerful forms of composition.

00:07:20.620 | So here's a simple example of writing a story where I have a makeStory component with three child components.

00:07:28.060 | One is defining the character, another is defining the setting, and a third is defining the plot.

00:07:35.740 | When we render this application, all three of those components are going to run in parallel, and they're all streaming in parallel.

00:07:45.580 | There's three concurrent LLM calls going on, and they're streaming their tokens back to the makeStory component in real time.

00:07:55.180 | And so as this is being rendered, all the tokens are streaming through.

00:08:00.620 | The makeStory component is then streaming its output out to the result of that render, which might result in a story that looks like this.

00:08:08.060 | So far, I'm just showing you some basic things with text to give you some intuition around the ideas.

00:08:13.060 | But of course, you can take this a lot further.

00:08:17.260 | One thing you might say about this tree-based structure in AIJSX is that it allows you to break free of your chains.

00:08:27.420 | Sorry, I know that was a groaner.

00:08:29.860 | Okay, sorry.

00:08:30.740 | Okay, so here's another example of what you might be able to do.

00:08:37.380 | Let's wrap one component in another in order to constrain the latter component's output.

00:08:42.740 | So we're going to define a KidSafe component.

00:08:45.220 | This component takes in a system message that says, rewrite the following text so it's safe for kids.

00:08:51.140 | And the child components of that component are placed into the user message of that prompt.

00:08:59.580 | Then when we just wrap any component we want in a KidSafe component, it automatically will rewrite the output to be KidSafe.

00:09:07.560 | Right?

00:09:08.300 | So very, very powerful composition.

00:09:09.980 | Let me show you a quick example of how you use AIJSX to call out to tools and third-party APIs.

00:09:17.500 | In this case, we're going to define a record, which is a set of tools that we want to give the LLM access to.

00:09:24.180 | We're just going to define one tool here.

00:09:26.580 | Remember, this is a tool that calls the GitHub GraphQL API.

00:09:31.140 | And we're going to give it an English description of the tool.

00:09:35.620 | And there's a JavaScript function there.

00:09:37.840 | I've taken out the code for the JavaScript function because that's not interesting for this talk.

00:09:41.720 | But that's just calling using the Fetch API to call the GraphQL endpoint at GitHub.

00:09:48.480 | To use the tool in an application, then, all I have to do is instantiate a useTools component, give it that set of tools, and then anything that might need to use those tools as part of the rendering process can now invoke them.

00:10:03.640 | And so I can build very powerful applications in this way.

00:10:06.760 | This is RAG, retrieval augmented generation, in something like 10 lines of code.

00:10:15.240 | In AI-JSX, we have a chat completion.

00:10:17.400 | There's two children of that, the system message and the user message.

00:10:20.980 | The system message says, use the following information to answer the user's query.

00:10:25.700 | And it gets that information by using a DocsQA component.

00:10:30.980 | The DocsQA component is configured with a corpus of documents that you've crawled and indexed and placed in a vector database.

00:10:37.760 | You provide the user's query, the DocsQA component returns the chunks that are relevant to that query, places them right there in the system message, and then the user message contains the query again, and the final result is you effectively have retrieval augmented generation.

00:10:57.260 | I think this is a lot easier to understand, it's a lot easier to manipulate, it's a lot easier to integrate with other applications when expressed this way,

00:11:06.440 | rather than have a whole lot of different libraries that you have to invoke.

00:11:10.240 | And finally, when you're done building an AI-JSX application and you want to place it into your website, or your web app, or your mobile app, or whatever it is,

00:11:22.380 | you can just drop it right in as a React component.

00:11:25.420 | So in this case, we're showing you use the floating fixie embed component, that when you instantiate this, gives you a fully rendered UI

00:11:35.560 | UI for your AI-JSX application, with a chat window, with session management, markdown, rendering, custom UI, all the things that you might want out of such a thing in effectively one line of code.

00:11:54.240 | So I've been talking a lot about AI-JSX as an open source project, of course, I'm standing in front of you as a founder of a startup, so we've got to make money somehow.

00:12:01.560 | And so we're talking about the fixie platform as a really effective way to take AI-JSX applications and host them and run them and manage them in the cloud.

00:12:13.480 | So we make it really easy to build and deploy these things.

00:12:16.760 | The fixie cloud service has a fully managed RAG pipeline that does document ingestion, chunking, embedding, vector database, document storage, and all of the things that you need there.

00:12:29.740 | It fully manages the conversational state between the end user and the agent that you've built, so that you can have full context as part of that interaction with the user.

00:12:42.060 | We provide interfaces to all the popular LLMs, and those are really tuned for high performance and low latency, so that we get really good performance out of this.

00:12:50.060 | You can build your applications either in a no-code way, completely in the web UI here, or you can use AI-JSX to build a completely custom thing.

00:13:00.380 | And as I showed earlier, embedding it on your website is super easy.

00:13:04.900 | So that's the fixie platform.

00:13:09.520 | Right, so with that, you know, I'd encourage you all to try it out.

00:13:13.160 | Just go to fixie.ai.

00:13:14.400 | You can sign up.

00:13:15.360 | It's free to start out.

00:13:17.220 | Start building.

00:13:18.660 | AI-JSX is open source.

00:13:20.140 | You can use that without using fixie if you want.

00:13:22.180 | So go check that out.

00:13:24.520 | I'd really, really love to get your feedback and hear more about what you want to build and what kind of things you want to do.

00:13:29.760 | But one more thing.

00:13:32.920 | This is invoking, of course, Steve.

00:13:39.380 | Another thing we built with AI-JSX is the ability to support real-time voice interaction.

00:13:45.520 | That means bi-directional.

00:13:47.360 | You can speak to the agent and it can speak back to you.

00:13:50.380 | And we've done a ton of work to make the performance as just incredible, just amazing as possible.

00:13:57.060 | If you've seen the ChatGPT voice demos that they did last week or so, there's something like a four or five second gap after you talk to it and it talks back to you.

00:14:06.900 | I'm going to show you a demo in a moment that I think will be fairly impressive.

00:14:13.040 | So we get state-of-the-art performance, real-time, bi-directional conversations and, of course, it's all grounded on your docs and your data.

00:14:21.180 | So I'm not foolish enough with the Wi-Fi that's here to do a live demo, so fortunately our CTO, Justin, recorded a video of this, but I want to assure you that nothing in this video is staged, nothing has been trimmed out, nothing has been sped up.

00:14:37.320 | This is real-time, you know, live video recording from yesterday, and you can try it out yourself at voice.fixie.ai if you don't believe me, just go try it on your own machine, not now, not here, please.

00:14:51.400 | But the idea of this demo is we have a fictional donut restaurant called Dr. Donut, and you're at the drive-thru.

00:14:59.060 | You drive up in your Tesla or whatever you're driving, roll down the window, and you're talking to the drive-thru window at the Dr. Donut, and the AI is acting to take your order.

00:15:09.960 | And so with that, I'm going to see if this works. Hopefully we have sound on the video, and let's take it away.

00:15:16.040 | Welcome to Dr. Donut, home of the best donuts in town. How can I help you?

00:15:21.180 | Yeah, can I get two of the pumpkin spice iced donuts?

00:15:25.540 | Sure, I've added two pumpkin spice iced donuts to your order. Can I assist you with anything else?

00:15:33.960 | What's your best coffee?

00:15:35.960 | Our pumpkin spice coffee is a popular choice this season, and our caramel macchiato is a favorite all year round.

00:15:45.340 | Which one would you prefer?

00:15:47.100 | Yeah, let's go with two of the macchiatos.

00:15:49.940 | Great. I've added two caramel macchiatos to your order. Can I get you anything else?

00:15:58.080 | Yeah, what are your thoughts on large language models?

00:16:04.200 | Sir, this is a Dr. Donut.

00:16:06.600 | Okay, just the donuts and the coffee then.

00:16:10.400 | Sure, your order is two pumpkin spice iced donuts and two caramel macchiatos.

00:16:18.580 | The total is $9.56.

00:16:21.300 | Please pull up to the next window.

00:16:23.980 | And that was all, like, you can go try it yourself.

00:16:27.340 | We've done a ton of work to make the latency really low, to make the quality really high.

00:16:31.060 | It's using the full Fixie platform as a back-end.

00:16:33.700 | And, of course, if you're building with Fixie and AIJSX, you can build the same thing for yourself.

00:16:37.780 | Donuts or not, right?

00:16:40.040 | Okay, so that's basically all I have.

00:16:43.260 | I'd really love to get you all to check it out.

00:16:45.800 | We are of course hiring and so there's our careers page and please, you know, don't forget to smash that like button. Thank you very much.

00:16:53.780 | *music*

Building Reactive AI Apps: Matt Welsh

Chapters