Scaling AI Agents Without Breaking Reliability — Preeti Somal, Temporal
00:00:00.040 | My name is Preeti and I am part of the engineering team at Temporal. How many
00:00:19.440 | people here have heard of Temporal? Perfect, great. So Temporal is the company
00:00:25.080 | that takes reliability incredibly seriously, so seriously that our mascot
00:00:30.720 | is a tardigrade. Does anybody know what a tardigrade is? Yes, some folks? Well it is
00:00:38.640 | what is also called a water bear and is the most resilient animal known to
00:00:43.960 | humankind and so that's how seriously we take reliability. Definitely stop by our
00:00:49.020 | booth for some stickers and some pins to just show how much you care about
00:00:53.560 | reliability. All right so my goal in the next 17 minutes or so is to convince all
00:01:00.520 | of you that Temporal is the right choice of platform as you go out to build agentic
00:01:07.600 | AI applications. So let's dive right in. We've heard a lot about how you know agents
00:01:15.460 | are just software, however they are complex distributed systems, they need to cope with
00:01:22.420 | with LLMs and they must scale and provide durability and reliability. Otherwise no one's going to trust
00:01:29.380 | your agent. You know we can build like really cool stuff but if it doesn't work nobody's using it.
00:01:34.980 | We also know that complex systems need orchestration. Decks use the word workflow a few times and essentially at the core of agentic AI applications is a complicated workflow that orchestrates multiple processes,
00:01:53.940 | needs to handle state potentially over long periods of time, there needs to be human interaction
00:02:00.900 | for approvals or other reasons and of course they need to be able to run in parallel for efficiency, call tools, there's just a lot of things going on. You know how many of you feel like building agents is really simple?
00:02:17.860 | You're just calling one switch statement, right? Yeah? I mean there's a lot of things that are interacting here and how can you actually keep track of that, make sure it's running reliably as well as tracing and looking at the visibility of all of these pieces.
00:02:34.820 | And again you know these systems are inherently unreliable. How many of you have called an LLM and it succeeded 100% of the time? Yeah? Nobody's raising their hands.
00:02:46.820 | So we've dealt with this. As you're building these applications you are seeing inherently how unreliable some of the tool chain here is.
00:02:54.980 | And then you know difficult to debug and test and Alex has been talking about agent ops being kind of built around this but clearly the insight into what's happening has been incredibly hard to get.
00:03:10.020 | It's been incredibly difficult to test this in pre-production.
00:03:15.380 | Well, the interesting thing here is that these problems have existed for some time in building complex distributed systems.
00:03:23.540 | And Temporal is a company that was founded around solving these problems.
00:03:29.540 | Our mission really is to outsource the reliability and scalability parts of building a complex distributed application seamlessly.
00:03:40.820 | So again, you can focus on the hard parts of writing your business logic.
00:03:44.980 | The way that Temporal works is that we built language idiomatic SDKs.
00:03:52.980 | The languages are available there, and in fact, one of the fun facts here is that Python has overtaken all the other languages in the month of January, and that sort of is showing how much of Python is being used in building these applications.
00:04:09.140 | We handle all the plumbing code for you, making sure that every process executes reliably, and providing you guardrails.
00:04:17.300 | And this is a battle-tested product.
00:04:21.300 | Temporal has been in production for over a decade.
00:04:25.300 | Temporal is used in mission-critical applications.
00:04:29.300 | Temporal has been in production for over a decade.
00:04:35.460 | So we feel that using Temporal for building your authentic AI applications gives you reliability out-of-box.
00:04:45.060 | But, you know, I can stand here and talk all day about it.
00:04:48.260 | You probably don't believe me.
00:04:50.260 | So we have some customer quotes here that will help you understand this.
00:04:56.180 | One of them is from a customer called Dust that is building their agents on top of Temporal.
00:05:01.780 | And the other is a company you may have heard of that recently talked about the tech stack they use.
00:05:08.980 | And Temporal is very clearly featured there as well.
00:05:11.940 | We also have a number of use cases published.
00:05:17.060 | So Gorgias, for instance, is using AI agents today in production built on Temporal.
00:05:24.500 | This is the company that does customer service for brands like Reebok or Timbuktu or Glossier.
00:05:31.220 | These are all household names.
00:05:33.380 | And the reason I'm bringing this up is just to help you understand that customers are today running
00:05:40.580 | agents on Temporal at scale in production.
00:05:45.300 | And what Temporal is bringing to them is incredible agility and speed because they can focus
00:05:52.740 | on writing their business logic and don't need to worry about reliability.
00:05:57.620 | The reason I put up the payments example here is to show you how mission critical some of the
00:06:04.420 | workloads that are running on Temporal are.
00:06:07.700 | And finally, you know, some more quotes here from customers, developers using Temporal around building
00:06:14.420 | agentic applications.
00:06:15.860 | I hope some of this is resonating in terms of the issues that you are seeing as well as you're going
00:06:21.460 | out and building these applications.
00:06:23.140 | All right.
00:06:25.300 | Let's talk a little bit about architecture and code because no talk here can be complete if we don't
00:06:31.140 | talk architecture and code.
00:06:32.580 | Right.
00:06:32.900 | All right.
00:06:34.180 | All right.
00:06:34.260 | I was hoping I would get some claps.
00:06:36.180 | So let's get this going.
00:06:37.620 | All right.
00:06:39.300 | So this is this is an example of an architecture before Temporal.
00:06:45.540 | What what we're seeing here is there is a lot of interaction and error handling that developers are
00:06:53.860 | being forced to code.
00:06:55.940 | And when you use code using Temporal essentially we can abstract all of that out into this concept of a
00:07:02.820 | workflow.
00:07:03.460 | A workflow is something that you write this is this is very much a developer focused tool.
00:07:09.620 | This is not a tool that a business user or a non-technical person would use.
00:07:14.260 | This is a developer going in and coding and building their applications using these SDKs
00:07:21.620 | around the concept of a workflow abstraction.
00:07:25.460 | And at the end of the day, when you think about agents, you essentially are just orchestrating
00:07:30.740 | a number of pieces around the interaction, the large language models, the chat history database,
00:07:38.500 | and the tools.
00:07:39.380 | Right.
00:07:39.620 | These are the key abstractions that are in play and you're orchestrating that using Temporal.
00:07:45.300 | What's the impact?
00:07:48.180 | You know, everybody's going to tell you they've got the best platform here.
00:07:51.620 | But what what is the impact for engineers?
00:07:54.660 | What we really are able to do is accelerate development.
00:07:59.140 | And you what you can do is take Temporal and you can put applications out in production in weeks.
00:08:07.140 | We've had customers with case studies where we've sped up their feature delivery velocity
00:08:14.260 | by over 6x once they've started using Temporal.
00:08:18.100 | You can reach greater scale.
00:08:20.340 | One of our customers is a consumer application that is scaling with events and they don't need to worry
00:08:29.220 | about handling any of that scale logic at all. So we, our cloud, will handle the scale for you.
00:08:37.380 | Of course, you know, once you've got reliability nailed, you can sleep better at night and that's always
00:08:42.900 | important. And with reliable applications, customers are happier. Now I've talked a lot. I wanted to
00:08:50.980 | walk through a example. I'm not doing a demo because of all the various issues.
00:08:57.860 | And it also seems like for some reason travel is like the the classic use case everybody seems to be
00:09:03.780 | demoing. So let's dive right in. So we've got a demo of a ticket booking agent. This demo is live at our
00:09:10.820 | booth as well. So if you're interested, you can go look at that at the booth. And what we'll do is I'll
00:09:17.380 | quickly walk you through a little bit of the architecture and how Temporal would work here. And so clearly,
00:09:24.020 | you know, some of the key pieces here are around the user, the system, which is Temporal and AI, your
00:09:32.020 | language models, goals and tools. And the way that Temporal works here is essentially going to be able to
00:09:42.340 | take this flow and wrap pieces of this in Temporal concepts. So for instance, the workflow
00:09:51.780 | defines the flow of the application, and it's written as code. So this is where you would orchestrate the
00:09:59.300 | interactive loops, you would receive, we have this notion of a signal, which is how the workflow gets
00:10:05.220 | input. We have a notion of a query. So there's a rich set of abstractions that you program against
00:10:12.580 | to build that workflow, that will essentially take kind of all of the pieces of this model that I'm showing
00:10:20.100 | you and translate that into code. And nowhere in there will you have any statements or code
00:10:27.860 | that we call plumbing code. That is, nowhere in there will there be statements like, if something fails,
00:10:35.060 | you know, keep retrying it. All of those pieces are handled by Temporal.
00:10:41.940 | We also store all of the workflow history so that you can go in and you can look at the visibility of what is
00:10:50.580 | happening as your agent is navigating this complex set of interactions. Temporal has the notion of
00:10:58.900 | activities. And so the tools that you use can be wrapped into activities. And again, this is just code.
00:11:09.540 | And of course, with LLMs, you can use whatever provider you need, we are able to help you validate
00:11:16.420 | the inputs and drive towards the goal. And again, failures are handled transparently by Temporal.
00:11:24.260 | And then interactions are managed through Temporal signals and queries. And they're stored in the workflow
00:11:34.820 | So there's a very clear record of how your agent is executing. And you can go in and look at that.
00:11:44.020 | You can also export that. One of the things we are hearing is customers want access to that history for
00:11:50.500 | compliance reasons or for being able to go and debug in their test dev environment. So we allow the capability to
00:12:00.020 | export that entire history. And you can use it for whatever sort of purpose you might need.
00:12:06.020 | And finally, here, you know, you can kind of look at the fact that we've got the ability to have
00:12:14.340 | loops. We can, you know, it's a very rich programming model where you can take the various sort of
00:12:23.460 | use case patterns for your agent and you can build them as a workflow pretty quickly and get up and running.
00:12:29.700 | And then Temporal Cloud, of course, is where we do all of the heavy lifting around the reliability and
00:12:38.260 | scalability pieces for you. So your agent, your workflow, the code actually runs in your environment.
00:12:45.540 | And Temporal Cloud is where all of the execution state, the call stack, the, you know, looking at all of the
00:12:55.060 | failures and retries, all of that is happening within Temporal Cloud.
00:12:59.620 | I know I'm speeding through a lot here, but definitely come by our booth as well. The worker is what I was just
00:13:08.020 | talking about. This is your code. It runs in your environment. It is essentially fitting into any of
00:13:16.020 | your own CICD practices. A big part of the Temporal focus has been on meeting developers where they are.
00:13:24.660 | We don't want you to change how you write code. We just want you to get more efficient and help you
00:13:31.700 | focus on writing your business logic and not having to worry about all of the reliability
00:13:37.860 | and scalability issues here. And this, for instance, is the worker code for the
00:13:46.500 | use case that I was just showing. I know this is a screenshot. What I wanted to show here is we've got
00:13:55.140 | this concept of a code exchange. Temporal, if you weren't aware, is an open source product as well.
00:14:02.180 | So you can go in and I know this conference loves QR codes for some reason. So you can go
00:14:07.700 | in and you can actually look at the code at the code exchange and see how Temporal operates there.
00:14:14.820 | Finally, Temporal Cloud is available. You can go sign up. We are giving away credits. So getting started
00:14:25.140 | and kicking the tires on using Temporal is fairly easy. You can go to code exchange. You can look at
00:14:31.860 | any example you want. You can run that in your local dev environment. You can run it against cloud.
00:14:38.980 | And you can be up and running pretty quickly here. And we are, like I said, we are on the expo floor.
00:14:46.900 | So come by and chat with us. We are Booth G3. Perfect. Thank you.