Agentic Workflows on Vertex AI: Rukma Sen
00:00:00.000 | I'm Rukma, I work at Google Cloud on our Vertex AI product and at towards the end of the talk for
00:00:20.880 | those of you who don't know what that is I will discuss it just a little bit more but where I
00:00:27.000 | want to start today is with agents so this slide you're like understatement much right you're like
00:00:34.760 | yeah yeah right that's why we're here at this conference because generative AI is transforming
00:00:40.680 | how we interact with technology and if any of you are wondering hey is this is the rest of this
00:00:45.880 | person's talk filled with such groundbreaking insights maybe maybe not stick around and find
00:00:51.400 | out I kid I kid the interesting thing about this statement that I want to think about
00:00:58.040 | is what is the interface of that interaction where do all of these all of us whether we're developers
00:01:07.240 | employees parents students interface with AI I would posit that for the vast majority of many of our use
00:01:16.600 | cases that we actually want to accomplish that interface of interaction with generative AI is
00:01:22.920 | going to be an agent of some kind so the power of generative AI as I'm sure I don't need to belabor
00:01:29.320 | this point to you guys is immense but it can be kind of intimidating and is inaccessible to many people
00:01:37.800 | perhaps many people who are not in the room right now with us but we can think about these personas
00:01:43.640 | people we want to help people we want to build for right and that's kind of where I think agents come in
00:01:50.440 | and where they're really powerful they're the bridge between the models and everyday users
00:01:56.520 | so they help you go from speaking model language to speaking natural language funny joke no no no no
00:02:05.240 | I'm very sad you guys give me a laugh
00:02:11.080 | no all right all right all right all right I'll try I'll try I'll try
00:02:14.920 | well we'll we'll we'll make it there we'll make it there
00:02:19.880 | what I think is actually really cool though is that for actually all of us in this room we speak both
00:02:28.760 | languages so we're going to be the ones developing these agents right so we're going to be designing
00:02:34.760 | how they interact with people what kinds of limits and frameworks we're putting around them to make
00:02:40.760 | sure that you know we're being ethical we're being helpful we're being humane we're being safe
00:02:45.480 | and that I think is kind of magical think back to the days before the internet existed right what was the
00:02:54.360 | human interaction interface with technology it was machines it was things like appliances in the home
00:03:02.200 | and then the internet came about and the whole way human beings and technology interact completely
00:03:08.680 | changed we're all looking at our screens we use gestures like swiping and zooming and scrolling
00:03:15.480 | think about how cool it could be if you were the one building the next interface the next kind of
00:03:22.840 | boundary of interaction between human beings and technology I'm wearing a little necklace that says
00:03:30.520 | wizard in training because I think this is actually kind of magical that one got a better reaction okay
00:03:37.480 | we like wizards in this room given that though to quote my favorite spider person with great power
00:03:47.400 | comes great responsibility we all know who to attribute this to uncle ben in every version of every spider-man
00:03:54.200 | ever I promised spider-man to someone in this room I did say there was spider-man coming up in my talk
00:03:59.800 | and I'm hoping I delivered on that promise he's here he's here but the point spider-man is making I think
00:04:06.680 | is actually serious and something we should we should be thinking about so with the power to really shape how
00:04:13.240 | how people are interacting with AI does come responsibility we must ensure that these interactions
00:04:19.960 | are like I said safe humane and helpful when you think about like what is this responsibility I would say
00:04:27.400 | there are several kind of sources but some I would just highlight for everybody to think about our first ethical
00:04:33.880 | considerations what are our moral obligations to protect users who are using these technologies that
00:04:39.800 | have really great unlimited powers in some ways how can we build guardrails that protects people that keeps
00:04:47.240 | them safe prevents kind of the spread of misinformation and make it really clear when let's say an agent is
00:04:55.640 | producing something that's generated versus when it's producing something that should be taken as a true fact
00:05:02.360 | we should also think a good bit I think about safety cyber security data privacy where are we storing the data
00:05:11.720 | that we reason over with these models with how are we thinking about making sure that we're safeguarding people's privacy
00:05:19.800 | with the rise of a lot of things like wearables and kind of just a lot of what I like to think about as like
00:05:27.160 | unobtrusive compute where it's just out there in the world these become I think even more important
00:05:33.400 | you know things to think about so great I talked a lot about agents and how we should think about making them
00:05:43.400 | but let's talk really quickly about what an agent is now real talk the reason this talk was supposed
00:05:51.800 | to be open models is because we did have a last minute schedule shift and fully true story before I
00:05:58.680 | knew I was going to deliver this talk and I was you know one week ago registering for this and they asked
00:06:04.280 | hey what is it you really want to learn I said what's an agent really so so actually really curious about
00:06:12.280 | this but this talk is not actually going to focus on kind of the philosophies and ontologies of agents
00:06:18.360 | if you want to chat with me about it please drop by the google cloud booth I would be happy to discuss
00:06:22.520 | this with you can we appreciate that I got a spider-man reference and ontology in the same talk very proud of
00:06:28.920 | myself okay so given that we're just going to move forward with a working definition and what is a working
00:06:38.360 | definition this is probably the kind of broadest most overarching definition you can think about
00:06:43.720 | for our purposes an AI agent simply is a system that's designed to achieve specific goals by interacting
00:06:51.160 | with its environment so let's break that kind of down into what its key components are so at the heart of
00:06:57.800 | every AI agent is a powerful model often this is based on large language models right this is the model that's
00:07:04.680 | responsible for reasoning over what are the goals of this agent kind of determining what the next best
00:07:10.920 | plan of action is and then guiding its behavior think about it as your agent's brain or executive center
00:07:17.880 | if you will then let's think tools so an AI agent doesn't just think it also acts and I think this is
00:07:25.480 | actually a key piece of the definition where you can separate it from something where the primary
00:07:31.720 | function is just thinking or reasoning or generating with an AI agent you do want to have an action
00:07:38.600 | included so this is where tools come in tools are if that if the model was the brain tools are your AI
00:07:44.680 | agents hands this is where you get to interact you can do things like fetch data from the internet more
00:07:51.960 | complex action calling external APIs to do things like say book flights process payments etc and then
00:07:58.360 | orchestration is the glue that kind of holds everything together it maintains memory and state which is
00:08:04.600 | really important it keeps sort of track of the goals and if in this analogy of brain and hands
00:08:10.920 | orchestration is really the nervous system tying it all kind of together so these three components work
00:08:17.160 | together kind of allowing the AI agent to function autonomously and accomplish tasks that being said
00:08:23.800 | I really quickly want to say that there are different types of AI agents and some of these you could say
00:08:29.720 | have existed for a very long time way before generative AI really you know boomed in the marketplace
00:08:36.120 | so there are deterministic agents generative agents and obviously kind of hybrid agents
00:08:42.680 | deterministic agents are basically following a fixed set of rules or algorithms to make decisions
00:08:49.560 | so given a specific input that type of agent is always consistently going to return the same output
00:08:56.360 | so I'm sure you can tell this is quite different from when you're say prompting with a generative agent
00:09:03.160 | an example a very simple example of this could be a calculator when you give it the input of two plus two it
00:09:11.080 | will always return four unless something's deeply wrong and you're in a mirror dimension let's hope not
00:09:16.520 | generative agents on the other hand are more creative they kind of will work best in use cases
00:09:27.400 | where you want to be creative you want to combine rules in ways that they haven't been combined together
00:09:33.720 | before and they are capable of a much wider range of diverse outputs kind of based on the input they
00:09:40.680 | receive so an example a simple example of a generative agent is a chatbot designed to answer kind of
00:09:46.200 | customer questions a customer service chatbot when asked about kind of a product it will generate generate
00:09:52.600 | and hopefully helpful and informative answers based on whatever data source it has about your company's
00:09:59.080 | products etc and hybrid agents combine sort of the strengths of the two an example of this could be like a financial advisor
00:10:05.640 | that uses deterministic agents to analyze the market and predict the right places to invest but then uses a generative
00:10:13.320 | agent to actually communicate this or go out and talk about this strategy to customers
00:10:20.600 | okay so this is i think where things get really interesting so given the different types of agents
00:10:28.200 | you can actually architect them quite differently across the spectrum so from single agent to multi-agent
00:10:36.280 | architecture i think increases the kind of sophistication and complexity that your agent is capable of
00:10:43.080 | very very very quickly so just to like kind of very quickly go over the single agent one this is not
00:10:50.280 | i think hopefully new to most people this is where a single model is just responsible for everything
00:10:55.560 | reasoning planning acting super straightforward architecture to implement you just provide it with
00:11:00.600 | instructions and a set of tools to kind of achieve a goal right so what is the problem here great like you
00:11:08.440 | know great tell it what to do it's gonna do it it's gonna return the output well have you ever tried
00:11:15.160 | a prompt like count how many instances of the letter a are in the word banana and the model will say four
00:11:22.040 | and then you say hey can you check that and then it will say two and then you say hey can you check that
00:11:27.640 | and then it'll say one so in cases where you're trying to deploy a production ready app something like
00:11:34.840 | this can you know really be a problem so now we get to a much more powerful way to design agents which
00:11:43.000 | is multi-agent architecture so just like complex human systems like let's say a company you work at
00:11:49.560 | have people specialized in different roles working together to achieve a common goal that's what multi-agent
00:11:55.240 | architecture does as an example of this is a customer service system so let's say there's three levels of agent
00:12:02.280 | level one you have a dispatcher agent the job of this agent is simply to triage everything that comes
00:12:08.120 | in assess the customer's issue and determine where to route it so it triages second level agents subject
00:12:15.000 | matter exports these agents are trained in specific subject matters but maybe specific product lines or
00:12:21.320 | specific regions if that's how your company functions and when they are assigned a case by that first agent
00:12:28.520 | they have the expertise to respond and then finally as a level three check you also have a supervisor
00:12:33.640 | agent that quality checks the work against a predefined data set it that agent has the ability to go in and
00:12:41.240 | solve some issues for example um fun story i created a multi-agent kind of architecture once and the
00:12:49.560 | supervisor agent was supposed to return the the sentence this is not good enough please try again if it
00:12:55.720 | wasn't happy with the output and it just kept doing that it did not like anything my first agent did
00:13:01.880 | until i went back and like recreated the whole whole thing okay so as agents are becoming more and more
00:13:10.280 | common across industries we're largely kind of seeing development in four types and i just wanted to
00:13:15.240 | give like show you really quickly like what a set of use cases for agents could look like so with customer
00:13:22.440 | i already talked for example through what it would look like for a customer support agent
00:13:27.400 | but also things like e-commerce being able to support b2b supporting travel if you are a travel vendor
00:13:35.720 | for example there's also internal facing employee agents hr things like enrollment benefits questions
00:13:42.520 | those things sales of course as i'm sure you can see would be a great opportunity payable supply chain
00:13:50.040 | so those are kind of thinking about who the agent is targeted to and then knowledge agents
00:13:55.160 | are specialized agents in terms of what exactly is their subject matter of expertise so you could have
00:14:02.600 | an agent that's specifically very good at answering legal questions for example
00:14:06.840 | and then finally we are also seeing through the use of multimodal use cases a huge uptick in voice agents
00:14:16.520 | especially in scenarios like say a fast food drive-through so i'm sure you can imagine what
00:14:22.920 | like where a voice agent would come in here you go in you make that order using your voice and the agent
00:14:29.320 | basically transcribes that and sends it through to the ordering system so that the person at the delivery
00:14:34.600 | window can go ahead and serve you okay so we're more than halfway through this talk so quick moment so we
00:14:43.080 | looked at why we should care about agent design then we kind of peeked under the hood really quickly to
00:14:47.880 | talk about what what the kind of components of agents are then we thought through architecture a little
00:14:53.880 | bit and kind of looked at what the top use cases are so just before wrapping up the last thing i'm going to do
00:15:01.160 | so you can see my shirt i'm going to talk about tooling and specifically google cloud's developer platform
00:15:08.680 | vertex ai so google cloud's developer platform vertex ai offers essentially a full life cycle
00:15:16.600 | ai development platform so whatever it is you want to do whether it's things i didn't talk about today
00:15:22.440 | like uh calling models and fine tuning them or it is stuff like i talked about today such as building
00:15:27.800 | agents we offer you a spectrum of ways to enable that whether that's super low code even no code in some
00:15:35.080 | cases all the way up to very high customization high code methods to do it vertex offers you access to
00:15:42.040 | 150 plus models obviously all of our first party google cloud models but we also have all of anthropics
00:15:48.600 | models on there llama 2 and llama 3 as well as a whole bunch of open source models we try to make it easy
00:15:55.080 | to prototype so you can get apis for all of this and start experimenting start building without having to you
00:16:01.640 | know go through a whole bunch of setup we also want to make it very simple to kind of be able to deploy
00:16:08.360 | and have peace of mind that your security and all those enterprise concerns i was talking about earlier
00:16:13.800 | when it comes to things like data privacy etc are taken care of so we back all of this with google cloud
00:16:20.120 | level enterprise readiness security uh you know things like compute orchestration so you're not ending up
00:16:27.400 | paying too much for something if you don't have to and all of that um i wanted to quickly flash model
00:16:34.680 | garden for you since this is the piece of vertex ai i did not cover in today's talk but model garden is
00:16:40.200 | where you can go in pick your model get you know fine tune it we have a couple of model eval workflows that
00:16:47.560 | you can run to try to match the model to your specific use case as well and then finally agent builder as i
00:16:54.680 | said all the way from no code to kind of full code ways to build those cool exciting agents that i was just
00:17:02.680 | telling you about the last thought i want to leave you with is this we're building four builders vertex ai is
00:17:10.920 | designed with developers first in mind and all the choices we make as we build this from training and
00:17:17.560 | quick start resources all the way through to deployment is for you so we love feedback please stop by our
00:17:25.800 | booth tell us if you've used the product what you love what you hate we would love to learn from all of
00:17:32.120 | you with that i will ask you to please do me a giant favor and take a quick survey to tell us how we did
00:17:40.920 | and mayveveen my colleague in the green skirt there will give you a cute vertex ai branded water bottle if
00:17:48.360 | you show her you completed the survey that's it thank you guys
00:17:59.960 | you