Okay so we're going to move on to chapter two of the Agents SDK course. In this chapter we're going to be taking a look at the built-in tracing that comes with the SDK. So just to give you a quick preview of what tracing looks like when you're using it, it looks like this.

So this is all built-in. I haven't, or at least for most of these, I have not done anything much beyond passing in an API key. So this is one thing that Agents SDK does very well in my opinion. Just by passing my OpenAI API key and turning tracing on in my dashboard, and I'll show you how to do that in a moment, we get all of this information about what is going on within our Agents when we're using Agents SDK.

So this is an example from a future chapter that we are going to be working on, which is where we're going to be building this essentially GFK RAG pipeline. And in this example we have this GFK document assistant. You can see that this is a chat response. So this is when we're getting the LLM to generate some text.

In this case I can see that the LLM generated some text. It decided to use the GFK file search tool. Here I must have been searching something to do with some guy here, Valerie. And we can see what was returned. Okay. And we also get to see how long each one of these steps took.

So that initial tool call from the LLM is about a second. The response time there, super fast, it's about, well, 100 milliseconds there. Then the final response, the final response has a bit more to process because we've just given it a ton of documents to read and it needs to respond.

You can see that it went through and got all this input here and then the output was this. So that's pretty cool. We have all that and you'll see it's super easy to sell. This is a pretty short chapter. So switching across to our code and make sure I'm using the agents SDK course environment there.

If you are in Colab, I will just pre-warn you that tracing doesn't work, at least not by default, in Colab as of today. Maybe in the future it will, but if you do want to run this, you will unfortunately just have to run this locally. You can't do this part in Colab.

At least not right now. In the future, maybe you can. So we come down. If you are running this anywhere remotely, of course, you will need OpenAI agents. So that is to install. And the first thing I'm going to do is just see how we set this up so that we can go into the OpenAI dashboard and see all these traces because by default, if you're part of an organization, you can't.

This information is by default only accessible by the organization owner. So if you're an engineer in a larger company, you will need to go through a few steps, which I have outlined here. So we first need to click that little settings icon at the top. We're then going to click on data controls.

We should find a little section here, which allows us to set the log visibility, right? And this includes traces. So you're looking at your response logs, stored, check completions, and traces. That will need to be set to be visible either to everyone or visible for selected projects or just visible so that you as a engineer can see it.

So once you have selected the required permissions there, you click save. Now, once you've gained access, you should actually be able to see traces already. And you can do so by going to your dashboard, go to traces, and then you see here we have all of these traces. So you should be able to see this page.

If you can't, you need to talk to whoever owns your OpenAI organization and fix that. Once you've confirmed, you can see everything. Let's switch back to the notebook, and we're going to start creating some traces. So we will need an OpenAI API key, of course. So you just enter that here.

Then we're going to go ahead and create a basic agent. Now, as usual, agent object will have that name instructions and model. You will see these in the tracing dashboard. Okay, so we're going to start with tracing prompt agent, speak like a pirate and the model there will be GT 4.1 nano.

So we initialize that, come down to here, and we're going to say, write a one sentence poem. We run that. And what we should see, if we navigate back to our browser, click on the most recent trace, and we just see, okay, we've got that tracing prompt agent. So that's the agent name that we just created.

And we can see what happened inside there. So it used 43 tokens. It used the 4.1 nano, and this is the April 14th model. The instructions were speak like a pirate. Input from us, the user was write one sentence poem. And we can see the assistant output there. So that is pretty self-explanatory.

There's nothing, we literally did nothing. We just entered our API key and it worked. So that's great, but there is a little more to this. So I want to go through and we'll just see what else we can gather from traces or how we can even set our own custom traces.

So starting with the custom traces, to customize our tracing, we use the trace function from the agent's SDK library. And with this, we can add or modify any of the parameters that we are tracking. And that will look a little like this. So we'll have this with trace and then we would have any of our code that we want to trace here.

And that doesn't necessarily actually even need to be agent's SDK code, right? If you have a particular tool and there's something you want to track with that tool, you can, you can do that using this. Although that isn't necessarily the intended use of the traces here. So let's come down a little bit and we're going to import that trace object.

And I'm doing with trace, I'm setting the workflow name. So I can, I can do this. I can set a default name. I'm going to call this the prompt agent workflow. You can see that I'm setting a group ID here. So if we want to group things together, we can do, I'm going to call this one agents SDK course tracing.

So all of our, the traces we create in this little chapter, we can put under this group ID. And what I'm going to do is run this. Okay. So I'm just running the same thing again, just within the trace. Let's run that. Switch over to our browser and we should see an update here.

You can see the workflow, the name is changed, prompt agent workflow. Now by default, if you don't set the workflow name, it will just be agent workflow. If you do set the workflow name, this is what will appear, right? Your custom workflow name. Now for all that other information, well, the group ID actually that we added, we'd have to click in to here.

We come across to details, I believe, and we can find a group ID there. Okay. So that's that agents SDK course tracing. Now, if you come up here, you can also search. So agents SDK course tracing. And now we can see, you know, I've done this a couple of times.

So you can see all the, all of the cases where I have set my course or group ID, sorry, to be agents SDK course tracing. So I could go into here, go into details, and you can see that there as well. As well as that, you can, as you can probably see here, search by the workflow name.

So I've got web search agent. Let's look for that or filter for that. Our filter for that, let's remove this filter and see how many of those we have probably. Oh, just three. Okay. So we have three workflows where I set to web search agent. So you can, you can filter by that as well.

Now let's come down and let's have a look at how we would trace tools. So we are going to be using the web search tool from open AI. We're going to be providing this as a tool, obviously to a agent. The agent's name will be tracing tool agent. Yeah.

You can see that the instructions here are your web search agent searches the web information on the user's queries. Okay. And we're using 4.1 mini here. Nano isn't supported for tool use or it's not recommended for tool use. So it's best to use mini there. Run that. And I'm going to say, what are the current world headlines?

And this will usually come up with something. Yeah. Yeah. Well, fun in, in, in some way. One thing that I will just know here is look how long that took. Yes. It's very long web search with the built-in opening our web search tool. I just wouldn't generally recommend it too much.

It's slow, slow, and not that good. So it's fine. You can use it, but I would use other tools if you can. So let's see what that looks like in our traces. Okay. So we have this agent workflow. That's the default name. So that must be it. Let's have a look.

Okay. Let's take a look in here. So we see, okay. It, there was the web search tool was used. We can see, okay, user, what the current world headlines search the web. It has, okay. Middle East. Ukraine. Ukraine. It opened a page for me. Okay. Back to this. International relations, natural disasters, science and technology list.

That one's nice. Very cool. So, yep. I mean, you know, world news. Lovely. That was actually better than a lot of the world. That's actually not bad for the OpenAI web search. A lot of time, I get a lot of rubbish from it. So it could be worse. So yeah, we can see what happened there.

Oh, and this is just the output, right? So this is like a really big response that we got from our agent with a ton of sources, ton of information. So really not too bad, actually. Even for that 13 seconds, I suppose. It could be, it could be worse. It could also be better.

Okay. So we can come down now. What, you know, what we just saw is that we had the, it was just called the tracing tool agent. Maybe what we'd like to do is actually give that a more, the workflow a more specific name. So in this case, we're going to call it that web search agent.

It's what we saw before. I'm also going to add metadata. Okay. So we can add metadata in here to, well, we can, we can search or filter based on that metadata. I'll show you that in a moment. So let's run this and jump across to our browser. So we can see, we have the website agent workflow in here.

Okay. It's, I think it's still running if I'm no, no, no, it has. Oh, that was, it was quite fast that time. So 6.6 seconds. And yeah, we got, we'll get more news there. The, we can look in the details. We have group ID. We can see metadata here and we can filter by that metadata.

So if I go into here and I write tools, web search tool, and we can see all of the cases where that metadata has been added. So you can, you'd add a lot of various metadata in that, whatever you need basically, which is, is really helpful. Great. So that is our quick introduction to the agent's SDK tracing.

As I think you've seen, it's pretty useful and it requires minimal setup on our side. We, you know, we kind of went in there, we're using the, the trace context, but you don't even need to do that. It's I, I honestly, I pro in most projects, I probably wouldn't do that.

I would just set my API key, allow the traces to be collected and go and review them when I need to review them, which I think is just really helpful. Now, that is it for this chapter. It's a quick one. So I will see you in the next one.

Thanks. Bye. Bye. Bye. Bye. Bye. Bye. you you