OpenAI have just released their API endpoints for TrackGPT. What we're going to do is have a quick look at what that looks like and how we can actually use it. So they've just announced it in this article here, introducing TrackGPT and Whisper APIs. Now, I'm not going to talk about Whisper APIs, but we've covered them before.

So you can go and watch those videos if you're interested. But what is really exciting is TrackGPT, finally, we can use it through the API, which is pretty cool. Now, this article doesn't really cover anything. They just kind of mentioned this, which is pretty impressive. So the cost reduction, and if we scroll down, I think they are more specific about it elsewhere, but we'll cover that in a moment.

But if we scroll down, we just see a couple of use cases, which is pretty cool. So you have Snapchat using it, these guys who I'm not really familiar with, Instacart, I think we also have Shopify here, which is pretty cool. They have TrackGPT, which is providing a chat interface, but they're also pulling in information from elsewhere.

So, you know, they're doing the whole knowledge base thing here and feeding that information into the chat interface. And we see more in-depth example of that here from Toby, who's the CEO of Shopify. And you see that they're bringing all these other resources and kind of fine tuning what they're searching for based on that.

Now returning back to the 2ChatGPT, let's come up here and we have somewhere near the bottom of the article, we have a little bit of code here, and it's just above this that they mentioned the price difference. So this is the ChatGPT model. All right, so if you've used OpenAI's endpoints before, you probably are familiar with Text Adventure 0.0.3.

Now what we can use is this model, which is apparently pretty easy to migrate from, small amount of adjustment needed to their prompts. Yeah, and it's a lot cheaper. It's a 10 times cheaper than existing GPT 3.5 models, which were already pretty cheap to start with. So this is seemingly better performance than this, and it's also 10 times cheaper.

So it's a pretty good upgrade. Now we have some demo code here, and then we can come to here to get a more in-depth guide. So we have chat completions, go down and some code here, right? I'm going to basically replicate this here. So we come to here, all I've done here is just pip and sort OpenAI.

I come to here, I set my API key. Now to get that API key, you need to go to platform.openai.com. You should see in the top right, there's like a signup or your actual account. So you go into there, view API keys, and then you just click create a secret key here and get going.

That secret key would go in here and then we would come to here. So this is our chat completion, create. So kind of what you just saw right here. So let me just copy this and we'll try it. So this is literally my first time trying this. So let's run this and just see what we get.

All right. So finish reason start context. So we get this. I haven't read this conversation. Was it who won the World Series in 2020 Los Angeles Dodgers? Where was it played? It was played at Globe Life Field in Arlington. Okay. That's pretty cool. Now what's probably more interesting is if we, you know, I want to see if this is actually the same model as what we would be getting from ChatGPT, like through their actual interface.

So I had a quick conversation earlier. It was, so I just asked, you know, what are sentence transformers? Gave me a bit of an explanation. Can you show me how to implement something like this? For question answering, gave me tip glasses. You know, it wasn't great. Give me some models that I, you know, wouldn't recommend using.

So I asked it, can you fix that? You know, so on. So I'm going to take these, let's just see how we'd actually implement this, but through the Python API. So I'm going to take my first question. Now here we have web user, so system user assistant, right? So this system here is like the primer for the model.

So this is kind of like before the chat. So this is before I even say anything up here. So we can just leave it as you are a helpful assistant. I don't know if that's going to produce the same sort of result as what we just got, but let's try and let's just say, you know, what are sentence transformers?

See what we get. Okay. Sentence transformers are a class of deep learning models that are specifically designed to generate high quality sentence embeddings. Okay. And let's compare that to what we've got on the other side. Okay. Uh, it's pretty similar, but this one is, I would say significantly longer.

So I imagine it's probably a token limit that we can set up here. Let's take a look. So maybe this is what we can, we can add in here. So this seems to be like a system message that they sometimes use to prime chat GPT. So let's replace the first part with that.

Let's put that up here. Primer. So knowledge cutoff is I think September. I don't know if we need to enter this, but September, 2021, and the current date is March, 2023. Kind of weird. All right. So primer, I'm going to put that in there and I'm going to rerun it and see what we get.

Oops. I need to also run this. Okay. Okay. So we get like slightly different output. They're slightly different to what we, what we got initially. Let's have a quick look at the chat completion and such a limit. In the case of max tokens, temperature and max tokens are too many options developers.

Okay. So I think this is the same as the usual inputs that we provide. So if we go max tokens and say 500, run this again. Okay. So we actually used not that many. Can I try and force it to go longer? Probably not. Yeah, but what we probably can do is make it go shorter if we wanted to.

Okay. Let's kind of short and let's do 50. Is that shorter? Completion tokens, 39, 55. So it is modifying it. So you can see the total tokens here, actually. So if I put that down to 50, we should not go 50, I think. Okay. So maybe it's 50 in completion.

Okay. It's cool. Right. So let's carry on that conversation with the other parts I added in there. So I'm going to increase max tokens. And what we will do is just copy in the other responses I got. So the first answer I got was this, pretty long. Let's pull that in.

So this one's going to be the role will be the assistant. And the content will be what I just got, but I need to format it a little bit better. Actually, I can go. Okay. It's messy, but it's fine. We can do that. And then I'm kind of curious.

What would it do if we fed that in and didn't say anything from the user? Let's see. Okay. So it's literally just like nothing else is needed, which is cool. So it knows. Right. Next one, I'm going to do user again. And my response to that was this. Let's see what we get.

Right. So we get a lot more here. Let me do response so I can format this a bit nicer. So we have choices. First one. So response choices. Then we have message. Let me run that. Okay, and I'm going to go with content. And let's print now. It's also in Markdown.

So we can actually print to Markdown. Let me have a quick look at how to do that. Naturally, let's ask chat EPT how to do that. How do I print Markdown in Jupyter? Jupyter notebook. How do I print it with Python? Via Python. So. All right, cool. So we do this.

Okay, let's try that. Perfect. So then we come to here and I'm going to display. Markdown. Let's see what we get. Right. It looks pretty cool. So. Implement sentence transformer of question answering. Okay. Yes. Yep. Okay. We using Squad 2. It's better than the one I got before actually, which was like this.

Which is like NLI meet tokens. I wonder if this one is more up to date. Curious. Okay, cool. So let me ask a similar question to this. I wonder here if we can go choices. Zero message. Can we feed this directly back into it? So we have content. We have role.

I think maybe we can. All right, so we'll just call it response one. Right. And what I'm going to do is come to here and I'm going to put response one. And then I'll put my next question. She has to be role user content. I would say do you have any other model was better suited to Q&A?

Right. Okay. See what we get. Response. Yes, there are a number. Okay, so that works. You can feed the responses directly back into there using these. I can't remember what it says. It's like a I shouldn't do that. Should do this. All right. So using this opening, I object cool.

It's good to know. Yes, there are a number of training models and then, you know, see we can we can take that out again. What was it? It's like choices. Zero message. I think yeah message and then it was content. So then we just display markdown. Let's have a look.

All right, nice. So Berkeley way Albert electric you a DPR nice. All right. So that I think looks pretty cool just as a you know, it's just a very quick intro to chat GP as I mentioned. This is the first time I'm using it as well, but it looks very cool.

It's pretty promising. I would like to try this with a few other things see how does the instructions and so on but we'll do that in another video for now. I'll leave it there. So I hope this has been interesting. Thank you very much for watching and I will see you again in the next one.

Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. (gentle music) you