Today, we're going to take a look at a ChatGPT plugin that is going to let us search through podcast episodes and basically answer questions based on those episodes and a few other things. So let me show you what we have so far. So here we're in ChatGPT. You can see up here, we have a little Lex Friedman logo.

That's because we are going to ask Lex some questions. So we're going to ask Lex about the future of AI. So ChatGPT is thinking. It decides that it needs to use the AskLex plugin. And then it's kind of just giving us a summary from a few different episodes here.

Okay, so we can see that it's also included the source of these episodes, which is pretty nice. And it's given us a few options. Okay, and then we also get this nice little, these cards at the bottom showing us where these are coming from. So we get a pretty good summary of AI being discussed in Lex Friedman podcast episodes.

And also this as well. This is, I think, one of the courses he did at MIT, right? So that's pretty cool. Now, what if I, maybe I want a little more information from a particular one of these episodes. Okay, so let's come down here. And I'm going to say, can you tell me more about the Eric Brian Jolson episode?

Okay, see what comes up. So it's again, referring to the plugin, even though I didn't say AskLex this time, it knows that I still want to use this plugin. Okay, cool. Okay, so there's a lot more information in this kind of summary of that particular episode. Right, that is really interesting.

And basically what it's doing there is searching through our entire database for that particular episode, rather than up here where we were searching for a particular topic, now it's searching for a particular episode, which is pretty cool. Now, let's just continue this conversation a little bit. So interesting, just let's talk about space exploration.

That's all. Let's see what we get. Okay, cool. Getting a few summaries like we did the other time. So we get, so it's two here. One with Darwin Newman and the other one with Ariel Ekblor. Now, that looks pretty cool. So this one's interesting talking about building space megastructures, that sounds interesting.

Let's ask about this. Can you tell me more about building space megastructures? Okay, and then it's kind of honing in on that specific episode. So it's seen, we haven't specified that we want to talk about that, but it's seen the previous interactions and decided that, yeah, we're probably talking about this particular episode, or at least this information can be found in that particular episode.

So again, it's kind of filtered down there, or maybe just from search, I'm not sure, but we definitely get a very interesting and relevant answer here. Now, that is just kind of a glimpse of what this sort of plugin can do. And naturally we can apply this to a ton of different podcasts, videos, or just anything you want really.

Let's kind of dive into how I actually built this and what is actually happening when ChatGPT is deciding to use the AskElects plugin. So let's come up to the top here and we have our first question, okay? AskElects about the future of AI. We can open this to see what is being sent to this plugin.

And we see that there's these queries, query the future of AI. All right, that's cool. So we're querying the future of AI. This is a semantic search. So we're using a vector database here. If that doesn't make sense, I'm going to explain it. But essentially you can think of this as being our question.

And these are the parts or the chunks of transcribed audio from Deluxe Freeman podcasts that are the most relevant or the most semantically meaningful to our particular question, okay? These are pretty big, but if we go through, we should be able to find somewhere where they're talking about AI, right?

You see there's a ton of things being spoken about here, but ChatGPT is actually managing to kind of find, okay, here, like when you start again, truly superhuman artificial intelligence, kind of by definition, I'll be able to do a ton of things that I couldn't have thought of in a greater world I couldn't imagine, you know, all this sort of stuff, right?

So that's the way it's getting that information from it then summarizes and gives us in that answer, okay? And we can see that there's three entries there, right? So we have that the Eric Brian Jolson episode, the Deep Learning State of the Art and the Andrew Ng episode, okay?

Which is what we see down there. And then there's this one. So I'm saying, can you tell me more about the Eric Brian Jolson episode? Here, if we take a look, you can see that the query has changed slightly. So I'm saying I want to know more about this specific episode.

And what we have done is actually given specific instructions to ChatGPT saying, you know, if someone wants to know more about a particular episode, you can search for that particular episode by adding in this filter and then filtering based on the title, okay? So the title, it actually knows the title from the previous interaction, okay?

So up here, it got the title here in the metadata of that result, right? So then it can search for that particular podcast episode, right? So I think that that is pretty cool. And you can see that the results here, then we're only returning chunks of texts from that particular episode.

And that's basically what is happening as we go through the rest of this. So here, it will just be a search, the query. And then down here, I'm not sure what it's doing actually. So, okay, so it's doing a search and it's also filtering for that particular episode because it knows the information is in there.

All right, so you can see a little bit of how this is working, but let's maybe dive into it a little more detail on how I actually built this. Okay, so let's just try and visualize what we have actually built here. So you just saw the chat GPT, you can think of it as almost like front end of this whole thing.

So we have chat GPT over here. Now, chat GPT, now these plugins that you can connect to chat GPT and the way that I think they work because the code for that isn't actually public is that you are basically passing your plugin, which is like a tool for your chat GPT instance to use.

You're basically passing that into the prompt or the initial prompt of chat GPT, maybe even like every prompt that is, maybe it's reformatted on its way to chat GPT, I'm not sure, but basically somewhere in the prompts, there is something that says, hey, chat GPT, you can use these plugins.

Okay, and then it will list each of these plugins with a little description about each one of these plugins, which we actually do write ourselves. Okay, so in our prompt, you know, may say whatever it normally says, and then it says, you can, by the way, you can use the Lex Friedman, I'm just gonna put Lex F database, and it will help you answer questions about the Lex Friedman podcast.

And when you see the words, ask Lex in the user's prompt, you should 100% use this. All right, that's basically what we wrote. So chat GPT sees this, sees your question, and it will sometimes decide that it needs to use this. If it does decide to use this, it is going to create a prompt to send to the Lex Friedman database.

Okay, and that prompt is going to be like, actually, you already saw it, it contains queries. And then within that, you have like a list of queries. Okay, so it can actually pass multiple queries in there, but we just have one query here. It's just how the thing is built.

And I'll explain that later. And yeah, so that is actually being passed to this ask Lex endpoint or plugin. And then we have this response from the plugin. Now, how do we get that response? That's probably the more complex bit that we will need to figure out here. So once we pass that query, okay, so we pass the query from chat GPT into our plugin, we are now in a space that is not necessarily anything to do with chat GPT.

Okay, so in this space is our own code or API, whatever that we have set up. The API that I've set up here is based on the chat GPT retrieval plugin API that was created by OpenAI. And you can actually see that here. So we go to github.com/openai/chat-gpt-retrieval-plugin. And what kind of tells you here that chat GPT retrieval plugin lets you easily search and find personal or work documents by asking questions in everyday language, right?

Sounds pretty familiar to what we have done. It's just rather than personal work documents, we've used Lex Friedman transcribed text. Cool, so we have that retrieval plugin, we'll call it the chat GPT retrieval plugin. CRP, okay, right? We've set it up to use OpenAI's text embedding model, ARDA002, okay?

So ARDA002, and you can basically think of this as the same as the GPT models that you've seen, but rather than generating text, this one actually generates numerical vectors based on the semantic meaning of the text that you're giving it, okay? So what I mean by that is, you're basically creating these vectors that go into this vector space here, and this vector and this vector here will have a similar meaning, whereas this vector all the way over here, because of this distance, this space between this vector and the others, this one will have some different meaning, okay?

So it's like we are mapping human meaning into numerical representations that we can then use to perform searches. So how do we perform searches through this space? Okay, so imagine we have our user query over here, it's going to be, what was our question? It was, ask Lex about the future of AI, right?

So the future of AI is basically our query. It's going to go into this R002 model, and that is going to create an embedding, right? And it's going to maybe place it over here, right? So then we know that these two items, these two documents that we already embedded, that they are very similar, and therefore probably highly likely to be relevant to our particular query, okay?

And all of this, this whole thing that you have going on here, where you have all these vectors and you're searching through them, all of that is handled by another service, which is the Pinecone Vector Database, okay? So the Pinecone Vector Database is basically a way of searching through this numerical vector space in a very efficient way at scale, right?

So you can have like hundreds of millions of these vectors or basically documents in there, and you'll be able to retrieve relevant items within around a 10th of a second or something super fast, right? So that is all handled by the Vector Database component there, okay? So we are then going to return those items out here, okay?

So we bring those here, and we'll see these are our, we'll say that we're going to return the top three most relevant items, okay? So let's say that was another item over here, right? So we're going to return those, and that is basically, this is going to be the output of our CRP, so our chat GPT retrieval plugin, okay?

So they go into here and they go back to chat GPT. Now, chat GPT has more information to answer the question, right? And it also, as well as the original text that we have over here, right? It also has where this text has come from. So it has the podcast episode titles and the URL to the podcast episode and everything else, which is exactly what you can see in here, right?

So these results, we have the original query, and then we have the things that we returned, right? So we have that text and the metadata title and the video ID here, which I think is actually being used for the URL. But that's something we should actually fix. So in the source here, we should add, or in the URL, we should actually add the actual URL for the video.

But that's something we can do another time. So what we have so far is, we've just basically augmented our original query with all of this other relevant information from the Lex Friedman podcast, right? So now, our query is going to be a ton of text based on all that stuff, and we've just returned it to JetGPT over here, and we're saying, okay, I want you to answer the user's question based on all of this information here.

The user's question is, and then it's whatever you, whatever you put in here, right? So that gets fed into there. And then, JetGPT is going to answer your question, and we saw that, right? So we got this kind of response where it's telling you a ton of stuff, right?

And it's sourcing the information. Look, we can actually click on here, and it's going to take us through to that episode, right? So this is, yeah, the Eric Brian Jolson episode. But there is still a little more to this whole thing than what I've just described. I mean, Wanda is actually the code to do all of this, which you can actually find if you go to GitHub.

The plugin code itself is actually all here, right? So this is the forked JetGPT repo. And yeah, I mean, this is, it basically contains the same code as what you would find in the JetGPT retrieval plugin. It's just I've modified it a little bit for our particular use case.

So for example, if you go to the well-known here, this is basically where you sell the instructions for JetGPT to understand your plugin. And you go to the AI plugin here, right? So I've changed the name for the model, okay? So the name that JetGPT we use to understand this model, the name that we understand it with, right?

We also have, okay, you can use this tool to answer user questions using Lex Friedman podcast. If the user says, "Ask Lex," okay? So we saw that "Ask Lex" earlier on. Use this tool to get the answer. This tool can also be used to follow up questions from the user and filters can be used to grab more information from specific podcasts like by filtering for a specific podcast title, right?

So that's the primary context, the primary instructions I've given JetGPT on how to use this plugin, which is they're pretty light. And then down here, we've just set up where our API is hosted, right? So we need to include that so that JetGPT knows where to find the opening API spec for our app.

So these are basically going to be the instructions that JetGPT will use in order to understand how to actually programmatically interact with our API or our plugin, okay? And there were some changes made there as well. Nothing significant, okay? So just put in the server, you have to do this.

And then there are some instructions around how JetGPT can use this again. So say this is an array of search query objects, each contain natural language query string, query, and an optional metadata filter, filter, okay? So we also say here filters can help refine search results based on criteria such as document title or time period.

So we can say these are particularly useful when a user asks for more information about a particular record. For example, the user may ask for more information after being provided with information from a specific podcast with the title, Our AI Future with JetGPT, right? So I'm using an example here so that JetGPT knows exactly what to do.

In that case, the filter field can then be used to return more information by using title, Our AI Future with JetGPT, right? So we've been very specific and explicit on how JetGPT can use this. Okay, so we build that. Let me show you some very quick code that just shows how I actually populated our database with all this stuff.

So there'll be a link to this. You'll be able to, in fact, it will be somewhere around the top of the video right now. To this notebook, you'll just be able to run it through and do exactly what I'm doing here. Okay, so there are a few items that you need, and we need this in general for the rest of the plugin as well.

So there's the open AI API key, which we get from platform.openai.com, the Pinecone API key, which we get from app.pinecone.io, and these other things as well. So we run through this. We're using a pre-built dataset in this example, but actually, if you wanted to do the download and transcribing yourself, you just open this, and you can actually see all of the code that I use there.

So I actually built this PodGPT library to help with that. It's just easier, okay? So in reality, there's just a few lines of code, okay? Cool. But that takes some time, right? Downloading all of these, like, mp3 files from the YouTube videos and then transcribing them using OpenAI's Whisper, you know, it takes a while.

So there's also this option, right? So I've already done all of this, obviously, for the Lex Rubin podcast. So you can actually just use my dataset here. This is hosted on HuggingFace datasets. There's 499 videos in there, okay? But they've already been transcribed. So then all you need to do is you initialize your -- this is your Pinecone index and also the embedding model that you're going to be using to actually create those embeddings and store everything in Pinecone, all right?

So that is going to be using OpenAI and specifically the OpenAI text embedding R002 model, okay? And then here, I'm just looping through all of the podcasts. We're indexing everything, and that is actually it, okay? That's all I did in order to index all of this text. Then from there, I just went over to DigitalOcean.

So DigitalOcean is just like a service where we can host APIs, which is pretty ideal for what we have, right? So we have this James Callum AskLex plugin. I basically just went over here, went to Create, Use, which one, Apps, okay? Clicked on Apps, clicked on Deploy from GitHub, and just deployed from this repo.

And then it deploys, and I end up with this, right? So I have the AskLex app here. You can click on there. This gives you the URL for your app. So you can copy that. You can open it, and we will probably see this, okay? So detail not found, right?

That's fine. It's just like the URL of the API. Without the endpoints or without any file extensions, you're going to find this, okay? So then I took that. I went over to -- let's go to New Chat. I went to Plugin Store. I said, "Develop your own plugin." My manifest is ready.

So this is the -- we saw this, the AIPlugin.json file. And then I just pasted in the URL, okay? So from there, we can see, okay, we've got ValidateManifest, ValidateOpenAPISpec. And then you can see, okay, we have, like, all the information that I include in there. Okay, so I've already installed it.

I'm not going to install it again. But then from there, all you do is you go to your Plugin Store. You go to Unverified Plugins, and you have your actual plugin here. And then you just go ahead and use it, right? So it's incredibly easy to pull this together.

Obviously, there are a few steps. It's not the most straightforward thing in the entire world, but it's definitely not the most complex. And given what you can do, it's pretty cool. So now I can just ask about random things. Like, you know, I can ask Lex. I want to know what he thinks about World War II, right?

I know this is, like, a favorite talking point of him, so there's probably plenty of things to talk about there. Okay, so, yeah, we get all this information. We're getting, like, pretty similar thing to before. And then we can kind of, like, draw down. We can ask, okay, cool, who's, like, this guy?

I have no idea. So we can ask about that. Like, tell me more about the thingy episode, this episode, and it will actually be able to do this. You know, it's just going to refer back. And we can also just use this. Like, we don't have to use it as Lex plug-in every single time, right?

So maybe I'll let this generate, and then we can ask another question. Maybe, you know, something about this makes us think, oh, I kind of want to know a little more about that. So maybe we can say, so there's this Ian Kershaw biography on Adolf Hitler. We can search that, see what it is.

So what do you know about that? Now, I'm not sure if it's going to use the SLX plug-in. Okay, so here it's not using the SLX plug-in because it doesn't need to. It already probably knows some stuff about this. Now, my internet isn't great, so it just cut out, but maybe I can try and regenerate that.

Okay, so we get a ton of information about this particular book that was kind of inspired by us reading through the Lex Friedman information. But this, in particular, we didn't need that in order to talk about this book. So Chad G. Petit knows this and has decided, okay, I don't need to ask Lex this question.

I can actually just give you all this information from my training memory, essentially. So, I mean, I think that is all fascinating and it's really cool how quickly we can just build something like this. It doesn't take that much. Now, I haven't dived super deep into the technical side of things here.

I showed you some of the things or I showed you some of the code. I showed you the GitHub repo. I think with that, you can pretty much do all of this. But if you are struggling to kind of follow along or you just want a bit more of a technical deep dive, I do actually have another video on Chad G.

Petit plugins and specifically the OpenAI retrieval plugin that we saw at the start of this video. So if you like and you want to go into technical details, I would definitely recommend following along with that video as I actually go through, like from start to finish, building a plugin, not specifically for podcasts, but for basically online code docs.

But yeah, for now, that's it for this video. I hope this has been interesting and insightful as to the sort of things that you can build with these Chad G. Petit plugins. If you are right now looking at where to find these plugins, there is right now still a wait list to get access to those.

So I will make sure that the wait list link is somewhere in either the description of the video or even at the top of the video right now. So you can go and sign up for that if you don't have access already. For now, I will leave it there.

Thank you very much for watching the video. I hope this has been helpful and I will see you again in the next one. Bye. (gentle music) (gentle music) (gentle music) (gentle music) (gentle music) you