so thank you everyone we have a very full room today so we'll get started with the repo here in a second there is a QR code here and then in a couple other slides that all link to the same place will be using a Python notebook and we'll be doing graph rag with some real data using Neo4j so it'll be myself I'm Zach and then we have Andreas or ABK in the back who will be helping and then we have Michael hunger somewhere else in the room who can help answer questions and and Tomaj as well hello Tomaj back there in the base who can help answer questions as well as we go through so this is an interactive section session we have only two hours so we're asking to save the big questions to the end but basically we'll be popping back and forth between the slides and then between hands-on work inside of a notebook so if you do get stuck with anything raise your hand we'll come over and try to help you through and try to stick to our schedule as well with around 10 to 15 minutes per each section and there's about five sections for us to walk through so before we start there's two things we want to do if you haven't done them already the first thing is we want to create a blank Neo4j graph data science sandbox this is going to be at sandbox.neo4j.com so basically what you're going to do if I go ahead and click this link you might have to create an account really quickly on this page and then you're going to click for data science here and then it's going to be this if you can see this blank graph data science thing here so you're going to click on that blank sandbox graph data science and then your that will create an instance for you and it should come up with username and password that you're going to use inside of the notebook so this will be the database that we'll use throughout the course the next thing that you're going to do is go to the notebook so the notebook is going to be in this link and also by the way this QR code so I'll go back to that in just a second if you missed it but before that I just want to show you this GitHub repository and it's gen AI workshop is the notebook so what you're going to want to do is you're going to want to go to this notebook and then you're going to want to click open in Colab this will require a Google account if you don't have a Google account you can use VS code or you can run it locally that's fine but then when you go in if you are using Google Colab the internet is a little bit slow so it might take a while for it to load here hopefully once everyone gets settled it will go a little bit faster but basically what will happen is you just want to go to the top menu and you're basically going to go like file and you're going to save a copy and drive just so that you have the ability to edit it so you go file and then you say save a copy and drive and then it will create another copy for you so I'm going to go back to the slide and I'll give everyone a little while to get that all set up actually too while we're waiting on that 15 people can still sit in the front if you don't want to stand in the back so feel free yes it might be a good idea for you to start running the notebook cells so especially the first cell the pip install just because that might take a little while if when you just start that might it's probably the longest running cell in the whole notebook is just loading all the requirements for today so to say again create your sandbox blank sandbox at blank sandbox gds graph data science and then you're going to open that notebook create a copy of it or have it locally and then start running that pip install cell right here okay so what happens i did the blank sandbox instead of the blank data science sandbox i did the blank sandbox instead of the blank data science sandbox that one will maybe work but what i'd recommend doing is just terminating that instance so if you go to the um if you go to the screen once it's there just uh press terminate there's like a garbage can icon and then go and create a graph uh data science sandbox yeah blank it's blank sandbox dash graph data science is the one we want so the main difference is actually that the graph data science sandbox has a little bit more of memory so as we're doing some more involved operations it's just faster to complete so the regular blank one should still work but it will take longer so the graph data science one will just be faster so it runs with a little bit more memory yeah i know it's we need to work on that it's a little bit tricky alrighty so um i'll leave this here is there can i get a show of hands of how many people have the notebook set up okay so we're pretty good um is there is there anyone who still needs to stay on this pr code okay we'll go ahead and move on um so i'm gonna do like my general spiel that i normally do i think for a lot of us this is gonna be review um we're gonna be talking a lot about retrieval augmented generation today um and for those who are you know unfamiliar right retrieval augmented generation is just when you have this step in the middle when you're communicating with a large language model where instead of sort of like in a chat gpt like way where you're just sending a question and getting a response back you're going to have that LLM application go and pull relevant data through some query um that it can then use to reduce hallucinations provide more domain specific context enable better traceability etc and when we talk about graph powered retrieval augmented generation and we'll use the word graph rag kind of throughout this course is we're talking about the combination of three things that we're gonna look at today so that's gonna be vector search which i'm sure a lot of you are already fairly familiar with knowledge graph traversals which is going to be traversing over structured data in addition to the vector search and then we're gonna do some cool things with graph data science so we're actually gonna create a new type of embedding a graph embedding and we're gonna use that that vector to do some very interesting things for recommendations with structured data um and today so um everyone does chat bots which chat bots are wonderful i wanted to do something a little bit different today um so we're actually going to do an example of using a large language model to construct an email um sort of like a recommendation recommendation type of email um and this is going to be for clothing so imagine um someone's looking for something like a halter neck top right how do you choose what to buy why well you need a list of things that are available in your online store you need to match that specifically for that specific customer right which different customers might have different preferences um and then you might want to have other recommendations to pair with that item like maybe a pair of pants um or a hat or other accessories and things of that nature and so what we're going to do is sort of build an AI fashion assistance so to speak to help scale this um so we're going to use real world data from a Kaggle competition from H&M um and we're going to do some search and retrieval we're going to add some context for personalization a small recommendation engine and we're going to wrap that into sort of an LLM chain inside of LANG chain to show you how to generate that response um and the idea behind this right is again we're combining sort of this we're combining sort of this knowledge graph which provides this context and this enrichment with the large language model giving that reasoning and that creativity and sort of the we're giving some liberty to the large language model to kind of take what's in the knowledge graph based on the customer's purchase behavior and other textual data and vectors um and and kind of say use your creativity to put something together for this customer basically um and while we're doing sort of a fashion recommendation thing here this same process could could work for different types of support agents um or a lot of different types of internal search and customer experience use cases um and this is what we're going to build today um um what's that we're going to be sharing the deck deck is in the github repo i need to it's not the most it's not exactly the one that i have here but i'll update it tonight pdf pdf the pdf of the workshop slides is in the repo yep yep yep um so this is what we're going to build today it's a little gradio app um where basically we're going to enter in some customer information their id a time of year um and some of the things they're interested in and we're going to generate an email that provides various recommendations to that customer so imagine you're trying to create targeted marketing content or whatever it may be this is sort of an example of how to do that and these are the specific technologies we'll be using um so we're going to use open ai for the language model we could of course substitute that out um with vertex ai or gemini or whatever we wanted to there we have lane chain that we're using to orchestrate everything so that's going to be creating the vector stores and also the large language model chain we're using colab obviously and then gradio just to kind of show you an example of a very simple ui that you can put on top of that just for demonstration purposes and this is going to be how we're splitting the course up um so basically there's this journey that we're going to follow we're going to start with building our graph that's going to be built inside of that sandbox instance that you just made after that we're going to be showing you how to do some very simple vector search then we're going to explore how to integrate knowledge graph traversal patterns into that to better personalize the retrieval data so not just based on sort of text similarity but also on similarity based on co-purchasing behavior and then when we get to recommendations we're actually going to use an unsupervised machine learning model to generate graph embeddings and we're going to use k nearest neighbor there to locate other recommendations that we can make inside of the graph and then we're going to put that all together inside of an llm chain to show you how you can use that to create a better more curated response with the language model alrighty so that was a lot i'll go ahead and get started with graph building again the way this will work is i'll spend a little bit of time in slides just talking about what we're going to do then we're going to hop over to the notebook and we we're going to run through those cells you you might already be running through those cells because i know a lot of us we it's going to be pretty easy to just sort of click through there's not actually a lot of hands-on coding you need to do but we're going to walk through that anyway so this is the data model that we're going to use a reminder right that we're working inside of a graph database which means that everything that we're working with is natively represented as nodes and relationships inside of a graph so we have these entities we have customers and articles and then products so basically what this graph is telling us is a sample of customers that exists different purchases that they made of articles of clothing so think of these like the halter tops the jeans the boots you know all the accessories a variant of so an article is a variant of a product in the sense that you can have a certain style of jeans but they can be you know have different color patterns different sizes etc so the product is sort of like the product and then the article is like the variant of that product we also have each department that the article is sold in we're not going to use that department node very much though in this specific workshop and of course all of these things can have properties associated with them so we have for example the customer id to uniquely identify a customer for articles we have product we have color and different design patterns and then for the product we have sort of the description of the product the garment group that is in the type of the product and the name of the product etc so again i think we've we've all done this step hopefully by now so i will go ahead and enter into our notebook so hopefully everyone can see this this was the first cell that we wanted to run before just because it takes a while if you haven't run the first cell for pip install please go ahead and do that now once you've run that go ahead and just kind of keep going down here run the import this just gets us all the libraries we need and now here's the part where we're going to have to inject some things here so when you made that sandbox instance i'll actually go to my i have to log out and go to a different account here you should get connection details so if i go and i just click this and i go to connection details so please don't use mine so we're not fighting on the same database um but copy copy the bolt url here um and you're going to go back into your uh workshop code and you're going to put that here for your neo4j uri is going to be that bolt connection the password as well is going to be um is going to be this one right here the password and then you're going to keep the username as neo4j so you don't have to change either of these two we're going to keep the llm as gpt4o and then for the open ai key if you have your own please please do use that if you don't if you click on this link uh it'll take you to a workshop key that we'll keep live for a little bit that you can use um to run through the workshop and so basically you're just going to copy it in kind of like i've done here um and you're going to give that a run and then this next cell is just if you wanted to do it the proper way and have a separate environments file which you can skip or you can do that if you want um uh but that uh is not necessary so once you've run that we get down to the knowledge graph building section and we've already talked about the graph we're going to import and basically what you're going to do is you're going to connect to neo4j so this is our graph data science client we have multiple different drivers and different wrappers around those drivers to connect to the database i'm using graph data science for this just to simplify things it's basically you're just going to hand it the uri the username and the password and that'll get you connected and there's some warnings there that we're going to get rid of and then if you can run this cell successfully right after it this gds debug that means you're connected and you're good to go so you're connected to a blank database once that is run there's one cell here that i have for pretty much loading 90 of the data so you're going to go ahead and run this um what this is doing is i've gone ahead and there's another notebook inside of this repository that's prepared these csvs so there's uh five csvs here um there's department product article uh customer and transaction um and so most of these are self-explanatory the department is the department node the product is the product node article is the article node etc customer the customer node and then transactions are all the purchase transactions between customers and articles um so the if you were to dig into this code and because we only have two hours i mean we can spend all our time talking about this but i decided uh not to so we can we can get to more of the vector stuff um this is going to create uniqueness constraints which are going to be useful for for querying later um in neo4j so it's just a general best practice um and then if i go there's basically two functions here one for loading the nodes and the next for the relationships um if you were doing this sort of in a non-automated setting um this is kind of what the cipher looks like so just like um um so just like in sql how you or for rdbms you have sql for graphs um and for neo4j specifically you have cipher so what this code is doing that i have above is basically for loading the different nodes it's generating this cipher so you see it says use this cipher query unwind merge set so that's the actual code behind the scenes right the cipher code that's being run to load this data um so if you are interested in how to load more data into neo4j you can take a look at that um and we have other graph academy resources as well if if you wanted to dig into that piece um in more detail around just how to model data and how to load data into a graph um and then we'll stop there and i think we're at 15 minutes now so we can probably do um i'd say five to ten minutes and we can wait for people to catch up as a quick show of hands how many people have made it through the loading awesome okay that's really good um and then michael and tomaj so instead of this um yeah so actually so don't you don't you've got the credentials as well okay so instead of uh single sign on there's user name password just add the password you see it says we're leaving it so this is like the built-in cipher query so one thing that's going to be doing is click on this database icon there it should end up looking for it looks like it might be blank still so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database and it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database and it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database and it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database so it's going to be in the middle of the database does anybody else need some help that I can tell Tomaj to help with everyone's good so there might still be some people having network connectivity issues we've seen a couple different variations on that we'll try to help figure those out and get that sorted I think we can maybe okay awesome oh it's crowd it's the stupid by the way I have to say you're all pretty good for such a big room everyone being on top of things I'm really impressed so well done everyone what's the software they installed on that yes fun security software on my laptop preventing me from connecting USB devices so hopefully it's there if it disappears again let me know one thing that I wanted to point out someone had a good question about what we were loading so there is there's a couple things that we do kind of at the end of this block one of them is I'm going to create a fake sort of dummy URL because it wasn't included in the data so we'll use that to help the LLM will basically provide a link to the product which won't be a real link but you can imagine substituting a real link into the database so we're just creating a new URL property for that and then this thing here is creating it's basically taking other properties that were in the database so when we looked at the product and we'll see what these fields actually look like in a second but there's a product name a type a group a description all this stuff and this statement here is just creating a new text property to kind of squish all of that together in one place and we're going to use that for our upcoming vector search so we're going to index that so that's why we're creating this one text property so just wanted to mention that really quickly before we move on to our text section so now we're going to start talking about vector search and we're going to do some initial search and retrieval so when we think about what embeddings are and I'm assuming a lot of people in this room are already quite familiar right but an embedding is basically a type of data compression that takes some messy data like text, images, or audio and it puts it into a compact format that's machine learning friendly right and so these are going to be most often numeric vectors or arrays with hundreds or thousands of elements and they preserve information such that similar items have proportionately similar embedding vectors and this similarity is going to be measured mathematically using something like cosine or Euclidean or etc and another thing that I want to mention is we will and maybe I'll save it a little bit for the end but just like you can embed audio text and videos you can also embed graph structures and so when you have co-purchase relationships in a graph or you're trying to do entity linking in a graph or anything like that that can become very valuable for then using downstream in vector search yes yep we will walk through that so similarity for something like a text embedding is referring to semantic similarity so the meaning behind a text sequence for a graph embedding right similarity can mean similar in position or structure inside of a graph and that can have semantic meaning too and right now what we'll be using vector search for is just on text embeddings and later when we get to recommendations is when we're going to use graph embeddings and specifically node embeddings that we're going to use there inside of vector search so talking a little bit about just what's inside of neo4j so neo4j has multiple indices and you can use these in combination with each other so we have things like range indices which is just a general kind of index that can do range comparisons and equality comparisons on numbers and dates and things like that point indices which are geospatial or I guess point space indices right text indices and full text indices so we have a full text index that's backed by Lucene for example that is tokenization analyzers and today what we're going to be focusing on primarily are vector indices and our vector index uses HNSW hierarchical navigable small world yes there we go for approximate nearest neighbor search And so a lot of this might be review for the folks in this room right but the idea behind this is that if we have some product some text about a product and let's say a search prompt on the right here for a halter neck top we can start to identify things that might be more similar to that using that embedding vector right and sort of the semantic similarity inside of that space And right and right and right and that semantic similarity can be measured by either in Euclidean distance right or cosine distance and like I said before we're using HNSW to perform approximate nearest neighbor search or we will be here in just a second So we're going to start you know let me just go ahead and jump into the notebooks I think it'll be easier to do live so Male Speaker 2: I have a question before we're going to ask for those who are not familiar with Neo4j, how to visualize a node for me.

We have created something that's not clear. Yes, so right when we're done with the vector indexing I will go into browser which is one of our tools for visualization and we can take a look at that but basically if you go into this page here I can go to open.

Male Speaker 2: Open and then it will take me basically into the browser tool. This is going to take a while because of the internet lag and we can visualize in there. If you're willing to bear with me in creating a vector index first because I want to just go over that very quickly and then we can go into the visualization portion.

Male Speaker 2: But yes, we will very soon be visualizing the data. What I wanted to do first though was just kind of get it, get our index set up and just show you how to do some searching. Male Speaker 2: So basically what we're going to do with our data.

We're specifically going to match the product nodes. So if you remember we have that product node which is a variant of an article and the way that this is going to work. Male Speaker 2: You can call embedding models externally too but what we're going to do here is basically match the product where it has a detailed description.

Male Speaker 2: So basically some of these products don't have complete information so we're just going to take the ones that have complete information and then all this thing is doing is it's using this code gen.ai vector encode batch to on the server side create embeddings and then set that embedding property on a property called text embedding.

Male Speaker 2: We're using OpenAI for this. There's a few other models that we can use here or a few other API providers rather. By default this is using text embedding ADA or ADA but you could also use Vertex.ai here as well as Azure OpenAI or AWS Bedrock. Male Speaker 2: And then you can also just generate your own embeddings externally and ingest them for any other type of model.

Male Speaker 2: So if you go ahead and give this a run it will start creating your embeddings and then the cell after that is going to create your vector index. Male Speaker 2: So this isn't creating an index all this is doing is basically populating in batch a text embedding property and then after you create that this cell down here if you run that is going to create your vector index.

Male Speaker 2: We're going to set it ahead of time to use cosine as a similarity function. Male Speaker 2: And then you're just this next call is just to wait for it to come online since once you set it it just takes a little bit of while for it to index all the data and become available for querying.

Male Speaker 2: Yes. Male Speaker 2: Can you just clarify the difference between generating the embedding, which is the semantic of the text, right? Versus creating the vector index and what's that? Male Speaker 2: Right, so when you're doing the embedding generation, all you're doing is you're taking this text field that I talked about earlier, you're sending it to OpenAI to say, hey embed this, you take that back and then you set that as a property.

Male Speaker 2: So all this cell is doing is basically making these vector array properties inside of the node and then the vector index actually sets the index to make that searchable. Male Speaker 2: Once you've gotten that done, we can do a couple examples of search. Male Speaker 2: And again, this is sort of just using Cypher here and you can do this externally as well.

Male Speaker 2: But in this cell, I'm taking a search prompt that is very simple, just denim jeans, and then I'm passing it to this function, which is running Cypher. Male Speaker 2: All this is doing actually is just encoding, so basically it's just creating the query vector, right?

So if I were to just encode denim jeans, this is the query vector that that creates. And then if you take that query and you go to the next cell here, what this will do is it will do that same encoding. Male Speaker 2: So this is the first part of the query that we just ran, which is creates a query vector.

And then inside of Cypher, I say, call database index vector query nodes. And I say, hit this index that we just created. Give me the top 10 with that query vector and yield the node as product and the score. And then give me some fields like the product text and the product code, et cetera.

And so when you run that, it should hopefully give you some results that look something like this. Male Speaker 2: And this is returning different products. So you see all of these are denim jeans or denim products, or they should at least be jeans and/or denim, right? I think all jeans are technically denim, maybe not.

But you see here we have the product descriptions and the product text that it brought back. So you can see all the descriptions and that's basic vector search inside of Neo4j. Male Speaker 2: Um, yeah. Male Speaker 2: Sure. Male Speaker 2: I have a question. Male Speaker 2: Yes.

Male Speaker 2: Do you guys, do you build the, Male Speaker 2: Do we build it on every node? Male Speaker 2: Well, we build the index and then every node has a property. So it's built on top of all those properties, right? Male Speaker 2: Maybe I don't understand.

Male Speaker 2: Maybe I don't understand. It's one index. Male Speaker 2: Well, HNSW is a total graph, right? Male Speaker 2: That's right. Male Speaker 2: Yes. Male Speaker 2: I don't understand. Male Speaker 2: Yeah, this, so our graph representation is not the HNSW graph representation. Male Speaker 2: Right.

Male Speaker 2: I'm just trying to understand the scale. Male Speaker 2: If you put an HNSW, you know we're talking graph graph. Male Speaker 2: HNSW don't graph for every node in your graph, or is it one HNSW for the entire? Male Speaker 2: Wouldn't it be one HNSW for every index?

Male Speaker 2: Yeah. Male Speaker 2: Yeah. Male Speaker 2: Yeah. Male Speaker 2: I'm not familiar, but are you talking about a leucine index? Male Speaker 2: Leucine. Male Speaker 2: We are using a leucine index on the back end. Male Speaker 2: . Male Speaker 2: Yeah.

Male Speaker 2: Yeah, this is a leucine implementation. Male Speaker 2: Yeah. Male Speaker 2: . Male Speaker 2: Other question? Male Speaker 2: Yes. Male Speaker 2: If I already have embeddings, would it be easier to check the block group? Male Speaker 2: Yeah. If you do, I have… Male Speaker 2: If you met the size of index.

Male Speaker 2: Yes. Male Speaker 2: Well, yes. If you already have embeddings, so I commented this out. I haven't run this block for a while, but basically, this is just showing, you know, how you can create, like, you know, use open AI embeddings here. I'm using LaneChain in particular, but this is a workflow where you have embeddings external, right, and then you can import them into Neo4j.

Male Speaker 2: And thank you. Same question for vector database. Male Speaker 2: So there's multiple different ways to do it. If you have your vector index inside of Neo4j, that can make certain things more efficient. But there's certain situations for scale where using something like LaneChain or another orchestration framework, for example, you can query your vector data inside of your chosen vector database, and then do your graph traversal stuff in Neo4j, right?

Yes. Male Speaker 2: Why do I need a graph database? Male Speaker 2: We're going to get to that in a second. Male Speaker 2: It's why do you need a graph database if you're already using vector search. And so what we're going to explore today is there's a couple different things.

One of them is using structured data or data that's been structured inside the graph to better sort of personalize or make the responses more relevant for a large language model for search. And then we're also going to do some things with graph embeddings, which is a different type of embedding that we can create inside of this database.

Male Speaker 2: Yes. We do that here. There's an example of that. We'll do KNN and we'll show you how to draw relationships for that. Yes. Male Speaker 2: So it's a good question and we should definitely have a longer conversation afterward, but the advantage -- there's a few different advantages.

Male Speaker 2: We have index free adjacency and we have very performant like multi-hop patterns. So you'll see some of the patterns in here are very complicated and it's easier to do inside of a graph. First for just writing the code to do it, but then also the performance on it will be better.

And so there's -- and there's a very flexible schema. So it's very robust in terms of being able to change and pivot things around very quickly. Male Speaker 2: Yep. Male Speaker 2: I have one more. Male Speaker 2: Not yet. Not yet. All righty. So I'm going to -- I'm going to move on a little bit here.

This next part of the vector search portion, I'm going to just integrate LangChain into the mix. Male Speaker 2: So basically the reason I'm going to start using LangChain here is because when we get to the LLM portion, it'll just be a natural fit where we'll create a chain for this.

But all we're doing here is we're going to create a vector store using Neo4j vector. So this is part of the LangChain vector stores. Again, instantiate an open AI embedding model. We're just going to use the default with 1536 as the default size. And then you see we can create a LangChain vector store from index with that embedding model and our same credentials and the index name that we created earlier.

And when we do that, basically what this will allow us to do is start running from our search prompt, which is denim jeans. We're going to be able to run the results through this vector store here. So this is what we're going to be using throughout the rest of the notebook.

Instead of just using our cipher, our query language directly, we're going to be using LangChain, which gets this done under the hood for us. So that's that. And you can see it brings back the different documents with the product descriptions and the representative URL and all the metadata. And if you want it to be in a pretty data frame so it's easier to see, you can run that to kind of get the, to get the view we had before.

And this should match what we were doing earlier. And of course you can try yourself with your own prompt. So if you were to type in something else here, you can see it'll bring back, right, a result that's relevant to that. So I typed in oversized sweaters and it will bring back different oversized sweaters.

All righty. So how many people were able to get through the vector search section? If you just raise your hands. Okay. Good. Very fast class. Okay. Anyone having trouble, struggling, need help? One person over there. Can I ask one more question? Yep. So, like, if you're using one Lucy, how do you determine how many Lucy indices do you need?

Do you pre-compute that on your side? Because the user has to get it out? Because PHSSW can only take out the property if it is. So right now you, I might not fully understand your question. When you create, you basically instantiate an index for one or a set of properties.

That can be on a node or it can be on a relationship, right? So if I make one index to cover, you know, the property of text, that's one index. I could create another index to cover, like, a product description or another name or something, right, if I wanted to.

But that's, there's nothing that would, you know, so it would be like it's one index at a time, right? Does that, is that getting towards you or is that totally off? Let's say we have, like, billions of documents and we're going to . So you said you're using Lucy, right?

Right. So Lucy is . So each Lucy index takes 50 gigs of data. What do you do on your side? I'm just curious. What do you do on your side so it doesn't puke? Because after 50 gigs, there's diminishing returns. Right. Are you, like, automatically auto-sharding it? Like, what are you doing to make sure -- There are, there are ways to federate the database.

Like, you could, you could create different shards based off of metadata values. But we're, that's stuff that we're, like, currently building on and experimenting with. And for very, very large, like, if you, if you truly had, like, tens of billions of documents, you can also integrate with your vector store of choice, right?

And then just basically you substitute out the vector retrieval call with, for that vector database and then you can use Neo4j for doing a lot of the graph traversals. But it's one of those things where we're continually improving. So, we're coming out with new features to handle that sort of stuff.

All right. So, let's start with semantic search with graph patterns. So, this is going to be, basically, building on top of vector search where we start to work with structured data. So, when we think about Neo4j and semantic search, there's sort of multiple layers. So, the first part is vector similarity search, which we just went over, which is just finding relevant documents.

After that, there's graph pattern matching, which is basically saying, hey, can I find entities associated to these documents? And the sort of patterns in that connective data. And then we have the more data science and machine learning part on top of it, where we can improve search relevance and do other things by running graph algorithms like embeddings, right?

So, when we think about semantic search patterns with graph, there's connections that we have based on purchase behavior. So, basically, given that you have one person who purchases relatively the same things as another person in this graph, you can start to make inferences about what they might want to purchase next, right?

So, this is sort of like if you've ever dealt with collaborative filtering or sort of those types of patterns. This is a very common recommendation pattern that we can do with structured data. And when we go back to the graph, we'll start exploring the data set now in a little bit more detail with visualizations.

So, what I want you to do, and I don't have these calls saved, but I'll go ahead and copy them over. Basically, you're going to go back to your browsing window, and you're going to open your blank instance. So, I'm going to click open, and hopefully, that'll work. And if not, I might actually start hot spotting myself.

Yeah. It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page.

It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page.

It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page.

It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page. It took me like a minute to refresh the page. Yeah, I'm hot spotting.

All right. Thanks. I'm on the, I think I'm on the Wi-Fi. All righty. So basically what I did is I just entered into something called browser from our sandbox. So again, you just go here, or I guess I'm back to this window now. You go here and you press open and then you can either do the SSO sign on or you can use your username and password.

And so once I'm here, I'm going to copy. If you go back to the notebook, you can copy these calls. This is just Cypher that you can put in to help visualize the data that we've ingested. So this will give me my schema, right? You won't have this relationship yet, but what you should have is these four nodes, which you can get rid of this window.

We'll give you an idea of your data model. That should reflect what we have in that picture inside of the notebook. We have our articles, our products, our departments, and our customers. Excuse me. And then if we go back to our notebook, I'm going to pull a sample of that data just so you can see the graph.

So this Cypher statement here is just saying, hey, get me all the products, match all of the articles that are variants of those products, and then all the customers that purchase those articles. And I'm just going to return all of that and I'm going to limit it to 150 results.

And so if I do that, what's going to happen? And again, the Wi-Fi is kind of hurting me right now. I'll get this pretty graph. So this is a sample of the data that we loaded. So customers are in orange, articles are in blue, and products are in red.

So what you'll start to see, right, is we have these customers. Some of them make multiple purchases. So here's a customer making multiple purchases. Make that a little bit bigger so you can see when I go to purchase, it should have a transaction date, a price, all that sort of stuff.

Here's my customer with the customer ID and some metadata about the customer. And then here's my article. And my article is a variant of a product. So what we've done so far is if you go down here, we've created, right, this text field for every product. That's what the text description looks like.

And then we've created this text embedding. This is our OpenAI embedding that we have here. And if we zoom out, the graph structure that I was talking about before in slides, we start to see that. So you start to see that you have these customers that are co-purchasing things, right, at different times.

And so we can start leveraging that graph structure. And we can use that graph structure to start informing our search and our LLM how to return better responses. So for example, what I'm going to do is I'm going to feed it a specific customer ID. So all this does is it basically caches this parameter for this specific customer ID.

So I can use that in later query calls. So I'm going to go ahead and put that in. And then I am going to take this query here. And basically what this query is just going to do is tell me the history of this specific customer, their purchase history.

So I'm going to go back here. And I'm going to say, hey, get me the customer with that ID. And then get me all the things that they purchased. And then all of the product categories that those were in basically. And just return some metadata about that. So I'm going to go ahead and give that a run.

And it will tell me this person purchased the specific product names right here. The product type that they belong to, et cetera. And the description of those products. And then the cool part about graph is I can basically start taking. I can pull the latest purchase history. And then I can start making recommendations off of that latest purchase history.

So there's lots of more complicated and fancy ways to do this. But the basic idea here, right, is that if I go with this, I can pull the latest purchases. So similar to what we did before. Just say, hey, take this customer. Get everything they purchased. Get sort of the max date as the latest purchases.

And then I'm going to do this long query here. And basically what this long match statement is saying is say, hey, take this customer. Take their latest purchases, right? Look at other customers that purchased the same thing. And then look at what they purchased, right? That's what it's saying.

So it's like, right, if ABK, if I buy the same things as ABK does, I can say, right, which, okay, thumbs up, right? Okay, and then I can basically say, well, for me, look at what ABK has purchased and a few other people that have similar purchasing behaviors. And is there anything there that I haven't bought yet?

That's basically what this query is doing. So when you run this query, what you'll see is you'll start to get these different sweaters and everything. And this will change for every customer. And so the idea here is that this structured data that can feed us this, you see here, you have these common purchase scores.

This is basically scoring the amount of times that you had common purchases between customers and then like sort of the next thing that they buy. So for example, eight other, there's been eight other common purchases between people that have bought this and between something that this particular customer has bought.

And so these top, go ahead. You can imagine this as your peer group, right? So the people that are like you, that buy the same stuff or interested in the same stuff. So that's your peer group. You just look what else has your peer group bought that you haven't bought yet, right?

So that's kind of what is behind that. Yes, exactly. And these are the top ring things in the peer group. So this specific sweater, this T-shirt. Say slowly again what the common purchase score means in plain language. Yep. And let me look at it again just to make sure I have it right here.

So we're basically counting the number of paths between the number of co-purchase paths. So basically this customer, when we go out and look at what other people have bought in that peer group basically, how many times has that item been bought in that peer group? Does that make sense?

Okay, so it's the score attached to an item. Yes. It says high score means lots of people have bought it. Yes, exactly. Thank you. Yep. Is that clear for everyone? How does it, sorry, how does it know it's a peer group? Peer group being that it's sort of the local graph structure.

So what I'm doing right is I'm basically, if I look at this graph, I'm basically going like I'm starting at a specific customer. Then I'm looking at the article. And then I'm looking at everyone that bought this. And then I'm going out one more hop, right, to see everything they bought.

And then I'm going to count up how many times those purchases appear. And the more purchases there are, the more likely it is, right, that that individual customer is likely to like that product just based on the common purchase behaviors in the graph. Does that make sense? So in this case then, the customer that you were just pointing out, the, well, yeah, before you scrolled away.

Oh, sorry. I'm just, I want to make sure I understand the graph. So you picked that one random customer and you said you go one hop away from, so yeah, so assuming that person. Then that blue product, I guess, so that's the one they bought. And then the one to your left, I suppose, that would, or I guess right, whatever, the next, the closest blue dot, that's another product.

Yep. Because there are links between, oh my goodness. Yes. There's lots of products, right? So it's basically going bought this, this person bought that, and then for all of, see, and there's a lot of interconnected relationships too. And then it, and then yes, for all of that, and then go out one more purchase relationship.

So you're basically going out and then you're saying, okay, how many times do these other blue nodes show up? Right. Okay. And the more that they show up inside of that graph, the more shared purchasing behavior you have. Yeah. Yeah. So it's basically just counting that. You could write the query to go out further if you wanted to.

You could. You could build it. Yeah. And, and part of the machine learning piece does that automatically. So, but yes. Can I ask a question about time window? So, for example, here you are considering like, you know, the person, no matter if it's one month or one year, that's one transaction.

Versus if I'm considering window, right? One month versus one year versus one way. Is there a way like, you know, to make this? Yeah. So you notice, let me get out of this view. If I go up to the query that I just made here and maybe I went through this a little bit too fast.

But you should see in here I'm saying, I'm just going on max date. But you could filter the date to say like TDAT, which is the purchase date variable. You could say, do that for only the last week or only the last two weeks or whatever filter was relevant for that.

And so that's basically the idea that we're going to use to provide additional, we're actually going to do a graph post filtering pattern with vector search. So what we're going to do is if you, if you come back to the notebook and you go down and run. So this is, here's the query, right?

So basically what we're doing is we're going to instantiate another vector store. Except here what we're going to do is we're going to say, we're going to put a retrieval query at the end. So what this is going to do is it's going to do vector search like before.

But then after it does the vector search, it's going to try to match for that customer. It's going to create that sort of peer group, that local graph. And then it's going to score, well, how many times does that product show up? So this matches maybe almost backwards, making it probably a little bit hard to read, right?

But it's the same query that we have before where we take the customer ID and then we go sort of one or I guess two hops. But really we're saying, okay, get me that peer group where we go to the article. We look at all the purchases, the common purchases between customers and then we compute that same score.

And there's some multiplication here that I do basically to combine the vector search score with the, what we're calling the purchase or the co-purchase score here. But basically what this does is it now allows me to pass a customer ID into the search result. And then when I do similarity search with that, it will start sorting things by basically this purchase score.

So now instead of getting, because denim jeans is like this really generic thing, right? So different customers, and we'll see when we run the app later, if you plug in different customer IDs, this result is going to be different. It's personalized now to this customer. So basically with vector search, we're now using this post-filtering pattern in the graph.

And we can also do a pre-filtering pattern. but we're using a filtering pattern, right, to now rescore these things. So now, okay, we're just returning, for example, these different denim items, which are much more relevant for this specific customer. Yes. Can you explain the difference between search score and purchase score?

Yes. Let me see what I did here. So search score is basically just the vector score that came back from the cosine similarity. So when you're doing similarity between two vectors, right, for the text description, that's going to be your search score. Purchase score is going to be what we were showing before, which is showing the popularity inside of the peer group.

And then I'm just multiplying them together to create a combined score here, but there's different ways that you can do that. So are you going to use this product as a way of ranking those results? Yes. So if you look down here, these results are now ranked differently, and it's actually different from the query we ran before.

So we ran the same search prompt with denim jeans. But here you can see we get like Rachel HW denim. We get Jade HW damn skinny and all this sort of stuff. If you look at what we ran before, if I were to scroll up to the top here, this one, you'll see that these are different.

These are actually different. These are different. It's a different ordering. There's some similarities here. Like this is number three where it was number one below. But the idea is that you're sort of re-ranking this right now that you're personalizing it to this specific customer. And this gets really powerful as you get more and more structured co-purchasing data, right?

Yes. Why are there a bunch of purchase scores zeroes? Because there's going to be, if you, it's an optional match. So if you notice, the sort of literal reason is that this is an optional match. But the other reason too is that there could be a situation where they're searching for something that isn't bought in their peer group yet.

So this is just organized so that even if you didn't catch anything in the graph structure, you would still have a way to return results based on the vector scoring. Yeah. Yes. And it will, and it will try to see if there are, if anyone else has bought anything of that nature.

Right? And if they have, then it will, it will help reorder the ranking. Yeah. And you could do it the other way around too where basically, and we have another app and I might take it to our booth where you could do the graph traversal first. Right? So like if you, if you had a scale problem that was, you know, very large.

So one of the, one of the ways we can scale is through basically, hey, I would do the graph search first. And then after that, I can do the, the vector search. So, whatever you are getting in this, how is that different from like, like, the recommendations for products that you may have to say?

You have a separate system competing with batch recommendations for you. Oh, people can't hear. So the question, the question was, why can't you use like an external batch recommendation? So you potentially could. What you're getting here is basically as this data is updated in real time in a transactional store.

Right? You're going to be able to re-query it. So you're getting everything kind of in real time. Right? So if you're doing rag and you want answers to be, you know, adjusted for what people had just purchased. Right? This would be a way to accomplish that inside of a graph database.

And then there's a lot of flexibility. You can change the way this query is, you know, you can shorten the time period. You could create a different type of traversal to go more than one hop out, et cetera. And I don't know. There's just so many of you who have a question.

Oh, okay. Go ahead. Okay. Sorry. I'm hiding in the corner over here. I was wondering if you could speak more about combining the scores. It seems like just multiplying them together is a good way to get started. But is there a better theory there? Or it seems like something that have pitfalls, you know, to just multiply the two scores together.

You don't know. Are they normalized? What's the distribution between them? Like how are you going to achieve the results? Yeah. That's true. Yeah. So again, this is just an example to get you started. I suppose like you could maintain a rank ordering where you say, hey, rank by like the search score first and then rank by the vector score, which would be kind of independent of scale.

But yeah, there's a few different ways you can do it. This was just a quick way for me to put it together and I could probably do it better. But yes. And you know, with these systems, it's always good to have labeled data and sort of figure out how you can fine tune things.

So like, you know, in a production system, you would have recommendations and whether or not those recommendations were successful. And then you can try different methodologies and see which ones get the best historic score too. For normalization. Just one second. One second. One second. One second. One second. You gotta .

Thank you. Yes. Please. Maybe a follow up. Right? If I thought of... If I had this problem in front the first time, I would probably gone filtering. So filtering on the graph relations and then filtering on a fixed threshold on the semantic matching. Can you speak to how you would think about doing it my way versus computing scores like this and multiplying them?

Yeah. So you're saying filtering. So basically you would do the graph pattern first. Excuse me. Yeah. I would restrict the graph to only the subgraph that matches these patterns. Yes. And then they're filtered to only the articles that have a similarity score of all threshold. Yep. Returned out. Yep.

You could do it that way. And in fact, we have a demo of exactly, I think, something very close to what you're describing. The issue is for this course, like if there's a chance that whatever you put inside of the graph, inside of that local graph could have a cold start problem, right, where it might not very well match the vector search.

So it is a way of doing it. It just might end up that you could be over restricting, right, to a local area of the graph that might not have, in this case, good denim jeans, for example. But you could do it that way. You could also do a filtering where, like you said, for example, give me...

I guess you could do vector search filtering first and then filter down to the graph. And then if there was nothing in the graph, default to the vector search, right, results in the scores there. So you could do it that way too. Any intuition on to, you know, what you expect to work best?

These filtering steps or multiplying into one score? I think multiplying into one score is a good way to... It's just a way for me to get started. And some of it will depend on the scale of your graph, right? So if you... The more sort of purchase transactions you have, the more well-connected the graph is going to be.

And the better you're going to be able to, you know, sort of match things almost like with a pre-filtering pattern, where you just restrict to that area of the graph first. When it's a little bit more sparse, then you're going to probably not have that same reliability. And you're going to need to either do an optional match or some post-filtering pattern where you make sure not to over-filter things.

Yep. Just two real quick. I wasn't going to ask you about scoring, but since there's been a couple. Normalization, a typical way to do it is reciprocal ring fusion. Is that available here? Not that specific one, no. We have some normalization. We have... It's the graph data science library.

There are some normalization functions, but they're not going to normalize things at query time in the way you're thinking. Okay, yeah. Yeah. Okay, and then the other one is just, and if I'm getting ahead of myself, just tell me and I'll be quiet. But everything we've done so far, it seems like creating the peer group, you can do that with a graph database, right?

Infer relationships. But what part of this is actually the knowledge graph part of it, which is infer unknown relationships based on unsupervised understandings? Have we gotten to that yet? Yeah, so we'll get to some of that in a little bit. Yes. So I will, we'll get through, we'll get through to that.

By the way, there are also some empty chairs here if someone wants to sit down. Thanks. Any other questions? Otherwise, you probably need to progress. By the way, we also have a really big booth with really comfy chairs over there. So you can always come the next two days and ask Zach and Andreas and Tomas and me many more questions.

Right, so. And we have swag at the store's booth as well, so. Awesome. Thank you, Michael. Alrighty. So now we're going to start augmenting semantic search for knowledge graph inference. So this will get a little bit into what you were talking about. There's a lot more, so I didn't like, we have a whole knowledge graph builder where we build relationships from unstructured data that we're not doing in this course.

But we do have it at our booth. So basically, this is where we're going to start using the graph data science portion of Neo4j. So, we have this thing called graph data science inside of our product. And basically, what it does is it offers various graph algorithms to you and this special workspace and data representation.

So you can effectively create a projection of part of your graph in sort of this sparse matrix representation. And that allows you to run a lot of algorithms very efficiently. So things like path finding, centrality, community detection, different machine learning algorithms, and of course embeddings and K&N similarity, which is what we're going to talk about here.

With a lot of focus on this graph embedding piece. This is a customer who created a very nice sort of periodic table of all of our algorithms for reference. But there's a lot of different areas, right? So there's a lot of things that we aren't going to get to talk about here.

So things like community detection, path finding, centrality, topological link prediction, and directed acyclical graph algorithms. But we are going to talk about similarity in embeddings. So basically, a node embedding works just like any other embedding except it's for a graph where the things you're embedding are nodes in a graph.

So this is a picture of a very small graph. I think it's Zach's Karate Club is the name of the graph. And it's basically the relationships between different people in a karate club. But the idea is that I can take this graph and I can turn it into a set of vectors.

And in this case, it's a 2D vector, which is a 2D representation. We're going to create longer vectors for the notebook. But the idea here, right, is that the yellow nodes that are similar in the graph in the sense that they're really close in the graph are going to be really close in the embedding space and the orange nodes and the green nodes and the blue nodes, etc.

So that's the idea of what we're doing. We're basically embedding what we call homophily, which is basically this fancy term for like the locality inside of the graph structure. And once we do that embedding, what we're going to be able to do is start inferring relationships. So someone mentioned being able to do sort of inference beforehand so calculations are available at query time.

And we're going to use k-nearest-neighbor for this. So basically this is happening outside of an index. What we're going to be doing is creating graph embeddings, specifically on the articles. And then we're going to be inferring these k-nearest neighbors between them based on k and n, which is this semi-supervised or unsupervised algorithm.

And you can do this for a lot of different things. We are going to use co-purchase relationships for this and kind of scale that process for multiple hops. But you could do this for other forms of link prediction. For example, things like entity resolution or if you're trying to resolve, you know, maybe you have people and locations in the same graph and you're trying to resolve whether or not a person is in a specific location.

You can use these sorts of models to start making those knowledge graph analytics and drawing those relationships. So let's dive into the notebook to see how that works. Question? Yes. You mentioned you're embedding the nodes. Are we essentially throwing away the edges? We're not embedding the edges? So we're doing a node embedding which is specifically focused on nodes.

We could do different types of knowledge graph embeddings which embed like the edge and like, right, it embeds the, what is it, the head and the edge to the tail. Right? So there are those types of embeddings and there's different types of subgraph embeddings. For today, we are, the relationships are used to understand where nodes fit inside of that vector space.

Right? In terms of clustering of, in this case, what's going to be co-purchase behavior. So we're not throwing them away. They're going to be used. But we, and we'll use the, yeah, you'll see as we go along. We'll continue to use the relationships. Sounds good. So basically, what we're going to do here is we're going to create a customers also like relationship.

So basically, this cell, so it's going to be the first cell after you get to augmented semantic search with knowledge graph and ML. And if you run that cell, it's going to run our GDS algorithms. So I have some utilities to kind of help clear, clean things up. And then basically, we're going to create something called a graph projection.

And we don't have a whole ton of time to go into detail on these in a two-hour course. But the basic idea, right, is that I'm going to look for these common co-purchase relationships between articles and customers. And then I'm going to project that as a co-purchase relationship to an in-memory graph.

So I'm creating basically this graph that contains articles and then a weighted co-purchase relationship between them. After that, I'm going to run our node embedding model. So we have supervised models that we can use. But in this case, I'm going to use something called fast RP, which stands for fast random projection.

And basically, what this is is a graph embedding that is going to use a lot of sort of this matrix multiplication with sort of this sparse random matrix. So the math behind it basically allows it to calculate an embedding really quickly without necessarily needing a neural network. So I'm going to make these embeddings 128 dimensions.

I'm going to specify certain things like an iteration weight that will tell me how many hops to go out when looking for other similar nodes. And then once I create that embedding, I'm going to write GDS KNN write. So basically what this is going to do is it's going to find nodes with similar embeddings.

And it's going to write a customer also likes relationship between those nodes. And then there's some sampling procedures that I can use inside of KNN to make it run faster, which I'm doing here. And then once I have that, I can write both my embeddings back to the graph.

And in KNN, I just wrote the relationships back. So now we have these customer also like relationships between articles inside of the graph. And I get some stats when that's run telling me about whether or not it converged. The number of new relationships written, which is going to be generally a lot because it's doing this quadratic comparison.

And each relationship is scored. I had a similarity cutoff at I think 75 and this was using cosine similarity. So it would be negative one to one, I think, that it was using. And we can visualize node embeddings as well. And that's interesting to do in the sense that it will tell me clusters of purchase behavior.

So these cells that I have here, I've commented out because they take a while to run. But basically, this should still be able to be rendered here. This visual. And basically, what this visual will show you is you see we get these clusters. And basically, what I did is I took those 128 dimensional embeddings.

And I used a TISNI, which is just a way to condense that down to a smaller space. And I condensed it down to a 2D space. And basically, what you'll see is you'll get these clusters of commonly purchased things. And so, this is sort of doing the multi-hop thing that you were referring to earlier, where now it's starting to look at, okay, well, these different clusters, right?

These clusters represent items that are often co-purchased together. Baskets, right? Baskets, yeah. And yes, baskets in a sort of sense. Right now, it's just co-purchasing. But you could organize into orders inside of the graph. And you could do co-purchasing baskets as well. And so, with that information, we can start to sort of scale how we power just plain recommendation.

So, I can create a new query here, where basically, instead of going to the vector database, I can just say, okay, we'll match the customers, get me everything they purchase, the articles, get me what else customers also like to buy, and get me the product behind that. So, it's similar to what we did before, except it's scaling it to sort of a, to a multi-hop pattern.

And you can substitute other machine learning models in here if you wanted to, too. If you had, for example, a supervised model that you wanted to use for recommendation, that could be substituted in here to make these inferences, and then you all of a sudden have these customer also purchase relationships.

You could also create a vector index for that graph embedding, which we're not doing here. We've sort of done the KNN beforehand, right, and we've written the relationships to the graph. And that allows me to do this customer also liked relationship that I can draw in the query. And that gives me a set of recommendations.

Again, not necessarily based off of a search pattern, but just based off of what the customer had purchased before. And to your point earlier, we could create time, date, and date cutoffs, and things like that. So, this is basically just a way to kind of scale things with the machine learning model and leverage graph embeddings.

So, that was it for the graph embedding and sort of the graph machine learning section. And again, you can use a lot of different types of machine learning models here to basically infer these relationships. some of which exists, like, inside of our platform through graph data science and others that you can bring in externally.

We're at 4:48, so we have, what, 45 minutes left, a little under. And we haven't taken a break yet. I'm wondering if we should power through. Yeah. How do people keep going? Yeah, keep power through. Okay. Yes. Do you always have to recalculate it? Is there, like, any communication going with customers coming in?

So, right now, you do, yes, have to recalculate the graph embedding. Ideally, right, if you, like, over time, if customers have been in there for a while, that graph embedding, like, well, it'll slow down how much it changes. We are working on things for the future that might make that easier in terms of getting things in real time.

But right now, for calculating graph embeddings, it is something where you have to recalculate it for the graph. Thankfully, though, with something like FastRP, that'll work. I mean, we've, I've seen it work on graphs with billions of nodes and relationships, and it can calculate in, like, half an hour.

But yes, right now, you do have to recalculate it. You could use other methods, too, like, averaging the neighborhood as well, for sort of, like, intermediate embeddings as well. So, if you didn't want to recalculate it right away, you could take, like, the average of the neighborhood and use that as, like, a stand-in embedding.

What's that? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Yes. Is there an apple limit? Is there an apple limit?

Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit?

Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit? Is there an apple limit?

I think you have an apple limit? Is there an apple limit? Is there an apple limit? Yeah. There's no upper limit to dimension. Right now, inside of this graph, we only have, let me see how many customers we have. So this graph only has a thousand customers, right? There's been other examples and demos that we have where it's like millions or tens of millions of customers, right?

So the more transactions you have, the more customers and articles you have, the higher that dimension needs to be, right? For demo purposes right now, and we're working off of sandbox instances, I kept it pretty small. But you can expand this to 256 or keep going up in orders of two.

I found that these generally don't need to be as big as like the text embeddings that you get from OpenAI because it's a smaller space, right? When you, even if you have billions of nodes in your graph, it's still a smaller space than the entire human language, right? So you, so they tend to be on the, tend to be smaller, but they can go up and there's no upper limit that we, that we inherently have inside of the product, no.

And what is the biggest use case you have at the moment deployed? For embeddings? Graph embeddings, yes. I'd say entity resolution, different forms of classification, like for fraud detection, and then also for recommendation and things like customer segmentation where you want to like create clusters based on, you know, behavior or other data inside of the graph.

Is it possible to semantically retrieve the information about the graph structure itself? So let's say that you have 10 different types of pants, but you want to ask a question, how many types of pants do you have, which is like it's not in the data itself specifically, it's kind of metadata about it.

Right. Yes, so very good question. So there's different types of retrieval patterns that are better at those sorts of queries. So we have something called text to cipher. We have it, it's sort of, it's a pattern, right, where you can actually get the LLM to generate like the cipher query to query the data and those tend to work well for aggregation.

So like if you were building out a more mature system, you can have like a semantic layer with different tools. And so if you knew that you had more of like an aggregation question about types of things, then you can send it to that tool, which can do like a text to cipher or another sort of a cipher snippet that was like pre-generated or template that was pre-generated.

Does that make sense? Yeah. Have you tried using that with like function calling? So you don't have to like specific, know in advance that you'd like to use it, but rather like... Yes, that is something that Michael, if he's still here, has done some work on for function calling.

Yeah. So... Yes, in general, you can make many of these graph operations available as functions to an LLM. And then basically depends on the selectivity of how you describe the function and what you pass in to make this available. So you can do this kind of embedding work, but also other graph queries can be made available as functions to an LLM.

And then it just selects them. So for instance, get a neighborhood of a node or find a shortest path between two nodes or find similar nodes to this nodes using graph embeddings or whatever. And then you have these kind of some generic functions that you offer to the LLM.

And then you could have also specific functions that are basically built by a subject matter expert that, for instance, for legal domain or for a fintech domain, they have specific graph queries that are really specific to that domain. And then add them as functions as well. So you can have, you know, five, ten functions available and then the LLM fixed ones that are necessary to answer the questions and such.

But it should work with all the LLM models that support function callings and such. Yes. Thanks. If you have more questions, you can always come to talk to us in detail. Yes. So I was just trying to gain a little more intuition about the idea of graph embeddings. I'm pretty new to graphs in general.

Well, it seems to me like the whole motivation for using embeddings, it's really taken off in the last four years or so. And it's because you've got transformers, you've got unsupervised pre-training where you can leverage all of, you know, the text on the internet or all of the images on the internet and whatnot and train these embedding models that are just general purpose foundation models.

And this is very different from that because you're just completely training your embeddings or if it even is training from scratch. I'm wondering if there's like a deeper intuition to why you'd be using embeddings for this instead of, it almost seems like there's a lot of more like traditional machine learning techniques that you could be using.

I just don't know what the state of the art is. And I'm curious what the intuition is for using embeddings for this. Yeah, I mean so embeddings here basically is helping you kind of scale these other graph traversals, right, to go, because when we do that embedding, what we're basically doing is we're taking a graph traversal that's similar to the one we went over and going multiple hops, right.

So you're sort of taking this structure of the graph and you're scaling your ability to search for related things inside of the graph without having to do a cypher query every single time. So it's basically and it can help you also infer information that you didn't know before. So like in this example, right, we're making these relationships on customer also liked.

We could also do things like say, hey, like if we had like an identity graph of people and different identifiers and transactions, we could start resolving people in the same household, for example, using embeddings. So even though it is kind of local to your graph, the idea behind embeddings is that you're able to infer relationships between things inside of your graph that, you know, would be very hard to do without some sort of machine learning or just traditional methods, if that makes sense.

A little bit? Yeah. Yeah, because you're basically using, depending on the graph embedding you're using, you're basically taking the structure of the relationships and that topology and you're using it as features, right, to power what something should look like in a representative space. And then you can then use those to basically predict where links should exist inside of the graph.

Hopefully. I don't know if it really did. I mean, it's that you're, you're providing this sort of intermediate representation where you can do more general purpose things with it. Yes. Is that good paraphrasing? Yes, I think so. Okay. Understood something. Alrighty. Awesome. So this is going to be the final step for tonight.

And then we'll see, maybe we'll even be able to get out a little bit early and get to the reception. So we're now going to take what we have and we're going to start using a large language model. So we're basically going to use the retrievers that we were just working on and we're going to take them into this prompt and we're going to have a language model generate an email for us.

So this might be a little bit hard to see, but it's also in the notebook for when we go there. This is the prompt that we're going to use. We're going to instruct an LLM. We're going to give it a name. We're going to say it works for a certain company.

And we're going to tell it to write an engaging email to a customer to promote unsummarized products relevant to them, given one, the current season and time of year. And two, recent searches and interests, which is going to be our search prompt. And so all of sort of the red things are where we're going to inject things inside of the prompt.

And then the rag part comes in. So we're going to tell it, hey, please only use the below relevant products. So there's going to be what we're calling search products and recommendation products, which search products is going to be based on a vector search combined with a graph traversal.

So that will give us something that matches the customer interest. The recommendation products are going to leverage our graph embeddings to give us additional recommendations that we can add to that. So there's going to be sort of two lists that it gets. And there's some other instruction there to tell it to only pick a certain number from those things and kind of be creative with that and make sure it aligns to the current season and time of year.

And this is kind of what it looks like. I'll just go to the end of this thing. So if you were to think about the chain that we're going to put together, we're going to come in with basically four things. That search prompt, the customer ID, the customer name and the time of year.

The search prompt is going to go with the customer ID for the personalized search, which is going to be that sort of peer group pattern. The customer ID is going to go directly to get recommendations based off of graph embeddings. And then these four things here, the products that were recommended here and the customer name and the time of year are going to go into the prompt to then generate an answer.

So we're doing basically two retrievers here. One for recommendations and then another for basically search results. You can think of it that way. All right. So let's go and dive into the notebook. So we're going to use lane chain for this. We're going to instantiate the LLM that's going to be based off of GPT for Omni, which we have here.

And then we are going to recreate two retrievers. So the first retriever here is going to be for the personalized search. This is going to use the text embeddings and then it's going to do this sort of post filtering pattern with that combined score, where we do this optional match to look at the peer group to kind of filter those results.

We're going to create another function here based off of that. I'll zoom into this a little bit more. Where basically we are going to take the, this function is just so we can pass a customer ID parameter in. So this is the KG personalized search function, which is going to use this retrieval pattern.

And then we have a KG recommendations function, which is going to use that customer also liked relationship that's going to just basically go off of customer history. And then both of these will sort of join, it'll do some formatting for the documents and return them for injection into the prompt.

Here's the prompt itself. So this is just the prompt we look at in the slide before. It mentions, you know, customer name, time of year, and then it has the relevant products where you put in the search products and then other recommendations generated from the graph embedding. And then when you create the chain, so you can see here on this chain, we have four inputs, right?

So the search products, which again comes from the personalized search pattern, the recommendation products, which comes from the recommendation pattern. So there's these two retrievers that are supplying each of these respectively, the customer name and the time of year and the customer interests as well. So I guess all five of these things, right, get passed and injected into the that prompt, go to the LLM.

This is all a LangChain expression language. This is kind of a LangChain thing where we have this pipe operator that's just basically taking this dictionary of things and then passing it to be injected into the prompt, calls the LLM, in this case GPT-4O, and then just gets outputted. And so if we look at what it looks like when we invoke that chain, again, using our search prompt, which is DenimJeans, our customer ID and then I gave it a customer name and a time of year.

And the customer name is probably something that could be stored in a database too, which is that this data was anonymized, so we don't have that, we don't have that in the database itself. But anyway, what this will do, you see it'll give me its responses, it'll make a subject header for the email.

"Hi Alex, I hope this email finds you well and enjoy the start of the summer. As the days get warmer, it's a perfect time to refresh your wardrobe." And then it will give you the denim jeans. So these are the items that came from the search that was personalized to this user.

And you see we get a set of five jean products here. "Summer essentials to pair with your denim." And then this is where it uses the graph embeddings to provide those other recommendations down here. So it's basically, right, using a combination of this vector search to pull things back, personalizing it so that it becomes specific to just this user, and then adding additional recommendations to that as well.

And of course you can change the prompt to be just the recommendations portion if you wanted to or just the search portion or, you know, whatever, whatever was appropriate for the application at hand. And if we just looked at, so basically this cell, if I run it, all this next cell will do is show you the prompt, the full prompt that was set to the language model.

So if we look at this, this is basically showing you, if I were to scroll all the way to the top, it provides quite a few products. You can see here this is the data that it retrieved and formatted from the database for the relevant products, all the different garments that it selected.

And then if I were to go down more, it would eventually bring me down to the recommendation products, which I think it already did here. So if I go up further, yes, relevant products. And then it should give me about 10 of those. And then other recommendations from the graph embeddings that go along with that going down here.

And then it basically chose from those products to generate that given response. Yes. Yeah. If you look at the way those queries were written in the cells before, I think the first one for the search responses, I limited them to 10. And then for the recommendations, I'm actually passing 100.

So I'm passing quite a few. But yeah, eventually, if you provide enough of them, you will hit a context window issue. And then, so basically, once we have that, we can run a demo app. So in this case, we're using Gradio. This is just some examples, preset examples for that we can use in the application.

And this is running the application. So I'll go ahead and give it a run. And if you have it in your notebook, it should just give you a link that you can go to. And basically, let me make this bigger so it's seeable. So here's just a UI of what we had before, right?

So this is the example that we ran earlier. So if I say submit for this specific user, it should pull back hopefully in the next 20 seconds or so because the internet is slow. Normally, it's around 10 to 15 seconds. Yep. And this will pull back the email response.

And the interesting thing about this is if I took, you see this customer ID is like D-A-A-E. If I select a different customer ID, but the same prompt, so say it's this person. So I click that and then, yes, this change to this other customer. This answer should be different.

And if it isn't different, it's because they have a very similar peer group and a very similar piece in the graph. But if you see HW denim and skinny jeans, and then this, you'll see they get a bunch of different recommendations and search patterns. So the idea here, right, is even though these two individuals had the same interest based on their purchase behavior in the graph, we're able to give them very different recommendations because they exist in very different parts of the graph and have different interests based on their purchase history.

And you can play around with this. Like if I, you'll see it'll give me some summer things here. If I change this to like February, right, this is nothing to do with graph. This is purely an LLM thing at this point and I submit it. It should, the email should change to winter and then it should give you, instead of summer must-haves, it should give you something about winter and it should be able to select different products down here that should hopefully, if the LLM is doing its job correctly, go for more winter related items like sweaters that you see here.

So that's the, that's the full app that kind of leverages all the things that we just went over. Cool. Thank you so much, everyone. This was really great. Thank you so much. Thank you so much. Thank you so much.