Why Your Agent’s Brain Needs a Playbook: Practical Wins from Using Ontologies

00:00:00.000 | Jesus Barraza: Welcome. Thank you for joining my session this morning. I was thinking that

00:00:19.200 | I was in some kind of niche space where people would not be interested, but I'm curious to

00:00:23.680 | hear what brought you to this session. I'm Jesus Barraza. I'm the field CTO for AI with Neo4j.

00:00:31.600 | For the next 14ish minutes, I'm going to be diving a bit deeper into this really powerful combination,

00:00:41.040 | which is knowledge graphs and large language models to build AI applications. In particular,

00:00:47.600 | I'm going to be focusing on these interesting knowledge artifacts that are ontologies and how

00:00:52.960 | we can use them to get some practical wins in our favorite architecture, which I'm sure I've heard

00:01:00.480 | about over the last couple of days, which is GraphRag. I mean, it is part of this track.

00:01:05.360 | But anyway, even if you attended the workshops or you didn't, I'm going to spend a minute just

00:01:13.040 | refreshing what GraphRag is. Basically, GraphRag is your rag where you have replaced your vector database

00:01:22.320 | with the knowledge graph that's built on a database, on a graph database. And the flow is the same,

00:01:29.280 | right? So your AI application will get a prompt from the user and instead of passing it on to the LLM

00:01:34.480 | directly, what it will do is we'll go out to a retrieval, to a controlled, curated, trusted sort of

00:01:40.800 | knowledge base to retrieve potentially relevant information that will be passed to the LLM in the

00:01:45.920 | context window so that the response of the task is accomplished based on more grounded information

00:01:52.000 | rather than purely generated. So now the interesting thing, of course, is that by using a knowledge

00:01:58.560 | graph, what you get is a much richer collection of retrieval strategies. And we're going to see that

00:02:04.320 | you have not only the vector semantic search over the vector index, you can also contextualize results,

00:02:10.640 | because the graph captures the connections between data points. So you're going to be able to go deeper

00:02:15.920 | into how that happens. Is it also possible to generate structured queries? So there's a rich collection

00:02:22.080 | of retrieval strategies, like I was saying, that augments what you traditionally have with you restrict

00:02:27.360 | yourself to only vector search. And the result is that you get better quality results, more completeness,

00:02:34.560 | better completeness, more relevance, better precision, more faithfulness. A lot of metrics that I'm sure

00:02:39.200 | have been mentioned over the last couple of days and I won't spend too much time on. But what does the graph

00:02:45.600 | look like? Again, especially for those of you who are not familiar with this. But the we, Neo4j implements

00:02:51.280 | the property graph model, which has two main primitives is nodes and relationships. Pretty simple. So for us,

00:02:56.960 | nodes represent entities, that's things, persons, objects, locations, events, documents, anything. And they're

00:03:02.320 | connected to each other with relationships that are directed. And then these, you can think of them as

00:03:06.720 | objects that will have a collection of key value pairs that characterize them, like description,

00:03:10.720 | like name, date of birth, whatever. And this is what we call the domain graph. Now, the thing is, when we build

00:03:17.440 | the graph out of unstructured data, we often enrich this representation with what we call the lexical graph,

00:03:24.080 | or some kind of form of description of the source documents from where we have extracted the graph.

00:03:31.280 | And you can see here that we will have a note representing a document, which maybe has a

00:03:35.680 | sequence of chunks, depending on how we partition the document. It can be, in the simplest case, a

00:03:39.680 | sequential list of chunks. But if it's, think of a legal document, like a contract with sections, subsections,

00:03:47.680 | like definitions, clauses, you might have a richer kind of tree description of what that document looks like. And that would be

00:03:54.480 | represented also as a graph. Now, of course, these chunks then will be connected to the domain graph by

00:04:00.880 | entity extraction. So we'll find that this person is mentioned in this chunk, and we'll keep track of

00:04:05.680 | where that's mentioned, because we're going to use that when we do the retrieval. And one interesting thing to

00:04:11.760 | keep in mind here is that these properties that we store in a node can also, well, can be numeric, can be string, can be all sorts of data

00:04:20.160 | types, but it can also be vectors. So these chunks will typically contain the text, the raw text that you have, in which you have, you have split your

00:04:28.160 | document, but you can also have the vectorized representation, and then you can index it. So it's pretty compelling, the fact that you can have in the same platform, a vector

00:04:36.560 | storage with enriched with the connected representation that the graph brings. Now, of course, in order to get to GraphRack, you have to build a graph. There's no GraphRack without a graph. And depending

00:04:48.560 | how, where you're getting your data from, whether it's structured or unstructured, you probably will follow different type of pipelines, right? So if you look at the more

00:04:57.760 | structured approach, you will have some maybe tabular representation in some database or even files, you will have some description of the target schema, the graph that you want to build, and you would map those, basically saying in the simplest case, these records will populate these type of nodes, these other records will define the relationships, and you will write that to the graph. What you have here in the URL is the UI, the interface that our product offers, and basically, it does what I was describing here in a visual way. So you define your graph model, you map to your data sources,

00:05:27.360 | you map, you connect the dots, and you get a graph generated. Behind, of course, there's an API base, and you can do the same thing programmatically. But important to understand that you have to pass the target schema. We're going to get to that in a minute. If you're loading, you're building a graph out of unstructured data, and this is the pipeline that our GraphRack package follows, but if you look at our integration with Langchain, with Laohan index, it will look pretty much the same. What you're doing is you're processing the document, you will split it in some way, based on what we described before, you will take these chunks, and you will embed them, and

00:05:56.960 | And you will run this kind of entity extraction process

00:05:59.740 | that I was mentioning before, to which you will have

00:06:02.140 | to inject your schema.

00:06:04.620 | So you will have to tell what kind of entities do I care about?

00:06:07.140 | What's my domain of discourse?

00:06:08.440 | Am I looking at medical documents, where I'm going to find

00:06:11.140 | information about patients and symptoms?

00:06:13.440 | Am I looking at financial information,

00:06:16.180 | like account holders, accounts, money transfers, et cetera?

00:06:19.980 | So that's the kind of information that you find in the schema.

00:06:22.140 | And then ultimately, you will write it to the graph.

00:06:24.280 | But the common bit there is the fact that we have

00:06:27.100 | to inject that schema.

00:06:28.360 | And that's where things diverge.

00:06:29.620 | Because depending on the path that you follow,

00:06:31.480 | maybe this approach will have a JSON-based representation

00:06:35.080 | of your entities and the relationships.

00:06:36.840 | Maybe here, you're using something

00:06:38.480 | like a pydantic description of the kind of entities

00:06:40.600 | that you want to get, or even a natural language one,

00:06:42.820 | if you're prompting your LLM to do the extraction.

00:06:46.560 | Now, this divergence is something that's not ideal to manage.

00:06:51.280 | And what you want to do is have some kind of agnostic

00:06:53.840 | and general approach to representing schemas.

00:06:56.640 | And that's exactly what ontologies bring.

00:06:58.600 | And an ontology is just a shared description of a domain.

00:07:02.880 | You can think of it as a formal schema.

00:07:05.200 | That's implementation agnostic.

00:07:07.360 | And what's interesting, looking at ontologies,

00:07:10.080 | there's a bunch of standards for defining ontologies.

00:07:12.860 | And there's some of them defined by the W3C.

00:07:16.000 | They can be serialized in a number of ways.

00:07:17.460 | I've picked the XML one.

00:07:19.600 | Why not?

00:07:20.060 | Who is using XML in 2025?

00:07:21.660 | But anyway, there's JSON-based as well.

00:07:23.420 | There's many others.

00:07:24.460 | The important thing is what an ontology describes.

00:07:26.420 | This is exactly what we're after.

00:07:27.980 | It is a domain, right?

00:07:29.020 | So you will find a definition of a class, which actually matches

00:07:32.100 | very well what a graph model looks like.

00:07:34.220 | So this is a fragment of the FIBO ontology, which is the financial

00:07:37.700 | industry ontology.

00:07:39.740 | That's a public one.

00:07:41.120 | And you see the definition of a privately held company.

00:07:44.880 | I don't know if you can read it there.

00:07:45.880 | But anyway, there's another class there that's a stock corporation.

00:07:49.500 | We are specifying that one is a subcategory of the other.

00:07:52.340 | So there's a subclass of this relationship between the two of them.

00:07:55.500 | Then there's even connections between classes of other nature.

00:07:58.260 | For example, we say that a stock corporation is governed by a board

00:08:01.880 | agreement.

00:08:02.880 | So all these sort of elements are described in the ontology.

00:08:05.900 | And that's a perfect, like I was saying, implementation agnostic approach to drive

00:08:11.900 | the knowledge graph creation.

00:08:13.060 | And that will work for both pipelines, the structure and the unstructured.

00:08:17.180 | And basically my take on this is fundamentally be model driven, right?

00:08:24.380 | Because not only because it's data engineering, but good practice, but also because it will

00:08:31.740 | build a better graph in the first place.

00:08:33.900 | And it will pay dividends in the long run.

00:08:37.920 | And we're going to see that in the retrieval phase.

00:08:39.780 | Because if you have a good description of your graph, your text to structure query is going

00:08:44.440 | to generate better and more accurate queries.

00:08:46.500 | But also when we use the vector plus contextualization, that's going to pay off as well.

00:08:50.820 | So you have there an example of use of an ontology to populate a graph with structured

00:08:55.800 | and unstructured data.

00:08:56.800 | I don't have time to go over that today.

00:08:58.720 | But you can have a look there.

00:09:00.440 | Now, on the retrieval side of things, let me spend a minute explaining how this magic happens.

00:09:05.120 | Because I said that you have better retrieval strategies, richer ones.

00:09:08.640 | And what's happening there, you see that the graph captures-- and I'm going to use a visual.

00:09:13.280 | Bear with me.

00:09:14.280 | I hope that's useful.

00:09:15.280 | So you have a graph, right?

00:09:16.280 | And the graph, some of those notes will contain pieces of text, the chunks that we have embedded.

00:09:20.820 | So when we embed them and put them in a vector index, we're effectively creating a new search

00:09:25.800 | space.

00:09:26.800 | Think of this as a new way of entering the graph, right?

00:09:31.420 | So a search in that space would be you embed your query that defines a point in your multidimensional

00:09:37.300 | space and you will find vectors in the vicinity.

00:09:39.800 | So that's what you do when you do a vector search, right?

00:09:41.660 | So the interesting thing is that in the graph, when you find the vectors that are in proximity,

00:09:48.400 | because these vectors are properties of a node, we can dereference them and get back to the

00:09:52.420 | graph.

00:09:53.420 | And from there, we can contextualize.

00:09:55.320 | We can navigate and enrich and filter out, aggregate.

00:09:58.280 | So that's why we do the contextualization and we can bring additional richness to the context

00:10:04.420 | that we pass to the LLM.

00:10:06.240 | Now what does that look like in code?

00:10:07.940 | So, oh, I don't know if it's going to be readable, but let's read from the model map.

00:10:12.780 | So this is, again, a fragment from our GraphRack package, Python package.

00:10:16.660 | But if you look at the integration with Langchain, it's going to be very, very similar.

00:10:21.120 | So we do a search, we specify, we pass our query.

00:10:24.360 | This is a movie database, what we're looking for.

00:10:27.300 | We have embedded and put in a vector index the plots of the movies, right?

00:10:31.040 | So we're looking for something, you can imagine the movie that I'm looking for, right?

00:10:34.360 | We're looking for computers, a hacker that discovers that the world he knows is simulated

00:10:39.100 | reality controlled by machines.

00:10:41.040 | So that's my search term.

00:10:43.900 | And I'm passing that to my retriever and specifying I want the top three results.

00:10:48.120 | That's for me to decide.

00:10:49.680 | And basically what I'm doing is I've created this handler that's a handler over the vector

00:10:54.520 | index.

00:10:55.520 | And all I'm doing is telling it to do this search, but I'm passing a retrieval query.

00:11:01.080 | So basically what you see up here is this retrieval query, which is basically the logic, the exploration

00:11:05.520 | logic that I'm running on every single result that I get from my vector index, right?

00:11:10.680 | So that's powerful.

00:11:13.240 | That's great.

00:11:14.240 | Now there is kind of a problem there.

00:11:16.440 | And my problem is that in order to create this retriever, I need to know my model.

00:11:22.280 | I need to be familiar with the fact that my graph contains actors that acted in movies.

00:11:28.120 | These movies have plots and things like that.

00:11:31.460 | And that introduces, I mean, that's great.

00:11:33.960 | But to some extent it's a little bit rigid because I have to hard code part of the logic in my retriever.

00:11:40.400 | So how could I work around that?

00:11:41.960 | Well, the great thing is that if we take the ontology that we saw before and we persisted

00:11:46.840 | in the graph, now we have the graph of our data, our movie database, and we have an ontology that describes that schema,

00:11:52.680 | what I can do is create a dynamic query.

00:11:56.680 | And I understand that you guys are not cypher experts, but, you know, if you bear with me on that,

00:12:02.680 | I mean, what we're doing is I'm querying my ontology where there's a definition of a relationship.

00:12:07.680 | So I know that actors act in movies and there's a definition of this relationship.

00:12:12.080 | That's what my domain model describes.

00:12:14.240 | Now, I'm going to query that and I'm looking for relationships that I have annotated as contextualizing relationships.

00:12:21.520 | So I'm defining a subcategory in this case, a property of.

00:12:24.000 | So what I can do is drive the behavior of my retriever by consulting the ontology.

00:12:30.880 | So I look at the ontology, I find the relationships that have been specified as contextualizing relationships,

00:12:35.440 | and those are the ones that I navigate.

00:12:37.200 | So this R is the one that I'm using to then find nodes that are connected to other things.

00:12:42.000 | So, in other words, my ontology is driving the behavior of this query,

00:12:46.160 | and it's determining the fact that when I find the movie The Matrix,

00:12:49.120 | I'm going to be navigating just the acted-in relationships and not the produced or the directed.

00:12:54.160 | I understand that, you know, it might be going a bit too deep into the cypher aspect

00:12:59.600 | without having, you know, you train a little bit on it, but I hope it makes sense.

00:13:03.120 | I mean, the idea is that's another way in which ontologies can drive, can give us,

00:13:06.720 | kind of a level of indirection that makes it possible for us to kind of build dynamic

00:13:12.240 | retrievers that are driven by data. So I can change my ontology, and by changing my ontology,

00:13:17.440 | which is a data artifact in my graph, I'm changing the behavior of my retriever

00:13:21.520 | on the fly and dynamically. So that's what I wanted to share with you.

00:13:25.920 | So the two takeaways, I mean, the first one is ontologies for knowledge graph creation as an

00:13:31.120 | implementation agnostic data model. And the second one, storing ontology can help us drive dynamic

00:13:38.880 | behavior in our retrievers. That's what I had for you today and just in time. So thank you very much,

00:13:43.360 | and happy to take the questions or the conversation back to our booth.

00:13:46.320 | Thank you.

Why Your Agent’s Brain Needs a Playbook: Practical Wins from Using Ontologies - Jesús Barrasa, Neo4j

Chapters