Hello everyone. I am here to talk about GraphRAG as we're here for the track, but I'm talking about what to do in the legal industry and what we do in the legal industry and what does it look like to turn documents into graphs and use those graphs in the age of AI.

I tend to have to qualify why I'm at places. There are various reasons why I could be talking today. You choose the one you want to, but generally I've been working in graphs for about a decade. I have a good relationship with the Neo4j team and I've been doing graphs for a long time, but primarily I am the technical founder of a company called whyhow.ai, and we find cases first before lawyers do and then give them to lawyers.

Now, how we find these cases is a process that will go through, but we use a variation of graphs, multi-agent systems, signals, et cetera. And I'll detail through today how we do that at a high level and a low level, and I'm happy to answer questions at any point.

This is broadly what we do. We work in law. This is an example. We find class action mass tort cases before other people do. We have agents. We have graphs. We store that information. We scrape the web. We qualify that with a proprietary process, and we deal with lawyers every day and understand exactly how they think and build these cases.

And the cases I'm referring to would be like many people used a pharmaceutical product. That product has caused them harm. Science has proved that harm, and we can collect those people and collectively sue the pharmaceutical company. So we support the law firms that do that. And as I'm talking, and everyone here for a graph rad track can start to imagine that I'm starting to develop a bit of a schema there.

I'm describing individuals. I'm describing products. Those products have ingredients. Those ingredients have concentrations. Those concentrations may have an ID number, and all of a sudden, you can start to imagine there is this large networked, schematized bit of data that has particular points in it that are very valuable and very visual and very useful to domain experts.

So I'm going to start to use some definitions, because knowledge graphs have been around for a long time, and ABK would know that more than I would. But I started my PhD and wore my master's in graphs in 2016, and it was not nearly as popular as it is now, and it's fascinating to see how far it's come.

But I do think it's important for me to define how we use them and how we think about them. Broadly, to me, graphs are relations. That's part of the visual element, and there's a back-end element as well. But it's the benefit of using graphs is that I can see what is connected to something else.

I can be explicit about what is connected to something else, and I can do mass analytics on what is connected to something else. All the way from I can see it down to I can do large-scale analytics on it is the value of the relations. And when I use relations, it's not necessarily node to node.

It can be node to node to node. It can be multi-hop. It can be as varied and as forked and as distributed as you want. This is why we use graphs in our process. Broadly, throughout the process of running this company and previously as an academic, this is what I think is easy about graphs.

People look at them and go, well, that's fantastic. I have a great understanding of what this is. And someone else says, me too. And there isn't necessarily a consistency in what those two people just said. They may have a different understanding of what is represented. Broadly, throughout my career, these are the things that are difficult about graphs, right?

And you can say that they're nodes connected to edges. You can say they're distributed. You can say they're backed up. There's a variety of ways in which people use the data that they have, the way they store it and the way they talk about it. And now, as graphs have become very necessary and consistent for things like graph rag, for things like structured data, etc., more and more people are coming to this relatively niche area previously that even at the time wasn't necessarily agreed upon what it was.

So, I do like to define what it is we're using. So, graphs and multi-agent systems. These are the two things that I want to define as there's a variety of ways that people use them. So, this is how we use multi-agent systems, right? So, now, multi-agent systems are all the way from very specifically define what you're dealing with and chain those together and use an LLM to glue it all together.

Or it is, in our case, break down a complicated white-collar workflow down into a specific set of steps that I can I/O test, right? And each of those steps have different requirements, different frequencies, different state, and that state can be controlled often, in our case, by a graph. This is why we like to use them.

When we're building an application for the legal industry, I'm not sure if you guys know this, but lawyers don't really like when things are incorrect, right? It is basically the whole industry is make this very specifically correct and proper and definitely in the right language. So, when it comes to building applications, probabilistic large language models don't necessarily work for that just in isolation.

I need to have a very specific control and structure and schema for the way that we build these systems, and I need to be able to test and be able to pinpoint exactly what is going right and wrong at any point in time. Here are some of the issues with that, right?

And we've heard about multi -- well, at least I've heard about multi-agent systems a lot. I'm sure other people have as well. Sometimes the part in the workflow is much more important than the other part. Sometimes there's parts in the workflow I don't particularly care about. There are also agents in the world.

Agents imply that these things are very capable, but I can write a bad prompt very easily, and all of a sudden I have a bad agent. So, when it comes to what is the agent that I trust, very few. We spend a lot of time guard railing as much as we possibly can.

We spend time making so that the memory is not just immediate, but it's episodic. We spend time capturing the information state over time and then pruning that state. And again, to bring it back, capturing, expanding, pruning, structuring, and then querying state for us happens in a graphical format. Because the necessity of having the structure, having the extendability, and then having the ability to remove that extension is really important for us.

And then finally, I'm trying not to make this too deep depth and too many numbers, but 95% accuracy for a single agent, I think, is a tall order at this point. Maybe people have entirely accurate agents. I'm very happy for you. I don't have that exactly right now. I have systems that I can put in place, like guardrails and humans in the loop, that can bring these agents to a point that it is accurate enough that people are willing to use them.

However, five 95% accurate agents chained together sequentially, that's 77% expected accurate accuracy. That's not that many agents in a row. If you think about a workflow, that's five steps. And if I'm basically saying that if each of those five steps are 95% accurate, already quite a hard thing to ask, especially if there's an LLM involved.

Now we're at 77% of the time, it gets to the end of that workflow in the way that I want. That is part of, probably, if I was to summarize my main problem with that. It would be decision making under uncertainty throughout the process of building these systems. That's the background.

That's how we understand these systems. We use multi-agent systems and we're naturally skeptical. We use graphs every day and we have a natural skepticism of exactly how these things are stored and structured, but we use them specifically and consistently in the way that we like. So I am using the term agent because everyone's using the term agent.

We build litigation agents. Litigation is the process of, well, I'm going to summarize, but we work with class action slash mass tort law. As I said before, get everyone together. They were harmed. Put that harm all in place and then sue a pharmaceutical company. Now, we don't do any of the litigating as a company or the suing, but we do support the lawyers who do that.

We do that in a few different ways. Here is one of the ways that we look at the legal industry, right? Without exception, everything needs to be perfect, needs to be accurate, needs to be written in the correct way, right? There's also, once you have that correct format, creative arguments.

The best lawyers are very, very, very detail-oriented and then very, very creative in the way that they can apply those details to a case. For example, there was an issue with Netflix and they were capturing data from their users, as they do and they should, and I'm a Netflix user and they capture my data and I appreciate it because they give me the better shows that I'd like to watch.

However, there is a legal limit as to how much information they can capture from me, right? And you cannot surpass that legal limit, or you can, but then you can go into the process of litigation. Now, if you surpass that, there needs to be a precedent as to why someone could say, "You cannot capture this much information." And the particular precedent I'm referring to is many years ago, Blockbuster was sued by keeping too many details about the literal physical DVDs that people rented.

That is a reasonably creative way to say, "Look, I remember that Blockbuster happened, and what Netflix is doing isn't that different. It may be in a digital format, it may be at a larger scale, it may be into an algorithm instead of someone who's recommending it. However, that is an interesting application of what I'm doing." So, these problems then, which is necessary accuracy and then creativity on top of that accuracy, and then all of that information is kept in separate places, and a lot of that creativity comes from the latent knowledge in the expert's head, starts to come to a bit of a fall when you say, "Well, I have these probabilistic agents that you could argue aren't that creative." Right?

I have these agents that most of the time do a pretty good job and can be creative in a way that, frankly, can be quite frustrating, especially to a lawyer. So, this butts heads in terms of exactly how lawyers want to deal with this information. And again, I'm painting a very broad brush.

I'm not a lawyer. My co-founder is. If anyone is a lawyer in the audience who's offended, I do apologize. But this is broadly what I've seen to be accurate. We help with legal discovery as well, right? Like I described before, there could be an unnamed pharmaceutical company. A pharmaceutical company's great, but they happen to have done some harm, right?

And it is in their best interest to give all of the information to the law firm and describe exactly -- well, not exactly -- describe in as many ways as possible, "Here is 500 gigabytes of emails that don't matter. Go nuts." Right? Figure out exactly what happened at what point and bring up the information.

Now, that is a challenge at the moment. A lot of the time it's manually reviewed. There are shortcuts and processes by necessity because a lot of these lawsuits are on a particular timeline. It is physically impossible to read all of the information that is given in the discovery of the processing of a lawsuit.

However -- and this is just a generic graph I used because I'm not allowed to use the ones that I'm currently working on -- however, if you can take all of that information, you can extract the information and structure it in such a way that it is consistent, all of a sudden that mountain of emails becomes a lot of information I can immediately dismiss and a bunch of generally, genuinely useful information that I can look at.

And not just that, when it comes to a graph, I can actually augment the information from discovery and then I can give that visual to the expert who can make an immediate decision. I'm going to loop back to the example I was working with, what I was describing before, which is the pharmaceutical example.

So again, if ingredients are a certain concentration, that concentration is at a problem, that problem happened at a certain time, there is only going to be a few people in that graph of potentially millions of nodes that are a problem, right? In the same way that there are only a few people in that mountain of documents that were a problem.

However, now I've changed the form factor such that I can specifically hone in on what matters, and not just hone in in a data-driven way, I can hone in in a visual way and natural language such that the lawyer who knows exactly what that natural language means, or the expert who knows exactly what that natural language means, can make a decision that's data-driven.

There's also a process if we can build this information exactly, and I'm giving the fundamentals. This is a graph rag talk. We want to bring this graph in. The graph I just described is not that large. The graph I just described has a consistent schema, and the graph I just described can be relatively easily retrieved.

I'm not going to say that retrieval is completely solved. I am going to say we have agents in production right now that lawyers can in natural language query and further understand the lawsuit and the individuals that they are representing. Now we get to case research. That was more discovery, right?

A mountain of documents. Case research would be a lot of people used said product, and they are complaining about it online. And this is a lot of the value of our company and what we do. People can complain all the time. They can shout into the void of a niche subreddit, or they can go on Twitter, or they can be on a forum that they're used to.

They can be in IRC. They can be wherever they want, right? But they're using similar language about a specific thing. And so when it comes to traditional case research, that information isn't really discovered. A lot of the time it happens through talking to another individual, subscribing to a newsletter, et cetera.

How do people find the information? So, and this is a graphic I've taken from our website, which I promise looks significantly better than the slides that I make, but I tend to try and talk to them. Here is how case research, in our case, for our business, works. And that is we start and scrape the entire web.

Now anyone can scrape the entire web. It's doable. It's a technical challenge, but it's doable. And you can scrape it at a frequency in the services, et cetera. What we do is scrape the web and then qualify the leads of that scraping. We filter down all of the information down to specifically what the individuals want.

We have schemas that we work with particular law firms and lawyers. And those schemas get us down to just the information that they care about. And look, maybe there is, but right now, at least for me, there's no such thing as a perfect case. There's no such thing as a perfect lawsuit.

It depends on the lawyer or the partner or the firm who's willing to take that on. So it is not a problem of best. It's a problem of specific and personalized. And that is where things like LLMs are particularly useful at the moment. That's where things like multi-agent systems are fantastic.

That's where things like structured information and graphs all of a sudden, a different lawyer can have a different multi-agent system and a different graph that backs up their specific way that they like to work, as opposed to having a compromise previously on the way that everyone else liked to work to maybe hear something if they can.

And from there, once we've honed down just to the signals that they care about, the qualified signals that are specific to them, that signal can then further generate a report, and that report can be entirely specific to the lawyer as well. So when it comes to report generation, again, a multi-agent system that's backed up by a schema, and that schema is consistent and pruned, and that schema looks like a controlled state with a graph that can build the report that the lawyer wants.

And every report is going to be different, but the structure is going to be the same for each lawyer, and each lawyer has a different process. What I'm broadly describing is mass scraping the web down to a specific signal generated just for the lawyer. It's entirely personalized service that's been automated.

And that is the process of what we do, and this is part of how we are able to manage and use state and graphs and multi-agent systems to bring the information together. Cool. I'm going to go through -- I think I have one case study that I want to describe, just conscious of time.

This happens. It's not great. No one really wants it to. There may be situations in which there's a bunch of people who bought a car who really wanted it to catch fire. We don't necessarily deal with them. What we do find is that there are people who are driving their car, and it starts to smoke, and then it catches fire.

That's not the behavior that they intended to happen. It was not on the brochure when they bought it. It's not what they want. Those people immediately go and complain, as they should. They go to government website. They go to carcomplaints.com. They're on a specific subreddit or forum. And once we can start to track that, which we can, and once we can start to scrape and then structure and then schematize and then analyze, we can start to basically build a density of complaints for a specific vehicle, for a specific year, for a specific problem.

And that density is a combination of how many complaints multiplied by the velocity of complaints, so a certain amount per month over a number of months. All of a sudden, we get to the point where we're finding these leads particularly early. And now, as we're building models, we're starting to find these leads early and earlier, and that we don't necessarily need the velocity straight away.

We can start to figure out what are the previous lawsuits, which were all public and very well documented, and exactly what happened in that process. And so, for a large law firm, maybe eight or nine months post people starting to complain, they can take that lawsuit on if they want to.

For us, we can find it within about 15 minutes. And then generally, it takes probably a month for you to be confident that this is the signal that you want. And so we can find things significantly earlier. That process, again, scraping the web, filtering down, producing the specific report.

This is an example that we did. And again, we deal with what the lawyers want. So this lawyer, again, he made the case that people's cars are catching flyers. They don't really want them to. Those are the cases that he would like to take on. It's of a certain amount of money.

It's of a certain make and model. It's in a certain jurisdiction, et cetera. Those specific filters, that schema can be applied throughout the entire process. That's basically the graph. Each of these lawyers have a specific graph that they want. And not just that, they can filter and feedback that information.

So it's not just a static graph. I mean, the benefit of a graph structure, at least one of the benefits of a graph structure, I should say, is that it's an extensible schema. And that I can update and I can query across and I can understand that information. So while we are dealing with RAG, I would say we have less of a chat RAG interface.

Well, the lawyers definitely do appreciate that. A lot of what we have when it comes to RAG or retrieval augmented generation would be generating these reports. Because as much as a lawyer does want an answer, what they also want is the form factor they're used to. And so all of these graphs are consistently made and built each day.

And then some subgraph from that broader monolithic structure is then brought in and composed into a report that a lawyer can action. Kind of what's next. And I'll talk about the future a little bit. I mean, what I described is kind of what we're doing. But this is what we're doing.

Find law suits early, compensate harm, and then people can have that information if they want to. We're able to do this entirely technically. We're able to scrape the web structure, etc. We're able to iteratively build up a schema as we want to. So this is not just a Gen AI problem.

And I think this is an important thing that I've seen around this conference and people may be seeing is that Gen AI is not better than machine learning. And LLMs are not you know, better than traditional ML systems. But there are situations in which one is fantastic and one is not.

If you look at multi-agent systems, and again, I was previously an academic and multi-agent systems and no one ever listened to me. So this is a bizarre situation. But that when you used to structure the multi-agent systems together, somewhere along that workflow, you would have to stop or say this is not doable, because I cannot plug these two bits of information together.

It's too probabilistic, or it's too random, or it's too inconsistent, or the way to describe it is not a binary feature, right? It is, I really just want to kind of type what I want, right? Now with LLMs, you can. But it's very much for us, not an LLM filtered system.

It's an ML filtered system that LLMs have allowed us to pipe together such that you can actually provide value completely end to end, which I think was previously not doable. And for us, again, we've been using graphs for a long time. For us, the ability to iteratively build that graph, prune that graph, and every single report gets better, because we're able to manage the state, is why people like working with us, because we can consistently follow and track exactly what they want specifically.

Cool. I think I'm just about at time, kind of got in early, but that's been the talk specifically around -- I'm happy to talk to anyone about the specifics, graph rag, et cetera, multi-agent systems, but that's how we use the process. Thank you very much. Thank you very much.

Thank you very much. Thank you very much. Thank you very much. Thank you. We'll be right back.