Wisdom-Driven Knowledge Augmented Generation at Scale

00:00:00.000 | So hi, hi everybody. My name is Ching Kyeong-Lam. I'm a founder and CEO of Patto.ai. A bit of a

00:00:23.120 | background about my company. Patto.ai started two years ago with an invitation from National Science

00:00:28.720 | Foundation, from the SBIR grant funding investigating LLM-BIRD. We did a LLM-BIRD driven discovery

00:00:37.040 | application. Since then, we branched out to leverage what we learned about building AI system

00:00:43.200 | for large corporations. We are currently building expert AI system for several clients. Currently,

00:00:49.280 | the system we build goes beyond RAC system. Many of our clients are asking for AI systems that perform

00:00:56.320 | tasks like research and advisory role based on their area of interest. Today, the talk is about

00:01:03.600 | sharing with our fellow AI engineer what we learned so far building this kind of system.

00:01:08.400 | Okay. What is knowledge? Generally, philosophically, I say knowledge is the understanding and awareness

00:01:15.120 | gained through experience, education, and the comprehension of facts and principle.

00:01:19.920 | And that leads to the next question is what is knowledge graph, right? So knowledge graph is a

00:01:25.200 | systematic method of preserving wisdom by connecting them and creating a network or interconnect relationship.

00:01:32.000 | That's important. The graph represents the thought process and comprehensive taxonomy of a specific

00:01:39.600 | domain of expertise. That's why this is very important for people's moving forward. It's about AI system. Then think a lot and return

00:01:48.080 | and advice instead of just retrieving data from your database, right? So that comes to the development of

00:01:55.360 | this KAG. What is KAG? KAG stands for Knowledge Augment Generations, and it's different from RAC, okay? It is enhanced

00:02:04.000 | language model by integrating structure knowledge graph for more accurate and insightful response, making it smarter,

00:02:11.200 | more structured approach than a simple RAC. KAG doesn't just retrieve. Remember, it understands that this is different.

00:02:19.600 | Okay. After interviewing a lot of my clients, okay? So we also expert in a certain area of scale, I found that there are

00:02:29.760 | common ways of their thinking, decision-making process. The way that make them expert in their area, knowledge graph, seems to be a perfect fit.

00:02:38.480 | So here is the graph or state diagram if you're a computer engineering grad like me.

00:02:43.760 | So it shows wisdom. The wisdom note, as you can see, it's a core, right? It's wisdom. It just isn't static.

00:02:55.760 | It actively guides decision and views by other elements.

00:03:01.760 | The output from the wisdom actually goes to decision-making in the blue, right? Wisdom isn't passive.

00:03:09.200 | It guides decision, helping us choose wisely, okay? And then the decision-making analyses the situation,

00:03:16.800 | given in the circle in the green, and decisions aren't made in a vacuum, okay? They analyze real-world

00:03:26.320 | situation. That's the difference, okay? So look at the wisdom input, okay? Look at the relationship feedback

00:03:32.640 | from the knowledge to wisdom in gold colour. Example of that is knowledge to wisdom, like

00:03:40.400 | All your book smart, encyclopedia, Wikipedia, whatever you store, plus once that data get absorbed by IRM, whatever model you use up there,

00:03:52.640 | it need to regurgitate that and understand. That's why it's very important that wisdom is able to synthesize the data after you ingested knowledge.

00:03:59.680 | You know, that's the kind of abstract, but I'll come to that later. How about talking about, okay? Okay, from inside, example of that is

00:04:07.120 | wisdom derive pattern from chaos, right? Some of my client has a lot of social media, their product,

00:04:14.800 | how do they, you know, track their product sediment from social media, right? So it's very chaotic,

00:04:20.240 | and from X, tweet, right? So from that, you can see some pattern of their competitor versus

00:04:24.960 | current what my product is. That is, like, an example of that, and I will go to that later. Okay,

00:04:31.520 | when all these connected nodes matter together, why do they matter? All the nodes relate to one another to

00:04:38.400 | ever enriching wisdom storing system. Okay, this talk is about storing wisdom, right? So knowledge tells you

00:04:47.360 | what it is, right? And experience tells you what worked before. Inside, invent what to try next. Like a pizza,

00:04:59.600 | knowledge is recipe. Experience is knowing your oven burnt crust. Inside is like, hey, it is

00:05:06.880 | adding, you know, honey to the crust, you caramelize perfectly, right? So the most important part of the

00:05:13.920 | knowledge graph is feedback loop, okay? Feedback isn't one-way street. It learns from itself. Look at the

00:05:20.320 | feedback from the, going back to all the nodes, from insight to wisdom, okay? Situation informs future

00:05:28.880 | wisdom. Experience, deepen it. Insight, sharpen it. Like a tree growing roots. The more it fed,

00:05:36.960 | the stronger it gets. Now, I want to ask you a question in general. Where do you see this circle in your

00:05:43.680 | life? Maybe a tough decision that, you know, taught you something? So one practical application for

00:05:51.840 | leadership is wisdom. Avoid knee-jack reaction by learning from feedback. As for personal

00:05:58.800 | growth, ever notice how past mistakes make you wiser? That's the loop in the action. All this. So,

00:06:06.880 | the takeaway from the slide in this is, wisdom isn't a trophy you earn. It is a muscle you exercise. The more you

00:06:15.120 | feed knowledge, experience, insight, the more they guide you. Now, I will show you how it being mapped to my current

00:06:24.480 | client. You know, all this is like very abstract, right? So, how I, one of my clients actually doing a

00:06:29.680 | competitive analysis. They used to have a marketing department doing that, but they want AI to do that,

00:06:36.160 | right? They asked me to build a system. This is exactly what I did with the same textonomy of storing all

00:06:42.160 | this. So, this textonomy will be, later on, I talk about how multi-agents are going to handle all that.

00:06:47.840 | Here is one of the chatbots that I build for my client to do, you know, not just some,

00:06:54.160 | we, not just some chatbot, okay? It's our wisdom graph power AI designed to turn data into strategy,

00:07:01.520 | right? Dominant. So, what kind of question I talk about? Talk about how do I win my competitor in this

00:07:05.680 | market space? That's kind of a very sophisticated question, right? So, without, uh, if you do simply

00:07:10.480 | just write my first speaker, talk about right, right? So, it's not going to cut it. They're not going to

00:07:15.040 | able to answer that kind of question, okay? What I did is this. We retained the same textonomy, and

00:07:20.640 | the wisdom is then mapped, the same engine there, the wisdom engine. The wisdom engine is like an

00:07:25.120 | orchestration agent that does a lot of decision-making, including advising what the ARAM is able to see,

00:07:31.760 | based on the current situation, what to do next, right? So, um, what I did is, uh, for the, uh,

00:07:37.680 | decision-making, I mapped it to a strategy generator. So, these customers are talking about a competitive

00:07:43.120 | analysis, right? So, um, I mapped the knowledge, in terms of knowledge, what do they have? They have

00:07:48.160 | market data, right? So, I mapped this experience to HP, it's one of a kind, okay, past campaign. So,

00:07:56.240 | they have a lot of campaign doing a lot of marketing, and then, um, the insight is actually

00:08:01.520 | mapped to, uh, in, industrial insight. So, they have a database doing, storing that, and then,

00:08:07.120 | of course, the most important is, is the situation. The situation is, how, how am I doing, how am I

00:08:13.440 | product selling, right? So, so that, that is like a situation, and then I mapped that to a competitor

00:08:19.040 | weakness. That means to say, if you make the ARAM aware of that, you probably get a very good answer,

00:08:26.320 | and then, you know, the chatbot will probably be doing the right thing, advising it. So, from here,

00:08:31.840 | very high level, you know, state diagram, all that. How do I mapped it to a system that drive? Well,

00:08:38.320 | here comes the trick. So, anybody here heard of N8N? All right, all right, it's all good. So, so, I, I first

00:08:45.920 | encountered similar situation when I, my past IoT project, which is not great developed by, uh, IBM,

00:08:52.720 | right? So, it's the same kind of thing. It's like, no code, but, but underneath the hood, there's a bunch

00:08:57.040 | of code, okay? It's on Node.js code, okay? So, uh, but, but for the, for, for proving your concept and all

00:09:02.880 | that, it's very, very, very flexible. And I, I, I highly recommend that, and, and, and here, here, you can take a look at the, the workflow, the

00:09:09.920 | the way from, I enable the implementation of these complicated state diagram with, um, uh, what I say is,

00:09:16.480 | there is a different community node. One of the very powerful node is the AI agent node. Well, previously,

00:09:21.520 | N8N is just workflow automation tool. I'm not selling for N8N here. I'm just telling you I'm using it, uh,

00:09:26.640 | for pro, prototyping, uh, further down the road, maybe the client say, "Oh, I, I, I really need to, you know, go

00:09:33.520 | lightweight. Maybe we will switch over to some other link chain or whatever." But, uh, we actually use this,

00:09:38.720 | like, I mapped the previous, uh, state diagram from the wisdom engine. I actually mapped that to our, uh,

00:09:45.280 | uh, wisdom agent, okay? Wisdom agent is now have the option to drive, uh, different model, like OpenAI

00:09:51.280 | model, Entropic model, and even on-prem model. And then that, the key in making the state, uh, the state

00:09:57.760 | machine work is that my wisdom agent is now overseeing, like a supervisory agent, or all these other agents

00:10:03.680 | that do, uh, whatever I say on the state diagram. Um, for example, the, uh, state of, uh, going into a

00:10:11.520 | node of insight. Insight agent will test to do, go to the social media, go for the sediment of all your

00:10:18.800 | product, and then collect that, and then pump that, that you can see that at the dot, or bottom that we,

00:10:24.320 | are connected to a, uh, uh, uh, uh, centralized, uh, graph. The centralized graph will be

00:10:30.880 | able to get updated by different agent. Uh, insight agent will update the, here, their perspective,

00:10:37.280 | like, part of that graph for the, uh, as I say, for this particular, uh, uh, insight note. So, so all the

00:10:44.880 | unified knowledge graph will contain the textonomy that eventually just think like the marketing

00:10:53.120 | strategies, the way that here, they will probably, if you are doing manually, they probably would think,

00:10:58.560 | you know, you know, SharePoint will all this, you know, folder will start the same kind of, uh, you

00:11:04.000 | know, wisdom, I call it, to make decisions based on that. So, the, the final decision is LLM. Also

00:11:09.280 | depend on the model that you use, uh, but I, I, I, I pretty much think that not really the way that

00:11:15.920 | I think the final decision come when you make the right decision from the advisor output is basically

00:11:21.200 | depend on all the textonomy, the graph structure. That's very important. So, come to that, I, I want

00:11:26.960 | to go deep down how I implement one of the nodes, uh, just to go a bit technical on this competitive

00:11:33.120 | node. How do I implement that? Okay, before I do that, okay, competitive analysis, right, what, what,

00:11:38.640 | well, you can actually just use, right, what do you want to use a knowledge graph like Neo4j? Well,

00:11:43.280 | if you ever be asked that question, tell them to seize five, uh, five reasons. The first reason is

00:11:49.360 | knowledge graph, you know, uh, system excel at capturing and representing complex relationship with

00:11:55.680 | the entities. That is covered by the first speaker, but I'll just reiterate that this lead to a deeper

00:12:00.240 | contextual understanding, which is crucial for competitive analysis, where this, in this case, the nuance

00:12:06.560 | insight can be significant, make a significant difference, okay? You want to find the gap in your

00:12:11.200 | computer weakness. Now, this is very important. The second is improve accuracy. By leveraging structure

00:12:16.720 | data and semantics relationship, knowledge graph can provide more accurate and relevant information

00:12:22.000 | compared to traditional vector racks. Um, this ensure generator content is not only relevant, but also

00:12:28.880 | precise and reduce the noise and improve decision-making, making this. In this case, the bot is supposed to help the guy that is

00:12:35.200 | marketing department make decisions. So, so you better make this work, improve accuracy. Any inaccurate data,

00:12:41.600 | you will be out of the contract, out of the door, right? So, it's very important, okay? You're talking

00:12:45.760 | about contract work like me. I have to make the rack as accurate as possible. So, the third is scalability and

00:12:51.120 | flexibility. Graphic, you know, knowledge graphs are entirely scalable and can integrate to new data source

00:12:57.200 | and relationship. The flexibility allows the continuous improvement. As I said, if your taxonomy is correct, you will continue to improve

00:13:03.840 | and enrich, right? So, so that is important. And also, rich query capability. Knowledge graphs support

00:13:09.920 | complex query, traverse to multiple relationship entity, provide richer and more detailed insight. This is

00:13:16.320 | particularly advantage for a competitive analysis when multi-facet query, like, like what the first speaker

00:13:21.680 | said, it is super authoritatively good in answering things that normal rack will fail. It's a multi-hop question, okay? This is very important. And then the final one is the

00:13:33.680 | enhanced data integration. Knowledge graph can seamlessly integrate diverse data source, pictures, graphics, videos.

00:13:41.440 | However, it is, now that LRN is so powerful, we have OCR capability, we can do that. As long as you have a right

00:13:48.160 | structure of the graph, semi-structure, unstructure, the holistic approach ensure compressive view of the

00:13:53.520 | competitive landscape and enable more informed strategy decision making. Okay. So, one of the, this is, I'm going to just very

00:14:01.360 | briefly go through this. It's just a, uh, uh, example of the, some of the thing, like, um, problem of vectors

00:14:07.120 | rack, you know, vector rack is really, really bad in answering limited numerical risk reasoning. Vector store,

00:14:13.360 | Excel, you know, add semantic similarity, but struggle with complex numerical calculation. This is why, uh, for, uh,

00:14:20.720 | marketing analysis, uh, that I'm building the chatbot for, uh, they actually rely on number instead of just,

00:14:26.880 | you know, returning example like this, if you ask like, what is the Apple revenue between, you know,

00:14:36.540 | what's the revenue in 2022, they probably would give you a bunch of this kind of a passage, right,

00:14:42.000 | retrieval graph, instead of this kind of a very, very precise thing, like the answer is, you know,

00:14:49.140 | because about the data is already there in the structure form, the data source assume a knowledge

00:14:56.200 | graph name, this particular, in this particular case, Apple financial data, the query will be able,

00:15:01.660 | the query engine will be able to select the revenue figure from 2021 to 2022, and then do a function call,

00:15:08.800 | the function call will eventually give, come up with 15.23, which is exactly what the marketing

00:15:14.200 | guy was looking for, a very quantitative stuff, that most of the decision were based on that,

00:15:19.180 | because you have the evidence, not just some passage that you retrieve from the data, it's basically evidence-based

00:15:25.520 | decision-making is very important for this kind of complicated RAC system that, you know.

00:15:30.980 | So, there's a jungle out there right now, you can use different kind of a, a, a, a thing to build your,

00:15:37.420 | you know, this is just a snapshot of that, you know, you can actually use Langchain plus Chroma to build your own RAC,

00:15:44.980 | and then you also can combine that with your knowledge graph.

00:15:48.040 | Depend on your user case, okay?

00:15:50.320 | If this slide show that the RAC and the KAG

00:15:54.240 | can be viewed with many, okay?

00:15:55.980 | I adopt that wisdom graph in red color.

00:15:59.100 | Normally you will see if client is just asking

00:16:02.060 | for a simple RAC that perform product information query,

00:16:05.680 | you can just use a simple chroma DB with our agent.

00:16:09.580 | And if you start to ask so complicated questions

00:16:12.180 | like how can I beat my competition

00:16:14.060 | based on my current market share?

00:16:15.800 | Well, this will be able,

00:16:18.160 | the thing that I will probably be adopting

00:16:21.100 | is knowledge graph here with graph DB plus cipher query

00:16:25.300 | and it will create ONA and also train my RAC

00:16:27.780 | to perform several loop of, we call multi-hop query.

00:16:32.220 | And this probably will give a very good answer.

00:16:35.300 | So, and then it comes to the another question.

00:16:38.580 | When I was trying to extract my,

00:16:41.220 | hold, I think my time is almost up.

00:16:44.760 | Okay, so anyway, this is like to say,

00:16:46.840 | the first speaker talk about the extraction, right?

00:16:49.240 | There's a very simple way to extract on the right side.

00:16:51.200 | It's like automated, totally automated LIM graph transformer.

00:16:55.360 | On the left is like manual.

00:16:57.020 | I probably recommend the center of hybrid model,

00:16:59.320 | which is like, after you use the LIM to extract your graph,

00:17:02.380 | you ask the interview, the expert that you're gonna build,

00:17:06.020 | to build the taxonomy, right?

00:17:07.900 | To prone the graph, we call it prone your graph.

00:17:09.820 | Remove a lot of relationship, then that will be okay.

00:17:12.820 | And I will try to just highlight this.

00:17:15.320 | This is the result of benchmark that we did.

00:17:17.780 | Okay, anybody ask you, you know,

00:17:19.300 | why you want to use graph, right?

00:17:20.820 | Or KAG.

00:17:21.560 | Okay, first is accuracy.

00:17:23.220 | I have achieved 91% because it's really good in extract structure.

00:17:26.400 | Second is flexibility, 85%.

00:17:28.920 | Third is reproducibility, deterministic.

00:17:32.780 | And then the fourth one, traceability.

00:17:35.540 | And finally, most important is scalability.

00:17:38.540 | So in conclusion, by leveraging structural nature

00:17:42.860 | of wisdom knowledge graph,

00:17:44.460 | we can significantly enhance the quantitative capability

00:17:46.960 | of KAG system and able more accurate

00:17:50.240 | and insightful response to complex query.

00:17:52.960 | By using wisdom-driven system as highlighted,

00:17:55.600 | together we can build smarter AI system that can scale

00:17:58.580 | and store wisdom with the right framing,

00:18:00.620 | potentially support the intelligence of the initial expert

00:18:04.100 | that we meant to serve.

00:18:06.380 | So, we talked to Jesus, you know.

00:18:09.880 | What did he just do?

00:18:10.660 | Talk to Jesus.

00:18:11.500 | He's in a not booth.

00:18:13.880 | This is my good friend.

00:18:15.040 | And anybody that wants to build graph,

00:18:17.900 | we have a good so-called LLM graph rack stack on GitHub

00:18:22.480 | that is sponsored by Neo4j.

00:18:24.380 | And out of the box, just spin up your Docker.

00:18:27.660 | The next thing you know, your text is going to be converted

00:18:29.960 | to your graph, and you can start happy pruning your graph.

00:18:32.820 | Thank you.

00:18:33.540 | - Thank you so much.

00:18:34.540 | - All right.

00:18:35.540 | - All right.

00:18:36.540 | - Thank you.

00:18:37.040 | - Thank you.

00:18:38.040 | We'll be right back.

Wisdom-Driven Knowledge Augmented Generation at Scale - Chin Keong Lam, Patho AI

Chapters