So hi, hi everybody. My name is Ching Kyeong-Lam. I'm a founder and CEO of Patto.ai. A bit of a background about my company. Patto.ai started two years ago with an invitation from National Science Foundation, from the SBIR grant funding investigating LLM-BIRD. We did a LLM-BIRD driven discovery application. Since then, we branched out to leverage what we learned about building AI system for large corporations.

We are currently building expert AI system for several clients. Currently, the system we build goes beyond RAC system. Many of our clients are asking for AI systems that perform tasks like research and advisory role based on their area of interest. Today, the talk is about sharing with our fellow AI engineer what we learned so far building this kind of system.

Okay. What is knowledge? Generally, philosophically, I say knowledge is the understanding and awareness gained through experience, education, and the comprehension of facts and principle. And that leads to the next question is what is knowledge graph, right? So knowledge graph is a systematic method of preserving wisdom by connecting them and creating a network or interconnect relationship.

That's important. The graph represents the thought process and comprehensive taxonomy of a specific domain of expertise. That's why this is very important for people's moving forward. It's about AI system. Then think a lot and return and advice instead of just retrieving data from your database, right? So that comes to the development of this KAG.

What is KAG? KAG stands for Knowledge Augment Generations, and it's different from RAC, okay? It is enhanced language model by integrating structure knowledge graph for more accurate and insightful response, making it smarter, more structured approach than a simple RAC. KAG doesn't just retrieve. Remember, it understands that this is different.

Okay. After interviewing a lot of my clients, okay? So we also expert in a certain area of scale, I found that there are common ways of their thinking, decision-making process. The way that make them expert in their area, knowledge graph, seems to be a perfect fit. So here is the graph or state diagram if you're a computer engineering grad like me.

So it shows wisdom. The wisdom note, as you can see, it's a core, right? It's wisdom. It just isn't static. It actively guides decision and views by other elements. The output from the wisdom actually goes to decision-making in the blue, right? Wisdom isn't passive. It guides decision, helping us choose wisely, okay?

And then the decision-making analyses the situation, given in the circle in the green, and decisions aren't made in a vacuum, okay? They analyze real-world situation. That's the difference, okay? So look at the wisdom input, okay? Look at the relationship feedback from the knowledge to wisdom in gold colour. Example of that is knowledge to wisdom, like All your book smart, encyclopedia, Wikipedia, whatever you store, plus once that data get absorbed by IRM, whatever model you use up there, it need to regurgitate that and understand.

That's why it's very important that wisdom is able to synthesize the data after you ingested knowledge. You know, that's the kind of abstract, but I'll come to that later. How about talking about, okay? Okay, from inside, example of that is wisdom derive pattern from chaos, right? Some of my client has a lot of social media, their product, how do they, you know, track their product sediment from social media, right?

So it's very chaotic, and from X, tweet, right? So from that, you can see some pattern of their competitor versus current what my product is. That is, like, an example of that, and I will go to that later. Okay, when all these connected nodes matter together, why do they matter?

All the nodes relate to one another to ever enriching wisdom storing system. Okay, this talk is about storing wisdom, right? So knowledge tells you what it is, right? And experience tells you what worked before. Inside, invent what to try next. Like a pizza, knowledge is recipe. Experience is knowing your oven burnt crust.

Inside is like, hey, it is adding, you know, honey to the crust, you caramelize perfectly, right? So the most important part of the knowledge graph is feedback loop, okay? Feedback isn't one-way street. It learns from itself. Look at the feedback from the, going back to all the nodes, from insight to wisdom, okay?

Situation informs future wisdom. Experience, deepen it. Insight, sharpen it. Like a tree growing roots. The more it fed, the stronger it gets. Now, I want to ask you a question in general. Where do you see this circle in your life? Maybe a tough decision that, you know, taught you something?

So one practical application for leadership is wisdom. Avoid knee-jack reaction by learning from feedback. As for personal growth, ever notice how past mistakes make you wiser? That's the loop in the action. All this. So, the takeaway from the slide in this is, wisdom isn't a trophy you earn. It is a muscle you exercise.

The more you feed knowledge, experience, insight, the more they guide you. Now, I will show you how it being mapped to my current client. You know, all this is like very abstract, right? So, how I, one of my clients actually doing a competitive analysis. They used to have a marketing department doing that, but they want AI to do that, right?

They asked me to build a system. This is exactly what I did with the same textonomy of storing all this. So, this textonomy will be, later on, I talk about how multi-agents are going to handle all that. Here is one of the chatbots that I build for my client to do, you know, not just some, we, not just some chatbot, okay?

It's our wisdom graph power AI designed to turn data into strategy, right? Dominant. So, what kind of question I talk about? Talk about how do I win my competitor in this market space? That's kind of a very sophisticated question, right? So, without, uh, if you do simply just write my first speaker, talk about right, right?

So, it's not going to cut it. They're not going to able to answer that kind of question, okay? What I did is this. We retained the same textonomy, and the wisdom is then mapped, the same engine there, the wisdom engine. The wisdom engine is like an orchestration agent that does a lot of decision-making, including advising what the ARAM is able to see, based on the current situation, what to do next, right?

So, um, what I did is, uh, for the, uh, decision-making, I mapped it to a strategy generator. So, these customers are talking about a competitive analysis, right? So, um, I mapped the knowledge, in terms of knowledge, what do they have? They have market data, right? So, I mapped this experience to HP, it's one of a kind, okay, past campaign.

So, they have a lot of campaign doing a lot of marketing, and then, um, the insight is actually mapped to, uh, in, industrial insight. So, they have a database doing, storing that, and then, of course, the most important is, is the situation. The situation is, how, how am I doing, how am I product selling, right?

So, so that, that is like a situation, and then I mapped that to a competitor weakness. That means to say, if you make the ARAM aware of that, you probably get a very good answer, and then, you know, the chatbot will probably be doing the right thing, advising it.

So, from here, very high level, you know, state diagram, all that. How do I mapped it to a system that drive? Well, here comes the trick. So, anybody here heard of N8N? All right, all right, it's all good. So, so, I, I first encountered similar situation when I, my past IoT project, which is not great developed by, uh, IBM, right?

So, it's the same kind of thing. It's like, no code, but, but underneath the hood, there's a bunch of code, okay? It's on Node.js code, okay? So, uh, but, but for the, for, for proving your concept and all that, it's very, very, very flexible. And I, I, I highly recommend that, and, and, and here, here, you can take a look at the, the workflow, the the way from, I enable the implementation of these complicated state diagram with, um, uh, what I say is, there is a different community node.

One of the very powerful node is the AI agent node. Well, previously, N8N is just workflow automation tool. I'm not selling for N8N here. I'm just telling you I'm using it, uh, for pro, prototyping, uh, further down the road, maybe the client say, "Oh, I, I, I really need to, you know, go lightweight.

Maybe we will switch over to some other link chain or whatever." But, uh, we actually use this, like, I mapped the previous, uh, state diagram from the wisdom engine. I actually mapped that to our, uh, uh, wisdom agent, okay? Wisdom agent is now have the option to drive, uh, different model, like OpenAI model, Entropic model, and even on-prem model.

And then that, the key in making the state, uh, the state machine work is that my wisdom agent is now overseeing, like a supervisory agent, or all these other agents that do, uh, whatever I say on the state diagram. Um, for example, the, uh, state of, uh, going into a node of insight.

Insight agent will test to do, go to the social media, go for the sediment of all your product, and then collect that, and then pump that, that you can see that at the dot, or bottom that we, are connected to a, uh, uh, uh, uh, centralized, uh, graph. The centralized graph will be able to get updated by different agent.

Uh, insight agent will update the, here, their perspective, like, part of that graph for the, uh, as I say, for this particular, uh, uh, insight note. So, so all the unified knowledge graph will contain the textonomy that eventually just think like the marketing strategies, the way that here, they will probably, if you are doing manually, they probably would think, you know, you know, SharePoint will all this, you know, folder will start the same kind of, uh, you know, wisdom, I call it, to make decisions based on that.

So, the, the final decision is LLM. Also depend on the model that you use, uh, but I, I, I, I pretty much think that not really the way that I think the final decision come when you make the right decision from the advisor output is basically depend on all the textonomy, the graph structure.

That's very important. So, come to that, I, I want to go deep down how I implement one of the nodes, uh, just to go a bit technical on this competitive node. How do I implement that? Okay, before I do that, okay, competitive analysis, right, what, what, well, you can actually just use, right, what do you want to use a knowledge graph like Neo4j?

Well, if you ever be asked that question, tell them to seize five, uh, five reasons. The first reason is knowledge graph, you know, uh, system excel at capturing and representing complex relationship with the entities. That is covered by the first speaker, but I'll just reiterate that this lead to a deeper contextual understanding, which is crucial for competitive analysis, where this, in this case, the nuance insight can be significant, make a significant difference, okay?

You want to find the gap in your computer weakness. Now, this is very important. The second is improve accuracy. By leveraging structure data and semantics relationship, knowledge graph can provide more accurate and relevant information compared to traditional vector racks. Um, this ensure generator content is not only relevant, but also precise and reduce the noise and improve decision-making, making this.

In this case, the bot is supposed to help the guy that is marketing department make decisions. So, so you better make this work, improve accuracy. Any inaccurate data, you will be out of the contract, out of the door, right? So, it's very important, okay? You're talking about contract work like me.

I have to make the rack as accurate as possible. So, the third is scalability and flexibility. Graphic, you know, knowledge graphs are entirely scalable and can integrate to new data source and relationship. The flexibility allows the continuous improvement. As I said, if your taxonomy is correct, you will continue to improve and enrich, right?

So, so that is important. And also, rich query capability. Knowledge graphs support complex query, traverse to multiple relationship entity, provide richer and more detailed insight. This is particularly advantage for a competitive analysis when multi-facet query, like, like what the first speaker said, it is super authoritatively good in answering things that normal rack will fail.

It's a multi-hop question, okay? This is very important. And then the final one is the enhanced data integration. Knowledge graph can seamlessly integrate diverse data source, pictures, graphics, videos. However, it is, now that LRN is so powerful, we have OCR capability, we can do that. As long as you have a right structure of the graph, semi-structure, unstructure, the holistic approach ensure compressive view of the competitive landscape and enable more informed strategy decision making.

Okay. So, one of the, this is, I'm going to just very briefly go through this. It's just a, uh, uh, example of the, some of the thing, like, um, problem of vectors rack, you know, vector rack is really, really bad in answering limited numerical risk reasoning. Vector store, Excel, you know, add semantic similarity, but struggle with complex numerical calculation.

This is why, uh, for, uh, marketing analysis, uh, that I'm building the chatbot for, uh, they actually rely on number instead of just, you know, returning example like this, if you ask like, what is the Apple revenue between, you know, what's the revenue in 2022, they probably would give you a bunch of this kind of a passage, right, retrieval graph, instead of this kind of a very, very precise thing, like the answer is, you know, because about the data is already there in the structure form, the data source assume a knowledge graph name, this particular, in this particular case, Apple financial data, the query will be able, the query engine will be able to select the revenue figure from 2021 to 2022, and then do a function call, the function call will eventually give, come up with 15.23, which is exactly what the marketing guy was looking for, a very quantitative stuff, that most of the decision were based on that, because you have the evidence, not just some passage that you retrieve from the data, it's basically evidence-based decision-making is very important for this kind of complicated RAC system that, you know.

So, there's a jungle out there right now, you can use different kind of a, a, a, a thing to build your, you know, this is just a snapshot of that, you know, you can actually use Langchain plus Chroma to build your own RAC, and then you also can combine that with your knowledge graph.

Depend on your user case, okay? If this slide show that the RAC and the KAG can be viewed with many, okay? I adopt that wisdom graph in red color. Normally you will see if client is just asking for a simple RAC that perform product information query, you can just use a simple chroma DB with our agent.

And if you start to ask so complicated questions like how can I beat my competition based on my current market share? Well, this will be able, the thing that I will probably be adopting is knowledge graph here with graph DB plus cipher query and it will create ONA and also train my RAC to perform several loop of, we call multi-hop query.

And this probably will give a very good answer. So, and then it comes to the another question. When I was trying to extract my, hold, I think my time is almost up. Okay, so anyway, this is like to say, the first speaker talk about the extraction, right? There's a very simple way to extract on the right side.

It's like automated, totally automated LIM graph transformer. On the left is like manual. I probably recommend the center of hybrid model, which is like, after you use the LIM to extract your graph, you ask the interview, the expert that you're gonna build, to build the taxonomy, right? To prone the graph, we call it prone your graph.

Remove a lot of relationship, then that will be okay. And I will try to just highlight this. This is the result of benchmark that we did. Okay, anybody ask you, you know, why you want to use graph, right? Or KAG. Okay, first is accuracy. I have achieved 91% because it's really good in extract structure.

Second is flexibility, 85%. Third is reproducibility, deterministic. And then the fourth one, traceability. And finally, most important is scalability. So in conclusion, by leveraging structural nature of wisdom knowledge graph, we can significantly enhance the quantitative capability of KAG system and able more accurate and insightful response to complex query.

By using wisdom-driven system as highlighted, together we can build smarter AI system that can scale and store wisdom with the right framing, potentially support the intelligence of the initial expert that we meant to serve. So, we talked to Jesus, you know. What did he just do? Talk to Jesus.

He's in a not booth. This is my good friend. And anybody that wants to build graph, we have a good so-called LLM graph rack stack on GitHub that is sponsored by Neo4j. And out of the box, just spin up your Docker. The next thing you know, your text is going to be converted to your graph, and you can start happy pruning your graph.

Thank you. - Thank you so much. - All right. - All right. - Thank you. - Thank you. We'll be right back.