Realtime Data Connectivity for AI: Tanmai Gopal
00:00:00.000 | SRABH SINGH:
00:00:01.000 | Cool.
00:00:14.100 | I'd originally titled this talk,
00:00:17.020 | "Connect real-time data to your AI," et cetera, et cetera.
00:00:20.320 | But really, it's more existential, right?
00:00:21.960 | The AI overlords are coming for us.
00:00:24.080 | And to help them be good rulers, to help us,
00:00:27.400 | let's just give them the data they need
00:00:29.960 | so that they can do a good job, right?
00:00:33.880 | Hopefully, this talk is going to be the simplest talk
00:00:35.820 | that you heard at this conference.
00:00:37.840 | If it's not, I'll go back to using GPT-4 for coding
00:00:40.560 | instead of Sonnet.
00:00:42.280 | But the real pain that I have as I work with LLMs
00:00:48.640 | is that they can ride a flappy bird for me with my face going up
00:00:52.840 | and down in 30 seconds.
00:00:55.280 | But they can't talk to my data intelligently.
00:00:58.980 | It's really stupid.
00:01:01.220 | If I want to connect it to my calendar and I just
00:01:07.160 | want to say, how many one-on-ones did I have last week?
00:01:11.040 | What's a good number to have with my team given their roles?
00:01:15.500 | Help me stagger them better and plan it out.
00:01:19.800 | I want to connect it to my sales force and say, why is this deal
00:01:23.260 | with Acme stuck in stage three?
00:01:25.160 | And I needed to do the right thing.
00:01:26.600 | I needed to figure out the things between stage two and stage three
00:01:30.100 | in my sales pipeline and tell me why that particular deal is blocked.
00:01:34.300 | I wanted to connect to my tickets and my product data and say,
00:01:37.140 | is this ticket from an enterprise customer?
00:01:39.060 | What's the name of their project?
00:01:40.680 | Can you tell me what the status of that project is and what part of the product funnel this project is in?
00:01:48.380 | I went to Amazon today in the morning and they have this Rufus thing.
00:01:52.080 | And I was like, OK, cool.
00:01:54.580 | Is this product - I'm going to tell you what that product is in a second -
00:01:58.080 | but is this product available for one-day delivery at my Harrison Street address?
00:02:03.080 | And just doesn't - like, what is this, right?
00:02:07.780 | Like, it's right here.
00:02:09.480 | Just do it.
00:02:11.080 | And it doesn't work.
00:02:12.780 | And you all know why it doesn't work, right?
00:02:15.080 | There's like a death by a thousand cuts and it's not secure,
00:02:18.180 | and I don't want to connect my calendar and make it into a GPT.
00:02:20.380 | Who even knows what the GPT is doing with this, right?
00:02:22.480 | Like, it's scary, and it doesn't work.
00:02:28.180 | So we solved this with a pretty simple idea,
00:02:31.980 | which is that you take your live data and business logic
00:02:35.680 | and you make that available as a tool to your LLM.
00:02:40.180 | No shit.
00:02:40.980 | It's not surprising, right?
00:02:42.480 | It's easy.
00:02:43.880 | Because - and we did a bunch of things -
00:02:46.180 | that makes it work really, really well, right?
00:02:50.180 | Let's see if you have time for a quick live demo here.
00:02:54.180 | Let me see if I'm connected to the internet, which I am.
00:02:58.380 | All right.
00:02:58.880 | I'm going to zoom this up.
00:02:59.880 | All right.
00:03:05.380 | So I am a blockbuster because, obviously,
00:03:13.380 | services businesses are the most important businesses now,
00:03:15.580 | and, like, movie streaming businesses
00:03:17.580 | are going to go nowhere in the AI world that is to come.
00:03:20.180 | And so, in my blockbuster database and transactions
00:03:23.880 | and all of this stuff that I have going on,
00:03:25.880 | I want to ask my data question and say,
00:03:28.380 | "What helped me write an email to my top customer,
00:03:36.580 | thanking them for their patronage?"
00:03:39.080 | Quote, mention some recent movies they watched, right?
00:03:51.080 | Straightforward request.
00:03:52.580 | "I have all this data.
00:03:54.780 | I just needed to do the right things.
00:03:56.280 | And I needed to write an email for me."
00:03:58.280 | Right?
00:03:59.280 | And it works.
00:04:00.780 | And it works despite the fact that it's going to two or three
00:04:03.780 | different places and getting data from them.
00:04:05.280 | And it works pretty well.
00:04:07.280 | It handles all kinds of situations.
00:04:09.280 | And I'm going to talk to you about three key ideas
00:04:11.280 | about how it works,
00:04:12.280 | and hopefully that's going to be useful to you as well.
00:04:14.280 | So, the first is this idea of a unified query language.
00:04:25.980 | Whether you're talking to structured data,
00:04:27.480 | or unstructured data, or APIs,
00:04:29.780 | what if your LLM could talk to everything the same way?
00:04:33.780 | Right?
00:04:34.780 | LLMs don't know what your API is.
00:04:36.080 | If you're a little honest with yourselves,
00:04:37.280 | you probably don't know what your API does.
00:04:38.780 | But LLMs know what SQL is.
00:04:42.280 | Right?
00:04:42.880 | Because when you say select star from x where id greater than 1,
00:04:47.680 | greater than has a semantic meaning
00:04:50.080 | that is embedded in the language.
00:04:51.980 | That in your API, that URL param, who knows what it means?
00:04:54.680 | Is it greater than, is it greater than, equal to?
00:04:56.180 | Is it greater than, but actually only works with Boolean?
00:04:58.080 | I don't know, right?
00:04:58.880 | But it works with SQL.
00:05:01.780 | Because LLMs know what that SQL is, right?
00:05:03.980 | So the first part of this is, let's just make everything
00:05:06.080 | one query language and deal with that.
00:05:08.480 | The second is an object model for authorization, right?
00:05:11.480 | Which is, again, kind of blows my mind of why it's so complicated.
00:05:14.780 | Look, I don't care where the data is coming from.
00:05:16.880 | The data has a schema, right?
00:05:18.980 | It's a property of the data, and it's a property of the session.
00:05:21.380 | And then just run the rule.
00:05:22.880 | And maybe there's 100 rules, but it should just work.
00:05:25.280 | And then however it gets accessed, it's fine, right?
00:05:28.180 | I should be able to use this wherever it's used.
00:05:31.280 | However it's accessed, the same authorization should be applied.
00:05:33.880 | So that's idea number two.
00:05:35.180 | And that's kind of embedded there as well.
00:05:37.180 | The third, and this is kind of interesting,
00:05:39.280 | is to get the LLM to figure out the plan to access data by itself.
00:05:43.480 | We don't have to hard code it, and we don't have to do the work.
00:05:45.780 | And then you're like, Tanmay, listen, what are you smoking, man?
00:05:48.680 | LLMs can't even reason.
00:05:49.980 | I can't even get it to count the number of Rs in strawberry.
00:05:52.280 | What are you going to do with--
00:05:53.480 | how are you going to make me fetch all of this data
00:05:55.480 | from three or four different places in DeSambuay and whatnot?
00:05:57.580 | And we're like, you know what?
00:05:58.780 | There's a really simple fix to this problem.
00:06:00.380 | But let me ask you a live question.
00:06:01.780 | How many of you can count the number of I's in supercalifragilisticexpialidocious?
00:06:06.880 | Can you?
00:06:07.680 | You can't, right?
00:06:09.380 | You're being mean to the LLM by asking it such questions.
00:06:11.880 | Don't be mean to the LLM.
00:06:13.580 | Set it up for success.
00:06:16.280 | Ask it to write Python code to solve the problem, and it works.
00:06:21.480 | And that's it.
00:06:22.780 | So when you're asking, and when we're asking our LLMs to figure out how to retrieve data,
00:06:26.980 | we just ask it to run Python code to fetch the data that we want.
00:06:30.880 | So if the AI singularity is coming, get ready for the data singularity.
00:06:38.980 | Put everything together.
00:06:40.580 | If you're doing AI, you need access to data.
00:06:42.480 | If you're doing data and you wish that it could talk to your AI,
00:06:45.480 | if you have AI and data and you need to get to talk to each other,
00:06:48.180 | come visit us at our booth.
00:06:49.380 | Everything's in the open at hasura/pasture.ai.
00:06:52.280 | Talk to you folks soon.
00:06:53.280 | Thank you for your time.
00:06:54.280 | Bye.
00:06:55.080 | Bye.
00:06:55.080 | Bye.
00:06:55.580 | Bye.
00:06:55.580 | Bye.
00:06:56.080 | Bye.
00:06:56.680 | Bye.
00:06:56.880 | Bye.
00:06:56.880 | Bye.
00:06:57.180 | Bye.
00:06:57.780 | Bye.
00:06:58.080 | Bye.
00:06:58.580 | Bye.
00:07:00.580 | Bye.
00:07:01.580 | Bye.
00:07:02.580 | Bye.
00:07:03.080 | Bye.
00:07:03.580 | Bye.
00:07:04.080 | Bye.
00:07:04.580 | Bye.
00:07:05.080 | Bye.
00:07:05.580 | Bye.
00:07:07.580 | Bye.
00:07:09.580 | We'll see you next time.