Extractive Q&A With Haystack and FastAPI in Python
Chapters
0:00:31 Jupiter Notebook
3:7 Create a Http Method
10:0 Query Our Data
15:55 What Is the Universe
00:00:00.000 | Okay, so we're gonna go through putting together
00:00:02.580 | our reader model and the fast API wrapper
00:00:06.220 | around all of this.
00:00:07.860 | So originally I was just gonna do the reader model
00:00:10.820 | in this video, but I realized it's actually super easy.
00:00:14.340 | So we'd almost be done already
00:00:17.940 | if we were just doing the reader model.
00:00:19.900 | So what we're gonna do, is wrap everything up in the API
00:00:23.980 | and then we'll add the reader model into that.
00:00:26.780 | So this is where we got to before.
00:00:30.300 | We have this, which is our Jupyter Notebook
00:00:33.320 | and this has everything.
00:00:35.300 | So it has our document store, the retriever,
00:00:38.640 | and we're just retrieving a few things.
00:00:41.220 | So we've got a few sort of contexts here.
00:00:44.060 | Now, obviously notebooks are fine
00:00:50.500 | for when we're putting things together and testing things,
00:00:52.860 | but we need to switch over to actual Python files.
00:00:57.900 | So we're gonna switch across to VS code over here.
00:01:02.900 | So this is our project directory on the left here.
00:01:07.420 | So we have the code and the labs in here,
00:01:10.360 | which we already put together.
00:01:11.860 | We have our data, which is meditations.
00:01:16.520 | And I also put together this requirements text file.
00:01:19.940 | So in here, these are all the libraries
00:01:21.980 | that we need to pip install.
00:01:23.460 | So we have fast API, farm haystack,
00:01:26.940 | which we already installed, and uvicorn.
00:01:29.900 | They're the two that I want to focus on.
00:01:31.860 | So fast API, we're obviously using that to build our API
00:01:34.900 | and uvicorn we're going to use to spin up the server
00:01:38.620 | that our API will be hosted on.
00:01:41.580 | So that's our requirements.txt.
00:01:47.980 | I'm gonna create a new folder in here.
00:01:52.340 | For now, I'm just gonna call it QA service.
00:01:57.100 | I mean, we can change these later on.
00:01:59.420 | I'm not really sure what to call them all.
00:02:01.660 | So we're gonna encode.
00:02:04.060 | Okay, and then in here, we'll create our first Python file.
00:02:12.220 | And this is just going to be our API.
00:02:15.620 | So we'll call it API.py.
00:02:17.620 | And now we'll start putting together our actual API.
00:02:26.800 | So let's switch across to the window.
00:02:31.800 | And the first thing we need to do is import fast API.
00:02:36.920 | So we do from fast API, import fast API.
00:02:43.840 | And then we want to initialize our API here.
00:02:52.960 | So let's call it app equals fast API.
00:02:57.960 | I just need to fix that there.
00:03:02.360 | Okay, and that initializes our API.
00:03:05.580 | And then whenever we want to create a HTTP method,
00:03:11.560 | all we need to do is this.
00:03:14.480 | So we do app.
00:03:15.520 | And then in here, we write the method
00:03:19.840 | that we'd like to add to our API.
00:03:22.240 | So the only one that I think we're gonna need
00:03:25.640 | at least at the moment is a get method.
00:03:29.080 | So we'll add that.
00:03:30.800 | And then we also need to add path to this method.
00:03:35.800 | And in this case, we're going to be querying our Q&A model.
00:03:41.000 | So we'll just create the query path.
00:03:43.600 | So it's pretty good.
00:03:47.640 | And then we do async.
00:03:49.520 | And here we have our function.
00:03:51.360 | So this is just going to be the query function.
00:03:54.760 | And we want this query function
00:03:58.720 | to at some point, accept a query.
00:04:02.000 | But for now, we'll just leave it.
00:04:03.760 | And we're gonna add that in a moment.
00:04:05.600 | Now, this will just return,
00:04:10.040 | no, hello world.
00:04:15.560 | So that we know that it's working.
00:04:18.080 | So let's spin this up.
00:04:21.960 | So I'm opening terminal in this folder at the moment.
00:04:30.200 | So at the moment, we're at the high level of this.
00:04:33.120 | So we'll go down
00:04:34.280 | a few items.
00:04:40.320 | So we have, we want to go into the code directory.
00:04:45.480 | And then in the code directory, it's QA service,
00:04:48.520 | which we made just a moment ago.
00:04:51.000 | And from inside here,
00:04:53.840 | we initialize our GuviCorn service,
00:04:57.920 | which will host our API.
00:04:59.800 | So for that, we just write GuviCorn.
00:05:02.320 | And then we want our file name, which is API.
00:05:07.280 | And then we want whatever we've called
00:05:09.960 | our fast API instance in our code,
00:05:12.960 | which is app up here.
00:05:15.520 | And then we'll use the reload flag as well.
00:05:18.360 | So this means that whenever we edit our code,
00:05:20.720 | we don't need to re-initialize the API instance
00:05:24.680 | in order for changes to load.
00:05:26.600 | Okay.
00:05:28.680 | So now we can see that here,
00:05:32.960 | we have our instance.
00:05:35.840 | So I'm gonna copy that and open it in my browser.
00:05:39.120 | So over here, enter that.
00:05:44.160 | Enter that.
00:05:45.000 | And we see this detail not found.
00:05:49.560 | So that's because we just went straight to the index.
00:05:53.120 | So let me just have the query endpoint
00:05:57.480 | and there we get hello world.
00:05:58.880 | So this is the endpoint that we're gonna use
00:06:01.640 | for our fast API.
00:06:02.800 | So we know that's working.
00:06:06.800 | And now let's add in the rest of our code
00:06:11.400 | so that we can begin querying our QA stack.
00:06:15.960 | So if we just check what we did in our Jupyter notebook,
00:06:19.080 | all we actually need to do
00:06:21.280 | is basically copy loads of this across.
00:06:23.760 | So let me just create a new cell here.
00:06:28.280 | And I'm gonna take all of this
00:06:30.480 | and we're just gonna copy it across.
00:06:34.160 | (keyboard clicking)
00:06:37.160 | And honestly, this should really be all we need.
00:06:48.440 | Okay, so we'll copy this and take it into our code here.
00:06:53.440 | So before we initialize our API,
00:06:56.560 | we're going to initialize the rest of our code.
00:07:02.800 | So we have our document store,
00:07:06.640 | we initialize that, it accesses the Aurelius index.
00:07:12.200 | And then we have our retriever here.
00:07:15.720 | Do we, so import them as well.
00:07:19.640 | And then as well as the retriever,
00:07:24.080 | we also have the reader model, which is the next step.
00:07:32.160 | So first, obviously we need to import that
00:07:35.520 | and we do from Haystack reader farm, import farm reader.
00:07:40.520 | And then when we put all of these together,
00:07:49.280 | we're also going to use something called a pipeline.
00:07:51.680 | And to use that, we just want to put from Haystack pipeline,
00:07:55.440 | import extractive QA pipeline.
00:08:02.080 | So the reason it's extractive QA
00:08:05.960 | is because it's question answering
00:08:08.440 | and we're extracting the answer from the text
00:08:10.680 | rather than generating it with model.
00:08:12.560 | So that's why we're using the extractive QA pipeline.
00:08:18.040 | Now, that should be everything that we need to import.
00:08:23.040 | And then all we need to do here
00:08:24.560 | is we initialize our reader model,
00:08:29.640 | which is farm reader.
00:08:32.160 | And then in here,
00:08:35.880 | we want to include our model name or path.
00:08:40.720 | And that's going to be equal to deep set.
00:08:46.520 | Let me do this on another line.
00:08:50.960 | So deep set, BERT base case squad two.
00:08:59.240 | And this is a, I don't know if we covered this already.
00:09:04.160 | I don't think we did.
00:09:05.200 | So this is a pretty standard Q&A model
00:09:10.040 | that I think a lot of people use.
00:09:13.040 | So it's pretty safe one to start with.
00:09:17.320 | And then after we've initialized
00:09:19.520 | these three parts of our stack,
00:09:21.960 | we need to initialize our pipeline object.
00:09:24.800 | So for that, we do pipeline.
00:09:28.440 | And then we're taking our extractive QA pipeline.
00:09:31.320 | And then we have our reader,
00:09:35.200 | which is going to be equal to our reader.
00:09:37.800 | And we also need to pass the retriever,
00:09:40.040 | which is equal to our retriever.
00:09:41.640 | So that should be everything that we need to do
00:09:47.880 | in order to initialize our model.
00:09:53.240 | So now what we want to do is
00:09:58.040 | we need to query our data.
00:10:02.600 | So we do pipeline.run.
00:10:06.760 | And then in here, I'm just going to put like a random query.
00:10:12.360 | So what was the one we used before?
00:10:15.320 | It was, what did your grandfather do?
00:10:21.280 | Grandfather teach you.
00:10:24.640 | Okay.
00:10:27.840 | And just pass that as the query.
00:10:33.840 | And then this will return,
00:10:36.720 | this is going to return a dictionary.
00:10:39.800 | So then we can just return it straight away.
00:10:42.200 | Now, let's see if our API is still running.
00:10:47.200 | I think it should be.
00:10:50.400 | (keyboard clicking)
00:10:52.640 | Okay.
00:10:53.480 | Yep, so it's just starting up again now.
00:10:59.240 | Okay, looks good.
00:11:03.400 | So I think that's good.
00:11:05.840 | And let's try and open it up.
00:11:08.400 | Over here.
00:11:12.360 | We should get loads of text.
00:11:16.440 | Yeah.
00:11:17.840 | So that's cool.
00:11:18.680 | So now we're getting the query.
00:11:20.440 | What did your grandfather teach you?
00:11:22.440 | And this is pretty hard to see what you see.
00:11:25.640 | We get this list.
00:11:26.760 | And then this goes all the way through here
00:11:29.760 | and we get all of these different answers.
00:11:31.880 | So the first, like the top rated answer is
00:11:35.880 | what did your grandfather teach you?
00:11:38.400 | And it's act and speak.
00:11:40.160 | Okay.
00:11:42.760 | In sleep scene, act and speak.
00:11:45.080 | Toilet children from their parents.
00:11:47.320 | Okay, well, it is interesting.
00:11:50.400 | Okay.
00:11:55.760 | So it's not amazing.
00:12:03.720 | So let me try, instead of using,
00:12:10.760 | instead of using the DPR retriever,
00:12:15.320 | let's try swapping it out
00:12:20.240 | for a VM25 retriever,
00:12:23.240 | which is, it's basically just an algorithmic retriever.
00:12:27.240 | And it should actually run faster as well.
00:12:33.080 | Elastic search retriever.
00:12:39.120 | And all we do is swap those around.
00:12:44.440 | And I think the only parameter we need
00:12:47.560 | is the document store.
00:12:49.200 | Yeah.
00:12:51.240 | So, okay.
00:12:55.000 | Let's let that start up again.
00:12:57.560 | Okay, just waiting.
00:13:01.800 | Let's try this one.
00:13:05.240 | See how it compares.
00:13:06.640 | And then we'll go with one of these, I think.
00:13:10.400 | So let's go.
00:13:14.000 | (mouse clicking)
00:13:16.760 | And just refresh the page again.
00:13:21.560 | Okay.
00:13:23.880 | Yeah, this is a lot better.
00:13:28.200 | Which is weird.
00:13:30.400 | I kind of thought the other one would do better.
00:13:32.160 | But that's fine.
00:13:34.120 | Maybe we need to fine tune it a bit more than what we have.
00:13:38.040 | So what we have here is,
00:13:42.280 | what did your grandfather teach you?
00:13:43.840 | And we get good morals and the government of my temper,
00:13:46.840 | which is, I think pretty exactly what I wanted it to return.
00:13:50.960 | So that's good.
00:13:53.000 | I see here there's like a new line character as well,
00:13:58.120 | which I think we need to format the text
00:14:00.840 | before we put it in to remove those.
00:14:03.440 | But that's cool.
00:14:05.320 | It's pretty cool.
00:14:11.840 | So that's really good.
00:14:14.000 | It's working, but we're not actually making our own queries.
00:14:17.000 | So we need to add that.
00:14:20.240 | And to do that, all we need to do is add Q in here.
00:14:25.280 | And this is our query string.
00:14:28.400 | And in here, we just make that equal to query.
00:14:33.920 | And now if we do that,
00:14:39.520 | rather than opening it in the browser,
00:14:41.360 | I'm going to open Insomnia here.
00:14:44.280 | So Insomnia, what you can see now is a,
00:14:49.600 | it's just a HTTP client.
00:14:51.720 | So it allows us to send requests really easily
00:14:55.280 | and just format everything in a nice way.
00:14:57.920 | I should have opened this before.
00:14:58.920 | I'm not sure why I didn't.
00:15:00.200 | I kind of just forgot about it.
00:15:02.280 | So localhost 8000.
00:15:06.600 | So localhost 8000, we're going to the query endpoint.
00:15:11.600 | And then in our query, we have the Q parameter.
00:15:17.720 | And in here, I want to say,
00:15:20.680 | what did your grandfather teach you?
00:15:25.680 | Okay, and then we send this
00:15:30.480 | and we should get a response on the right.
00:15:33.200 | Cool.
00:15:35.480 | And this is a lot better.
00:15:36.320 | So it's good.
00:15:38.480 | And let's try something else.
00:15:40.120 | So I think I've read a lot of the time,
00:15:50.440 | he just like talks about the universe.
00:15:53.800 | So what is the universe?
00:15:57.480 | And send that.
00:16:01.520 | Takes a while to run at the moment.
00:16:06.520 | Cool.
00:16:11.200 | So we'll get a query and then we get answers.
00:16:14.080 | And it's a well-arranged universe.
00:16:17.920 | Universe loves to make whatever it is about to be.
00:16:22.440 | Interesting.
00:16:26.040 | That which knows beginning and end
00:16:30.040 | and knows the reason,
00:16:32.040 | which pervades all substance and through all time
00:16:34.720 | by fixed periods, revolutions and minuses the universe.
00:16:38.040 | Yeah, it's pretty deep.
00:16:42.000 | Yeah, that was cool.
00:16:47.360 | So I think that's everything.
00:16:52.360 | I don't want to include anything else.
00:16:56.280 | So that's pretty cool.
00:16:58.320 | So now if we have a look at this again,
00:17:02.080 | we've just done like this entire section here and the API.
00:17:08.240 | I mean, we'll probably make changes in the future,
00:17:13.400 | but that's kind of all this done.
00:17:15.040 | So you can cross that off.
00:17:18.040 | And then the next bit is going to be
00:17:20.560 | probably the most difficult bit, at least for me,
00:17:23.560 | which is going to be the front end in Angular.
00:17:27.000 | So it should be pretty interesting.
00:17:30.160 | So let's see how that goes.
00:17:33.840 | But for now, I think that's it.
00:17:35.160 | So I'll see you again in the next one.