How I Built a Custom GPT-Based Chatbot in Under 10 Minutes with LlamaIndex

Not long ago, I read an article from Jerry Liu that introduced LlamaIndex, an interface that utilizes GPT to synthesize responses to queries using the information provided by the user.

It immediately got my attention, and I knew I had to try it out for myself.

After all, many of the large language models (LLMs) that have taken the world by storm have limited use cases since they aren’t trained with the data available to their users.

Thus, there is still a demand for customizable LLMs that are tailored to the needs of their users. Simply put, businesses need LLMs that perform tasks like text summarization, question and answering (Q&A), and text generation with their own information.

To see if LlamaIndex has the potential to meet this demand, I’ve played around with its features and was genuinely surprised at what I was able to do. I even ended up building a Q&A chatbot in just around 10 minutes!

Here, I walk you through my journey.

What Does LlamaIndex Do?

LlamaIndex generates responses to queries by connecting LLMs to the information provided by users.

As detailed in the documentation, the usage of LlamaIndex entails the following steps:

1. Load in the documents
2. Parse the documents into nodes (optional)
3. Construct the index
4. Build Indices on top of the constructed indices (optional)
5. Query the index

Essentially, LlamaIndex loads your data into a document object and then converts it into an index. When the index is provided a query, it passes the query to a GPT prompt to synthesize a response. By default, this is done with OpenAI’s text-davinci-003 model.

While this process sounds pretty convoluted, it can be executed with very little code, as you are about to find out.

Setup

To test the versatility of LlamaIndex, I ended up building 3 different chatbots, with each bot being constructed with a different data source. For the sake of brevity, the previously mentioned optional steps (i.e., steps 2 and 4) will be omitted.

First, let’s take care of the prerequisites.

LlamaIndex and OpenAI can be installed from pip using the following commands:

pip install llama-index
pip install openai

Users will also need an API key from OpenAI:

import os

os.environ['OPENAI_API_KEY'] = 'API_KEY'

The project will also need the following imports:

Loading the data

Data can be loaded either manually or through a data loader. For this case, I have loaded 3 types of data:

• a local .txt file in which I write about my favorite fruits (accessible in the GitHub repository)
• a Wikipedia page on apples
• a YouTube video showing a recipe for a vanilla cake

The first index will be created with the local .txt file, which is located in a folder named data. This data will be loaded manually.

The second index will be created using data from the Wikipedia page on apples. It can be loaded with one of LlamaIndex’s data loaders.

The third index will be constructed with the YouTube video showing how to bake a vanilla cake. This data will also be loaded with a data loader.

Construct the Indices

With all the data loaded into document objects, we can construct the index for each chatbot.

Each index can be constructed from the document object with a one-liner.

Surprised? From loading the data to creating the index, the usage of LlamaIndex requires just a few lines of code!

Query the Index

The constructed indices can now generate responses for any given query. Once again, this step can be completed with a one-liner.

1. Querying the index build with the .txt file

In case you were wondering, this is the right answer.

2. Querying the index built with the Wikipedia page (topic: apples)

3. Querying the index build with the YouTube video (topic: vanilla cake recipe)

Finally, it’s also worth noting that the indices will only provide answers to queries when they contain the needed context.

Here’s how the same index created with the YouTube video’s data would respond to a completely irrelevant query.

Thankfully, it seems that LlamaIndex has taken measures against hallucination (i.e when a model confidently gives an answer not justified by the given data).

Deploying the Chatbots with a Web App

Finally, we can create a web app in order to share the constructed indices with end users.

To do so, we need to first save the indices using the save_to_disk method.

These indices will be used in a Streamlit app. The underlying code for the entire app is as follows:

In the app, users can select the data source that they wish to base their questions on and write their query in the provided box.

We can see how the indices perform after running the app:

streamlit run app.py

Querying the index built with the .txt file (my favorite fruits):

Querying the index built with the Wikipedia page (apples):

Querying the index built with the Youtube video (vanilla cake recipe):

Pretty cool, right? We’ve built a functioning web app in just 10 minutes!

Final Remarks

So far, I have only implemented the basic functionalities of the LlamaIndex interface. There are many areas that haven’t been explored in this project, such as customizing LLMs and using non-default settings.

For more information, I invite you to visit the documentation.

If you plan on experimenting with this tool yourself, do be wary of the costs that are incurred from the use of OpenAI’s API. This project cost me a mere $1, but that can be attributed to my working with small documents (the pricing is based on the number of tokens you use). If you get too carried away, you may end up picking up a nasty bill.

Finally, all of the source code used to create the Q&A chatbots can be accessed in this Github repository:

Thank you for reading!