---
title: "Deploy Your Own Knowledge Base Chatbot with DeepSeek on AWS Bedrock (working-in-progress)"
version: "v2"
status: "DEPRECATED"
description: "AutoFlow is a Graph RAG based and conversational knowledge base chat assistant built with TiDB vector store and LlamaIndex, open source and free to use."
---
# Deploy Your Own Knowledge Base Chatbot with DeepSeek on AWS Bedrock (working-in-progress)
## Overview
[AutoFlow](https://github.com/pingcap/autoflow) is a Graph RAG based and conversational knowledge base chat assistant built with TiDB vector store and LlamaIndex, open source and free to use.
+ In this lab, you will learn how to:
+ Import your own knowledge (Word/PDF/TXT/Markdown etc.) to a knowledge base.
+ Use the chatbot to search for information in the knowledge base.
+ Connect to and run queries on a TiDB Cloud Serverless cluster provisioned for you. This cluster serves as both the vector store and the graph database.
+ Deploy your own Knowledge Base Chatbot like [this](https://tidb.ai):
### Duration
+ This lab should take you approximately 1 hour to complete.
### Tasks
+ Deploy the knowledge base application with [pingcap/autoflow](https://github.com/pingcap/autoflow) repository.
+ Configure your knowledge base application.
+ Have a random chat with the AI chatbot to see if it can answer your questions using information from the knowledge base.
> [!TIP]
> Click **Start** to launch the lab environment.
> Once the environment is ready, select **Next** on the bottom toolbar to advance to **Module 1**.
> And, follow the step-by-step lab guide to connect to the hands-on environment and begin your tasks.
## Module 1: Prepare The Runtime Environment and Initialize The AutoFlow Service
### Step 1. Log into the application server VM
+ Click here - [Terminal 1](ssh://monitor1-ssh#1) to open a new terminal connected to the virtual machine assigned for your hands-on lab.
+ The banner after you logged in:
```output
ubuntu@ip-10-90-1-154:~$
```
### Step 2. Install necessary packages like Docker, Docker Compose, MySQL client
+ Uninstall the old versions of Docker and Docker Compose.
```bash
for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done
```
```output
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
...
Reading state information... Done
Package 'runc' is not installed, so not removed
0 upgraded, 0 newly installed, 0 to remove and 30 not upgraded.
```
+ Add Docker's official GPG key.
```bash
sudo apt-get update -y
sudo apt-get install ca-certificates curl -y
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
```
```output
Hit:1 http://us-west-2.ec2.archive.ubuntu.com/ubuntu noble InRelease
Get:2 http://us-west-2.ec2.archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB]
...
Get:15 http://us-west-2.ec2.archive.ubuntu.com/ubuntu noble-updates/main Translation-en [151 kB]
Fetched 30.6 MB in 5s (6133 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
ca-certificates is already the newest version (20240203).
ca-certificates set to manually installed.
curl is already the newest version (8.5.0-2ubuntu10.4).
curl set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 33 not upgraded.
```
+ Add the repository to `apt` sources.
```bash
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update -y
```
```output
Reading package lists... Done
...
Get:6 https://download.docker.com/linux/ubuntu noble/stable amd64 Packages [15.3 kB]
Fetched 64.2 kB in 1s (79.9 kB/s)
Reading package lists... Done
```
+ Install Docker packages.
```bash
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
```
```output
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
...
No VM guests are running outdated hypervisor (qemu) binaries on this host.
```
+ Verify that Docker is installed correctly by running the `hello-world` image.
```bash
sudo docker run hello-world
```
```output
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
c1ec31eb5944: Pull complete
Digest: sha256:d211f485f2dd1dee407a80973c8f129f00d54604d2c90732e8e320e5038a0348
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
...
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
```
+ Install MySQL client.
```bash
sudo apt-get install -y mysql-client-core-8.0
```
```output
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
mysql-client-core-8.0
0 upgraded, 1 newly installed, 0 to remove and 33 not upgraded.
Need to get 2794 kB of archives.
After this operation, 61.6 MB of additional disk space will be used.
Get:1 http://us-west-2.ec2.archive.ubuntu.com/ubuntu noble-updates/main amd64 mysql-client-core-8.0 amd64 8.0.39-0ubuntu0.24.04.2 [2794 kB]
Fetched 2794 kB in 0s (34.7 MB/s)
Selecting previously unselected package mysql-client-core-8.0.
(Reading database ... 68102 files and directories currently installed.)
Preparing to unpack .../mysql-client-core-8.0_8.0.39-0ubuntu0.24.04.2_amd64.deb ...
Unpacking mysql-client-core-8.0 (8.0.39-0ubuntu0.24.04.2) ...
Setting up mysql-client-core-8.0 (8.0.39-0ubuntu0.24.04.2) ...
Processing triggers for man-db (2.12.0-4build2) ...
Scanning processes...
Scanning linux images...
Running kernel seems to be up-to-date.
No services need to be restarted.
No containers need to be restarted.
No user sessions are running outdated binaries.
No VM guests are running outdated hypervisor (qemu) binaries on this host.
```
### Step 3. Clone and configure AutoFlow
+ Clone the pingcap/autoflow repository and navigate to the `autoflow` directory.
```bash
git clone https://github.com/pingcap/autoflow;
cd autoflow;
git fetch origin v0.4.1;
git checkout v0.4.1;
git switch --detach d2300a3a9f43c8d3545d057c2943c9ecbf4b6b8a;
```
```output
Cloning into 'autoflow'...
remote: Enumerating objects: 12041, done.
remote: Counting objects: 100% (538/538), done.
remote: Compressing objects: 100% (264/264), done.
remote: Total 12041 (delta 312), reused 446 (delta 268), pack-reused 11503 (from 1)
Receiving objects: 100% (12041/12041), 4.33 MiB | 22.07 MiB/s, done.
Resolving deltas: 100% (7431/7431), done.
```
+ Update the `.env` file with the follow values:
```bash
cat > .env <<'EOF'
ENVIRONMENT=production
SECRET_KEY="some_secret_key_that_is_at_least_32_characters_long"
TIDB_HOST=serverless_host
TIDB_PORT=serverless_port
TIDB_USER='serverless_username'
TIDB_PASSWORD='serverless_password'
TIDB_DATABASE=tidbai_test
TIDB_SSL=true
EMBEDDING_DIMS=1536
EMBEDDING_MAX_TOKENS=8191
EOF
cat .env
```
```output
ENVIRONMENT=production
SECRET_KEY="some_secret_key_that_is_at_least_32_characters_long"
TIDB_HOST=serverless_host
TIDB_PORT=serverless_port
TIDB_USER='serverless_username'
TIDB_PASSWORD='serverless_password'
TIDB_DATABASE=tidbai_test
TIDB_SSL=true
EMBEDDING_DIMS=1536
EMBEDDING_MAX_TOKENS=8191
```
### Step 4. Ensure your database is clean and ready for the next steps
+ Connect to the TiDB Cloud Serverless cluster with the following command.
```bash
mysql \
--comments -u 'serverless_username' -p'serverless_password' \
-h serverless_host -P serverless_port \
--ssl-mode=VERIFY_IDENTITY --ssl-ca=/etc/ssl/certs/ca-certificates.crt
```
```output
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2168969
Server version: 5.7.28-TiDB-Serverless TiDB Server (Apache License 2.0) Community Edition, MySQL 5.7 compatible
Copyright (c) 2000, 2024, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
```
+ Drop the existing database and create a new one.
```bash
DROP DATABASE IF EXISTS tidbai_test;
CREATE DATABASE tidbai_test;
EXIT;
```
```output
mysql> DROP DATABASE IF EXISTS tidbai_test;
Query OK, 0 rows affected (0.00 sec)
mysql> CREATE DATABASE tidbai_test;
Query OK, 0 rows affected (0.12 sec)
mysql> EXIT;
Bye
```
### Step 5. Create schema
+ Run data migration to create table schemas and setup initial admin users. Pay attention to the generated admin password in red lines in the output.
```bash
sudo docker compose run backend /bin/sh -c "alembic upgrade head";
sudo docker compose run backend /bin/sh -c "python bootstrap.py";
```
```output
[+] Creating 1/0
✔ Container tidb-ai-redis-1 Running 0.0s
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade -> 2fc10c21bf88, empty message
INFO [alembic.runtime.migration] Running upgrade 2fc10c21bf88 -> 94b198e20946, empty message
INFO [alembic.runtime.migration] Running upgrade 94b198e20946 -> 04d4f05116ed, empty message
INFO [alembic.runtime.migration] Running upgrade 04d4f05116ed -> 197bc8be72d1, empty message
INFO [alembic.runtime.migration] Running upgrade 197bc8be72d1 -> ac6e4d58580d, empty message
INFO [alembic.runtime.migration] Running upgrade ac6e4d58580d -> a8c79553c9f6, empty message
INFO [alembic.runtime.migration] Running upgrade a8c79553c9f6 -> bd17a4ebccc5, empty message
INFO [alembic.runtime.migration] Running upgrade bd17a4ebccc5 -> e32f1e546eec, empty message
INFO [alembic.runtime.migration] Running upgrade e32f1e546eec -> 04d81be446c3, empty message
INFO [alembic.runtime.migration] Running upgrade 04d81be446c3 -> 041fbef26e3a, empty message
INFO [alembic.runtime.migration] Running upgrade 041fbef26e3a -> 10f36e8a25c4, empty message
INFO [alembic.runtime.migration] Running upgrade 10f36e8a25c4 -> 00534dc350db, empty message
INFO [alembic.runtime.migration] Running upgrade 00534dc350db -> eb0b85608c0a, empty message
INFO [alembic.runtime.migration] Running upgrade eb0b85608c0a -> dfee070b8abd, empty message
WARN[0000] Found orphan containers ([tidb-ai-backend-run-d4d35b6b5909]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
[+] Creating 1/0
✔ Container tidb-ai-redis-1 Running 0.0s
Bootstrapping the application...
/usr/local/lib/python3.11/site-packages/pydantic/_internal/_config.py:341: UserWarning: Valid config keys have changed in V2:
* 'orm_mode' has been renamed to 'from_attributes'
warnings.warn(message, UserWarning)
User 019309f5-2a8f-7e9d-b848-cc81312e598a has registered.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[IMPORTANT] Admin user created with email: admin@example.com and password:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Default chat engine created.
Bootstrapping completed.
```
### Step 6. Start the knowledge base application by running the following `docker compose` command
```bash
sudo docker compose up -d
```
```output
[+] Running 43/33
✔ backend Pulled 51.9s
✔ background Pulled 51.9s
✔ redis Pulled 6.5s
✔ frontend Pulled 36.8s
[+] Running 5/5
✔ Network tidb-ai_default Created 0.3s
✔ Container tidb-ai-redis-1 Started 35.0s
✔ Container tidb-ai-backend-1 Started 0.7s
✔ Container tidb-ai-background-1 Started 0.7s
✔ Container tidb-ai-frontend-1 Started 0.9s
```
## Module 2: Configure The Azure OpenAI
+ In this module, we will introduce how to configure the Azure OpenAI Service, Embedding Model, and Large Language Model in the Azure OpenAI service.
+ For more details, please refer to:
+ [Azure OpenAI documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference)
+ [Create and deploy an Azure OpenAI Service resource](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal)
### Step 1. Create an Azure OpenAI Service
+ In the first step, we will create an Azure OpenAI Service. The first step is to login to the [Azure portal](https://portal.azure.com/#home). If it is your first time to login, you will see the following page.
+ Click on the Azure AI services button.
+ Click on the Create button in the Azure OpenAI account section.
+ Follow the instructions to create an Azure OpenAI Service.
> **Note**:
>
> + The Azure OpenAI models are limited to the region of the Azure OpenAI Service. For example, if you create an Azure OpenAI Service in the `Southeast Asia` region, the Azure OpenAI models most likely cannot be created.
> + Please select "All networks, including the internet, can access this resource." option in the `Network` section. So that the Azure OpenAI Service can be accessed from the internet.
> + Azure OpenAI Service is free to use, please refer to [Azure OpenAI pricing](https://azure.microsoft.com/en-us/pricing/details/cognitive-services/) for more details.
+ After the Azure OpenAI Service is created, you will see the following page.
+ Once we created the Azure OpenAI Service, we can deploy an Embedding Model. Please be aware the Azure OpenAI Service like a container more or less, so you need to deploy the Embedding Model and Large Language Model in the Azure OpenAI Service.
### Step 2. Deploy an Embedding Model
+ In this step, we will deploy an Embedding Model in the Azure OpenAI Service. We need to enter to a seperated page, [Azure AI Foundry](https://ai.azure.com/resource/deployments) , to deploy the Embedding Model.
+ Click on the Deploy model then Deploy base model button.
+ Select the `text-embedding-3-small` model.
+ Click on the Confirm button.
+ Click on the Deploy button in the popup window.
+ Congratulations! You have successfully deployed an Embedding Model in the Azure OpenAI Service.
### Step 3. Deploy a Large Language Model
+ In this step, we will deploy a Large Language Model in the Azure OpenAI Service. We will deploy the `gpt-4o` model in the Azure AI Foundry, just like the Embedding Model.
+ Click on the Deployments tab on the left side of the page to go back to the **Model deployments** page.
+ Click on the Deploy model then Deploy base model button.
+ Select the `gpt-4o` model.
+ Click on the Confirm button.
+ Click on the Deploy button in the popup window.
+ Congratulations! You have successfully deployed a Large Language Model in the Azure OpenAI Service.
### Step 4. Collect the Parameters
+ Now, we need to collect the parameters of the Azure OpenAI Service, Embedding Model, and Large Language Model and use them in the AutoFlow application later.
+ Click the `text-embedding-3-small` model, and you will see the `text-embedding-3-small` model page.
+ Click the `gpt-4o` model, and you will see the `gpt-4o` model page.
+ Now, we have collected the parameters of the Azure OpenAI Service, Embedding Model, and Large Language Model. We will use these parameters in the AutoFlow application later.
+ Azure OpenAI Endpoint
+ Azure OpenAI API Key
+ Embedding API Version
+ LLM API Version
+ LLM Deployment Name
## Module 3: Configure And Populate The Knowledge Base
### Step 1. Configure the LLM
+ Access and configure the knowledge base app by visiting [http://monitor1_address:3000](http://monitor1_address:3000) in your browser, you will see the following page.
+ Login with the default admin user `admin@example.com` and the password you received in Module 1, Step 5.
+ Click on the Models then LLMs tab, then click on the New LLM button.
+ Input as the following information and then click on the Create LLM button like the image below. Please replace the placeholder with the parameters you collected in Module 2, Step 4:
+ Name: my-azure-openai-llm
+ Provider: Azure OpenAI
+ Model: gpt-4o
+ Azure OpenAI API Key: ${Azure OpenAI API Key}
+ Is Default LLM: Yes
+ Advanced Settings:
```json
{
"azure_endpoint": "${Azure OpenAI Endpoint}",
"api_version": "${LLM API Version}",
"engine": "${LLM Deployment Name}"
}
```
### Step 2. Configure the embedding model
+ Click on the Models then Embedding Models tab, then click on the New Embedding Model button.
+ Input as the following information and then click on the Create Embedding Model button like the image below. Please replace the placeholder with the parameters you collected in Module 2, Step 4:
+ Name: my-azure-openai-embedding
+ Provider: Azure OpenAI
+ Model: text-embedding-3-small
+ Azure OpenAI API Key: ${Azure OpenAI API Key}
+ Vector Dimensions: 1536
+ Advanced Settings:
```json
{
"azure_endpoint": "${Azure OpenAI Endpoint}",
"api_version": "${Embedding API Version}"
}
```
### Step 3. Populate the knowledge base
+ Click the Knowledge Base tab, then click on the New Knowledge Base button.
+ Input as the following information and then click on the Create button:
+ Name: tidb overview
+ Description: tidb overview
+ LLM: my-azure-openai-llm
+ Embedding Model: my-azure-openai-embedding
+ Knowledge Graph Index: Yes
### Step 4. Create a Data Source
+ In the Knowledge Base page, click on the Data Sources tab, then click on the Web Pages button.
+ Click on the New Item button under the Page URL section, and input as the following information then click on the Create button:
+ Page URL: https://docs.pingcap.com/tidb/stable/overview
+ Datasource Name: tidb overview
### Step 5. Observe the indexing progress
+ Click on the Index Progress tab to navigate to the indexing progress status page to check the progress of the indexing task.
+ You are nearly done! Now the app is running a background task to download the content from the URL you provided.
+ If the circle color is blue, it means the content is being downloaded.
+ If the circle color is green, it means the content has been downloaded and imported (indexed) successfully, that means you can start chatting with the AI chatbot.
### Step 6. Configure the Default Chat Engine
+ Now, we already have a knowledge base, with a LLM, and an embedding model, we can configure the chat engine to chat with the AI chatbot.
+ Click on the Chat Engines tab, then click on the default chat engine.
+ Click on the Retrieval tab, then select the Knowledge Base you just created, then click on the Save button.
### Step 7. Use the Chatbot you just built
+ Open [http://monitor1_address:3000](http://monitor1_address:3000) in your browser or click Home to start chatting with the AI chatbot! Just Ask: "What is TiDB?":
+ Have fun chatting with your own AI chatbot!
## Module 4: Real Scenario
In the real scenario, the material is usually more complex. Let us show you real cases from a customer of autoflow.
> **Disclaimer**:
>
> The following questions are based on a real customer of autoflow, but we refined it to remove the sensitive information and make it unrecognizable. Also for the material that needs to import, we made it by ourself, it is **NOT** the real material from the customer.
### Step 1. Determine the Scope of the Questions
This is a customer from fast food industry, they want to build a knowledge base for their staff to check the standard operating procedures (SOP), recipes, food storage guidelines, and other knowledge.
So there are some typical questions from the customer:
+ Can I ask customers to pay a tip?
+ How to make a Surf and Turf?
+ When should raw salmon fish be discarded?
### Step 2. Download the material
In order to answer these questions, we need to import the material into the knowledge base. This is a fake food industry material repository, you can download it from [the release page](https://github.com/Icemap/fake-food-industry-material/releases) of repository to your local machine.
You can download the zip file or the tar.gz file, and then unzip it to your local machine.
The result is a folder named `fake-food-industry-material-`, and the folder structure is as follows:
```output
.
├── recipes
│ ├── chicken_nuggets.md
│ ├── chicken_sandwich.md
│ ├── classic_hamburger.md
│ ├── crispy_fried_chicken.md
│ ├── fish_and_chips.md
│ ├── french_fries.md
│ ├── hot_dogs.md
│ ├── milkshakes.md
│ ├── onion_rings.md
│ ├── pizza.md
│ ├── soft_serve_ice_cream.md
│ └── surf_and_turf.md
├── sop.md
└── storage_guidelines
├── bread_products.md
├── dry_goods.md
├── frozen_foods.md
└── prepared_foods.md
```
### Step 3. Import the material into the knowledge base
+ Click on the Knowledge Base tab, then click on the New Knowledge Base button.
+ Input as the following information and then click on the Create button:
+ Name: Food Industry Knowledge Base
+ Knowledge Graph Index: Yes
+ Once the knowledge base is created, click on the Data Sources tab, then click on the Files button.
+ Click on the Select files... button, and select the `fake-food-industry-material-` folder you downloaded in Module 4, Step 2.
+ Fill the Data Source Name as `Food Industry Data Source`, then click on the Create button.
+ Once the data source is created, click on the Index Progress tab to check the indexing progress.
+ Wait for the indexing progress to be completed.
### Step 4. Create a Chat Engine using the Food Industry Knowledge Base
+ Click on the Chat Engines tab, then click on the New Chat Engine button.
+ Input `Food Industry Chat Engine` in the Name section, then click on the Retrieval tab, select the `Food Industry Knowledge Base` in the Knowledge Base section, then click on the Create Chat Engine button.
### Step 5. Use the chatbot to answer the questions
+ Click on the Home tab, then click on the Select Chat Engine button.
+ Then select the `Food Industry Chat Engine`.
+ Input your question we mentioned in Module 4, Step 1, and start to chat with the chatbot.
+ Question: Can I ask customers to pay a tip?
+ Question: How to make a Surf and Turf?
+ Question: When should raw salmon fish be discarded?
Now, you can handle the real questions from the customer.
## Conclusion
In this lab, you have learned how to deploy your own Knowledge Base Chatbot like [https://tidb.ai](https://tidb.ai), you can import your own knowledge to the knowledge base, use the chatbot to search for information in the knowledge base, and handle the real questions from the customer.
If you want to use this AI chatbot in your work environment, you can deploy it on your own server, you need:
+ Get the source code at [pingcap/autoflow](https://github.com/pingcap/autoflow).
+ Get your own TiDB Cloud Serverless cluster at [https://tidbcloud.com](https://tidbcloud.com).
+ Get your own [Azure OpenAI Service](https://azure.microsoft.com/en-us/products/ai-services).