--- title: "Deploy Your Own Knowledge Base Chatbot with OpenAI" version: "v2" status: "DEPRECATED" description: "AutoFlow is a Graph RAG based and conversational knowledge base chat assistant built with TiDB vector store and LlamaIndex, open source and free to use." --- # Deploy Your Own Knowledge Base Chatbot with OpenAI ## Overview [AutoFlow](https://github.com/pingcap/autoflow) is a Graph RAG based and conversational knowledge base chat assistant built with TiDB vector store and LlamaIndex, open source and free to use. + In this lab, you will learn how to: + Import your own knowledge (Word/PDF/TXT/Markdown etc.) to a knowledge base. + Use the chatbot to search for information in the knowledge base. + Connect to and run queries on a TiDB Cloud Serverless cluster provisioned for you. This cluster serves as both the vector store and the graph database. + Deploy your own Knowledge Base Chatbot like [this](https://tidb.ai): ![70%](https://lab-static.pingcap.com/quick-demo/taa-example-does-tidb-support-fk.jpeg) ### Duration + This lab should take you approximately 30 minutes to complete. ### Tasks + Deploy the knowledge base application with [pingcap/autoflow](https://github.com/pingcap/autoflow) repository. + Configure your knowledge base application. + Have a random chat with the AI ​​chatbot to see if it can answer your questions using information from the knowledge base. ### Prerequisites + You must bring your own OpenAI API key for this hands-on lab. > [!TIP] > Click **Start** to launch the lab environment. > Once the environment is ready, select **Next** on the bottom toolbar to advance to **Module 1**. > And, follow the step-by-step lab guide to connect to the hands-on environment and begin your tasks. ## Module 1: Prepare The Runtime Environment and Initialize The AutoFlow Service ### Step 1. Log into the application server VM + Click here - [Terminal 1](ssh://monitor1-ssh#1) to open a new terminal connected to the virtual machine assigned for your hands-on lab. + The banner after you logged in: ```output ubuntu@ip-10-90-1-154:~$ ``` ### Step 2. Install necessary packages like Docker, Docker Compose, MySQL client + Uninstall the old versions of Docker and Docker Compose. ```bash for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done ``` ```output Reading package lists... Done Building dependency tree... Done Reading state information... Done . . . Reading state information... Done Package 'runc' is not installed, so not removed 0 upgraded, 0 newly installed, 0 to remove and 30 not upgraded. ``` + Add Docker's official GPG key. ```bash sudo apt-get update -y sudo apt-get install ca-certificates curl -y sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc ``` ```output Hit:1 http://us-west-2.ec2.archive.ubuntu.com/ubuntu noble InRelease Get:2 http://us-west-2.ec2.archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB] . . . Get:15 http://us-west-2.ec2.archive.ubuntu.com/ubuntu noble-updates/main Translation-en [151 kB] Fetched 30.6 MB in 5s (6133 kB/s) Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done ca-certificates is already the newest version (20240203). ca-certificates set to manually installed. curl is already the newest version (8.5.0-2ubuntu10.4). curl set to manually installed. 0 upgraded, 0 newly installed, 0 to remove and 33 not upgraded. ``` + Add the repository to `apt` sources. ```bash echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update -y ``` ```output Reading package lists... Done . . . Get:6 https://download.docker.com/linux/ubuntu noble/stable amd64 Packages [15.3 kB] Fetched 64.2 kB in 1s (79.9 kB/s) Reading package lists... Done ``` + Install Docker packages. ```bash sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin ``` ```output Reading package lists... Done Building dependency tree... Done Reading state information... Done . . . No VM guests are running outdated hypervisor (qemu) binaries on this host. ``` + Verify that Docker is installed correctly by running the `hello-world` image. ```bash sudo docker run hello-world ``` ```output Unable to find image 'hello-world:latest' locally latest: Pulling from library/hello-world c1ec31eb5944: Pull complete Digest: sha256:d211f485f2dd1dee407a80973c8f129f00d54604d2c90732e8e320e5038a0348 Status: Downloaded newer image for hello-world:latest Hello from Docker! . . . To try something more ambitious, you can run an Ubuntu container with: $ docker run -it ubuntu bash Share images, automate workflows, and more with a free Docker ID: https://hub.docker.com/ For more examples and ideas, visit: https://docs.docker.com/get-started/ ``` + Install MySQL client. ```bash sudo apt-get install -y mysql-client-core-8.0 ``` ```output Reading package lists... Done Building dependency tree... Done Reading state information... Done The following NEW packages will be installed: mysql-client-core-8.0 0 upgraded, 1 newly installed, 0 to remove and 33 not upgraded. Need to get 2794 kB of archives. After this operation, 61.6 MB of additional disk space will be used. Get:1 http://us-west-2.ec2.archive.ubuntu.com/ubuntu noble-updates/main amd64 mysql-client-core-8.0 amd64 8.0.39-0ubuntu0.24.04.2 [2794 kB] Fetched 2794 kB in 0s (34.7 MB/s) Selecting previously unselected package mysql-client-core-8.0. (Reading database ... 68102 files and directories currently installed.) Preparing to unpack .../mysql-client-core-8.0_8.0.39-0ubuntu0.24.04.2_amd64.deb ... Unpacking mysql-client-core-8.0 (8.0.39-0ubuntu0.24.04.2) ... Setting up mysql-client-core-8.0 (8.0.39-0ubuntu0.24.04.2) ... Processing triggers for man-db (2.12.0-4build2) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. ``` ### Step 3. Clone and configure AutoFlow + Clone the pingcap/autoflow repository and navigate to the `autoflow` directory. ```bash git clone https://github.com/pingcap/autoflow; cd autoflow; git switch --detach adb5f5483b83c5af98868b8a5f080b3a6f4f4116; ``` ```output Cloning into 'autoflow'... remote: Enumerating objects: 12041, done. remote: Counting objects: 100% (538/538), done. remote: Compressing objects: 100% (264/264), done. remote: Total 12041 (delta 312), reused 446 (delta 268), pack-reused 11503 (from 1) Receiving objects: 100% (12041/12041), 4.33 MiB | 22.07 MiB/s, done. Resolving deltas: 100% (7431/7431), done. ``` + Update the `.env` file with the follow values: ```bash cat > .env <<'EOF' ENVIRONMENT=production SECRET_KEY="some_secret_key_that_is_at_least_32_characters_long" TIDB_HOST=serverless_host TIDB_PORT=serverless_port TIDB_USER='serverless_username' TIDB_PASSWORD='serverless_password' TIDB_DATABASE=tidbai_test TIDB_SSL=true EMBEDDING_DIMS=1536 EMBEDDING_MAX_TOKENS=8191 EOF cat .env ``` ```output ENVIRONMENT=production SECRET_KEY="some_secret_key_that_is_at_least_32_characters_long" TIDB_HOST=serverless_host TIDB_PORT=serverless_port TIDB_USER='serverless_username' TIDB_PASSWORD='serverless_password' TIDB_DATABASE=tidbai_test TIDB_SSL=true EMBEDDING_DIMS=1536 EMBEDDING_MAX_TOKENS=8191 ``` ### Step 4. Ensure your database is clean and ready for the next steps + Connect to the TiDB Cloud Serverless cluster with the following command. ```bash mysql \ --comments -u 'serverless_username' -p'serverless_password' \ -h serverless_host -P serverless_port \ --ssl-mode=VERIFY_IDENTITY --ssl-ca=/etc/ssl/certs/ca-certificates.crt ``` ```output mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2168969 Server version: 5.7.28-TiDB-Serverless TiDB Server (Apache License 2.0) Community Edition, MySQL 5.7 compatible Copyright (c) 2000, 2024, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> ``` + Drop the existing database and create a new one. ```bash DROP DATABASE IF EXISTS tidbai_test; CREATE DATABASE tidbai_test; EXIT; ``` ```output mysql> DROP DATABASE IF EXISTS tidbai_test; Query OK, 0 rows affected (0.00 sec) mysql> CREATE DATABASE tidbai_test; Query OK, 0 rows affected (0.12 sec) mysql> EXIT; Bye ``` ### Step 5. Create schema + Run data migration to create table schemas and setup initial admin users. Pay attention to the generated admin password in red lines in the output. ```bash sudo docker compose run backend /bin/sh -c "alembic upgrade head"; sudo docker compose run backend /bin/sh -c "python bootstrap.py"; ``` ```output [+] Creating 1/0 ✔ Container tidb-ai-redis-1 Running 0.0s INFO [alembic.runtime.migration] Context impl MySQLImpl. INFO [alembic.runtime.migration] Will assume non-transactional DDL. INFO [alembic.runtime.migration] Running upgrade -> 2fc10c21bf88, empty message INFO [alembic.runtime.migration] Running upgrade 2fc10c21bf88 -> 94b198e20946, empty message INFO [alembic.runtime.migration] Running upgrade 94b198e20946 -> 04d4f05116ed, empty message INFO [alembic.runtime.migration] Running upgrade 04d4f05116ed -> 197bc8be72d1, empty message INFO [alembic.runtime.migration] Running upgrade 197bc8be72d1 -> ac6e4d58580d, empty message INFO [alembic.runtime.migration] Running upgrade ac6e4d58580d -> a8c79553c9f6, empty message INFO [alembic.runtime.migration] Running upgrade a8c79553c9f6 -> bd17a4ebccc5, empty message INFO [alembic.runtime.migration] Running upgrade bd17a4ebccc5 -> e32f1e546eec, empty message INFO [alembic.runtime.migration] Running upgrade e32f1e546eec -> 04d81be446c3, empty message INFO [alembic.runtime.migration] Running upgrade 04d81be446c3 -> 041fbef26e3a, empty message INFO [alembic.runtime.migration] Running upgrade 041fbef26e3a -> 10f36e8a25c4, empty message INFO [alembic.runtime.migration] Running upgrade 10f36e8a25c4 -> 00534dc350db, empty message INFO [alembic.runtime.migration] Running upgrade 00534dc350db -> eb0b85608c0a, empty message INFO [alembic.runtime.migration] Running upgrade eb0b85608c0a -> dfee070b8abd, empty message WARN[0000] Found orphan containers ([tidb-ai-backend-run-d4d35b6b5909]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. [+] Creating 1/0 ✔ Container tidb-ai-redis-1 Running 0.0s Bootstrapping the application... /usr/local/lib/python3.11/site-packages/pydantic/_internal/_config.py:341: UserWarning: Valid config keys have changed in V2: * 'orm_mode' has been renamed to 'from_attributes' warnings.warn(message, UserWarning) User 019309f5-2a8f-7e9d-b848-cc81312e598a has registered. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! [IMPORTANT] Admin user created with email: admin@example.com and password: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used. Default chat engine created. Bootstrapping completed. ``` ### Step 6. Start the knowledge base application + Start the application by running the following `docker compose` command. ```bash sudo docker compose up -d ``` ```output [+] Running 43/33 ✔ backend Pulled 51.9s ✔ background Pulled 51.9s ✔ redis Pulled 6.5s ✔ frontend Pulled 36.8s [+] Running 5/5 ✔ Network tidb-ai_default Created 0.3s ✔ Container tidb-ai-redis-1 Started 35.0s ✔ Container tidb-ai-backend-1 Started 0.7s ✔ Container tidb-ai-background-1 Started 0.7s ✔ Container tidb-ai-frontend-1 Started 0.9s ``` ## Module 2: Configure The OpenAI Service ### Step 1. Access the application + Access and configure the knowledge base app by visiting [http://monitor1_address:3000](http://monitor1_address:3000) in your browser. + Login with the default admin user `admin@example.com` and the password you received in Module 1, Step 5. ![60%](https://lab-static.pingcap.com/quick-demo/taa-app-credentials.jpeg) ### Step 2. Configure the LLM + Click on the Models then LLMs tab, then click on the New LLM button. + Input as the following information and then click on the Create LLM button: + Name: my-openai-llm + Provider: OpenAI + Model: gpt-4o + OpenAI API Key: ${YOUR_OPENAI_API_KEY} + Is Default LLM: Yes ![60%](https://lab-static.pingcap.com/quick-demo/taa-default-llm.jpeg) ![60%](https://lab-static.pingcap.com/quick-demo/taa-setup-progress.jpeg) ### Step 3. Configure the embedding model + Click on the Models then Embedding Models tab, then click on the New Embedding Model button. + Input as the following information and then click on the Create Embedding Model button: + Name: my-openai-embedding + Provider: OpenAI + Model: text-embedding-3-small + OpenAI API Key: ${YOUR_OPENAI_API_KEY} + Vector Dimensions: 1536 ![60%](https://lab-static.pingcap.com/quick-demo/setup-embedding-model-start.jpeg) ![60%](https://lab-static.pingcap.com/quick-demo/setup-embedding-model-done.png) ### Step 4. Create a Knowledge Base + Click the Knowledge Base tab, then click on the New Knowledge Base button. + Input as the following information and then click on the Create button: + Name: tidb overview + Description: tidb overview + LLM: my-openai-llm + Embedding Model: my-openai-embedding + Knowledge Graph Index: Yes ### Step 5. Create a Data Source + In the Knowledge Base page, click on the Data Sources tab, then click on the Web Pages button. + Click on the New Item button under the Page URL section, and input as the following information then click on the Create button: + Page URL: https://docs.pingcap.com/tidb/stable/overview + Datasource Name: tidb overview ![60%](https://lab-static.pingcap.com/quick-demo/setup-datasource-start.jpeg) ![60%](https://lab-static.pingcap.com/quick-demo/setup-datasource-done.jpeg) ### Step 6. Observe the indexing progress + Click on the Index Progress tab to navigate to the indexing progress status page to check the progress of the indexing task. + You are nearly done! Now the app is running a background task to download the content from the URL you provided. + If the circle color is blue, it means the content is being downloaded. + If the circle color is green, it means the content has been downloaded and imported (indexed) successfully, that means you can start chatting with the AI chatbot. ![60%](https://lab-static.pingcap.com/quick-demo/build-index-done.jpeg) ### Step 7. Configure the Default Chat Engine + Click on the Chat Engines tab, then click on the default chat engine. + Click on the Retrieval tab, then select the Knowledge Base you just created, then click on the Save button. ### Step 8. Use the Chatbot you just built + Open [http://monitor1_address:3000](http://monitor1_address:3000) in your browser or click Home to start chatting with the AI chatbot! Just Ask: "What is TiDB?" + Have fun chatting with your own AI chatbot! ![60%](https://lab-static.pingcap.com/quick-demo/chat-start.jpeg) ![60%](https://lab-static.pingcap.com/quick-demo/chat-done.jpeg) ## Conclusion In this lab, you have learned how to deploy your own Knowledge Base Chatbot like [https://tidb.ai](https://tidb.ai), you can import your own knowledge to the knowledge base, and use the chatbot to search for information in the knowledge base. But we must remind you that the data stored in this lab environment is not guaranteed to be persistent. It will be released after the lab session ends. If you want to use this AI chatbot in your work environment, you can deploy it on your own server, you need: + Get the source code at [pingcap/autoflow](https://github.com/pingcap/autoflow). + Get your own TiDB Cloud Serverless cluster at [https://tidbcloud.com](https://tidbcloud.com). + Get your own OpenAI API key at [https://platform.openai.com](https://platform.openai.com).