2. Build locally, see [Build Docker Image](./build_image.md)
You can **choose any one** during actual use.
## Deploy With Proxy Model
In this deployment, you don't need an GPU environment.
1. Pull from the official image repository, [Eosphoros AI Docker Hub](https://hub.docker.com/u/eosphorosai)
```bash
docker pull eosphorosai/dbgpt-openai:latest
```
2. Run the Docker container
This example requires you previde a valid API key for the SiliconFlow API. You can obtain one by signing up at [SiliconFlow](https://siliconflow.cn/) and creating an API key at [API Key](https://cloud.siliconflow.cn/account/ak).
```bash
docker run -it--rm-eSILICONFLOW_API_KEY=${SILICONFLOW_API_KEY}\
Please replace `${SILICONFLOW_API_KEY}` with your own API key.
Then you can visit [http://localhost:5670](http://localhost:5670) in the browser.
## Deploy With GPU (Local Model)
In this deployment, you need an GPU environment.
Before running the Docker container, you need to install the NVIDIA Container Toolkit. For more information, please refer to the official documentation [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
In this deployment, you will use a local model instead of downloading it from the Hugging Face or ModelScope model hub. This is useful if you have already downloaded the model to your local machine or if you want to use a model from a different source.
### Step 1: Download the Model
Before running the Docker container, you need to download the model to your local machine. You can use either Hugging Face or ModelScope (recommended for users in China) to download the model.
<Tabs>
<TabItemvalue="modelscope"label="Download from ModelScope">
1. Install `git` and `git-lfs` if you haven't already:
```bash
sudo apt-get install git git-lfs
```
2. Create a `models` directory in your current working directory:
```bash
mkdir-p ./models
```
3. Use `git` to clone the model repositories into the `models` directory:
-`--ipc host`: Enables host IPC mode for better performance.
-`--gpus all`: Allows the container to use all available GPUs.
-`-v ./dbgpt-local-gpu.toml:/app/configs/dbgpt-local-gpu.toml`: Mounts the local configuration file into the container.
-`-v ./models:/app/models`: Mounts the local `models` directory into the container.
-`eosphorosai/dbgpt`: The Docker image to use.
-`dbgpt start webserver --config /app/configs/dbgpt-local-gpu.toml`: Starts the webserver with the specified configuration file.
---
### Step 4: Access the Application
Once the container is running, you can visit [http://localhost:5670](http://localhost:5670) in your browser to access the application.
---
### Step 5: Persist Data (Optional)
To ensure that your data is not lost when the container is stopped or removed, you can map the `pilot/data` and `pilot/message` directories to your local machine. These directories store application data and messages.
1. Create local directories for data persistence:
```bash
mkdir-p ./pilot/data
mkdir-p ./pilot/message
mkdir-p ./pilot/alembic_versions
```
2. Modify the `dbgpt-local-gpu.toml` configuration file to point to the correct paths:
```toml
[service.web.database]
type="sqlite"
path="/app/pilot/message/dbgpt.db"
```
3. Run the Docker container with the additional volume mounts:
This ensures that the `pilot/data` and `pilot/message` directories are persisted on your local machine.
---
### Summary of Directory Structure
After completing the steps, your directory structure should look like this:
```
.
├── dbgpt-local-gpu.toml
├── models
│ ├── Qwen2.5-Coder-0.5B-Instruct
│ └── bge-large-zh-v1.5
├── pilot
│ ├── data
│ └── message
```
This setup ensures that the models and application data are stored locally and mounted into the Docker container, allowing you to use them without losing data.
This example requires you previde a valid API key for the SiliconFlow API. You can obtain one by signing up at [SiliconFlow](https://siliconflow.cn/) and creating an API key at [API Key](https://cloud.siliconflow.cn/account/ak).
```bash
SILICONFLOW_API_KEY=${SILICONFLOW_API_KEY} docker compose up -d
```
You will see the following output if the deployment is successful.
```bash
[+] Running 3/3
✔ Network dbgptnet Created 0.0s
✔ Container db-gpt-db-1 Started 0.2s
✔ Container db-gpt-webserver-1 Started 0.2s
```
## View log
```bash
docker logs db-gpt-webserver-1 -f
```
:::info note
For more configuration content, you can view the `docker-compose.yml` file
:::
## Visit
Open the browser and visit [http://localhost:5670](http://localhost:5670)
In this example, we will show how to use the Elasticsearch as in DB-GPT RAG Storage. Using a Elasticsearch database to implement RAG can, to some extent, alleviate the uncertainty and interpretability issues brought about by Elasticsearch database retrieval.
### Install Dependencies
First, you need to install the `dbgpt elasticsearch storage` library.
In this example, we will show how to use the ClickHouse as in DB-GPT Datasource. Using a column-oriented database to implement Datasource can, to some extent, alleviate the uncertainty and interpretability issues brought about by vector database retrieval.
### Install Dependencies
First, you need to install the `dbgpt clickhouse datasource` library.
DuckDB is a high-performance analytical database system. It is designed to execute analytical SQL queries fast and efficiently, and it can also be used as an embedded analytical database.
In this example, we will show how to use DuckDB as in DB-GPT Datasource. Using DuckDB to implement Datasource can, to some extent, alleviate the uncertainty and interpretability issues brought about by vector database retrieval.
### Install Dependencies
First, you need to install the `dbgpt duckdb datasource` library.
In this example, we will show how to use the Graph RAG framework in DB-GPT. Using a graph database to implement RAG can, to some extent, alleviate the uncertainty and interpretability issues brought about by vector database retrieval.
You can refer to the python example file `DB-GPT/examples/rag/graph_rag_example.py` in the source code. This example demonstrates how to load knowledge from a document and store it in a graph store. Subsequently, it recalls knowledge relevant to your question by searching for triplets in the graph store.
### Install Dependencies
First, you need to install the `dbgpt graph_rag` library.
```bash
uv sync--all-packages\
--extra"base"\
--extra"proxy_openai"\
--extra"rag"\
--extra"storage_chromadb"\
--extra"dbgpts"\
--extra"graph_rag"
````
### Prepare Graph Database
To store the knowledge in graph, we need an graph database, [TuGraph](https://github.com/TuGraph-family/tugraph-db) is the first graph database supported by DB-GPT.
Visit github repository of TuGraph to view [Quick Start](https://tugraph-db.readthedocs.io/zh-cn/latest/3.quick-start/1.preparation.html#id5) document, follow the instructions to pull the TuGraph database docker image (latest / version >= 4.5.1) and launch it.
```
docker pull tugraph/tugraph-runtime-centos7:4.5.1
docker run -d -p 7070:7070 -p 7687:7687 -p 9090:9090 --name tugraph_demo tugraph/tugraph-runtime-centos7:latest lgraph_server -d run --enable_plugin true
```
The default port for the bolt protocol is `7687`.
> **Download Tips:**
>
> There is also a corresponding version of the TuGraph Docker image package on OSS. You can also directly download and import it.
In this example, we will show how to use the Hive as in DB-GPT Datasource. Using Hive to implement Datasource can, to some extent, alleviate the uncertainty and interpretability issues brought about by vector database retrieval.
### Install Dependencies
First, you need to install the `dbgpt hive datasource` library.
In this example, we will show how to use the Milvus as in DB-GPT RAG Storage. Using a graph database to implement RAG can, to some extent, alleviate the uncertainty and interpretability issues brought about by vector database retrieval.
### Install Dependencies
First, you need to install the `dbgpt milvus storage` library.
In this example, we will show how to use the MSSQL as in DB-GPT Datasource. Using MSSQL to implement Datasource can, to some extent, alleviate the uncertainty and interpretability issues brought about by vector database retrieval.
### Install Dependencies
First, you need to install the `dbgpt mssql datasource` library.
In this example, we will show how to use the Oceanbase Vector as in DB-GPT RAG Storage. Using a graph database to implement RAG can, to some extent, alleviate the uncertainty and interpretability issues brought about by vector database retrieval.
### Install Dependencies
First, you need to install the `dbgpt Oceanbase Vector storage` library.
Postgres is a powerful, open source object-relational database system. It is a multi-user database management system and has sophisticated features such as Multi-Version Concurrency Control (MVCC), point in time recovery, tablespaces, asynchronous replication, nested transactions (savepoints), online/hot backups, a sophisticated query planner/optimizer, and write ahead logging for fault tolerance.
In this example, we will show how to use Postgres as in DB-GPT Datasource. Using Postgres to implement Datasource can, to some extent, alleviate the uncertainty and interpretability issues brought about by vector database retrieval.
### Install Dependencies
First, you need to install the `dbgpt postgres datasource` library.
All the following operations are completed through the `dbgpt` command. To use the `dbgpt` command, you first need to install the `DB-GPT` project. You can install it through the following command
```shell
$ pip install-e".[default]"
```
It can also be used in script mode
```shell
$ python pilot/scripts/cli_scripts.py
```
## Start Model Controller
```shell
$ dbgpt start controller
```
## View log
```shell
$ docker logs db-gpt-webserver-1 -f
```
By default, `Model Server` will start on port `8000`
## Start Model Worker
:::tip
Start `glm-4-9b-chat` model Worker
:::
```shell
dbgpt start worker --model_name glm-4-9b-chat \
--model_path /app/models/glm-4-9b-chat \
--port 8001 \
--controller_addr http://127.0.0.1:8000
```
:::tip
Start `vicuna-13b-v1.5` model Worker
:::
```shell
dbgpt start worker --model_name vicuna-13b-v1.5 \
--model_path /app/models/vicuna-13b-v1.5 \
--port 8002 \
--controller_addr http://127.0.0.1:8000
```
:::info note
⚠️ Make sure to use your own model name and model path.
:::
## Start Embedding Model Worker
```shell
dbgpt start worker --model_name text2vec \
--model_path /app/models/text2vec-large-chinese \
--worker_type text2vec \
--port 8003 \
--controller_addr http://127.0.0.1:8000
```
:::info note
⚠️ Make sure to use your own model name and model path.
:::
## Start Reranking Model Worker
```shell
dbgpt start worker --worker_type text2vec \
--rerank\
--model_path /app/models/bge-reranker-base \
--model_name bge-reranker-base \
--port 8004 \
--controller_addr http://127.0.0.1:8000
```
:::info note
⚠️ Make sure to use your own model name and model path.
The model service deployed as above can be used through dbgpt_server. First modify the `.env` configuration file to change the connection model address
```shell
dbgpt start webserver --light
```
## Start Webserver
```shell
LLM_MODEL=vicuna-13b-v1.5
# The current default MODEL_SERVER address is the address of the Model Controller
MODEL_SERVER=http://127.0.0.1:8000
```
`--light` means not to start the embedded model service.
Or it can be started directly by command to formulate the model.
By default, the `dbgpt start webserver command` will start the `webserver`, `model controller`, and `model worker` through a single Python process. In the above command, port `6006` is specified.
