Skip to content

GitLab

Commit 396700dd authored by chenzk's avatar chenzk
Browse files

v1.0

parents
Pipeline #2603 failed with stages
in 0 seconds
Hide whitespace changes
Inline Side-by-side
Showing with 2613 additions and 0 deletions
docs/docs/config-reference/vector_store/elastic_store_elasticsearchstoreconfig_15bdb6.mdx 0 → 100644
View file @ 396700dd
---
title: "Elastic Vector Config Configuration"
description: "Elasticsearch vector config."
---
import { ConfigDetail } from "@site/src/components/mdx/ConfigDetail";
<ConfigDetail config={{
"name": "ElasticsearchStoreConfig",
"description": "Elasticsearch vector config.",
"documentationUrl": null,
"parameters": [
{
"name": "user",
"type": "string",
"required": false,
"description": "The user of vector store, if not set, will use the default user."
},
{
"name": "password",
"type": "string",
"required": false,
"description": "The password of vector store, if not set, will use the default password."
},
{
"name": "uri",
"type": "string",
"required": false,
"description": "",
"defaultValue": "localhost"
},
{
"name": "port",
"type": "string",
"required": false,
"description": "",
"defaultValue": "9200"
},
{
"name": "alias",
"type": "string",
"required": false,
"description": "",
"defaultValue": "default"
},
{
"name": "index_name",
"type": "string",
"required": false,
"description": "",
"defaultValue": "index_name_test"
},
{
"name": "metadata_field",
"type": "string",
"required": false,
"description": "",
"defaultValue": "metadata"
},
{
"name": "secure",
"type": "string",
"required": false,
"description": "",
"defaultValue": ""
}
]
}} />
docs/docs/config-reference/vector_store/index.mdx 0 → 100644
View file @ 396700dd
---
title: "vector_store"
description: "vector_store Configuration"
---
# vector_store Configuration
This document provides an overview of all configuration classes in vector_store type.
import { ConfigClassTable } from '@site/src/components/mdx/ConfigClassTable';
## Configuration Classes
<ConfigClassTable classes={[
{
"name": "ChromaVectorConfig",
"description": "Chroma vector store config.",
"link": "./chroma_store_chromavectorconfig_16224f"
},
{
"name": "ElasticsearchStoreConfig",
"description": "Elasticsearch vector config.",
"link": "./elastic_store_elasticsearchstoreconfig_15bdb6"
},
{
"name": "MilvusVectorConfig",
"description": "Milvus vector config.",
"link": "./milvus_store_milvusvectorconfig_20af52"
},
{
"name": "OceanBaseConfig",
"description": "OceanBase vector store config.",
"link": "./oceanbase_store_oceanbaseconfig_220e36"
},
{
"name": "PGVectorConfig",
"description": "PG vector config.",
"link": "./pgvector_store_pgvectorconfig_3ef448"
},
{
"name": "WeaviateVectorConfig",
"description": "Weaviate vector config.",
"link": "./weaviate_store_weaviatevectorconfig_093ce3"
},
]} />
docs/docs/config-reference/vector_store/milvus_store_milvusvectorconfig_20af52.mdx 0 → 100644
View file @ 396700dd
---
title: "Milvus Config Configuration"
description: "Milvus vector config."
---
import { ConfigDetail } from "@site/src/components/mdx/ConfigDetail";
<ConfigDetail config={{
"name": "MilvusVectorConfig",
"description": "Milvus vector config.",
"documentationUrl": null,
"parameters": [
{
"name": "user",
"type": "string",
"required": false,
"description": "The user of vector store, if not set, will use the default user."
},
{
"name": "password",
"type": "string",
"required": false,
"description": "The password of vector store, if not set, will use the default password."
},
{
"name": "uri",
"type": "string",
"required": false,
"description": "The uri of milvus store, if not set, will use the default uri."
},
{
"name": "port",
"type": "string",
"required": false,
"description": "The port of milvus store, if not set, will use the default port.",
"defaultValue": "19530"
},
{
"name": "alias",
"type": "string",
"required": false,
"description": "The alias of milvus store, if not set, will use the default alias.",
"defaultValue": "default"
},
{
"name": "primary_field",
"type": "string",
"required": false,
"description": "The primary field of milvus store, if not set, will use the default primary field.",
"defaultValue": "pk_id"
},
{
"name": "text_field",
"type": "string",
"required": false,
"description": "The text field of milvus store, if not set, will use the default text field.",
"defaultValue": "content"
},
{
"name": "embedding_field",
"type": "string",
"required": false,
"description": "The embedding field of milvus store, if not set, will use the default embedding field.",
"defaultValue": "vector"
},
{
"name": "metadata_field",
"type": "string",
"required": false,
"description": "The metadata field of milvus store, if not set, will use the default metadata field.",
"defaultValue": "metadata"
},
{
"name": "secure",
"type": "string",
"required": false,
"description": "The secure of milvus store, if not set, will use the default ",
"defaultValue": ""
}
]
}} />
docs/docs/config-reference/vector_store/oceanbase_store_oceanbaseconfig_220e36.mdx 0 → 100644
View file @ 396700dd
---
title: "OceanBase Config Configuration"
description: "OceanBase vector store config."
---
import { ConfigDetail } from "@site/src/components/mdx/ConfigDetail";
<ConfigDetail config={{
"name": "OceanBaseConfig",
"description": "OceanBase vector store config.",
"documentationUrl": null,
"parameters": [
{
"name": "user",
"type": "string",
"required": false,
"description": "The user of vector store, if not set, will use the default user."
},
{
"name": "password",
"type": "string",
"required": false,
"description": "The password of vector store, if not set, will use the default password."
},
{
"name": "ob_host",
"type": "string",
"required": false,
"description": "The host of oceanbase, if not set, will use the default host."
},
{
"name": "ob_port",
"type": "integer",
"required": false,
"description": "The port of oceanbase, if not set, will use the default port."
},
{
"name": "ob_user",
"type": "string",
"required": false,
"description": "The user of oceanbase, if not set, will use the default user."
},
{
"name": "ob_password",
"type": "string",
"required": false,
"description": "The password of oceanbase, if not set, will use the default password"
},
{
"name": "ob_database",
"type": "string",
"required": false,
"description": "The database for vector tables, if not set, will use the default database."
}
]
}} />
docs/docs/config-reference/vector_store/pgvector_store_pgvectorconfig_3ef448.mdx 0 → 100644
View file @ 396700dd
---
title: "PGVector Config Configuration"
description: "PG vector config."
---
import { ConfigDetail } from "@site/src/components/mdx/ConfigDetail";
<ConfigDetail config={{
"name": "PGVectorConfig",
"description": "PG vector config.",
"documentationUrl": null,
"parameters": [
{
"name": "user",
"type": "string",
"required": false,
"description": "The user of vector store, if not set, will use the default user."
},
{
"name": "password",
"type": "string",
"required": false,
"description": "The password of vector store, if not set, will use the default password."
},
{
"name": "connection_string",
"type": "string",
"required": false,
"description": ""
}
]
}} />
docs/docs/config-reference/vector_store/weaviate_store_weaviatevectorconfig_093ce3.mdx 0 → 100644
View file @ 396700dd
---
title: "Weaviate Config Configuration"
description: "Weaviate vector config."
---
import { ConfigDetail } from "@site/src/components/mdx/ConfigDetail";
<ConfigDetail config={{
"name": "WeaviateVectorConfig",
"description": "Weaviate vector config.",
"documentationUrl": null,
"parameters": [
{
"name": "user",
"type": "string",
"required": false,
"description": "The user of vector store, if not set, will use the default user."
},
{
"name": "password",
"type": "string",
"required": false,
"description": "The password of vector store, if not set, will use the default password."
},
{
"name": "weaviate_url",
"type": "string",
"required": false,
"description": ""
},
{
"name": "persist_path",
"type": "string",
"required": false,
"description": ""
}
]
}} />
docs/docs/cookbook/agents/codeagent_develop.md 0 → 100644
View file @ 396700dd
# CodeAgent Develop
\ No newline at end of file
docs/docs/cookbook/app/data_analysis_app_develop.md 0 → 100644
View file @ 396700dd
# Data App Develop Guide
In this document, we will guide you through the process of developing a data analysis app using DB-GPT.
# Target
In this case, our goal is to build a data assistant application that includes the following capabilities:
1. Intelligent question and answer based on the documents.
2. Conduct data dialogue based on database.
3. Internet search based on tool usage.
These three capabilities can be utilized within a single conversation based on the intent recognition ability provided by DB-GPT. The data assistant will match appropriate sub-agent applications to answer questions in corresponding domains based on the user's inquiries.
:::tip
Note: This case is mainly for demonstration purposes of application building, and actual applications in production environments still need further optimization.
:::
# Prepare
Before starting to build the application, you first need to complete the installation and deployment of the project. For relevant tutorials, please refer to the [deployment documentation](../../installation/sourcecode.md).
# Sub-Data App Construction
First, we need to create three sub-intelligent applications separately, and then use the intent recognition capability provided by AppLink to integrate the intelligent applications into a unified intelligent entity and unify the dialogue interaction entrance.
## 1. Building a question answering assistant based on RAG
We use the agent module provided by DB-GPT to build a RAG-based question-answering assistant. DB-GPT has some built-in agents, such as
- Intent Recognition Expert Agent
- CodeEnginner Agent
- Report Generator Agent
- Data Scientist Agent
- Document Summarizer Agent
- ToolExpert Agent
- ...
In this case, intelligent question answering mainly relies on the domain knowledge base and document summarization agent (Summarizer), so we first need to build the domain knowledge base. The process is as follows:
1. Domain Knowledge Cleaning and Organization
2. Upload to DB-GPT Knowledge
3. Create Knowledge-Based Data App
4. Chat with KBQA
### Domain Knowledge Cleaning and Organization
The organization and processing of domain knowledge is a very important task and has a very important impact on the final effect. You need to organize and clean up the files according to your actual application. In this example, we use the default PDF for uploading. We prepare the official DB-GPT document as demonstration material.
### Create a knowledge base
On the product interface, select the knowledge base, click [Create Knowledge], and fill in the corresponding parameters. We provide multiple storage types. 1. Embedding vector 2. Knowledge graph 3. Full text. In this example, we use the Embedding solution for construction.
<p align="center">
<img src={'/img/cookbook/knowledge_base.png'} width="800" />
</p>
After filling in the corresponding parameters, click [Next] to select the document type and upload the document.
<p align="center">
<img src={'/img/cookbook/knowledge_base_upload.png'} width="800" />
</p>
Select the appropriate slicing method and wait for the document to be uploaded. At this point, our knowledge base has been built and we can proceed with the subsequent intelligent question and answer application
<p align="center">
<img src={'/img/cookbook/knowledge_base_success.png'} width="800" />
</p>
### Create a KBQA App
Select [Application Management] -> [Create Application], and select Single Agent Mode in the pop-up dialog box.
<p align="center">
<img src={'/img/cookbook/app_create_with_agent.png'} width="800" />
</p>
Click [OK], in the pop-up dialog box
1. Select the Summarizer agent
2. The prompt word is empty by default. If you need to modify it, you can customize the prompt first. For a tutorial on prompt definition, see the documentation.
3. Model strategy: Supports multiple model strategies. If there are multiple models, they can be configured according to priority.
4. Add resources: In this case, we rely on the previously created knowledge base, so select the resource type [knowledge] and the parameter is the name of the knowledge base just created.
5. Add a recommended question, [Whether it takes effect] to control the effectiveness of the recommended question.
<p align="center">
<img src={'/img/cookbook/qa_app_build_parameters.png'} width="800" />
</p>
Click [Save] to complete the creation of the smart application.
### Start Chat
<p align="center">
<img src={'/img/cookbook/qa_app_chat.png'} width="800" />
</p>
:::tip
Note: The agent application shown in this tutorial is built based on the Summarizer agent. The Summarizer agent is a built-in agent of DB-GPT. See the [source code](https://github.com/eosphoros-ai/DB-GPT/blob/main/dbgpt/agent/expand/summary_assistant_agent.py) for the relevant code implementation. In actual use, the relevant code can be further modified according to specific scenarios. Customization and optimization. Or customize the agent based on this case
:::
## Data ChatBot Assistant
In the same way, a data dialogue assistant can be built based on similar ideas. The data dialogue assistant can conduct simple data dialogue based on a database and draw corresponding charts. It mainly includes the following steps:
1. Data Preparation
2. Create Datasource
3. Create Data Chat App
4. Chat
### Data Preparation
For data preparation, please refer to the [data preparation](https://github.com/eosphoros-ai/DB-GPT/blob/main/docker/examples/dashboard/test_case_mysql_data.py) section in the document.
### Create Datasource
After preparing the data, you need to add the database to the data source for subsequent use. Select [Application Management] -> [Database] -> [Add Data Source]
<p align="center">
<img src={'/img/cookbook/datasource.png'} width="800" />
</p>
### Create Data Chat App
As shown in the figure below, select [Application Management] -> [Application] -> [Create Application], select a single agent application, fill in the corresponding parameters, and click OK.
<p align="center">
<img src={'/img/cookbook/data_app_create.png'} width="800" />
</p>
Select the corresponding parameters in turn:
- Agent: Select the `DataScientist` agent
- Prompt: The default is empty. For customization, please refer to the Prompt management tutorial.
- Model strategy: The priority strategy is selected here. You can use the `proxyllm` and `tongyi_proxyllm` models according to the priority.
- Available resources: Select the database type as the resource type, and select the database we added before as the parameter.
- Recommended questions: Default questions can be set based on data conditions.
<p align="center">
<img src={'/img/cookbook/data_app_build_parameters.png'} width="800" />
</p>
### Start Chat
Click to start the conversation and enter the corresponding questions for data Q&A.
<p align="center">
<img src={'/img/cookbook/data_app_chat.png'} width="800" />
</p>
## Search Assistant
The weather assistant needs to call the search engine to query relevant information, so the Tool call needs to be designed, and the construction process is relatively complicated. In order to simplify application creation, we have built the relevant capabilities into an AWEL workflow, which can be installed and used directly.
### AWEL workflow install
First execute the command `dbgpt app list-remote` to view all AWEL sample processes in the remote warehouse. `awel-flow-web-info-search` provides the ability to search the Internet.
```
dbgpt app list-remote
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ 存储库 ┃ 类型 ┃ 名称 ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ eosphoros/dbgpts │ operators │ awel-simple-operator │
│ eosphoros/dbgpts │ resources │ jina-web-reader │
│ eosphoros/dbgpts │ resources │ simple-calculator-example │
│ eosphoros/dbgpts │ workflow │ all-in-one-entrance │
│ eosphoros/dbgpts │ workflow │ andrewyng-translation-agent │
│ eosphoros/dbgpts │ workflow │ awel-flow-example-chat │
│ eosphoros/dbgpts │ workflow │ awel-flow-rag-chat-example │
│ eosphoros/dbgpts │ workflow │ awel-flow-rag-summary-example │
│ eosphoros/dbgpts │ workflow │ awel-flow-simple-streaming-chat │
│ eosphoros/dbgpts │ workflow │ awel-flow-web-info-search │
│ eosphoros/dbgpts │ workflow │ db-expert-assisant │
│ eosphoros/dbgpts │ workflow │ financial-report-knowledge-factory │
│ eosphoros/dbgpts │ workflow │ financial-robot-app │
│ eosphoros/dbgpts │ workflow │ rag-save-url-to-vstore │
│ eosphoros/dbgpts │ workflow │ rag-url-knowledge-example │
└──────────────────┴───────────┴────────────────────────────────────┘
```
Execute the `dbgpt app install awel-flow-web-info-search` command to install it locally.
```
dbgpt app install awel-flow-web-info-search
>
Installing collected packages: awel-flow-web-info-search
Successfully installed awel-flow-web-info-search-0.1.0
Installed dbgpts at ~/.dbgpts/packages/ae442685cde998fe51eb565a23180544/awel-flow-web-info-search.
dbgpts 'awel-flow-web-info-search' installed successfully.
```
Refresh the interface. In the AWEL workflow interface, you can see that the corresponding workflow has been installed.
<p align="center">
<img src={'/img/cookbook/awel_web_search.png'} width="800" />
</p>
Click on the AWEL workflow and we can see the content inside. Here is a brief explanation.
1. Agent Resource: The resource that the agent depends on, in this case baidu_search
2. ToolExpert: Tool expert, used to implement tool invocation.
3. Summarizer agent: used to summarize the query results.
To summarize: This AWEL workflow uses two agents, ToolExpert and Summarizer. ToolExpert relies on the built-in tool baidu_search. Summarizer further summarizes the results of the tool expert's execution and generates the final answer.
<p align="center">
<img src={'/img/cookbook/awel_web_search_tool.png'} width="800" />
</p>
### Create a search assistant
At the same time, [Create Application] -> [Task Flow Orchestration Mode]
<p align="center">
<img src={'/img/cookbook/search_app.png'} width="800" />
</p>
Select the corresponding workflow, add recommended questions, and click Save.
<p align="center">
<img src={'/img/cookbook/search_app_build.png'} width="800" />
</p>
### Chat
<p align="center">
<img src={'/img/cookbook/search_app_chat.png'} width="800" />
</p>
# Unified intelligent application construction
According to the above process, we have created intelligent applications for each sub-scenario, but in actual applications. We need to complete all questions and answers at one entrance, so we need to integrate agents from these sub-fields. Unify the interaction portal through AppLink and intent recognition capabilities.
In order to implement problem routing, a core capability is intent recognition and classification. In order to make application construction more flexible in design, we provide intent recognition and classification capabilities based on knowledge base and Agent. And supports customization based on AWEL.
### Intent knowledge base construction
To implement intent classification and route user questions to corresponding intelligent applications, we first need to define and describe the capabilities of each application. Here we build it through a knowledge base. The following is a simple intent definition document used to describe the capabilities of each intelligent application. There are four main types of information that need to be filled in
1. Intent: Intent type
2. App Code: Can be copied in the application interface.
<p align="center">
<img src={'/img/cookbook/app_code.png'} width="800" />
</p>
3. Describe: Describe the capabilities of the agent.
4. Slots: Slot information, used to represent the parameters that the agent relies on in actual question and answer, such as [time] and [location] information required in weather queries.
```
#######################
Intent:DB答疑 App Code:a41d0274-8ac4-11ef-8735-3ea07eeef889 Describe: 所有DB领域相关知识的咨询答疑，包含了日常DBA的FAQ问题数据、OceanBase(OB)的官方文档手册，操作手册、问题排查手册、日常疑难问题的知识总结、可以进行专业的DBA领域知识答疑。 只要和DB相关的不属于其他应用负责范畴的都可以使用我来回答 问题范例: 1.怎么查看OB抖动？ 2.DMS权限如何申请 3.如何确认xxxxx 类型:知识库咨询
#######################
Intent:数据对话 App Code:516963c4-8ac9-11ef-8735-3ea07eeef889 Describe: 通过SQL查询分析当前数据库(dbgpt-test:包含用户和用户销售订单数据的数据库） 类型:数据查询
#######################
Intent:天气检索助手 App Code:f93610cc-8acc-11ef-8735-3ea07eeef889 Describe: 可以进行天气查询 Slots:
位置: 要获取天气信息的具体位置
时间: 要获取的天气信息的时间，如果没有明确提到，使用当前时间
```
### Create an intent classification knowledge base
As shown in the figure below, create an intent classification knowledge base.
<p align="center">
<img src={'/img/cookbook/app_intent_knowledge.png'} width="800" />
</p>
It should be noted that the delimiter needs to be separated by our custom delimiter, that is, # in the document.
<p align="center">
<img src={'/img/cookbook/chunk_sep.png'} width="800" />
</p>
### AWEL workflow installation editor
Again, to simplify usage. We have written the corresponding AWEL workflow for intent recognition and can be installed and used directly.
```
dbgpt app install db-expert-assisant
> Installing collected packages: db-expert-assisant
Successfully installed db-expert-assisant-0.1.0
Installed dbgpts at ~/.dbgpts/packages/ae442685cde998fe51eb565a23180544/db-expert-assisant.
dbgpts 'db-expert-assisant' installed successfully.
```
Open the front-end interface. In the AWEL workflow interface, we can see db_expert_assisant. In order to facilitate our subsequent editing, we copy a process for editing. Click [Copy] in the upper right corner, customize the name and description, and complete the copy.
<p align="center">
<img src={'/img/cookbook/awel_db_expert.png'} width="800" />
</p>
We open the copied AWEL process, here we name it `db_expert_assistant_v1`, and open the workflow. We can see the following orchestration process. Similarly, the following agents are used in this workflow
1. `Intent Recognition Expert`: Intent recognition expert is specially used for intent recognition. It relies on a knowledge base resource, that is, the knowledge base resource for intent recognition we defined earlier.
2. `AppLauncher`: used to call experts in each field.
3. `Summarizer`: Summarizes the entire question and answer. If there is no routing in all scenarios, a default answer will be given based on the database knowledge base.
<p align="center">
<img src={'/img/cookbook/awel_expert_v1.png'} width="800" />
</p>
### Application creation
Create an application and select the task flow orchestration mode.
<p align="center">
<img src={'/img/cookbook/data_app_build.png'} width="800" />
</p>
Click OK, select the workflow, enter the recommended questions, and save.
<p align="center">
<img src={'/img/cookbook/data_app_awel.png'} width="800" />
</p>
### Chat
<p align="center">
<img src={'/img/cookbook/data_expert_chat.png'} width="800" />
</p>
docs/docs/cookbook/rag/graph_rag_app_develop.md 0 → 100644
View file @ 396700dd
# Graph RAG User Manual
In this example, we will show how to use the Graph RAG framework in DB-GPT. Using a graph database to implement RAG can, to some extent, alleviate the uncertainty and interpretability issues brought about by vector database retrieval.
You can refer to the python example file `DB-GPT/examples/rag/graph_rag_example.py` in the source code. This example demonstrates how to load knowledge from a document and store it in a graph store. Subsequently, it recalls knowledge relevant to your question by searching for triplets in the graph store.
### Install Dependencies
First, you need to install the `dbgpt` library.
```bash
uv sync --all-packages --frozen \
--extra "proxy_openai" \
--extra "rag" \
--extra "storage_chromadb" \
--extra "dbgpts"
--extra "graph_rag"
````
### Prepare Graph Database
To store the knowledge in graph, we need an graph database, [TuGraph](https://github.com/TuGraph-family/tugraph-db) is the first graph database supported by DB-GPT.
Visit github repository of TuGraph to view [Quick Start](https://tugraph-db.readthedocs.io/zh-cn/latest/3.quick-start/1.preparation.html#id5) document, follow the instructions to pull the TuGraph database docker image (latest / version >= 4.5.1) and launch it.
```
docker pull tugraph/tugraph-runtime-centos7:4.5.1
docker run -d -p 7070:7070 -p 7687:7687 -p 9090:9090 --name tugraph_demo tugraph/tugraph-runtime-centos7:latest lgraph_server -d run --enable_plugin true
```
The default port for the bolt protocol is `7687`.
> **Download Tips:**
>
> There is also a corresponding version of the TuGraph Docker image package on OSS. You can also directly download and import it.
>
> ```
> wget 'https://tugraph-web.oss-cn-beijing.aliyuncs.com/tugraph/tugraph-4.5.1/tugraph-runtime-centos7-4.5.1.tar' -O tugraph-runtime-centos7-4.5.1.tar
> docker load -i tugraph-runtime-centos7-4.5.1.tar
> ```
### Prepare LLM
To build a Graph RAG program, we need a LLM, here are some of the LLMs that DB-GPT supports:
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
<Tabs
defaultValue="openai"
values={[
{label: 'Open AI(API)', value: 'openai'},
{label: 'YI(API)', value: 'yi_proxy'},
{label: 'API Server(cluster)', value: 'model_service'},
]}>
<TabItem value="openai">
Then set your API key in the environment `OPENAI_API_KEY`.
```python
from dbgpt.model.proxy import OpenAILLMClient
llm_client = OpenAILLMClient()
```
</TabItem>
<TabItem value="yi_proxy">
You should have a YI account and get the API key from the YI official website.
Then set your API key in the environment variable `YI_API_KEY`.
</TabItem>
<TabItem value="model_service">
If you have deployed [DB-GPT cluster](/docs/installation/model_service/cluster) and
[API server](/docs/installation/advanced_usage/OpenAI_SDK_call)
, you can connect to the API server to get the LLM model.
The API is compatible with the OpenAI API, so you can use the OpenAILLMClient to
connect to the API server.
First you should install the `openai` library.
```bash
pip install openai
```
```python
from dbgpt.model.proxy import OpenAILLMClient
llm_client = OpenAILLMClient(api_base="http://localhost:8100/api/v1/", api_key="{your_api_key}")
```
</TabItem>
</Tabs>
### TuGraph Configuration
Set variables below in `.env` file, let DB-GPT know how to connect to TuGraph.
```
GRAPH_STORE_TYPE=TuGraph
TUGRAPH_HOST=127.0.0.1
TUGRAPH_PORT=7687
TUGRAPH_USERNAME=admin
TUGRAPH_PASSWORD=73@TuGraph
GRAPH_COMMUNITY_SUMMARY_ENABLED=True # enable the graph community summary
TRIPLET_GRAPH_ENABLED=True # enable the graph search for the triplets
DOCUMENT_GRAPH_ENABLED=True # enable the graph search for documents and chunks
KNOWLEDGE_GRAPH_CHUNK_SEARCH_TOP_SIZE=5 # the number of the searched triplets in a retrieval
KNOWLEDGE_GRAPH_EXTRACTION_BATCH_SIZE=20 # the batch size of triplet extraction from the text
COMMUNITY_SUMMARY_BATCH_SIZE=20 # the batch size of parallel community summary process
```
### Load into Knowledge Graph
When using a graph database as the underlying knowledge storage platform, it is necessary to build a knowledge graph to facilitate the archiving and retrieval of documents. DB-GPT leverages the capabilities of large language models to implement an integrated knowledge graph, while still maintaining the flexibility to freely connect to other knowledge graph systems and graph database systems.
We created a knowledge graph with graph community summaries based on `CommunitySummaryKnowledgeGraph`.
```python
from dbgpt.model.proxy.llms.chatgpt import OpenAILLMClient
from dbgpt.storage.knowledge_graph.community_summary import (
CommunitySummaryKnowledgeGraph,
CommunitySummaryKnowledgeGraphConfig,
)
llm_client = OpenAILLMClient()
model_name = "gpt-4o-mini"
def __create_community_kg_connector():
"""Create community knowledge graph connector."""
return CommunitySummaryKnowledgeGraph(
config=CommunitySummaryKnowledgeGraphConfig(
name="community_graph_rag_test",
embedding_fn=DefaultEmbeddingFactory.openai(),
llm_client=llm_client,
model_name=model_name,
graph_store_type="TuGraphGraph",
),
)
```
### Retrieve from Knowledge Graph
Then you can retrieve the knowledge from the knowledge graph, which is the same with vector store.
```python
import os
from dbgpt.configs.model_config import ROOT_PATH
from dbgpt.core import Chunk, HumanPromptTemplate, ModelMessage, ModelRequest
from dbgpt_ext.rag import ChunkParameters
from dbgpt_ext.rag.assembler import EmbeddingAssembler
from dbgpt_ext.rag.knowledge import KnowledgeFactory
from dbgpt.rag.retriever import RetrieverStrategy
async def test_community_graph_rag():
await __run_graph_rag(
knowledge_file="examples/test_files/graphrag-mini.md",
chunk_strategy="CHUNK_BY_MARKDOWN_HEADER",
knowledge_graph=__create_community_kg_connector(),
question="What's the relationship between TuGraph and DB-GPT ?",
)
async def __run_graph_rag(knowledge_file, chunk_strategy, knowledge_graph, question):
file_path = os.path.join(ROOT_PATH, knowledge_file).format()
knowledge = KnowledgeFactory.from_file_path(file_path)
try:
chunk_parameters = ChunkParameters(chunk_strategy=chunk_strategy)
# get embedding assembler
assembler = await EmbeddingAssembler.aload_from_knowledge(
knowledge=knowledge,
chunk_parameters=chunk_parameters,
index_store=knowledge_graph,
retrieve_strategy=RetrieverStrategy.GRAPH,
)
await assembler.apersist()
# get embeddings retriever
retriever = assembler.as_retriever(1)
chunks = await retriever.aretrieve_with_scores(question, score_threshold=0.3)
# chat
print(f"{await ask_chunk(chunks[0], question)}")
finally:
knowledge_graph.delete_vector_name(knowledge_graph.get_config().name)
async def ask_chunk(chunk: Chunk, question) -> str:
rag_template = (
"Based on the following [Context] {context}, "
"answer [Question] {question}."
)
template = HumanPromptTemplate.from_template(rag_template)
messages = template.format_messages(context=chunk.content, question=question)
model_messages = ModelMessage.from_base_messages(messages)
request = ModelRequest(model=model_name, messages=model_messages)
response = await llm_client.generate(request=request)
if not response.success:
code = str(response.error_code)
reason = response.text
raise Exception(f"request llm failed ({code}) {reason}")
return response.text
```
### Chat Knowledge via GraphRAG
> Note: The current test data is in Chinese.
Here we demonstrate how to achieve chat knowledge through Graph RAG on web page.
First, create a knowledge base using the `Knowledge Graph` type.
<p align="left">
<img src={'/img/chat_knowledge/graph_rag/create_knowledge_graph.png'} width="1000px"/>
</p>
Then, upload the documents ([graphrag-test.md](https://github.com/eosphoros-ai/DB-GPT/blob/main/examples/test_files/graphrag-test.md)) and process them automatically (markdown header by default).
<p align="left">
<img src={'/img/chat_knowledge/graph_rag/upload_file.png'} width="1000px"/>
</p>
After indexing, the graph data may look like this.
<p align="left">
<img src={'/img/chat_knowledge/graph_rag/graph_data.png'} width="1000px"/>
</p>
Start to chat on knowledge graph.
<p align="left">
<img src={'/img/chat_knowledge/graph_rag/graph_rag_chat.png'} width="1000px"/>
</p>
### Performance Testing
Performance testing is based on the `gpt-4o-mini` model.
#### Indexing Performance
| | DB-GPT | GraphRAG(microsoft) |
| ------------------- | --------------------- | -------------------- |
| Doc Tokens | 42631 | 42631 |
| Triplets Graph | 734 nodes, 1064 edges | 779 nodes, 967 edges |
| Doc Structure Graph | 76 nodes, 1090 edges | N/A |
| Prompt Tokens | 375768 | 744990 |
| Completion Tokens | 41797 | 227230 |
| Total Tokens | **417565** | 972220 |
| Indexing Time | **170s** | 210s |
#### Querying Performance
**Global Search**
| | DB-GPT | GraphRAG(microsoft) |
| ------ | ------ | ------------------- |
| Time | 8s | 40s |
| Tokens | 7432 | 63317 |
**Question**
```
请总结知识库的内容，并进行摘要说明。
```
- **DB-GPT Answer**
```
· 知识图谱概述: 知识图谱集中展示了与图数据库及相关处理系统的多种实体和其相互关联，涵盖了分布式内存图数据库、图计算模型及多种图处理框架和技术，强调了这些技术在图处理· 领域中的应用和相互影响。
· TuGraph DB: TuGraph DB是一种高效的图数据库，支持大数据容量和低延迟查找，具备ACID特性。其社区版适用于学习和小型项目，支持实时数据管理和多图结构。
· DB-GPT: DB-GPT是一个私有化的LLM技术数据库交互方式，支持多种高级功能，旨在推动数据3.0时代的到来。它提供了多模型管理、信息检索优化和智能体协作等功能。
· 应用领域: 知识库中提到的图技术应用于金融风控、社交网络和数据分析等多个领域，展示了图数据库的快速发展及其在各行各业中的重要性。
· 开发者生态: 知识库还强调了开发者在开源项目中的活跃表现，尤其是在Apache Flink和Spark等项目中的贡献，表明了良好的合作与开发生态。
整体而言，知识库展示了图数据库及其相关技术的广泛应用和快速进步，强调了DB-GPT和TuGraph等工具在数据智能和处理中的重要角色。
```
- **GraphRAG(microsoft) Answer**
```
知识库内容总结
概述
知识库涵盖了与图处理相关的多个社区和框架，强调了这些实体在推动技术解决方案和协作努力中的重要性。主要关注的领域包括图处理、人工智能开发和开源贡献。
关键实体
1. 图处理框架：知识库中提到的多个重要框架，如 GraphChi、X-Stream、GraphQ、PowerLyra、Gemini 和 TuGraph，均在处理大规模图数据方面发挥了重要作用。这些框架的相互关联性表明了一个促进创新的协作环境。
2. 企业贡献：一些主要企业如 Ant Group、Google、Tencent 和 Alibaba 在图处理和数据管理技术的发展中扮演了关键角色。它们的合作关系强调了在推动技术创新方面的伙伴关系的重要性。
3. 开源项目：Apache 软件基金会的多个开源项目（如 Apache Hudi、Iceberg、Pulsar 和 Kafka）展示了全球开发者的贡献，增强了数据管理和分析能力。
发展趋势
知识库还指出了动态图处理、人工智能驱动的数据库管理和自动化工具集成等新兴趋势，这些趋势对于提高现代软件开发的效率和有效性至关重要。
结论
整体而言，知识库展示了一个充满活力的技术生态系统，其中各个实体通过合作和创新共同推动图处理和相关技术的发展。这种相互联系的网络不仅促进了技术的进步，也为解决复杂的技术挑战提供了支持。
```
**Local Search**
| | DB-GPT | GraphRAG(microsoft) |
| ------ | ------ | ------------------- |
| Time | 15s | 15s |
| Tokens | 9230 | 11619 |
**Question**
```
DB-GPT社区和TuGraph社区在社区贡献，社区生态，开发者的这几个方面的联系和区别分别是什么？
```
- **DB-GPT Answer**
```
· 社区贡献:
· DB-GPT社区: 由多个开发者和公司支持，开发者积极参与代码提交、审查和问题反馈，展现出强大的社区协作能力。主要合作项目包括Langchain-Chatchat、LLaMA-Factory、FastChat等。
· TuGraph社区: 吸引了众多开发者参与，专注于提升TuGraph DB的功能和性能，与多个开源工具共同开发，形成了活跃的贡献模式。
· 社区生态:
· DB-GPT社区: 主要围绕私有化LLM技术数据库交互方式，支持多种高级功能，促进数据智能应用开发，构建了一个以AI和数据处理为核心的生态。
· TuGraph社区: 依托于图数据库领域，尤其是TuGraph DB和TuGraph Analytics，形成了以大数据处理和实时计算为核心的生态，涉及多个行业应用，如金融风控和社交网络。
· 开发者:
· DB-GPT社区: 包含多个GitHub用户，开发者主要集中在AI和数据处理领域，积极参与开源项目的开发和维护。
· TuGraph社区: 涉及来自不同国家的开发者，特别是在图数据库和大数据分析领域的专业人员，强调多领域协作与贡献。
总体来看，DB-GPT社区和TuGraph社区在贡献方式和生态领域有所不同，前者更侧重于AI和数据交互技术，后者则专注于图数据库技术及其应用。
```
- **GraphRAG(microsoft) Answer**
```
DB-GPT社区与TuGraph社区的比较
DB-GPT社区和TuGraph社区在多个方面展现了各自的特点和贡献，尤其是在社区贡献、生态系统和开发者参与等方面。以下是对这两个社区的联系与区别的详细分析。
社区贡献
DB-GPT社区
DB-GPT社区围绕DB-GPT框架展开，整合了多个组织和资源，致力于AI和数据驱动应用的开发。该社区的主要贡献者包括Hiyouga、LM-Sys和Langchain-AI等组织，这些组织通过合作推动AI模型和应用的发展。DB-GPT的开发者们积极参与知识共享和技术创新，推动了AI应用的多样化和实用性。
TuGraph社区
TuGraph社区则专注于图数据库的开发，尤其是TuGraph及其相关项目。该社区的贡献者包括Ant Group和Tsinghua University等，致力于提供高效的图数据管理和分析解决方案。TuGraph社区的开发者们通过开源项目和技术文档，促进了图数据库技术的普及和应用。
社区生态
DB-GPT社区
DB-GPT社区的生态系统是一个多元化的合作网络，涵盖了多个组织和技术平台。该社区通过整合不同的技术和数据源，支持从聊天系统到企业报告等多种应用，展现出其在AI领域的广泛适用性。DB-GPT的生态系统强调了组织间的协作与知识共享，促进了技术的快速发展。
TuGraph社区
相较之下，TuGraph社区的生态系统更为专注于图数据的管理和分析。TuGraph及其相关项目（如TuGraph DB和TuGraph Analytics）共同构成了一个完整的图技术体系，支持大规模数据的实时处理和复杂分析。该社区的生态系统强调了图数据库在金融、工业和政务服务等领域的应用，展现了其在特定行业中的深度影响。
开发者参与
DB-GPT社区
在DB-GPT社区中，开发者的参与主要体现在对AI应用的开发和优化上。社区内的开发者通过贡献代码、参与讨论和解决问题，推动了DB-GPT框架的不断完善。该社区的开发者们来自不同国家和地区，展现了全球范围内对AI技术的关注和参与。
TuGraph社区
TuGraph社区的开发者则主要集中在图数据库的构建和优化上。该社区的开发者们通过GitHub等平台积极参与项目的开发、代码审查和问题解决，推动了TuGraph技术的进步。TuGraph社区的开发者们同样来自中国及其他国家，展现了对图数据管理技术的广泛兴趣。
总结
总体而言，DB-GPT社区和TuGraph社区在社区贡献、生态系统和开发者参与等方面各具特色。DB-GPT社区更侧重于AI应用的多样性和组织间的合作，而TuGraph社区则专注于图数据的高效管理和分析。两者的共同点在于都强调了开源和社区合作的重要性，推动了各自领域的技术进步和应用发展。
```
### Retrieval Of Document Structure
In version 0.6.1 of DB-GPT, we have added a new feature:
- Retrieval of triplets with the **retrieval of document structure**
We have expanded the definition scope of 'Graph' in GraphRAG:
```
Knowledge Graph = Triplets Graph + Document Structure Graph
```
<p align="left">
<img src={'/img/chat_knowledge/graph_rag/image_graphrag_0_6_1.png'} width="1000px"/>
</p>
Thanks to the Document Structure Graph, GraphRAG now can provide references to the original text when answering:
<p align="left">
<img src={'/img/chat_knowledge/graph_rag/doc_structure_graph_demo.png'} width="1000px"/>
</p>
How?
We decompose standard format files (currently best support for Markdown files) into a directed graph based on their hierarchy and layout information, and store it in a graph database. In this graph:
- Each node represents a chunk of the file
- Each edge represents the structural relationship between different chunks in the original document
- Merge the document structure graph to the triplets graph
What is the next?
We aim to construct a more complex Graph that covers more comprehensive information to support more sophisticated retrieval algorithms in our GraphRAG.
### Similarity Search in GraphRAG:
In the latest version of DB-GPT, we have implemented a new feature:
- **Similarity search** for GraphRAG retrieval
#### How to use?
Use TuGraph 4.5.1 and above.
Set the variables below in the `.env` file to enable similarity search in DB-GPT.
```
SIMILARITY_SEARCH_ENABLED=True # enable the similarity search for entities and chunks
KNOWLEDGE_GRAPH_EMBEDDING_BATCH_SIZE=20 # the batch size of embedding from the text
KNOWLEDGE_GRAPH_SIMILARITY_SEARCH_TOP_SIZE=5 # set the topk of the vector similarity search
KNOWLEDGE_GRAPH_SIMILARITY_SEARCH_RECALL_SCORE=0.3 # set the reacall score of the vector similarity search
```
Additionally, you need to choose an embedding model in the `.env` file
```
## Openai embedding model, See dbgpt/model/parameter.py
# EMBEDDING_MODEL=proxy_openai
# proxy_openai_proxy_server_url=https://api.openai.com/v1
# proxy_openai_proxy_api_key={your-openai-sk}
# proxy_openai_proxy_backend=text-embedding-ada-002
## qwen embedding model, See dbgpt/model/parameter.py
# EMBEDDING_MODEL=proxy_tongyi
# proxy_tongyi_proxy_backend=text-embedding-v1
# proxy_tongyi_proxy_api_key={your-api-key}
## qianfan embedding model, See dbgpt/model/parameter.py
#EMBEDDING_MODEL=proxy_qianfan
#proxy_qianfan_proxy_backend=bge-large-zh
#proxy_qianfan_proxy_api_key={your-api-key}
#proxy_qianfan_proxy_api_secret={your-secret-key}
```
#### Why to use?
TuGraph now offers comprehensive vector capabilities, including vector storage, indexing, and similarity search functionalities. These features enable GraphRAG to achieve superior retrieval performance compared to traditional keyword-based approaches.
To leverage these capabilities, we've introduced an `_embedding` field in both entity and chunk objects to store embedding data, enabling similarity search to identify the most relevant results for a given query.
#### Comparison of Similarity Search Results
Given identical documents and questions in the same environment, the results of the keyword mode are as follows:
<p align="left">
<img src={'/img/chat_knowledge/graph_rag/comparison_result_for_keywords.png'} width="1000px"/>
</p>
The results of the similarity search mode are as follows:
<p align="left">
<img src={'/img/chat_knowledge/graph_rag/comparison_result_for_similarity_search.png'} width="1000px"/>
</p>
Compared to the keyword search method, the similarity search method can cover more comprehensive information. For instance, when dealing with the term 清北大学 in the keyword search mode, it is hard to extract useful keywords. However, the similarity search mode can identify similar words, enabling it to retrieve relevant information related to Tsinghua University and thus include it in the search results.
This implies that in scenarios where queries are imprecise, the similarity search approach can retrieve more pertinent information compared to keyword-based search patterns.
Furthermore, as shown in the following figure, compared to RAG, GraphRAG with similarity search can obtain more relevant information, ensuring richer answers.
<p align="left">
<img src={'/img/chat_knowledge/graph_rag/comparison_with_rag.png'} width="1000px"/>
</p>
In conclusion, enabling similarity search in GraphRAG significantly expands the scope and relevance of its responses.
### Text2GQL Search in GraphRAG:
In the latest version of DB-GPT, we have implemented a new feature:
- **Text2GQL search** for GraphRAG retrieval
#### How to use?
Set the variables below in the `.env` file to enable text2gql search in DB-GPT.
```
TEXT2GQL_SEARCH_ENABLED=True # enable the text2gql search for entities and relations.
```
#### Why to use?
Keywords or vectors based retrieval will generate large multihop subgraph for LLM to summarize information, but this method is costive when questions asked by users can be simply expressed by a single graph query. Text2GQL search can effectively reduce the cost of graph search and increase the accuracy of the retrieved subgraph under above situation.
In the future, we hope to further improve the ability of Text2GQL translation to compete with keywords or vectors based retrieval under complicated questions with both prompt based method and finetune based method.
#### Comparison of Text2GQL Search Results
Given identical documents and questions in the same environment, the results of the keyword mode are as follows:
<p align="left">
<img src={'/img/chat_knowledge/graph_rag/comparison_result_for_keywords_search.png'} width="1000px"/>
</p>
The results of the text2gql search mode are as follows:
<p align="left">
<img src={'/img/chat_knowledge/graph_rag/comparison_result_for_text2gql_search.png'} width="1000px"/>
</p>
Compared to the keyword search method, the text2gql search method can generate an accurate graph query laguage to query the entity of DB-GPT in knowledge graph, which is
```cypher
MATCH (n) WHERE n.id = 'DB-GPT' RETURN n LIMIT 10
```
This implies that in scenarios where questions can be expressed by a single graph query, the text2gql search approach can retrieve more accurate information with lower cost.
In conclusion, enabling text2gql search in GraphRAG significantly increase the accuracy and lower the cost when questions are concise and clear.
docs/docs/cookbook/rag/keyword_rag_app_develop.md 0 → 100644
View file @ 396700dd
# Keyword Search RAG User Manual
In this example, we will show how to use the Full Text Search RAG framework in DB-GPT. Using traditional full-text search to implement RAG can, to some extent, alleviate the uncertainty and interpretability issues brought about by vector database retrieval.
You can refer to the python example file `DB-GPT/examples/rag/keyword_rag_example.py` in the source code. This example demonstrates how to load knowledge from a document and persist it in a full text store. Subsequently, it recalls knowledge relevant to your question by searching for keywords in the full text store.
### The Constraints of Vector Retrieve
Vector Retrieve offers clear advantages, the technology does have some constraints:
- Computationally Intensive - Generating vectors for entire corpora of documents and querying based on vector similarity requires significantly more processing power than keyword indexing and matching. Latency can be an issue if systems are not properly optimized.
- Requires Massive Training Data - The semantic connections made by models like BERT rely on being trained on massive, diverse datasets over long periods. This data may not be readily available for specialized corpora, limiting the quality of vectors.
- Less Effective for Precise Keyword Queries - Vector search adds little benefit when queries contain clear, precise keywords and intent. Searching for "apple fruit" would likely return poorer results than just "apple" because the vector focuses on overall meaning more than keywords.
### How to choose Between Vector Retrieve and Keyword Retrieve ?
When is vector search preferable over keyword search, and vice versa? Here are some best practices on when to use each:
When to Use Vector Search
Early stage research when query intent is vague or broad
Need to grasp concepts and subject matter more than keywords
Exploring a topic with loose information needs
User search queries are more conversational
The semantic capabilities of vector search allow it to shine for these use cases. It can point users in the right direction even with limited keywords or understanding of a topic.
When to Use Keyword Search:
- Looking for something ultra-specific and already understand the topic
- Research is narrowly focused with clear objectives
- Queries contain unique proper nouns like brand names
- Needs require fast results more than exhaustive relevancy
For precise or time-sensitive queries, keyword search will target the exact terms efficiently. Vector search may meander with unnecessary semantic expansion.
The search method should align with the user's intent and specificity needs. Vector search for exploration, keyword search for precision. With both available, users get the best of both worlds.
### Install Dependencies
First, you need to install the `dbgpt` library.
```bash
pip install "dbgpt[rag]>=0.5.8"
````
### Prepare Full Text Search Engine
`Elasticsearch` is the distributed search and analytics engine at the heart of the Elastic Stack. Logstash and Beats facilitate collecting, aggregating, and enriching your data and storing it in Elasticsearch. Kibana enables you to interactively explore, visualize, and share insights into your data and manage and monitor the stack. Elasticsearch is where the indexing, search, and analysis magic happens.
refer https://www.elastic.co/guide/en/elasticsearch/reference/current/elasticsearch-intro.html
Install Elasticsearch refer https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html
### Keyword Search Configuration
Set variables below in `.env` file, let DB-GPT know how to connect to Full Text Search Engine Storage.
```
ELASTICSEARCH_URL=localhost
ELASTICSEARCH_PORT=9200
ELASTICSEARCH_USERNAME=elastic
ELASTICSEARCH_PASSWORD=dbgpt
```
### Load into Full Text Search Engine
When using a `Elaticsearch` full text engine as the underlying knowledge storage platform, it is necessary to build document inverted index to facilitate the archiving and retrieval of documents.
The following code demonstrates how to create a connection to the Elasticsearch search engine.
```python
from dbgpt_ext.storage.full_text.elasticsearch import ElasticDocumentConfig, \
ElasticDocumentStore
def _create_es_connector():
"""Create es connector."""
config = ElasticDocumentConfig(
name="keyword_rag_test",
uri="localhost",
port="9200",
user="elastic",
password="dbgpt",
)
return ElasticDocumentStore(config)
```
### Keyword Retrieve from Full Text Search Engine
Keyword Retrieve is a simple and efficient way to retrieve relevant information from a large number of documents. It is based on the full-text search engine Elasticsearch. The user can input a query and retrieve the most relevant documents based on the query.
```python
import os
from dbgpt.configs.model_config import ROOT_PATH
from dbgpt_ext.rag import ChunkParameters
from dbgpt_ext.rag.assembler import EmbeddingAssembler
from dbgpt_ext.rag.knowledge import KnowledgeFactory
async def main():
file_path = os.path.join(ROOT_PATH, "docs/docs/awel/awel.md")
knowledge = KnowledgeFactory.from_file_path(file_path)
keyword_store = _create_es_connector()
chunk_parameters = ChunkParameters(chunk_strategy="CHUNK_BY_SIZE")
# get embedding assembler
assembler = EmbeddingAssembler.load_from_knowledge(
knowledge=knowledge,
chunk_parameters=chunk_parameters,
index_store=keyword_store,
)
assembler.persist()
# get embeddings retriever
retriever = assembler.as_retriever(3)
chunks = await retriever.aretrieve_with_scores("what is awel talk about", 0.3)
print(f"keyword rag example results:{chunks}")
```
### Chat Knowledge via Keyword RAG
Here we demonstrate how to achieve chat knowledge through Keyword RAG on web page.
First, create a knowledge base using the `Full Text` type. Upload the knowledge documents and wait for the slicing to complete.
<p align="left">
<img src={'/img/chat_knowledge/keyword_rag/create_keyword_rag.jpg'} width="1000px"/>
</p>
Start chat to knowledge based on Keyword RAG.
<p align="left">
<img src={'/img/chat_knowledge/keyword_rag/keyword_search_chat.jpg'} width="1000px"/>
</p>
docs/docs/dbgpts/introduction.md 0 → 100644
View file @ 396700dd
# dbgpts
[dbgpts](https://github.com/eosphoros-ai/dbgpts) contains some data apps, AWEL operators, AWEL workflows, agents and resources
which build upon the DB-GPT.
## Introduction
### Why We Need `dbgpts`
In a production-level LLM's application, there are many components that need to be
integrated, and you want to start your research and creativity quickly by using the
existing components.
At the same time, we hope that the core components of DB-GPT keep simple and easy to
maintain, and some complex components can be developed in the form of plugins.
So, we need a plugin system to extend the capabilities of DB-GPT, and `dbgpts` is the
plugin system or a part of ecosystem of DB-GPT.
### What Is `dbgpts`
There are some concepts in `dbgpts`:
- `app`: It includes data apps, AWEL operators, AWEL workflows, agents and resources, sometimes, we
call it `dbgpts` app or `dbgpts` package.
- `repo`: It is a repository of `dbgpts` apps, you can install a `dbgpts` app from a `dbgpts` repo,
the default `dbgpts` repo is [eosphoros-ai/dbgpts](https://github.com/eosphoros-ai/dbgpts), you can
also create your own `dbgpts` repo or use other's `dbgpts` repo.
### How To Run `dbgpts`
1. When you install a `dbgpts` app, it will be loaded to your DB-GPT webserver automatically,
and you can use it in the DB-GPT webserver or trigger it by command line `dbgpt run ...`.
2. You can also run a `dbgpts` app as a command line tool, you can use it in your terminal by
`dbgpt app run ...` with `--local` option, it will run the app in your local environment.
## Quick Start
Let's install a `dbgpts` package named [awel-flow-simple-streaming-chat](https://github.com/eosphoros-ai/dbgpts/tree/main/workflow/awel-flow-simple-streaming-chat)
```bash
dbgpt app install awel-flow-simple-streaming-chat -U
```
### Run The App Locally
Then, you can run the app in your terminal:
```bash
dbgpt run flow --local chat \
--name awel-flow-simple-streaming-chat \
--model "gpt-3.5-turbo" \
--messages "hello" \
--stream
```
- `dbgpt run flow`: Means you want to run a AWEL workflow.
- `--local`: Means you want to run the workflow in your local environment without
starting the DB-GPT webserver, it will find the `app` installed in your local
environment, then run it, also, you can use `--file` to specify the python file.
- `--name`: The name of the app.
- `--model`: The LLM model you want to use, `awel-flow-simple-streaming-chat` will
use OpenAI LLM by default if you run it with `--local`.
- `--messages`: The messages you want to send to the LLM.
- `--stream`: Means you want to run the workflow in streaming mode.
The output will be like this:
```bash
You: hello
[~info] Chat stream started
[~info] JSON data: {"model": "gpt-3.5-turbo", "messages": "hello", "stream": true}
Bot:
Hello! How can I assist you today?
🎉 Chat stream finished, timecost: 1.12 s
```
### Run The App In DB-GPT Webserver
After you install the `awel-flow-simple-streaming-chat` app, you can run it in the DB-GPT webserver.
Also, you can use the `dbgpt` command line tool to trigger the app.
```bash
dbgpt run flow chat \
--name awel-flow-simple-streaming-chat \
--model "chatgpt_proxyllm" \
--messages "hello" \
--stream
```
You just remove the `--local` option, then the command will connect to the DB-GPT webserver and run the app.
And you should modify the `--model` option to your model name in the DB-GPT webserver.
The output will be like this:
```bash
You: hello
[~info] Chat stream started
[~info] JSON data: {"model": "chatgpt_proxyllm", "messages": "hello", "stream": true, "chat_param": "1ecd35d4-a60a-420b-8943-8fc44f7f054a", "chat_mode": "chat_flow"}
Bot:
Hello! How can I assist you today?
🎉 Chat stream finished, timecost: 0.98 s
```
## Run The App With `command` Mode
In previous examples, we run the app in `chat` mode, but not all `dbgpts` apps support `chat` mode,
some apps support `command` mode, you can run the app with `dbgpt run flow cmd` command.
### Run The App Locally
```bash
dbgpt run flow --local cmd \
--name awel-flow-simple-streaming-chat \
-d '
{
"model": "gpt-3.5-turbo",
"messages": "hello",
"stream": true
}
'
```
We replace the `chat` mode with `cmd` mode, and use `-d` option to specify the data in JSON format.
The output will be like this:
```bash
[~info] Flow started
[~info] JSON data: {"model": "gpt-3.5-turbo", "messages": "hello", "stream": true}
Command output:
Hello! How can I assist you today?
🎉 Flow finished, timecost: 1.35 s
```
### Run The App In DB-GPT Webserver
Just remove the `--local` option, then the command will connect to the DB-GPT webserver and run the app.
```bash
dbgpt run flow cmd \
--name awel-flow-simple-streaming-chat \
-d '
{
"model": "chatgpt_proxyllm",
"messages": "hello",
"stream": true
}
'
```
The output will be like this:
```bash
[~info] Flow started
[~info] JSON data: {"model": "chatgpt_proxyllm", "messages": "hello", "stream": true}
Command output:
Hello! How can I assist you today?
🎉 Flow finished, timecost: 1.09 s
```
## `chat` Mode vs `command` Mode
In short, `chat` mode is used for chat applications, and `command` mode is used to
trigger the app with a command.
For example, you want to load your documents to the DB-GPT, you can use `command` mode
to trigger the app to load the documents, it always runs once and the result will be
returned.
And `chat` mode is a special case of `command` mode, it provides a chat interface to
the user, and you can chat with the LLM in an interactive way.
## Run You App With Python Script
If you run app locally, it will find the app which is installed in your local environment,
also, you can run the app by providing the python file.
Let's create a python file named `simple_chat_app.py`:
```python
import os
from dbgpt._private.pydantic import BaseModel, Field
from dbgpt.core import ModelMessage, ModelRequest
from dbgpt.core.awel import DAG, HttpTrigger, MapOperator
from dbgpt.model.proxy import OpenAILLMClient
from dbgpt.model.operators import LLMOperator
class TriggerReqBody(BaseModel):
model: str = Field(..., description="Model name")
messages: str = Field(..., description="User input")
class RequestHandleOperator(MapOperator[TriggerReqBody, ModelRequest]):
def __init__(self, **kwargs):
super().__init__(**kwargs)
async def map(self, input_value: TriggerReqBody) -> ModelRequest:
messages = [ModelMessage.build_human_message(input_value.messages)]
return ModelRequest.build_request(input_value.model, messages)
with DAG("dbgpts_simple_chat_app") as dag:
# Receive http request and trigger dag to run.
trigger = HttpTrigger(
"/dbgpts/simple_chat_app", methods="POST", request_body=TriggerReqBody
)
llm_client = OpenAILLMClient(
model_alias="gpt-3.5-turbo", # or other models, eg. "gpt-4o"
api_base=os.getenv("OPENAI_API_BASE"),
api_key=os.getenv("OPENAI_API_KEY"),
)
request_handle_task = RequestHandleOperator()
llm_task = LLMOperator(llm_client=llm_client)
model_parse_task = MapOperator(lambda out: out.text)
trigger >> request_handle_task >> llm_task >> model_parse_task
```
Then you can run the app by providing the python file:
```bash
dbgpt run flow --local --file simple_chat_app.py \
chat \
--name dbgpts_simple_chat_app \
--model "gpt-3.5-turbo" \
--messages "hello"
```
The output will be like this:
```bash
You: hello
[~info] Chat started
[~info] JSON data: {"model": "gpt-3.5-turbo", "messages": "hello", "stream": false}
Bot:
Hello! How can I assist you today?
🎉 Chat stream finished, timecost: 1.06 s
```
And you can run previous examples with `command` mode.
```bash
dbgpt run flow --local --file simple_chat_app.py \
cmd \
--name dbgpts_simple_chat_app \
-d '
{
"model": "gpt-3.5-turbo",
"messages": "hello"
}'
```
The output will be like this:
```bash
[~info] Flow started
[~info] JSON data: {"model": "gpt-3.5-turbo", "messages": "hello"}
Command output:
Hello! How can I assist you today?
🎉 Flow finished, timecost: 1.04 s
```
## Show Your App In DB-GPT Webserver
When you install the workflow, you can see the workflow in the DB-GPT webserver, you can open
the **AWEL Flow** page, then you can see the workflow named `awel_flow_simple_streaming_chat`.
<p align="left">
<img src={'/img/dbgpts/awel_flow_simple_streaming_chat_1.png'} width="720px" />
</p>
Then you can click the `edit` button to see the details of the workflow.
<p align="left">
<img src={'/img/dbgpts/awel_flow_simple_streaming_chat_2.png'} width="720px" />
</p>
Note: Not all workflows support editing, there are two types of workflows according to the
definition type: `json` and `python`, the `json` type workflow can be edited in the DB-GPT,
We will show you more details in the next sections.
docs/docs/faq/install.md 0 → 100644
View file @ 396700dd
# Installation FAQ
### Q1: sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
make sure you pull latest code or create directory with mkdir pilot/data
### Q2: The model keeps getting killed.
your GPU VRAM size is not enough, try replace your hardware or replace other llms.
### Q3: How to access website on the public network
You can try to use gradio's [network](https://github.com/gradio-app/gradio/blob/main/gradio/networking.py) to achieve.
```python
import secrets
from gradio import networking
token=secrets.token_urlsafe(32)
local_port=5670
url = networking.setup_tunnel('0.0.0.0', local_port, token)
print(f'Public url: {url}')
time.sleep(60 * 60 * 24)
```
Open `url` with your browser to see the website.
### Q4: (Windows) execute `pip install -e .` error
The error log like the following:
```
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [11 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-310
creating build\lib.win-amd64-cpython-310\cchardet
copying src\cchardet\version.py -> build\lib.win-amd64-cpython-310\cchardet
copying src\cchardet\__init__.py -> build\lib.win-amd64-cpython-310\cchardet
running build_ext
building 'cchardet._cchardet' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
[end of output]
```
Download and install `Microsoft C++ Build Tools` from [visual-cpp-build-tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
### Q5: `Torch not compiled with CUDA enabled`
```
2023-08-19 16:24:30 | ERROR | stderr | raise AssertionError("Torch not compiled with CUDA enabled")
2023-08-19 16:24:30 | ERROR | stderr | AssertionError: Torch not compiled with CUDA enabled
```
1. Install [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit-archive)
2. Reinstall PyTorch [start-locally](https://pytorch.org/get-started/locally/#start-locally) with CUDA support.
### Q6: `How to migrate meta table chat_history and connect_config from duckdb to sqlite`
```commandline
python docker/examples/metadata/duckdb2sqlite.py
```
### Q7: `How to migrate meta table chat_history and connect_config from duckdb to mysql`
```commandline
1. update your mysql username and password in docker/examples/metadata/duckdb2mysql.py
2. python docker/examples/metadata/duckdb2mysql.py
```
### Q8: `How to manage and migrate my database`
You can use the command of `dbgpt db migration` to manage and migrate your database.
See the following command for details.
```commandline
dbgpt db migration --help
```
First, you need to create a migration script(just once unless you clean it).
This command with create a `alembic` directory in your `pilot/meta_data` directory and a initial migration script in it.
```commandline
dbgpt db migration init
```
Then you can upgrade your database with the following command.
```commandline
dbgpt db migration upgrade
```
Every time you change the model or pull the latest code from DB-GPT repository, you need to create a new migration script.
```commandline
dbgpt db migration migrate -m "your message"
```
Then you can upgrade your database with the following command.
```commandline
dbgpt db migration upgrade
```
### Q9: `alembic.util.exc.CommandError: Target database is not up to date.`
**Solution 1:**
Run the following command to upgrade the database.
```commandline
dbgpt db migration upgrade
```
**Solution 2:**
Run the following command to clean the migration script and migration history.
```commandline
dbgpt db migration clean -y
```
**Solution 3:**
If you have already run the above command, but the error still exists,
you can try the following command to clean the migration script, migration history and your data.
warning: This command will delete all your data!!! Please use it with caution.
```commandline
dbgpt db migration clean --drop_all_tables -y --confirm_drop_all_tables
```
or
```commandline
rm -rf pilot/meta_data/alembic/versions/*
rm -rf pilot/meta_data/alembic/dbgpt.db
```
\ No newline at end of file
docs/docs/faq/kbqa.md 0 → 100644
View file @ 396700dd
# KBQA FAQ
### Q1: text2vec-large-chinese not found
make sure you have download text2vec-large-chinese embedding model in right way
```tip
centos:yum install git-lfs
ubuntu:apt-get install git-lfs -y
macos:brew install git-lfs
```
```bash
cd models
git lfs clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
```
### Q2:How to change Vector DB Type in DB-GPT.
Update .env file and set VECTOR_STORE_TYPE.
DB-GPT currently support Chroma(Default), Milvus(>2.1), Weaviate, OceanBase vector database.
If you want to change vector db, Update your .env, set your vector store type, VECTOR_STORE_TYPE=Chroma (now only support Chroma and Milvus(>2.1), if you set Milvus, please set MILVUS_URL and MILVUS_PORT).
If you want to use OceanBase, please first start a docker container via the following command:
```shell
docker run --name=ob433 -e MODE=slim -p 2881:2881 -d quay.io/oceanbase/oceanbase-ce:4.3.3.0-100000142024101215
```
Download the partner package:
```shell
pip install --upgrade --quiet pyobvector
```
Check the connection to OceanBase and set the memory usage ratio for vector data:
```python
from pyobvector import ObVecClient
tmp_client = ObVecClient()
tmp_client.perform_raw_text_sql(
"ALTER SYSTEM ob_vector_memory_limit_percentage = 30"
)
```
Then set the following variables in the .env file:
```shell
VECTOR_STORE_TYPE=OceanBase
OB_HOST=127.0.0.1
OB_PORT=2881
OB_USER=root@test
OB_DATABASE=test
## Optional
# OB_PASSWORD=
## Optional: If {OB_ENABLE_NORMALIZE_VECTOR} is set, the vector stored in OceanBase is normalized.
# OB_ENABLE_NORMALIZE_VECTOR=True
```
If you want to support more vector db, you can integrate yourself.[how to integrate](https://db-gpt.readthedocs.io/en/latest/modules/vector.html)
```commandline
#*******************************************************************#
#** VECTOR STORE SETTINGS **#
#*******************************************************************#
VECTOR_STORE_TYPE=Chroma
#MILVUS_URL=127.0.0.1
#MILVUS_PORT=19530
#MILVUS_USERNAME
#MILVUS_PASSWORD
#MILVUS_SECURE=
#WEAVIATE_URL=https://kt-region-m8hcy0wc.weaviate.network
```
### Q3:When I use vicuna-13b, found some illegal character like this.
<p align="left">
<img src="https://github.com/eosphoros-ai/DB-GPT/assets/13723926/088d1967-88e3-4f72-9ad7-6c4307baa2f8" width="800px" />
</p>
Set KNOWLEDGE_SEARCH_TOP_SIZE smaller or set KNOWLEDGE_CHUNK_SIZE smaller, and reboot server.
### Q4:space add error (pymysql.err.OperationalError) (1054, "Unknown column 'knowledge_space.context' in 'field list'")
1.shutdown dbgpt_server(ctrl c)
2.add column context for table knowledge_space
```commandline
mysql -h127.0.0.1 -uroot -p {your_password}
```
3.execute sql ddl
```commandline
mysql> use knowledge_management;
mysql> ALTER TABLE knowledge_space ADD COLUMN context TEXT COMMENT "arguments context";
```
4.restart dbgpt serve
### Q5:Use Mysql, how to use DB-GPT KBQA
build Mysql KBQA system database schema.
```bash
$ mysql -h127.0.0.1 -uroot -p{your_password} < ./assets/schema/knowledge_management.sql
```
\ No newline at end of file
docs/docs/faq/llm.md 0 → 100644
View file @ 396700dd
# LLM USE FAQ
### Q1:how to use openai chatgpt service
change your LLM_MODEL
````shell
LLM_MODEL=proxyllm
````
set your OPENAPI KEY
````shell
PROXY_API_KEY={your-openai-sk}
PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions
````
make sure your openapi API_KEY is available
### Q2 What difference between `python dbgpt_server --light` and `python dbgpt_server`
:::tip
python dbgpt_server --light` dbgpt_server does not start the llm service. Users can deploy the llm service separately by using `python llmserver`, and dbgpt_server accesses the llm service through set the LLM_SERVER environment variable in .env. The purpose is to allow for the separate deployment of dbgpt's backend service and llm service.
python dbgpt_server service and the llm service are deployed on the same instance. when dbgpt_server starts the service, it also starts the llm service at the same time.
:::
### Q3 how to use MultiGPUs
DB-GPT will use all available gpu by default. And you can modify the setting `CUDA_VISIBLE_DEVICES=0,1` in `.env` file
to use the specific gpu IDs.
Optionally, you can also specify the gpu ID to use before the starting command, as shown below:
````shell
# Specify 1 gpu
CUDA_VISIBLE_DEVICES=0 python3 dbgpt/app/dbgpt_server.py
# Specify 4 gpus
CUDA_VISIBLE_DEVICES=3,4,5,6 python3 dbgpt/app/dbgpt_server.py
````
You can modify the setting `MAX_GPU_MEMORY=xxGib` in `.env` file to configure the maximum memory used by each GPU.
### Q4 Not Enough Memory
DB-GPT supported 8-bit quantization and 4-bit quantization.
You can modify the setting `QUANTIZE_8bit=True` or `QUANTIZE_4bit=True` in `.env` file to use quantization(8-bit quantization is enabled by default).
Llama-2-70b with 8-bit quantization can run with 80 GB of VRAM, and 4-bit quantization can run with 48 GB of VRAM.
Note: you need to install the latest dependencies according to [requirements.txt](https://github.com/eosphoros-ai/DB-GPT/blob/main/requirements.txt).
Note: you need to install the latest dependencies according to [requirements.txt](https://github.com/eosphoros-ai/DB-GPT/blob/main/requirements.txt).
\ No newline at end of file
docs/docs/installation/advanced_usage/Llamacpp_server.md 0 → 100644
View file @ 396700dd
# LLama.cpp Server
DB-GPT supports native [llama.cpp server](https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md),
which supports concurrent requests and continuous batching inference.
## Install dependencies
```bash
pip install -e ".[llama_cpp_server]"
```
If you want to accelerate the inference speed, and you have a GPU, you can install the following dependencies:
```bash
CMAKE_ARGS="-DGGML_CUDA=ON" pip install -e ".[llama_cpp_server]"
```
## Download the model
Here, we use the `qwen2.5-0.5b-instruct` model as an example. You can download the model from the [Huggingface](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF).
```bash
wget https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF/resolve/main/qwen2.5-0.5b-instruct-q4_k_m.gguf?download=true -O /tmp/qwen2.5-0.5b-instruct-q4_k_m.gguf
````
## Modify configuration file
In the `.env` configuration file, modify the inference type of the model to start `llama.cpp` inference.
```bash
LLM_MODEL=qwen2.5-0.5b-instruct
LLM_MODEL_PATH=/tmp/qwen2.5-0.5b-instruct-q4_k_m.gguf
MODEL_TYPE=llama_cpp_server
```
## Start the DB-GPT server
```bash
python dbgpt/app/dbgpt_server.py
```
\ No newline at end of file
docs/docs/installation/advanced_usage/More_proxyllms.md 0 → 100644
View file @ 396700dd
# ProxyLLMs
DB-GPT can be deployed on servers with lower hardware through proxy LLMs, and now dbgpt support many proxy llms, such as OpenAI、Azure、Wenxin、Tongyi、Zhipu and so on.
### Proxy model
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
<Tabs
defaultValue="openai"
values={[
{label: 'Open AI', value: 'openai'},
{label: 'Azure', value: 'Azure'},
{label: 'Qwen', value: 'qwen'},
{label: 'ChatGLM', value: 'chatglm'},
{label: 'WenXin', value: 'erniebot'},
]}>
<TabItem value="openai" label="open ai">
Install dependencies
```python
pip install -e ".[openai]"
```
Download embedding model
```python
cd DB-GPT
mkdir models and cd models
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
```
Configure the proxy and modify LLM_MODEL, PROXY_API_URL and API_KEY in the `.env`file
```python
# .env
LLM_MODEL=chatgpt_proxyllm
PROXY_API_KEY={your-openai-sk}
PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions
# If you use gpt-4
# PROXYLLM_BACKEND=gpt-4
```
</TabItem>
<TabItem value="Azure" label="Azure">
Install dependencies
```python
pip install -e ".[openai]"
```
Download embedding model
```python
cd DB-GPT
mkdir models and cd models
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese # change this to other embedding model if needed.
```
Configure the proxy and modify LLM_MODEL, PROXY_API_URL and API_KEY in the `.env`file
```python
# .env
LLM_MODEL=proxyllm
PROXY_API_KEY=xxxx
PROXY_API_BASE=https://xxxxxx.openai.azure.com/
PROXY_API_TYPE=azure
PROXY_SERVER_URL=xxxx
PROXY_API_VERSION=2023-05-15
PROXYLLM_BACKEND=gpt-35-turbo
API_AZURE_DEPLOYMENT=xxxx[deployment_name]
```
</TabItem>
<TabItem value="qwen" label="通义千问">
Install dependencies
```python
pip install dashscope
```
Download embedding model
```python
cd DB-GPT
mkdir models and cd models
# embedding model
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
or
git clone https://huggingface.co/moka-ai/m3e-large
```
Configure the proxy and modify LLM_MODEL, PROXY_API_URL and API_KEY in the `.env`file
```python
# .env
# Aliyun tongyiqianwen
LLM_MODEL=tongyi_proxyllm
TONGYI_PROXY_API_KEY={your-tongyi-sk}
PROXY_SERVER_URL={your_service_url}
```
</TabItem>
<TabItem value="chatglm" label="chatglm" >
Install dependencies
```python
pip install zhipuai
```
Download embedding model
```python
cd DB-GPT
mkdir models and cd models
# embedding model
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
or
git clone https://huggingface.co/moka-ai/m3e-large
```
Configure the proxy and modify LLM_MODEL, PROXY_API_URL and API_KEY in the `.env`file
```python
# .env
LLM_MODEL=zhipu_proxyllm
PROXY_SERVER_URL={your_service_url}
ZHIPU_MODEL_VERSION={version}
ZHIPU_PROXY_API_KEY={your-zhipu-sk}
```
</TabItem>
<TabItem value="erniebot" label="文心一言" default>
Download embedding model
```python
cd DB-GPT
mkdir models and cd models
# embedding model
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
or
git clone https://huggingface.co/moka-ai/m3e-large
```
Configure the proxy and modify LLM_MODEL, MODEL_VERSION, API_KEY and API_SECRET in the `.env`file
```python
# .env
LLM_MODEL=wenxin_proxyllm
WEN_XIN_MODEL_VERSION={version} # ERNIE-Bot or ERNIE-Bot-turbo
WEN_XIN_API_KEY={your-wenxin-sk}
WEN_XIN_API_SECRET={your-wenxin-sct}
```
</TabItem>
</Tabs>
:::info note
⚠️ Be careful not to overwrite the contents of the `.env` configuration file
:::
\ No newline at end of file
docs/docs/installation/advanced_usage/OpenAI_SDK_call.md 0 → 100644
View file @ 396700dd
# OpenAI SDK Calls Local Multi-model
The call of multi-model services is compatible with the OpenAI interface, and the models deployed in DB-GPT can be directly called through the OpenAI SDK.
:::info note
⚠️ Before using this project, you must first deploy the model service, which can be deployed through the [cluster deployment tutorial](../model_service/cluster.md).
:::
## Start apiserver
After deploying the model service, you need to start the API Server. By default, the model API Server uses port `8100` to start.
```bash
dbgpt start apiserver --controller_addr http://127.0.0.1:8000 --api_keys EMPTY
```
## Verify
### cURL validation
After the apiserver is started, the service call can be verified. First, let's look at verification through curl.
:::tip
List models
:::
```bash
curl http://127.0.0.1:8100/api/v1/models \
-H "Authorization: Bearer EMPTY" \
-H "Content-Type: application/json"
```
:::tip
Chat
:::
```bash
curl http://127.0.0.1:8100/api/v1/chat/completions \
-H "Authorization: Bearer EMPTY" \
-H "Content-Type: application/json" \
-d '{"model": "glm-4-9b-chat", "messages": [{"role": "user", "content": "hello"}]}'
```
:::tip
Embedding
:::
```bash
curl http://127.0.0.1:8100/api/v1/embeddings \
-H "Authorization: Bearer EMPTY" \
-H "Content-Type: application/json" \
-d '{
"model": "text2vec",
"input": "Hello world!"
}'
```
## Verify via OpenAI SDK
```bash
import openai
openai.api_key = "EMPTY"
openai.api_base = "http://127.0.0.1:8100/api/v1"
model = "glm-4-9b-chat"
completion = openai.ChatCompletion.create(
model=model,
messages=[{"role": "user", "content": "hello"}]
)
# print the completion
print(completion.choices[0].message.content)
```
## (Experimental) Rerank Open API
The rerank API is an experimental feature that can be used to rerank the candidate list.
1. Use cURL to verify the rerank API.
```bash
curl http://127.0.0.1:8100/api/v1/beta/relevance \
-H "Authorization: Bearer EMPTY" \
-H "Content-Type: application/json" \
-d '{
"model": "bge-reranker-base",
"query": "what is awel talk about?",
"documents": [
"Agentic Workflow Expression Language(AWEL) is a set of intelligent agent workflow expression language specially designed for large model application development.",
"Autonomous agents have long been a research focus in academic and industry communities",
"AWEL is divided into three levels in deign, namely the operator layer, AgentFream layer and DSL layer.",
"Elon musk is a famous entrepreneur and inventor, he is the founder of SpaceX and Tesla."
]
}'
```
2. Use python to verify the rerank API.
```python
from dbgpt.rag.embedding import OpenAPIRerankEmbeddings
rerank = OpenAPIRerankEmbeddings(api_key="EMPTY", model_name="bge-reranker-base")
rerank.predict(
query="what is awel talk about?",
candidates=[
"Agentic Workflow Expression Language(AWEL) is a set of intelligent agent workflow expression language specially designed for large model application development.",
"Autonomous agents have long been a research focus in academic and industry communities",
"AWEL is divided into three levels in deign, namely the operator layer, AgentFream layer and DSL layer.",
"Elon musk is a famous entrepreneur and inventor, he is the founder of SpaceX and Tesla."
]
)
```
The output is as follows:
```bash
[
0.9685816764831543,
3.7338297261158004e-05,
0.03692878410220146,
3.73825132555794e-05
]
```
docs/docs/installation/advanced_usage/ollama.md 0 → 100644
View file @ 396700dd
# ollama
ollama is a model serving platform that allows you to deploy models in a few seconds.
It is a great tool.
### Install ollama
If your system is linux.
```bash
curl -fsSL https://ollama.com/install.sh | sh
```
other environments, please refer to the [official ollama website](https://ollama.com/).
### Pull models.
1. Pull LLM
```bash
ollama pull qwen:0.5b
```
2. Pull embedding model.
```bash
ollama pull nomic-embed-text
```
3. install ollama package.
```bash
pip install ollama
```
### Use ollama proxy model in DB-GPT `.env` file
```bash
LLM_MODEL=ollama_proxyllm
PROXY_SERVER_URL=http://127.0.0.1:11434
PROXYLLM_BACKEND="qwen:0.5b"
PROXY_API_KEY=not_used
EMBEDDING_MODEL=proxy_ollama
proxy_ollama_proxy_server_url=http://127.0.0.1:11434
proxy_ollama_proxy_backend="nomic-embed-text:latest"
```
### run dbgpt server
```bash
python dbgpt/app/dbgpt_server.py
```
\ No newline at end of file
docs/docs/installation/advanced_usage/vLLM_inference.md 0 → 100644
View file @ 396700dd
# vLLM Inference
DB-GPT supports [vLLM](https://github.com/vllm-project/vllm) inference, a fast and easy-to-use LLM inference and service library.
## Install dependencies
`vLLM` is an optional dependency in DB-GPT. You can install it manually through the following command.
```bash
pip install -e ".[vllm]"
```
## Modify configuration file
In the `.env` configuration file, modify the inference type of the model to start `vllm` inference.
```bash
LLM_MODEL=glm-4-9b-chat
MODEL_TYPE=vllm
# modify the following configuration if you possess GPU resources
# gpu_memory_utilization=0.8
```
For more information about the list of models supported by `vLLM`, please refer to the [vLLM supported model document](https://docs.vllm.ai/en/latest/models/supported_models.html#supported-models).
docs/docs/installation/build_image.md 0 → 100644
View file @ 396700dd
---
id: docker-build-guide
title: DB-GPT Docker Build Guide
sidebar_label: Docker Build Guide
description: Comprehensive guide for building DB-GPT Docker images with various configurations
keywords:
- DB-GPT
- Docker
- Build
- CUDA
- OpenAI
- VLLM
- Llama-cpp
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CodeBlock from '@theme/CodeBlock';
# DB-GPT Docker Build Guide
This guide provides comprehensive instructions for building DB-GPT Docker images with various configurations using the `docker/base/build_image.sh` script.
## Overview
The DB-GPT build script allows you to create Docker images tailored to your specific requirements. You can choose from predefined installation modes or customize the build with specific extras, environment variables, and other settings.
## Available Installation Modes
<Tabs>
<TabItem value="default" label="Default" default>
CUDA-based image with standard features.
```bash
bash docker/base/build_image.sh
```
Includes: CUDA support, proxy integrations (OpenAI, Ollama, Zhipuai, Anthropic, Qianfan, Tongyi), RAG capabilities, graph RAG, Hugging Face integration, and quantization support.
</TabItem>
<TabItem value="openai" label="OpenAI">
CPU-based image optimized for OpenAI API usage.
```bash
bash docker/base/build_image.sh --install-mode openai
```
Includes: Basic functionality, all proxy integrations, and RAG capabilities without GPU acceleration.
</TabItem>
<TabItem value="vllm" label="VLLM">
CUDA-based image with VLLM for optimized inference.
```bash
bash docker/base/build_image.sh --install-mode vllm
```
Includes: All default features plus VLLM support for high-performance inference.
</TabItem>
<TabItem value="llama-cpp" label="Llama-cpp">
CUDA-based image with Llama-cpp support.
```bash
bash docker/base/build_image.sh --install-mode llama-cpp
```
Includes: All default features plus Llama-cpp and Llama-cpp server with CUDA acceleration enabled via `CMAKE_ARGS="-DGGML_CUDA=ON"`.
</TabItem>
<TabItem value="full" label="Full">
CUDA-based image with all available features.
```bash
bash docker/base/build_image.sh --install-mode full
```
Includes: All features from other modes plus embedding capabilities.
</TabItem>
</Tabs>
## Basic Usage
### View Available Modes
To see all available installation modes and their configurations:
```bash
bash docker/base/build_image.sh --list-modes
```
### Get Help
Display all available options:
```bash
bash docker/base/build_image.sh --help
```
## Customization Options
### Python Version
DB-GPT requires Python 3.10 or higher. The default is Python 3.11, but you can specify a different version:
```bash
bash docker/base/build_image.sh --python-version 3.10
```
### Custom Image Name
Set a custom name for the built image:
```bash
bash docker/base/build_image.sh --image-name mycompany/dbgpt
```
### Image Name Suffix
Add a suffix to the image name for versioning or environment identification:
```bash
bash docker/base/build_image.sh --image-name-suffix v1.0
```
This will generate `eosphorosai/dbgpt-v1.0` for the default mode or `eosphorosai/dbgpt-MODE-v1.0` for specific modes.
### PIP Mirror
Choose a different PIP index URL:
```bash
bash docker/base/build_image.sh --pip-index-url https://pypi.org/simple
```
### Ubuntu Mirror
Control whether to use Tsinghua Ubuntu mirror:
```bash
bash docker/base/build_image.sh --use-tsinghua-ubuntu false
```
### Language Preference
Set your preferred language (default is English):
```bash
bash docker/base/build_image.sh --language zh
```
## Advanced Customization
### Custom Extras
You can customize the Python package extras installed in the image:
<Tabs>
<TabItem value="override" label="Override Extras" default>
Completely replace the default extras with your own selection:
```bash
bash docker/base/build_image.sh --extras "base,proxy_openai,rag,storage_chromadb"
```
</TabItem>
<TabItem value="add" label="Add Extras">
Keep the default extras and add more:
```bash
bash docker/base/build_image.sh --add-extras "storage_milvus,storage_elasticsearch,datasource_postgres"
```
</TabItem>
<TabItem value="mode-specific" label="Mode-Specific">
Add specific extras to a particular installation mode:
```bash
bash docker/base/build_image.sh --install-mode vllm --add-extras "storage_milvus,datasource_postgres"
```
</TabItem>
</Tabs>
#### Available Extra Options
Here are some useful extras you can add:
| Extra Package | Description |
|--------------|-------------|
| `storage_milvus` | Vector store integration with Milvus |
| `storage_elasticsearch` | Vector store integration with Elasticsearch |
| `datasource_postgres` | Database connector for PostgreSQL |
| `vllm` | VLLM integration for optimized inference |
| `llama_cpp` | Llama-cpp Python bindings |
| `llama_cpp_server` | Llama-cpp HTTP server |
You can run `uv run install_help.py list` in your local DB-GPT repository to see all available extras.
### Environment Variables
DB-GPT build supports environment variables for specialized builds. The main environment variable used is `CMAKE_ARGS` which is particularly important for Llama-cpp compilation.
<Tabs>
<TabItem value="override-env" label="Override Env Vars" default>
Replace the default environment variables:
```bash
bash docker/base/build_image.sh --env-vars "CMAKE_ARGS=\"-DGGML_CUDA=ON -DLLAMA_CUBLAS=ON\""
```
</TabItem>
<TabItem value="add-env" label="Add Env Vars">
Add additional environment variables:
```bash
bash docker/base/build_image.sh --install-mode llama-cpp --add-env-vars "FORCE_CMAKE=1"
```
</TabItem>
</Tabs>
:::note
For Llama-cpp mode, `CMAKE_ARGS="-DGGML_CUDA=ON"` is automatically set to enable CUDA acceleration.
:::
### Docker Network
Specify a Docker network for building:
```bash
bash docker/base/build_image.sh --network host
```
### Custom Dockerfile
Use a custom Dockerfile:
```bash
bash docker/base/build_image.sh --dockerfile Dockerfile.custom
```
## Example Scenarios
### Enterprise DB-GPT with PostgreSQL and Elasticsearch
Build a full-featured enterprise version with PostgreSQL and Elasticsearch support:
```bash
bash docker/base/build_image.sh --install-mode full \
--add-extras "storage_elasticsearch,datasource_postgres" \
--image-name-suffix enterprise \
--python-version 3.10 \
--load-examples false
```
### Optimized Llama-cpp for Specific Hardware
Build with custom Llama-cpp optimization flags:
```bash
bash docker/base/build_image.sh --install-mode llama-cpp \
--env-vars "CMAKE_ARGS=\"-DGGML_CUDA=ON -DGGML_AVX2=OFF -DGGML_AVX512=ON\"" \
--python-version 3.11
```
### Lightweight OpenAI Proxy
Build a minimal OpenAI proxy image:
```bash
bash docker/base/build_image.sh --install-mode openai \
--use-tsinghua-ubuntu false \
--pip-index-url https://pypi.org/simple \
--load-examples false
```
### Development Build with Milvus
Build a development version with Milvus support:
```bash
bash docker/base/build_image.sh --install-mode vllm \
--add-extras "storage_milvus" \
--image-name-suffix dev
```
## Troubleshooting
<details>
<summary>Common Build Issues</summary>
### CUDA Not Found
If you encounter CUDA-related errors:
```bash
# Try building with a different CUDA base image
bash docker/base/build_image.sh --base-image nvidia/cuda:12.1.0-devel-ubuntu22.04
```
### Package Installation Failures
If extras fail to install:
```bash
# Try building with fewer extras to isolate the problem
bash docker/base/build_image.sh --extras "base,proxy_openai,rag"
```
### Network Issues
If you encounter network problems:
```bash
# Use a specific network
bash docker/base/build_image.sh --network host
```
</details>
## API Reference
### Script Options
| Option | Description | Default Value |
|--------|-------------|---------------|
| `--install-mode` | Installation mode | `default` |
| `--base-image` | Base Docker image | `nvidia/cuda:12.4.0-devel-ubuntu22.04` |
| `--image-name` | Docker image name | `eosphorosai/dbgpt` |
| `--image-name-suffix` | Suffix for image name | ` ` |
| `--pip-index-url` | PIP mirror URL | `https://pypi.tuna.tsinghua.edu.cn/simple` |
| `--language` | Interface language | `en` |
| `--load-examples` | Load example data | `true` |
| `--python-version` | Python version | `3.11` |
| `--use-tsinghua-ubuntu` | Use Tsinghua Ubuntu mirror | `true` |
| `--extras` | Extra packages to install | Mode dependent |
| `--add-extras` | Additional extra packages | ` ` |
| `--env-vars` | Build environment variables | Mode dependent |
| `--add-env-vars` | Additional environment variables | ` ` |
| `--dockerfile` | Dockerfile to use | `Dockerfile` |
| `--network` | Docker network to use | ` ` |
## Additional Resources
- [DB-GPT Documentation](https://github.com/eosphoros-ai/DB-GPT)
- [Docker Documentation](https://docs.docker.com/)
- [CUDA Documentation](https://docs.nvidia.com/cuda/)
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment