Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
ChatGLM3_fastllm
Commits
81e6b0e9
Commit
81e6b0e9
authored
Jan 12, 2024
by
zhouxiang
Browse files
完善readme
parent
7cd68ade
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
3 additions
and
3 deletions
+3
-3
README.md
README.md
+2
-2
api_server_demo/requirements.txt
api_server_demo/requirements.txt
+1
-1
No files found.
README.md
View file @
81e6b0e9
...
...
@@ -32,7 +32,7 @@ ChatGLM3-6B基于GLM架构开发。GLM是一种基于Transformer的语言模型
<img
src=
"doc/GLM.png"
width=
"550"
height=
"200"
>
</div>
本项目主要针对ChatGLM
2
-6B推理性能优化,达到DCU平台较快的对话效果
本项目主要针对ChatGLM
3
-6B推理性能优化,达到DCU平台较快的对话效果
## 环境配置
...
...
@@ -97,7 +97,7 @@ streamlit run web_demo.py chatglm3-6b-fp16.bin
cd api_server_demo
pip install -r requirements.txt
# 运行api_server服务,使用-p指定转换后的模型文件,客户端代码可以参考openai-client.py实现:
python fastllm-openai.py -p chatglm3-6b-fp16.bin
python fastllm-openai.py -p
../
chatglm3-6b-fp16.bin
# 如果需要测试服务的并发性能,可以使用openai-client.py,修改其中的prompt和concurrencys变量值后执行:
python openai-client.py
```
...
...
api_server_demo/requirements.txt
View file @
81e6b0e9
...
...
@@ -2,4 +2,4 @@ uvicorn==0.23.2
pydantic==2.5.1
fastapi==0.103.1
sse_starlette
openai
openai==0.28
openai==0.28
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment