Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenzhuo
lmdeployEx
Commits
75cf0593
"vscode:/vscode.git/clone" did not exist on "4acd87ff4e6a363b47d13b8a960b4b8340a3d615"
Commit
75cf0593
authored
May 21, 2024
by
chenzhuo
Browse files
readme
parent
374c78ca
Pipeline
#1013
canceled with stages
Changes
2
Pipelines
1
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
8 additions
and
3 deletions
+8
-3
README.md
README.md
+7
-2
sugon_readme.md
sugon_readme.md
+1
-1
No files found.
README.md
View file @
75cf0593
...
...
@@ -11,8 +11,9 @@ LMDeploy 由 [MMDeploy](https://github.com/open-mmlab/mmdeploy) 和 [MMRazor](ht
-
**persistent batch 推理**
:进一步优化模型执行效率。
persistent batch 推理:进一步优化模型执行效率。
LMdeploy官方github地址:
[
https://github.com/InternLM/lmdeploy
](
https://github.com/InternLM/lmdeploy
)
persistent batch 推理:进一步优化模型执行效率。
<br>
LMdeploy官方github地址:
[
https://github.com/InternLM/lmdeploy
](
https://github.com/InternLM/lmdeploy
)
<br>
现在支持qwen1.5 详见sugon_readme.md
## 暂不支持的官方功能
-
**量化推理**
:目前仅支持fp16的推理,awq-int4的权重量化和kv-cache int8推理方案暂不支持
...
...
@@ -28,6 +29,10 @@ LMdeploy官方github地址:[https://github.com/InternLM/lmdeploy](https://github
| QWen-7B | Yes | Yes |
| QWen-14B | Yes | Yes |
| QWen-72B | Yes | Yes |
| QWen1.5-7B | Yes | Yes |
| QWen1.5-14B | Yes | Yes |
| QWen1.5-72B | Yes | Yes |
| QWen1.5-110B | Yes | Yes |
| Baichuan-7B | Yes | Yes |
| Baichuan2-7B | Yes | Yes |
| wizardlM | Yes | Yes |
...
...
sugon_readme.md
View file @
75cf0593
### 修改部分(qwen1.5)
1.
requirements/runtime.txt transformers==4.38.2
1.
requirements/runtime.txt transformers==4.38.2
<br>
2.
lmdeploy/turbomind/deploy/source_model/qwen.py
<br>
要将文件内容变成下面的 添加qwen的模型权重读取对应
<br>
```
python
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment