Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
Lmdeploy
Commits
386c0a76
Commit
386c0a76
authored
May 30, 2024
by
gaoqiong
Browse files
根据ubuntu编译做一些源码调整
parent
1884fd60
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
16 additions
and
9 deletions
+16
-9
README.md
README.md
+11
-6
requirements/build.txt
requirements/build.txt
+1
-0
setup.py
setup.py
+4
-3
No files found.
README.md
View file @
386c0a76
...
...
@@ -55,13 +55,18 @@ docker run -it --name baichuan --shm-size=1024G -v /opt/hyhal:/opt/hyhal --devic
2、要是非光源提供镜像,配置环境:(若安装过慢,可以添加源:pip3 install xxx -i https://pypi.tuna.tsinghua.edu.cn/simple/)
```
#centos7.6环境
pip3 install -r requirements.txt
pip3 install urllib3==1.24
yum install rapidjson
# gcc版本需要>=9 安装高版本gcc 要是必须使用gcc7,可以下载对应的gcc7的分支
yum install -y centos-release-scl
yum install -y devtoolset-9
scl enable devtoolset-9 bash
#ubuntu20.04环境
export NCCL_LIB_DIR=/opt/dtk/cuda/lib64
pip3 install -r requirements.txt
pip3 install urllib3==1.24
#apt-get 换源,添加清华源
sudo apt-get update
apt-get install rapidjson-dev
# 执行nccl环境变量
export NCCL_LAUNCH_MODE=GROUP
...
...
@@ -156,8 +161,8 @@ lmdeploy convert ${model_name} ${model_path} --model_format awq --group-size ${g
#step1:模型运行
lmdeploy chat turbomind ./workspace
--tp
${
tp
}
```
注意事项:
1.
该版本暂时仅支持tp=1 单卡量化推理;
注意事项:
1.
该版本暂时仅支持tp=1 单卡量化推理
,仅支持卡型KM-AI,暂不支持K100/Z100/Z100L
;
2.
该版本量化推理功能仅支持先通过convert模型转换为turbomind格式,然后进行推理运行,暂时不知道hf模型直接量化推理;
3.
该版本暂时不支持通过数据集进行量化功能,需要在别处获取量化模型;
...
...
requirements/build.txt
View file @
386c0a76
pybind11
setuptools
wheel
\ No newline at end of file
setup.py
View file @
386c0a76
...
...
@@ -60,12 +60,12 @@ def get_version_add(sha: Optional[str] = None) -> str:
version
+=
".dtk"
+
rocm_version
# torch version
version
+=
".torch"
+
torch
.
__version__
[:
4
]
version
+=
".torch"
+
torch
.
__version__
[:
5
]
lines
=
[]
with
open
(
add_version_path
,
'r'
,
encoding
=
'utf-8'
)
as
file
:
lines
=
file
.
readlines
()
lines
[
2
]
=
"__dcu_version__ = '0.2.6+{}'
\n
"
.
format
(
version
)
lines
[
2
]
=
"__dcu_version__ = '0.2.6+
das1.1.
{}'
\n
"
.
format
(
version
)
with
open
(
add_version_path
,
encoding
=
"utf-8"
,
mode
=
"w"
)
as
file
:
file
.
writelines
(
lines
)
file
.
close
()
...
...
@@ -212,6 +212,7 @@ if __name__ == '__main__':
author
=
'OpenMMLab'
,
author_email
=
'openmmlab@gmail.com'
,
packages
=
find_packages
(
exclude
=
()),
package_data
=
{
'lmdeploy'
:
lmdeploy_package_data
,
},
...
...
@@ -235,4 +236,4 @@ if __name__ == '__main__':
'Intended Audience :: Science/Research'
,
],
entry_points
=
{
'console_scripts'
:
[
'lmdeploy = lmdeploy.cli:run'
]},
)
)
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment