Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
BLOOM_fastertransformer
Commits
b26bccb5
Commit
b26bccb5
authored
Nov 08, 2023
by
zhuwenwen
Browse files
add modules and update readme
parent
21de54f8
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
24 additions
and
2 deletions
+24
-2
.gitmodules
.gitmodules
+4
-0
README.md
README.md
+20
-2
No files found.
.gitmodules
0 → 100644
View file @
b26bccb5
[submodule "fastertransformer"]
path = fastertransformer
url = http://developer.hpccube.com/codes/aicomponent/fastertransformer.git
branch = develop
\ No newline at end of file
README.md
View file @
b26bccb5
...
@@ -2,7 +2,7 @@
...
@@ -2,7 +2,7 @@
*
@Author: zhuww
*
@Author: zhuww
*
@email: zhuww@sugon.com
*
@email: zhuww@sugon.com
*
@Date: 2023-09-08 11:08:07
*
@Date: 2023-09-08 11:08:07
*
@LastEditTime: 2023-
09-19 14:10
:01
*
@LastEditTime: 2023-
11-08 17:47
:01
-->
-->
# BLOOM
# BLOOM
...
@@ -48,6 +48,23 @@ docker run -it --name llama --shm-size=32G --device=/dev/kfd --device=/dev/dri/
...
@@ -48,6 +48,23 @@ docker run -it --name llama --shm-size=32G --device=/dev/kfd --device=/dev/dri/
## 推理
## 推理
### 编译
```
bash
git clone http://developer.hpccube.com/codes/modelzoo/bloom_fastertransformer.git
cd
bloom_fastertransformer
mkdir
build
cd
build
git submodule init
&&
git submodule update
cmake
-DCMAKE_BUILD_TYPE
=
Release
-DBUILD_MULTI_GPU
=
ON
-DCMAKE_CXX_COMPILER
=
nvcc ..
export
C_INCLUDE_PATH
=
$PWD
/_deps/googletest-src/googletest/include
${
C_INCLUDE_PATH
:+:
${
C_INCLUDE_PATH
}}
export
CPLUS_INCLUDE_PATH
=
$PWD
/_deps/googletest-src/googletest/include
${
CPLUS_INCLUDE_PATH
:+:
${
CPLUS_INCLUDE_PATH
}}
make
-j12
运行前:
export
LD_LIBRARY_PATH
=
$PWD
/src/fastertransformer/utils/gemm_test/CMakeFiles/gpt_gemm_func.dir:
$LD_LIBRARY_PATH
```
注:若不重新编译,直接使用测试目录代码即可。
### 模型下载
### 模型下载
[
bloom 7B
](
https://huggingface.co/bigscience/bloomz-7b1-mt
)
[
bloom 7B
](
https://huggingface.co/bigscience/bloomz-7b1-mt
)
...
@@ -110,7 +127,8 @@ build/
...
@@ -110,7 +127,8 @@ build/
```
```
执行一下命令可以解析out结果:
执行一下命令可以解析out结果:
```
bash
```
bash
python bloom_token_converter.py
python bloom_token_converter.py
# 使用镜像代码
python ../examples/cpp/multi_gpu_gpt/bloom_tokenizer.py
# 若重新编译
其中,
`
tokenizer
`
为原模型路径
其中,
`
tokenizer
`
为原模型路径
```
```
测试数据:"Translate to English: Je t’aime." (token id: 153772, 427, 9522, 6395, 76721, 68258, 17),使用的加速卡:1张 DCU-Z100L-32G
测试数据:"Translate to English: Je t’aime." (token id: 153772, 427, 9522, 6395, 76721, 68258, 17),使用的加速卡:1张 DCU-Z100L-32G
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment