Commit b26bccb5 authored by zhuwenwen's avatar zhuwenwen
Browse files

add modules and update readme

parent 21de54f8
[submodule "fastertransformer"]
path = fastertransformer
url = http://developer.hpccube.com/codes/aicomponent/fastertransformer.git
branch = develop
\ No newline at end of file
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
* @Author: zhuww * @Author: zhuww
* @email: zhuww@sugon.com * @email: zhuww@sugon.com
* @Date: 2023-09-08 11:08:07 * @Date: 2023-09-08 11:08:07
* @LastEditTime: 2023-09-19 14:10:01 * @LastEditTime: 2023-11-08 17:47:01
--> -->
# BLOOM # BLOOM
...@@ -48,6 +48,23 @@ docker run -it --name llama --shm-size=32G --device=/dev/kfd --device=/dev/dri/ ...@@ -48,6 +48,23 @@ docker run -it --name llama --shm-size=32G --device=/dev/kfd --device=/dev/dri/
## 推理 ## 推理
### 编译
```bash
git clone http://developer.hpccube.com/codes/modelzoo/bloom_fastertransformer.git
cd bloom_fastertransformer
mkdir build
cd build
git submodule init && git submodule update
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_MULTI_GPU=ON -DCMAKE_CXX_COMPILER=nvcc ..
export C_INCLUDE_PATH=$PWD/_deps/googletest-src/googletest/include${C_INCLUDE_PATH:+:${C_INCLUDE_PATH}}
export CPLUS_INCLUDE_PATH=$PWD/_deps/googletest-src/googletest/include${CPLUS_INCLUDE_PATH:+:${CPLUS_INCLUDE_PATH}}
make -j12
运行前:
export LD_LIBRARY_PATH=$PWD/src/fastertransformer/utils/gemm_test/CMakeFiles/gpt_gemm_func.dir:$LD_LIBRARY_PATH
```
注:若不重新编译,直接使用测试目录代码即可。
### 模型下载 ### 模型下载
[bloom 7B](https://huggingface.co/bigscience/bloomz-7b1-mt) [bloom 7B](https://huggingface.co/bigscience/bloomz-7b1-mt)
...@@ -110,7 +127,8 @@ build/ ...@@ -110,7 +127,8 @@ build/
``` ```
执行一下命令可以解析out结果: 执行一下命令可以解析out结果:
```bash ```bash
python bloom_token_converter.py python bloom_token_converter.py # 使用镜像代码
python ../examples/cpp/multi_gpu_gpt/bloom_tokenizer.py # 若重新编译
其中,`tokenizer`为原模型路径 其中,`tokenizer`为原模型路径
``` ```
测试数据:"Translate to English: Je t’aime." (token id: 153772, 427, 9522, 6395, 76721, 68258, 17),使用的加速卡:1张 DCU-Z100L-32G 测试数据:"Translate to English: Je t’aime." (token id: 153772, 427, 9522, 6395, 76721, 68258, 17),使用的加速卡:1张 DCU-Z100L-32G
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment