Commit b26bccb5 authored by zhuwenwen's avatar zhuwenwen
Browse files

add modules and update readme

parent 21de54f8
[submodule "fastertransformer"]
path = fastertransformer
url = http://developer.hpccube.com/codes/aicomponent/fastertransformer.git
branch = develop
\ No newline at end of file
......@@ -2,7 +2,7 @@
* @Author: zhuww
* @email: zhuww@sugon.com
* @Date: 2023-09-08 11:08:07
* @LastEditTime: 2023-09-19 14:10:01
* @LastEditTime: 2023-11-08 17:47:01
-->
# BLOOM
......@@ -48,6 +48,23 @@ docker run -it --name llama --shm-size=32G --device=/dev/kfd --device=/dev/dri/
## 推理
### 编译
```bash
git clone http://developer.hpccube.com/codes/modelzoo/bloom_fastertransformer.git
cd bloom_fastertransformer
mkdir build
cd build
git submodule init && git submodule update
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_MULTI_GPU=ON -DCMAKE_CXX_COMPILER=nvcc ..
export C_INCLUDE_PATH=$PWD/_deps/googletest-src/googletest/include${C_INCLUDE_PATH:+:${C_INCLUDE_PATH}}
export CPLUS_INCLUDE_PATH=$PWD/_deps/googletest-src/googletest/include${CPLUS_INCLUDE_PATH:+:${CPLUS_INCLUDE_PATH}}
make -j12
运行前:
export LD_LIBRARY_PATH=$PWD/src/fastertransformer/utils/gemm_test/CMakeFiles/gpt_gemm_func.dir:$LD_LIBRARY_PATH
```
注:若不重新编译,直接使用测试目录代码即可。
### 模型下载
[bloom 7B](https://huggingface.co/bigscience/bloomz-7b1-mt)
......@@ -110,7 +127,8 @@ build/
```
执行一下命令可以解析out结果:
```bash
python bloom_token_converter.py
python bloom_token_converter.py # 使用镜像代码
python ../examples/cpp/multi_gpu_gpt/bloom_tokenizer.py # 若重新编译
其中,`tokenizer`为原模型路径
```
测试数据:"Translate to English: Je t’aime." (token id: 153772, 427, 9522, 6395, 76721, 68258, 17),使用的加速卡:1张 DCU-Z100L-32G
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment