Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
llama_inference_pytorch
Commits
b0177f9b
Commit
b0177f9b
authored
Oct 16, 2023
by
hepj987
Browse files
标准格式
parent
de4884a1
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
19 additions
and
11 deletions
+19
-11
README.md
README.md
+16
-8
llama-inf.jpg
llama-inf.jpg
+0
-0
model.properties
model.properties
+3
-3
No files found.
README.md
View file @
b0177f9b
...
...
@@ -45,8 +45,6 @@ SwiGLU 激活函数 [PaLM]。使用 SwiGLU 激活函数替换 ReLU 非线性以

# LLAMA推理
## 环境配置
推荐使用docker方式运行,提供
[
光源
](
https://www.sourcefind.cn/
)
拉取的docker镜像:
...
...
@@ -65,13 +63,17 @@ pip install tensor_parallel==1.2.5 --no-dependencies
pip install transformers==4.28.1 sentencepiece==0.1.99
```
## 数据集
无
## 模型权重下载
[
llama chat 7B
](
https://huggingface.co/Linly-AI/ChatFlow-7B
)
[
llama caht 13b
](
https://huggingface.co/Linly-AI/ChatFlow-13B
)
##
模型
推理
## 推理
### 单卡推理
...
...
@@ -121,6 +123,16 @@ SPM_PATH 为下载的llama 模型tokenizer路径
#其他参数与单卡推理相同
```
## result
`多轮对话`

## 精度
无
## 应用场景
### 算法类别
...
...
@@ -131,14 +143,10 @@ SPM_PATH 为下载的llama 模型tokenizer路径
`nlp,智能聊天助手`
## 多轮对话推理效果

## 源码仓库及问题反馈
https://developer.hpccube.com/codes/hepj/llama_pytorch
## 参考
## 参考
资料
https://github.com/ProjectD-AI/llama_inference
\ No newline at end of file
llama-inf.jpg
0 → 100644
View file @
b0177f9b
71.5 KB
model.properties
View file @
b0177f9b
# 模型唯一标识
modelCode
=
405
# 模型名称
modelName
=
LL
aMA_inference_pytorch
modelName
=
ll
aMA_inference_pytorch
# 模型描述
modelDescription
=
基于tencentpretrain框架的llama模型推理
# 应用场景
appScenario
=
推理,
inference,
nlp,智能聊天助手
appScenario
=
推理,nlp,智能聊天助手
# 框架类型
frameType
=
P
ytorch,
T
ransformers,
D
eepspeed
frameType
=
p
ytorch,
t
ransformers,
d
eepspeed
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment