Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
llama3_pytorch
Commits
6f267cd1
Commit
6f267cd1
authored
May 24, 2024
by
Rayyyyy
Browse files
Modify in README
parent
15d1457a
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
8 additions
and
3 deletions
+8
-3
README.md
README.md
+7
-2
model.properties
model.properties
+1
-1
No files found.
README.md
View file @
6f267cd1
...
...
@@ -133,7 +133,7 @@ torchrun --nproc_per_node 1 example_chat_completion.py \
--max_seq_len
512
--max_batch_size
6
```
## 多轮对话
##
#
多轮对话
1.
确认环境安装及模型下载完毕;
2.
修改
[
chat.sh
](
./chat.sh
)
文件中的
`--ckpt_dir`
、
`--tokenizer_path`
参数为本地模型地址,
`--max_seq_len`
根据自身需求进行修改,调整该值可以增加多轮对话模型的记忆长度,不过需要注意的是这可能会增加模型运算的时间和内存需求;
3.
执行:
...
...
@@ -141,12 +141,15 @@ torchrun --nproc_per_node 1 example_chat_completion.py \
bash chat.sh
```
##
Evaluation
##
# 验证
1.
安装
`llama-recipes`
、
`lm-eval`
库
```
bash
# llama-recipes 下载
git clone http://developer.hpccube.com/codes/chenych/llama-recipes.git
cd
llama-recipes
# 移动exact_match.py文件
mv
exact_match.py ~/.cache/huggingface/evaluate/downloads/
cd
../
# lm-eval 下载
git clone http://developer.hpccube.com/codes/chenych/lm-evaluation-harness.git
cd
./lm-evaluation-harness/
...
...
@@ -156,7 +159,9 @@ pip install -e .
2.
修改待测模型
**pretrained**
参数地址,例如
`/home/Meta-Llama-3-8B-Instruct`
,特别地,当前仅支持
`hellaswag`
数据集进行测试验证。执行以下命令:
```
bash
cd
/path_of/llama-recipes/recipes/evaluation
# 必须添加HF_ENDPOINT环境变量
export
HF_ENDPOINT
=
https://hf-mirror.com
# 执行
python eval.py
--model
hf
--model_args
pretrained
=
/home/llama3/Meta-Llama-3-8B-Instruct,dtype
=
"float"
--tasks
hellaswag
--device
cuda
--batch_size
8
```
<div
align=
center
>
...
...
model.properties
View file @
6f267cd1
...
...
@@ -5,6 +5,6 @@ modelName=llama3_pytorch
# 模型描述
modelDescription
=
Meta最新开源模型llama3
# 应用场景
appScenario
=
推理,对话问答,制造,广媒,家居,教育
appScenario
=
推理,
训练,
对话问答,制造,广媒,家居,教育
# 框架类型
frameType
=
pytorch
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment