Commit 15d1457a authored by Rayyyyy's avatar Rayyyyy
Browse files

update lm-evaluation-harness llama-recipes in README

parent fd1124b0
......@@ -145,28 +145,18 @@ bash chat.sh
1. 安装`llama-recipes``lm-eval`
```bash
# llama-recipes 下载
git clone https://github.com/meta-llama/llama-recipes.git
cd ./llama-recipes/recipes/evaluation/
# 修改eval.py第15行代码,将from lm_eval.utils import make_table 改为
from lm_eval.evaluator import make_table
# 修改eval.py第121行代码,num_fewshot参数的默认值改为0
default=0
# 修改eval.py第215行代码,use_cache=args.use_cache 修改为
no_cache=args.use_cache
# 返回根目录
cd ~
git clone http://developer.hpccube.com/codes/chenych/llama-recipes.git
mv exact_match.py ~/.cache/huggingface/evaluate/downloads/
# lm-eval 下载
git clone http://developer.hpccube.com/codes/chenych/lm-evaluation-harness.git
cd ./lm-evaluation-harness/
pip install -e .
cd ../
```
2. 修改待测模型**pretrained**参数地址,例如`/home/Meta-Llama-3-8B-Instruct`,特别地,当前仅支持`hellaswag`数据集进行测试验证。执行以下命令:
```bash
cd /path_of/llama-recipes/recipes/evaluation
export HF_ENDPOINT=https://hf-mirror.com
python eval.py --model hf --model_args pretrained=/home/llama3/Meta-Llama-3-8B-Instruct,dtype="float" --tasks hellaswag --device cuda --batch_size 8
```
<div align=center>
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment