Commit 185e8d8c authored by luopl's avatar luopl
Browse files

Update README.md

parent aeeb1b71
...@@ -101,33 +101,30 @@ pip install e . ...@@ -101,33 +101,30 @@ pip install e .
``` ```
## 数据集 ## 数据集
运行推理时会自动连接huggingface下载最新数据集于缓存目录 运行推理时会自动连接huggingface下载最新数据集于缓存目录,如无法连接到huggingface,可通过export HF_ENDPOINT=https://hf-mirror.com 设置镜像地址
``` ```
#数据集格式 #数据集格式
piqa/ hellaswag/
└── plain_text └── default
└── 1.1.0 └── 0.1.0
├── 6c611c1a9bf220943c4174e117d3b660859665baf1d43156230116185312d011 ├── 362ac471216900f3f7c021863caac4eb7886347d0f76d90b6b4361f59ffea4d7
│ ├── cache-081f361bf081c0bf.arrow
│ ├── cache-5d43362a7601c065.arrow
│ ├── dataset_info.json │ ├── dataset_info.json
│ ├── piqa-test.arrow │ ├── hellaswag-test.arrow
│ ├── piqa-train.arrow │ ├── hellaswag-train.arrow
│ └── piqa-validation.arrow │ └── hellaswag-validation.arrow
├── 6c611c1a9bf220943c4174e117d3b660859665baf1d43156230116185312d011_builder.lock ├── 362ac471216900f3f7c021863caac4eb7886347d0f76d90b6b4361f59ffea4d7_builder.lock
└── 6c611c1a9bf220943c4174e117d3b660859665baf1d43156230116185312d011.incomplete_info.lock └── 362ac471216900f3f7c021863caac4eb7886347d0f76d90b6b4361f59ffea4d7.incomplete_info.lock
... ...
``` ```
也可下载离线数据,放于缓存目录~/.cache/huggingface/hub/datasets/,根据自己的缓存地址存放: 也可下载离线数据,放于缓存目录~/.cache/huggingface/datasets/,根据自己的缓存地址存放:
数据集SCNet快速下载链接[datasets](http://113.200.138.88:18080/aidatasets/mamba2_data_test) 数据集SCNet快速下载链接[datasets](http://113.200.138.88:18080/aidatasets/mamba2_data_test)
或者使用modelscope下载相关数据集
```
#数据集下载示例
#需要pip install modelscope安装modelscope
from modelscope.msdatasets import MsDataset
ds = MsDataset.load('opencompass/piqa')
```
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment