Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
DISC-FinLLM_pytorch
Commits
4a637dad
Commit
4a637dad
authored
Oct 24, 2024
by
dcuai
Browse files
Update README.md ——data部分
parent
a1db219c
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
8 additions
and
4 deletions
+8
-4
README.md
README.md
+8
-4
No files found.
README.md
View file @
4a637dad
...
@@ -82,8 +82,8 @@ pip install transformers=4.40.1
...
@@ -82,8 +82,8 @@ pip install transformers=4.40.1
```
```
import json
import json
jsonl_file_path = '.
..
/data/dataset_new.jsonl'
jsonl_file_path = './data/dataset_new.jsonl'
json_file_path = '.
.
/data/dataset_new.json'
json_file_path = './data/dataset_new.json'
data = []
data = []
with open(jsonl_file_path, 'r', encoding='utf-8') as file:
with open(jsonl_file_path, 'r', encoding='utf-8') as file:
for line in file:
for line in file:
...
@@ -101,15 +101,19 @@ with open(json_file_path, 'w', encoding='utf-8') as file:
...
@@ -101,15 +101,19 @@ with open(json_file_path, 'w', encoding='utf-8') as file:
print(data)
print(data)
```
```
项目中已提供用于试验训练的迷你数据集,
训练数据目录结构如下,用于正常训练的完整数据集请按此目录结构进行制备:
训练数据目录结构如下,用于正常训练的完整数据集请按此目录结构进行制备
,
:
```
```
cp LLaMA-Factory/data/dataset_info.json data/
── data
── data
│ ├── computing_part.json
│ ├── computing_part.json
│ ├── consulting_part.json
│ ├── consulting_part.json
│ ├── retrieval_part.json
│ ├── retrieval_part.json
│ └── task_part.json
│ ├── task_part.json
│ └── dataset_info.json
│——————————
│——————————
```
```
项目中已提供用于试验训练的迷你数据集,即脚本中的默认数据集路径
[
LLaMA-Factory/data
](
https://developer.sourcefind.cn/codes/modelzoo/disc-finllm_pytorch/-/tree/main/data
)
## 训练
## 训练
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment