Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
Megatron-LM
Commits
527584c3
"git@developer.sourcefind.cn:modelzoo/resnet50_tensorflow.git" did not exist on "d3258fe9f42b2f0e146a942c89afb95f9108498d"
Commit
527584c3
authored
Nov 29, 2024
by
wxj
Browse files
Update README.md
parent
1a317380
Pipeline
#2022
passed with stage
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
README.md
README.md
+1
-1
No files found.
README.md
View file @
527584c3
...
@@ -71,7 +71,7 @@ python tools/preprocess_data.py \
...
@@ -71,7 +71,7 @@ python tools/preprocess_data.py \
# 参数说明
# 参数说明
# --input 输入数据集路径,即oscar-1GB.jsonl.xz解压后的文件路径
# --input 输入数据集路径,即oscar-1GB.jsonl.xz解压后的文件路径
# --output-prefix 输出数据路径,处理后会自动加上_text_document后缀
# --output-prefix 输出数据路径
(需要输出目录已创建)
,处理后会自动加上_text_document后缀
# --vocab-file 下载的gpt2-vocab.json词表文件路径
# --vocab-file 下载的gpt2-vocab.json词表文件路径
# --tokenizer-type tokenizer类型
# --tokenizer-type tokenizer类型
# --merge-file 下载的gpt2-merges.txt文件路径
# --merge-file 下载的gpt2-merges.txt文件路径
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment