Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
Megatron-LM
Commits
edb026d5
Commit
edb026d5
authored
Apr 17, 2025
by
wxj
Browse files
Update README.md
parent
a65d5678
Pipeline
#2653
passed with stage
Changes
1
Pipelines
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
15 additions
and
0 deletions
+15
-0
README.md
README.md
+15
-0
No files found.
README.md
View file @
edb026d5
...
@@ -210,6 +210,21 @@ DATA_PATH="/datasets/oscar-1GB-llama_text_document"
...
@@ -210,6 +210,21 @@ DATA_PATH="/datasets/oscar-1GB-llama_text_document"
在
`Llama_pretraining.log`
中查看训练日志
在
`Llama_pretraining.log`
中查看训练日志
# 微调
将hf格式转为pt格式
```
shell
python tools/checkpoint/convert.py
\
--model-type
GPT
\
--loader
llama_mistral
\
--saver
megatron
\
--target-tensor-parallel-size
1
\
--checkpoint-type
hf
\
--model-size
llama2-7Bf
\
--load-dir
/models/llama2/Llama-2-7b-hf/
\
--save-dir
./Llama-2-7b-megatron-lm-0108
\
--tokenizer-model
/models/llama2/Llama-2-7b-hf
```
# 参考
# 参考
-
[
README_ORIGIN
](
README_ORIGIN.md
)
-
[
README_ORIGIN
](
README_ORIGIN.md
)
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment