Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
Gemma-2_pytorch
Commits
154afe21
Commit
154afe21
authored
Nov 05, 2024
by
chenych
Browse files
Delete git submodule
parent
ead0d966
Pipeline
#1844
canceled with stages
Changes
4
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
80 additions
and
2 deletions
+80
-2
Llama-Factory
Llama-Factory
+0
-1
README.md
README.md
+1
-1
llama-factory-v0.8.3/examples/train_full/gemma2_full_sft_ds3.yaml
...ctory-v0.8.3/examples/train_full/gemma2_full_sft_ds3.yaml
+39
-0
llama-factory-v0.8.3/examples/train_lora/gemma2_lora_sft_ds3.yaml
...ctory-v0.8.3/examples/train_lora/gemma2_lora_sft_ds3.yaml
+40
-0
No files found.
Llama-Factory
@
8bcc3dc7
Subproject commit 8bcc3dc7ab44595f0cf07607f0937f3d2ee428d7
README.md
View file @
154afe21
...
...
@@ -58,7 +58,7 @@ DTK驱动: dtk24.04.2
python: 3.10
torch: 2.1.0
llama-factory: 0.8.3
transformers: 4.4
2.4
transformers:
>=
4.4
1.2
vllm: 0.5.4
```
`Tips:以上dtk驱动、python、torch等DCU相关工具版本需要严格一一对应`
...
...
llama-factory-v0.8.3/examples/train_full/gemma2_full_sft_ds3.yaml
0 → 100644
View file @
154afe21
### model
model_name_or_path
:
google/gemma-2-2b
### method
stage
:
sft
do_train
:
true
finetuning_type
:
full
deepspeed
:
examples/deepspeed/ds_z3_config.json
### dataset
dataset
:
identity,alpaca_en_demo
template
:
gemma
cutoff_len
:
1024
max_samples
:
1000
overwrite_cache
:
true
preprocessing_num_workers
:
16
### output
output_dir
:
saves/gemma-2-2b/full/sft
logging_steps
:
10
save_steps
:
500
plot_loss
:
true
overwrite_output_dir
:
true
### train
per_device_train_batch_size
:
1
gradient_accumulation_steps
:
2
learning_rate
:
1.0e-5
num_train_epochs
:
3.0
lr_scheduler_type
:
cosine
warmup_ratio
:
0.1
bf16
:
true
ddp_timeout
:
180000000
### eval
val_size
:
0.1
per_device_eval_batch_size
:
1
eval_strategy
:
steps
eval_steps
:
500
llama-factory-v0.8.3/examples/train_lora/gemma2_lora_sft_ds3.yaml
0 → 100644
View file @
154afe21
### model
model_name_or_path
:
google/gemma-2-2b
### method
stage
:
sft
do_train
:
true
finetuning_type
:
lora
lora_target
:
all
deepspeed
:
examples/deepspeed/ds_z3_config.json
### dataset
dataset
:
identity,alpaca_en_demo
template
:
gemma
cutoff_len
:
1024
max_samples
:
1000
overwrite_cache
:
true
preprocessing_num_workers
:
16
### output
output_dir
:
saves/gemma-2-2b/lora/sft
logging_steps
:
10
save_steps
:
500
plot_loss
:
true
overwrite_output_dir
:
true
### train
per_device_train_batch_size
:
1
gradient_accumulation_steps
:
2
learning_rate
:
1.0e-4
num_train_epochs
:
3.0
lr_scheduler_type
:
cosine
warmup_ratio
:
0.1
bf16
:
true
ddp_timeout
:
180000000
### eval
val_size
:
0.1
per_device_eval_batch_size
:
1
eval_strategy
:
steps
eval_steps
:
500
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment