Skip to content

GitLab

Commit 2de27772 authored by zhaoying1's avatar zhaoying1
Browse files

Update lora_7B.sh

parent d767e5f5
Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 3 deletions
lora_7B.sh
View file @ 2de27772
...@@ -11,14 +11,11 @@ deepspeed --include localhost:0 --master_port $MASTER_PORT src/train_bash.py \ ...@@ -11,14 +11,11 @@ deepspeed --include localhost:0 --master_port $MASTER_PORT src/train_bash.py \
--output_dir ./checkpoint_7b \ --output_dir ./checkpoint_7b \
--overwrite_output_dir \ --overwrite_output_dir \
--per_device_train_batch_size 32 \ --per_device_train_batch_size 32 \
--per_device_eval_batch_size 1 \
--lr_scheduler_type cosine \ --lr_scheduler_type cosine \
--logging_steps 1 \ --logging_steps 1 \
--save_steps 1000 \ --save_steps 1000 \
--eval_steps 1000 \
--learning_rate 5e-5 \ --learning_rate 5e-5 \
--gradient_accumulation_steps 1 \ --gradient_accumulation_steps 1 \
--max_steps 3000 \ --max_steps 3000 \
--evaluation_strategy steps \
--bf16 \ --bf16 \
--deepspeed bf16_deepspeed.json \ --deepspeed bf16_deepspeed.json \
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment