Skip to content

GitLab

Commit 2bd6a72f authored by zhaoying1's avatar zhaoying1
Browse files

Update lora_13B.sh

parent 2de27772
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 3 deletions
lora_13B.sh
View file @ 2bd6a72f
...@@ -11,12 +11,10 @@ HIP_VISIBLE_DEVICES=2 deepspeed --num_gpus=1 --master_port $MASTER_PORT src/trai ...@@ -11,12 +11,10 @@ HIP_VISIBLE_DEVICES=2 deepspeed --num_gpus=1 --master_port $MASTER_PORT src/trai
--output_dir ./checkpoint_13b \ --output_dir ./checkpoint_13b \
--overwrite_output_dir \ --overwrite_output_dir \
--per_device_train_batch_size 16 \ --per_device_train_batch_size 16 \
--per_device_eval_batch_size 1 \
--lr_scheduler_type cosine \ --lr_scheduler_type cosine \
--logging_steps 1 \ --logging_steps 1 \
--save_steps 1000 \ --save_steps 1000 \
--eval_steps 1000 \
--learning_rate te-5 \ --learning_rate te-5 \
--gradient_accumulation_steps 1 \ --gradient_accumulation_steps 1 \
--bf16 \ --bf16 \
--deepspeed bf16_deepspeed.json \ --deepspeed bf16_deepspeed.json \
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment