Skip to content

GitLab

Commit 428e38f1 authored by wxj's avatar wxj
Browse files

Update qwen1.5_14b.sh

parent 9d8e86df
Pipeline #2575 passed with stage
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 1 deletion
examples/qwen/qwen1.5_14b.sh
View file @ 428e38f1
...@@ -99,6 +99,7 @@ TRAINING_ARGS=( ...@@ -99,6 +99,7 @@ TRAINING_ARGS=(
--hidden-dropout 0 --hidden-dropout 0
# --no-gradient-accumulation-fusion # --no-gradient-accumulation-fusion
--swiglu --swiglu
--add-qkv-bias
--lr 3.0e-5 --lr 3.0e-5
--lr-decay-style cosine --lr-decay-style cosine
--min-lr 3.0e-6 --min-lr 3.0e-6
...@@ -249,4 +250,4 @@ case ${LOCAL_RANK} in ...@@ -249,4 +250,4 @@ case ${LOCAL_RANK} in
numactl --cpunodebind=7 --membind=7 ${APP} numactl --cpunodebind=7 --membind=7 ${APP}
# hipprof --hip-trace --trace-off numactl --cpunodebind=0 --membind=0 ${APP} # hipprof --hip-trace --trace-off numactl --cpunodebind=0 --membind=0 ${APP}
;; ;;
esac esac
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment