Skip to content

GitLab

Unverified Commit 55dcd305 authored by BlueRum's avatar BlueRum Committed by GitHub
Browse files

[chatgpt] fix readme (#3025)

parent 287d6049
Hide whitespace changes
Inline Side-by-side
Showing with 5 additions and 5 deletions
applications/ChatGPT/examples/README.md
View file @ 55dcd305
...@@ -15,9 +15,9 @@ Use these code to train your reward model. ...@@ -15,9 +15,9 @@ Use these code to train your reward model.
```shell ```shell
# Naive reward model training # Naive reward model training
python train_reward_model.py --pretrain <your model path> python train_reward_model.py --pretrain <your model path> --model <your model type> --strategy naive
# if to use LoRA # use colossalai_zero2
python train_reward_model.py --pretrain <your model path> --lora_rank 16 torchrun --standalone --nproc_per_node=2 train_reward_model.py --pretrain <your model path> --model <your model type> --strategy colossalai_zero2
``` ```
## Train with dummy prompt data (Stage 3) ## Train with dummy prompt data (Stage 3)
...@@ -44,7 +44,7 @@ DDP strategy and ColossalAI strategy support multi GPUs training: ...@@ -44,7 +44,7 @@ DDP strategy and ColossalAI strategy support multi GPUs training:
# run DDP on 2 GPUs # run DDP on 2 GPUs
torchrun --standalone --nproc_per_node=2 train_dummy.py --strategy ddp torchrun --standalone --nproc_per_node=2 train_dummy.py --strategy ddp
# run ColossalAI on 2 GPUs # run ColossalAI on 2 GPUs
torchrun --standalone --nproc_per_node=2 train_dummy.py --strategy colossalai torchrun --standalone --nproc_per_node=2 train_dummy.py --strategy colossalai_zero2
``` ```
## Train with real prompt data (Stage 3) ## Train with real prompt data (Stage 3)
...@@ -63,7 +63,7 @@ python train_prompts.py prompts.csv --strategy naive ...@@ -63,7 +63,7 @@ python train_prompts.py prompts.csv --strategy naive
# run DDP on 2 GPUs # run DDP on 2 GPUs
torchrun --standalone --nproc_per_node=2 train_prompts.py prompts.csv --strategy ddp torchrun --standalone --nproc_per_node=2 train_prompts.py prompts.csv --strategy ddp
# run ColossalAI on 2 GPUs # run ColossalAI on 2 GPUs
torchrun --standalone --nproc_per_node=2 train_prompts.py prompts.csv --strategy colossalai torchrun --standalone --nproc_per_node=2 train_prompts.py prompts.csv --strategy colossalai_zero2
``` ```
## Inference example(After Stage3) ## Inference example(After Stage3)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment