Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
Vary-toy_pytorch
Commits
b5dbd0eb
Commit
b5dbd0eb
authored
May 27, 2024
by
wanglch
Browse files
Update README.md
parent
2927283f
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
34 deletions
+1
-34
README.md
README.md
+1
-34
No files found.
README.md
View file @
b5dbd0eb
...
...
@@ -74,40 +74,7 @@ pip install ninja
无, 本项目暂未开放数据集
## 训练
需自己构建数据集
可参考本项目github论文
-
本项目gitlab地址
[
Ucas-HaoranWei/Vary-toy
](
https://github.com/Ucas-HaoranWei/Vary-toy/tree/main
)
```
deepspeed vary/train/train_qwen_vary.py --deepspeed /vary/zero_config/zero2.json
--model_name_or_path /Vary-toy/path/
--vision_tower /vit-large-patch14/path/
--freeze_vision_tower True
--freeze_lm_model False
--vision_select_layer -2
--use_im_start_end True
--bf16 True
--per_device_eval_batch_size 4
--gradient_accumulation_steps 1
--evaluation_strategy "no"
--save_strategy "steps"
--save_steps 5000
--save_total_limit 1
--weight_decay 0.
--warmup_ratio 0.03
--lr_scheduler_type "cosine"
--logging_steps 1 --tf32 True
--model_max_length 4096
--gradient_checkpointing True
--dataloader_num_workers 4
--report_to none
--per_device_train_batch_size 4
--num_train_epochs 1
--learning_rate 5e-5
--datasets data_name1+data_name2+data_name3
--output_dir /path/to/output/
```
无
## 推理
**需严格按照本仓库代码目录进行排列**
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment