Commit 82cd38ce authored by suily's avatar suily
Browse files

Update README.md

parent b1270ba3
...@@ -81,7 +81,7 @@ deepspeed0.14.2只能使用transformers4.31.0,此版本不支持k100ai使用bf ...@@ -81,7 +81,7 @@ deepspeed0.14.2只能使用transformers4.31.0,此版本不支持k100ai使用bf
``` ```
pip show pip #查看依赖库安装地址site-packages pip show pip #查看依赖库安装地址site-packages
1、site-packages/transformers/utils/import_utils.py,修改def is_torch_bf16_gpu_available(): 1、site-packages/transformers/utils/import_utils.py,修改def is_torch_bf16_gpu_available()277行后:
... ...
#TODO:if torch.cuda.is_available() and torch.version.cuda is not None: #TODO:if torch.cuda.is_available() and torch.version.cuda is not None:
if torch.cuda.is_available(): if torch.cuda.is_available():
...@@ -93,8 +93,9 @@ if torch.cuda.is_available(): ...@@ -93,8 +93,9 @@ if torch.cuda.is_available():
return False return False
else: else:
return False return False
return True
2、site-packages/transformers/utils/import_utils.py,修改def is_torch_tf32_available(): 2、site-packages/transformers/utils/import_utils.py,修改def is_torch_tf32_available()322行后:
... ...
#TODO:if not torch.cuda.is_available() or torch.version.cuda is None: #TODO:if not torch.cuda.is_available() or torch.version.cuda is None:
if not torch.cuda.is_available(): if not torch.cuda.is_available():
...@@ -181,7 +182,7 @@ huggingface-cli download --resume-download THUDM/chatglm3-6b --local-dir checkpo ...@@ -181,7 +182,7 @@ huggingface-cli download --resume-download THUDM/chatglm3-6b --local-dir checkpo
``` ```
以基于VTimeLLM-7B的VTimeLLM为例,模型目录结构如下: 以基于VTimeLLM-7B的VTimeLLM为例,模型目录结构如下:
``` ```
VTimeLLM: VTimeLLM/checkpoints:
── clip ── clip
│   └── ViT-L-14.pt │   └── ViT-L-14.pt
── vicuna-7b-v1.5 ── vicuna-7b-v1.5
...@@ -190,6 +191,7 @@ VTimeLLM: ...@@ -190,6 +191,7 @@ VTimeLLM:
以基于Vicuna v1.5的VTimeLLM为例,训练运行代码: 以基于Vicuna v1.5的VTimeLLM为例,训练运行代码:
``` ```
# 若使用小数据集需修改sh文件,例如per_device_train_batch_size=gradient_accumulation_steps=2【per_device_train_batch_size*n_gpus<10不能超过小数据集的size】
cd VTimeLLM cd VTimeLLM
wandb off wandb off
sh scripts/stage1.sh sh scripts/stage1.sh
...@@ -210,7 +212,7 @@ tar -xzvf vtimellm-vicuna-v1-5-7b.tar.gz ...@@ -210,7 +212,7 @@ tar -xzvf vtimellm-vicuna-v1-5-7b.tar.gz
``` ```
以基于Vicuna v1.5的VTimeLLM为例,模型目录结构如下: 以基于Vicuna v1.5的VTimeLLM为例,模型目录结构如下:
``` ```
VTimeLLM: VTimeLLM/checkpoints:
── clip ── clip
│   └── ViT-L-14.pt │   └── ViT-L-14.pt
── vtimellm-vicuna-v1-5-7b-stage1 ── vtimellm-vicuna-v1-5-7b-stage1
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment