Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
suily
VTimeLLM_pytorch
Commits
82cd38ce
"...composable_kernel_rocm.git" did not exist on "05d382183f431d6166c814015527f473814480b6"
Commit
82cd38ce
authored
Nov 28, 2024
by
suily
Browse files
Update README.md
parent
b1270ba3
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
4 deletions
+6
-4
README.md
README.md
+6
-4
No files found.
README.md
View file @
82cd38ce
...
@@ -81,7 +81,7 @@ deepspeed0.14.2只能使用transformers4.31.0,此版本不支持k100ai使用bf
...
@@ -81,7 +81,7 @@ deepspeed0.14.2只能使用transformers4.31.0,此版本不支持k100ai使用bf
```
```
pip show pip #查看依赖库安装地址site-packages
pip show pip #查看依赖库安装地址site-packages
1、site-packages/transformers/utils/import_utils.py,修改def is_torch_bf16_gpu_available():
1、site-packages/transformers/utils/import_utils.py,修改def is_torch_bf16_gpu_available()
277行后
:
...
...
#TODO:if torch.cuda.is_available() and torch.version.cuda is not None:
#TODO:if torch.cuda.is_available() and torch.version.cuda is not None:
if torch.cuda.is_available():
if torch.cuda.is_available():
...
@@ -93,8 +93,9 @@ if torch.cuda.is_available():
...
@@ -93,8 +93,9 @@ if torch.cuda.is_available():
return False
return False
else:
else:
return False
return False
return True
2、site-packages/transformers/utils/import_utils.py,修改def is_torch_tf32_available():
2、site-packages/transformers/utils/import_utils.py,修改def is_torch_tf32_available()
322行后
:
...
...
#TODO:if not torch.cuda.is_available() or torch.version.cuda is None:
#TODO:if not torch.cuda.is_available() or torch.version.cuda is None:
if not torch.cuda.is_available():
if not torch.cuda.is_available():
...
@@ -181,7 +182,7 @@ huggingface-cli download --resume-download THUDM/chatglm3-6b --local-dir checkpo
...
@@ -181,7 +182,7 @@ huggingface-cli download --resume-download THUDM/chatglm3-6b --local-dir checkpo
```
```
以基于VTimeLLM-7B的VTimeLLM为例,模型目录结构如下:
以基于VTimeLLM-7B的VTimeLLM为例,模型目录结构如下:
```
```
VTimeLLM:
VTimeLLM
/checkpoints
:
── clip
── clip
│ └── ViT-L-14.pt
│ └── ViT-L-14.pt
── vicuna-7b-v1.5
── vicuna-7b-v1.5
...
@@ -190,6 +191,7 @@ VTimeLLM:
...
@@ -190,6 +191,7 @@ VTimeLLM:
以基于Vicuna v1.5的VTimeLLM为例,训练运行代码:
以基于Vicuna v1.5的VTimeLLM为例,训练运行代码:
```
```
# 若使用小数据集需修改sh文件,例如per_device_train_batch_size=gradient_accumulation_steps=2【per_device_train_batch_size*n_gpus<10不能超过小数据集的size】
cd VTimeLLM
cd VTimeLLM
wandb off
wandb off
sh scripts/stage1.sh
sh scripts/stage1.sh
...
@@ -210,7 +212,7 @@ tar -xzvf vtimellm-vicuna-v1-5-7b.tar.gz
...
@@ -210,7 +212,7 @@ tar -xzvf vtimellm-vicuna-v1-5-7b.tar.gz
```
```
以基于Vicuna v1.5的VTimeLLM为例,模型目录结构如下:
以基于Vicuna v1.5的VTimeLLM为例,模型目录结构如下:
```
```
VTimeLLM:
VTimeLLM
/checkpoints
:
── clip
── clip
│ └── ViT-L-14.pt
│ └── ViT-L-14.pt
── vtimellm-vicuna-v1-5-7b-stage1
── vtimellm-vicuna-v1-5-7b-stage1
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment