Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
InstructBLIP_pytorch
Commits
e25524e9
Commit
e25524e9
authored
Sep 13, 2024
by
dongchy920
Browse files
Update README.md, icon.png, model.properties files
parent
4a835b73
Changes
3
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
11 additions
and
2 deletions
+11
-2
README.md
README.md
+10
-1
icon.png
icon.png
+0
-0
model.properties
model.properties
+1
-1
No files found.
README.md
View file @
e25524e9
...
...
@@ -68,6 +68,9 @@ export HF_ENDPOINT=https://hf-mirror.com
```
## 数据集
下载
[
ScienceQA
](
https://huggingface.co/datasets/derek-thomas/ScienceQA/tree/main
)
数据集
SCNet快速下载链接:
-
http://113.200.138.88:18080/aimodels/findsource-dependency/vicuna-13b-v1.1
对ScienceQA进行预处理:
```
# 修改预处理代码中scienceqa数据集解压位置
...
...
@@ -79,6 +82,12 @@ python scienceqa_data_preprocess.py
```
## 训练
使用预训练权重
[
vicuna-7b-v1.1
](
https://huggingface.co/lmsys/vicuna-7b-v1.1
)
或者
[
vicuna-13b-v1.1
](
https://huggingface.co/lmsys/vicuna-13b-v1.1
)
进行LoRA微调
SCNet vicuna-7b-v1.1快速下载链接:
-
http://113.200.138.88:18080/aimodels/findsource-dependency/minigpt-4
SCNet vicuna-13b-v1.1快速下载链接:
-
http://113.200.138.88:18080/aimodels/findsource-dependency/vicuna-13b-v1.1
修改lavis/config/datasets/scienceqa/default.yaml的数据集路径
开始训练:
...
...
@@ -115,7 +124,7 @@ python projects/instructblip/run_demo.py
## 应用场景
### 算法类别
多模态
文生图
### 热点应用行业
AIGC,设计,教育
...
...
icon.png
0 → 100644
View file @
e25524e9
64.5 KB
model.properties
View file @
e25524e9
...
...
@@ -5,7 +5,7 @@ modelName=InstructBLIP_pytorch
# 模型描述
modelDescription
=
InstructBLIP是BLIP系列的最新模型,将LLM领域的指令微调技术应用到视觉语言多模态领域中,在多项视觉和语言任务取得SOTA。
# 应用场景
appScenario
=
推理,训练,
多模态
,AIGC,设计,教育
appScenario
=
推理,训练,
文生图
,AIGC,设计,教育
# 框架类型
frameType
=
pytorch
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment