Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
bert-large_oneflow
Commits
54727d34
Commit
54727d34
authored
Jul 26, 2024
by
chenzk
Browse files
Merge branch 'main' into 'main'
add icon See merge request
!2
parents
124d7959
7c1437bb
Pipeline
#1493
failed with stages
Changes
3
Pipelines
2
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
49 additions
and
3 deletions
+49
-3
.gitlab-ci.yml
.gitlab-ci.yml
+41
-0
README.md
README.md
+8
-3
icon.png
icon.png
+0
-0
No files found.
.gitlab-ci.yml
0 → 100644
View file @
54727d34
# This file is a template, and might need editing before it works on your project.
# To contribute improvements to CI/CD templates, please follow the Development guide at:
# https://docs.gitlab.com/ee/development/cicd/templates.html
# This specific template is located at:
# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Bash.gitlab-ci.yml
# See https://docs.gitlab.com/ee/ci/yaml/README.html for all available options
# you can delete this line if you're not using Docker
image
:
busybox:latest
before_script
:
-
echo "Before script section"
-
echo "For example you might run an update here or install a build dependency"
-
echo "Or perhaps you might print out some debugging details"
after_script
:
-
echo "After script section"
-
echo "For example you might do some cleanup here"
build1
:
stage
:
build
script
:
-
echo "Do your build here"
test1
:
stage
:
test
script
:
-
echo "Do a test here"
-
echo "For example run a test suite"
test2
:
stage
:
test
script
:
-
echo "Do another parallel test here"
-
echo "For example run a lint test"
deploy1
:
stage
:
deploy
script
:
-
echo "Do your deploy here"
README.md
View file @
54727d34
...
@@ -49,6 +49,13 @@ pip3 install -e . -i https://mirrors.aliyun.com/pypi/simple
...
@@ -49,6 +49,13 @@ pip3 install -e . -i https://mirrors.aliyun.com/pypi/simple
├── bert-base-chinese-vocab
├── bert-base-chinese-vocab
├── gpt2-merges
├── gpt2-merges
└── gpt2-vocab
└── gpt2-vocab
训练数据:
-
https://oneflow-static.oss-cn-beijing.aliyuncs.com/ci-files/dataset/libai/gpt_dataset/gpt2-merges.txt
-
https://oneflow-static.oss-cn-beijing.aliyuncs.com/ci-files/dataset/libai/gpt_dataset/gpt2-vocab.json
数据集快速下载中心:
[
SCNet AIDatasets
](
http://113.200.138.88:18080/aidatasets
)
,项目中数据集可从快速下载通道下载:
[
gpt_dataset
](
http://113.200.138.88:18080/aidatasets/project-dependency/bert-large_oneflow
)
。
## 训练
## 训练
该预训练脚本运行环境为1节点,4张DCU-Z100-16G。
该预训练脚本运行环境为1节点,4张DCU-Z100-16G。
...
@@ -72,8 +79,6 @@ train.dist.pipeline_parallel_size = 1
...
@@ -72,8 +79,6 @@ train.dist.pipeline_parallel_size = 1
### 精度
### 精度
训练数据:
[
https://oneflow-static.oss-cn-beijing.aliyuncs.com/ci-files/dataset/libai/gpt_dataset
](
链接
)
使用的GPGPU:4张DCU-Z100-16G。
使用的GPGPU:4张DCU-Z100-16G。
模型精度:
模型精度:
...
@@ -86,7 +91,7 @@ train.dist.pipeline_parallel_size = 1
...
@@ -86,7 +91,7 @@ train.dist.pipeline_parallel_size = 1
### 算法类别
### 算法类别
`
自然语言处理
`
`
对话问答
`
### 热点应用行业
### 热点应用行业
...
...
icon.png
0 → 100644
View file @
54727d34
53.8 KB
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment