Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
gpu-base-image-build
Commits
aa879d4a
"examples/git@developer.sourcefind.cn:OpenDAS/megatron-lm.git" did not exist on "0d99ae1fe84f8d191abe5ed1c2f4fdc5a9f9a773"
Commit
aa879d4a
authored
Oct 24, 2024
by
chenpangpang
Browse files
feat: 2.16.1的bug修复
parent
9eaaf126
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
22 additions
and
4 deletions
+22
-4
script/2_text_test.sh
script/2_text_test.sh
+11
-2
script/3_image_test.sh
script/3_image_test.sh
+11
-2
No files found.
script/2_text_test.sh
View file @
aa879d4a
...
@@ -9,8 +9,17 @@ fi
...
@@ -9,8 +9,17 @@ fi
if
[[
"
$1
"
==
*
"pytorch"
*
]]
;
then
if
[[
"
$1
"
==
*
"pytorch"
*
]]
;
then
docker run
--rm
--platform
=
linux/amd64
--gpus
all
-v
./
$TARGET_DIR
:/workspace
--workdir
/workspace/pytorch/gpt2
$1
python infer.py
;
fi
docker run
--rm
--platform
=
linux/amd64
--gpus
all
-v
./
$TARGET_DIR
:/workspace
--workdir
/workspace/pytorch/gpt2
$1
python infer.py
;
fi
if
[[
"
$1
"
==
*
"tensorflow"
*
]]
;
then
if
[[
"
$1
"
==
*
"tensorflow"
*
]]
;
then
docker run
--rm
--platform
=
linux/amd64
--gpus
all
-v
./
$TARGET_DIR
:/workspace
--workdir
/workspace/tensorflow/bert
$1
python infer.py
;
fi
tensorflow_version
=
$(
echo
"
$1
"
|
cut
-d
:
-f2
|
cut
-d-
-f1
)
# 当tensorflow版本为2.16.1时,不添加环境变量找不到cuda,所以需要这样执行验证。在正常交互式启动容器时,会默认激活/etc/bash.bashrc,可以正常找到cuda
if
[[
"
$tensorflow_version
"
==
"2.16.1"
]]
;
then
python_version
=
$(
echo
$1
|
awk
-F
'[-:]'
'{for(i=3;i<=NF;i++) if($i ~ /^py[0-9]+\.[0-9]+$/) {gsub(/^py/,"",$i); print $i; exit}}'
)
docker run
--rm
--platform
=
linux/amd64
--gpus
all
\
-e
CUDNN_PATH
=
"/opt/conda/lib/python
$python_version
/site-packages/nvidia/cudnn"
\
-e
LD_LIBRARY_PATH
=
"/opt/conda/lib/python
$python_version
/site-packages/nvidia/cudnn/lib:/usr/local/cuda/lib64"
\
-v
./
$TARGET_DIR
:/workspace
--workdir
/workspace/tensorflow/bert
$1
python infer.py
else
docker run
--rm
--platform
=
linux/amd64
--gpus
all
-v
./
$TARGET_DIR
:/workspace
--workdir
/workspace/tensorflow/bert
$1
python infer.py
;
fi
;
fi
if
[[
"
$1
"
==
*
"paddle"
*
]]
;
then
if
[[
"
$1
"
==
*
"paddle"
*
]]
;
then
TARGET_DIR
=
gpu-base-image-test/paddletest
TARGET_DIR
=
gpu-base-image-test/paddletest
docker run
--rm
--platform
=
linux/amd64
--gpus
all
-v
./
$TARGET_DIR
:/workspace
--workdir
/workspace
$1
python text.py
;
fi
docker run
--rm
--platform
=
linux/amd64
--gpus
all
-v
./
$TARGET_DIR
:/workspace
--workdir
/workspace
$1
python text.py
;
fi
...
...
script/3_image_test.sh
View file @
aa879d4a
...
@@ -9,8 +9,17 @@ fi
...
@@ -9,8 +9,17 @@ fi
if
[[
"
$1
"
==
*
"pytorch"
*
]]
;
then
if
[[
"
$1
"
==
*
"pytorch"
*
]]
;
then
docker run
--rm
--platform
=
linux/amd64
--gpus
all
-v
./
$TARGET_DIR
:/workspace
--workdir
/workspace/pytorch/stable-diffusion-v1-4
$1
python infer.py
;
fi
docker run
--rm
--platform
=
linux/amd64
--gpus
all
-v
./
$TARGET_DIR
:/workspace
--workdir
/workspace/pytorch/stable-diffusion-v1-4
$1
python infer.py
;
fi
if
[[
"
$1
"
==
*
"tensorflow"
*
]]
;
then
if
[[
"
$1
"
==
*
"tensorflow"
*
]]
;
then
docker run
--rm
--platform
=
linux/amd64
--gpus
all
-v
./
$TARGET_DIR
:/workspace
--workdir
/workspace/tensorflow/mnist
$1
python train.py
;
fi
tensorflow_version
=
$(
echo
"
$1
"
|
cut
-d
:
-f2
|
cut
-d-
-f1
)
# 当tensorflow版本为2.16.1时,不添加环境变量找不到cuda,所以需要这样执行验证。在正常交互式启动容器时,会默认激活/etc/bash.bashrc,可以正常找到cuda
if
[[
"
$tensorflow_version
"
==
"2.16.1"
]]
;
then
python_version
=
$(
echo
$1
|
awk
-F
'[-:]'
'{for(i=3;i<=NF;i++) if($i ~ /^py[0-9]+\.[0-9]+$/) {gsub(/^py/,"",$i); print $i; exit}}'
)
docker run
--rm
--platform
=
linux/amd64
--gpus
all
\
-e
CUDNN_PATH
=
"/opt/conda/lib/python
$python_version
/site-packages/nvidia/cudnn"
\
-e
LD_LIBRARY_PATH
=
"/opt/conda/lib/python
$python_version
/site-packages/nvidia/cudnn/lib:/usr/local/cuda/lib64"
\
-v
./
$TARGET_DIR
:/workspace
--workdir
/workspace/tensorflow/mnist
$1
python train.py
else
docker run
--rm
--platform
=
linux/amd64
--gpus
all
-v
./
$TARGET_DIR
:/workspace
--workdir
/workspace/tensorflow/mnist
$1
python train.py
;
fi
;
fi
if
[[
"
$1
"
==
*
"paddle"
*
]]
;
then
if
[[
"
$1
"
==
*
"paddle"
*
]]
;
then
TARGET_DIR
=
gpu-base-image-test/paddletest
TARGET_DIR
=
gpu-base-image-test/paddletest
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment