Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
651f6c31
Unverified
Commit
651f6c31
authored
Nov 23, 2024
by
Nishidha
Committed by
GitHub
Nov 23, 2024
Browse files
For ppc64le, disabled tests for now and addressed space issues (#10538)
parent
86a44fb8
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
41 deletions
+3
-41
.buildkite/run-cpu-test-ppc64le.sh
.buildkite/run-cpu-test-ppc64le.sh
+3
-41
No files found.
.buildkite/run-cpu-test-ppc64le.sh
View file @
651f6c31
...
@@ -4,49 +4,11 @@
...
@@ -4,49 +4,11 @@
# It serves a sanity check for compilation and basic model usage.
# It serves a sanity check for compilation and basic model usage.
set
-ex
set
-ex
# Try building the docker image
docker build
-t
cpu-test
-f
Dockerfile.ppc64le
.
# Setup cleanup
# Setup cleanup
remove_docker_container
()
{
docker
rm
-f
cpu-test
||
true
;
}
remove_docker_container
()
{
docker
rm
-f
cpu-test
||
true
;
docker system prune
-f
;
}
trap
remove_docker_container EXIT
trap
remove_docker_container EXIT
remove_docker_container
remove_docker_container
# Run the image, setting --shm-size=4g for tensor parallel.
# Try building the docker image
source
/etc/environment
docker build
-t
cpu-test
-f
Dockerfile.ppc64le
.
#docker run -itd --entrypoint /bin/bash -v ~/.cache/huggingface:/root/.cache/huggingface --privileged=true --network host -e HF_TOKEN --env VLLM_CPU_KVCACHE_SPACE=4 --shm-size=4g --name cpu-test cpu-test
docker run
-itd
--entrypoint
/bin/bash
-v
~/.cache/huggingface:/root/.cache/huggingface
--privileged
=
true
--network
host
-e
HF_TOKEN
=
"
$HF_TOKEN
"
--name
cpu-test cpu-test
function
cpu_tests
()
{
set
-e
# Run basic model test
docker
exec
cpu-test bash
-c
"
set -e
pip install pytest pytest-asyncio
\
decord einops librosa peft Pillow sentence-transformers soundfile
\
transformers_stream_generator matplotlib datamodel_code_generator
pip install torchvision --index-url https://download.pytorch.org/whl/cpu
pytest -v -s tests/models/decoder_only/language -m cpu_model
pytest -v -s tests/models/embedding/language -m cpu_model
pytest -v -s tests/models/encoder_decoder/language -m cpu_model
pytest -v -s tests/models/decoder_only/audio_language -m cpu_model
pytest -v -s tests/models/decoder_only/vision_language -m cpu_model"
# online inference
docker
exec
cpu-test bash
-c
"
set -e
python3 -m vllm.entrypoints.openai.api_server --model facebook/opt-125m &
timeout 600 bash -c 'until curl localhost:8000/v1/models; do sleep 1; done' || exit 1
python3 benchmarks/benchmark_serving.py
\
--backend vllm
\
--dataset-name random
\
--model facebook/opt-125m
\
--num-prompts 20
\
--endpoint /v1/completions
\
--tokenizer facebook/opt-125m"
}
# All of CPU tests are expected to be finished less than 25 mins.
export
-f
cpu_tests
timeout
25m bash
-c
"cpu_tests"
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment