Skip to content

GitLab

Unverified Commit 14b7899d authored by Michael Goin's avatar Michael Goin Committed by GitHub
Browse files

[CI] Fix failing FP8 cpu offload test (#13170) 


Signed-off-by: default avatarmgoin <mgoin64@gmail.com>
parent 09972e71
Show whitespace changes
Inline Side-by-side
Showing with 6 additions and 6 deletions
tests/quantization/test_cpu_offload.py
View file @ 14b7899d
......@@ -14,13 +14,13 @@ from ..utils import compare_two_settings
reason="fp8 is not supported on this GPU type.")
def test_cpu_offload_fp8():
# Test quantization of an unquantized checkpoint
compare_two_settings("meta-llama/Meta-Llama-3-8B-Instruct",
compare_two_settings("meta-llama/Llama-3.2-1B-Instruct",
["--quantization", "fp8"],
["--quantization", "fp8", "--cpu-offload-gb", "2"],
["--quantization", "fp8", "--cpu-offload-gb", "1"],
max_wait_seconds=480)
# Test loading a quantized checkpoint
compare_two_settings("neuralmagic/Meta-Llama-3-8B-Instruct-FP8", [],
["--cpu-offload-gb", "2"],
compare_two_settings("neuralmagic/Qwen2-1.5B-Instruct-FP8", [],
["--cpu-offload-gb", "1"],
max_wait_seconds=480)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment