Skip to content

GitLab

Unverified Commit d919ecc7 authored by Alexander Matveev's avatar Alexander Matveev Committed by GitHub
Browse files

add gptq_marlin test for bug report https://github.com/vllm-project/vllm/issues/5088 (#5145)

parent e691918e
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 0 deletions
tests/models/test_gptq_marlin.py
View file @ d919ecc7
...@@ -40,6 +40,9 @@ MODELS = [ ...@@ -40,6 +40,9 @@ MODELS = [
("TheBloke/TinyLlama-1.1B-Chat-v1.0-GPTQ", "gptq-8bit-128g-actorder_True"), ("TheBloke/TinyLlama-1.1B-Chat-v1.0-GPTQ", "gptq-8bit-128g-actorder_True"),
# 8-bit, act_order==True, group_size=32 # 8-bit, act_order==True, group_size=32
("TheBloke/TinyLlama-1.1B-Chat-v1.0-GPTQ", "gptq-8bit-32g-actorder_True"), ("TheBloke/TinyLlama-1.1B-Chat-v1.0-GPTQ", "gptq-8bit-32g-actorder_True"),
# 4-bit, act_order==True, group_size=128
("TechxGenus/gemma-1.1-2b-it-GPTQ", "main")
] ]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment