Commit a12e8ab4 authored by Aubrey Li's avatar Aubrey Li
Browse files

yaml: fix Marlin AssertionError

Marlin quantized linear only supports GPU device, when change generate_op
to "KLinearMarlin", generate_device need to be changed to "cuda" accordingly.

Fixes: e5b001d7 ("Update readme; Format code; Add example yaml.")
parent 05f6cede
...@@ -22,7 +22,7 @@ ...@@ -22,7 +22,7 @@
replace: replace:
class: ktransformers.operators.linear.KTransformersLinear class: ktransformers.operators.linear.KTransformersLinear
kwargs: kwargs:
generate_device: "cpu" generate_device: "cuda"
prefill_device: "cuda" prefill_device: "cuda"
generate_op: "KLinearMarlin" generate_op: "KLinearMarlin"
prefill_op: "KLinearTorch" prefill_op: "KLinearTorch"
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment