Skip to content

GitLab

Unverified Commit ab7fba0e authored by yinfan98's avatar yinfan98 Committed by GitHub
Browse files

Fix nightly ci Gsm8k & Fix flashinfer backend kvcache quant (#4147)

parent bc1534ff
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
python/sglang/srt/layers/attention/flashinfer_backend.py
View file @ ab7fba0e
...@@ -904,6 +904,7 @@ class FlashInferIndicesUpdaterPrefill: ...@@ -904,6 +904,7 @@ class FlashInferIndicesUpdaterPrefill:
self.head_dim, self.head_dim,
1, 1,
q_data_type=self.q_data_type, q_data_type=self.q_data_type,
kv_data_type=self.data_type,
custom_mask=custom_mask, custom_mask=custom_mask,
non_blocking=True, non_blocking=True,
) )
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment