Fix KV Offloading + MLA AssertionError by using num_kv_heads=1 in cpu… (#37536)
Signed-off-by:xueliangyang-oeuler <yxl546827391@gmail.com> Co-authored-by:
xueliangyang-oeuler <yxl546827391@gmail.com>
Showing
Please register or sign in to comment