GitLab

Fixes `is_marlin` not being passed into `get_default_config`

Also allow `--tensor-parallel-size` in addition to `-tp` and `--tp-size`
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

Signed-off-by: Divakar Verma <divakar.verma@amd.com>

Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>

Signed-off-by: Divakar Verma <divakar.verma@amd.com>

Signed-off-by: Divakar Verma <divakar.verma@amd.com>

2.修复medusa推理时内存访问越界问题
3.benchmark_moe添加deepseek_v2模型

Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Xiaoyu Zhang <BBuf@users.noreply.github.com>

2.更新medusa readme
3.解决benchmark_moe报错问题

Signed-off-by: ElizaWszola <eliza@neuralmagic.com>

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

Signed-off-by: wangshuai09 <391746016@qq.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

Co-authored-by: Michael Goin <michael@neuralmagic.com>

Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>

Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

The 2nd PR for #4532.

This PR supports loading FP8 kv-cache scaling factors from a FP8 checkpoint (with .kv_scale parameter).