Commits · 6116ca8cd79b642c64f4ae6f050a6bc12b96d037 · OpenDAS / vllm_cscc

26 Jan, 2025 1 commit
- [Bugfix] Disable w16a16 2of4 sparse CompressedTensors24 (#12417) · aa2cd2c4
  Tyler Michael Smith authored Jan 26, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  aa2cd2c4
25 Jan, 2025 1 commit
- [ROCm][MoE] MI300 tuned configs Mixtral-8x(7B,22B) | fp16, fp8 (#12408) · bf21481d
  Divakar Verma authored Jan 24, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  bf21481d
23 Jan, 2025 3 commits

[BugFix] Fix parameter names and `process_after_weight_loading` for W4A16 MoE... · eb5cb5e5

Dipika Sikka authored Jan 23, 2025


[BugFix] Fix parameter names and `process_after_weight_loading` for W4A16 MoE Group Act Order  (#11528)
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

eb5cb5e5

[FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) · e97f802b

Gregory Shtrasberg authored Jan 23, 2025


Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>

e97f802b

[AMD][Quantization] Add TritonScaledMMLinearKernel since int8 is broken for AMD (#12282) · 68c4421b
rasmith authored Jan 22, 2025
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
68c4421b

21 Jan, 2025 2 commits
- [BugFix] Fix GGUF tp>1 when vocab_size is not divisible by 64 (#12230) · 5fe6bf29
  Nicolò Lucchesi authored Jan 21, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  5fe6bf29
- [AMD][Build] Porting dockerfiles from the ROCm/vllm fork (#11777) · d4b62d46
  Gregory Shtrasberg authored Jan 20, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  d4b62d46
19 Jan, 2025 3 commits

[Model] Support for fairseq2 Llama (#11442) · bbe5f9de

Martin Gleize authored Jan 19, 2025


Signed-off-by: Martin Gleize <mgleize@meta.com>
Co-authored-by: mgleize user <mgleize@a100-st-p4de24xlarge-4.fair-a100.hpcaas>

bbe5f9de

[V1] Add V1 support of Qwen2-VL (#12128) · 81763c58

Roger Wang authored Jan 19, 2025


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: imkero <kerorek@outlook.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

81763c58

[Misc] Support register quantization method out-of-tree (#11969) · 32eb0da8
yancong authored Jan 19, 2025

32eb0da8

17 Jan, 2025 2 commits
- [AMD][FP8] Using MI300 FP8 format on ROCm for block_quant (#12134) · b5b57e30
  Gregory Shtrasberg authored Jan 17, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  b5b57e30
- [CI/Build][CPU][Bugfix] Fix CPU CI (#12150) · d4e61945
  Li, Jiang authored Jan 17, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  d4e61945
16 Jan, 2025 3 commits
- Support torchrun and SPMD-style offline inference (#12071) · bf53e0c7
  youkaichao authored Jan 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  bf53e0c7
- Various cosmetic/comment fixes (#12089) · 9aa1519f
  Michael Goin authored Jan 16, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  9aa1519f
- [Core] Default to using per_token quantization for fp8 when cutlass is supported. (#8651) · fa0050db
  Elfie Guo authored Jan 15, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  fa0050db
15 Jan, 2025 3 commits

[Misc][Quark] Upstream Quark format to VLLM (#10765) · de0526f6

kewang-xlnx authored Jan 16, 2025


Signed-off-by: kewang-xlnx <kewang@xilinx.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

de0526f6

Fix: cases with empty sparsity config (#12057) · cbe94391
Rahul Tuli authored Jan 15, 2025
```
Signed-off-by: Rahul Tuli <rahul@neuralmagic.com>
```
cbe94391
[Kernel] Support MulAndSilu (#11624) · 42f5e7c5
Jee Jee Li authored Jan 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
42f5e7c5

13 Jan, 2025 2 commits
- [Bugfix] Fix deepseekv3 gate bias error (#12002) · f35ec461
  Steve Luo authored Jan 14, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  f35ec461
- [Model] Support GGUF models newly added in `transformers` 4.46.0 (#9685) · d14e98d9
  Isotr0py authored Jan 13, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d14e98d9
12 Jan, 2025 1 commit
- [Hardware][TPU] workaround fix for MoE on TPU (#11764) · 263a870e
  Avshalom Manevich authored Jan 12, 2025
  
  263a870e
11 Jan, 2025 1 commit

[Bugfix] fused_experts_impl wrong compute type for float32 (#11921) · c32a7c7c

shaochangxu authored Jan 11, 2025


Signed-off-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Co-authored-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>

c32a7c7c

10 Jan, 2025 3 commits
- [Hardware][CPU] Support MOE models on x86 CPU (#11831) · aa1e77a1
  Li, Jiang authored Jan 11, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  aa1e77a1
- [platform] support custom torch.compile backend key (#11318) · 20410b2f
  wangxiyuan authored Jan 10, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  20410b2f
- [misc] remove python function call for custom activation op (#11885) · d907be7d
  cennn authored Jan 10, 2025
```
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  d907be7d
09 Jan, 2025 1 commit

[Misc] Move `print_*_once` from utils to logger (#11298) · d848800e

Cyrus Leung authored Jan 09, 2025

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Co-authored-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>

d848800e

08 Jan, 2025 4 commits
- [Kernel][Triton][AMD] Use block size heuristic for avg 2.8x speedup for int8 models (#11698) · 526de822
  rasmith authored Jan 08, 2025
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  526de822
- [TPU][Quantization] TPU `W8A8` (#11785) · 56fe4c29
  Robert Shaw authored Jan 08, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  56fe4c29
- [Bugfix][XPU] fix silu_and_mul (#11823) · 78f4590b
  Yan Ma authored Jan 09, 2025
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  78f4590b
- [CI/Build][Bugfix] Fix CPU CI image clean up (#11836) · 2f702498
  Li, Jiang authored Jan 08, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  2f702498
07 Jan, 2025 1 commit
- [optimization] remove python function call for custom op (#11750) · 869579a7
  youkaichao authored Jan 08, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  869579a7
06 Jan, 2025 1 commit

[mypy] Forward pass function type hints in lora (#11740) · 9c749713

Lucas Tucker authored Jan 06, 2025


Signed-off-by: lucast2021 <lucast2021@headroyce.org>
Co-authored-by: lucast2021 <lucast2021@headroyce.org>

9c749713

04 Jan, 2025 1 commit

[Model] Remove unnecessary weight initialization logic (#11736) · 65c08928

Cyrus Leung authored Jan 04, 2025


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

65c08928

03 Jan, 2025 1 commit
- [Bugfix] Check chain_speculative_sampling before calling it (#11673) · 07064cb1
  Lu Fang authored Jan 02, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  07064cb1
01 Jan, 2025 1 commit
- [Doc] Fix typo (#11666) · 6d70198b
  Kazuhiro Serizawa authored Jan 01, 2025
```
Signed-off-by: Kazuhiro Serizawa <nserihiro@gmail.com>
```
  6d70198b
30 Dec, 2024 1 commit
- [CI/Build][CPU] Fix CPU CI by lazy importing triton FP8 kernels (#11618) · 5dbf8545
  Li, Jiang authored Dec 30, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  5dbf8545
29 Dec, 2024 2 commits
- Remove print statement in DeepseekScalingRotaryEmbedding (#11604) · 0aa38d16
  Michael Goin authored Dec 29, 2024
  
  0aa38d16
- [v1][bugfix] fix cudagraph with inplace buffer assignment (#11596) · dba4d9de
  youkaichao authored Dec 29, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dba4d9de
27 Dec, 2024 2 commits
- [Bugfix] Fix for ROCM compressed tensor support (#11561) · ac797994
  Selali authored Dec 27, 2024
  
  ac797994
- [MODEL] LoRA support for Jamba model (#11209) · 55509c21
  ErezSC42 authored Dec 27, 2024
```
Signed-off-by: Erez Schwartz <erezs@ai21.com>
```
  55509c21