Commits · dcb1a944d4cf95b4a7b3522ddf970e6d3a28b8b5 · OpenDAS / vllm_cscc · GitLab

26 Dec, 2024 1 commit

[V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (#10681) · dcb1a944

sroy745 authored Dec 26, 2024


Signed-off-by: Sourashis Roy <sroy@roblox.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

dcb1a944

23 Dec, 2024 1 commit
- [Misc] Add assertion and helpful message for marlin24 compressed models (#11388) · b866cdbd
  Dipika Sikka authored Dec 23, 2024
  
  b866cdbd
21 Dec, 2024 1 commit
- [Bugfix] update should_ignore_layer (#11354) · 51ff216d
  George authored Dec 21, 2024
```
Signed-off-by: George Ohashi <george@neuralmagic.com>
```
  51ff216d
20 Dec, 2024 1 commit
- [Bugfix] Fix spec decoding when seed is none in a batch (#10863) · 86c2d8fd
  Wallas Henrique authored Dec 20, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  86c2d8fd
19 Dec, 2024 2 commits
- [Bugfix] Fix broken CPU compressed-tensors test (#11338) · 276738ce
  Isotr0py authored Dec 20, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  276738ce
- [Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311) · 5a9da2e6
  Tyler Michael Smith authored Dec 18, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  5a9da2e6
18 Dec, 2024 1 commit

[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd

Dipika Sikka authored Dec 18, 2024


Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

60508ffd

15 Dec, 2024 1 commit
- [[Misc]Upgrade bitsandbytes to the latest version 0.45.0 (#11201) · 15859f23
  Jee Jee Li authored Dec 15, 2024
  
  15859f23
13 Dec, 2024 1 commit
- [Frontend] Separate pooling APIs in offline inference (#11129) · eeec9e33
  Cyrus Leung authored Dec 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  eeec9e33
10 Dec, 2024 2 commits
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
- [Model] Add has_weight to RMSNorm and re-enable weights loading tracker for Mamba (#10739) · d1f6d1c8
  Isotr0py authored Dec 10, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  d1f6d1c8
01 Dec, 2024 1 commit
- [Model] Replace embedding models with pooling adapter (#10769) · 13370712
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  13370712
29 Nov, 2024 1 commit
- [Model] Clean up MiniCPMV (#10751) · fa6ecb9a
  Cyrus Leung authored Nov 29, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  fa6ecb9a
27 Nov, 2024 3 commits
- [Bugfix] Fix GGUF inference with FP16 unquantized checkpoint (#10675) · b98c62ba
  Isotr0py authored Nov 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b98c62ba
- [bugfix] fix the default value of llm_int8_threshold in BitsAndBytesConfig (#10657) · cfb3bf25
  yansh97 authored Nov 27, 2024
  
  cfb3bf25
- Remove hard-dependencies of Speculative decode to CUDA workers (#10587) · 0a71900b
  Chendi.Xue authored Nov 26, 2024
```
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
```
  0a71900b
26 Nov, 2024 1 commit
- [Bugfix] Check bnb_4bit_quant_storage for bitsandbytes (#10642) · 7576cd38
  Michael Goin authored Nov 26, 2024
  
  7576cd38
25 Nov, 2024 1 commit

Support Cross encoder models (#10400) · 214efc2c

Maximilien de Bayser authored Nov 24, 2024


Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>

214efc2c

22 Nov, 2024 1 commit
- [Bugfix] Fix Phi-3 BNB quantization with tensor parallel (#9948) · b6374e09
  Isotr0py authored Nov 22, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b6374e09
21 Nov, 2024 2 commits
- [Bugfix] Embedding model pooling_type equals ALL and multi input's bug (#10494) · 4d676f08
  Xiaoyu Zhang authored Nov 21, 2024
  
  4d676f08
- [torch.compile] limit inductor threads and lazy import quant (#10482) · 388ee3de
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  388ee3de
19 Nov, 2024 2 commits
- [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) · b00b33d7
  ElizaWszola authored Nov 19, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  b00b33d7
- [Bugfix]Fix Phi-3 BNB online quantization (#10417) · 7eb719df
  Jee Jee Li authored Nov 19, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  7eb719df
18 Nov, 2024 5 commits
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
- [Kernel] Explicitly specify other value in tl.load calls (#9014) · c2170a5b
  Angus Wang authored Nov 18, 2024
```
Signed-off-by: Angus Wang <wangjadehao@gmail.com>
```
  c2170a5b
- [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) · 6b2d25ef
  Yan Ma authored Nov 19, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  6b2d25ef
- [5/N][torch.compile] torch.jit.script --> torch.compile (#10406) · 7851b451
  youkaichao authored Nov 18, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7851b451
- [Model] Remove redundant softmax when using PoolingType.STEP (#10415) · 01aae1cc
  Maybewuss authored Nov 18, 2024
  
  01aae1cc
17 Nov, 2024 1 commit
- [Bugfix] Fix mrope_position_delta in non-last prefill chunk (#10403) · 80d85c5d
  电脑星人 authored Nov 17, 2024
```
Signed-off-by: imkero <kerorek@outlook.com>
```
  80d85c5d
16 Nov, 2024 2 commits
- [Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (#10388) · 361c29e1
  电脑星人 authored Nov 17, 2024
```
Signed-off-by: imkero <kerorek@outlook.com>
```
  361c29e1
- [BugFix] [Kernel] Fix GPU SEGV occuring in fused_moe kernel (#10385) · 1d754726
  rasmith authored Nov 16, 2024
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  1d754726
15 Nov, 2024 2 commits
- [Misc] Consolidate pooler config overrides (#10351) · 2ac6d0e7
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2ac6d0e7
- [bugfix] Fix static asymmetric quantization case (#10334) · bf2ddc66
  Luka Govedič authored Nov 14, 2024
```
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
```
  bf2ddc66
09 Nov, 2024 1 commit

[Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8... · 127c0748

rasmith authored Nov 08, 2024


[Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8 and int8 SmoothQuant, symmetric case (#9857)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>

127c0748

07 Nov, 2024 3 commits
- Adds method to read the pooling types from model's files (#9506) · aa9078fa
  Flávia Béo authored Nov 07, 2024
```
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
```
  aa9078fa
- [Hardware][CPU] Update torch 2.5 (#9911) · a4b3e0c1
  Li, Jiang authored Nov 07, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a4b3e0c1
- [Bugfix] Fix FP8 torch._scaled_mm fallback for torch>2.5 with CUDA<12.4 (#10095) · 4ab32566
  Michael Goin authored Nov 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  4ab32566
06 Nov, 2024 3 commits

[V1] Make v1 more testable (#9888) · d58268c5
Joe Runde authored Nov 06, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
d58268c5
Remove ScaledActivation for AWQ (#10057) · 399c7986
Michael Goin authored Nov 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
399c7986

[Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) · a02a50e6

Konrad Zawora authored Nov 06, 2024


Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzynie...

a02a50e6