Commits · 96ae75ad7f96c70c436f9297b06d059ad85140f9 · OpenDAS / vllm_cscc

03 Jan, 2025 1 commit
- fix run error · 184b50f7
  zhuwenwen authored Jan 03, 2025
  
  184b50f7
27 Dec, 2024 2 commits
- [BugFix] Fix quantization for all other methods (#11547) · 2339d59f
  Robert Shaw authored Dec 27, 2024
  
  2339d59f
- Deepseek v3 (#11502) · f49777ba
  Simon Mo authored Dec 26, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: robertgshaw2-neuralmagic <rshaw@neuralmagic.com>
```
  f49777ba
26 Dec, 2024 2 commits

[Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (#11523) · 2072924d

Michael Goin authored Dec 26, 2024


Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>

2072924d

[V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (#10681) · dcb1a944

sroy745 authored Dec 26, 2024


Signed-off-by: Sourashis Roy <sroy@roblox.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

dcb1a944

23 Dec, 2024 1 commit
- [Misc] Add assertion and helpful message for marlin24 compressed models (#11388) · b866cdbd
  Dipika Sikka authored Dec 23, 2024
  
  b866cdbd
21 Dec, 2024 1 commit
- [Bugfix] update should_ignore_layer (#11354) · 51ff216d
  George authored Dec 21, 2024
```
Signed-off-by: George Ohashi <george@neuralmagic.com>
```
  51ff216d
20 Dec, 2024 1 commit
- [Bugfix] Fix spec decoding when seed is none in a batch (#10863) · 86c2d8fd
  Wallas Henrique authored Dec 20, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  86c2d8fd
19 Dec, 2024 3 commits
- [Bugfix] Fix broken CPU compressed-tensors test (#11338) · 276738ce
  Isotr0py authored Dec 20, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  276738ce
- 增加w8a8 triton warmup代码 · 7ef78993
  gaoqiong authored Dec 19, 2024
  
  7ef78993
- [Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311) · 5a9da2e6
  Tyler Michael Smith authored Dec 18, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  5a9da2e6
18 Dec, 2024 1 commit

[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd

Dipika Sikka authored Dec 18, 2024


Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

60508ffd

15 Dec, 2024 1 commit
- [[Misc]Upgrade bitsandbytes to the latest version 0.45.0 (#11201) · 15859f23
  Jee Jee Li authored Dec 15, 2024
  
  15859f23
13 Dec, 2024 1 commit
- [Frontend] Separate pooling APIs in offline inference (#11129) · eeec9e33
  Cyrus Leung authored Dec 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  eeec9e33
11 Dec, 2024 1 commit
- [feat]优化medusa代码，通过VLLM_TREE_DECODING环境变量控制是否采用tree-style解码，计算逻辑主干隔离 · 44e3ca68
  王敏 authored Dec 11, 2024
  
  44e3ca68
10 Dec, 2024 2 commits
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
- [Model] Add has_weight to RMSNorm and re-enable weights loading tracker for Mamba (#10739) · d1f6d1c8
  Isotr0py authored Dec 10, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  d1f6d1c8
05 Dec, 2024 2 commits
- skip tests about cross-attn and not supported · 78800ecf
  zhuwenwen authored Dec 05, 2024
  
  78800ecf
- update DCU info and skip tests same to nv · 129fce94
  zhuwenwen authored Dec 05, 2024
  
  129fce94
03 Dec, 2024 1 commit
- 增加w8a8的triton调度支持 · 1c77f16e
  gaoqiong authored Dec 03, 2024
  
  1c77f16e
01 Dec, 2024 1 commit
- [Model] Replace embedding models with pooling adapter (#10769) · 13370712
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  13370712
29 Nov, 2024 1 commit
- [Model] Clean up MiniCPMV (#10751) · fa6ecb9a
  Cyrus Leung authored Nov 29, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  fa6ecb9a
27 Nov, 2024 4 commits
- [Bugfix] Fix GGUF inference with FP16 unquantized checkpoint (#10675) · b98c62ba
  Isotr0py authored Nov 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b98c62ba
- 1.优化medusa推理，节省cpu耗时 · 28375803
  王敏 authored Nov 27, 2024
```
2.更新medusa readme
3.解决benchmark_moe报错问题
```
  28375803
- [bugfix] fix the default value of llm_int8_threshold in BitsAndBytesConfig (#10657) · cfb3bf25
  yansh97 authored Nov 27, 2024
  
  cfb3bf25
- Remove hard-dependencies of Speculative decode to CUDA workers (#10587) · 0a71900b
  Chendi.Xue authored Nov 26, 2024
```
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
```
  0a71900b
26 Nov, 2024 1 commit
- [Bugfix] Check bnb_4bit_quant_storage for bitsandbytes (#10642) · 7576cd38
  Michael Goin authored Nov 26, 2024
  
  7576cd38
25 Nov, 2024 1 commit

Support Cross encoder models (#10400) · 214efc2c

Maximilien de Bayser authored Nov 24, 2024


Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>

214efc2c

22 Nov, 2024 1 commit
- [Bugfix] Fix Phi-3 BNB quantization with tensor parallel (#9948) · b6374e09
  Isotr0py authored Nov 22, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b6374e09
21 Nov, 2024 2 commits
- [Bugfix] Embedding model pooling_type equals ALL and multi input's bug (#10494) · 4d676f08
  Xiaoyu Zhang authored Nov 21, 2024
  
  4d676f08
- [torch.compile] limit inductor threads and lazy import quant (#10482) · 388ee3de
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  388ee3de
19 Nov, 2024 2 commits
- [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) · b00b33d7
  ElizaWszola authored Nov 19, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  b00b33d7
- [Bugfix]Fix Phi-3 BNB online quantization (#10417) · 7eb719df
  Jee Jee Li authored Nov 19, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  7eb719df
18 Nov, 2024 5 commits
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
- [Kernel] Explicitly specify other value in tl.load calls (#9014) · c2170a5b
  Angus Wang authored Nov 18, 2024
```
Signed-off-by: Angus Wang <wangjadehao@gmail.com>
```
  c2170a5b
- [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) · 6b2d25ef
  Yan Ma authored Nov 19, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  6b2d25ef
- [5/N][torch.compile] torch.jit.script --> torch.compile (#10406) · 7851b451
  youkaichao authored Nov 18, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7851b451
- [Model] Remove redundant softmax when using PoolingType.STEP (#10415) · 01aae1cc
  Maybewuss authored Nov 18, 2024
  
  01aae1cc
17 Nov, 2024 1 commit
- [Bugfix] Fix mrope_position_delta in non-last prefill chunk (#10403) · 80d85c5d
  电脑星人 authored Nov 17, 2024
```
Signed-off-by: imkero <kerorek@outlook.com>
```
  80d85c5d
16 Nov, 2024 1 commit
- [Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (#10388) · 361c29e1
  电脑星人 authored Nov 17, 2024
```
Signed-off-by: imkero <kerorek@outlook.com>
```
  361c29e1