Commits · 96ae75ad7f96c70c436f9297b06d059ad85140f9 · OpenDAS / vllm_cscc · GitLab

26 Dec, 2024 1 commit

[Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (#11523) · 2072924d

Michael Goin authored Dec 26, 2024


Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>

2072924d

23 Dec, 2024 1 commit
- update test_cutlass.py · 300892fe
  zhuwenwen authored Dec 23, 2024
  
  300892fe
19 Dec, 2024 1 commit
- [Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311) · 5a9da2e6
  Tyler Michael Smith authored Dec 18, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  5a9da2e6
18 Dec, 2024 1 commit

[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd

Dipika Sikka authored Dec 18, 2024


Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

60508ffd

13 Dec, 2024 1 commit

[torch.compile] Dynamic fp8 + rms_norm fusion (#10906) · 30870b4f

Luka Govedič authored Dec 12, 2024


Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

30870b4f

11 Dec, 2024 1 commit
- [feat]优化medusa代码，通过VLLM_TREE_DECODING环境变量控制是否采用tree-style解码，计算逻辑主干隔离 · 44e3ca68
  王敏 authored Dec 11, 2024
  
  44e3ca68
07 Dec, 2024 1 commit
- [BugFix][Kernel]: fix illegal memory access in causal_conv1d when conv_states is None (#10928) · 78029b34
  zhou fan authored Dec 08, 2024
```
Signed-off-by: xffxff <1247714429@qq.com>
```
  78029b34
05 Dec, 2024 3 commits
- update DCU info and skip tests same to nv · 129fce94
  zhuwenwen authored Dec 05, 2024
  
  129fce94
- change AMD GPU and dcu to hcu · 3f78216a
  zhuwenwen authored Dec 05, 2024
  
  3f78216a
- added support for kernels tests with torch 2.3 · e150cf11
  zhuwenwen authored Dec 05, 2024
  
  e150cf11
04 Dec, 2024 1 commit
- remove unsupported tests from kernels and add pytest html · a3d96521
  zhuwenwen authored Dec 04, 2024
  
  a3d96521
03 Dec, 2024 1 commit
- 修改cutlass 单测 · 26513bb5
  gaoqiong authored Dec 03, 2024
  
  26513bb5
02 Dec, 2024 1 commit
- [Kernel] Use `out` arg in flash_attn_varlen_func (#10811) · 073a4bd1
  Woosuk Kwon authored Dec 01, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  073a4bd1
27 Nov, 2024 1 commit
- add VLLM_OPTEST_MODELS_PATH/OPTEST_MODELS_PATH to load models from local path... · 3c9817d2
  zhuwenwen authored Nov 27, 2024
```
add VLLM_OPTEST_MODELS_PATH/OPTEST_MODELS_PATH  to load models from local path instead of Hugging Face Hub
```
  3c9817d2
25 Nov, 2024 2 commits
- [Bugfix] Fix chunked prefill with model dtype float32 on Turing Devices (#9850) · c27df94e
  Wallas Henrique authored Nov 25, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  c27df94e
- [misc] move functions to config.py (#10624) · 05d1f8c9
  youkaichao authored Nov 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  05d1f8c9
22 Nov, 2024 1 commit
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
20 Nov, 2024 2 commits
- [Bugfix] Marlin 2:4 temp fix for large M dim (>256) (#10464) · d200972e
  Lucas Wilkinson authored Nov 19, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  d200972e
- remove xops · 1a493a24
  zhuwenwen authored Nov 20, 2024
  
  1a493a24
19 Nov, 2024 3 commits
- [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) · b00b33d7
  ElizaWszola authored Nov 19, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  b00b33d7
- update fa interface tests · fc7db442
  zhuwenwen authored Nov 19, 2024
  
  fc7db442
- [Platform][Refactor] Extract func `get_default_attn_backend` to `Platform` (#10358) · 8c1fb507
  Mengqing Cao authored Nov 19, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  8c1fb507
18 Nov, 2024 1 commit
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
15 Nov, 2024 3 commits
- [Misc] Bump up test_fused_moe tolerance (#10364) · 79ee45b4
  ElizaWszola authored Nov 15, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  79ee45b4
- [fix]修复test_attention单测中paged_attention_v1和paged_attention_v2 opcheck找不到attn_masks错误 · fa57cb7f
  王敏 authored Nov 15, 2024
  
  fa57cb7f
- [bugfix] Fix static asymmetric quantization case (#10334) · bf2ddc66
  Luka Govedič authored Nov 14, 2024
```
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
```
  bf2ddc66
10 Nov, 2024 1 commit
- update tests of kernels and basic_correctness · c012f7f6
  zhuwenwen authored Nov 10, 2024
  
  c012f7f6
09 Nov, 2024 1 commit

[Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8... · 127c0748

rasmith authored Nov 08, 2024


[Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8 and int8 SmoothQuant, symmetric case (#9857)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>

127c0748

08 Nov, 2024 1 commit
- [torch.compile] Fuse RMSNorm with quant (#9138) · 4f93dfe9
  Luka Govedič authored Nov 08, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@126.com>
```
  4f93dfe9
06 Nov, 2024 2 commits
- [V1] Make v1 more testable (#9888) · d58268c5
  Joe Runde authored Nov 06, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  d58268c5
- [CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
  Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  21063c11
05 Nov, 2024 1 commit
- [CI] Prune back the number of tests in tests/kernels/* (#9932) · 235366fe
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  235366fe
02 Nov, 2024 1 commit
- [Encoder Decoder] Add flash_attn kernel support for encoder-decoder models (#9559) · a78dd330
  sroy745 authored Nov 01, 2024
  
  a78dd330
01 Nov, 2024 2 commits
- [Core][VLM] Add precise multi-modal placeholder tracking (#8346) · 6c0b7f54
  Peter Salas authored Nov 01, 2024
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
  6c0b7f54
- [Bugfix/Core] Flashinfer k_scale and v_scale (#9861) · 598b6d7b
  Pavani Majety authored Nov 01, 2024
  
  598b6d7b
31 Oct, 2024 1 commit
- [BugFix][Kernel] Fix Illegal memory access in causal_conv1d in H100 (#9838) · 9fb12f78
  Mor Zusman authored Oct 31, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  9fb12f78
29 Oct, 2024 1 commit
- [Hardware] using current_platform.seed_everything (#9785) · 622b7ab9
  wangshuai09 authored Oct 29, 2024
```
Signed-off-by: wangshuai09 <391746016@qq.com>
```
  622b7ab9
28 Oct, 2024 2 commits
- [torch.compile] support moe models (#9632) · 32176fee
  youkaichao authored Oct 27, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  32176fee
- [Hardware][ROCM] using current_platform.is_rocm (#9642) · 4e2d95e3
  wangshuai09 authored Oct 28, 2024
```
Signed-off-by: wangshuai09 <391746016@qq.com>
```
  4e2d95e3
26 Oct, 2024 1 commit
- [Hardware][openvino] is_openvino --> current_platform.is_openvino (#9716) · 5cbdccd1
  Mengqing Cao authored Oct 26, 2024
  
  5cbdccd1