Commits · ed920135c8490440453a64e197fce5e1e6459225 · OpenDAS / vllm_cscc · GitLab

15 Oct, 2024 1 commit
- [Misc] Directly use compressed-tensors for checkpoint definitions (#8909) · 22f8a695
  Michael Goin authored Oct 15, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  22f8a695
11 Oct, 2024 2 commits
- [Kernel] adding fused moe kernel config for L40S TP4 (#9245) · f710090d
  Burkhard Ringlein authored Oct 11, 2024
  
  f710090d
- [misc] hide best_of from engine (#9261) · cbc2ef55
  youkaichao authored Oct 10, 2024
```
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
```
  cbc2ef55
10 Oct, 2024 1 commit
- [Bugfix] Fix lm_head weights tying with lora for llama (#9227) · 07c11cf4
  Isotr0py authored Oct 10, 2024
  
  07c11cf4
09 Oct, 2024 2 commits
- [Hardware][CPU] Support AWQ for CPU backend (#7515) · ca77dd7a
  Li, Jiang authored Oct 10, 2024
  
  ca77dd7a
- support bitsandbytes quantization with more models (#9148) · 2f4117c3
  chenqianfzh authored Oct 08, 2024
  
  2f4117c3
07 Oct, 2024 3 commits
- [Model] Support NVLM-D and fix QK Norm in InternViT (#9045) · 151ef4ef
  Cyrus Leung authored Oct 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  151ef4ef
- [Core] Refactor GGUF parameters packing and forwarding (#8859) · f19da648
  Isotr0py authored Oct 07, 2024
  
  f19da648
- [core] remove beam search from the core (#9105) · 18b296fd
  youkaichao authored Oct 06, 2024
  
  18b296fd
04 Oct, 2024 1 commit
- [Kernel] Zero point support in fused MarlinMoE kernel + AWQ Fused MoE (#8973) · 05d68643
  ElizaWszola authored Oct 04, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika Sikka <ds3822@columbia.edu>
```
  05d68643
03 Oct, 2024 1 commit
- [Misc] log when using default MoE config (#8971) · 01843c89
  Divakar Verma authored Oct 02, 2024
  
  01843c89
01 Oct, 2024 1 commit
- [Spec Decode] (1/2) Remove batch expansion (#8839) · 15702038
  Lily Liu authored Oct 01, 2024
  
  15702038
29 Sep, 2024 3 commits
- [Kernel][Model] Varlen prefill + Prefill chunking support for mamba kernels and Jamba model (#8533) · f13a07b1
  Mor Zusman authored Sep 30, 2024
  
  f13a07b1
- [Model] Support Qwen2.5-Math-RM-72B (#8896) · bc2ef1f7
  Zilin Zhu authored Sep 29, 2024
  
  bc2ef1f7
- [Bugfix] Fix Marlin MoE act order when is_k_full == False (#8741) · d081da00
  ElizaWszola authored Sep 29, 2024
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  d081da00
28 Sep, 2024 1 commit
- [CI/Build] Update models tests & examples (#8874) · e1a3f5e8
  Cyrus Leung authored Sep 29, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  e1a3f5e8
27 Sep, 2024 2 commits
- [Bugfix] fix for deepseek w4a16 (#8906) · c5d55356
  Lucas Wilkinson authored Sep 27, 2024
```
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  c5d55356
- [Kernel] AQ AZP 4/4: Integrate asymmetric quantization to linear method (#7271) · 172d1cd2
  Luka Govedič authored Sep 27, 2024
  
  172d1cd2
25 Sep, 2024 4 commits
- [Misc] Support FP8 MoE for compressed-tensors (#8588) · 873edda6
  Michael Goin authored Sep 25, 2024
  
  873edda6
- [Kernel] Fullgraph and opcheck tests (#8479) · 300da091
  bnellnm authored Sep 25, 2024
  
  300da091
- [Core][Bugfix] Support prompt_logprobs returned with speculative decoding (#8047) · 01b6f9e1
  Travis Johnson authored Sep 24, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  01b6f9e1
- [[Misc]Upgrade bitsandbytes to the latest version 0.44.0 (#8768) · 13f9f7a3
  Jee Jee Li authored Sep 25, 2024
  
  13f9f7a3
24 Sep, 2024 1 commit
- [Bugfix] Fix torch dynamo fixes caused by `replace_parameters` (#8748) · 72fc97a0
  Lucas Wilkinson authored Sep 24, 2024
  
  72fc97a0
23 Sep, 2024 2 commits
- Fix typical acceptance sampler with correct recovered token ids (#8562) · 5f7bb584
  jiqing-feng authored Sep 24, 2024
  
  5f7bb584
- [Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin (#7701) · 86e9c8df
  Lucas Wilkinson authored Sep 23, 2024
```
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  86e9c8df
22 Sep, 2024 1 commit
- [SpecDec][Misc] Cleanup, remove bonus token logic. (#8701) · c6bd70d7
  Lily Liu authored Sep 22, 2024
  
  c6bd70d7
21 Sep, 2024 1 commit
- [Kernel][Triton][AMD] Remove tl.atomic_add from awq_gemm_kernel, 2-5x speedup... · ec4aaad8
  rasmith authored Sep 21, 2024
```
[Kernel][Triton][AMD] Remove tl.atomic_add from awq_gemm_kernel, 2-5x speedup MI300, minor improvement for MI250 (#8646)
```
  ec4aaad8
19 Sep, 2024 1 commit
- [Core] simplify logits resort in _apply_top_k_top_p (#8619) · e42c634a
  盏一 authored Sep 20, 2024
  
  e42c634a
18 Sep, 2024 4 commits
- [Kernel] Change interface to Mamba selective_state_update for continuous batching (#8039) · db9120cd
  Tyler Michael Smith authored Sep 18, 2024
  
  db9120cd
- [AMD][ROCm]Quantization methods on ROCm; Fix _scaled_mm call (#8380) · b3195bc9
  Gregory Shtrasberg authored Sep 18, 2024
```
Co-authored-by: Alexei-V-Ivanov-AMD <156011006+Alexei-V-Ivanov-AMD@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  b3195bc9
- [CI/Build] Update Ruff version (#8469) · 9d104b5b
  Aaron Pham authored Sep 18, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  9d104b5b
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31
17 Sep, 2024 3 commits
- [Kernel] Change interface to Mamba causal_conv1d_update for continuous batching (#8012) · 8110e445
  Tyler Michael Smith authored Sep 17, 2024
  
  8110e445
- [Feature][kernel] tensor parallelism with bitsandbytes quantization (#8434) · 9855b995
  chenqianfzh authored Sep 17, 2024
  
  9855b995
- [refactor] remove triton based sampler (#8524) · 546034b4
  Simon Mo authored Sep 16, 2024
  
  546034b4
16 Sep, 2024 2 commits
- [Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270) · 5d73ae49
  Luka Govedič authored Sep 16, 2024
  
  5d73ae49
- [Kernel] Enable 8-bit weights in Fused Marlin MoE (#8032) · a091e2da
  ElizaWszola authored Sep 16, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
```
  a091e2da
15 Sep, 2024 1 commit
- [Bugfix][Kernel] Add `IQ1_M` quantization implementation to GGUF kernel (#8357) · fc990f97
  Isotr0py authored Sep 16, 2024
  
  fc990f97
13 Sep, 2024 2 commits
- [Hardware][intel GPU] bump up ipex version to 2.3 (#8365) · 85172520
  Kunshang Ji authored Sep 14, 2024
```
Co-authored-by: Yan Ma <yan.ma@intel.com>
```
  85172520
- [BugFix] fix group_topk (#8430) · 8f44a92d
  Dipika Sikka authored Sep 12, 2024
  
  8f44a92d