Commits · 96ae75ad7f96c70c436f9297b06d059ad85140f9 · OpenDAS / vllm_cscc

04 Jan, 2025 1 commit
- update qwen&qwen2-moe and baichuan layout · f9f4a735
  zhuwenwen authored Jan 04, 2025
  
  f9f4a735
03 Jan, 2025 3 commits
- remove gemm and fa pad · ca7f98e5
  zhuwenwen authored Jan 03, 2025
  
  ca7f98e5
- update qwen2 and mixtral layout · 57adffa2
  zhuwenwen authored Jan 03, 2025
  
  57adffa2
- fix run error · 184b50f7
  zhuwenwen authored Jan 03, 2025
  
  184b50f7
27 Dec, 2024 2 commits
- [BugFix] Fix quantization for all other methods (#11547) · 2339d59f
  Robert Shaw authored Dec 27, 2024
  
  2339d59f
- Deepseek v3 (#11502) · f49777ba
  Simon Mo authored Dec 26, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: robertgshaw2-neuralmagic <rshaw@neuralmagic.com>
```
  f49777ba
26 Dec, 2024 4 commits

[Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (#11523) · 2072924d

Michael Goin authored Dec 26, 2024


Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>

2072924d

[Misc] Add placeholder module (#11501) · eec906d8
Cyrus Leung authored Dec 26, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
eec906d8
[Model] Modify MolmoForCausalLM MLP (#11510) · f57ee565
Jee Jee Li authored Dec 26, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
f57ee565

[V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (#10681) · dcb1a944

sroy745 authored Dec 26, 2024


Signed-off-by: Sourashis Roy <sroy@roblox.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

dcb1a944

25 Dec, 2024 2 commits
- [Model] Update Qwen2-Audio model support · df81235f
  zhuwenwen authored Dec 25, 2024
  
  df81235f
- [Model] Add Qwen2-Audio model support · 3c23ce2d
  zhuwenwen authored Dec 25, 2024
  
  3c23ce2d
24 Dec, 2024 3 commits
- [Model] Automatic conversion of classification and reward models (#11469) · 3f3e92e1
  Cyrus Leung authored Dec 25, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3f3e92e1
- [Misc] Move weights mapper (#11443) · 196c34b0
  Jee Jee Li authored Dec 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  196c34b0
- [Bugfix] Fix Qwen2-VL LoRA weight loading (#11430) · b1b1038f
  Jee Jee Li authored Dec 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  b1b1038f
23 Dec, 2024 4 commits
- [Bugfix] Add kv cache scales to gemma2.py (#11269) · 60fb4f3b
  Michael Goin authored Dec 23, 2024
  
  60fb4f3b
- [Misc] Add assertion and helpful message for marlin24 compressed models (#11388) · b866cdbd
  Dipika Sikka authored Dec 23, 2024
  
  b866cdbd
- [Bugfix] Fix CFGGuide and use outlines for grammars that can't convert to GBNF (#11389) · 5bfb30a5
  Michael Goin authored Dec 23, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  5bfb30a5
- 1.解决qwen lora模型推理结果异常问题 · da39222e
  王敏 authored Dec 23, 2024
```
2.修复medusa推理时内存访问越界问题
3.benchmark_moe添加deepseek_v2模型
```
  da39222e
21 Dec, 2024 2 commits
- [Bugfix] Fix issues for `Pixtral-Large-Instruct-2411` (#11393) · c2d1b075
  Roger Wang authored Dec 21, 2024
```
Signed-off-by: ywang96 <ywang@example.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  c2d1b075
- [Bugfix] update should_ignore_layer (#11354) · 51ff216d
  George authored Dec 21, 2024
```
Signed-off-by: George Ohashi <george@neuralmagic.com>
```
  51ff216d
20 Dec, 2024 2 commits
- [Core] Loading model from S3 using RunAI Model Streamer as optional loader (#10192) · 995f5623
  omer-dayan authored Dec 20, 2024
```
Signed-off-by: OmerD <omer@run.ai>
```
  995f5623
- [Bugfix] Fix spec decoding when seed is none in a batch (#10863) · 86c2d8fd
  Wallas Henrique authored Dec 20, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  86c2d8fd
19 Dec, 2024 9 commits
- [Bugfix] Fix broken CPU compressed-tensors test (#11338) · 276738ce
  Isotr0py authored Dec 20, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  276738ce
- [Model] Refactor Qwen2-VL to use merged multimodal processor (#11258) · e24113a8
  Isotr0py authored Dec 20, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e24113a8
- [V1] Fix multimodal profiling for `Molmo` (#11325) · 7379b3d4
  Roger Wang authored Dec 19, 2024
```
Signed-off-by: ywang96 <ywang@example.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  7379b3d4
- [Model] Add JambaForSequenceClassification model (#10860) · 6c7f8815
  Yehoshua Cohen authored Dec 19, 2024
```
Signed-off-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6c7f8815
- [Bugfix] Cleanup Pixtral HF code (#11333) · a0f7d53b
  Cyrus Leung authored Dec 19, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a0f7d53b
- 增加w8a8 triton warmup代码 · 7ef78993
  gaoqiong authored Dec 19, 2024
  
  7ef78993
- [VLM] Merged multimodal processor for Qwen2-Audio (#11303) · 6142ef0a
  Cyrus Leung authored Dec 19, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6142ef0a
- [CI] Expand test_guided_generate to test all backends (#11313) · a30482f0
  Michael Goin authored Dec 18, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  a30482f0
- [Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311) · 5a9da2e6
  Tyler Michael Smith authored Dec 18, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  5a9da2e6
18 Dec, 2024 3 commits
- [Bugfix] Fix broken phi3-v mm_processor_kwargs tests (#11263) · 996aa70f
  Isotr0py authored Dec 19, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  996aa70f
- [Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd
  Dipika Sikka authored Dec 18, 2024
```
Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  60508ffd
- [Bugfix] Fix guided decoding with tokenizer mode mistral (#11046) · 8b79f9e1
  Wallas Henrique authored Dec 18, 2024
  
  8b79f9e1
17 Dec, 2024 1 commit
- [V1][VLM] Proper memory profiling for image language models (#11210) · 59c9b6eb
  Roger Wang authored Dec 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  59c9b6eb
16 Dec, 2024 2 commits
- [Model] Refactor Ultravox to use merged input processor (#11198) · d927dbcd
  Isotr0py authored Dec 16, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d927dbcd
- [Model] Support Cohere2ForCausalLM (Cohere R7B) (#11203) · bddbbcb1
  Jani Monoses authored Dec 16, 2024
  
  bddbbcb1
15 Dec, 2024 2 commits
- [Bugfix] Fix error handling of unsupported sliding window (#11213) · 96d673e0
  Cyrus Leung authored Dec 16, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  96d673e0
- [[Misc]Upgrade bitsandbytes to the latest version 0.45.0 (#11201) · 15859f23
  Jee Jee Li authored Dec 15, 2024
  
  15859f23