Commits · 4d3a2c284ecc47748273664c9a6ef302ff3adcbe · OpenDAS / vllm_cscc · GitLab

25 Dec, 2024 2 commits
- [Model] Update Qwen2-Audio model support · df81235f
  zhuwenwen authored Dec 25, 2024
  
  df81235f
- [Model] Add Qwen2-Audio model support · 3c23ce2d
  zhuwenwen authored Dec 25, 2024
  
  3c23ce2d
23 Dec, 2024 1 commit
- 1.解决qwen lora模型推理结果异常问题 · da39222e
  王敏 authored Dec 23, 2024
```
2.修复medusa推理时内存访问越界问题
3.benchmark_moe添加deepseek_v2模型
```
  da39222e
19 Dec, 2024 1 commit
- 增加w8a8 triton warmup代码 · 7ef78993
  gaoqiong authored Dec 19, 2024
  
  7ef78993
17 Dec, 2024 1 commit
- [V1][VLM] Proper memory profiling for image language models (#11210) · 59c9b6eb
  Roger Wang authored Dec 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  59c9b6eb
16 Dec, 2024 2 commits
- [Model] Refactor Ultravox to use merged input processor (#11198) · d927dbcd
  Isotr0py authored Dec 16, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d927dbcd
- [Model] Support Cohere2ForCausalLM (Cohere R7B) (#11203) · bddbbcb1
  Jani Monoses authored Dec 16, 2024
  
  bddbbcb1
15 Dec, 2024 2 commits
- [Bugfix] Fix error handling of unsupported sliding window (#11213) · 96d673e0
  Cyrus Leung authored Dec 16, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  96d673e0
- [[Misc]Upgrade bitsandbytes to the latest version 0.45.0 (#11201) · 15859f23
  Jee Jee Li authored Dec 15, 2024
  
  15859f23
14 Dec, 2024 2 commits
- [VLM] Fully dynamic prompt replacement in merged input processor (#11199) · 93abf23a
  Cyrus Leung authored Dec 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  93abf23a
- [Core] Update outlines and increase its threadpool size (#11140) · 48259264
  Russell Bryant authored Dec 14, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  48259264
13 Dec, 2024 6 commits
- [V1][VLM] Fix edge case bug for InternVL2 (#11165) · 969da7d7
  Roger Wang authored Dec 13, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  969da7d7
- [Frontend] Separate pooling APIs in offline inference (#11129) · eeec9e33
  Cyrus Leung authored Dec 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  eeec9e33
- PaliGemma 2 support (#11142) · 7cd74091
  Jani Monoses authored Dec 13, 2024
  
  7cd74091
- [core] clean up cudagraph batchsize padding logic (#10996) · be39e3cd
  youkaichao authored Dec 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  be39e3cd
- [Bugfix] Update starcoder2 to remap k/v scale names for kv_cache quantization (#11148) · 3989a798
  Dipika Sikka authored Dec 13, 2024
  
  3989a798
- [Bugfix] Use runner_type instead of task in GritLM (#11144) · 1efce686
  Pooya Davoodi authored Dec 12, 2024
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  1efce686
12 Dec, 2024 3 commits
- [Misc] Validate grammar and fail early (#11119) · 2c97eca1
  Cody Yu authored Dec 12, 2024
  
  2c97eca1
- [Bugfix] Quick fix to make Pixtral-HF load correctly again after 39e227c7 . (#11024) · 5d712571
  Jeff Cook authored Dec 12, 2024
  
  5d712571
- [Model] Add support for embedding model GritLM (#10816) · 1da8f0e1
  Pooya Davoodi authored Dec 11, 2024
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  1da8f0e1
11 Dec, 2024 5 commits
- [Misc] Split up pooling tasks (#10820) · 8f10d5e3
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f10d5e3
- [Bugfix] Fix Idefics3 fails during multi-image inference (#11080) · 2e32f5d2
  B-201 authored Dec 11, 2024
```
Signed-off-by: B-201 <Joy25810@foxmail.com>
```
  2e32f5d2
- [ci/build] Fix entrypoints test and pin outlines version (#11088) · 9974fca0
  Kevin H. Luu authored Dec 11, 2024
  
  9974fca0
- [Model] PP support for Mamba-like models (#10992) · ffa48c91
  Mor Zusman authored Dec 11, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  ffa48c91
- [feat]优化medusa代码，通过VLLM_TREE_DECODING环境变量控制是否采用tree-style解码，计算逻辑主干隔离 · 44e3ca68
  王敏 authored Dec 11, 2024
  
  44e3ca68
10 Dec, 2024 5 commits
- [Core] Update to outlines >= 0.1.8 (#10576) · e7391949
  Russell Bryant authored Dec 10, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  e7391949
- [Bugfix] Fix xgrammar failing to read a vocab_size from LlavaConfig on PixtralHF. (#11043) · e35879c2
  Jeff Cook authored Dec 09, 2024
  
  e35879c2
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
- [Pixtral] Improve loading (#11040) · bc192a2b
  Patrick von Platen authored Dec 10, 2024
  
  bc192a2b
- [Model] Add has_weight to RMSNorm and re-enable weights loading tracker for Mamba (#10739) · d1f6d1c8
  Isotr0py authored Dec 10, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  d1f6d1c8
09 Dec, 2024 1 commit
- [Model] merged input processor for Phi-3-Vision models (#10977) · a811dd66
  Isotr0py authored Dec 10, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  a811dd66
08 Dec, 2024 1 commit
- [V1] Initial support of multimodal models for V1 re-arch (#10699) · a11f3265
  Roger Wang authored Dec 08, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  a11f3265
07 Dec, 2024 4 commits
- [Doc] Explicitly state that PP isn't compatible with speculative decoding yet (#10975) · c889d588
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c889d588
- [Model] Update multi-modal processor to support Mantis(LLaVA) model (#10711) · 39e227c7
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  39e227c7
- [Model] Composite weight loading for multimodal Qwen2 (#10944) · bf0e382e
  Cyrus Leung authored Dec 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  bf0e382e
- [3/N] Support and implement merged input processor for LLaVA model (#10676) · 955fa953
  Cyrus Leung authored Dec 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  955fa953
06 Dec, 2024 2 commits
- [Core] Support Lark grammars for XGrammar (#10870) · 8b596318
  Michael Goin authored Dec 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  8b596318
- [feat]并行解码支持多卡推理 · 4a4e3601
  王敏 authored Dec 06, 2024
  
  4a4e3601
05 Dec, 2024 2 commits
- skip tests about cross-attn and not supported · 78800ecf
  zhuwenwen authored Dec 05, 2024
  
  78800ecf
- update DCU info and skip tests same to nv · 129fce94
  zhuwenwen authored Dec 05, 2024
  
  129fce94