Commits · 381ac93bb5a41347a025367bc58119cb45357095 · OpenDAS / vllm_cscc

04 Dec, 2024 1 commit

[Benchmark] Benchmark structured output with datasets (#10557) · 381ac93b

Chendi.Xue authored Dec 03, 2024


Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>

381ac93b

03 Dec, 2024 14 commits
- [Build][Bugfix] Using the correct type hint (#10866) · a061fe60
  Gregory Shtrasberg authored Dec 03, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  a061fe60
- [Frontend] correctly record prefill and decode time metrics (#10853) · 7c32b686
  tomeras91 authored Dec 03, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  7c32b686
- [Bugfix] Only require XGrammar on x86 (#10865) · 7090c27b
  Michael Goin authored Dec 03, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  7090c27b
- [MISC][XPU] quick fix for XPU CI (#10859) · 2f2cdc74
  Yan Ma authored Dec 04, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  2f2cdc74
- [V1] VLM - Run the mm_mapper preprocessor in the frontend process (#10640) · 3bc94cab
  Alexander Matveev authored Dec 03, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  3bc94cab
- [Speculative Decoding] Move indices to device before filtering output (#10850) · f6084f63
  Yang Zheng authored Dec 03, 2024
```
Co-authored-by: Yang Zheng(SW)(Alex) <you@example.com>
```
  f6084f63
- [Core][Performance] Add XGrammar support for guided decoding and set it as default (#10785) · 9323a315
  Aaron Pham authored Dec 03, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  9323a315
- [Misc] Remove deprecated names (#10817) · 3257d449
  Cyrus Leung authored Dec 03, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3257d449
- [Doc] Add github links for source code references (#10672) · ef51831e
  Russell Bryant authored Dec 03, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ef51831e
- [torch.compile] remove compilation_context and simplify code (#10838) · dc5ce861
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dc5ce861
- [core][distributed] add pynccl broadcast (#10843) · 21fe7b48
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  21fe7b48
- [Bugfix] Fix QKVParallelLinearWithShardedLora bias bug (#10844) · a4cf2561
  Jee Jee Li authored Dec 03, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a4cf2561
- [Model] support bitsandbytes quantization with minicpm model (#10842) · d746268e
  zixuanzhang226 authored Dec 02, 2024
```
Signed-off-by: Ubuntu <zixuanzhang@bytedance.com>
```
  d746268e
- [Bugfix] Prevent benchmark_throughput.py from using duplicated random prompts (#10753) · 4433195a
  Michael Goin authored Dec 02, 2024
  
  4433195a
02 Dec, 2024 14 commits
- [Model] Add TP and BNB quantization support to LlavaMultiModalProjector (#10834) · 4c05edb3
  Isotr0py authored Dec 03, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  4c05edb3
- Fix openvino on GPU (#10793) · 9b14d978
  Jani Monoses authored Dec 02, 2024
  
  9b14d978
- [Misc][XPU] Avoid torch compile for XPU platform (#10747) · 519cc6ca
  Yan Ma authored Dec 03, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  519cc6ca
- [Misc][LoRA] Move the implementation of lora bias to punica.py (#10829) · b45f0d79
  Jee Jee Li authored Dec 03, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  b45f0d79
- [misc] use out argument for flash attention (#10822) · a4c4daf3
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  a4c4daf3
- [CI/Build] Update `mistral_common` version for tests and docs (#10825) · e95f275f
  Cyrus Leung authored Dec 02, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e95f275f
- [Model]: add some tests for aria model (#10770) · ef31eabc
  zhou fan authored Dec 02, 2024
```
Signed-off-by: xffxff <1247714429@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  ef31eabc
- [doc]Update config docstring (#10732) · 995a1485
  wangxiyuan authored Dec 02, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  995a1485
- [misc] remove xverse modeling file (#10814) · 63a16417
  youkaichao authored Dec 01, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  63a16417
- Fill TorchSDPAAttentionMetadata seq_lens_field for prefill (#10799) · e25810ae
  Maximilien de Bayser authored Dec 01, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  e25810ae
- [Kernel] Use `out` arg in flash_attn_varlen_func (#10811) · 073a4bd1
  Woosuk Kwon authored Dec 01, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  073a4bd1
- [core] Avoid metrics log noise when idle - include speculative decodi… (#10809) · b7954776
  cduk authored Dec 02, 2024
  
  b7954776
- [Model] Add BNB support to Llava and Pixtral-HF (#10795) · b18c9bba
  Isotr0py authored Dec 02, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b18c9bba
- [Core] Implement disagg prefill by StatelessProcessGroup (#10502) · 0590ec3f
  Kuntai Du authored Dec 01, 2024
```
This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>
```
  0590ec3f
01 Dec, 2024 5 commits
- [Misc] Adding `MMMU-Pro` vision dataset to serving benchmark (#10804) · c11f1721
  Roger Wang authored Dec 01, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  c11f1721
- [doc] add warning about comparing hf and vllm outputs (#10805) · 169a0ff9
  youkaichao authored Dec 01, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  169a0ff9
- [Misc] Rename embedding classes to pooling (#10801) · d2f058e7
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d2f058e7
- [Misc] Improve type annotations for `support_torch_compile` (#10763) · f877a7d1
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f877a7d1
- [Model] Replace embedding models with pooling adapter (#10769) · 13370712
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  13370712
30 Nov, 2024 4 commits
- [doc] format fix (#10789) · 7e4bbda5
  wangxiyuan authored Nov 30, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  7e4bbda5
- [Interleaved ATTN] Support for Mistral-8B (#10591) · e7cfc4ef
  Patrick von Platen authored Nov 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  e7cfc4ef
- [Model] Refactor Molmo weights loading to use AutoWeightsLoader (#10771) · 16ee07f2
  Isotr0py authored Nov 30, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  16ee07f2
- [Bugfix] Fix OpenVino/Neuron `driver_worker` init (#10779) · 40bc2425
  Nicolò Lucchesi authored Nov 30, 2024
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  40bc2425
29 Nov, 2024 2 commits
- [platform] Add verify_quantization in platform. (#10757) · 661175bc
  wangxiyuan authored Nov 29, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  661175bc
- [Bugfix] Fix Idefics3 bug (#10778) · 3132aac0
  Jee Jee Li authored Nov 29, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  3132aac0