Commits · 39c89e71a84779c0758ec603efcded7a48bb5fc0 · OpenDAS / vllm_cscc · GitLab

05 Dec, 2024 6 commits
- [Misc] Update llama 3.2 template to support system prompt with images (#10901) · 39c89e71
  Travis Johnson authored Dec 04, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  39c89e71
- [Bugfix] Fix BNB loader target_modules (#10720) · 1f958a7d
  Jee Jee Li authored Dec 05, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1f958a7d
- [Doc] Create a new "Usage" section (#10827) · aa39a8e1
  Cyrus Leung authored Dec 05, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aa39a8e1
- [Bugfix] Fallback to outlines for complex json schemas (#10899) · 8d370e91
  Michael Goin authored Dec 04, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  8d370e91
- [benchmark] Make H100 benchmark optional (#10908) · 7883c2bb
  Kevin H. Luu authored Dec 04, 2024
  
  7883c2bb
- [V1] Fix when max_model_len is not divisible by block_size (#10903) · 2a56e126
  Woosuk Kwon authored Dec 04, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2a56e126
04 Dec, 2024 11 commits
- [CI/Build] improve python-only dev setup (#9621) · e4c34c23
  Daniele authored Dec 04, 2024
```
Signed-off-by: Daniele Trifirò <dtrifiro@redhat.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  e4c34c23
- Benchmark serving structured output (#10880) · 82eb5ea8
  Chendi.Xue authored Dec 04, 2024
```
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  82eb5ea8
- [Model] Consolidate ViTs attention implementation without mask (#10893) · 10398b47
  Isotr0py authored Dec 05, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  10398b47
- [LoRA] Change lora_tokenizers capacity (#10796) · 01d079fd
  Xin Yang authored Dec 04, 2024
```
Signed-off-by: Xin Yang <xyang19@gmail.com>
```
  01d079fd
- [ci/build] Update vLLM postmerge ECR repo (#10887) · c92acb96
  Kevin H. Luu authored Dec 04, 2024
  
  c92acb96
- [bugfix] fixed parameter “n” when set parameter “bestof” > 1 (#10854) · 8db957ee
  jianzheng authored Dec 04, 2024
```
Signed-off-by: jianzheng <57654625+o2363286@users.noreply.github.com>
```
  8db957ee
- [ci/build] Job to build and push release image (#10877) · c9ca4fce
  Kevin H. Luu authored Dec 03, 2024
  
  c9ca4fce
- [ci/build] Change queue name for Release jobs (#10875) · fa2dea61
  Kevin H. Luu authored Dec 03, 2024
  
  fa2dea61
- Drop ROCm load format check (#10767) · b5b647b0
  wangxiyuan authored Dec 04, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  b5b647b0
- [CI/Build] Replace mean with torch.all in test_pynccl.py (#10876) · d2bd88b1
  Tyler Michael Smith authored Dec 03, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  d2bd88b1
- [Benchmark] Benchmark structured output with datasets (#10557) · 381ac93b
  Chendi.Xue authored Dec 03, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
```
  381ac93b
03 Dec, 2024 14 commits
- [Build][Bugfix] Using the correct type hint (#10866) · a061fe60
  Gregory Shtrasberg authored Dec 03, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  a061fe60
- [Frontend] correctly record prefill and decode time metrics (#10853) · 7c32b686
  tomeras91 authored Dec 03, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  7c32b686
- [Bugfix] Only require XGrammar on x86 (#10865) · 7090c27b
  Michael Goin authored Dec 03, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  7090c27b
- [MISC][XPU] quick fix for XPU CI (#10859) · 2f2cdc74
  Yan Ma authored Dec 04, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  2f2cdc74
- [V1] VLM - Run the mm_mapper preprocessor in the frontend process (#10640) · 3bc94cab
  Alexander Matveev authored Dec 03, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  3bc94cab
- [Speculative Decoding] Move indices to device before filtering output (#10850) · f6084f63
  Yang Zheng authored Dec 03, 2024
```
Co-authored-by: Yang Zheng(SW)(Alex) <you@example.com>
```
  f6084f63
- [Core][Performance] Add XGrammar support for guided decoding and set it as default (#10785) · 9323a315
  Aaron Pham authored Dec 03, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  9323a315
- [Misc] Remove deprecated names (#10817) · 3257d449
  Cyrus Leung authored Dec 03, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3257d449
- [Doc] Add github links for source code references (#10672) · ef51831e
  Russell Bryant authored Dec 03, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ef51831e
- [torch.compile] remove compilation_context and simplify code (#10838) · dc5ce861
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dc5ce861
- [core][distributed] add pynccl broadcast (#10843) · 21fe7b48
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  21fe7b48
- [Bugfix] Fix QKVParallelLinearWithShardedLora bias bug (#10844) · a4cf2561
  Jee Jee Li authored Dec 03, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a4cf2561
- [Model] support bitsandbytes quantization with minicpm model (#10842) · d746268e
  zixuanzhang226 authored Dec 02, 2024
```
Signed-off-by: Ubuntu <zixuanzhang@bytedance.com>
```
  d746268e
- [Bugfix] Prevent benchmark_throughput.py from using duplicated random prompts (#10753) · 4433195a
  Michael Goin authored Dec 02, 2024
  
  4433195a
02 Dec, 2024 9 commits
- [Model] Add TP and BNB quantization support to LlavaMultiModalProjector (#10834) · 4c05edb3
  Isotr0py authored Dec 03, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  4c05edb3
- Fix openvino on GPU (#10793) · 9b14d978
  Jani Monoses authored Dec 02, 2024
  
  9b14d978
- [Misc][XPU] Avoid torch compile for XPU platform (#10747) · 519cc6ca
  Yan Ma authored Dec 03, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  519cc6ca
- [Misc][LoRA] Move the implementation of lora bias to punica.py (#10829) · b45f0d79
  Jee Jee Li authored Dec 03, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  b45f0d79
- [misc] use out argument for flash attention (#10822) · a4c4daf3
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  a4c4daf3
- [CI/Build] Update `mistral_common` version for tests and docs (#10825) · e95f275f
  Cyrus Leung authored Dec 02, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e95f275f
- [Model]: add some tests for aria model (#10770) · ef31eabc
  zhou fan authored Dec 02, 2024
```
Signed-off-by: xffxff <1247714429@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  ef31eabc
- [doc]Update config docstring (#10732) · 995a1485
  wangxiyuan authored Dec 02, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  995a1485
- [misc] remove xverse modeling file (#10814) · 63a16417
  youkaichao authored Dec 01, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  63a16417