Commits · 250ee65d72a0c7b86ec5cea9cbe9377da21d6439 · OpenDAS / vllm_cscc · GitLab

10 Dec, 2024 15 commits
- [BUG] Remove token param #10921 (#11022) · 250ee65d
  Flávia Béo authored Dec 10, 2024
```
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
```
  250ee65d
- [Bugfix] Backport request id validation to v0 (#11036) · 9b9cef31
  Joe Runde authored Dec 10, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  9b9cef31
- [Misc][LoRA] Add PEFTHelper for LoRA (#11003) · d05f8867
  Jee Jee Li authored Dec 10, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  d05f8867
- [Bugfix] Handle <|tool_call|> token in granite tool parser (#11039) · beb16b2c
  Travis Johnson authored Dec 10, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  beb16b2c
- Add example of helm chart for vllm deployment on k8s (#9199) · fe2e10c7
  Maxime Fournioux authored Dec 10, 2024
```
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
```
  fe2e10c7
- [Bugfix] cuda error running llama 3.2 (#11047) · 82c73fd5
  Gene Der Su authored Dec 09, 2024
  
  82c73fd5
- Update README.md (#11034) · bfd61043
  Diego Marinho authored Dec 10, 2024
  
  bfd61043
- [Bugfix] Fix xgrammar failing to read a vocab_size from LlavaConfig on PixtralHF. (#11043) · e35879c2
  Jeff Cook authored Dec 09, 2024
  
  e35879c2
- monitor metrics of tokens per step using cudagraph batchsizes (#11031) · ebf77806
  youkaichao authored Dec 09, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ebf77806
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
- [Pixtral] Improve loading (#11040) · bc192a2b
  Patrick von Platen authored Dec 10, 2024
  
  bc192a2b
- [Frontend] Use request id from header (#10968) · 980ad394
  Joe Runde authored Dec 09, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  980ad394
- [Bugfix] Fix usage of `deprecated` decorator (#11025) · 391d7b27
  Cyrus Leung authored Dec 10, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  391d7b27
- [Model] Add has_weight to RMSNorm and re-enable weights loading tracker for Mamba (#10739) · d1f6d1c8
  Isotr0py authored Dec 10, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  d1f6d1c8
- [Docs] Add dedicated tool calling page to docs (#10554) · 6d525288
  Michael Goin authored Dec 09, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  6d525288
09 Dec, 2024 17 commits
- [V1] Do not store `None` in self.generators (#11038) · 6faec545
  Woosuk Kwon authored Dec 09, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6faec545
- Build tpu image in release pipeline (#10936) · 5ed5d5f1
  Richard Liu authored Dec 09, 2024
```
Signed-off-by: Richard Liu <ricliu@google.com>
Co-authored-by: Kevin H. Luu <kevin@anyscale.com>
```
  5ed5d5f1
- [ROCm][bugfix] scpecilative decoding worker class (#11035) · b63ba848
  Gregory Shtrasberg authored Dec 09, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  b63ba848
- [Neuron] Upgrade neuron to 2.20.2 (#11016) · 9c6459e4
  xendo authored Dec 09, 2024
```
Signed-off-by: Jerzy Zagorski <jzagorsk@amazon.com>
Co-authored-by: Jerzy Zagorski <jzagorsk@amazon.com>
```
  9c6459e4
- [v1] fix use compile sizes (#11000) · 1a2f8fb8
  youkaichao authored Dec 09, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1a2f8fb8
- [Bugfix][Hardware][Gaudi] Bump vllm_hpu_extension version (#11028) · cbcbdb1c
  Konrad Zawora authored Dec 09, 2024
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
```
  cbcbdb1c
- [Model] merged input processor for Phi-3-Vision models (#10977) · a811dd66
  Isotr0py authored Dec 10, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  a811dd66
- [Misc][LoRA] Abstract PunicaWrapper (#10955) · ca871491
  Jee Jee Li authored Dec 10, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ca871491
- [V1] Further reduce CPU overheads in flash-attn (#10989) · 3b61cb45
  Woosuk Kwon authored Dec 09, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3b61cb45
- [ci/build] Recompile CI dependencies list with Python 3.12 (#11013) · edc4fa31
  Kevin H. Luu authored Dec 09, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  edc4fa31
- [V1] Input Batch Relocation (#10962) · 25b79d9f
  Varun Sundar Rabindranath authored Dec 09, 2024
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  25b79d9f
- [Platform] Move `async output` check to platform (#10768) · aea2fc38
  wangxiyuan authored Dec 10, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  aea2fc38
- [Core] Require xgrammar >= 0.1.6 (#11021) · e691b26f
  Russell Bryant authored Dec 09, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  e691b26f
- [V1] Fix Detokenizer loading in `AsyncLLM` (#10997) · c6903579
  Roger Wang authored Dec 09, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  c6903579
- [torch.compile] add dynamo time tracking (#11005) · d1c2e15e
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d1c2e15e
- [Doc][V1] Add V1 support column for multimodal models (#10998) · af7c4a92
  Roger Wang authored Dec 08, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  af7c4a92
- [misc] clean up and unify logging (#10999) · 46004e83
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  46004e83
08 Dec, 2024 3 commits
- [torch.compile][misc] fix comments (#10993) · 43b05fa3
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  43b05fa3
- [V1] Initial support of multimodal models for V1 re-arch (#10699) · a11f3265
  Roger Wang authored Dec 08, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  a11f3265
- [torch.compile] allow candidate compile sizes (#10984) · fd57d2b5
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  fd57d2b5
07 Dec, 2024 5 commits
- [core][misc] remove use_dummy driver for _run_workers (#10920) · 7be15d93
  youkaichao authored Dec 07, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7be15d93
- [core][executor] simplify instance id (#10976) · 1b62745b
  youkaichao authored Dec 07, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1b62745b
- [BugFix][Kernel]: fix illegal memory access in causal_conv1d when conv_states is None (#10928) · 78029b34
  zhou fan authored Dec 08, 2024
```
Signed-off-by: xffxff <1247714429@qq.com>
```
  78029b34
- [Doc] Explicitly state that PP isn't compatible with speculative decoding yet (#10975) · c889d588
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c889d588
- [Model] Update multi-modal processor to support Mantis(LLaVA) model (#10711) · 39e227c7
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  39e227c7