Commits · e35879c27601b09aab49e054786ce2a459f7a384 · OpenDAS / vllm_cscc · GitLab

10 Dec, 2024 8 commits
- [Bugfix] Fix xgrammar failing to read a vocab_size from LlavaConfig on PixtralHF. (#11043) · e35879c2
  Jeff Cook authored Dec 09, 2024
  
  e35879c2
- monitor metrics of tokens per step using cudagraph batchsizes (#11031) · ebf77806
  youkaichao authored Dec 09, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ebf77806
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
- [Pixtral] Improve loading (#11040) · bc192a2b
  Patrick von Platen authored Dec 10, 2024
  
  bc192a2b
- [Frontend] Use request id from header (#10968) · 980ad394
  Joe Runde authored Dec 09, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  980ad394
- [Bugfix] Fix usage of `deprecated` decorator (#11025) · 391d7b27
  Cyrus Leung authored Dec 10, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  391d7b27
- [Model] Add has_weight to RMSNorm and re-enable weights loading tracker for Mamba (#10739) · d1f6d1c8
  Isotr0py authored Dec 10, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  d1f6d1c8
- [Docs] Add dedicated tool calling page to docs (#10554) · 6d525288
  Michael Goin authored Dec 09, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  6d525288
09 Dec, 2024 17 commits
- [V1] Do not store `None` in self.generators (#11038) · 6faec545
  Woosuk Kwon authored Dec 09, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6faec545
- Build tpu image in release pipeline (#10936) · 5ed5d5f1
  Richard Liu authored Dec 09, 2024
```
Signed-off-by: Richard Liu <ricliu@google.com>
Co-authored-by: Kevin H. Luu <kevin@anyscale.com>
```
  5ed5d5f1
- [ROCm][bugfix] scpecilative decoding worker class (#11035) · b63ba848
  Gregory Shtrasberg authored Dec 09, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  b63ba848
- [Neuron] Upgrade neuron to 2.20.2 (#11016) · 9c6459e4
  xendo authored Dec 09, 2024
```
Signed-off-by: Jerzy Zagorski <jzagorsk@amazon.com>
Co-authored-by: Jerzy Zagorski <jzagorsk@amazon.com>
```
  9c6459e4
- [v1] fix use compile sizes (#11000) · 1a2f8fb8
  youkaichao authored Dec 09, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1a2f8fb8
- [Bugfix][Hardware][Gaudi] Bump vllm_hpu_extension version (#11028) · cbcbdb1c
  Konrad Zawora authored Dec 09, 2024
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
```
  cbcbdb1c
- [Model] merged input processor for Phi-3-Vision models (#10977) · a811dd66
  Isotr0py authored Dec 10, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  a811dd66
- [Misc][LoRA] Abstract PunicaWrapper (#10955) · ca871491
  Jee Jee Li authored Dec 10, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ca871491
- [V1] Further reduce CPU overheads in flash-attn (#10989) · 3b61cb45
  Woosuk Kwon authored Dec 09, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3b61cb45
- [ci/build] Recompile CI dependencies list with Python 3.12 (#11013) · edc4fa31
  Kevin H. Luu authored Dec 09, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  edc4fa31
- [V1] Input Batch Relocation (#10962) · 25b79d9f
  Varun Sundar Rabindranath authored Dec 09, 2024
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  25b79d9f
- [Platform] Move `async output` check to platform (#10768) · aea2fc38
  wangxiyuan authored Dec 10, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  aea2fc38
- [Core] Require xgrammar >= 0.1.6 (#11021) · e691b26f
  Russell Bryant authored Dec 09, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  e691b26f
- [V1] Fix Detokenizer loading in `AsyncLLM` (#10997) · c6903579
  Roger Wang authored Dec 09, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  c6903579
- [torch.compile] add dynamo time tracking (#11005) · d1c2e15e
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d1c2e15e
- [Doc][V1] Add V1 support column for multimodal models (#10998) · af7c4a92
  Roger Wang authored Dec 08, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  af7c4a92
- [misc] clean up and unify logging (#10999) · 46004e83
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  46004e83
08 Dec, 2024 3 commits
- [torch.compile][misc] fix comments (#10993) · 43b05fa3
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  43b05fa3
- [V1] Initial support of multimodal models for V1 re-arch (#10699) · a11f3265
  Roger Wang authored Dec 08, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  a11f3265
- [torch.compile] allow candidate compile sizes (#10984) · fd57d2b5
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  fd57d2b5
07 Dec, 2024 12 commits
- [core][misc] remove use_dummy driver for _run_workers (#10920) · 7be15d93
  youkaichao authored Dec 07, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7be15d93
- [core][executor] simplify instance id (#10976) · 1b62745b
  youkaichao authored Dec 07, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1b62745b
- [BugFix][Kernel]: fix illegal memory access in causal_conv1d when conv_states is None (#10928) · 78029b34
  zhou fan authored Dec 08, 2024
```
Signed-off-by: xffxff <1247714429@qq.com>
```
  78029b34
- [Doc] Explicitly state that PP isn't compatible with speculative decoding yet (#10975) · c889d588
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c889d588
- [Model] Update multi-modal processor to support Mantis(LLaVA) model (#10711) · 39e227c7
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  39e227c7
- [Doc] Explicitly state that InternVL 2.5 is supported (#10978) · 1c768fe5
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1c768fe5
- [Model] Composite weight loading for multimodal Qwen2 (#10944) · bf0e382e
  Cyrus Leung authored Dec 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  bf0e382e
- [Misc][LoRA] Refactor and clean MergedQKVParallelLinearWithLora implementation (#10958) · b26b4cd0
  Isotr0py authored Dec 07, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b26b4cd0
- [Build] Fix for the Wswitch-bool clang warning (#10060) · f13cf9ad
  Gregory Shtrasberg authored Dec 07, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  f13cf9ad
- [3/N] Support and implement merged input processor for LLaVA model (#10676) · 955fa953
  Cyrus Leung authored Dec 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  955fa953
- [Bugfix] Fix test-pipeline.yaml (#10973) · acf092d3
  Jee Jee Li authored Dec 07, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  acf092d3
- [Core] Cleanup startup logging a bit (#10961) · 69d357ba
  Russell Bryant authored Dec 06, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  69d357ba