Commits · beb16b2c810a87b28e7b8a7aa29d26f842f654b9 · OpenDAS / vllm_cscc

"lib/llm/src/entrypoint/input/text.rs" did not exist on "3e8e38a942468cebbb39dbc50ec00130f9e6f07d"

10 Dec, 2024 9 commits
- [Bugfix] Handle <|tool_call|> token in granite tool parser (#11039) · beb16b2c
  Travis Johnson authored Dec 10, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  beb16b2c
- [Bugfix] cuda error running llama 3.2 (#11047) · 82c73fd5
  Gene Der Su authored Dec 09, 2024
  
  82c73fd5
- [Bugfix] Fix xgrammar failing to read a vocab_size from LlavaConfig on PixtralHF. (#11043) · e35879c2
  Jeff Cook authored Dec 09, 2024
  
  e35879c2
- monitor metrics of tokens per step using cudagraph batchsizes (#11031) · ebf77806
  youkaichao authored Dec 09, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ebf77806
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
- [Pixtral] Improve loading (#11040) · bc192a2b
  Patrick von Platen authored Dec 10, 2024
  
  bc192a2b
- [Frontend] Use request id from header (#10968) · 980ad394
  Joe Runde authored Dec 09, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  980ad394
- [Bugfix] Fix usage of `deprecated` decorator (#11025) · 391d7b27
  Cyrus Leung authored Dec 10, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  391d7b27
- [Model] Add has_weight to RMSNorm and re-enable weights loading tracker for Mamba (#10739) · d1f6d1c8
  Isotr0py authored Dec 10, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  d1f6d1c8
09 Dec, 2024 13 commits
- [V1] Do not store `None` in self.generators (#11038) · 6faec545
  Woosuk Kwon authored Dec 09, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6faec545
- [ROCm][bugfix] scpecilative decoding worker class (#11035) · b63ba848
  Gregory Shtrasberg authored Dec 09, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  b63ba848
- [Neuron] Upgrade neuron to 2.20.2 (#11016) · 9c6459e4
  xendo authored Dec 09, 2024
```
Signed-off-by: Jerzy Zagorski <jzagorsk@amazon.com>
Co-authored-by: Jerzy Zagorski <jzagorsk@amazon.com>
```
  9c6459e4
- [v1] fix use compile sizes (#11000) · 1a2f8fb8
  youkaichao authored Dec 09, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1a2f8fb8
- [Bugfix][Hardware][Gaudi] Bump vllm_hpu_extension version (#11028) · cbcbdb1c
  Konrad Zawora authored Dec 09, 2024
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
```
  cbcbdb1c
- [Model] merged input processor for Phi-3-Vision models (#10977) · a811dd66
  Isotr0py authored Dec 10, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  a811dd66
- [Misc][LoRA] Abstract PunicaWrapper (#10955) · ca871491
  Jee Jee Li authored Dec 10, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ca871491
- [V1] Further reduce CPU overheads in flash-attn (#10989) · 3b61cb45
  Woosuk Kwon authored Dec 09, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3b61cb45
- [V1] Input Batch Relocation (#10962) · 25b79d9f
  Varun Sundar Rabindranath authored Dec 09, 2024
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  25b79d9f
- [Platform] Move `async output` check to platform (#10768) · aea2fc38
  wangxiyuan authored Dec 10, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  aea2fc38
- [V1] Fix Detokenizer loading in `AsyncLLM` (#10997) · c6903579
  Roger Wang authored Dec 09, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  c6903579
- [torch.compile] add dynamo time tracking (#11005) · d1c2e15e
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d1c2e15e
- [misc] clean up and unify logging (#10999) · 46004e83
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  46004e83
08 Dec, 2024 3 commits
- [torch.compile][misc] fix comments (#10993) · 43b05fa3
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  43b05fa3
- [V1] Initial support of multimodal models for V1 re-arch (#10699) · a11f3265
  Roger Wang authored Dec 08, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  a11f3265
- [torch.compile] allow candidate compile sizes (#10984) · fd57d2b5
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  fd57d2b5
07 Dec, 2024 8 commits
- [core][misc] remove use_dummy driver for _run_workers (#10920) · 7be15d93
  youkaichao authored Dec 07, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7be15d93
- [core][executor] simplify instance id (#10976) · 1b62745b
  youkaichao authored Dec 07, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1b62745b
- [Doc] Explicitly state that PP isn't compatible with speculative decoding yet (#10975) · c889d588
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c889d588
- [Model] Update multi-modal processor to support Mantis(LLaVA) model (#10711) · 39e227c7
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  39e227c7
- [Model] Composite weight loading for multimodal Qwen2 (#10944) · bf0e382e
  Cyrus Leung authored Dec 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  bf0e382e
- [Misc][LoRA] Refactor and clean MergedQKVParallelLinearWithLora implementation (#10958) · b26b4cd0
  Isotr0py authored Dec 07, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b26b4cd0
- [3/N] Support and implement merged input processor for LLaVA model (#10676) · 955fa953
  Cyrus Leung authored Dec 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  955fa953
- [Core] Cleanup startup logging a bit (#10961) · 69d357ba
  Russell Bryant authored Dec 06, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  69d357ba
06 Dec, 2024 6 commits
- [ci] fix broken tests (#10956) · dcdc3faf
  youkaichao authored Dec 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dcdc3faf
- [misc] fix typo (#10960) · c05cfb67
  youkaichao authored Dec 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c05cfb67
- [Core] Support Lark grammars for XGrammar (#10870) · 8b596318
  Michael Goin authored Dec 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  8b596318
- [torch.compile] fix deprecated code (#10948) · a1887f2c
  youkaichao authored Dec 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  a1887f2c
- [torch.compile] add logging for compilation time (#10941) · b031a455
  youkaichao authored Dec 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b031a455
- [torch.compile] use size tuning for specific sizes (#10933) · db87eb6c
  youkaichao authored Dec 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  db87eb6c
05 Dec, 2024 1 commit
- [Misc][Gaudi] Avoid torch.compile and enable lazy collectives (#10897) · a4306527
  Konrad Zawora authored Dec 05, 2024
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
```
  a4306527