Commits · 4db72e57f6e8da5e78285e9868e9327167bea973 · OpenDAS / vllm_cscc

"tests/utils_/test_argparse_utils.py" did not exist on "8a297115e2367d463b781adb86b55ac740594cf6"

01 Jan, 2025 2 commits
- [Bugfix][Refactor] Unify model management in frontend (#11660) · 4db72e57
  Joe Runde authored Dec 31, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  4db72e57
- [Benchmark] Add benchmark script for CPU offloading (#11533) · 0c6f9985
  Yihua Cheng authored Dec 31, 2024
```
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: KuntaiDu <kuntai@uchicago.edu>
```
  0c6f9985
31 Dec, 2024 6 commits

[V1][VLM] V1 support for selected single-image models. (#11632) · e7c7c5e8

Roger Wang authored Dec 31, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>

e7c7c5e8

[V1] Simpify vision block hash for prefix caching by removing offset from hash (#11646) · 8c3230d8
Chen Zhang authored Dec 31, 2024

8c3230d8

[Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to... · 2c571880

sakunkun authored Dec 31, 2024

[Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to after the check for allocating new blocks. (#11565)

2c571880

[Misc][LoRA] Support Rank Stabilized LoRA (RSLoRA) (#6909) · 82c49d32

John Giorgi authored Dec 31, 2024


Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

82c49d32

[Bugfix] Fix OpenAI parallel sampling when using xgrammar (#11637) · 74fa1d12
Michael Goin authored Dec 30, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
74fa1d12

[Model][LoRA]LoRA support added for MolmoForCausalLM (#11439) · a2a40bcd

Matthias Vogler authored Dec 31, 2024


Signed-off-by: Matthias Vogler <matthias.vogler@joesecurity.org>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Matthias Vogler <matthias.vogler@joesecurity.org>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

a2a40bcd

30 Dec, 2024 9 commits
- [benchmark] Remove dependency for H100 benchmark step (#11572) · ccb1aabc
  Kevin H. Luu authored Dec 30, 2024
  
  ccb1aabc
- [Bugfix] Validate and concatenate image embeddings in MiniCPMVBaseModel (#11631) · 36e76700
  whyiug authored Dec 31, 2024
  
  36e76700
- [V1] [6/N] API Server: Better Shutdown (#11586) · 5886aa49
  Robert Shaw authored Dec 30, 2024
  
  5886aa49
- [VLM] Abstract out multi-modal data parsing in merged processor (#11620) · 8d9b6721
  Cyrus Leung authored Dec 30, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8d9b6721
- [platforms] enable platform plugins (#11602) · b12e87f9
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  b12e87f9
- [CI/Build][CPU] Fix CPU CI by lazy importing triton FP8 kernels (#11618) · 5dbf8545
  Li, Jiang authored Dec 30, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  5dbf8545
- [Build][Kernel] Update CUTLASS to v3.6.0 (#11607) · 970d6d07
  Tyler Michael Smith authored Dec 30, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  970d6d07
- [Docker] bump up neuron sdk v2.21 (#11593) · 628ec6c1
  Liangfu Chen authored Dec 29, 2024
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  628ec6c1
- [v1] fix compilation cache (#11598) · 3682e33f
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3682e33f
29 Dec, 2024 4 commits
- Remove print statement in DeepseekScalingRotaryEmbedding (#11604) · 0aa38d16
  Michael Goin authored Dec 29, 2024
  
  0aa38d16
- [Misc] KV cache transfer connector registry (#11481) · faef77c0
  Kuntai Du authored Dec 29, 2024
```
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
```
  faef77c0
- [v1][bugfix] fix cudagraph with inplace buffer assignment (#11596) · dba4d9de
  youkaichao authored Dec 29, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dba4d9de
- [Doc] Convert list tables to MyST (#11594) · 32b4c63f
  Cyrus Leung authored Dec 29, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  32b4c63f
28 Dec, 2024 10 commits
- [V1] [5/N] API Server: unify `Detokenizer` and `EngineCore` input (#11545) · 4fb8e329
  Robert Shaw authored Dec 28, 2024
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  4fb8e329
- [bugfix] interleaving sliding window for cohere2 model (#11583) · 328841d0
  youkaichao authored Dec 29, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  328841d0
- [Doc] Minor documentation fixes (#11580) · d427e5cf
  Cyrus Leung authored Dec 28, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d427e5cf
- [V1][Minor] Set pin_memory=False for token_ids_cpu tensor (#11581) · 42bb201f
  Woosuk Kwon authored Dec 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  42bb201f
- [Hardware][AMD]: Replace HIPCC version with more precise ROCm version (#11515) · 59d6bb4c
  hj-wei authored Dec 28, 2024
```
Signed-off-by: hjwei <hjwei_xd@163.com>
```
  59d6bb4c
- [Model] Remove hardcoded image tokens ids from Pixtral (#11582) · b7dcc003
  Roger Wang authored Dec 28, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  b7dcc003
- [Model] Support InternLM2 Reward models (#11571) · d34be24b
  Isotr0py authored Dec 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d34be24b
- [Bugfix] Last token measurement fix (#11376) · b5cbe8ee
  Rajveer Bachkaniwala authored Dec 27, 2024
```
Signed-off-by: rajveerb <46040700+rajveerb@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  b5cbe8ee
- [V1] [4/N] API Server: ZMQ/MP Utilities (#11541) · df04dffa
  Robert Shaw authored Dec 27, 2024
  
  df04dffa
- [Doc] Update mllama example based on official doc (#11567) · a6073124
  Chen Zhang authored Dec 28, 2024
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  a6073124
27 Dec, 2024 9 commits
- [Bugfix] Fix for ROCM compressed tensor support (#11561) · ac797994
  Selali authored Dec 27, 2024
  
  ac797994
- [Misc] Improve BNB loader to handle mixture of sharded and merged weights with same suffix (#11566) · dde1fa18
  Isotr0py authored Dec 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  dde1fa18
- [Misc]Add BNB quantization for MolmoForCausalLM (#11551) · 0240402c
  Jee Jee Li authored Dec 28, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  0240402c
- [MODEL] LoRA support for Jamba model (#11209) · 55509c21
  ErezSC42 authored Dec 27, 2024
```
Signed-off-by: Erez Schwartz <erezs@ai21.com>
```
  55509c21
- [VLM] Support caching in merged multi-modal processor (#11396) · 10141809
  Cyrus Leung authored Dec 28, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  10141809
- [Doc] Add xgrammar in doc (#11549) · 5ce4627a
  Chen1022 authored Dec 27, 2024
```
Signed-off-by: ccjincong <chenjincong11@gmail.com>
```
  5ce4627a
- [Misc] Abstract the logic for reading and writing media content (#11527) · 7af553ea
  Cyrus Leung authored Dec 27, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7af553ea
- [Bugfix] Fix TeleChat2ForCausalLM weights mapper (#11546) · 2c9b8ea2
  Jee Jee Li authored Dec 27, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  2c9b8ea2
- Update deploying_with_k8s.md with AMD ROCm GPU example (#11465) · d003f3ea
  AlexHe99 authored Dec 27, 2024
```
Signed-off-by: Alex He <alehe@amd.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d003f3ea