Commits · 03f48b3db6cff93ffff3ca440383aea1fe4b1a4c · OpenDAS / vllm_cscc

"tests/models/test_blip2.py" did not exist on "cff6a1fec15dba524c162d20a4e8de4df2b0a3d5"

25 Feb, 2025 2 commits
- [Core] LoRA V1 - Add add/pin/list/remove_lora functions (#13705) · 03f48b3d
  Varun Sundar Rabindranath authored Feb 25, 2025
  
  03f48b3d
- Remove unused kwargs from model definitions (#13555) · cdc1fa12
  Harry Mellor authored Feb 25, 2025
  
  cdc1fa12
24 Feb, 2025 3 commits
- [V1] V1 engine implements parallel sampling (AsyncLLM and LLMEngine) (#10980) · befc402d
  afeldman-nm authored Feb 24, 2025
```
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  befc402d
- Expert Parallelism (EP) Support for DeepSeek V2 (#12583) · 781096e3
  Jongseok Park authored Feb 24, 2025
  
  781096e3
- [ci] Add logic to change model to S3 path only when S3 CI env var is on (#13727) · f90a3755
  Kevin H. Luu authored Feb 23, 2025
```
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-63-253.us-west-2.compute.internal>
```
  f90a3755
23 Feb, 2025 4 commits

[V1][BugFix] Fix engine core client shutdown hangs (#13298) · cbae7af5

Nick Hill authored Feb 23, 2025

Even though ZMQ context.destroy() is meant to close open sockets before terminating the context, it appears to be necessary to do this explicitly or else it can hang in the context.term() method.

Close zmq sockets explicitly before terminating context, make shutdown of client resource more robust, shut down engine core process prior to terminating zmq context.
Signed-off-by: Nick Hill <nhill@redhat.com>

cbae7af5

[v1] torchrun compatibility (#13642) · eb24dc4a
youkaichao authored Feb 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
eb24dc4a
[LMM] Implement merged multimodal processor for whisper (#13278) · ba5106e5
Isotr0py authored Feb 23, 2025

ba5106e5
[ci] Use env var to control whether to use S3 bucket in CI (#13634) · 2c5e637b
Kevin H. Luu authored Feb 22, 2025

2c5e637b

22 Feb, 2025 8 commits
- [V1][Kernel] Refactor the prefix_prefill kernel so that the caller no longer... · 558db808
  Sage Moore authored Feb 22, 2025
```
[V1][Kernel] Refactor the prefix_prefill kernel so that the caller no longer has to pass in the context lengths (#13095)
```
  558db808
- [NVIDIA] Support nvfp4 cutlass gemm (#13571) · e109e598
  Kaixi Hou authored Feb 22, 2025
  
  e109e598
- Support SSL Key Rotation in HTTP Server (#13495) · 8db1b9d0
  Keyun Tong authored Feb 22, 2025
  
  8db1b9d0
- [Misc] Reduce LoRA-related static variable (#13166) · 105b8ce4
  Jee Jee Li authored Feb 22, 2025
  
  105b8ce4
- [Metrics] Add `--show-hidden-metrics-for-version` CLI arg (#13295) · 2cb8c154
  Mark McLoughlin authored Feb 22, 2025
  
  2cb8c154
- [V1][Metrics] Support `vllm:cache_config_info` (#13299) · 1cd981da
  Mark McLoughlin authored Feb 22, 2025
  
  1cd981da
- [v1] Support allowed_token_ids in v1 Sampler (#13210) · bb78fb31
  Lu Fang authored Feb 21, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  bb78fb31
- [HTTP Server] Make model param optional in request (#13568) · 0ffdf8ce
  Keyun Tong authored Feb 21, 2025
  
  0ffdf8ce
21 Feb, 2025 4 commits
- [Attention] MLA with chunked prefill (#12639) · 288cc6c2
  Lucas Wilkinson authored Feb 21, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  288cc6c2
- [ci] Fix metrics test model path (#13635) · 34ad27fe
  Kevin H. Luu authored Feb 20, 2025
  
  34ad27fe
- [FEATURE] Enables /score endpoint for embedding models (#12846) · 1c3c9757
  Gabriel Marinho authored Feb 21, 2025
  
  1c3c9757
- [Neuron][Kernel] Vectorize KV cache load in FlashPagedAttention to maximize DMA bandwidth (#13245) · 33170081
  Lingfan Yu authored Feb 20, 2025
```
Signed-off-by: Lingfan Yu <lingfany@amazon.com>
```
  33170081
20 Feb, 2025 7 commits
- [Frontend] Add backend-specific options for guided decoding (#13505) · bfbc0b32
  Joe Runde authored Feb 20, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  bfbc0b32
- Merge similar examples in `offline_inference` into single `basic` example (#12737) · 992e5c3d
  Harry Mellor authored Feb 20, 2025
  
  992e5c3d
- [2/n][ci] S3: Use full model path (#13564) · a64a8443
  Kevin H. Luu authored Feb 20, 2025
```
Signed-off-by: <>
```
  a64a8443
- [ci] Fix spec decode test (#13600) · aa1e62d0
  Kevin H. Luu authored Feb 20, 2025
  
  aa1e62d0
- [core] add sleep and wake up endpoint and v1 support (#12987) · ba811639
  youkaichao authored Feb 20, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: cennn <2523403608@qq.com>
Co-authored-by: cennn <2523403608@qq.com>
```
  ba811639
- [Misc] Qwen2.5 VL support LoRA (#13261) · 512368e3
  Jee Jee Li authored Feb 20, 2025
  
  512368e3
- [3/n][CI] Load Quantization test models with S3 (#13570) · 473f51cf
  Kevin H. Luu authored Feb 19, 2025
```
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
```
  473f51cf
19 Feb, 2025 6 commits
- [VLM][Bugfix] Pass processor kwargs properly on init (#13516) · 377d10bd
  Cyrus Leung authored Feb 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  377d10bd
- [Feature] Pluggable platform-specific scheduler (#13161) · 42333026
  Yannick Schnider authored Feb 19, 2025
```
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
```
  42333026
- [V1][Core] Generic mechanism for handling engine utility (#13060) · caf7ff44
  Nick Hill authored Feb 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  caf7ff44
- [Model][Speculative Decoding] DeepSeek MTP spec decode (#12755) · f525c0be
  Lucia Fang authored Feb 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f525c0be
- [Bugfix] Fix Positive Feature Layers in Llava Models (#13514) · 983a40a8
  Alex Brooks authored Feb 19, 2025
```
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
```
  983a40a8
- [1/n][CI] Load models in CI from S3 instead of HF (#13205) · d5d214ac
  Kevin H. Luu authored Feb 18, 2025
```
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
```
  d5d214ac
18 Feb, 2025 5 commits
- [V1] Optimize handling of sampling metadata and req_ids list (#13244) · 30172b49
  Nick Hill authored Feb 18, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  30172b49
- [V1][Tests] Adding additional testing for multimodal models to V1 (#13308) · a4d577b3
  Murali Andoorveedu authored Feb 18, 2025
```
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
```
  a4d577b3
- [Bugfix] Fix invalid rotary embedding unit test (#13431) · 38094584
  Liangfu Chen authored Feb 18, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  38094584
- Add outlines fallback when JSON schema has enum (#13449) · b53d7998
  Michael Goin authored Feb 18, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b53d7998
- [Model] Enable quantization support for `transformers` backend (#12960) · 67ef8f66
  Isotr0py authored Feb 18, 2025
  
  67ef8f66
17 Feb, 2025 1 commit
- [V1][Spec decode] Move drafter to model runner (#13363) · cd4a72a2
  Woosuk Kwon authored Feb 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  cd4a72a2