Commits · 389ebcf734ec1217d152a0adcc49ab7b5d26a504 · OpenDAS / vllm_cscc · GitLab

11 Mar, 2025 1 commit
- add VLLM_USE_TRITON_OPT_MLA to use optimized MLA attention · 1ee40c4d
  zhuwenwen authored Mar 11, 2025
```
update mla optest
```
  1ee40c4d
20 Feb, 2025 6 commits
- Merge similar examples in `offline_inference` into single `basic` example (#12737) · 992e5c3d
  Harry Mellor authored Feb 20, 2025
  
  992e5c3d
- [2/n][ci] S3: Use full model path (#13564) · a64a8443
  Kevin H. Luu authored Feb 20, 2025
```
Signed-off-by: <>
```
  a64a8443
- [ci] Fix spec decode test (#13600) · aa1e62d0
  Kevin H. Luu authored Feb 20, 2025
  
  aa1e62d0
- [core] add sleep and wake up endpoint and v1 support (#12987) · ba811639
  youkaichao authored Feb 20, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: cennn <2523403608@qq.com>
Co-authored-by: cennn <2523403608@qq.com>
```
  ba811639
- [Misc] Qwen2.5 VL support LoRA (#13261) · 512368e3
  Jee Jee Li authored Feb 20, 2025
  
  512368e3
- [3/n][CI] Load Quantization test models with S3 (#13570) · 473f51cf
  Kevin H. Luu authored Feb 19, 2025
```
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
```
  473f51cf
19 Feb, 2025 6 commits
- [VLM][Bugfix] Pass processor kwargs properly on init (#13516) · 377d10bd
  Cyrus Leung authored Feb 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  377d10bd
- [Feature] Pluggable platform-specific scheduler (#13161) · 42333026
  Yannick Schnider authored Feb 19, 2025
```
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
```
  42333026
- [V1][Core] Generic mechanism for handling engine utility (#13060) · caf7ff44
  Nick Hill authored Feb 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  caf7ff44
- [Model][Speculative Decoding] DeepSeek MTP spec decode (#12755) · f525c0be
  Lucia Fang authored Feb 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f525c0be
- [Bugfix] Fix Positive Feature Layers in Llava Models (#13514) · 983a40a8
  Alex Brooks authored Feb 19, 2025
```
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
```
  983a40a8
- [1/n][CI] Load models in CI from S3 instead of HF (#13205) · d5d214ac
  Kevin H. Luu authored Feb 18, 2025
```
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
```
  d5d214ac
18 Feb, 2025 5 commits
- [V1] Optimize handling of sampling metadata and req_ids list (#13244) · 30172b49
  Nick Hill authored Feb 18, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  30172b49
- [V1][Tests] Adding additional testing for multimodal models to V1 (#13308) · a4d577b3
  Murali Andoorveedu authored Feb 18, 2025
```
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
```
  a4d577b3
- [Bugfix] Fix invalid rotary embedding unit test (#13431) · 38094584
  Liangfu Chen authored Feb 18, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  38094584
- Add outlines fallback when JSON schema has enum (#13449) · b53d7998
  Michael Goin authored Feb 18, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b53d7998
- [Model] Enable quantization support for `transformers` backend (#12960) · 67ef8f66
  Isotr0py authored Feb 18, 2025
  
  67ef8f66
17 Feb, 2025 4 commits
- [V1][Spec decode] Move drafter to model runner (#13363) · cd4a72a2
  Woosuk Kwon authored Feb 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  cd4a72a2
- [V1] Get input tokens from scheduler (#13339) · 4c21ce9e
  Woosuk Kwon authored Feb 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4c21ce9e
- [Model] Support Mamba2 (Codestral Mamba) (#9292) · 1f69c4a8
  Tyler Michael Smith authored Feb 17, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
```
  1f69c4a8
- [Feature][Spec Decode] Simplify the use of Eagle Spec Decode (#12304) · 46cdd595
  shangmingc authored Feb 17, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  46cdd595
16 Feb, 2025 4 commits
- [Bugfix] Fix 2 Node and Spec Decode tests (#13341) · 5d2965b7
  Cyrus Leung authored Feb 16, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  5d2965b7
- [ci] skip failed tests for flashinfer (#13352) · 124776eb
  youkaichao authored Feb 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  124776eb
- [BugFix] Enhance test_pos_encoding to support execution on multi-devices (#13187) · dc0f7ccf
  wchen61 authored Feb 16, 2025
```
Signed-off-by: wchen61 <wchen61@foxmail.com>
```
  dc0f7ccf
- [V1][Spec Decode] Ngram Spec Decode (#12193) · 80f63a39
  Lily Liu authored Feb 15, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  80f63a39
15 Feb, 2025 4 commits
- [V1][PP] Run engine busy loop with batch queue (#13064) · 9206b3d7
  Cody Yu authored Feb 15, 2025
  
  9206b3d7
- [V1][Metrics] Add iteration_tokens_total histogram from V0 (#13288) · 2ad1bc7a
  Mark McLoughlin authored Feb 15, 2025
  
  2ad1bc7a
- [V1][CI] Fix failed v1-test because of min_p (#13316) · e7eea5a5
  Woosuk Kwon authored Feb 14, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e7eea5a5
- update test_moe.py · f112086f
  zhuwenwen authored Feb 15, 2025
  
  f112086f
14 Feb, 2025 10 commits
- [V1][Core] min_p sampling support (#13191) · a12934d3
  Aoyu authored Feb 15, 2025
```
Signed-off-by: Aoyu <aoyuzhan@amazon.com>
Co-authored-by: Aoyu <aoyuzhan@amazon.com>
```
  a12934d3
- [Core] Reduce TTFT with concurrent partial prefills (#10235) · 3bcb8c75
  Joe Runde authored Feb 14, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  3bcb8c75
- [Quant][Perf] Use moe_wna16 kernel by default for MoEs with many experts (#13236) · 5e5c8e09
  Michael Goin authored Feb 14, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  5e5c8e09
- Support logit_bias in v1 Sampler (#13079) · 6224a9f6
  Lu Fang authored Feb 14, 2025
  
  6224a9f6
- [WIP] TPU V1 Support Refactored (#13049) · 45f90bcb
  Alexander Matveev authored Feb 14, 2025
  
  45f90bcb
- [Bugfix][V1] GPUModelRunner._update_states should return True when there is a... · b0ccfc56
  Kero Liang authored Feb 14, 2025
```
[Bugfix][V1] GPUModelRunner._update_states should return True when there is a finished request in batch (#13126)
```
  b0ccfc56
- [V1] LoRA - Enable Serving Usecase (#12883) · cbc40128
  Varun Sundar Rabindranath authored Feb 14, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  cbc40128
- Consolidate Llama model usage in tests (#13094) · f2b20fe4
  Harry Mellor authored Feb 14, 2025
  
  f2b20fe4
- [Bugfix/CI] Turn test_compressed_tensors_2of4_sparse back on (#13250) · 09545c0a
  Tyler Michael Smith authored Feb 13, 2025
  
  09545c0a
- [Kernel][Bugfix] Refactor and Fix CUTLASS 2:4 Sparse Kernels (#13198) · c1e37bf7
  Tyler Michael Smith authored Feb 13, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  c1e37bf7