Commits · 467a96a5415dc896170cecc0bb83d9c49c2f3c5e · OpenDAS / vllm_cscc · GitLab

06 Feb, 2025 19 commits
- [V1] LoRA Support (#10957) · 467a96a5
  Varun Sundar Rabindranath authored Feb 06, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  467a96a5
- [Bugfix] Fix unsupported FA version check for Turing GPU (#12828) · 8108ac84
  Isotr0py authored Feb 07, 2025
  
  8108ac84
- [Doc] double quote cmake package in build.inc.md (#12840) · afe74f7a
  Jitse Klomp authored Feb 06, 2025
  
  afe74f7a
- [torch.compile] PyTorch 2.6 and nightly compatibility (#12393) · 09b95e36
  youkaichao authored Feb 07, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  09b95e36
- [Kernel] Make rotary_embedding ops more flexible with input shape (#12777) · 85ac82d2
  Isotr0py authored Feb 07, 2025
  
  85ac82d2
- [Misc] Remove unnecessary decode call (#12833) · 1e57b1ee
  Cyrus Leung authored Feb 07, 2025
  
  1e57b1ee
- [misc] Reduce number of config file requests to HuggingFace (#12797) · e152f295
  Kevin H. Luu authored Feb 06, 2025
```
Signed-off-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
```
  e152f295
- [Attention] Use FA3 for MLA on Hopper (#12807) · c786e757
  Lucas Wilkinson authored Feb 06, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  c786e757
- [Docs] Add Google Cloud Slides (#12814) · cefd56ee
  Simon Mo authored Feb 06, 2025
  
  cefd56ee
- [Misc] Update w2 scale loading for GPTQMarlinMoE (#12757) · 7ca9934f
  Dipika Sikka authored Feb 06, 2025
  
  7ca9934f
- [Misc] Improve error message for incorrect pynvml (#12809) · 0408efc6
  youkaichao authored Feb 06, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  0408efc6
- [Misc] Remove duplicated DeepSeek V2/V3 model definition (#12793) · 449d1bce
  Michael Goin authored Feb 06, 2025
  
  449d1bce
- Improve `TransformersModel` UX (#12785) · 1a6fcad4
  Harry Mellor authored Feb 06, 2025
  
  1a6fcad4
- [Bugfix] Fix the test_ultravox.py's license (#12806) · 56534cd5
  Lu Fang authored Feb 05, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  56534cd5
- [Model] LoRA Support for Ultravox model (#11253) · d88506dd
  Sumit Vij authored Feb 05, 2025
  
  d88506dd
- [Misc][Easy] Remove the space from the file name · 9cdea30b
  Lu Fang authored Feb 05, 2025
  
  9cdea30b
- [Bugfix] Better FP8 supported defaults · 76abd0c8
  Lucas Wilkinson authored Feb 05, 2025
  
  76abd0c8
- [ROCm][Kernel] Using the correct warp_size value · 5b19b930
  Gregory Shtrasberg authored Feb 05, 2025
  
  5b19b930
- [VLM] Update compatibility with transformers 4.49 · 75404d04
  Cyrus Leung authored Feb 06, 2025
  
  75404d04
05 Feb, 2025 20 commits
- [VLM] Qwen2.5-VL · bf3b79ef
  Roger Wang authored Feb 05, 2025
  
  bf3b79ef
- [Docs] Drop duplicate [source] links · 9a5b1554
  Russell Bryant authored Feb 05, 2025
  
  9a5b1554
- [VLM] Use shared field to pass token ids to model · a4ce74c1
  Cyrus Leung authored Feb 06, 2025
  
  a4ce74c1
- Add: Support for Sparse24Bitmask Compressed Models · 3b2005e1
  Rahul Tuli authored Feb 05, 2025
  
  3b2005e1
- [Hardware][Intel-Gaudi] Enable FusedSDPA support for Intel Gaudi (HPU) · af8486de
  Sanju C Sudhakaran authored Feb 06, 2025
  
  af8486de
- Merging PR #12536 · 4c3aac51
  Chen Zhang authored Feb 06, 2025
```
Merged via CLI script
```
  4c3aac51
- [core][distributed] exact ray placement control (#12732) · bc1bdece
  youkaichao authored Feb 06, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  bc1bdece
- [Bugfix] Fix 'ModuleNotFoundError: No module named... · 022bcc70
  Akash kaothalkar authored Feb 05, 2025
```
[Bugfix] Fix 'ModuleNotFoundError: No module named 'intel_extension_for_pytorch'' for --tensor-parallel-size more than 1  (#12546)
```
  022bcc70
- [Doc] Remove performance warning for auto_awq.md (#12743) · c53dc466
  Michael Goin authored Feb 05, 2025
  
  c53dc466
- [V1][Misc] Shorten `FinishReason` enum and use constant strings (#12760) · 3d09e592
  Nick Hill authored Feb 04, 2025
  
  3d09e592
- [Bugfix] Fix OpenVINO model runner (#12750) · fcf2e3d7
  Harry Mellor authored Feb 05, 2025
  
  fcf2e3d7
- [Doc] Update PR Reminder with link to Developer Slack (#12748) · 58b218d7
  Michael Goin authored Feb 05, 2025
  
  58b218d7
- [Model][Quant] Fix GLM, Fix fused module mappings for quantization (#12634) · 7ff7a638
  Kyle Sayers authored Feb 05, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  7ff7a638
- [Misc] Bump the compressed-tensors version (#12736) · 686006a2
  Dipika Sikka authored Feb 04, 2025
  
  686006a2
- [VLM] Add MLA with pure RoPE support for deepseek-vl2 models (#12729) · 98fd089f
  Isotr0py authored Feb 05, 2025
  
  98fd089f
- Refactor `Linear` handling in `TransformersModel` (#12727) · 249824c3
  Harry Mellor authored Feb 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  249824c3
- [ROCM][AMD][TRITON] Halving warps number for fw_prefill to reduce spilling (#12713) · 64862d10
  Aleksandr Malyshev authored Feb 04, 2025
```
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
```
  64862d10
- [Core] add and implement `VLLM_LOGITS_PROCESSOR_THREADS` (#12368) · b3a0d01e
  Aviv Keshet authored Feb 04, 2025
```
Signed-off-by: Aviv Keshet <akeshet@scaledcognition.com>
```
  b3a0d01e
- [Perf] Mem align KV caches for CUDA devices (MLA perf improvement) (#12676) · 75e94309
  Lucas Wilkinson authored Feb 04, 2025
```
Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: Lucas Wilkinson <lcwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
```
  75e94309
- [V1][Metrics] Add request_success_total counter, labelled with finish reason (#12579) · 233df6f5
  Mark McLoughlin authored Feb 05, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  233df6f5
04 Feb, 2025 1 commit
- [Bugfix] Fix CI failures for InternVL and Mantis models (#12728) · 18016a5e
  Cyrus Leung authored Feb 04, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  18016a5e