Commits · ff9ddbceee63efba6ba1f8d4dc66a92f1191da04 · OpenDAS / vllm_cscc · GitLab

22 Jun, 2024 3 commits
- [Misc] Remove #4789 workaround left in vllm/entrypoints/openai/run_batch.py (#5756) · ff9ddbce
  zifeitong authored Jun 21, 2024
  
  ff9ddbce
- [Model] Support Qwen-VL and Qwen-VL-Chat models with text-only inputs (#5710) · 9c62db07
  Jie Fu (傅杰) authored Jun 22, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  9c62db07
- [CI][Hardware][Intel GPU] add Intel GPU(XPU) ci pipeline (#5616) · cf90ae01
  Kunshang Ji authored Jun 22, 2024
  
  cf90ae01
21 Jun, 2024 11 commits
- [LoRA] Add support for pinning lora adapters in the LRU cache (#5603) · f5dda63e
  rohithkrn authored Jun 21, 2024
  
  f5dda63e
- [ci][test] fix ca test in main (#5746) · 71875073
  youkaichao authored Jun 21, 2024
  
  71875073
- [BugFix] exclude version 1.15.0 for modelscope (#5668) · f1e72cc1
  zhyncs authored Jun 22, 2024
  
  f1e72cc1
- [Doc] Documentation on supported hardware for quantization methods (#5745) · 5b15bde5
  Michael Goin authored Jun 21, 2024
  
  5b15bde5
- [Kernel][CPU] Add Quick `gelu` to CPU (#5717) · bd620b01
  Roger Wang authored Jun 20, 2024
  
  bd620b01
- [Core][Distributed] add shm broadcast (#5399) · d9a252bc
  youkaichao authored Jun 20, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  d9a252bc
- [Bugfix] Add fully sharded layer for QKVParallelLinearWithLora (#5665) · 67005a07
  Jee Li authored Jun 21, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  67005a07
- [BugFix] Fix test_phi3v.py (#5725) · c35e4a3d
  Chang Su authored Jun 20, 2024
  
  c35e4a3d
- [Kernel] Add punica dimension for Qwen2 LoRA (#5441) · 1f567421
  Jinzhen Lin authored Jun 21, 2024
  
  1f567421
- [Model] MLPSpeculator speculative decoding support (#4947) · b12518d3
  Joshua Rosenkranz authored Jun 20, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Davis Wertheimer <Davis.Wertheimer@ibm.com>
```
  b12518d3
- [distributed][misc] use fork by default for mp (#5669) · 6c5b7af1
  youkaichao authored Jun 20, 2024
  
  6c5b7af1
20 Jun, 2024 7 commits
- [Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718) · 8065a7e2
  Michael Goin authored Jun 20, 2024
  
  8065a7e2
- [Bugfix] Fix the CUDA version check for FP8 support in the CUTLASS kernels (#5715) · 3f3b6b21
  Tyler Michael Smith authored Jun 20, 2024
  
  3f3b6b21
- [Kernel] Update Cutlass int8 kernel configs for SM80 (#5275) · a7dcc620
  Varun Sundar Rabindranath authored Jun 20, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  a7dcc620
- [Model] Port over CLIPVisionModel for VLMs (#5591) · ad137cd1
  Roger Wang authored Jun 20, 2024
  
  ad137cd1
- [Kernel] Update Cutlass int8 kernel configs for SM90 (#5514) · 111af1fa
  Varun Sundar Rabindranath authored Jun 20, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  111af1fa
- [Bugfix][Doc] FIx Duplicate Explicit Target Name Errors (#5703) · 1b2eaac3
  Roger Wang authored Jun 19, 2024
  
  1b2eaac3
- [Misc] Improve conftest (#5681) · 3730a1c8
  Cyrus Leung authored Jun 20, 2024
  
  3730a1c8
19 Jun, 2024 16 commits
- [ci] Limit num gpus if specified for A100 (#5694) · 949e49a6
  Kevin H. Luu authored Jun 19, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  949e49a6
- [Misc] Add per channel support for static activation quantization; update w8a8... · 4a30d7e3
  Dipika Sikka authored Jun 19, 2024
```
[Misc] Add per channel support for static activation quantization; update w8a8 schemes to share base classes (#5650)
```
  4a30d7e3
- [Doc] Update docker references (#5614) · e83db9e7
  Rafael Vasquez authored Jun 19, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  e83db9e7
- [Bugfix] AsyncLLMEngine hangs with asyncio.run (#5654) · 78687504
  zifeitong authored Jun 19, 2024
  
  78687504
- [ci][distributed] add tests for custom allreduce (#5689) · d571ca01
  youkaichao authored Jun 19, 2024
  
  d571ca01
- [Frontend][Bugfix] Fix preemption_mode -> preemption-mode for CLI arg in arg_utils.py (#5688) · afed90a0
  Michael Goin authored Jun 19, 2024
  
  afed90a0
- [ci] Add A100 queue into AWS CI template (#5648) · 3ee5c4bc
  Kevin H. Luu authored Jun 19, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  3ee5c4bc
- [CI/Build] Add tqdm to dependencies (#5680) · e9c2732b
  Cyrus Leung authored Jun 19, 2024
  
  e9c2732b
- [Misc]Add param max-model-len in benchmark_latency.py (#5629) · d8714530
  DearPlanet authored Jun 19, 2024
  
  d8714530
- [Bugfix] Fix sampling_params passed incorrectly in Phi3v example (#5684) · 7d46c8d3
  Isotr0py authored Jun 19, 2024
  
  7d46c8d3
- [Model] Add FP8 kv cache for Qwen2 (#5656) · da971ec7
  Michael Goin authored Jun 19, 2024
  
  da971ec7
- [misc][distributed] use 127.0.0.1 for single-node (#5619) · 3eea7488
  youkaichao authored Jun 19, 2024
  
  3eea7488
- [Bugfix][CI/Build][AMD][ROCm]Fixed the cmake build bug which generate garbage... · f758aed0
  Hongxia Yang authored Jun 19, 2024
```
[Bugfix][CI/Build][AMD][ROCm]Fixed the cmake build bug which generate garbage on certain devices (#5641)
```
  f758aed0
- [Bugfix] Added test for sampling repetition penalty bug. (#5659) · e5150f2c
  Thomas Parnell authored Jun 19, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  e5150f2c
- [Bugfix] Fix Phi-3 Long RoPE scaling implementation (#5628) · 59a1eb59
  Shukant Pal authored Jun 18, 2024
  
  59a1eb59
- [Bugfix] Fix w8a8 benchmarks for int8 case (#5643) · 6820724e
  Tyler Michael Smith authored Jun 18, 2024
  
  6820724e
18 Jun, 2024 3 commits
- [Bugfix] Fix CUDA version check for mma warning suppression (#5642) · b23ce920
  Tyler Michael Smith authored Jun 18, 2024
  
  b23ce920
- [Doc] Added cerebrium as Integration option (#5553) · 2bd231a7
  milo157 authored Jun 18, 2024
  
  2bd231a7
- [Bugfix] Fix for inconsistent behaviour related to sampling and repetition penalties (#5639) · 8a173382
  Thomas Parnell authored Jun 18, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  8a173382