Commits · 6c5b7af1525a2013d7b1806dd6c0c9a53404be6d · OpenDAS / vllm_cscc

"csrc/vscode:/vscode.git/clone" did not exist on "6ccc0bfffbcf1b7e927cc3dcf4159fc74ff94d40"

21 Jun, 2024 1 commit
- [distributed][misc] use fork by default for mp (#5669) · 6c5b7af1
  youkaichao authored Jun 20, 2024
  
  6c5b7af1
20 Jun, 2024 7 commits
- [Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718) · 8065a7e2
  Michael Goin authored Jun 20, 2024
  
  8065a7e2
- [Bugfix] Fix the CUDA version check for FP8 support in the CUTLASS kernels (#5715) · 3f3b6b21
  Tyler Michael Smith authored Jun 20, 2024
  
  3f3b6b21
- [Kernel] Update Cutlass int8 kernel configs for SM80 (#5275) · a7dcc620
  Varun Sundar Rabindranath authored Jun 20, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  a7dcc620
- [Model] Port over CLIPVisionModel for VLMs (#5591) · ad137cd1
  Roger Wang authored Jun 20, 2024
  
  ad137cd1
- [Kernel] Update Cutlass int8 kernel configs for SM90 (#5514) · 111af1fa
  Varun Sundar Rabindranath authored Jun 20, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  111af1fa
- [Bugfix][Doc] FIx Duplicate Explicit Target Name Errors (#5703) · 1b2eaac3
  Roger Wang authored Jun 19, 2024
  
  1b2eaac3
- [Misc] Improve conftest (#5681) · 3730a1c8
  Cyrus Leung authored Jun 20, 2024
  
  3730a1c8
19 Jun, 2024 16 commits
- [ci] Limit num gpus if specified for A100 (#5694) · 949e49a6
  Kevin H. Luu authored Jun 19, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  949e49a6
- [Misc] Add per channel support for static activation quantization; update w8a8... · 4a30d7e3
  Dipika Sikka authored Jun 19, 2024
```
[Misc] Add per channel support for static activation quantization; update w8a8 schemes to share base classes (#5650)
```
  4a30d7e3
- [Doc] Update docker references (#5614) · e83db9e7
  Rafael Vasquez authored Jun 19, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  e83db9e7
- [Bugfix] AsyncLLMEngine hangs with asyncio.run (#5654) · 78687504
  zifeitong authored Jun 19, 2024
  
  78687504
- [ci][distributed] add tests for custom allreduce (#5689) · d571ca01
  youkaichao authored Jun 19, 2024
  
  d571ca01
- [Frontend][Bugfix] Fix preemption_mode -> preemption-mode for CLI arg in arg_utils.py (#5688) · afed90a0
  Michael Goin authored Jun 19, 2024
  
  afed90a0
- [ci] Add A100 queue into AWS CI template (#5648) · 3ee5c4bc
  Kevin H. Luu authored Jun 19, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  3ee5c4bc
- [CI/Build] Add tqdm to dependencies (#5680) · e9c2732b
  Cyrus Leung authored Jun 19, 2024
  
  e9c2732b
- [Misc]Add param max-model-len in benchmark_latency.py (#5629) · d8714530
  DearPlanet authored Jun 19, 2024
  
  d8714530
- [Bugfix] Fix sampling_params passed incorrectly in Phi3v example (#5684) · 7d46c8d3
  Isotr0py authored Jun 19, 2024
  
  7d46c8d3
- [Model] Add FP8 kv cache for Qwen2 (#5656) · da971ec7
  Michael Goin authored Jun 19, 2024
  
  da971ec7
- [misc][distributed] use 127.0.0.1 for single-node (#5619) · 3eea7488
  youkaichao authored Jun 19, 2024
  
  3eea7488
- [Bugfix][CI/Build][AMD][ROCm]Fixed the cmake build bug which generate garbage... · f758aed0
  Hongxia Yang authored Jun 19, 2024
```
[Bugfix][CI/Build][AMD][ROCm]Fixed the cmake build bug which generate garbage on certain devices (#5641)
```
  f758aed0
- [Bugfix] Added test for sampling repetition penalty bug. (#5659) · e5150f2c
  Thomas Parnell authored Jun 19, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  e5150f2c
- [Bugfix] Fix Phi-3 Long RoPE scaling implementation (#5628) · 59a1eb59
  Shukant Pal authored Jun 18, 2024
  
  59a1eb59
- [Bugfix] Fix w8a8 benchmarks for int8 case (#5643) · 6820724e
  Tyler Michael Smith authored Jun 18, 2024
  
  6820724e
18 Jun, 2024 16 commits
- [Bugfix] Fix CUDA version check for mma warning suppression (#5642) · b23ce920
  Tyler Michael Smith authored Jun 18, 2024
  
  b23ce920
- [Doc] Added cerebrium as Integration option (#5553) · 2bd231a7
  milo157 authored Jun 18, 2024
  
  2bd231a7
- [Bugfix] Fix for inconsistent behaviour related to sampling and repetition penalties (#5639) · 8a173382
  Thomas Parnell authored Jun 18, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  8a173382
- [Model] LoRA support added for command-r (#5178) · 07feecde
  sergey-tinkoff authored Jun 18, 2024
  
  07feecde
- [ci] Setup Release pipeline and build release wheels with cache (#5610) · 19091efc
  Kevin H. Luu authored Jun 18, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  19091efc
- [Misc] Add channel-wise quantization support for w8a8 dynamic per token... · 95db455e
  Dipika Sikka authored Jun 18, 2024
```
[Misc] Add channel-wise quantization support for w8a8 dynamic per token activation quantization (#5542)
```
  95db455e
- [Misc] Add OpenTelemetry support (#4687) · 7879f24d
  Ronen Schaffer authored Jun 18, 2024
```
This PR adds basic support for OpenTelemetry distributed tracing.
It includes changes to enable tracing functionality and improve monitoring capabilities.

I've also added a markdown with print-screens to guide users how to use this feature. You can find it here
```
  7879f24d
- [ci] Deprecate original CI template (#5624) · 13db4369
  Kevin H. Luu authored Jun 18, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  13db4369
- [CI/Build][Misc] Update Pytest Marker for VLMs (#5623) · 4ad7b53e
  Roger Wang authored Jun 18, 2024
  
  4ad7b53e
- [Misc] Remove import from transformers logging (#5625) · f0cc0e68
  Chang Su authored Jun 18, 2024
  
  f0cc0e68
- [bugfix][distributed] improve p2p capability test (#5612) · db5ec52a
  youkaichao authored Jun 18, 2024
```
[bugfix][distributed] do not error if two processes do not agree on p2p capability (#5612)
```
  db5ec52a
- [CI] Avoid naming different metrics with the same name in performance benchmark (#5615) · 114d7270
  Kuntai Du authored Jun 17, 2024
  
  114d7270
- [Misc] Fix typo (#5618) · 32c86e49
  Cyrus Leung authored Jun 18, 2024
  
  32c86e49
- [misc][typo] fix typo (#5620) · 8eadcf0b
  youkaichao authored Jun 17, 2024
  
  8eadcf0b
- [Kernel] Add punica dimensions for Granite 13b (#5559) · 5002175e
  Joe Runde authored Jun 17, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  5002175e
- [Model] Initialize Phi-3-vision support (#4986) · daef218b
  Isotr0py authored Jun 18, 2024
  
  daef218b