Commits · df1a21131d951ba8ee65363aeb9b9486f569aa4f · OpenDAS / vllm_cscc · GitLab

22 Aug, 2024 2 commits
- [Model] Fix Phi-3.5-vision-instruct 'num_crops' issue (#7710) · df1a2113
  zifeitong authored Aug 21, 2024
  
  df1a2113
- [Kernel] Replaced `blockReduce[...]` functions with `cub::BlockReduce` (#7233) · 7937009a
  Luka Govedič authored Aug 21, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  7937009a
21 Aug, 2024 9 commits
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527) · 8678a69a
  Dipika Sikka authored Aug 21, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  8678a69a
- [Model] Add UltravoxModel and UltravoxConfig (#7615) · 1ca0d4f8
  Peter Salas authored Aug 21, 2024
  
  1ca0d4f8
- [Frontend] Improve Startup Failure UX (#7716) · 970dfdc0
  Robert Shaw authored Aug 21, 2024
  
  970dfdc0
- [Bugfix][Frontend] Fix Issues Under High Load With `zeromq` Frontend (#7394) · f7e3b0c5
  Robert Shaw authored Aug 21, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  f7e3b0c5
- [BUG] fix crash on flashinfer backend with cudagraph disabled, when attention... · 53328d75
  LI MOU authored Aug 21, 2024
```
[BUG] fix crash on flashinfer backend with cudagraph disabled, when attention group_size not in [1,2,4,8] (#7509)
```
  53328d75
- [BugFix] Avoid premature async generator exit and raise all exception variations (#7698) · c75363fb
  Nick Hill authored Aug 21, 2024
  
  c75363fb
- [mypy] Enable following imports for entrypoints (#7248) · baaedfdb
  Cyrus Leung authored Aug 21, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Fei <dfdfcai4@gmail.com>
```
  baaedfdb
- [Model] Add AWQ quantization support for InternVL2 model (#7187) · 12e1c65b
  Isotr0py authored Aug 21, 2024
  
  12e1c65b
- [ci][test] adjust max wait time for cpu offloading test (#7709) · 9e51b6a6
  youkaichao authored Aug 20, 2024
  
  9e51b6a6
20 Aug, 2024 4 commits
- [Core] Add `AttentionState` abstraction (#7663) · 3b682179
  Antoni Baum authored Aug 20, 2024
  
  3b682179
- [VLM][Model] Add test for InternViT vision encoder (#7409) · aae6927b
  Isotr0py authored Aug 20, 2024
  
  aae6927b
- [Kernel] (1/N) Machete - Hopper Optimized Mixed Precision Linear Kernel (#7174) · 5288c06a
  Lucas Wilkinson authored Aug 20, 2024
  
  5288c06a
- [Speculative Decoding] Fixing hidden states handling in batch expansion (#7508) · 312f7612
  Abhinav Goyal authored Aug 20, 2024
  
  312f7612
19 Aug, 2024 5 commits
- [Core] Support tensor parallelism for GGUF quantization (#7520) · 7601cb04
  Isotr0py authored Aug 20, 2024
  
  7601cb04
- [core] Multi Step Scheduling (#7000) · 47b65a55
  William Lin authored Aug 19, 2024
```
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
```
  47b65a55
- [MISC] Add prefix cache hit rate to metrics (#7606) · 3ac50b47
  Cody Yu authored Aug 19, 2024
  
  3ac50b47
- [Core] Use flashinfer sampling kernel when available (#7137) · f710fb52
  Peng Guanwen authored Aug 19, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  f710fb52
- [Core] Optimize SPMD architecture with delta + serialization optimization (#7109) · ff7ec82c
  SangBin Cho authored Aug 18, 2024
  
  ff7ec82c
18 Aug, 2024 2 commits
- [CI/Build] Add text-only test for Qwen models (#7475) · 40e1360b
  Alex Brooks authored Aug 18, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  40e1360b
- [ Bugfix ] Fix Prometheus Metrics With `zeromq` Frontend (#7279) · e3b31821
  Robert Shaw authored Aug 18, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  e3b31821
17 Aug, 2024 4 commits
- [VLM] Refactor `MultiModalConfig` initialization and profiling (#7530) · bbf55c48
  Roger Wang authored Aug 17, 2024
  
  bbf55c48
- [ci][test] allow longer wait time for api server (#7629) · 832163b8
  youkaichao authored Aug 17, 2024
  
  832163b8
- [ci][test] fix engine/logger test (#7621) · 5bf45db7
  youkaichao authored Aug 16, 2024
  
  5bf45db7
- [aDAG] Unflake aDAG + PP tests (#7600) · 4706eb62
  SangBin Cho authored Aug 16, 2024
  
  4706eb62
16 Aug, 2024 10 commits
- [Core] Fix tracking of model forward time in case of PP>1 (#7440) · 93478b63
  Mahesh Keralapura authored Aug 16, 2024
```
[Core] Fix tracking of model forward time to the span traces in case of PP>1 (#7440)
```
  93478b63
- [Kernel] W8A16 Int8 inside FusedMoE (#7415) · 7fc23be8
  Mor Zusman authored Aug 16, 2024
  
  7fc23be8
- [Feature][Hardware][Amd] Add fp8 Linear Layer for Rocm (#7210) · e837b624
  Charlie Fu authored Aug 16, 2024
  
  e837b624
- register custom op for flash attn and use from torch.ops (#7536) · 54bd9a03
  youkaichao authored Aug 15, 2024
  
  54bd9a03
- [Misc/Testing] Use `torch.testing.assert_close` (#7324) · 50b8d08d
  jon-chuang authored Aug 15, 2024
  
  50b8d08d
- [CI] Move quantization cpu offload tests out of fastcheck (#7574) · e1655287
  Michael Goin authored Aug 16, 2024
  
  e1655287
- Chat method for offline llm (#5049) · 3b19e39d
  nunjunj authored Aug 16, 2024
```
Co-authored-by: nunjunj <ray@g-3ff9f30f2ed650001.c.vllm-405802.internal>
Co-authored-by: nunjunj <ray@g-1df6075697c3f0001.c.vllm-405802.internal>
Co-authored-by: nunjunj <ray@g-c5a2c23abc49e0001.c.vllm-405802.internal>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3b19e39d
- [ci/test] rearrange tests and make adag test soft fail (#7572) · 4cd7d47f
  youkaichao authored Aug 15, 2024
  
  4cd7d47f
- [Feature]: Add OpenAI server prompt_logprobs support #6508 (#7453) · f878c8fe
  Grant Pinkert authored Aug 16, 2024
  
  f878c8fe
- [Misc] Add quantization config support for speculative model. (#7343) · b67ae00c
  shangmingc authored Aug 16, 2024
  
  b67ae00c
14 Aug, 2024 4 commits
- [Misc] Revert `compressed-tensors` code reuse (#7521) · f55a9aea
  Kyle Sayers authored Aug 14, 2024
  
  f55a9aea
- [VLM][Core] Support profiling with multiple multi-modal inputs per prompt (#7126) · 3f674a49
  Cyrus Leung authored Aug 15, 2024
  
  3f674a49
- [Misc] Deprecation Warning when setting --engine-use-ray (#7424) · 70b746ef
  Wallas Henrique authored Aug 14, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: youkaichao <youkaichao@126.com>
```
  70b746ef
- [misc][ci] fix cpu test with plugins (#7489) · ea49e6a3
  youkaichao authored Aug 13, 2024
  
  ea49e6a3