Commits · dbfe254eda918bc3cc52cf448d518824ad6593b9 · OpenDAS / vllm_cscc

14 Jul, 2024 2 commits
- [Feature] vLLM CLI (#5090) · dbfe254e
  Ethan Xu authored Jul 14, 2024
```
Co-authored-by: simon-mo <simon.mo@hey.com>
```
  dbfe254e
- [Model] Initialize Fuyu-8B support (#3924) · 540c0368
  Isotr0py authored Jul 14, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  540c0368
13 Jul, 2024 1 commit

[ci] try to add multi-node tests (#6280) · 41708e50

youkaichao authored Jul 12, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

41708e50

12 Jul, 2024 6 commits
- [Misc] Add generated git commit hash as `vllm.__commit__` (#6386) · 111fc6e7
  Michael Goin authored Jul 12, 2024
  
  111fc6e7
- [Misc] add fixture to guided processor tests (#6341) · b039cbbc
  Yihuan Bu authored Jul 12, 2024
  
  b039cbbc
- [Bugfix] Fix dtype mismatch in PaliGemma (#6367) · 024ad87c
  Cyrus Leung authored Jul 12, 2024
  
  024ad87c
- [ Misc ] Support Models With Bias in `compressed-tensors` integration (#6356) · aea19f09
  Robert Shaw authored Jul 12, 2024
  
  aea19f09
- [ROCm][AMD] unify CUDA_VISIBLE_DEVICES usage in cuda/rocm (#6352) · b6c16cf8
  Hongxia Yang authored Jul 12, 2024
  
  b6c16cf8
- [Misc] Remove flashinfer warning, add flashinfer tests to CI (#6351) · d6ab5289
  Lily Liu authored Jul 11, 2024
  
  d6ab5289
11 Jul, 2024 3 commits
- [ BugFix ] Prompt Logprobs Detokenization (#6223) · 7ed6a4f0
  Robert Shaw authored Jul 11, 2024
```
Co-authored-by: Zifei Tong <zifeitong@gmail.com>
```
  7ed6a4f0
- [bug fix] Fix llava next feature size calculation. (#6339) · 1df43de9
  xwjiang2010 authored Jul 11, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
```
  1df43de9
- [ Misc ] Refactor Marlin Python Utilities (#6082) · b675069d
  Robert Shaw authored Jul 11, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
```
  b675069d
10 Jul, 2024 3 commits
- [Speculative Decoding] Enabling bonus token in speculative decoding for KV... · ae151d73
  sroy745 authored Jul 10, 2024
```
[Speculative Decoding] Enabling bonus token in speculative decoding for KV cache based models (#5765)
```
  ae151d73
- [core][distributed] zmq fallback for broadcasting large objects (#6183) · da78caec
  youkaichao authored Jul 09, 2024
```
[core][distributed] add zmq fallback for broadcasting large objects (#6183)
```
  da78caec
- [Speculative Decoding] Medusa Implementation with Top-1 proposer (#4978) · 2416b26e
  Abhinav Goyal authored Jul 10, 2024
  
  2416b26e
09 Jul, 2024 1 commit

[CORE] Adding support for insertion of soft-tuned prompts (#4645) · 4d6ada94

Swapnil Parekh authored Jul 09, 2024


Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

4d6ada94

08 Jul, 2024 2 commits
- [Bugfix] Mamba cache Cuda Graph padding (#6214) · ddc369fb
  tomeras91 authored Jul 08, 2024
  
  ddc369fb
- [Kernel] Correctly invoke prefill & decode kernels for cross-attention... · 543aa485
  afeldman-nm authored Jul 08, 2024
```
[Kernel] Correctly invoke prefill & decode kernels for cross-attention (towards eventual encoder/decoder model support) (#4888)
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  543aa485
07 Jul, 2024 2 commits
- [ Misc ] Support Fp8 via `llm-compressor` (#6110) · abfe705a
  Robert Shaw authored Jul 07, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  abfe705a
- [Model] Add PaliGemma (#5189) · 6206dcb2
  Roger Wang authored Jul 06, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6206dcb2
05 Jul, 2024 1 commit
- [Frontend] Continuous usage stats in OpenAI completion API (#5742) · f1e15da6
  jvlunteren authored Jul 05, 2024
  
  f1e15da6
04 Jul, 2024 2 commits
- [Kernel][Model] logits_soft_cap for Gemma2 with flashinfer (#6051) · 69ec3ca1
  Lily Liu authored Jul 04, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  69ec3ca1
- [CI/Build] Cleanup VLM tests (#6107) · 3dd50708
  Cyrus Leung authored Jul 04, 2024
  
  3dd50708
03 Jul, 2024 6 commits
- [ Misc ] Clean Up `CompressedTensorsW8A8` (#6113) · 62963d12
  Robert Shaw authored Jul 03, 2024
  
  62963d12
- [vlm] Remove vision language config. (#6089) · d9e98f42
  xwjiang2010 authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d9e98f42
- [Kernel] Expand FP8 support to Ampere GPUs using FP8 Marlin (#5975) · 47f0954a
  Michael Goin authored Jul 03, 2024
  
  47f0954a
- [CI] Fix base url doesn't strip "/" (#6087) · d18bab35
  SangBin Cho authored Jul 03, 2024
  
  d18bab35
- [Core] Dynamic image size support for VLMs (#5276) · 9831aec4
  Cyrus Leung authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  9831aec4
- [hardware][misc] introduce platform abstraction (#6080) · 482045ee
  youkaichao authored Jul 02, 2024
  
  482045ee
02 Jul, 2024 8 commits

[Model] Jamba support (#4115) · 9d6a8daa

Mor Zusman authored Jul 03, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Erez Schwartz <erezs@ai21.com>
Co-authored-by: Mor Zusman <morz@ai21.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Tomer Asida <tomera@ai21.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

9d6a8daa

[CORE] Quantized lm-head Framework (#4442) · ee93f4f9
Qubitium-ModelCloud authored Jul 03, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: ZX <zx@lbx.dev>
```
ee93f4f9

[ Misc ] Refactor MoE to isolate Fp8 From Mixtral (#5970) · 7c008c51

Robert Shaw authored Jul 02, 2024


Co-authored-by: Robert Shaw <rshaw@neuralmagic>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

7c008c51

Update conftest.py (#6076) · 4d26d806
Robert Shaw authored Jul 02, 2024

4d26d806
[Core] Pipeline Parallel Support (#4412) · c5832d2a
Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
c5832d2a
[Speculative Decoding] MLPSpeculator Tensor Parallel support (1/2) (#6050) · 15aba081
Sirej Dua authored Jul 02, 2024
```
Co-authored-by: Sirej Dua <sirej.dua@databricks.com>
Co-authored-by: Sirej Dua <Sirej Dua>
```
15aba081

[VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c

xwjiang2010 authored Jul 02, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

98d6682c

[Core] Optimize block_manager_v2 vs block_manager_v1 (to make V2 default) (#5602) · 3476ed08
Alexander Matveev authored Jul 01, 2024

3476ed08

01 Jul, 2024 3 commits
- [Bugfix] adding chunking mechanism to fused_moe to handle large inputs (#6029) · 12a59959
  Avshalom Manevich authored Jul 02, 2024
  
  12a59959
- [Speculative Decoding 2/2 ] Integrate typical acceptance sampler into Spec Decode Worker (#5348) · 80ca1e6a
  sroy745 authored Jul 01, 2024
  
  80ca1e6a
- [misc][cuda] use nvml to avoid accidentally cuda initialization (#6007) · 614aa512
  youkaichao authored Jun 30, 2024
  
  614aa512