Commits · a3c9435d93fb7609977da5d90f839b9987c8b264 · OpenDAS / vllm_cscc

"tests/vscode:/vscode.git/clone" did not exist on "987506bca63d95d9ff4ebb3a37e3f2447fa0757c"

09 Jul, 2024 1 commit
- [hardware][cuda] use device id under CUDA_VISIBLE_DEVICES for get_device_capability (#6216) · a3c9435d
  youkaichao authored Jul 08, 2024
  
  a3c9435d
08 Jul, 2024 5 commits
- [Bugfix] Mamba cache Cuda Graph padding (#6214) · ddc369fb
  tomeras91 authored Jul 08, 2024
  
  ddc369fb
- [Bugfix] use diskcache in outlines _get_guide #5436 (#6203) · 185ad31f
  Eric authored Jul 09, 2024
  
  185ad31f
- [Kernel] Correctly invoke prefill & decode kernels for cross-attention... · 543aa485
  afeldman-nm authored Jul 08, 2024
```
[Kernel] Correctly invoke prefill & decode kernels for cross-attention (towards eventual encoder/decoder model support) (#4888)
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  543aa485
- [Kernel] reloading fused_moe config on the last chunk (#6210) · f7a8fa39
  Avshalom Manevich authored Jul 08, 2024
  
  f7a8fa39
- do not exclude `object` field in CompletionStreamResponse (#6196) · 16620f43
  kczimm authored Jul 07, 2024
  
  16620f43
07 Jul, 2024 3 commits
- [misc][frontend] log all available endpoints (#6195) · 3b08fe2b
  youkaichao authored Jul 07, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  3b08fe2b
- [ Misc ] Support Fp8 via `llm-compressor` (#6110) · abfe705a
  Robert Shaw authored Jul 07, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  abfe705a
- [Model] Add PaliGemma (#5189) · 6206dcb2
  Roger Wang authored Jul 06, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6206dcb2
06 Jul, 2024 1 commit
- [Doc] Move guide for multimodal model and other improvements (#6168) · 93893800
  Cyrus Leung authored Jul 06, 2024
  
  93893800
05 Jul, 2024 5 commits
- bump version to v0.5.1 (#6157) · abad5746
  Simon Mo authored Jul 05, 2024
  
  abad5746
- [Bugfix] Add verbose error if scipy is missing for blocksparse attention (#5695) · e58294dd
  JGSweets authored Jul 05, 2024
  
  e58294dd
- [Frontend] Continuous usage stats in OpenAI completion API (#5742) · f1e15da6
  jvlunteren authored Jul 05, 2024
  
  f1e15da6
- [VLM] Cleanup validation and update docs (#6149) · ea4b5704
  Cyrus Leung authored Jul 05, 2024
  
  ea4b5704
- [VLM] Improve consistency between feature size calculation and dummy data for profiling (#6146) · a41357e9
  Roger Wang authored Jul 04, 2024
  
  a41357e9
04 Jul, 2024 6 commits
- [VLM] Calculate maximum number of multi-modal tokens by model (#6121) · ae96ef8f
  Cyrus Leung authored Jul 05, 2024
  
  ae96ef8f
- [Kernel][Model] logits_soft_cap for Gemma2 with flashinfer (#6051) · 69ec3ca1
  Lily Liu authored Jul 04, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  69ec3ca1
- [Hardware][Intel CPU] Adding intel openmp tunings in Docker file (#6008) · 81d7a50f
  Yuan authored Jul 05, 2024
```
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
```
  81d7a50f
- [ROCm][AMD][Model]Adding alibi slopes support in ROCm triton flash attention... · 56b325e9
  Gregory Shtrasberg authored Jul 04, 2024
```
[ROCm][AMD][Model]Adding alibi slopes support in ROCm triton flash attention and naive flash attention (#6043)
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
```
  56b325e9
- [CI/Build] Cleanup VLM tests (#6107) · 3dd50708
  Cyrus Leung authored Jul 04, 2024
  
  3dd50708
- [Distributed][Core] Support Py39 and Py38 for PP (#6120) · 0ed646b7
  Murali Andoorveedu authored Jul 03, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  0ed646b7
03 Jul, 2024 12 commits
- [Bugfix] set OMP_NUM_THREADS to 1 by default for multiprocessing (#6109) · 1dab9bc8
  Travis Johnson authored Jul 03, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  1dab9bc8
- [core][distributed] support n layers % pp size != 0 (#6115) · 3de6e6a3
  youkaichao authored Jul 03, 2024
  
  3de6e6a3
- [ Misc ] Clean Up `CompressedTensorsW8A8` (#6113) · 62963d12
  Robert Shaw authored Jul 03, 2024
  
  62963d12
- [vlm] Remove vision language config. (#6089) · d9e98f42
  xwjiang2010 authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d9e98f42
- [core][distributed] custom allreduce when pp size > 1 (#6117) · 3c6325f0
  youkaichao authored Jul 03, 2024
  
  3c6325f0
- [Kernel] Expand FP8 support to Ampere GPUs using FP8 Marlin (#5975) · 47f0954a
  Michael Goin authored Jul 03, 2024
  
  47f0954a
- [Bugfix] Fix `compute_logits` in Jamba (#6093) · 7cd2ebb0
  Roger Wang authored Jul 03, 2024
  
  7cd2ebb0
- [VLM][Frontend] Proper Image Prompt Formatting from OpenAI API (#6091) · 3a86b54f
  Roger Wang authored Jul 02, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  3a86b54f
- [misc][distributed] error on invalid state (#6092) · f6662071
  youkaichao authored Jul 02, 2024
  
  f6662071
- [BugFix] Avoid unnecessary Ray import warnings (#6079) · d830656a
  Nick Hill authored Jul 02, 2024
  
  d830656a
- [Core] Dynamic image size support for VLMs (#5276) · 9831aec4
  Cyrus Leung authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  9831aec4
- [hardware][misc] introduce platform abstraction (#6080) · 482045ee
  youkaichao authored Jul 02, 2024
  
  482045ee
02 Jul, 2024 7 commits

[Model] Jamba support (#4115) · 9d6a8daa

Mor Zusman authored Jul 03, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Erez Schwartz <erezs@ai21.com>
Co-authored-by: Mor Zusman <morz@ai21.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Tomer Asida <tomera@ai21.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

9d6a8daa

[CORE] Quantized lm-head Framework (#4442) · ee93f4f9
Qubitium-ModelCloud authored Jul 03, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: ZX <zx@lbx.dev>
```
ee93f4f9

[ Misc ] Refactor MoE to isolate Fp8 From Mixtral (#5970) · 7c008c51

Robert Shaw authored Jul 02, 2024


Co-authored-by: Robert Shaw <rshaw@neuralmagic>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

7c008c51

[Core] Pipeline Parallel Support (#4412) · c5832d2a
Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
c5832d2a
[Speculative Decoding] MLPSpeculator Tensor Parallel support (1/2) (#6050) · 15aba081
Sirej Dua authored Jul 02, 2024
```
Co-authored-by: Sirej Dua <sirej.dua@databricks.com>
Co-authored-by: Sirej Dua <Sirej Dua>
```
15aba081

[VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c

xwjiang2010 authored Jul 02, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

98d6682c

[Frontend] Add template related params to request (#5709) · 2c37540a
danieljannai21 authored Jul 02, 2024

2c37540a