Commits · 16620f439db1f2cc91b5582b59fc8845cbb02881 · OpenDAS / vllm_cscc

08 Jul, 2024 1 commit
- do not exclude `object` field in CompletionStreamResponse (#6196) · 16620f43
  kczimm authored Jul 07, 2024
  
  16620f43
07 Jul, 2024 4 commits
- [misc][frontend] log all available endpoints (#6195) · 3b08fe2b
  youkaichao authored Jul 07, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  3b08fe2b
- [ Misc ] Support Fp8 via `llm-compressor` (#6110) · abfe705a
  Robert Shaw authored Jul 07, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  abfe705a
- add benchmark for fix length input and output (#5857) · 333306a2
  Haichuan authored Jul 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  333306a2
- [Model] Add PaliGemma (#5189) · 6206dcb2
  Roger Wang authored Jul 06, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6206dcb2
06 Jul, 2024 3 commits
- [Doc] Move guide for multimodal model and other improvements (#6168) · 93893800
  Cyrus Leung authored Jul 06, 2024
  
  93893800
- [Doc] Reorganize Supported Models by Type (#6167) · 175c43ec
  Roger Wang authored Jul 05, 2024
  
  175c43ec
- Move release wheel env var to Dockerfile instead (#6163) · bc96d5c3
  Simon Mo authored Jul 05, 2024
  
  bc96d5c3
05 Jul, 2024 9 commits
- Fix release wheel build env var (#6162) · f0250620
  Simon Mo authored Jul 05, 2024
  
  f0250620
- Update wheel builds to strip debug (#6161) · 2de490d6
  Simon Mo authored Jul 05, 2024
  
  2de490d6
- [Docs] Fix readthedocs for tag build (#6158) · 79d406e9
  Simon Mo authored Jul 05, 2024
  
  79d406e9
- bump version to v0.5.1 (#6157) · abad5746
  Simon Mo authored Jul 05, 2024
  
  abad5746
- [Bugfix] Add verbose error if scipy is missing for blocksparse attention (#5695) · e58294dd
  JGSweets authored Jul 05, 2024
  
  e58294dd
- [Frontend] Continuous usage stats in OpenAI completion API (#5742) · f1e15da6
  jvlunteren authored Jul 05, 2024
  
  f1e15da6
- [Bugfix] Use templated datasource in grafana.json to allow automatic imports (#6136) · 0097bb18
  Christian Rohmann authored Jul 05, 2024
```
Signed-off-by: Christian Rohmann <christian.rohmann@inovex.de>
```
  0097bb18
- [VLM] Cleanup validation and update docs (#6149) · ea4b5704
  Cyrus Leung authored Jul 05, 2024
  
  ea4b5704
- [VLM] Improve consistency between feature size calculation and dummy data for profiling (#6146) · a41357e9
  Roger Wang authored Jul 04, 2024
  
  a41357e9
04 Jul, 2024 7 commits
- [VLM] Calculate maximum number of multi-modal tokens by model (#6121) · ae96ef8f
  Cyrus Leung authored Jul 05, 2024
  
  ae96ef8f
- [Kernel][Model] logits_soft_cap for Gemma2 with flashinfer (#6051) · 69ec3ca1
  Lily Liu authored Jul 04, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  69ec3ca1
- [Hardware][Intel CPU] Adding intel openmp tunings in Docker file (#6008) · 81d7a50f
  Yuan authored Jul 05, 2024
```
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
```
  81d7a50f
- [misc][doc] try to add warning for latest html (#5979) · 27902d42
  youkaichao authored Jul 04, 2024
  
  27902d42
- [ROCm][AMD][Model]Adding alibi slopes support in ROCm triton flash attention... · 56b325e9
  Gregory Shtrasberg authored Jul 04, 2024
```
[ROCm][AMD][Model]Adding alibi slopes support in ROCm triton flash attention and naive flash attention (#6043)
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
```
  56b325e9
- [CI/Build] Cleanup VLM tests (#6107) · 3dd50708
  Cyrus Leung authored Jul 04, 2024
  
  3dd50708
- [Distributed][Core] Support Py39 and Py38 for PP (#6120) · 0ed646b7
  Murali Andoorveedu authored Jul 03, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  0ed646b7
03 Jul, 2024 15 commits
- [Bugfix] set OMP_NUM_THREADS to 1 by default for multiprocessing (#6109) · 1dab9bc8
  Travis Johnson authored Jul 03, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  1dab9bc8
- [core][distributed] support n layers % pp size != 0 (#6115) · 3de6e6a3
  youkaichao authored Jul 03, 2024
  
  3de6e6a3
- [doc][misc] bump up py version in installation doc (#6119) · 966fe721
  youkaichao authored Jul 03, 2024
  
  966fe721
- [ Misc ] Clean Up `CompressedTensorsW8A8` (#6113) · 62963d12
  Robert Shaw authored Jul 03, 2024
  
  62963d12
- [vlm] Remove vision language config. (#6089) · d9e98f42
  xwjiang2010 authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d9e98f42
- [core][distributed] custom allreduce when pp size > 1 (#6117) · 3c6325f0
  youkaichao authored Jul 03, 2024
  
  3c6325f0
- [Kernel] Expand FP8 support to Ampere GPUs using FP8 Marlin (#5975) · 47f0954a
  Michael Goin authored Jul 03, 2024
  
  47f0954a
- [Bugfix] Fix `compute_logits` in Jamba (#6093) · 7cd2ebb0
  Roger Wang authored Jul 03, 2024
  
  7cd2ebb0
- [Doc] Fix Mock Import (#6094) · f1c78138
  Roger Wang authored Jul 03, 2024
  
  f1c78138
- [VLM][Frontend] Proper Image Prompt Formatting from OpenAI API (#6091) · 3a86b54f
  Roger Wang authored Jul 02, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  3a86b54f
- [misc][distributed] error on invalid state (#6092) · f6662071
  youkaichao authored Jul 02, 2024
  
  f6662071
- [BugFix] Avoid unnecessary Ray import warnings (#6079) · d830656a
  Nick Hill authored Jul 02, 2024
  
  d830656a
- [CI] Fix base url doesn't strip "/" (#6087) · d18bab35
  SangBin Cho authored Jul 03, 2024
  
  d18bab35
- [Core] Dynamic image size support for VLMs (#5276) · 9831aec4
  Cyrus Leung authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  9831aec4
- [hardware][misc] introduce platform abstraction (#6080) · 482045ee
  youkaichao authored Jul 02, 2024
  
  482045ee
02 Jul, 2024 1 commit

[Model] Jamba support (#4115) · 9d6a8daa

Mor Zusman authored Jul 03, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Erez Schwartz <erezs@ai21.com>
Co-authored-by: Mor Zusman <morz@ai21.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Tomer Asida <tomera@ai21.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

9d6a8daa