Commits · 2416b26e119b9d1932ba30790ecaddfac1ae4143 · OpenDAS / vllm_cscc · GitLab

10 Jul, 2024 1 commit
- [Speculative Decoding] Medusa Implementation with Top-1 proposer (#4978) · 2416b26e
  Abhinav Goyal authored Jul 10, 2024
  
  2416b26e
09 Jul, 2024 8 commits
- [Bugfix]fix and needs_scalar_to_array logic check (#6238) · d3a24513
  Baoyuan Qi authored Jul 10, 2024
```
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  d3a24513
- [Docs] Docs update for Pipeline Parallel (#6222) · 673dd4ca
  Murali Andoorveedu authored Jul 09, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  673dd4ca
- [CORE] Adding support for insertion of soft-tuned prompts (#4645) · 4d6ada94
  Swapnil Parekh authored Jul 09, 2024
```
Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  4d6ada94
- Add support for multi-node on CI (#5955) · a0550cbc
  Kevin H. Luu authored Jul 09, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  a0550cbc
- [Bugfix][TPU] Fix outlines installation in TPU Dockerfile (#6256) · 08c5bdec
  Woosuk Kwon authored Jul 09, 2024
  
  08c5bdec
- [Bugfix][TPU] Add missing None to model input (#6245) · 5d5b4c5f
  Woosuk Kwon authored Jul 09, 2024
  
  5d5b4c5f
- [core][distributed] fix ray worker rank assignment (#6235) · 70c232f8
  youkaichao authored Jul 08, 2024
  
  70c232f8
- [hardware][cuda] use device id under CUDA_VISIBLE_DEVICES for get_device_capability (#6216) · a3c9435d
  youkaichao authored Jul 08, 2024
  
  a3c9435d
08 Jul, 2024 7 commits
- Add FlashInfer to default Dockerfile (#6172) · 4f0e0ea1
  Simon Mo authored Jul 08, 2024
  
  4f0e0ea1
- [Bugfix] Mamba cache Cuda Graph padding (#6214) · ddc369fb
  tomeras91 authored Jul 08, 2024
  
  ddc369fb
- [Bugfix] use diskcache in outlines _get_guide #5436 (#6203) · 185ad31f
  Eric authored Jul 09, 2024
  
  185ad31f
- [Kernel] Correctly invoke prefill & decode kernels for cross-attention... · 543aa485
  afeldman-nm authored Jul 08, 2024
```
[Kernel] Correctly invoke prefill & decode kernels for cross-attention (towards eventual encoder/decoder model support) (#4888)
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  543aa485
- [Kernel] reloading fused_moe config on the last chunk (#6210) · f7a8fa39
  Avshalom Manevich authored Jul 08, 2024
  
  f7a8fa39
- Feature/add benchmark testing (#5947) · 717f4bce
  Haichuan authored Jul 08, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  717f4bce
- do not exclude `object` field in CompletionStreamResponse (#6196) · 16620f43
  kczimm authored Jul 07, 2024
  
  16620f43
07 Jul, 2024 4 commits
- [misc][frontend] log all available endpoints (#6195) · 3b08fe2b
  youkaichao authored Jul 07, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  3b08fe2b
- [ Misc ] Support Fp8 via `llm-compressor` (#6110) · abfe705a
  Robert Shaw authored Jul 07, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  abfe705a
- add benchmark for fix length input and output (#5857) · 333306a2
  Haichuan authored Jul 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  333306a2
- [Model] Add PaliGemma (#5189) · 6206dcb2
  Roger Wang authored Jul 06, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6206dcb2
06 Jul, 2024 3 commits
- [Doc] Move guide for multimodal model and other improvements (#6168) · 93893800
  Cyrus Leung authored Jul 06, 2024
  
  93893800
- [Doc] Reorganize Supported Models by Type (#6167) · 175c43ec
  Roger Wang authored Jul 05, 2024
  
  175c43ec
- Move release wheel env var to Dockerfile instead (#6163) · bc96d5c3
  Simon Mo authored Jul 05, 2024
  
  bc96d5c3
05 Jul, 2024 9 commits
- Fix release wheel build env var (#6162) · f0250620
  Simon Mo authored Jul 05, 2024
  
  f0250620
- Update wheel builds to strip debug (#6161) · 2de490d6
  Simon Mo authored Jul 05, 2024
  
  2de490d6
- [Docs] Fix readthedocs for tag build (#6158) · 79d406e9
  Simon Mo authored Jul 05, 2024
  
  79d406e9
- bump version to v0.5.1 (#6157) · abad5746
  Simon Mo authored Jul 05, 2024
  
  abad5746
- [Bugfix] Add verbose error if scipy is missing for blocksparse attention (#5695) · e58294dd
  JGSweets authored Jul 05, 2024
  
  e58294dd
- [Frontend] Continuous usage stats in OpenAI completion API (#5742) · f1e15da6
  jvlunteren authored Jul 05, 2024
  
  f1e15da6
- [Bugfix] Use templated datasource in grafana.json to allow automatic imports (#6136) · 0097bb18
  Christian Rohmann authored Jul 05, 2024
```
Signed-off-by: Christian Rohmann <christian.rohmann@inovex.de>
```
  0097bb18
- [VLM] Cleanup validation and update docs (#6149) · ea4b5704
  Cyrus Leung authored Jul 05, 2024
  
  ea4b5704
- [VLM] Improve consistency between feature size calculation and dummy data for profiling (#6146) · a41357e9
  Roger Wang authored Jul 04, 2024
  
  a41357e9
04 Jul, 2024 7 commits
- [VLM] Calculate maximum number of multi-modal tokens by model (#6121) · ae96ef8f
  Cyrus Leung authored Jul 05, 2024
  
  ae96ef8f
- [Kernel][Model] logits_soft_cap for Gemma2 with flashinfer (#6051) · 69ec3ca1
  Lily Liu authored Jul 04, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  69ec3ca1
- [Hardware][Intel CPU] Adding intel openmp tunings in Docker file (#6008) · 81d7a50f
  Yuan authored Jul 05, 2024
```
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
```
  81d7a50f
- [misc][doc] try to add warning for latest html (#5979) · 27902d42
  youkaichao authored Jul 04, 2024
  
  27902d42
- [ROCm][AMD][Model]Adding alibi slopes support in ROCm triton flash attention... · 56b325e9
  Gregory Shtrasberg authored Jul 04, 2024
```
[ROCm][AMD][Model]Adding alibi slopes support in ROCm triton flash attention and naive flash attention (#6043)
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
```
  56b325e9
- [CI/Build] Cleanup VLM tests (#6107) · 3dd50708
  Cyrus Leung authored Jul 04, 2024
  
  3dd50708
- [Distributed][Core] Support Py39 and Py38 for PP (#6120) · 0ed646b7
  Murali Andoorveedu authored Jul 03, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  0ed646b7
03 Jul, 2024 1 commit
- [Bugfix] set OMP_NUM_THREADS to 1 by default for multiprocessing (#6109) · 1dab9bc8
  Travis Johnson authored Jul 03, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  1dab9bc8