Commits · b1c9aa3daa7dcd981f0f77231b46883624b72dd0 · OpenDAS / vllm_cscc

04 Aug, 2024 1 commit

[Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size to 1 when... · b1c9aa3d

Thomas Parnell authored Aug 04, 2024


[Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size to 1 when using MLPSpeculator (#7105)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

b1c9aa3d

03 Aug, 2024 1 commit
- [Frontend] Warn if user `max_model_len` is greater than derived `max_model_len` (#7080) · 825b0448
  Jeff Fialho authored Aug 03, 2024
```
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  825b0448
01 Aug, 2024 2 commits
- [Models] Support Qwen model with PP (#6974) · fc912e08
  Murali Andoorveedu authored Aug 01, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  fc912e08
- [Model] Pipeline parallel support for Qwen2 (#6924) · 1d2e7fb7
  xuyi authored Aug 01, 2024
  
  1d2e7fb7
31 Jul, 2024 2 commits
- [Misc] Add compressed-tensors to optimized quant list (#7006) · a0dce938
  Michael Goin authored Jul 31, 2024
  
  a0dce938
- [mypy] Enable following imports for some directories (#6681) · da1f7cc1
  Cyrus Leung authored Jul 31, 2024
  
  da1f7cc1
27 Jul, 2024 2 commits
- Add Nemotron to PP_SUPPORTED_MODELS (#6863) · b1366a95
  Michael Goin authored Jul 27, 2024
  
  b1366a95
- enforce eager mode with bnb quantization temporarily (#6846) · bb549467
  chenqianfzh authored Jul 26, 2024
  
  bb549467
23 Jul, 2024 5 commits
- [bitsandbytes]: support read bnb pre-quantized model (#5753) · 87525fab
  dongmao zhang authored Jul 23, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  87525fab
- [Model] Pipeline Parallel Support for DeepSeek v2 (#6519) · 507ef787
  Travis Johnson authored Jul 23, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  507ef787
- [BugFix] Fix RoPE error in Llama 3.1 (#6693) · a112a84a
  Woosuk Kwon authored Jul 23, 2024
  
  a112a84a
- [Bugfix] Fix a log error in chunked prefill (#6694) · 461089a2
  Woosuk Kwon authored Jul 23, 2024
  
  461089a2
- support ignore patterns in model loader (#6673) · 3eda4ec7
  Simon Mo authored Jul 22, 2024
  
  3eda4ec7
21 Jul, 2024 2 commits
- [Kernel][Core] Add AWQ support to the Marlin kernel (#6612) · 396d92d5
  Alexander Matveev authored Jul 21, 2024
  
  396d92d5
- [Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both... · 14f91fe6
  sroy745 authored Jul 20, 2024
```
[Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both draft and target models. (#6485)
```
  14f91fe6
20 Jul, 2024 2 commits
- [ Misc ] `fbgemm` checkpoints (#6559) · 683e3cb9
  Robert Shaw authored Jul 20, 2024
  
  683e3cb9
- [Core] Allow specifying custom Executor (#6557) · 7bd82002
  Antoni Baum authored Jul 19, 2024
  
  7bd82002
19 Jul, 2024 2 commits
- [Core] Multiprocessing Pipeline Parallel support (#6130) · b5672a11
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  b5672a11
- Add support for a rope extension method (#6553) · c5df56f8
  Simon Mo authored Jul 18, 2024
  
  c5df56f8
18 Jul, 2024 3 commits
- [core][model] yet another cpu offload implementation (#6496) · 1c27d25f
  youkaichao authored Jul 17, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  1c27d25f
- [ Kernel ] Fp8 Channelwise Weight Support (#6487) · 18fecc35
  Robert Shaw authored Jul 17, 2024
  
  18fecc35
- [Model] Pipeline parallel support for Mixtral (#6516) · b5af8c22
  Cody Yu authored Jul 17, 2024
  
  b5af8c22
12 Jul, 2024 1 commit
- [ROCm][AMD] unify CUDA_VISIBLE_DEVICES usage in cuda/rocm (#6352) · b6c16cf8
  Hongxia Yang authored Jul 12, 2024
  
  b6c16cf8
11 Jul, 2024 1 commit
- [Misc] refactor(config): clean up unused code (#6320) · 3963a533
  aniaan authored Jul 11, 2024
  
  3963a533
09 Jul, 2024 1 commit

[CORE] Adding support for insertion of soft-tuned prompts (#4645) · 4d6ada94

Swapnil Parekh authored Jul 09, 2024


Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

4d6ada94

03 Jul, 2024 5 commits
- [core][distributed] support n layers % pp size != 0 (#6115) · 3de6e6a3
  youkaichao authored Jul 03, 2024
  
  3de6e6a3
- [vlm] Remove vision language config. (#6089) · d9e98f42
  xwjiang2010 authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d9e98f42
- [core][distributed] custom allreduce when pp size > 1 (#6117) · 3c6325f0
  youkaichao authored Jul 03, 2024
  
  3c6325f0
- [BugFix] Avoid unnecessary Ray import warnings (#6079) · d830656a
  Nick Hill authored Jul 02, 2024
  
  d830656a
- [Core] Dynamic image size support for VLMs (#5276) · 9831aec4
  Cyrus Leung authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  9831aec4
02 Jul, 2024 4 commits

[Model] Jamba support (#4115) · 9d6a8daa

Mor Zusman authored Jul 03, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Erez Schwartz <erezs@ai21.com>
Co-authored-by: Mor Zusman <morz@ai21.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Tomer Asida <tomera@ai21.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

9d6a8daa

[Core] Pipeline Parallel Support (#4412) · c5832d2a
Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
c5832d2a
[Speculative Decoding] MLPSpeculator Tensor Parallel support (1/2) (#6050) · 15aba081
Sirej Dua authored Jul 02, 2024
```
Co-authored-by: Sirej Dua <sirej.dua@databricks.com>
Co-authored-by: Sirej Dua <Sirej Dua>
```
15aba081

[VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c

xwjiang2010 authored Jul 02, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

98d6682c

01 Jul, 2024 1 commit
- [Speculative Decoding 2/2 ] Integrate typical acceptance sampler into Spec Decode Worker (#5348) · 80ca1e6a
  sroy745 authored Jul 01, 2024
  
  80ca1e6a
28 Jun, 2024 4 commits
- Support Deepseek-V2 (#4650) · be0b3af9
  wangding zeng authored Jun 29, 2024
```
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
```
  be0b3af9
- [Bugfix] Better error message for MLPSpeculator when `num_speculative_tokens`... · ec1ad004
  Thomas Parnell authored Jun 28, 2024
```
[Bugfix] Better error message for MLPSpeculator when `num_speculative_tokens` is set too high (#5894)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  ec1ad004
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
- [Core] Registry for processing model inputs (#5214) · 5cbe8d15
  Cyrus Leung authored Jun 28, 2024
```
Co-authored-by: ywang96 <ywang@roblox.com>
```
  5cbe8d15
27 Jun, 2024 1 commit
- [Model] Add Gemma 2 (#5908) · 79c92c7c
  Woosuk Kwon authored Jun 27, 2024
  
  79c92c7c