Commits · 7c008c51a9aa4c0d53d09ab3a1ba61ecec354565 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "225d1090a0996710a23d58cfcd1d4d2b089cc553"

02 Jul, 2024 6 commits
- [ Misc ] Refactor MoE to isolate Fp8 From Mixtral (#5970) · 7c008c51
  Robert Shaw authored Jul 02, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  7c008c51
- [Core] Pipeline Parallel Support (#4412) · c5832d2a
  Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  c5832d2a
- [Speculative Decoding] MLPSpeculator Tensor Parallel support (1/2) (#6050) · 15aba081
  Sirej Dua authored Jul 02, 2024
```
Co-authored-by: Sirej Dua <sirej.dua@databricks.com>
Co-authored-by: Sirej Dua <Sirej Dua>
```
  15aba081
- [VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c
  xwjiang2010 authored Jul 02, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  98d6682c
- [Frontend] Add template related params to request (#5709) · 2c37540a
  danieljannai21 authored Jul 02, 2024
  
  2c37540a
- [Core] Optimize block_manager_v2 vs block_manager_v1 (to make V2 default) (#5602) · 3476ed08
  Alexander Matveev authored Jul 01, 2024
  
  3476ed08
01 Jul, 2024 7 commits
- [Model] Changes to MLPSpeculator to support tie_weights and input_scale (#5965) · 54600709
  Thomas Parnell authored Jul 02, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Joshua Rosenkranz <jmrosenk@us.ibm.com>
```
  54600709
- [BugFix] Ensure worker model loop is always stopped at the right time (#5987) · c87ebc3e
  Nick Hill authored Jul 01, 2024
  
  c87ebc3e
- [Bugfix] Add explicit `end_forward` calls to flashinfer (#6044) · c4059ea5
  Antoni Baum authored Jul 01, 2024
  
  c4059ea5
- [Bugfix] adding chunking mechanism to fused_moe to handle large inputs (#6029) · 12a59959
  Avshalom Manevich authored Jul 02, 2024
  
  12a59959
- [Bugfix] Use RayActorError for older versions of Ray in RayTokenizerGroupPool (#6039) · dec6fc6f
  Antoni Baum authored Jul 01, 2024
  
  dec6fc6f
- [Speculative Decoding 2/2 ] Integrate typical acceptance sampler into Spec Decode Worker (#5348) · 80ca1e6a
  sroy745 authored Jul 01, 2024
  
  80ca1e6a
- [misc][cuda] use nvml to avoid accidentally cuda initialization (#6007) · 614aa512
  youkaichao authored Jun 30, 2024
  
  614aa512
30 Jun, 2024 4 commits
- [ Misc ] Refactor w8a8 to use `process_weights_after_load` (Simplify Weight Loading) (#5940) · af9ad46f
  Robert Shaw authored Jun 30, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  af9ad46f
- [Misc] Fix `get_min_capability` (#5971) · 7836fdcc
  Dipika Sikka authored Jun 30, 2024
  
  7836fdcc
- [Lora] Use safetensor keys instead of adapter_config.json to find unexpected modules. (#5909) · f5e73c9f
  SangBin Cho authored Jul 01, 2024
```
Co-authored-by: sang <sangcho@anyscale.com>
```
  f5e73c9f
- [Frontend]: Support base64 embedding (#5935) · c6c240aa
  llmpros authored Jun 30, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  c6c240aa
29 Jun, 2024 9 commits
- [CI/Build] Add TP test for vision models (#5892) · 99397da5
  Cyrus Leung authored Jun 29, 2024
  
  99397da5
- [ CI/Build ] Added E2E Test For Compressed Tensors (#5839) · 8dbfcd35
  Robert Shaw authored Jun 29, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  8dbfcd35
- [Kernel] Raise an exception in MoE kernel if the batch size is larger then 65k (#5939) · f7dac83d
  Cody Yu authored Jun 29, 2024
  
  f7dac83d
- [Core] Optimize `SequenceStatus.is_finished` by switching to IntEnum (#5974) · 7c01f706
  Antoni Baum authored Jun 29, 2024
  
  7c01f706
- [Bugfix] Support `eos_token_id` from `config.json` (#5954) · 51e971d3
  Cyrus Leung authored Jun 29, 2024
  
  51e971d3
- [Bugfix] Fix precisions in Gemma 1 (#5913) · 580353da
  Woosuk Kwon authored Jun 28, 2024
  
  580353da
- [Misc] Extend vLLM Metrics logging API (#5925) · 906a19cd
  William Lin authored Jun 28, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  906a19cd
- [Bugfix][TPU] Fix pad slot id (#5977) · 7f83f40d
  Woosuk Kwon authored Jun 28, 2024
  
  7f83f40d
- [Bugfix][TPU] Fix TPU sampler output (#5978) · 54814fd8
  Woosuk Kwon authored Jun 28, 2024
  
  54814fd8
28 Jun, 2024 14 commits
- [Kernel] Flashinfer for prefill & decode, with Cudagraph support for decode (#4628) · 7041de43
  Lily Liu authored Jun 28, 2024
```
Co-authored-by: LiuXiaoxuanPKU &lt;llilyliupku@gmail.com&gt;, bong-furiosa <bongwon.jang@furiosa.ai>
```
  7041de43
- [Bugfix] Fix Engine Failing After Invalid Request - AsyncEngineDeadError (#5963) · 6a62cb82
  Robert Shaw authored Jun 28, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  6a62cb82
- Unmark more files as executable (#5962) · 5d2a1a9c
  Tyler Michael Smith authored Jun 28, 2024
  
  5d2a1a9c
- [Bugfix] Only add `Attention.kv_scale` if kv cache quantization is enabled (#5936) · 4bf35ed9
  Michael Goin authored Jun 28, 2024
  
  4bf35ed9
- Support Deepseek-V2 (#4650) · be0b3af9
  wangding zeng authored Jun 29, 2024
```
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
```
  be0b3af9
- [ Bugfix ] Enabling Loading Models With Fused QKV/MLP on Disk with FP8 (#5921) · 2cd402e1
  Robert Shaw authored Jun 28, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  2cd402e1
- [ Misc ] Remove `fp8_shard_indexer` from Col/Row Parallel Linear (Simplify Weight Loading) (#5928) · b1852307
  Robert Shaw authored Jun 28, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  b1852307
- [Spec Decode] Introduce DraftModelRunner (#5799) · b2c62023
  Cody Yu authored Jun 28, 2024
  
  b2c62023
- [Distributed] Make it clear that % should not be in tensor dict keys. (#5927) · b90d8cd8
  xwjiang2010 authored Jun 28, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
```
  b90d8cd8
- [Bugfix] Better error message for MLPSpeculator when `num_speculative_tokens`... · ec1ad004
  Thomas Parnell authored Jun 28, 2024
```
[Bugfix] Better error message for MLPSpeculator when `num_speculative_tokens` is set too high (#5894)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  ec1ad004
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
- Unmark fused_moe config json file as executable (#5960) · 59326344
  Tyler Michael Smith authored Jun 28, 2024
  
  59326344
- [Core] Registry for processing model inputs (#5214) · 5cbe8d15
  Cyrus Leung authored Jun 28, 2024
```
Co-authored-by: ywang96 <ywang@roblox.com>
```
  5cbe8d15
- [Bugfix][Hardware][Intel CPU] Fix unpassed multi_modal_kwargs for CPU runner (#5956) · 0d0e3a42
  Isotr0py authored Jun 28, 2024
  
  0d0e3a42